Using a Self-Assessment Tool (SAT) to Review National Health
Information Systems in Ireland
Sarah Craig
Health Research Board, Dublin, Ireland
Keywords: Health Information Systems, Self-Assessment, National Standards, Ireland.
Abstract: This paper presents a review of four national health information systems in Ireland using a nationally agreed
self-assessment tool (SAT) developed in Ireland by the health information regulatory body there. The review
was undertaken using documentary analysis of written materials about the systems from both primary and
secondary data sources. The findings show high levels of compliance with the standards identified but that
there is still some work to do to ensure that all aspects of the standards are met.
1 INTRODUCTION
This paper presents a review of four national health
information systems using a self-assessment tool
(SAT) which was developed by Ireland’s health
information regulatory body, the Health Information
and Quality Authority (HIQA, 2017a) and which
have since become an agreed set of information
management standards for evaluating national health
and social care data collections (HIQA, 2017b).
The value of having key standards for monitoring
data quality is well-recognised internationally (see
Canadian Institute for Health Information, 2009;
Australian Institute of Health Welfare, 2014) and
several tools have been developed to assist the
managers of health information systems to ensure
good information governance is at the heart of their
processes (e.g. American Health Information
Management Association, 2011; Health and Social
Care Information Centre, 2015) in their work.
Building on the international evidence, HIQA
developed a self-assessment tool (SAT) for use on
national health information systems and this is in
process of being applied to the major Irish health and
social care data collections currently (e.g HIQA,
2019).
The research questions for the study were:
a) To what extent, if at all, do the Health Research
Board’s (HRB) systems comply with the HIQA
standards?
b) What does the HRB need to do to ensure full
compliance with the standards?
The paper highlights the value of having
standards that are informed by international best
practice and that create targets for national bodies in
achievement of this best practice. The overall benefits
are better quality data which in turn results in better
service planning at the national and local level, safer,
better care for patients and service users and
improved population health (HIQA, 2017b).
2 BACKGROUND
The Health Information and Quality Authority
(HIQA) is an independent statutory authority in
Ireland that was established to promote safety and
quality in the provision of health and social care
services for the benefit of the health and welfare of
the public in Ireland. Its role in health information is
focused on advising on the efficient and secure
collection and sharing of health information, setting
standards, evaluating information resources and
publishing information on the delivery and
performance of Ireland’s health and social care
services. In recent years HIQA’s health information
function has been particularly focused on producing
resources for the managers of health information
systems that are designed to support the
implementation of common standards in relation to
information governance and data quality.
The SAT devised by HIQA is based on six broad
themes which are then broken down into ten
standards (see Table 1).
Craig, S.
Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland.
DOI: 10.5220/0009166302490253
In Proceedings of the 13th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2020) - Volume 5: HEALTHINF, pages 249-253
ISBN: 978-989-758-398-8; ISSN: 2184-4305
Copyright
c
2022 by SCITEPRESS Science and Technology Publications, Lda. All rights reser ved
249
Table 1: Themes and National Standards.
Theme Standard
Person-centred
1. Arrangements to protect
p
rivac
y
of
p
eo
p
le
Governance,
leadership and
management
2. Effective governance
3. Publicly available statement of
purpose
4. Compliance with relevant
le
g
islation
Use of data
5. Compliance with health
information standards
6. Monitors quality of data
7. Effective and appropriate
dissemination strategies
Information
governance
8. Effective information
governance
Workforce
9. Workforce to deliver
objectives
Use of resources
10. Effective allocation of
resources
The themes and standards were agreed nationally
through expert collaboration and public consultation
and are based on robust international evidence in this
area (see American Health Information Management
Association, 2011, Australian Institute of Health
Welfare, 2014). The themes, standards and metrics
for measuring compliance are all set out in the self-
assessment tool (HIQA, 2017a) which is available on
the HIQA website (www.hiqa.ie ).
Following the publication of the self-assessment
tool for health and social care data collections in
2017, the HRB committed to undertaking an
evaluation of its four national systems using the tool.
The HRB is a public service body under the remit of
the Department of Health in Ireland. It collects data
in the areas of drugs and alcohol (the National Drug
Treatment Reporting System (NDTRS) and the
National Drug-related Deaths Index (NDRDI)); in
relation to disability (the National Ability Supports
System (NASS)) and mental health (the National
Psychiatric In-patient Reporting System (NPIRS)).
Data collection began as far back as the 1960s in
the mental health area, followed by drug treatment
and disability data systems in the 1990s and drug-
related deaths in 2005. Appendix 1 presents a
summary overview of the four systems. Twenty-five
people are employed across the four systems and the
annual budget is around €1.2 million. All of the
systems generate timely and accurate data at a
national level to assist with service planning and
monitoring of key policies in the areas identified as
well as reporting at national, EU and international
levels.
3 METHODS
During 2018, a project team was formed in the HRB
consisting of staff members working in each of the
four systems along with the HRB’s Head of National
Health Information Systems. The project team was
chaired by a HIQA representative. The method
adopted was largely documentary where key
protocols for each of the systems were reviewed and
the SAT was completed based on this written
evidence as well as staff knowledge about practices
within each system. Staff worked in pairs so as to
ensure peer-review of the completed SAT which
improved the quality of the information recorded.
Analysis was undertaken of written materials for
each of the Health Research Board’s (HRB) four
national health information systems including
primary sources such as protocols for data collection
and collation, data validation and reporting as well as
secondary sources such as evaluations and reviews.
The work took place over a period of 3 months.
4 FINDINGS OF THE
SELF-ASSESSMENT
Overall, the HRB’s health information systems
displayed high levels of compliance with the themes
and standards set out by HIQA. The specific findings
in relation to each of the themes and standards are set
out below.
4.1 Person-centredness
A person-centred approach focuses on the need to
protect the privacy of the individuals about whom the
data are collected (HIQA, 2017b). One of the
fundamental principles in this regard is that the data
subject is aware of how their data is being used. Ways
in which this can be achieved include having a
statement of information practice and undertaking
privacy impact assessments on a regular basis.
In relation to how the HRB’s information systems
rate on effective arrangements in to protect the
privacy of people about whom it holds information,
compliance is high as there are privacy policies
implemented for each system. However, there is a
need to make statements of information practice
publicly available on the organisation’s website to
ensure greater transparency.
HEALTHINF 2020 - 13th International Conference on Health Informatics
250
4.2 Governance, Leadership and
Management
This theme requires bodies that manage health
information to have good governance, leadership and
management arrangements in at the level of the
organization. Ways in which this can be achieved is
through clarity in relation to the objectives of the data
collection, having an identified individual with
overall responsibility for the data collection, a
statement of purpose and a well-documented
organizational chart and an identified governance
group consisting of key stakeholders.
In relation to the standards captured in this theme
the HRB’s national health information systems rate
highly but there is variability in the practices across
systems. Two of the four systems have formalised
arrangements in place (Table 2) to bring together key
stakeholders on a regular basis to consider the
strategic direction of the system and to make
decisions about any required changes.
Table 2: Governance Arrangements for the HRB’s health
information systems.
System Governance arrangement
NDTRS
None
NDRDI
National Steering Committee
comprising all stakeholders
NASS
National Committee comprising all
stakeholders
NPIRS
None
Source: Health Research Board, 2014.
However, the HRB does not currently publish
reports on the effectiveness of the national data
collections it holds although all publications from
these systems refer to the quality of the data contained
within. The assessment also found that there should
be more key performance indicators in place to
measure and report on the effectiveness of the HRB’s
information systems.
Within the HRB, some policies are in place on
privacy and confidentiality but more transparency is
needed on how and how often statements of purpose
are reviewed.
The assessment also showed high levels of
compliance with legislation, particularly data
protection legislation but proactive identification of
risks and issues in upcoming legislation was seen to
be needed.
4.3 Use of Data
HIQA recommends in its information management
standards (HIQA, 2017b) that all organisations
involved in the collection of health information
should ensure that the use of the information is
optimized to achieve the best value for money and to
maximize social gain. This includes ensuring
accuracy, completeness, legibility, relevance,
reliability and timeliness of the data. It also includes
having published data dictionaries for each data
collection and a framework for data quality that is
agreed at the level of all stakeholders as well as the
incorporation of international classifications.
In relation to this theme, all but one of the HRB’s
systems have incorporated international
classifications. Whilst a data dictionary has been
developed for each system it is not currently made
publicly available.
Similarly, within the HRB, there is a recognition
of the need for quality data in all the information
systems and considerable time and effort go in to
ensuring the quality of the data. However, there is no
formalized data quality framework in place. Some
work has been undertaken on audit internally and two
of the systems have invested in external audit and
evaluation, but these are outdated, and further work is
required in this regard. The SAT findings suggest
that the HRB needs to plan for further internal and
external audit of the data and consider the use of the
data quality framework dimensions as an audit
framework.
With regard to dissemination of data there is a
high level of compliance in the HRBs information
systems. The assessment found that more could be
done to support users of the data, data quality
statements and notifying, in advance, on an annual
basis what publications will be available. In
particular, the findings suggested that more efforts
were needed to provide training to data users around
the use of the data and greater regard needed to be
given to recording metrics in relation to the timeliness
of responding to requests for the data. The SAT also
found that a simple calendar of when publications
from the information systems, if disseminated to
stakeholder, would improve timeliness and raise
awareness about when reports would come on stream.
4.4 Information Governance
This theme is about ensuring that those involved in
the collection of health data build in processes to
promote security and privacy in the collection and
reporting of data. It includes reliable information
governance practices in areas such as ensuring
consent is obtained from the data subject where it is
necessary, having statements of information
Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland
251
practices, arrangements around the appropriate
sharing of information and ongoing audit on
information governance practice.
There are high levels of compliance in the HRB
with the standard on information governance but the
assessment found that more work is required on
information governance audits and training for staff
on information governance issues.
Each of the four systems managed by the HRB has
a publicly available statement of purpose, setting out
how the objectives of each system are achieved (see
Table 3). The statement of purpose has been agreed
by those involved in the governance structures that
are in place to oversee the systems (where they exist).
The statement is maintained by the staff responsible
for managing each system and is reviewed regularly
to ensure that it is fit-for-purpose. All statements are
published in HIQA’s catalogue of health and social
care data collections (HIQA, 2017c).
Table 3: Statements of Purpose for the HRB’s health
information systems.
NASS
The NASS is a service-planning tool
designed to capture data on the usage
of and need for specialist disability
services among people with a
disability.
NDTRS
The NDTRS was established as an
epidemiological database on treated
drug and alcohol misuse in Ireland. It
records incidence of drug and alcohol
treatment.
NDRDI
The NDRDI provides epidemiological
monitoring of cases of drug- and
alcohol-related deaths, and deaths
among drug users and among
alcoholics in Ireland.
NPIRS
The NPIRS is a psychiatric database,
which provides detailed information
on all admissions and discharges to in-
patient psychiatric services in Ireland.
4.5 Workforce
HIQA defines the workforce standard to include all of
those who work in or for the national health
information system and recommends that staff with
specialist skills and qualifications are needed to
ensure that aspects of quality are factored in to the
work and that there is good workforce planning to
deal with expected and unexpected events. It is about
having the ‘right people with the right knowledge’
(HIQA, 2017b: 41).
In relation to the HRB’s workforce there are high
levels of compliance with the organisation of its
workforce and its ongoing development. Workforce
planning is undertaken regularly to anticipate any
likely changes in staffing levels.
4.6 Use of Resources
HIQA defines resources as including human,
physical, financial and ICT resources and
recommends that organisations involved in the
collection and reporting of health data should strive
to ensure that its resources are adequate to ensure the
sustainability, continual relevance and maximum
impact of the data for which they are responsible.
The SAT exercise showed that in relation to the
allocation and use of resources within the HRB, there
are high levels of compliance with planning and
management of the necessary resources. This is
particularly the case in relation to skills audit,
succession planning and the organisation’s training
and development fund.
5 DISCUSSION
Following the completion of the SAT, the HRB
reviewed its findings with HIQA and devised a set of
actions which were then presented to senior
management in the organisation as an improvement
plan. Overall, the review of the HRB’s health
information systems shows that there is a broad range
of processes at play within and between the four
systems in achieving their goals and objectives in the
delivery of key national health data. The systems
vary significantly in how they collect, organise,
manage and disseminate data but, for the most part,
there is good practice in a number of the standards
and themes that constituted the SAT. Areas for
improvement are as follows:
Publish statements of information practice on the
HRB website.
Set up oversight committees for the remaining
two systems, the NPIRS and the NDTRS.
Put in place formalised agreements with data
providers.
Consider adding more detail on
performance/effectiveness of HRB systems in
publications.
Develop an annual process to review statements
of purpose.
HEALTHINF 2020 - 13th International Conference on Health Informatics
252
Put in place a process for reviewing upcoming
legislation relevant to the HRB data collections.
Publish data dictionaries for each system.
Formalise a data quality framework for the in-
formation systems that incorporates the elements
of good practice already applied to the data.
Plan for further internal and external audit of the
data and consider the use of the data quality
framework dimensions as an audit framework.
Use the organisation’s website monitoring to
record access to health information publications
Add more metrics to monitor the use of the data
such as tracking the number of times data are
accessed on the HRB’s website.
Provide training to data users around the value of
the HRB’s data.
Add some performance metrics to the existing
log of data requests to include timeliness of
response to these requests.
Develop an annual calendar on publication of
reports from the HRB’s systems.
Include more detailed data quality statements in
each HRB publication.
Devise a schedule for internal and external audit
of the HRB systems.
Devise a training programme for staff on
information governance.
6 CONCLUSION
The study set out to address two research questions:
a) To what extent, if at all, do the Health Research
Board’s (HRB) systems comply with the HIQA
standards?
b) What does the HRB need to do to ensure full
compliance with the standards?
The findings indicate that there are high levels of
compliance with the standards across all four systems
but that there is variability in the level of this
compliance. The areas for improvement identified as
part of the study show a pathway towards full
compliance. A follow-up review will be undertaken
to track progress in this regard.
Overall, the study highlights the value of having
agreed national standards for health information
systems such as those managed by the HRB.
REFERENCES
American Health Information Management Association,
2011. Health Data Analysis Toolkit. AHIMA. Chicago.
Australian Institute of Health Welfare, 2014. Data
Governance Framework. AIHW. Canberra.
Canadian Institute for Health Information, 2009. The CIHI
Data Quality Framework. CIHI. Ottawa.
Health and Social Care Information Centre, 2015.
Information Governance Toolkit. HSCIC. London.
Health Research Board, 2016. Research, Evidence, Action.
HRB Strategy 2016-2020, HRB. Dublin.
HIQA, 2013. Guiding Principles for National Health and
Social Care Data Collections, HIQA. Dublin.
HIQA, 2017a. Self-Assessment Tool for National Health
and Social Care Data Collections, HIQA, Dublin.
HIQA 2017b. Information management standards for
national health and social care data collections. HIQA.
Dublin.
HIQA 2017c. HIQA Catalogue for national health and
social care data collections. HIQA. Dublin.
HIQA 2018. Guidance on a data quality framework for
health and social care. HIQA. Dublin.
HIQA 2018. Review of information management practices
in the HSE Primary Care Reimbursement Service.
HIQA. Dublin.
APPENDICE
Appendix 1: Overview of HRB Systems.
Objectives Activity
National
Drug
Treatment
Reporting
System
(NDTRS)
To gather data on
treated drug and
alcohol misuse in
Ireland that can be
used by policy
makers and
service providers
30,000 records
annually
Data items: 64
National
Drug-
related
Deaths
Index
(NDRDI)
To collect
information on
drug and alcohol-
related deaths and
deaths among
drug and alcohol
users in Ireland
600 deaths
annually
Data items: 70
15,000
coroners files
consulted each
y
ea
r
National
Ability
Supports
System
(NASS)
To capture details
of current service
provision and the
future service
requirements of
individuals with a
disability
30,000
records
Data items:
200
National
Psychiatric
In-patient
Reporting
System
(NPIRS)
To collect and
report on all
admissions and
discharges to
inpatient
psychiatric units
both public and
private. A regular
psychiatric census
is carried out.
38,000 records
annually -
19,000
admissions
and 19,000
discharges
67 sites
Number of
data items: 28
Using a Self-Assessment Tool (SAT) to Review National Health Information Systems in Ireland
253