Practical Application of Maturity Models in Healthcare: Findings
from Multiple Digitalization Case Studies
Anja Burmann
1a
and Sven Meister
1,2 b
1
Digitization in Healthcare, Fraunhofer Institute for Software and Systems Engineering, Dortmund, Germany
2
Health Informatics, Faculty of Health, Witten/Herdecke University, Witten, Germany
Keywords: Maturity Model, Digitalization, Hospital, Literature Review, Case Study Research.
Abstract: Maturity models (MMs) are widely applied means for describing a current status of development across
numerous domains, but also within the healthcare sector. They offer orientation for systematic development
and improvement regarding the aspect examined. However, experience from the practical application of MMs
is scarcely described. Within this article two projects in which MMs were used in collaboration with
practitioner groups from hospital environments are presented. The general project intentions, motivation for
incorporating MMs and generated results are described. By deriving observations across the two cases,
general tensions between healthcare practice and research concerning maturity modelling are identified.
Additionally, the suitability of existing MMs to support especially hospitals in coping with the challenges of
the digital transformation is discussed. This study’s findings may be incorporated into development and
refinement of MMs and may thus contribute to increasing practical value created by such means.
1 INTRODUCTION
Manufacturing industries all over the world are facing
the challenges of the digital age: a change of
production, services as well as business models itself
is required (Bauernhansl, 2014). The same challenge
applies to healthcare provision: the increased
availability of electronically held information and
data offers new ways of providing healthcare. Digital
interfaces between care providers, sectors, divisions,
wards and functional areas offer a seamless
availability of information, but also require new ways
of organizing and steering the health provision
process. The peculiarity of the digital transformation
is the prospective designation as an industrial
revolution, what has usually been named as such in
historical review. Regardless of whether the scope
was correctly anticipated, this withholds the
opportunity and the burden for societies, industry,
research, but also healthcare providers to actively
shape the future (Hermann et al., 2016). Hitherto,
practitioners and researchers pursued the
systemization of change processes, in order to
constitute a phenomenon, and to be able to
a
https://orcid.org/0000-0002-6989-1230
b
https://orcid.org/0000-0003-0522-986X
structurally evolve it (Meister et al., 2019). The
shaping of this as a revolution labelled future working
world requires the disruptive rearrangement of
processes according to a constantly developing
vision (Deiters et al., 2018).
Up to this point, maturity models (MMs) have
been widely established as tools for describing a
specific development status, circumstance or
condition of an organization, a process or a structure
(Wendler, 2012). At the same time, they usually offer
a path of evolution for systematic development and
improvement with regard to the assessed aspect
(Pöppelbuß et al., 2011; Pöppelbuß & Röglinger,
2011). Current research efforts focus on the
development of increasingly rigorous MMs
particularly addressing the challenges of the digital
transformation in various domains. An increasing
number of MMs is dealing with digitalization aspects
in healthcare settings. At the same time, there is little
narrative about the implementation and effectiveness
of applying MMs in highly complex environments
like hospitals (Waring & Currie, 2009). However,
such experience should have sustainable impact on
the development of such models itself (Blondiau et
100
Burmann, A. and Meister, S.
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies.
DOI: 10.5220/0010228601000110
In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 5: HEALTHINF, pages 100-110
ISBN: 978-989-758-490-9
Copyright
c
2021 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
al., 2016). Barret and Oborn describe the division
between practice and research as twofold: on the one
side researchers are “lost in translation” struggling in
explaining the relevance of theories to practice. On
the other side research is “lost before translation”
developing practically irrelevant theories (Barrett &
Oborn, 2018). In order to bridge the mentioned
research-practice-division, within this article we
present experiences from applying MMs as means for
achieving specific practical purposes in two separate
research projects. According to Braa & Vidgen (Braa
& Vidgen, 1999) the studying of IS artefacts in-
context is able to contribute to understanding the
“organizational laboratory”, the artefact itself and
integration mechanisms. We thus endeavour to
contribute to answer the following research questions
by investigating two practical applications of MMs:
1) What challenges does the practical field face
when applying MMs as tools for structured
development of healthcare provision?
2) What challenges arise from using MMs as
means to structurally cope with the digital
transformation in healthcare?
For this purpose, we firstly outline the relevant
conceptual background regarding maturity modelling
and modelling digital maturity in the healthcare
domain. Secondly, we present the research approach
and data collection methods, as well as the two
projects we derive our findings from. Subsequently,
we depict the project results and the observations
made from incorporating MMs into practice. We
conclude with a summary of the contributions,
limitations and potential for future research.
2 BACKGROUND
2.1 Maturity Modelling
The concept of modelling maturity originates from
the classification of software development processes
(Humphrey, 1988) within the Capability Maturity
Model (CMM) (Paulk et al., 1993). Since then, a
continuously rising number of publications with
regard to MMs have been made (Lee et al., 2019;
Wendler, 2012).
The modelling of maturity distinguishes between
general approaches and domain specific descriptions
of maturity aspects (Wendler, 2012). This is due to
the challenge of distinctive MMs of on the one hand
addressing individual challenges and generating a
hands-on benefit for the assessed organization, and at
the same time provide general expressiveness and
abstraction, for which the research community is
striving (Becker et al., 2009; Blondiau et al., 2016).
MMs mostly categorize a defined context in discrete
stages (Gottschalk, 2009; Schuh et al., 2017), and
differ in their degree of descriptivity and
prescriptivity (de Bruin et al., 2005).
Most of the existing measurement means were
developed following a top-down approach, which led
to the criticism regarding relevance and rigor of these
procedures. Following that, bottom-up methods and
tools have been proposed (Lahrmann et al., 2011;
Rönkkö et al., 2008).
Within this paper the focus lies on the depiction
of maturity with regard of general healthcare
provision, institutions and processes, and in particular
regarding digitalization aspects within this domain.
2.2 Digital Maturity in Healthcare
Healthcare provision substantially differs from
providing conventional services, or from production
in the manufacturing sector. This is obvious in the
requirements dynamics, interdisciplinarity and
human-centred process control (Söylemez & Tarhan,
2016). Furthermore, and contrary to other
organizations, actors in healthcare organizations are
not subordinate to a central strategy, but follow
diverging aims (e.g. different aims in different clinics,
conflicting economic and medical aims). For this
reason, the healthcare domain puts its trust in domain
specific models rather than adopting general models
from industry.
In 2016 Carvalho et al. identified 14 maturity
models dealing with information systems technology
in healthcare (Carvalho et al., 2016), and since then
others have been added (Gomes & Romão, 2018;
Kolukısa Tarhan et al., 2020).
The probably most widely applied MM approach
regarding digitalization in healthcare is the Electronic
Medical Record Adoption Model (EMRAM),
developed and provided by the Healthcare
Information and Management Systems Society
(HIMMS Analytics, 2017). EMRAM measures the
degree of the integration of an electronic health
record (EHR) of hospitals. This model was first
introduced in 2005, revised in 2018 and categorizes
hospitals in eight levels of maturity from 0 (no
digitalization) to 7 (paperless hospital) (Stephani et
al., 2019).
Other exemplary approaches in the hospital sector
dealing with digitalization aspects set a focus on the
maturation and evolvability of PACS (van de
Wetering & Batenburg, 2009) or digital imaging
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies
101
(Studzinski, 2017), the adoption of data analytics
(Sanders et al., 2013), the degree of correspondence
of the IT architecture with strategic goals (Mettler et
al., 2014; Mettler & Pinto, 2018), or the ability to
implement IT innovation (Esdar et al., 2017).
While a considerable number of methods
operationalize and measure particular maturity
aspects of healthcare institutions, Carvalho et al. in
their literature review remark, that no identified
model covers all organizational areas and systems of
healthcare organizations. This notation is e.g. picked
up by the regularly published “IT Healthcare Report”
(Hübner et al., 2015; Hübner et al., 2018; Hübner et
al., 2020), which points out one particular aspect per
issue. One of these aspects is the clinical information
logistics, operationalized in the workflow composite
score (WCS) (Liebe et al., 2015). The score breaks
down the availability of data along and across clinical
processes.
Most of the mentioned approaches neglect the
dependence of healthcare processes on interaction
(purely social as well as socio-technical) and
commitment of human actors. Krasuska et al. i.e.
identify multimodal capabilities relevant for “digital
excellence” in hospitals (Krasuska et al., 2020). Pak
& Song (Pak & Song, 2016) explicitly address the
human interaction aspect, while Burmann et al.
suggest a combination of technical and human factors
(Burmann et al., 2019).
However, successful implementation and
deployment in practice is scarcely described
(Blondiau et al., 2016). Mettler et al. addressed this
issue by discussing experiences and pitfalls from the
translation of their intervention-cycle into application
(Blondiau et al., 2016; Mettler, 2010). They conclude
that implementation of maturity assessments in
practice requires especially in hospitals a high level
of support (Blondiau et al., 2016; Caldwell & Atwal,
2003; Conwell et al., 2000).
3 METHODOLOGY
The presented research methodologically builds on a
structured literature review (SLR) and case study
research (CSR). The SLR provides a comprehensive
overview of the current status of research and
application. Thereby, substantial developments of
maturity modelling in general and particularly in the
healthcare domain were identified. The findings
presented in this article are based on the retrospective
analysis of two projects, which were conducted from
2017 to 2020. Both projects followed the assumptions
regarding case study research made by Yin (Yin,
1987). The presented analysis combines aspects from
the pragmatism and interpretivism paradigms in IS
research as explained by Goldkuhl (Goldkuhl, 2012).
The “data generation” followed the former since the
two projects were carried out in joint teams with
representatives from research and practice, while the
retrospective “data analysis” can be merely assigned
to interpretivism. The SLR served as a continuous
knowledge basis for both of the presented case
studies.
3.1 Literature Review
In order to gain an overview of the literature in the
field of maturity modelling, a structured literature
review, as suggested by Webster and Watson
(Webster & Watson, 2002) and vom Brocke et al.
(vom Brocke et al., 2009) was conducted. This search
was carried out accompanying both projects and
updated regularly. Since literature in the field of
Information Systems (IS) is gathered across a large
number of databases, a combination of search sources
is advised (Levy & J. Ellis, 2006). The databases
Scopus, AIS eLibrary and IEEEXplore were chosen
in order to incorporate the leading journals and
conference proceedings. Firstly, the authors aimed at
collecting the conceptual foundation of maturity
models in general and therefore used the search string
“maturity model” OR “maturity assessment” OR
“maturity measurement”. Subsequently, languages
other than English and were excluded, and the results
were limited to secondary sources. From initially over
6800 identified publications 118 documents
remained, of which 4 sources referred to
digitalization MMs. Via a backward and forward
search of the relevant literature we identified the
major works and contributions to general maturity
modelling and modelling the status of digitalization.
Additionally, a summary of activities with particular
regard to maturity modelling in the healthcare domain
was generated by combining the search string above
with AND “Hospital” OR “Healthcare” OR “clinical
workflow”. This search lead to a total number of 146
results, which were subsequently screened for actual
models presented. The total number of maturity
models particularly for the healthcare domain rose
from around 30 in 2017 to more than 60 in 2020
(Kolukısa Tarhan et al., 2020). In order to also
incorporate approaches which might not be described
on a peer reviewed research level, but are part of
practical application, the search was conducted in
GoogleScholar as well. The identified maturity
models served as a knowledge basis for solving
domain problems within several projects related to
HEALTHINF 2021 - 14th International Conference on Health Informatics
102
hospital digitalization since 2017, and particularly the
two referred to in this article. Section 3.4 describes
the two cases in further detail.
3.2 Case Study Research
In order to derive general findings associated with the
practical application of MMs in healthcare context
and contribute to answering the formulated research
questions the recent project past of the authoring
working group was screened for cases suitable to be
included into this analysis. Since we aimed at breadth
and generalization across different practical
scenarios, we identified two projects as heterogenous
as possible regarding timely distance to each other,
data collection means, combination of applied MMs,
type of action (intervention or description) and
number of maturity assessments. Both projects were
assigned to the field of case study research. A case
study generally analyses a real-world phenomenon,
and contributes to understanding a complex problem
(Yin, 2003). The investigation is defined in time and
location, and follows a specific research question
(Baxter & Jack, 2008; Ridder, 2016)., related to the
description of digitalization of healthcare processes,
were carried out under the CSR notion. Both projects
were carried out and analysed separate from each
other (Gustaffson, 2017). Section 3.4 describes the
adduced projects, while Table 1 compares the case’s
parameter.
3.3 Data Collection Means
Within the two projects primary qualitative data
collection methods were used. That encloses in-depth
interviews and focus groups (Gill et al., 2008) with
designated experts as well as guest visits and
observations of the daily working environment by one
of the researchers (shadowing) (Quinlan, 2008). The
in-depth interviews followed a predefined semi-
structured guideline, and were conducted by phone
and documented by the interviewer (Longhurst,
2010). The focus groups were carried out with
healthcare representatives and scientists from the
research institute on the site of the respective project
partner. Focus groups strengthen the relevance as
they enable joint problem-oriented research activity
with practitioners (Gill et al., 2008). Within these
focus groups processes were modelled and analysed
with regard to the current status of digitalization and
optimization potentials. Therefore, tools such as the
Business Process Model and Notation (BPMN)
(Allweyer, 2010), group discussions or the Digital
Imaging Adoption Model (DIAM) (Studzinski, 2017)
were used. The focus groups were documented by the
research representative (doctoral candidates in the
subject area) with whiteboards, flipcharts and
protocols.
3.4 Case Description
The two mentioned cases were driven by user needs
and aimed for different purposes. Common ground is
that both incorporated the application of maturity
modelling concepts into practice. They were carried
out independently and included into this analysis
retrospectively. The projects were initiated from
institutions of healthcare provision and can be
assigned to two specific and disparate functional
areas: diagnostic imaging (DI) and oral and
maxillofacial surgery (OMS). The project objectives
are summarized in the following, and the project
parameter relevant for this article are comparatively
depicted in table 1.
3.4.1 Case 1: Diagnostic Imaging
The scope of Project 1: Diagnostic Imaging was to
support the substitution of an analogue radiology
imaging system with a digital workplace solution and
to assess the impact of an increase of the degree of
digitalization on workflow efficiency. In order to
achieve that, six radiology departments (four within
hospitals and two in resident practice) were
identified, all of which were just about to upgrade
from an analogue to a digital imaging system. in a
first step the initial workflow and the stage of
development with regard to digitalization were
modelled. Therefore, as MM means parts from the
EMRAM were combined with aspects from DIAM.
Additionally, the classic process modelling notation
BPMN (Allweyer, 2010) was used. By using these
means, the initial process was modelled and assessed
regarding workflow and status of digitalization.
Additionally, workflow efficiency was monitored by
a person from the research team in a time frame of
two days of shadowing the everyday work routine.
Following that, the current workflow was analysed
regarding optimization potential through
digitalization. Based on this a digital imaging system
was selected according to the customers’ needs and
configured correspondingly to the greatest possible
extent. The respective imaging system was then
integrated into each of the six departments, and the
department teams were trained on the system.
Following an initial familiarization phase of 6-8
weeks, the new workflow was assessed again with
regard to process flow by applying the same means as
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies
103
used in the initial analysis (two days of shadowing).
Both assessments were comparatively examined and
changes in workflow efficiency were identified.
3.4.2 Case 2: OMS
Within Project 2: Oral and Maxillofacial Surgery the
goal was to describe the current status of
digitalization in this particular medical discipline,
with the goal to describe and share this with the
medical community. This description was supposed
to reveal fields of action, structural internal and
external barriers as well as best practices. In order to
achieve that goal, in a first step the standard
procedures and information flows of the discipline
were modelled along the intersectoral patient
pathway. Additionally, the literature review with
regard to modelling digital maturity in healthcare
settings was updated. Therefrom, suitable aspects of
MM tools were identified and merged into an
interview guideline, which was supposed to extract
the relevant information from direct interaction with
medical professionals from the discipline. The
interviews were restricted to a timeframe of 40
minutes. Subsequently, the interview protocols were
pseudonymized and confirmed by the person
interviewed. Following that, a combination of open
and axial coding protocol as suggested by Wiesche et
al. (Wiesche et al., 2017) was carried out. The coding
and grouping of text passages led to a content
classification and analysis. The analysis was split into
general findings about process bottlenecks of the
discipline, the digitalization status of the field OMS
Table 1: Categorization of the two projects implementing MM means.
Name Diagnostic Imaging Oral and Maxillofacial Surgery
Project goal Assess the initial radiology workflow based on
an analogue imaging system. Integrate a digital
imaging system and adapt customizable settings
suitable to an envisaged digital workflow.
Assess efficiency changes between workflows.
Assess the current status of digitalization of the
medical discipline “oral and maxillofacial
surgery”. Identify internal and external barriers
remarkable structural achievements and easily
accessible adjusting screws for leveraging the
potential of digitalization for the discipline.
Scenarios
investigated
3 overall: 1
st
in hospital, integration of a mobile
imaging system, 2
nd
in hospital, stationary
imaging system, 3
rd
in resident practice,
stationary imaging system
2: 1
st
in resident practice, 2
nd
in hospital
MM goal Assess, improve MM level, reassess, display
improvement
Assess, interpret, suggest actions
MMs used EMRAM, DIAM WCS, DHMI, EMRAM
MM proceeding Identification of meaningful parameter from the
mentioned models, reduction and fusion of the
models into a project specific set of maturity
assessment points
Identification of meaningful parameter from the
mentioned models, reduction, fusion, and
transfer to the relevant data points and interfaces
in OMS
Type intervention description
Assessing person External (researcher), in close collaboration
with the participants
External (researcher), in close collaboration
with the participants
Data collection
method
Focus groups and guest visit (shadowing) In-depth interviews
Number of
participants
number of participants according to team size
and availability with alternating line-up,
according to roster: 2 departments per scenario,
leading to a total number of 6 departments
8 in total, 4 per scenario
Profile of
participants
Mainly medical technical radiology assistants
(MTRA), and radiologists
Medical specialists categorized by years of
working experience (>10, <10), medical
assistants (at least one representative of each
role per scenario)
Data collection
points
3 per intervention: “before” (focus group and
shadowing), “during” installation (shadowing)
and “after” replacing an analogue with a digital
radiology system (shadowing)
1 interview per participant
Project Year 2017 2020
HEALTHINF 2021 - 14th International Conference on Health Informatics
104
and best-practices in both respects. Additionally,
external and internal influencing factors on successful
digitalization were clustered.
3.5 Retrospective Analysis
The two described projects were retrospectively
analysed with regard to the formulated research
questions. In a first step the project content, its goals
and the applied means and methods were
summarised. Following that, the conducted steps and
tools were comparatively displayed on a timeline.
The project documentation (interview transcripts,
focus group documentation and notes, workflow
analysis and notes from the shadowing sessions) was
then inductively analysed by a coding protocol
combining open and axial coding as suggested by
Wiesche et al. (Wiesche et al., 2017). The axial
grouping led to 5 classes of findings observed in both
cases associated with the formulated research
questions. The findings are described in section 4.3.
4 RESULTS
The project’s timelines of steps and methods
conducted showed a common proceeding of the
following steps: 1. to assess the specific user goals, 2.
To narrow down the object that was to be described,
3. to identify a suitable tool to depict the respective
field of interest, 4. to adapt available tools to the
specific need, 5. to translate this into a real
application, 6. to derive knowledge from this
application and optional (only for the diagnostic
imaging project) 7. to integrate an intervention in
order to increase the digital maturity, and 8. to
reassess and examine efficiency differences between
the initial and final workflow. The application of
MMs was thus not an end in itself, and not
particularly scope of the observation, but served as a
tool for achieving a practical goal. However, the
translation of formalized MMs into practice is still a
field of research these projects contribute to.
Therefore, the following sections briefly outline the
two project’s results regarding their general scope,
and present central findings from the application of
MMs in these two healthcare contexts.
4.1 Case 1: Diagnostic Imaging
The efficiency of diagnostic imaging workflows in
resident practice as well as in hospitals based on an
analogue imaging system was assessed. Following
the replacement of the analogue with a digital
imaging system, the efficiency of the old workflow
was compared to the new workflow in the same
setting. In the resident practices a stationary imaging
system and in the hospital environment stationary and
mobile systems were examined. For each scenario,
two systems from differing manufacturers were
compared to a digital system from one manufacturer
(who also fabricated one of the initial systems in each
scenario). The digital workflow was found to be more
efficient compared to the analogue workflow to a
varying extent in all the scenarios. The comparison of
two different manufacturers of the initial system
showed no significant divergence by contrast with the
digital systems for none of the scenarios. Since this
project was commissioned by the digital system’s
manufacturer and the results intended for internal
purposes only, the study has not in all its details been
made available to the community.
4.2 Case 2: MOS
Based on the conducted interviews, the general
workflow, involved parties and actors, interfaces,
data and transfer points were described. The
digitalization status of the working environment,
workflow and the organizations was modelled based
on the set of maturity parameter merged from WCS,
DHMI and EMRAM for each interviewee as well as
abstracted across all participants. General
digitalization hurdles like literacy and sovereignty in
handling digital services and workflows, or the lack
of integrated process support were identified.
Particularly within hospitals, the competition of
disciplines for investment budgets was mentioned as
an inhibiting factor. The development status of
digitalization especially between resident practices
was highly heterogeneous. The close interrelationship
with the field of diagnostic imaging was identified as
a driver for development: innovations often diffuse
from this area to the discipline of OMS. The
comprehensive analysis was made available to the
medical discipline and the scientific community in a
separate article (Meister et al., 2020).
4.3 Retrospective Findings across
Cases
Only sparse experience from practical application of
MMs in healthcare has been reported thus far.
Blondiau et al. (Blondiau et al., 2016) made their
experiences available, based on the intervention-
cycle presented by Mettler (Mettler, 2010),
differentiating between findings related to MM
design and implementation. Referring to the latter, the
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies
105
authors brought MMs into application, and identified
challenges exceeding the ones addressed by the
experiences and suggestions by Blondiau et al. In this
section the authors present and formalize 5
observations noted in both of the two case studies
with regard to translating MMs into real-world
application in healthcare context. The observations
are as follows:
1) Continuous Translation between Research and
Practitioners is Required.
Not only the knowledge of the relevant state of the art
regarding MMs was found to be fairly limited on the
practitioners’ side (which is understandable since it is
a huge field and not particularly the core business of
the hospitals’ representatives). At the same time,
although the interest to be involved and contribute to
the arrangement was present, the comprehensibility
of the available literature seemed rather formalized
and impractical to the user groups. This led to the
necessity of intensive collaboration between research
and practice for identification, extraction and merging
of suitable means into a depiction protocol. Also
during the protocol execution, a close guidance and
affirmation was asked for. This emblematically
shows the misunderstanding between research and
practice. Researchers are striving for abstraction and
formalization and not necessarily provide an
overview of explicit technical or processual
innovations and best practices. On the other hand,
practitioners are hoping for precise details and
guidance. The interface in our two projects was
bridged by human efforts. Both sides need to move
towards each other by the means of education and
simplification.
2) Implementation of MMs and Taking Actions
based on This Requires Support of All Affected
Professional Disciplines and Hierarchy Levels.
In Hospitals we have the special situation, that
managerial and medical leaders work detached from
each other in terms of content and are not authorized
to issue instructions to each other with regard to the
specific professional matters handled by the
respective division. Since interests between these
areas not always fully overlap, joint efforts require the
involvement and support of all affected stakeholders.
Vice versa, that implies that MMs which explicitly
target a particular profession, while the object of
examination (or especially the evolution of it) affects
areas or actors beyond the included group, are hardly
able to contribute to mutual interests. While there are
reasonable intentions of addressing an explicit
profession with a MM (e.g. for tailoring the language
or conceptual level for the needs of the user group
(Blondiau et al., 2016), or strengthening the
negotiation position of individuals), the concomitant
non-consideration of other interest groups may even
affect the implementation of measures adversely.
3) Structural Development of Hospitals Requires
Modular and Holistic MMs.
Numerous MMs with differing level of detail and
focus exist, some of which were found suitable for
depicting a project-relevant aspect of maturity within
very specific boundaries. However, neither a
conceptually holistic and encompassing MM nor one
suitable approach for that very specific project goal
could be identified. This can of course vary in
conjunction with the respective area or goal one
wants to examine or achieve. Nonetheless, for the two
projects we had to extract different aspects from
different MMs and combine them into a presentation
appropriate for the respective group of practitioners.
We certainly did not expect to identify a perfectly
suitable MM able to model the two specific medical
disciplines, workflows or data exchange points.
Addressing the conducted effort in both project of
merging suitable modules of existing MMs into a
project-specific set of parameters by the development
of a modular and holistic MM can significantly
reduce the application hurdles for representatives
from practice. From a scientific point of view the
focus on depth rather than breadth in MMs makes
sense: comparability and schematization of similarly
parameterizable aspects naturally leads to narrowing
the observation field. At the same time, it impairs
practical application, since seldom only one particular
aspect is part of a real-world project. In order to
increase value creation for the hospital domain
multivariate and modular analysis means, or an
integration of specialized solutions are required.
4) A Differentiation between “Visible Result” and
Necessary Action to Achieve That Result is
Required.
MMs usually offer a path of evolution with regard to
the specific aspect of examination. This presentation
suggests necessary actions for achieving a higher
level of maturity by displaying the required
characteristics for the next stage. However, especially
MMs dealing with digitalization tend to derive a
maturity model from a set of “checkable
characteristics”. This seems intuitive since
digitalization becomes obviously technologically
verifiable. At the same time, it neglects that the digital
transformation requires processual and organizational
action, which eventually leads to a technological
HEALTHINF 2021 - 14th International Conference on Health Informatics
106
status. So checkable maturity parameter and
evolutionary actions not necessarily need to be the
same. Most MMs do not clearly distinguish between
these “checkable characteristics” and necessary
actions to achieve such a state. The development of
an organization like a hospital is multivariate and
complex itself. Correlation and causation remain
largely undescribed. Especially MMs with a focus on
data flows and degree of information integration
suggest a rather technical approach to developing a
state of digital maturity. From our experience, the
depiction of such “measurable” indicators may be
mistaken by practitioners for the necessary actions
required to achieve the next level. Especially in such
an interprofessional environment this outcome may
not in all cases be achieved by “just” implementing
technological requirements for the next maturity
level. E.g. in rather technical MMs the processual,
organizational and human prerequisites are necessary
to shape the digital transformation successfully and
encompassing. We suggest a stronger emphasis on
the differentiation between measurable maturity level
parameter and evolutionary actions required to
achieve the next stage.
5) The Static Definition of Maturity States in Times
of Disruptive Change Needs to be Questioned.
Some MMs provide a firm definition of maturity
levels, and some comprise a certain degree of
dynamics and adaptability to future developments.
The general concept of maturity modelling in the past
decades was mainly intended to support incremental
development. Today, great uncertainty engages the
healthcare provision domain on how to shape the
disruptiveness of the digital revolution. A clear and
encompassing vision of the future digital hospital is
neither by practice nor by research formulated yet.
This is also reflected in the respective definitions of
high sophistication in existing MMs. Therefore, the
depiction of the current status of maturity of a certain
skill, process or organization seemed capable within
the two applications. On the other hand, the
evolutionary paths provided by the selected MMs
could not cope with the expectations regarding
disruptive reorganization of healthcare provision.
Some of the selected MMs even acknowledge this
uncertain target state and leave the room for future
definition. That certainly reflects the current situation
of the domain in a sufficient way from the research
perspective. At the same time, it leaves practice alone
with the empty room the anticipated disruptiveness of
the digital revolution provides. Statically defined
maturity states are only able to describe our current
knowledge and imagination. However, they set
boundaries when they are used as means to
prospectively shape disruptive change. It could be
incorporated explicitly that current depictions only
reflect current knowledge, and that the digital
transformation requires solutions and arrangements
exceeding that. Nonetheless, in order to increase
value creation for practitioners in the future, this
uncertainty needs to be addressed by joint efforts to
foster clarification (this probably exceeds research
and practice and also requires politics and society) of
how healthcare delivery should be transformed in the
digital age.
5 CONCLUSIONS
With this article we intend to support building the
bridge between the developers and implementers of
MMs. The growing number of publications with
regard to maturity models emphasizes developments
and discussions of design methodologies rather than
reports on practical application and efficacy of that
implementation itself and potential change efforts
(Blondiau et al., 2016).
The need for application guidance, as described
by Caldwell and Atwal as a result of the
interdisciplinary organization “hospital” (Caldwell &
Atwal, 2003), was found to be still very high.
In both depicted cases, the definition of an
envisaged digital target state required a joint effort
between practitioners and researchers. A systematic
uncertainty within the domain of how exactly the
“digital hospital” as a maturity status can be defined
remains. For the purpose of guiding through the
disruptiveness and dynamics of the digital
transformation, existing MMs were perceived as too
focused on incremental development and static states.
With this prevailing design the capability of MMs of
guiding the healthcare domain through the
prospectively postulated and not yet fully described
digital revolution is arguable. However, applying
MMs with the aim of monitoring progress as
described in case 1 was found appropriate.
5.1 Contribution
The presented article provides various practical and
managerial implications. The observations derived
from projects applying MMs in practice may serve
practitioners and collaborative implementing teams
in their decision making when it comes to reflecting
the practical capabilities and expectations with
applying MMs. Furthermore, insights into challenges
to expect in the process of identifying, adapting and
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies
107
implementing suitable means for a specific maturity
purpose are provided.
At the same time, the observations presented are
equally suited to be taken into account by the
developers of MMs. The consideration of these
during the creation of MM solutions may contribute
to the practical applicability and thus on the value
creation for the practitioner’s scope of application as
well as the structural development of healthcare
organizations in general.
5.2 Limitations
At this stage, however, it is necessary to also share the
limitations of the presented study. This particularly
refers to the generalizability of the results. Firstly, the
findings result from two case studies from very
delimitable medical disciplines, and the
transferability to other fields or wider scopes is not
necessarily given. Secondly, the observed
implications arose from the interaction of humans
during the projects. We acknowledge the theory
formulated by Walsham, that the enquirer’s
interaction with the study object or environment and
the perception of all involved parties not only
influences the sensing of events, but also the reality
itself (Walsham, 1995). Perception and interaction
impact the reality created within a situation and such
observations are thus not compulsory applicable to
other social settings or general environments.
5.3 Future Research
Prospectively, it will be essential to invest further
research into the determination of success factors and
capabilities a hospital must have to be able to
collaboratively shape the change for the benefit of the
patient. The expert organization itself is only partially
described and can thus hardly be evolved structurally.
Therefore, further knowledge about the underlying
causal relationships of communicated and subliminal
aspirations of individuals and professional groups in
this special setting of an expert organization is
required.
Besides, further invest is needed in the definition
of a common ground of a potential “target state” of
the digital hospital.
Furthermore, since the transferability of the
presented observations is limited due to the two cases,
insights into the work of other research or practitioner
groups with regard to practical application of MMs
would be beneficial in the future.
ACKNOWLEDGEMENTS
Parts of the research presented in this paper stems
from the Center of Excellence for Logistics and IT
located in Dortmund (Leistungszentrum Logistik &
IT).
REFERENCES
Allweyer, T. (2010). Bpmn 2.0: Introduction to the
standard for business process modeling. Books on
Demand.
Barrett, M., & Oborn, E. (2018). Bridging the research-
practice divide: Harnessing expertise collaboration in
making a wider set of contributions. Information and
Organization, 28(1), 44–51. https://doi.org/10.1016/j.
infoandorg.2018.02.006
Bauernhansl, T. (2014). Die Vierte Industrielle Revolution
Der Weg in ein wertschaffendes
Produktionsparadigma. In T. Bauernhansl, M. ten
Hompel, & B. Vogel-Heuser (Eds.), Industrie 4.0 in
Produktion, Automatisierung und Logistik (pp. 5–35).
Springer Vieweg. https://doi.org/10.1007/978-3-658-
04682-8_1
Baxter, P., & Jack, S. J. (2008). Qualitative Case Study
Methodology: Study Design and Implementation for
Novice Researchers. The Qualitative Report, 13, 544–
559.
Becker, J., Knackstedt, R., & Pöppelbuß, J. (2009).
Developing Maturity Models for IT Management.
Business & Information Systems Engineering, 1(3),
213–222. https://doi.org/10.1007/s12599-009-0044-5
Blondiau, A., Mettler, T., & Winter, R. (2016). Designing
and implementing maturity models in hospitals: An
experience report from 5 years of research. Health
Informatics Journal, 22(3), 758–767. https://doi.org/
10.1177/1460458215590249
Braa, K., & Vidgen, R. (1999). Interpretation, intervention,
and reduction in the organizational laboratory: a
framework for in-context information system research.
Accounting, Management and Information
Technologies, 9(1), 25–47. https://doi.org/10.1016/
S0959-8022(98)00018-6
Burmann, A., Deiters, W., & Meister, S. (2019). Digital
Health Maturity Index: Analyse des
Digitalisierungsgrades im Krankenhaus. In M. A.
Pfannstiel, P. Da-Cruz, & H. Mehlich (Eds.), Digitale
Transformation von Dienstleistungen im
Gesundheitswesen VI (pp. 3–18). Springer Fachmedien
Wiesbaden.
Caldwell, K., & Atwal, A. (2003). The problems of
interprofessional healthcare practice in hospitals.
British Journal of Nursing (Mark Allen Publishing),
12(20), 1212–1218. https://doi.org/10.12968/bjon.
2003.12.20.11844
Carvalho, J. V., Rocha, Á., & Abreu, A. (2016). Maturity
Models of Healthcare Information Systems and
HEALTHINF 2021 - 14th International Conference on Health Informatics
108
Technologies: A Literature Review. Journal of Medical
Systems, 40(6), 131. https://doi.org/10.1007/s10916-
016-0486-5
Conwell, C. L., Enright, R., & Stutzman, M. A. (2000,
December). Capability Maturity Models support of
modeling and simulation verification, validation, and
accreditation. In 2000 Winter Simulation Conference
Proceedings (Cat. No.00CH37165) (pp. 819–828).
IEEE. https://doi.org/10.1109/WSC.2000.899880
de Bruin, T., Freeze, R., Kulkarni, U., & Rosemann, M.
(2005). Understanding the Main Phases of Developing
a Maturity Assessment Model. Australasian
Conference on Information Systems.
Deiters, W., Burmann, A., & Meister, S. (2018).
Digitalisierungsstrategien für das Krankenhaus der
Zukunft [Strategies for digitalizing the hospital of the
future]. Der Urologe. Ausg. A, 57(9), 1031–1039.
https://doi.org/10.1007/s00120-018-0731-2
Esdar, M., Liebe, J.-D., Weiß, J.-P., & Hübner, U. (2017).
Exploring Innovation Capabilities of Hospital CIOs:
An Empirical Assessment. Studies in Health
Technology and Informatics, 235, 383–387.
Gill, P., Stewart, K., Treasure, E., & Chadwick, B. (2008).
Methods of data collection in qualitative research:
Interviews and focus groups. British Dental Journal,
204(6), 291–295. https://doi.org/10.1038/bdj.2008.192
Goldkuhl, G. (2012). Pragmatism vs interpretivism in
qualitative information systems research. European
Journal of Information Systems, 21(2), 135–146.
https://doi.org/10.1057/ejis.2011.54
Gomes, J., & Romão, M. (2018). Information System
Maturity Models in Healthcare. Journal of Medical
Systems, 42(12), 235. https://doi.org/10.1007/s10916-
018-1097-0
Gottschalk, P. (2009). Maturity levels for interoperability in
digital government. Government Information Quarterly,
26(1), 75–81. https://doi.org/10.1016/j.giq.2008.03.003
Gustaffson, J. T. (2017). Single case studies vs. multiple
case studies: A comparative study. Sociology.
Hermann, M., Pentek, T., & Otto, B. (2016, January).
Design Principles for Industrie 4.0 Scenarios. In 2016
49th Hawaii International Conference on System
Sciences (HICSS) (pp. 3928–3937). IEEE.
https://doi.org/10.1109/HICSS.2016.488
HIMMS Analytics. (2017). „Electronic Medical Record
Adoption Model“. http://www.himss.eu/healthcare-
providers/emram
Hübner, U., Esdar, M., Hüsers, J., Liebe, J.-D., Naumann,
L., Thye, J., & Weiß, J.-P. (2020). IT-Report
Gesundheitswesen: Wie reif ist die Gesundheits-IT aus
Anwenderperspektive? Schriftenreihe der Hochschule
Osnabrück.
Hübner, U., Esdar, M., Hüsers, J., Liebe, J.-D., Rauch, J.,
Thye, J., & Weiß, J.-P. (2018). IT-Report
Gesundheitswesen: Wie reif ist die IT in deutschen
Krankenhäusern? Hochschule Osnabrück - IGW.
Hübner, U., Liebe, J.-D., Hüsers, J., Thye, J., Egbert, N.,
Hackl, W., & Ammenwerth, E. (2015). IT-Report
Gesundheitswesen: Pflege im Informationszeitalter.
Schriftenreihe der Hochschule Osnabrück.
Humphrey, W. S. (1988). Characterizing the software
process: a maturity framework. IEEE Software, 5(2),
73–79. https://doi.org/10.1109/52.2014
Kolukısa Tarhan, A., Garousi, V., Turetken, O., Söylemez,
M., & Garossi, S. (2020). Maturity assessment and
maturity models in health care: A multivocal literature
review. Digital Health, 6, 2055207620914772.
https://doi.org/10.1177/2055207620914772
Krasuska, M., Williams, R., Sheikh, A., Franklin, B. D.,
Heeney, C., Lane, W., Mozaffar, H., Mason, K., Eason,
S., Hinder, S., Dunscombe, R., Potts, H. W. W., &
Cresswell, K. (2020). Technological Capabilities to
Assess Digital Excellence in Hospitals in High
Performing Health Care Systems: International eDelphi
Exercise. Journal of Medical Internet Research, 22(8),
e17022. https://doi.org/10.2196/17022
Lahrmann, G., Marx, F., Mettler, T., Winter, R., &
Wortmann, F. (2011). Inductive Design of Maturity
Models: Applying the Rasch Algorithm for Design
Science Research. In H. Jain, A. P. Sinha, & P.
Vitharana (Eds.), Lecture Notes in Computer Science.
Service-Oriented Perspectives in Design Science
Research (Vol. 6629, pp. 176–191). Springer Berlin
Heidelberg. https://doi.org/10.1007/978-3-642-20633-
7_13
Lee, D., Gu, J. W., & Jung, H. W. (2019). Process
maturity models: Classification by application sectors
and validities studies. Journal of Software: Evolution
and Process, 31(4), e2161. https://doi.org/
10.1002/smr.2161
Leistungszentrum Logistik & IT. http://leistungszentrum-
logistik-it.de
Levy, Y., & J. Ellis, T. (2006). A Systems Approach to
Conduct an Effective Literature Review in Support of
Information Systems Research. Informing Science: The
International Journal of an Emerging Transdiscipline,
9, 181–212. https://doi.org/10.28945/479
Liebe, J.-D., Hübner, U., Straede, M. C., & Thye, J. (2015).
Developing a Workflow Composite Score to Measure
Clinical Information Logistics. A Top-down Approach.
Methods of Information in Medicine, 54(5), 424–433.
https://doi.org/10.3414/ME14-02-0025
Longhurst, R. (2010). Semi-structured interviews and focus
groups. In N. J. Clifford & G. Valentine (Eds.), Key
methods in geography (2nd ed., pp. 103–115). Sage
Publications.
Meister, S., Burmann, A., & Deiters, W. (2019). Digital
Health Innovation Engineering: Enabling Digital
Transformation in Healthcare: Introduction of an
Overall Tracking and Tracing at the Super Hospital
Aarhus Denmark. In N. Urbach & M. Röglinger (Eds.),
Management for Professionals. Digitalization Cases
(pp. 329–341). Springer International Publishing.
https://doi.org/10.1007/978-3-319-95273-4_17
Meister, S., Haßfeld, S., & Burmann, A. (2020).
Digitalisierung in der Mund-, Kiefer- und
Gesichtschirurgie: Vom Status-Quo zur Vision. Der
MKG-Chirurg.
Mettler, T. (2010). Supply-Management im Krankenhaus:
Konstruktion und Evaluation eines konfigurierbaren
Practical Application of Maturity Models in Healthcare: Findings from Multiple Digitalization Case Studies
109
Reifegradmodells zur zielgerichteten Gestaltung (1.
Aufl.). Sierke.
Mettler, T., Fitterer, R., Rohner, P., & Winter, R. (2014).
Does a hospital’s IT architecture fit with its strategy?
An approach to measure the alignment of health
information technology. Health Systems, 3(1), 29–42.
https://doi.org/10.1057/hs.2013.10
Mettler, T., & Pinto, R. (2018). Evolutionary paths and
influencing factors towards digital maturity: An
analysis of the status quo in Swiss hospitals.
Technological Forecasting and Social Change, 133,
104–117. https://doi.org/10.1016/j.techfore.2018.03.
009
Pak, J., & Song, Y.-t. (2016). Health Capability Maturity
Model: Person-centered approach in Personal Health
Record System. In Surfing the IT innovation wave:
22nd Americas Conference on Information Systems
(AMCIS 2016) : San Diego, California, USA, 11-14
August 2016. Curran Associates Inc.
Paulk, M. C., Curtis, B., Chrissis, M. B., & Weber, C. V.
(1993). Capability maturity model, version 1.1. IEEE
Software, 10(4), 18–27. https://doi.org/10.1109/52.
219617
Pöppelbuß, J., Niehaves, B., Simons, A., & Becker, J.
(2011). Maturity Models in Information Systems
Research: Literature Search and Analysis.
Communications of the Association for Information
Systems, 29. https://doi.org/10.17705/1CAIS.02927
Pöppelbuß, J., & Röglinger, M. (2011). What makes a
useful maturity model? A framework for general design
principles for maturity models and its demonstration in
business process management. 19th European
Conference on Information Systems (ECIS).
Quinlan, E. (2008). Conspicuous Invisibility: Shadowing as
a Data Collection Strategy. Qualitative Inquiry, 14(8),
1480–1499. https://doi.org/10.1177/1077800408318318
Ridder, H.-G. (2016). Sozialwissenschaftliche
Forschungsmethoden. Case Study Research:
Approaches, Methods, Contribution to Theory (B.
Kittel, C. F. Altobelli, C. Strobl, G. Tutz, H. Hinz, I.
Borg, J. Reinecke, J. Wagner, K.-U. Schnapp, M. Spieß,
M. Berlemann, M. Kraft, P. Sedlmeier, R. Schnell, R.
Oesterreich, S. Liebig, U. Jirjahn, W. Seidel, & W.
Matiaske, Eds.). Rainer Hampp Verlag.
Rönkkö, M., Järvi, A., & Mäkelä, M. M. (2008). Measuring
and Comparing the Adoption of Software Process
Practices in the Software Product Industry. In Q. Wang,
D. Pfahl, & D. M. Raffo (Eds.), Lecture Notes in
Computer Science. Making Globally Distributed
Software Development a Success Story (Vol. 5007,
pp. 407–419). Springer Berlin Heidelberg.
https://doi.org/10.1007/978-3-540-79588-9_35
Sanders, D., Burton, D., & Protti, D. (2013). Healthcare
Analytics Adoption Model: A Framework and Roadmap.
https://www.healthcatalyst.com/white-
paper/healthcare-analytics-adoption-model/
Schuh, G., Anderl, R., Gausemeier, J., & Hompel, M. ten
(2017). Industrie 4.0 Maturity Index.
Söylemez, M., & Tarhan, A. (2016). The Use of
Maturity/Capability Frameworks for Healthcare
Process Assessment and Improvement. In P. M. Clarke,
R. V. O'Connor, T. Rout, & A. Dorling (Eds.),
Communications in Computer and Information Science.
Software Process Improvement and Capability
Determination (Vol. 609, pp. 31–42). Springer
International Publishing. https://doi.org/10.1007/978-
3-319-38980-6_3
Stephani, V., Busse, R., & Geissler, A. (2019).
Benchmarking der Krankenhaus-IT: Deutschland im
internationalen Vergleich. In J. Klauber, M. Geraedts, J.
Friedrich, & J. Wasem (Eds.), Krankenhaus-Report
2019 (pp. 17–32). Springer Berlin Heidelberg.
https://doi.org/10.1007/978-3-662-58225-1_2
Studzinski, J. (2017). Bestimmung des Reifegrades der IT-
gestützten klinischen Bildgebung und Befundung mit
dem Digital Imaging Adoption Model: Ist Ihre klinische
Bildgebung bereit für das digitale Zeitalter?
[Evaluating the maturity of IT-supported clinical
imaging and diagnosis using the Digital Imaging
Adoption Model: Are your clinical imaging processes
ready for the digital era?]. Der Radiologe, 57(6), 466–
469. https://doi.org/10.1007/s00117-017-0253-8
van de Wetering, R., & Batenburg, R. (2009). A PACS
maturity model: A systematic meta-analytic review on
maturation and evolvability of PACS in the hospital
enterprise. International Journal of Medical
Informatics, 78(2), 127–140. https://doi.org/10.1016/
j.ijmedinf.2008.06.010
vom Brocke, J., Simons, A., Niehaves, B., Riemer, K.,
Plattfaut, R., & Cleven, A. (2009). Reconstructing the
Giant: On the Importance of Rigour in Documenting
the Literature Search Process. http://www.Alexandria.
Unisg.Ch/Publikationen/67910.
Walsham, G. (1995). The Emergence of Interpretivism in
IS Research. Information Systems Research, 6(4), 376–
394. https://doi.org/10.1287/isre.6.4.376
Waring, J., & Currie, G. (2009). Managing Expert
Knowledge: Organizational Challenges and Managerial
Futures for the UK Medical Profession. Organization
Studies, 30(7), 755–778. https://doi.org/10.1177/
0170840609104819
Webster, J., & Watson, R. T. (2002). Analyzing the Past to
Prepare for the Future: Writing a Literature Review.
MIS Quarterly, 26(2), xiii–xxiii. www.jstor.org/stable/
4132319
Wendler, R. (2012). The maturity of maturity model
research: A systematic mapping study. Information and
Software Technology, 54(12), 1317–1339.
https://doi.org/10.1016/j.infsof.2012.07.007
Wiesche, M., Jurisch, M. C., Yetton, P. W., & Krcmar, H.
(2017). Grounded Theory Methodology in Information
Systems Research. MIS Quarterly, 41(3), 685–701.
https://doi.org/10.25300/MISQ/2017/41.3.02
Yin, R. K. (1987). Case study research: Design and
methods (6. print). Applied social research methods
series: Vol. 5
. Sage Publ.
Yin, R. K. (2003). Case study research: Design and
methods (3. ed.). Applied social research methods
series: Vol. 5. Sage. http://www.loc.gov/catdir/
enhancements/fy0658/2002152696-d.html
HEALTHINF 2021 - 14th International Conference on Health Informatics
110