ADmed: An Adaptive Technical Process for the Agile Development of
Medical Devices
Maren Martens, Anna Schidek
a
, Markus Schmidtner
b
and Holger Timinger
c
Institute for Data and Process Science, Am Lurzenhof 1, Landshut, Germany
Keywords:
Medical Devices, Adaptive, Process Model, ADmed, Flexible Development.
Abstract:
Agile project management is an established approach in software development and over time has also been
adapted to different fields of work. While there are proven advantages for an agile development process, not
all branches have incorporated agile methods in their project and development practices yet. One of these
branches is the medical device industry. They often fall back on traditional, plan-based process models due
to regulatory and normative requirements because of perceived conformity. ADmed is a process model that
combines agile and plan-based processes while still adhering to the necessary regulatory requirements. In
order to make it more accessible and approachable for a wide range of users it also incorporates adaptive
elements, which are adjusted based on the user context.
1 INTRODUCTION
In everyday life, medical devices support and improve
the quality of life in different ways (Austromed, 2018)
ranging from daily products for fitness diagnos-
tics to high end devices for medical operations. The
medical device industry covers a wide field of differ-
ent products. This leads to diverse industry, which
in Germany alone generates sales of over 34 billion
Euro in 2021 (BVMed e.V., 2021). Sales have been
steadily rising over the last few years and with an ag-
ing society the demand is expected to rise even fur-
ther. It is clear, that the medical is an important eco-
nomic factor not only in Germany but worldwide. De-
spite this the industry is also facing new challenges
(BVMed e.V., 2021). Like in most other branches,
digitalization is currently one of the main topics in
developing new medical devices. While it may re-
quire new ideas and expertise on the developer side,
it is also an opportunity to create new and innova-
tive products (Dispan, 2020). Companies have re-
alized this challenge and currently about 9% of the
generated turnover is reinvested into research and de-
velopment. The industry is splitting their efforts be-
tween developing new products and further improv-
ing existing medical devices (BVMed e.V., 2021). As
a
https://orcid.org/0000-0003-3724-4626
b
https://orcid.org/0000-0002-1209-7968
c
https://orcid.org/0000-0001-7992-0392
medical devices are usually very close to patients or
users, there is always a strict focus on the safety of
every person involved, which includes the product
safety and also the product usability (Donaldson et al.,
2021). Both of these aspects can be achieved by tra-
ditional, plan-based project management, but they are
also perfectly achievable with an agile development
approach. Especially the iterative, incremental ap-
proach and the involvement of different stakeholders
ensure not only the usability but also that the critical
functionality is often tested multiple times due to the
iterations. While agile methods can provide valuable
advantages during product development, there is still
hesitation from developers to employ them. This is
mainly caused by caution as they want to guarantee
compliance with the strict regulations, specifications
and guidelines to which they must adhere. So they
trust their established methods instead of incorporat-
ing new and possibly beneficial methods. However,
many manufacturers desire more agility in manage-
ment and execution of medical device projects (Jon-
nalagadda et al., 2019). The innovative, flexible pro-
cess model ADmed (Agile Development of Medical
Devices) was developed in response to this desire for
more agility and demonstrates how the compatibility
of plan-based and agile development with the spec-
ifications of regulations and standards can be real-
ized (Schidek and Timinger, 2022). The model of-
fers insight in how agile methods in the development
of medical devices can be achieved. However, it re-
Martens, M., Schidek, A., Schmidtner, M. and Timinger, H.
ADmed: An Adaptive Technical Process for the Agile Development of Medical Devices.
DOI: 10.5220/0011543100003335
In Proceedings of the 14th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2022) - Volume 3: KMIS, pages 177-184
ISBN: 978-989-758-614-9; ISSN: 2184-3228
Copyright
c
2022 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
177
quires developers and project managers to understand
agile methods in order to be able to use it. As agile
methods are rather new in the field of medical device
development, the knowledge about them on the prac-
titioners side is on an early level. To alleviate this
drawback, the ADmed model could be reconstructed
as an adaptive process model that supports and guides
its user throughout the tailoring of the process model.
2 ADmed MODEL
This section describes the structure of the ADmed
Model with respect to its functionality and the roles
that take part in the process model.
2.1 Structure and Functionality of
ADmed
2.1.1 Top Level
Figure 1 provides an overview of the basic structure
of the process model ADmed. It consists in parts of
traditional, plan-based project phases, which makes
the model rather easy to understand. The main phases
are initialization, project design, realization and extra
phases especially for a final verification and valida-
tion, and the design transfer of the medical device. In
the realization phase, the model offers a potentially
iterative way of working. The top level of the pro-
cess model ADmed is in line with the medical device
regulation of the European Union, detailed in the har-
monized standard EN ISO 13485, chapter 7. As more
details on each phase are required, beneath the more
detailed level of each phase is presented.
Figure 1: Main structure of the flexible process model
(Schidek and Timinger, 2022).
2.1.2 Detailed Level
In order to show the relevant activities for the process
as well as the agile building blocks that were specifi-
cally set, the process model was expanded by a more
detailed level, shown in Figure 2, in addition to the
top level and is explained in the following.
Even before the actual project start, the process
model begins with the initialization. According to the
traditional understanding of project management, this
phase is regarded as an orientation to obtain a rough
overview of the project and its scope. Checking the
necessary resources in the company is also part of
this step. After the official project release, the project
design begins. In this phase, the framework condi-
tions for the implementation of the project are de-
fined. Starting with the definition of the user needs (as
well as the superior intended use of the medical de-
vice), the first agile building blocks are used here. Re-
quirements are defined together by the entire project
team – and in particular by the relevant stakeholders
on the basis of user needs, in the form of user stories,
use cases or similar. By involving relevant stakehold-
ers, requirements can be gathered qualitatively with-
out expecting quantitative details from them, which
often cannot be formulated directly at the beginning
of the project. In order to find concrete user needs
as early as possible, this process can take place in an
iterative form and in collaboration with the stakehold-
ers. Various creative methods and techniques can be
used for this purpose. The formulated user needs are
recorded in the backlog in a way that is comprehensi-
ble and accessible to the entire project team. Parallel
to this step, the risk management process according
to EN ISO 14971 can already be started here and the
first cycle can be run through. For the sake of clarity,
the risk management activities are not shown in the
rest of the process model. These are considered to be
a prerequisite with the start of the risk management
process and should be carried out in parallel with the
development. In the next step of the project design,
further framework conditions for the implementation
of the project are set by the project manager. These
include prioritizing the backlog and defining the as-
sociated definition of ”done”, assembling the project
team on the basis of the required competencies and
defining the time box for the iteration and the activ-
ities defined therein. Only when these factors have
been set, the next phase can begin. The realization is
executed iteratively and incrementally. For this pur-
pose, user needs prioritized in the backlog are first
transferred into quantitative requirements during iter-
ation planning and an iteration target is defined jointly
by the project team and recorded in the iteration back-
log. The requirements definition is taken over directly
by the developers similar to scrum in order to
strengthen the necessary understanding and commit-
ment. Planned activities for verification and valida-
tion of the set requirements are also already defined.
To focus on implementation and to improve com-
munication within the project team, short daily stand-
up meetings are conducted. They also help to com-
municate the upcoming work of the day of each team
member and to address potential impediments. As
KMIS 2022 - 14th International Conference on Knowledge Management and Information Systems
178
Figure 2: Detailed view of the defined activities within the individual phases (Schidek and Timinger, 2022).
a result of the implementation of the iteration back-
log, a functional product increment is created, which
is then evaluated by the entire project team and com-
pared with the specified definition of ”done” in the
subsequent review. At this point, relevant stakehold-
ers can make possible change requests. However, the
EU regulations require that changes must not acci-
dentally introduce additional risks or changes of the
intended use of the medical device.
For this reason, potential changes follow a well-
defined change management process. Approved
changes are documented in the backlog in a traceable
manner, as required by EN ISO 13485. Another qual-
ity assurance activity in the realization is the usability
check during development in the form of a formative
evaluation. This can be defined at certain intervals
in the iteration planning and carried out after the im-
plementation of the iteration backlog. Typically, ag-
ile development processes end with a retrospective, to
reflect one’s own work and processes. Through open
communication, processes can be reflected and opti-
mized, and problems can be directly addressed and
solved.
After the retrospective, it is possible to start the
planning for the following iteration. The activities of
the realization phase are executed iteratively as often
as open user needs are listed in the backlog.
Once all user needs of the backlog have been
implemented, the final verification of the developed
medical device can start. According to EU regula-
tions, the verification has to be performed in a sepa-
rate development phase. Test cases for the verification
have already been collected in the iteration backlogs
and can now be executed and documented.
As the final phase, the medical device can now be
transferred to production in the design transfer. In the
validation phase, the fully developed medical device
is tested by comparing it to the intended use and the
user needs. Similarly, the usability of the medical de-
vice is also finally tested by means of the summative
evaluation in accordance with IEC 62366.
2.2 Roles
In order to fulfill the regulatory requirement for de-
fined responsibilities according to EN ISO 13485,
the project team consists of three persons (groups)
who perform defined tasks: project manager, devel-
oper, and customer (representatives). The roles de-
scribed below act on an equal level to enable the most
open and direct communication and action possible.
Traditional project management tasks, such as defin-
ing the framework parameters in the project design,
are performed by the project manager. Maintain-
ing the backlog of requirements is also one of the
project manager’s tasks. As a link between stakehold-
ers and developers, the project manager ensures that
the project runs efficiently and effectively.
The project is implemented by the developers.
They act and work according to the agile values in
an interdisciplinary and (mostly) self-organized way.
The active involvement of the developers through-
out the process promotes identification and motiva-
tion with regard to the development of the project ob-
ject. Various activities, such as the daily stand-up
or the retrospective, also require direct communica-
tion channels. This open communication and regu-
lar reflection on one’s own actions and processes ulti-
mately benefits not only the project itself, but the en-
tire company. The last relevant group of people is the
group of project-relevant stakeholders. In addition to
the customer, these include, for example, users, sup-
pliers, production or sales. All relevant stakehold-
ers are involved throughout the entire development
process. Beginning with the collaborative definition
of user needs through to validation and design trans-
fer, the stakeholders are actively involved and can in-
troduce change requests according to a well defined
change process at any time in order to create a med-
ical device of highest possible quality and customer-
specification.
One of the most demanding tasks in working with
a reference model is to adapt it for a specific environ-
ment, especially for such a heterogeneous field like
ADmed: An Adaptive Technical Process for the Agile Development of Medical Devices
179
Figure 3: Screenshot of the ADAMO Modeler: The different colors of the model represent different adaptation options, which
are later selected during process execution, dependent on the values of the specified parameters for adaptation.
medical devices are. To make it easier for users to
apply the reference model for their specific case, it
is beneficial to use an adaptive modeling technique.
In this way the model also transfers knowledge about
best practices on how to adapt it. First the ADAMO
Modeler, a tool that enables adaptive modeling, will
be discussed, followed by an example on how it could
be used in the ADmed model.
3 ADAMO MODELER
Information modeling is a standard instrument of
business informatics that is frequently used to model
processes and company data (Seel, 2010). Currently,
there are hardly any tools that provide sufficient func-
tionality to both create and evaluate adaptive refer-
ence models (Seel et al., 2016). It is therefore nec-
essary to extend an existing and established mod-
eling language by the necessary elements (Hilpolt-
steiner et al., 2019). For this reason, the ADAMO
Modeler is being developed at the Institute for Data
and Process Science at Landshut University of Ap-
plied Sciences (Institute for Data and Process Sci-
ence Landshut, 2020). This not only enables pro-
cesses to be modeled in conformity with BPMN 2.0,
but also extends its meta-model to include parameters
and variables. While parameters are available glob-
ally throughout the process and can thus be evaluated
in all terms, a term is always linked to a BPMN ele-
ment in the process (Hilpoltsteiner et al., 2018).
Once the process model has been created, the
ADAMO Modeler also offers the possibility of eval-
uation. First, the user is asked to enter values for all
parameters required in the model. These values, then,
automatically replace all parameters in the terms of
the process model. Subsequently, all terms are eval-
uated using the values. On the basis of these terms,
the model can later decide to remove unneeded parts
of the model (according to the logic), thus suggesting
the user an implementation of the process based on
his parameters. An example what the tool looks like
can be seen is Figure 3.
3.1 Boolean Decision Making
A previous release of the ADAMO modeler was
solely based on Boolean logic as described by Becker
(Becker, 2002) and Delfmann (Delfmann, 2006).
Each element, like tasks, flows, events, or others
could be assigned with a Boolean term. The most
central piece, as with any term, were the variables
that can be used. As the possible use cases should
be as broad as possible, we opted for an open ap-
proach with the following data types: numeric values,
texts, and truth values. As the software is based on
an open source project in JavaScript, this corresponds
to the data types of Number, String, and Boolean. A
term, however, can not only contain string variables,
the whole term itself is also saved as a string (e.g.,
KMIS 2022 - 14th International Conference on Knowledge Management and Information Systems
180
Figure 4: Example of the fractional 0-1 Decision Making approach.
variable names are strings). Hence, it is important
to make variables clearly distinguishable. For this
reason a delimiter is defined that allows to mark a
variable as such. This delimiter may not use oper-
ators or limit the possible content of text variables.
While staying within these technical limitations, the
goal was also to keep the term as readable as possi-
ble for humans. In order to fulfill this criteria, square
brackets are used as delimiter. As the evaluation logic
of JavaScript does not consider them as commands
this works out well. A possible term looks like the
example below.
[participants] <= 12
In this case [participants] represents the variable
name, which can be substituted with the variable
value by implementing a simple search and replace
algorithm before the evaluation takes place. In or-
der to separate between text strings and decimal vari-
ables, it is important to introduce various identifiers.
Within an ADAMO term, character strings must be
introduced and terminated by quotation marks to en-
able a safe evaluation.
3.2 Fractional 0-1 Decision Making
While the approach with Boolean variables is enough
to satisfy basic use cases there are also some types
of decisions that are less clear-cut and require a more
gradual decision-making process. The Boolean ap-
proach will by definition always result in a clear true
or false for an element to stay in the reference model
or not. While this may work for some decisions, other
problems require a more gradual approach in the out-
put (Schmidtner et al., 2021). For example, in project
management, Scrum is generally attributed to be us-
able by small teams. If small teams are defined as
less than 12 people, in the Boolean approach Scrum
immediately becomes useless for teams with 13 or
more people, as it is a clear decision between ”more
than 12” or not. However, this does not reflect re-
ality, because Scrum is not unusable with 13 people
albeit it becomes less attractive the more people join
the team or it requires additional structures for large
scaled Scrum.
Also users of reference models often would like to
prioritize certain aspects in regard to their individual
project. For example, a project may value the person-
nel or culture parameter as far more important than
the actual team size. Both of the aforementioned rea-
sons make simple Boolean terms unsuitable for an ap-
proach where the user wants a less clear cut for vari-
ables or wants to weigh the parameters differently.
Today, thus, ADAMO offers another solution
which is based on the Boolean logic but enables an
even more user specific evaluation. To reflect this, the
fractional 0-1 approach allows for the full spectrum in
between. A solution becomes gradually less attractive
the more the values differ from their optimum. So we
do not have one point that suddenly reverses the de-
cision whether the element is deleted from the model
but instead we have a numerical evaluation how good
the element fits the variables based on the attached
term. To explain this new interpretation of parame-
ters, at first the variable [participants] is reconsidered,
which settles in the range between a minimum of 2
and a maximum of 20. Now we need to define the
optimum for a specific approach. In case of the afore-
mentioned Scrum example, a participant number of
12 or below would be best suited for this approach.
In this case, the following term can be defined to the
relevant elements in the model.
[participants 12-]
This will lead to a score of 1 if the parameter for
[participants] is equal to 12 or below and gradually
decreases towards 0 the further the parameter exceeds
12. At 20 participants, the score is 0. As we now have
a numerical value on how good an element fits into the
model, we must make a decision on which elements to
keep and which to remove. Therefore each path along
the process model is analyzed and the values of all el-
ements along the path are multiplied. The path that
has the closest value to 1 after all calculations is the
ADmed: An Adaptive Technical Process for the Agile Development of Medical Devices
181
Figure 5: Adaptive ADmed Example.
one most suitable for the given parameters. This is,
therefore, recommended to the user as the final pro-
cess.
An example for this can be seen in Figure 4. In
that simple example we have three possible paths
throughout the model. The model is defined with 4
parameters ([software] and [hardware] ranging from
0 to 1, [teamsize] ranging from 2 to 20 and [critical]
with a range from 1 to 100). The user selects the pa-
rameters as follows: [critical] = 0.86, [software]=1,
[hardware]=1 and [teamsize]=12. The first path us-
ing the Vee-Model is calculated with a score of 0.86,
the second path is to have no software development
calculated with a 0 and the third is with Scrum and
a calculated score of 0.14. According to the values
given, the Vee-Model path is suggested to the user.
4 ADAPTIVE ADmed EXAMPLE
If we now apply the possibilities ADAMO offers to
the ADmed model it enables recommendations to be
given to users on how to best tailor the model to their
specific needs. An example is the tailoring of the du-
ration of iterations, depending on how much software
in relation to hardware is part of the specific project.
The idea here is that the more software-oriented the
project is, the shorter should be the iteration to collect
feedback. If the project leans more toward hardware
orientation, the software iterations can take more time
to match up with the hardware development speed,
which typically is slower than that of software. To
model this, we define a parameter [software] that can
range between the values of 0 and 100 (see Figure 5
for an illustration). The user can then choose to in-
put a value depending on how much software devel-
opment is needed in the project under consideration.
100 denotes a pure software development project. 0
denotes a pure hardware development project. If the
user chooses a value in between, the reference model
calculates the approach of closest distance to its opti-
mum and suggests this as the final process.
Initially five alternative paths are present in the
ADmed process model. In case the user specifies
a pure software development project (value equal to
100), the leftmost part with six one week intervals re-
ceives a score of 1, while the path to its right ([Soft-
ware 99]) has a score slightly below 1, because the
given value 100 is (only) very close to 99, where that
path would be optimal. All other paths are even fur-
ther from 1, ending at the rightmost path with a score
of 0. Hence, the leftmost path is recommend to the
user as the most suitable process for the project. If,
on the other hand, the user specifies a value of 0 to
the variable [software], it is a pure hardware project
and with the reverse argumentation the rightmost path
without any software development is recommended.
If the given value is in between, meaning in the range
of 1 to 99, the process model recommends the path
[Software 99] (”Hardware Project with Software”), if
the given value is closer to 99 than to 50 or 1, [Soft-
ware 1] (”Software Project with Hardware”), if the
given value is closer to 1 than to 50 or 99, or [Soft-
ware 50] (”Hard- and Software Project”), if the given
value is closest to 50. Please note, that this simple ex-
ample focuses only on one part of the ADmed Process
KMIS 2022 - 14th International Conference on Knowledge Management and Information Systems
182
Model with only one variable. The logic can be for-
mulated much more in depth with additional variables
and if the best path along the whole model is calcu-
lated it may come to a different result, depending on
the other parameters involved.
5 CONCLUSION AND OUTLOOK
If the ADmed Process Model is reconstructed as an
adaptive model, it will provide benefits for users tai-
loring it to their needs. This is especially true for
the targeted audience group of medical device project
managers. The adaptive model supports the individ-
ual configuration of the development process while
ensuring compatibility to the regulatory requirements
which are unavoidable for the development of medi-
cal devices. The combination of plan-based and agile
process models in ADmed facilitates a great degree
of flexibility. ADmed is also suited to guide inexperi-
enced managers through the development process of
medical devices. However, there remain open tasks to
be solved in the future: In order to further implement
the adaptive model, the relation between influencing
parameters and required processes must be examined
and modeled in more detail. This also includes the
acquisition of knowledge about suitable ranges of the
values of the parameters. For this, expert interviews
and observations are planned in order to complete the
model. Additional case studies will, then, help to
evaluate and further refine the model.
ACKNOWLEDGEMENTS
The project is financed with funding provided by the
Federal Ministry of Education and Research and the
European Social Fund under the ”Future of work”
programme.
REFERENCES
Austromed (2018). Lebensqualitt braucht medizin-
produkte; quality of life needs medical de-
vices (in german). Das Medizinprodukt, avail-
able online at https://www.medmedia.at/das-
medizinprodukt/lebensqualitaet-braucht-
medizinprodukte/, last checked 29.03.2022, 2/18:14.
Becker, J. (2002). Wissensmanagement mit Referenzmod-
ellen: Konzepte fr die Anwendungssystem- und Or-
ganisationsgestaltung; Knowledge management with
reference models: Concepts for application system
and organization design (in german). Physica-Verl.,
Heidelberg.
BVMed e.V. (2021). Der Markt fr Medizin-
technologien; The market for medical tech-
nologies (in german), available online at
https://www.bvmed.de/download/charts-medtech-
markt.pdf, last checked 29.03.2022.
Delfmann, P. (2006). Adaptive Referenzmodellierung:
Methodische Konzepte zur Konstruktion und Anwen-
dung wiederverwendungsorientierter Informations-
modelle; Adaptive Reference Modeling: Methodolog-
ical Concepts for the Construction and Application of
Reuse-Oriented Information Models (in german), vol-
ume 25 of Advances in information systems and man-
agement science. Logos-Verl., Berlin.
Dispan, J. (2020). Der Markt fr Medizintechnolo-
gien: Arbeits-, Markt- und Innovationstrends;Industry
analysis medical technology: employment, market
and innovation trends (in german), available online
at https://www.bvmed.de/download/branchenanalyse-
medizintechnik-boeckler-stiftung-mai-2020.pdf, last
checked 17.01.2022.
Donaldson, L., Ricciardi, W., Sheridan, S., and Tartaglia,
R., editors (2021). Textbook of Patient Safety and
Clinical Risk Management. Springer Nature, DOI:
10.1007/978-3-030-59403-9.
Hilpoltsteiner, D., Schmidtner, M., and Seel, C. (2019).
Prototypische konzeption und implementierung eines
softwarewerkzeugs zur konstruktion adaptiver bpmn-
prozessmodelle; prototypical design and implementa-
tion of a software tool for the construction of adaptive
bpmn process models (in german). Landshuter Ar-
beitsberichte zur Wirtschaftsinformatik, pages 1–49.
Hilpoltsteiner, D., Seel, C., and D
¨
orndorfer, J. (2018).
Konzeption und implementierung eines soft-
warewerkzeuges zum management von bpmn-
prozessvarianten; conception and implementation of
a software tool for the management of bpmn process
variants (in german). In Hofmann, R. and Alm, W.,
editors, Wissenstransfer in der Wirtschaftsinformatik,
pages 15–24. Hochschule Aschaffenburg, Information
Management Institut, Aschaffenburg.
Institute for Data and Process Science Landshut
(2020). Github repository, available online at
https://github.com/hawmobilesystems/adamo, last
checked 28.06.2022.
Jonnalagadda, K., Fleisch, D., Hultman, P., and Berez,
S. (2019). How Agile Is Powering Health-
care Innovation: Life sciences and healthcare
companies need new solutions to meet rapidly
changing customer needs, available online at
https://www.bain.com/insights/how-agile-is-
powering-healthcare-innovation/, last checked
29.01.2022.
Schidek, A. and Timinger, H. (2022). Agilization of tech-
nical development processes for medical devices. In
IEEE ICE - IAMOT Conference 2022 (accepted and
currently waiting for publication). IEEE.
Schmidtner, M., Doering, C., Hilpoltsteiner, D., Martens,
M., and Timinger, H. (2021). A fractional 0–1 de-
cision making approach for process variant manage-
ment. In Co-creating our future: scaling-up innova-
tion capacities through the design and engineering of
ADmed: An Adaptive Technical Process for the Agile Development of Medical Devices
183
immersive, collaborative, empathic and cognitive sys-
tems, pages 1–8, Piscataway, NJ. IEEE.
Seel, C. (2010). Reverse Method Engineering: Meth-
ode und Softwareuntersttzung zur Konstruktion und
Adaption semiformaler Informationsmodellierung-
stechniken; Reverse Method Engineering: Method
and software support for the construction and adap-
tation of semiformal information modeling techniques
(in german), volume 20 of Wirtschaftsinformatik -
Theorie und Anwendung. Logos, Berlin.
Seel, C., D
¨
orndorfer, J., Schmidtner, M., and Schubel,
A. (2016). Vergleichende analyse von open-source-
modellierungswerkzeugen als basis fr forschungspro-
totypen; comparative analysis of open source model-
ing tools as basis for research prototypes (in german).
In Prozesse, Technologie, Anwendungen, Systeme und
Management 2016, pages 35–44. Barton, Thomas and
Herrmann, Frank and Meister, Vera and M
¨
uller, Chris-
tian and Seel, Christian.
KMIS 2022 - 14th International Conference on Knowledge Management and Information Systems
184