Techniques for Usability Risk Assessment during Medical Device

Design

Alice Ravizza

, Andres Diaz Lantada

, Luis Ignacio Ballesteros Sánchez

, Federico Sternini

and

Cristina Bignardi

DIMEAS, Politecnico di Torino, Torino, Italy

ETSI Industriales, Universidad Politecnica de Madrid, Madrid, Spain

Keywords: Usability, Human Factors, Risk Management, Medical Device Design.

Abstract: Human errors during the use of medical devices, due to pitfalls in the design of the user interface, may lead

to substantial risk to users and to patients. There are multiple techniques for the identification and for the

assessment of user related risks, that may be chosen according to the step of the design (preliminary

feasibility studies, minimum viable product assessment, verification and validation) and considering

cognitive processes and information processing mechanisms of users, which may lead to errors. Some

techniques are more adequate for a quick-and-dirty approach, during early stages of design: these include

expert reviews, discussions among focus groups, standard reviews and heuristic analyses. Other techniques

are adequate for a more detailed and systematic analysis of risk, in more advanced design stages, with a

failure mode and effect analysis (FMEA) approach, including time-and-motion studies and task analyses.

Lastly, user tests with the help of rapid prototypes, perhaps involving alternative embodiments to be studied,

are very adequate for verification and validation of the interface. Usability analysis techniques should be

part of the toolbox of a biomedical engineer and they should be carefully chosen. Each technique, regardless

the step it is used, should allow the designers to define a precise level of risk in terms of probability,

severity. Moreover, usability risk minimisation measures shall be measurable and able to be quantified, as

well as the impact of risk mitigation strategies. For this reason, usability risk minimisation measures should

be classified according to regulatory requirements as “safe by design”; “alarms and protections” and

“information for safe use”. Each class of risk minimisation measure should be then given a measurable risk

reduction score, so that the risk assessment can be completed in a repeatable and regulatory compliant way.

1 INTRODUCTION

Biomedical engineers routinely include “users’

needs” in the design requirements of medical

devices. But what is a “user need”? Not only the

patient clinical condition, but also the need of a

device that is adequate to his skills, education and

capabilities and can be used safely.

Usability is defined, by the standard IEC 62366,

as “the characteristics or features of the user

interface that facilitate use and thereby effectiveness,

efficiency and user satisfaction in the intended

environment of use” (International Electrotechnical

Commission [IEC], 2015).

It is an essential concept in the design process of

any medical device, for the benefit of healthcare

professionals, patients and all stakeholders. We

believe that usability should be part of the modern

academic education of biomedical engineers,

worldwide. Our group has tested this approach to the

design of innovative open source devices as part of

the UBORA project, including a drop foot frame, a

face splint, a hand rehabilitation tool and more.

Design decisions should be driven not only by

performance, cost or environmental impacts of the

device, but also by its ergonomics and aesthetics,

connected to usability, safety and user experience.

2 POSITION PROPOSAL

In this paper, we present a structured method for the

identification and assessment of use-related risks of

medical devices. These risks need to be considered

during the whole design process, from specification

and conceptualization, towards detailed design,

prototyping, preparation of production and whole

product life cycle.

Ravizza, A., Lantada, A., Sánchez, L., Sternini, F. and Bignardi, C.

Techniques for Usability Risk Assessment during Medical Device Design.

DOI: 10.5220/0007483102070214

In Proceedings of the 12th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2019), pages 207-214

ISBN: 978-989-758-353-7

207

We present different techniques as part of this

method, to be chosen according to the kind of device

under assessment, to the level of development of the

device - in terms of ideation, testing and verification

- and to the available resources for the analysis. We

also present how to identify minimisation measures

and how to evaluate their effectiveness.

The method is fully compliant to internationally

recognised standards ISO 14971 (International

Organization for Standardization, 2007) and IEC

62366. The application of this method would be

appropriate during early design stages, during

regulatory approvals and during health technology

assessment, posing important benefits that

overweight the current barriers (Shah, 2007). This

method can be applied by designers, by legal

manufacturers and by authorities involved in health

technology assessment studies.

2.1 Why Is Usability Important for a

BME?

In the upcoming Medical Device Regulation EU

2017/745, the usability of medical devices acquires

extreme importance in the process of certification.

Usability tests are part of an engineering process

that sees the collaboration of a team of experts in the

specific medical sector (physicians, nurses, medical

personnel), clinical and biomedical engineers and

product designers, and include analysis of past

adverse events, related to the use of the device,

design thinking of devices in accordance with the

repetitive and repeatable mental patterns of the

human user, considerations on experience and

technical knowledge of different types of user

(laymen or professional) and the application of

ergonomic principles on the design of the devices.

The classical techniques of human factors

engineering allow to systematize the approach to

medical device design with a view on usability,

because they allow to describe the different types of

users and to build around them a personalized

interface. In fact, the entire process of usability

assessment allows putting the patient and his/her

needs at the center of the medical device design.

3 ERROR DEFINITION AND

IDENTIFICATION

3.1 What is a User Error

User error is any error made by the user in

interfacing with a device, i.e. any situation caused by

the user that leads to device uses unintended by the

manufacturer. It includes two distinct types of error:

use error and abnormal use. Abnormal use is a

“conscious, intentional act or intentional omission of

an act that is counter to or violates normal use and is

also beyond any further reasonable means of user

interface-related risk control by the manufacturer”

(IEC 62366, 2015).

Use error is “user action or lack of user action

while using the medical device that leads to a

different result than that intended by the

manufacturer or expected by the user” (IEC, 2015).

3.2 The Two Steps of the Usability

Assessment

Assessments regarding usability start early during

the design and are iteratively performed to increase

knowledge about user needs and expectations,

interface solutions that better match those needs,

risks and their mitigation measures.

The standard defines two main steps of usability

assessment: a formative (typically iterative) phase

that is integrated in the development of further

iterations and then a summative phase that is

intended to validate and provide objective evidence

regarding the latest (approved) iteration of the

interface design.

3.2.1 Formative

Formative evaluation is a “user interface evaluation

conducted with the intent to explore user interface

design strengths, weaknesses, and unanticipated use

errors” (IEC 62366, 2015).

It is generally iterative and should be performed

until the manufacturer has reached a finalised

version. Formative evaluation improves user

interface, solving issues in preliminary analysis.

During formative iterations, it may be useful to

identify early phase versus late phase studies.

Early phase studies are characterised by a higher

uncertainty in the possible device variants, with

many specifications not yet completely defined. At

this stage, many prototypes are still available and

they can be radically different one to another, so the

employment of rapid and low-cost prototyping

techniques (i.e. 3D printing, cardboard modelling)

proves quite beneficial for first conceptual

assessments.

Late phase studies are characterised by a better

defined list of requirements and of specifications,

which leads to a shorter list of device variants, with

potentially small but very significant differences.

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

208

Figure 1: Usability engineering workflow.

3.2.2 Summative

The summative evaluation is conducted at the end of

the development, on the finalized user interface,

with the intent to obtain objective evidence that the

user interface can be used safely. Summative

verifies an acceptable risk-benefit profile under a

usability point of view and also determines and

confirms the expected effectiveness and clinical

benefit.

3.3 Techniques for Usability

Assessment in IEC 62366

System Description. It is an analysis of the main

users and scenarios where the device will be used.

Main functions and sub functions of the device

should be well defined and understood.

Task Analysis. Task analysis broadens the system

description, identifying all the relevant human

interventions in the use of the device and where

errors can occur. It has the objective to understand

and represent in an organized manner the set of tasks

that the human element carries out in the use of the

device. Task analysis may include analysis of

cognitive processes and performance shaping factors

(individual, social and ergonomic) influencing on

the device use. Some methods (Kirwan &

Ainsworth, 1992) for this purpose are divided in

“Task data collection” techniques and “Task

description” techniques.

Task data collection techniques are techniques

which are primarily used for collecting data on

human-system interactions, and which then feed into

other techniques. Some of these techniques are:

Walkthrough, Talkthrough, Critical Incident

Technique, Observation, Questionnaires, Structured

interviews.

Task description techniques represent and

structure the information collected into a systematic

format, serving as a reference material. Some of

these techniques are: Hierarchical Task Analysis,

Tabular Task Analysis, Timeline Analysis,

Decision-Action Diagram. Task description of

various medical devices is present in literature, with

a different level of detail, for example for volumetric

infusion pumps (Chung, 2003)

Human Error Analysis. Including the identification

of possible human errors based on previous Task

analysis. Human errors modes can be analyzed at

two layers: External Errors (Actions) or Internal

Errors (Cognitive). Some techniques to allow human

error identification are Human Hazard and

Operability Study [HAZOP]. Incident analysis, or

use of a Taxonomies and Checklists of possible

generic human errors that might occur during the use

of any device. Human error analysis includes

assessment of probability and severity of each error.

Human Error Reduction and Mitigation. Based

on previous stages a set of recommendations and

requirements for device design are proposed to

reduce and mitigate human errors. Combining the

previous aspects, we can set priorities and propose

strategies for human error reduction.

3.3.1 Which Technique in Each Phase?

The international standard IEC 62366 (IEC, 2016)

presents a series of techniques, that we assessed in:

- early formative

- late formative

- summative

In early formative, a quick-and-dirty approach

identifies the best interface. Then, in late feasibility,

a more structured approach may help to refine the

interface. Lastly, during summative, a frozen version

of the interface is validated to confirm its risk-

benefit profile. Criteria to choose the most

appropriate technique(s) for each phase are:

- Need to involve experts in the technology.

- Need to involve real user(s).

- Time required to assess.

- Time required to report.

- Qualitative results (opinions) vs. quantitative

results (usability scores).

- Depth of analysis.

We have identified some techniques that we

consider particularly appropriate for each step. A

detailed list is shown in Table 1.

The use of rapid prototyping and rapid tooling

techniques, proves interesting for the straightforward

creation of physical models, which can be used to

support most of the aforementioned techniques for

usability assessment. These physical prototypes or

models can support decision making processes for

selecting among different product ideas, on the basis

Techniques for Usability Risk Assessment during Medical Device Design

209

of ergonomics, aesthetics, basic performance,

overall usability and safety, to reach the device

concept in the first stages of the development

process. They can also support in the creation of a

first minimally viable product for interacting with

healthcare professionals, patients, layperson.

Prototypes according to different design iterations,

consequence of the different decisions taken to

mitigate risks and to improve usability, can,

consequently, support the whole methodology and

approach we propose here.

This evaluation has shown that a risk-based

approach is easily adapted to a resource wise

approach. During early formative, low resource

review techniques such as expert reviews, standard

analysis, cognitive walkthrough are easily performed

on documentation and by design experts. They do

not require the participation of a large number of

real users nor the availability of a finalised

prototype, while low-cost replication tools may

provide effective samples to boost discussion.

Later stages of formative assessments may

benefit of more structured techniques, such as a

detailed task analysis that is linked to the FMEA

technique. User tests with 5-10 users may be

planned at later formative steps in order to allow

refinement.

3.3.2 Which Technique for Which Device?

Medical devices belong to varied categories in terms

of technology, intended use, intended users

(layperson or professional), invasiveness in the

human body or expected useful life, which affect

design decisions in connection with usability and

safety. For this reason, we have also assessed each

technique presented by the norm IEC 62366 (IEC,

2016) in terms of adequateness to different kinds of

devices. A detailed evaluation is shown in Table 2 .

In Table 2, the same technique is considered as

adequate or inadequate for devices that may be

apparently very similar from the usability point of

view. However, this is explained by the

technological differences in the device. As an

example, the technique “standard review” proves

“adequate” for very different devices such as heart

valves and nasogastric tubes, but is considered

“adequate with reserve” for Software as a Medical

Device (SaMD). This is due to the poor

standardisation that is still present in the SaMD

sector, while traditional devices can be assessed by

very consistent and complete international standards

and guidelines. Also consider, the technique

“participatory design” that is considered “not

appropriate” for traditional electromedical devices

for the layperson, such as pulse oximeters, but on the

other hand is “adequate” for SaMD and apps for the

layperson. This again is justified as participatory

design may allow the designers to align the medical

app to users’ expectations, by allowing users to

design an intuitive and user friendly app, with a user

interface as similar as possible to a consumer app.

3.4 Linking Usability to Risk

Identification

Each usability evaluation technique allows the

designers to identify risks and potentially hazardous

situations. We describe here some of the techniques

identified above, in terms of capability of the

assessment to be easily linked to a formal risk

analysis as per ISO 14971 (ISO, 2007).

The preferred methods for early feasibility help

the designers to identify risks in general terms and

are potentially adequate to determine risk severity

(worst case consequences of the risk scenario). For

example, at very early stages of ideation of a

electromedical device to be used in emergencies

(e.g. a defibrillator) designers may already be aware

of the importance of high visibility and audibility of

the device, since it is expected to be used in loud,

dark, confusing environments. During late

feasibility, we propose a more structured method, by

application of the Failure Modes and Effects

Analysis FMEA technique. The Application FMEA

technique yields the best results if the question

“what happens if…” is posed at each application

step or phase. So, we propose an integrated

technique: firstly describe the use of the device in

very fine detail by task analysis and then perform

Application FMEA on each step.

We propose a very detailed task analysis and,

where applicable, also a function analysis or use

flowchart. Description of the intended use interface

by a flowchart is particularly adequate for medical

device software, both stand alone and integrated in

an electromedical device. Use of this integrated

method allows a very precise assessment of risk

severity, thanks also to the possibility of obtaining a

description of the chain of events that arise from an

hazardous situation, for example thanks to

brainstorming or focused expert reviews.

If the designers do not have enough past data or

experience-based estimations to determine risk-

related probability, a user test can be very useful to

estimate probability of each hazard. If the user test

are planned in this phase, the task list and use

flowchart already available to designers from the

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

210

Application FMEA activity will be used to plan and

record the user tests. For each use error or use

uncertainty observed during user tests, designers can

determine severity and estimate probability.

In late feasibility, user tests can also be

integrated to other techniques to reach a new and

refined iteration of the device interface. We

encourage designers to plan, after user testing

session, additional sessions with the users to gather

information through interviews, SUS questionnaires

(Brooke, 1996) and open-ended questions intended

to encourage participatory design. These interactive

sessions with end users are also very useful to gather

information about expected probability of each

encountered error or uncertainty. For example, late

feasibility studies of a surgically invasive device for

professional use, e.g. catheter for angioplasty, may

include the definition of a task list based on standard

reviews, guidelines, state of the art and interviews.

For each task, designers may identify potential

hazardous situations and their consequences. Then,

user tests on a simulator or dummy may confirm or

improve the estimated risk list; the same users may

be involved after the test to discuss their errors,

determine root causes and suggest improvements in

the catheter shape, pliability or accessory list.

During user tests in the formative phase,

assessments and integrations to reports for

Perception-Cognition-Action technique PCA are

also very common; users can also be invited to

express their thoughts and impressions while they

perform the tasks, as part of participatory design.

A detailed description of the use of different

techniques is given in Table 3.

4 RISK MITIGATION

TECHNIQUES

4.1 Risk Control Measures in Usability

Regulatory requirements (for example Medical

Device Regulation EU 2017/745, Annex I) on risk

minimisation are clearly indicating a preferred order

in the identification and selection of risk

minimisation measures.

Safe-by- design solutions are preferred and, if

not available or not sufficient, other measures shall

be added in terms of protections and alarms.

Moreover, information for safe use shall be

provided. Designers shall plan in eliminating the

most severe risks by safe-by-design solutions from

very early stages of design. To continue with the

defibrillator example given for early feasibility,

designers may decide to place all the interface

commands on the same (front) side of the device and

review standards for colours and icons at a

preliminary stage of the ideation. Alarms and

protections can be included during all iterations of

the formative stage even adjunct to safe-by- design

measures. For example while designing a software

interface of an electromedical device, the designers

may allow only an “admin” user (e.g. a qualified

medical professional) to set performance parameters

in a predefined interval, as based on state of the art

clinical guidelines. Then designers may place

adequate screens for password input as protection

measures for the “admin” access. Moreover, for all

interface screens designers may provide information

for safe use with reference to the allowed interval for

clinical parameters, tips to proceed to the next

clinically relevant step of the therapy and so forth.

4.2 Summative as Part of Device

Validation

The goal of device validation is to determine if the

device is adequate for its intended purpose and to

confirm its estimated risk benefit profile. No major

modifications are expected at this phase. While not

all parts of the interface may be subject to

summative, designers should plan to validate all the

critical ones. For example, summative assessment of

the interface for the assistance and maintenance

personnel of an electromedical device, when

personnel is directly trained by the device legal

manufacturer, may not be needed.

We propose to plan the summative evaluation by

mirroring activities of the late formative step, on the

final and frozen iteration of the interface. A

complete task analysis should be available and

checked for coherence to the user manual or

instruction leaflet. Moreover, if applicable to the

kind of device, also a complete use flowchart should

be available.

Summative evaluation should be performed with

real users and in a very well simulated or real use

environment, depending on device kind and ethics

considerations. During user tests, additional

techniques may be integrated to determine the length

of time needed for each task (by time-and-motion

studies) and the workload of the user.

It should be noted that, while very adequate for

summative activities, time of use and workload

assessment are not easily evaluated during formative

tests. The interface is still under modification and,

more often than not, the tasks may be interrupted for

Techniques for Usability Risk Assessment during Medical Device Design

211

clarifications and comments from the users, a very

common event if the participating users are aware

that the device is under development and not under

validation: most users are very keen to provide their

feedback and opinions as part of participatory design

activities. Interrupted and commented tasks disrupt

the workload assessment and the time estimation.

The outcome of the summative step is the

confirmation of all parts of the device interface,

including the information for safe use. No additional

risks should be encountered and all the foreseen

risks should be confirmed in terms of severity and

probability. Risk control measures should be

formally reviewed for final implementation and

effectiveness and the risk-benefit profile confirmed.

5 CONCLUSIONS

An integrated approach to usability and risk

management, while complex in general terms, can

be easily adapted to the design step, kind of device

under assessment and available resources. Designers

should be provided with a complete usability

toolbox and be able to choose a adequate tools for

each of their designs.

Integration of usability assessments in the wider

risk management leads to safer and more intuitive

medical devices, for the benefit of patient and

professional users alike.

While our group has tested this method in

multiple instances, we wish that it would be used

widely. With more experience, this method can be

refined, adapted to different cultural settings and

various technical skills, and updated with device-

specific tools. Moreover, this technique may be

integrated with the risk mitigation measures required

for the adequate management and protection of

patient data.

ACKNOWLEDGEMENTS

Authors acknowledge the UBORA “Euro-African

Open Biomedical Engineering e-Platform for

Innovation through Education” project, funded by

the European Union’s “Horizon 2020” research and

innovation programme under grant agreement No

731053.

REFERENCES

Brooke, J. (1996, September). SUS - A quick and dirty

usability scale. Usability evaluation in industry, 189, 4-7.

Council of the European Communities. (1993, July 12).

COUNCIL DIRECTIVE 93/42/EEC of 14 June 1993

concerning medical devices - ANNEX I. Official

Journal of the European Communities, 36, pp. 13-18.

European Parliament and Council of the European Union.

(2017, May 5). Regulation 2017/745 on medical

devices, amending Directive 2001/83/EC, Regulation

(EC) No 178/2002 and Regulation (EC) No 1223/2009

and repealing Council Directives 90/385/EEC and

93/42/EEC. Official Journal of the European Union,

pp. 1-175.

International Electrotechnical Commission. (2015). IEC

62366-1:2015 Medical devices -- Part 1: Application

of usability engineering to medical devices (1st ed.).

International Electrotechnical Commission. (2016). IEC

TR 62366-2:2016 Medical devices – Part 2: Guidance

on the application of usability engineering to medical

devices (1st ed.).

International Organization for Standardization. (2007).

ISO 14971:2007 Medical devices -- Application of risk

management to medical devices (2nd ed.).

International Organization for Standardization. (2016).

ISO 13485:2016 Medical devices -- Quality

management systems -- Requirements for regulatory

purposes (3rd ed.).

Kirwan, B., & Ainsworth, L. (1992). A Guide To Task

Analysis: The Task Analysis Working Group. Taylor &

Francis Ltd.

U.S. Food & Drug Administration. (2016, February 3).

Applying Human Factors and Usability Engineering to

Medical Devices. Guidance for Industry and Food and

Drug Administration Staff. Retrieved from (last access

Nov. 2018)

https://www.fda.gov/downloads/medicaldevices/.../uc

m259760.pdf

Shah, S., & Robinson, I. (2007).Benefits of and barriers to

involving users in medical device technology

development and evaluation.International Journal of

Technology Assessment in Health

Chung, P. H., Zhang, J., Johnson, T. R., & Patel, V. L.

(2003). An extended hierarchical task analysis for

error prediction in medical devices. AMIA ... Annual

Symposium proceedings. AMIA Symposium, 2003,

165-9.

APPENDIX

Tables.

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

212

Table 1: Assessment of each evaluation technique according to predefined criteria.

Method as per

table E.1 of IEC

62366-2:2016

Involve

experts

Involve

real

user(s)

Time to

assess

Time to

report

Qualitative

results

(opinions)

Quantitative

results

(scores)

Depth of

analysis

Proposed for step

Advisory panel

reviews

Yes

Medium

Low

Yes

Low

Early formative

Brainstorm use

scenarios

Yes

Low

Yes

Low

Early formative

Cognitive

walkthrough

Yes

Low

Yes

Medium

Early formative

Expert reviews

Yes

Low

Yes

Low

Early formative

and summative

FMEA and FTA

Yes

High

Yes

High

Late formative

Focus groups

Low

Yes

Low

Early formative

Function

analysis

Yes

Medium

Low

Yes

High

Early formative

Heuristic

analysis

Yes

Medium

Yes

High

Late formative

Observation

Yes

Medium

Yes

Medium

Early formative

One-on-one

interviews

Yes

Medium

Yes

Medium

Early formative

and late formative

Participatory

design

Yes

Medium

Yes

Medium

Late formative

PCE analysis

Yes

High

Yes

High

Late formative

Simulation

Yes

High

Yes

High

Late formative and

summative

Standards

reviews

Yes

Low

Yes

Medium

Early formative

Surveys

Yes

Low

Yes

Low

Late formative and

summative

Task analysis

Yes

High

Yes

High

Late formative and

summative

Time-and-

motion studies

Yes

Medium

Yes

Medium

Late formative and

summative

Usability tests

Yes

High

Yes

High

Late formative and

summative

Workload

assessment

Yes

High

Yes

Medium

Late formative and

summative

Table 2: Assessment of each evaluation technique related to device kind.

Advisory panel

reviews

Brainstorm use

scenarios

Cognitive

walkthrough

Expert reviews

FMEA

Focus groups

Function

analysis

Heuristic

analysis

Observation

One-on-one

interviews

Participatory

design

PCA analysis

Simulation

Standards

reviews

Surveys

Task analysis

Time

-and-

motion studies

Usability tests

Workload

assessment

Method as per table

E.1 of IEC 62366-

2:2016

Implantable,

electro-

medical

e.g.:

implantable

defibrillator

Yes

May

Yes

Implantable,

not electro-

Medical

e.g.: heart

valve

Yes

May

Yes

May

Yes

May

Electro-

Medical for

professional

e.g: ecg

Yes

May

Electro-

Medical for

layperson

e.g: home

thermometer

Yes

May

Yes

May

Yes

May

Yes

May

Yes

Samd for

professional

e.g: surgical

planning

Yes

May

Yes

May

Yes

May

Samd for

layperson

e.g: app for

treatment

adherence

Yes

May

Yes

May

Yes

May

Yes

May

Yes

Not active

device-

professional

e.g:

nasogastric

tube

Yes

May

Yes

May

Not active

device-

layperson

e.g.: contact

lenses

Yes

May

Yes

May

Yes

May

Yes

May

Yes

Techniques for Usability Risk Assessment during Medical Device Design

213

Table 3: Risk identification related to each evaluation technique. Methods as per Table E.1 of IEC 62366-2:2016.

Advisory

panel reviews

By brainstorming and review of past experiences, panels may identify potentially hazardous situations,

assess probability and severity, describe the risk minimization measures present in the state of the art

Brainstorm

use scenarios

Designers involved in the brainstorming may identify user errors and misuse/abnormal use; designers

may also identify risk control measures

Cognitive

walkthrough

Designers involved in the brainstorming may identify usability pitfalls in the design and describe the

hazardous situations that may arise; designers may also identify risk control measures

Expert

reviews

Experts may point out usability strengths and pitfalls during their review. Usability pitfalls may then be

linked to the hazardous situations; designers may also identify risk control measures. We recommend

that experts answers to questions as per ISO 14971 ann. C

FMEA and

FTA

FMEA technique is a very thorough method for the identification of risks. We recommend that this

method is used in conjunction with a very detailed task analysis

Focus groups

During a focus group, designers may guide the discussion with users leading to the identification

usability pitfalls in the design and describe the hazardous situations that may arise; designers may also

identify risk control measures and ask participants to the focus group to comment the proposed measures

Function

analysis

A functional flow diagram, commented with the identification of machine functions and user functions

may be used in conjunction with a FMEA technique for a thorough identification of risks

Heuristic

analysis

During heuristic analysis, usability experts may use heuristic principles to identify and give usability

scores to usability pitfalls. They may describe the hazardous situations that may arise; designers may

also identify risk control measures and ask experts participating to the heuristic review to comment the

proposed measures and score their capability to lower the risk

Observation

During observation, designers may identify user uncertainties or errors; root cause should be discussed

with the users to ensure that the hazardous situation is well understood by the designers; we believe that

observation alone cannot provide sufficient information regarding risk and that it should be backed up

with interviews or surveys as a de-brief activity

One-on-one

interviews

Interviews are useful when used in conjunction with techniques involving users that perform actual

tasks on the device, from observation to cognitive walkthrough to usability tests. Interviews are best

used as de-briefing activities as they allow to identify not only the hazardous situations, but also their

root causes

Participatory

design

Partecipatory design very powerful tool when used in conjunction with techniques involving users that

perform actual tasks on the device, from observation to cognitive walkthrough to usability tests.

Focused on defining risk mitigation measures and their perceived effectiveness

PCA analysis

PCA analysis can be integrated in the task analysis and therefore in the FMEA analysis to provide a

complete evaluation of risk; most applicable to complex tasks and/or interfaces

Simulation

We believe that simulation is one of the core techniques, as it can easily be adapted to all devices thanks

to the use of mockups, dummies, animal models and other simulated settings. This allows the planning

of all usability assessment activities in a cost-effective and ethical fashion

Standards

reviews

We believe that standards review should be applied whenever an internationally recognized document is

available, be it an ISO norm, a guideline from a scientific society, a local procedure. Non- fulfillment of

standard requirements is a potential source of significant risk

Surveys

Surveys are useful tools in some situations, where the use of the medical device is difficult to observe;

typically if it is used by the layperson as part of private life (contact lenses, in vitro testing for

pregnancy, and so on). Surveys are not adequate to investigate root causes of hazardous situations

Task analysis

Task analysis is the most powerful tool for linking usability assessment to risk management. It is best

used as an input to the FMEA technique but can also be used during preliminary steps of the device

design to determine the user needs and consequent testable technical requirements. Non fulfillment of

one of those requirements shall be treated as significant risk

Time-and-

motion

studies

We believe that time and motion studies are most adequate to assess risk of those devices in which the

time of execution is a risk control measure, e.g. If a fast execution improves patient safety (for example,

lowering chances of bacterial contamination or improving chances of patient recovery)

Usability

tests

Usability tests are a very powerful tool to determine those risks that not identified by the designers,

using techniques that do not directly involve users (such as brainstorming, standard reviews…).

Usability tests allow to estimate the probability of an hazardous situation; they also allow designers to

consolidate the task list

Workload

assessment

Workload assessment reviews may allow designers to identify some kind of use errors related to

overload or environmental distractions; we believe that this technique is most appropriate when

professional users are involved, as they are more prone to burn out and also more aware of the impact of

overload on their performance at work

BIODEVICES 2019 - 12th International Conference on Biomedical Electronics and Devices

214