Exloration of High Risk Medical Devices Methodologies for

Optimized Evaluations

Thomas Lihoreau

, Chrystelle Vidal

, Tianguang Wang

and Lionel Pazart

CHU Besançon, INSERM CIC 1431, Centre d'Investigation Clinique, Besançon, France

Institut Supérieur d'Ingénieurs de Franche-Comté, University of Franche-Comté, Besançon, France

Keywords: High Risk Medical Devices, Evaluation, Methodology.

Abstract: Medical devices are developed by manufacturers that need to provide proofs of safety, efficacy, efficiency.

In the same time they could be specialists in the technologies, they could not be necessary experts for the

targeted medical field and need to be surrounded to build the correct clinical evaluation strategy. Skills

required are specific to these particular instruments, and need to be optimized and innovative, as there is as

much different devices than the start-ups in the arena. Even if works are performed on the methodological

aspects since years, we propose to state a snap of the situation thanks to clinical trials databases exploration,

with the aim to extract typical cases for future help and support for the actors. The current article offer to

present our strategy of work as well as first quantitative results.

1 INTRODUCTION

In Clinmed special session of Biostec 2020 in Malta,

we discussed the adapted methodologies for medical

devices field (Vidal, 2020), which is characterized

with specificities well documented, on the subjects of

randomization, comparator, blinding, acceptance, or

endpoints selection… French Haute Autorité de santé

(2013: https://www.has-sante.fr/jcms/c_1696842/en/

methodological-choices-for-the-clinical-developmen

t-of-medical-devices), as well as American Food and

Drug Administration (2016, https://www.fda.gov/

media/92671/download) for example underlined

these points since years.

In Europe, the European commission adopted in 2017

an updated regulation on medical devices EU MDR

2017/745, and on in vitro diagnostic medical devices

EU IVDR 2017/746, repealing previous directives

(https://ec.europa.eu/health/md_sector/overview_en).

Guidance documents are developed to help actors for

implementation of these directives, previously

Meddevs, going onto updated Medical Device

Coordination Group: https://ec.europa.eu/health/

md_sector/new_regulations/guidance_en.

https://orcid.org/0000-0001-8417-6609

https://orcid.org/0000-0002-0882-5299

https://orcid.org/0000-0002-3652-5461

https://orcid.org/0000-0002-9104-0862

The general context have been related in previous

Clinmed sessions -and will be also debated in other

articles of this session. In a synthetic approach, we

can observe an updated framework around medical

devices requiring more clinical evidences, through

clinical investigations conceived, realized and

analysed with independent medical and clinicians

experts, high risk medical devices being the main

impacted by these considerations. The way a

technological innovation needs to be evaluated being

different that the historical well-known ones drugs.

We propose then to:

- formalize an analysis of the registered studies

mixing high risk medical devices and interesting

methodologies,

- discuss the quantitative results,

- analyse the studies retained in our approach,

- ultimately, we will try to sort out and propose

some recommendations for the actors.

In the present paper and to match with the

pedagogical objective of the Clinmed session we will

focus on the strategy of research and present the first

quantitative results, the final report being planned for

2021.

Lihoreau, T., Vidal, C., Wang, T. and Pazart, L.

Exloration of High Risk Medical Devices Methodologies for Optimized Evaluations.

DOI: 10.5220/0010386502850289

In Proceedings of the 14th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2021) - Volume 1: BIODEVICES, pages 285-289

ISBN: 978-989-758-490-9

285

2 EXPLORATION STRATEGY

In this aim to explore the methodologies adapted to

medical devices, we follow a the work from Vidal

2020, and Pruniaux 2021.

2.1 High Risk Medical Devices

The definition of high risk medical devices join the

classical criteria defining the classification linked to

the level of risk of each device (from I, IIa, IIb, to III),

but can not be only reduced to.

European commission even recently open a call in the

Horizon 2020 framework, named “Developing

methodological approaches for improved clinical

investigation and evaluation of high-risk medical

devices” (https://ec.europa.eu/info/funding-tenders/

opportunities/portal/screen/opportunities/topic-

details/sc1-hco-18-2020), without using namely the

“class” reference to continental regulations.

Brunotte (2020) add notions of delicate targeted

anatomical area, implantability character, or novelty

in the technology or the material used.

Following a work performed by Pruniaux et al. in

summer 2020 that defined an algorithm (Matlab and

Scilab softwares) allowing to search occurences of

keywords in databases, we use the same key-words

mixing the aspects of implantability, risk in

morbidity/mortality, adverse event and misuse risks.

2.2 Methodology

In the same way and following the work of Vidal

2020, we extend her research focused on adaptive

methodologies, with the concepts of : Zelen

randomization (Zelen et al. 1990), adaptive design

(response adaptive randomization, Jiang, F, et al.,

2013, or adaptive enrichment, Simon et al., 2013, Lai

TL et al., 2019), cross-over, flexible design,

sequential trial (Hamilton et al., 2012), treatment

switching, sequential multiple assignment

randomized trial (SMART) (Tamura et al., 2016, Wei

et al., 2018, Meurer et al., 2017), multi-arm multi-

stage trial (Simon et al., 1985), stepwise multiple

arms, cluster trial, tracker study (for fast technology

evolution, Lilford, et al., 2000), Bayesian approaches

(Pennello et al., 2008, Campbell et al., 2011,

Campbell et al., 2016), sample size reassessment (re-

estimation/adjustment) (Magirr et al., 2016), or trial

without informed consent or within cohort (Kim,

Weijner 2018)…

3 RESULTS

Our first researches focused on https://www.clinical

trials.gov/ database. The explorations allowed to

detected the defined keywords in brief titles, official

titles or brief summaries/detailed descriptions.

Figure 1: Repartition of the studies concerning high risk

medical devices and specific designs (source: map tool of

https://www.clinicaltrials.gov/).

We observed 7155 studies on high risk medical

devices. With methodology key-words, we detected

61156 studies. Crossing high risk + methodology

key-words, we obtained 859 studies.

On this total of 859 trials (on date November 25

2020) matching both, 341 were completed, 14

withdraw, 1 suspended, 54 not yet recruiting, 142

with results, 130 accepting healthy volunteers, 6 with

usability key-word.

A quick overview of the map provided by clinical

trials website presents that these kind of studies are

ainly performed in Europe (345), North America

(309+77), and to a lesser extent in India (75) (figure

1).

We can also observe an evolution in terms of

number of concerned studies (figure 2), with 16

referenced before 2001, and a regular increase (105 in

2020).

Figure 2: Evolution of trials on medical devices involving

specific methodologies, over time.

ClinMed 2021 - Special Session on Dealing with the Change in European Regulations for Medical Devices

286

Table 1 present also the distribution of the different

methodologies in the obtained results.

Table 1: Repartition of methodologies in the identified

studies: at least one key word concerning methodology –

some could have more than one, that is why we get here a

total of 933.

We performed the same extraction on Medline

(https://pubmed.ncbi.nlm.nih.gov/), with article type

filter “clinical trial”:

- we detected 10380 publications on high risk

medical devices,

- with methodology key-words, we obtained 88560,

- crossing high risk + methodology key-words, we

observed 864 articles (still on November 25

2020).

4 NEXT STEPS

4.1 Selection of the Studies of Interest

The selection phase will consist in defining criteria

allowing to retain relevant studies.

We will rank the trials by type of methodology.

The brief titles, official titles, then details of the

studies will be successivelly read by two experts; a

consensus will be reached on the retained trials to be

explored and retained in our discussion.

Criteria of selection will focus on the fact that the

study addresses well an high risk medical device (the

definition is not exactly shared), and that an

“innovative” / interesting design was provided; the

first identified methodologies will be discussed in

order to determine the originalty and relevancy for

medical devices field.

4.2 Analysis

In term of analysis, we will considerate the possible

problems, challenges, key points or strong points

enhanced by the investigators: we will pay attention

to the duration of the study compared to the planned

duration: was the study performed until the end? If

yes, quicker or slower than planned? If stopped, what

was the reason ? Devices was in question ? Was there

any discussion about the relevancy of the choosen

methodology ? ...

We will also extract the possible usability

information that could be detected with specific

methodologies, usability that is well know to be

important for considerations on technological

innovations.

For the results published, we will explore these

information thanks to https://pubmed.ncbi.nlm.

nih.gov/ website and based on our first articles

selection, to see if the extractions between the

registrered studies and the publications of the results

match, and in which way they are complementary or

different. We could have there an interesting view of

the ability of the available data to provide enough

information compared to our hypothesis. Are the

documentation provided in Clinical trials or

publications results systematically sufficient for

confirming our questions.

4.3 Extension to Other Databases

We plan to test the same strategy on adapted

algorithms for Medline website, Cochrane library

(https://www.cochranelibrary.com/ ), or databases for

clinical trials performed in other part of the world

(example in Asia: http://www.chictr.org.cn/abouten.

aspx, https://www.umin.ac.jp/ctr/ ).

5 PERSPECTIVES AND

PRACTICAL CASES

Mixing quantitative and qualitative analysis (looking

onto details inside the studies), we will then

considerate the state and number of employed

methodologies, depending of the device evaluated as

well as its stage of development.

Based on our experience, we extracted few

illustrations allowing to provide an idea of possible

typical cases and the way evaluations could be

provided to adequatly answer to requirements from

between authorities (for market assess, for studies

Type of methodology Number of concerned studies

Cross-Over 259

Trials within Cohorts 162

Flexible 132

Randomization Adjustment 124

Sequential 86

Adaptive 62

Trial without Informed Consent 46

Cluster 28

Treatment Switching 13

Tracker Study 7

Bayesian 6

Sample Size Adjustment 3

Dose Finding 2

Tracker Trial 2

Stepwise Multiple Arm 1

Exloration of High Risk Medical Devices Methodologies for Optimized Evaluations

287

approvals), industrials, scientists and cliniciens, and

for sure patients.

5.1 A Device without Assess to Market

Approval

In our practice, we had the case on which the strategy

of market acess was different depending of the

country/area of the world on which the manufacturer

would like to apply, which underlined different

procedures, but also lectures by authorities

concerning the way a device need to bring proofs.

5.2 A Software without Assess to

Market

The european regulation address specific sections for

software, that could have strong impact in the

diagnosis or care of the patients.

We can meet the case on which industrials

selected the strategy to play on some edges (or

adaptations?) in the lecture of the regulations, that

permit them to diffuse an “inoffensive observational”

version of their device onto hospitals and clinics on

very early phases; in that way, the aim consists in

aggregation of data, without any intervention on

patients. The data accumaleted need for sure to comply

with General Data Protection Regulation (GDPR

https://gdpr-info.eu/), but could help the manufacturer

to adapt its future device (especially in machine

learning/artificial intelligence considerations), and to

feed his future FDA or CE mark with in fact data

provided by the real life.

Another example relies on a software for which

we were involved since its very early stage for

development; the manufacturer had no experience

and was not structured for medical devices field. In

order to well understand the context, as well as giving

time to adapt the device and securize the things before

going on patients, we structured a two sequences

study, the first one being dedicated to observe the

current practices (without the software) and to define

the scenarii of use, the second one introducing the

software in simulation experimentation.

The regulatory positionning could then go on a

software first dedicated to training of caregivers,

before going on a high risk / class III medical device

- if enough proofs accumulated in these simulated

envirmnements.

5.3 A Device with a CE Mark

On medical devices already on the market since years,

we had to provide specific medico-economic

evaluations, taking into account these efficiency

considerations, in terms of duration of

hospitalization, back to work time, quality of life…

And in fact impacting the way the care and the

evaluations need to be arranged, with sometimes a

strong gap between what imagine the industrial at the

very beginning, and the proposed organization.

5.4 Use of Real Data

More and more and with the available big amount of

data, it is possible to imagine some “virtual”

controlled group, with possible pairing between a real

patient prospectively enrolled in a study, and his pair

selected onto database. This need strong thoughts on

criteria of inclusion (pairing), as well as available data

linked to the criteria of judgements. And then even if

it’s seems quite interesting (reducing the number of

patients involved in a research, gaining time, reducing

cost of a study…), it is not possible in many cases and

must not make forget to respect the chronological

steps of testing.

5.5 Implementation of a Device along

the Trial

For a very diruptive innovation involving not only

technology, but also organizations around patient and

its environment, we can pay attention to a tracker trial

that in fact allowed different phases introducing

implemented version of a prototype, each phases

turning profit from the previous ones, and feeding the

next ones (in term of adaptation of the device, but also

for the evaluations).

6 CONCLUSION

Considering our results, we plan to extend analysis

and to in end build recommendations, illustrating and

guiding the skateholders on the pathway to the

selection of a relevant methodology, depending of the

device, its level of risk but also its destination, its use,

or its stage of development. We will also think about

a “bottom up” approach by starting from the

methodology/design point of view.

We built a working group constituted of

methodologist, medical doctors, specialists in

medical devices evaluations, usability experts, in

order to set up this identification and study of cases.

Our results will be confronted to different actors of

the field, disseminated and adapted along the

feedbacks we’ll received, and finally to provide better

ClinMed 2021 - Special Session on Dealing with the Change in European Regulations for Medical Devices

288

evaluations, better medical devices, for better

healthcare for patients.

REFERENCES

Barker, D, et al., 2016. Stepped wedge cluster randomised

trials: a review of the statistical methodology used and

available. BMC Med Res Methodol 6 ;16:6..

Brunotte, G., Beuscart, R., Pariset, A. and Pazart, L. Place

of High-risk Medical Devices in European

recommendations with a Focus on End-users. In

Proceedings of the 13th International Joint Conference

on Biomedical Engineering Systems and Technologies

(BIOSTEC 2020) - Volume 1: BIODEVICES, pages

350-360. ISBN: 978-989-758-398-8.

Campbell, G, 2011. Bayesian statistics in medical devices:

innovation sparked by the FDA. J Biopharm Stat.

21:871-87.

Campbell, G, et al., 2016. Statistical innovations in the

medical device world sparked by the FDA. J Biopharm

Stat. 26 (1):3-16.

Guetterman, TC, et al., 2015. Reflections on the Adaptive

Designs Accelerating Promising Trials Into Treatments

(ADAPT-IT) Process-Findings from a Qualitative

Study. Clin Res Regul Aff. 32(4):121-130.

Hamilton, C,et al., 2012. Sequential design for clinical trials

evaluating a prosthetic heart valve. Ann Thorac Surg,

93(4): 1162-1166.

Jiang, F, et al., 2013. A Bayesian decision-theoretic

sequential response-adaptive randomization design.

Stat Med, 32 (12):1975-94.

Kim SY, Flory J, Relton C. Ethics and practice of Trials

within Cohorts: An emerging pragmatic trial design.

Clin Trials. 2018 Feb; 15(1):9-16.

Lai, TL, et al., 2019. Adaptive enrichment designs for

confirmatory trials. Stat Med 38 (4) :613-624.

Lihoreau, T., Chatelain, B., Rolin, G., Vidal, C., Butterlin,

N., Jacquet, E., Elouneg, Chambert, J., Bertrand, X.,

Meyer, C. and Louvrier, A. End-user Need based

Creation of a Medical Device: An Experience of Co-

design to Struggle Pathological Scars. In Proceedings

of the 13th International Joint Conference on

Biomedical Engineering Systems and Technologies

(BIOSTEC 2020) - Volume 1: BIODEVICES, pages

317-322. ISBN: 978-989-758-398-8.

Lilford, R, et al., 2000. Trials and fast changing

technologies: the case for tracker studies. BMJ, 320

(7226): 43–46.

Magirr, D, et al., 2016. Sample Size Reassessment and

Hypothesis Testing in Adaptive Survival Trials. PLoS

One. 11(2):e0146465.

Meurer, WJ, et al., 2017. Multiple assignment randomized

Trials: An opportunity for improved design of stroke

reperfusion trials. J Stroke Cerebrovasc Dis. 26 (4):717-

724.

Pennello, G, et al., 2008. Experience with reviewing

Bayesian medical device trials. J Biopharm Stat 18 :81-

115.

Simon, N, et al., 2013. Adaptive enrichment designs for

clinical trials. Biostatistics 14(4):613-25. Tamura, RN,

et al., 2016. A small n sequential multiple assignment

randomized trial design for use in rare disease research.

Contemp Clin Trials. ; 46:48-51.

Vidal, C., Beuscart, R. and Chevallier, T. Contribution of

Methodologies Adapted to Clinical Trials Focusing on

High Risk Medical Devices. In Proceedings of the 13th

International Joint Conference on Biomedical

Engineering Systems and Technologies (BIOSTEC

2020) - Volume 1: BIODEVICES, pages 337-343.

ISBN: 978-989-758-398-8.

Wei, B, et al., 2018. A Bayesian analysis of small n

sequential multiple assignment randomized trials

(snSMARTs). Stat Med 20 ; 37 (26) :3723-3732.

Weijer C, Goldstein CE, Taljaard M. TwiC or treat? Are

trials within cohorts ethically defensible?. Clin Trials.

2018;15(1):21-24. doi:10.1177/1740774517746622.

Zelen, M, et al., 1990. Randomized consent designs for

clinical trials: an update. Stat Med 9 (6): 645-56.

Exloration of High Risk Medical Devices Methodologies for Optimized Evaluations

289