A VIRTUAL REALITY SIMULATOR FOR TRAINING

WRIST ARTHROSCOPIC SURGERY

Fadi Yaacoub

1,4

, Yskandar Hamam

1,2,3

and Antoine Abche

Université Paris-Est, ESIEE-Paris, Laboratoire A²SI, Cité Descartes, 93162 Noisy Le Grand, France

Université de Versailles SQ, Laboratoire LISV, 10/12 Avenue de l'Europe, 78140 Velizy, France

F’SATIE -Tshwane University of Technology, Private Bag X680, Pretoria 0001, South Africa

University of Balamand, Dept. of Electrical Engineering, P.O.BOX 100, Tripoli, Lebanon

Keywords: Virtual Reality, Surgical Simulation, Convex Hull, Collision Detection, Healthcare Technology, Wrist

Arthroscopy, Haptic Feedback, 3D Modeling and Visualization.

Abstract: The minimally invasive approach of arthroscopy means less pain and faster recovery time for patients

compared to open surgery. However, it implies a high difficulty of performance. In this paper, a functional

prototype of a virtual reality simulator for training wrist arthroscopic surgery is introduced. This simulator

allows medical students as well as surgeons to interact with anatomical structures by modeling and

operating on virtual objects displayed on the computer screen. A 3-D virtual representation of the bones

constituting the wrist of a patient is shown. Also, algorithms that model objects using the convex hull

approaches and simulate real time collision detection between virtual objects during the training on the

operation are presented. In addition, a force feedback device is used as a haptic interface with the computer

simulation system. This leads in the development of a low cost system that is used by trainees with the same

benefits as professional devices. In this regard, the wrist arthroscopy can be simulated and medical students

can easily acquire the system and can learn the basic skills required with safety, flexibility and less cost.

1 INTRODUCTION

Virtual Environment (VE) provides a new

dimension of graphical simulation (Goebel, 1993). It

is described as an application that allows users to

navigate and interact with a computer-generated

three dimensional (3-D) space in real time. In this

context, Virtual Reality (VR) is not only a hardware

system but also an emerging technology that

changes the way individuals interact with computers.

Recently, medicine has entered a period of intense

technological transition driven by the need to

provide improved care at lower cost. Since, the

outcome of surgical procedures is closely related to

the skills of the surgeon, the latter should remain at a

high level of technical and professional expertise.

These skills are being developed over years of

surgical training on animals, cadavers and patients.

For surgical trainees to reach a high level, new and

alternative ways of performing surgical training are

required. In addition, the low availability and high

cost of cadaver and animal specimens for traditional

medical training and the public concern with the

inhuman treatment of animals have become another

impetus for surgeons and medical students to use

new technology in their education and their training

to gain valuable information and experience.

VR technology has opened new realms in the

practice of medicine. The graphics capabilities of

VR tools, particularly in modeling and displaying

medical data can be of great assistance in teaching,

learning, training and experimenting surgeries.

Furthermore, researchers on medical education

depend heavily on VR simulators that have become

one of the main components of changing radically

the traditional medical training and the surgical

certification scenarios (Immersion Corporation).

They allow the process of iterative learning through

assessment, evaluation, decision making and error

correction which create a much stronger learning

environment.

An important application is the arthroscopic

surgery simulation. In arthroscopy, the object is

visualized and accessed through small portals. An

optical endoscope equipped with a video camera

allows the visualization of the procedure through

one of the portals, while surgical probes and other

instruments are inserted through additional portals.

Yaacoub F., Hamam Y. and Abche A. (2009).

A VIRTUAL REALITY SIMULATOR FOR TRAINING WRIST ARTHROSCOPIC SURGERY .

In Proceedings of the International Conference on Health Informatics, pages 74-81

DOI: 10.5220/0001552500740081

 SciTePress

Arthroscopy decreases soft tissue disruption which

leads to less pain and less chance for infection.

However, it implies a high difficulty of performance

and necessitates the surgeon to acquire psychomotor

skills which are essential to become expert. On the

other hand, arthroscopy is increasingly being used in

the treatment of the hand. Wrist arthroscopy, in

particular, has proven to be extremely valuable in

both diagnosis and therapy. It is an important skill

for all hand surgeons (

Haisman et al., 2005), in exactly

the same way as shoulder and knee arthroscopy.

The skills required for arthroscopy are taught

through hands-on clinical experience. As

arthroscopy becomes a more common procedure, it

is now obvious that special trainings are necessary to

master surgical operations and guarantee

qualification of the surgeons. Different research

groups have shown significant advantage of using

medical simulation systems over existing

conventional methods that use live patients. Hence, a

VR training system to simulate wrist arthroscopic

procedures in VE is proposed in this paper. Two

main issues are addressed: the 3-D reconstruction

process and the 3-D interaction. The proposed

system provides a VE with realistic representation of

the region of interest. Based on a sequence of CT

images a realistic representation of the wrist joint is

obtained suitable for the computer simulation. Two

main components of the computer-based system

interface are illustrated: the 3-D interaction to guide

the surgical instruments and the user interface for

haptic feedback. In this context, algorithms that

model objects using the Convex Hull (CH)

approaches and simulate real time exact Collision

Detection (CD) between virtual objects during the

training on the surgical operation are presented.

Also, a force feedback device is used as a haptic

interface with the computer simulation system.

The rest of the paper is structured as follows:

section 2 reviews some of the previous surgical

simulators. The design criteria of the proposed VR

system are presented in section 3. Section 4 shows

the segmentation of the CT images and the

generation of the 3-D virtual wrist model. In section

5, algorithm to construct the CH is implemented,

then the CD problem is formulated and a linear

programming solution is obtained to test whether a

collision exists or not. A force feedback device that

is used as a haptic interface with the computer

simulation system is presented in section 6. Section

7 shows a virtual simulation of dorsal percutaneous

scaphoid fixation. Finally, conclusions are given in

section 8.

2 RELATED WORK

VR Surgical simulators have been developed for a

wide range of procedures. However, they are often

associated with specific involvements. The VR

simulators presented are classified based on their

applications and their relation to the organs or areas

they treat.

Many simulators are associated with laparoscopy

such as LapSim (Surgical Science). The LapSim

simulator focuses on implanting basic skills that

would be needed by the trainee towards performing

bigger procedures. The Lapmentor (Simbionix) is a

force feedback enabled laparoscopic training

simulator. Medical students can train on either the

basic skills or perform full procedures. This system

offers as well the opportunity to perform a complete

surgery. Moreover, The LASSO project (G. Szekely

et al., 2000) is an integrated development effort to

construct a laparoscopic simulation platform. The

abdominal cavity is modeled using data from the

Visible Human. Organ surface features are generated

using a combination of texture analysis/synthesis. In

addition, MIST is an endoscopic simulator where

trainees are guided through a series of exercises of

progressive complexity, enabling them to develop

the skills essential for good clinical practice and

VIST is a simulator for catheter based procedures

for angiography and interventional procedures

(Mentice). With VIST, trainees are able to practice

on many operations such as carotid, coronary, renal

and vena cava. Furthermore, VR simulations of

cystoscopy and ureteroscopy procedures are done

using the UroMentor (Simbionix). The UroMentor

has a mass of practice modules and patient profiles

that can be used to perform safe surgical procedures.

Besides, the (Simbionix) GI Mentor II simulator,

associated with colonoscopy, is an interactive

computerized simulator that provides hands-on

training in endoscopic procedures. Also, Bro-

Nielsen et al. (1999) described a PC-based

bronchoscopy simulator. In addition to realistic

visual effects, this system uses a haptic interface

designed to provide realistic force feedback during

scope insertion. The system has been expanded to

include colonoscopy and flexible sigmoidoscopy.

However, most simulators described above are

expensive to acquire and need maintenance.

Regarding arthroscopy simulators, most

developments have been for knee training (Heng et

al., 2004), the second case of arthroscopy that was

treated is the shoulder arthroscopy simulations

(Bayonat et al. 2006) and very little work has been

done for wrist arthroscopy even though the wrist is a

A VIRTUAL REALITY SIMULATOR FOR TRAINING WRIST ARTHROSCOPIC SURGERY

very important joint in the body and it handles many

activities. Thus, the problem of building an

inexpensive and practical simulator for training

medical students and treat the issue of the wrist

arthroscopy remained.

3 DESIGN CRITEREA

The design of the proposed computer-based

arthroscopy simulator was based on a trade-off

between medical professor’s needs and VR

limitations. Wrist arthroscopy is selected due to

several reasons:

1- Wrist arthroscopy is a frequent pathology

(study of essential nature of disease) that has been

less studied and practiced than knee and shoulder

arthroscopy.

2- A varied type of involvements and specific

surgeries can be covered by wrist arthroscopy

simulation such as: dorsal percutaneous scaphoid

fixation, volar percutaneous scaphoid fixation,

capitolunate arthrodesis ...

Two major aims are addressed:

1- Applying VR and physical simulation

techniques to generate 3-D models and to simulate

operations with fidelity and realism.

2- Trying to cover different requirements for the

apprentice learning process and providing the user

with tools to facilitate teaching, learning and training

on several experiments.

Therefore, medical images are processed to

generate volumetric object models. These 3-D

models are presented both visually via rendering on

the computer monitor and haptically with a force

feedback device. Visual parameters such as

viewpoint, zooming, color and lighting effects can

be interactively controlled and object models can be

manipulated with force feedback to change relative

probe and object positions, and to simulate many

surgical procedures. The interaction between the

haptic device and the computer closes the feedback

loop between the user and the simulator, offering a

better understanding of the anatomical structures.

Figure 1 outlines the main components of the

proposed VR simulation system.

Figure 1: A Flowchart of the Simulation System.

4 3-D OBJECT GENERATION

Segmentation subdivides an image into its

constituent parts. The watershed segmentation

(Couprie et al., 2005) has proven to be a powerful

and fast technique for both contour detection and

region based segmentation. This method allows,

from a gradient image, to find a thin separation

between the components of a given set of points

called markers. Figure 2 shows the original image

and figure 3 shows the gradient with the markers.

Figure 2: Original Image.

Figure 3: Gradient of Original Image with Markers.

HEALTHINF 2009 - International Conference on Health Informatics

The watershed algorithm is implemented by

region growing based on the set of markers to avoid

over-segmentation. At the end of the process all

minima are completely separated by dams, called

watershed lines. The watershed result is shown in

figure 4.

Figure 4: Watershed Result.

The final result of segmenting a set of CT images

is a volumetric image that represents the labeled

bones. Figure 5 shows the final 3-D image of the

wrist.

Figure 5: 3-D Image of the Wrist.

After the segmentation of the CT images, the

Marching Cube algorithm is used to construct the

boundary of the objects in the scene. The algorithm

finds the appropriate surface patch in a look-up table

and builds this patch, interpolated according to the

values of the eight corners of this unit cube. The

union of all these patches constitutes the

approximated iso-surface and a list of facets is

generated (Daragon et al., 2003). Figure 6 shows a

high resolution 3-D virtual representation of the

bones constituting the wrist of a patient. This

representation provides the surgeon with precise and

detailed information of the region of interest that he

will be working on.

Figure 6: 3-D Virtual Model of the Wrist Bones.

5 COLLISION DETECTION

The goal of a medical simulator is to support

medical students during training and practicing on

surgeries with high precision. In this regard, medical

objects are modeled with a tightness fit i.e. each

object is modeled by its corresponding CH. This will

give the simulator a high degree of precision but at

the same time an increase in the cost of the

complexity and the computational time for collision

check. Therefore, by taking advantages of the speed

and robustness of Linear Programming (LP)

techniques the problem of CD is formulated and

solved (Yaacoub et al, 2007). In addition, convex

objects allow the LP algorithm to converge quickly

and detect the collision if it exists. Thus, the CH of

each object is reconstructed. Then, the CD problem

is formulated as an optimization problem based on

convex objects and solved using linear programming

(simplex method).

5.1 Convex Hull Algorithm

Most exact collision detection systems work almost

exclusively with convex objects because they allow

CD algorithms to converge quickly. Moreover,

convex envelopes have less contact points than real

objects. This leads to a decrease in the size of the

system of equations needed to calculate the

collision. A new hybrid CH technique is developed

to construct the convex envelope of a 3-D medical

object (Yaacoub et al., 2006). The corresponding

pseudo-code is shown as follow:

A VIRTUAL REALITY SIMULATOR FOR TRAINING WRIST ARTHROSCOPIC SURGERY

Algorithm 1: The Convex Hull Approach.

1: find an initial plane from the min and max abscise and

the max distance with respect to (x

min

, x

max

)

2: construct a polyhedron from the initial plane and the

max distance to this plane

3: for each facet F of the polyhedra do

4: for each unassigned point p do

5: if p is above F then

6: assign p to F's outside set

7: end if

8: end for

9: end for

10: Discard all points inside the polyhedron forming a

new

input set n

new

11: find a starting edge (a, b) using the 2D Gift

Wrapping

algorithm on the XY projection

12: for i = 1 ... n

new

13: find point p

corresponding to min angle between

plane P in XY containing (a, b) and plane T = (a, b, p

)

14: replace c ← p

15: save (a, b, c) into Q

16: wrap the edge (a, c)

17: if facet has been explored then

18: wrap the edge (b, c)

19: if facet has been explored then

20: return

21: end if

22: end if

23: end for

As a result of applying the CH hybrid technique,

figure 7 shows different bones constituting the 3-D

wrist model: 1

Metacarpal (a), 2

Metacarpal (b),

Metacarpal (c), Scaphoid (d), Capitate (e),

Hamate (f), Radius (g) and Ulna (h). Each bone is

covered with its corresponding convex envelope.

Figure 7: Bones from the 3-D wrist model enclosed by

their corresponding CHs.

5.2 Linear Programming Solution

To formulate the problem, each facet i of the convex

envelope is represented by the plane inequality in

the form of:

iiii

dzcybxa ≤

(1)

Any point lying on the object must satisfy the

inequalities of the planes constituting the object.

These equations form the constraints of the collision

problem and represent the facets that separate two

regions in space. Therefore, if a point satisfies two

sets of inequalities simultaneously, it belongs to the

corresponding convex objects. Thus, a collision is

detected at that point between these two objects.

The problem is reduced to maximize an objective

function in the form of (x + y + z). It is formulated

as follows:

X c

max

(2)

subject to:

bAX

≤

(3)

[

]

x y zwhere X =

⎥

⎦

⎤

⎢

⎣

⎡

...

cba

333

222

111

(4)

[

][]

, c. . . d dd b 111

321

(5)

The coefficients of the matrices A and b are

calculated using the facets obtained from the CHs

reconstructed by the approach presented in the

previous subsection. Using the duality property, the

problem becomes:

π b

min

(6)

subject to:

cπA

≥

(7)

Having formulated the problem, the dual system is

solved using a linear programming algorithm. If the

system is bounded, a feasible solution exists and

consequently, a collision is detected. Otherwise,

there is no collision.

HEALTHINF 2009 - International Conference on Health Informatics

6 HAPTIC INTERFACE

Force feedback is a very interesting technology in

the context of human machine interface. It is used

as a haptic interface in order to make 3-D models

and simulations accessible to users and

participants. In this work, a 3-DOF force feedback

device is used. This will enhance the surgical

performance by guiding the (surgeon, student ...)

and give him a sense of touch and resistance when

collision is detected.

When the user moves the haptic device, the

position of the probe changes allowing dynamic

interactions with the virtual environment. That is,

the position of the medical probe is updated at

every step and the CD is checked by applying the

proposed algorithm on the updated matrices that

formulate the collision problem, i.e. solving the

system of equations at every step change. If

collision is detected, a force is applied against the

motion of the user of the haptic device. Therefore,

the user can feel the resistance of the applied force

against his hand’s motion, i.e. against the force

applied by the user to move the haptic device. This

force-reflecting device enables medical students

during the training to experience the real feeling of

touch. Touching virtual objects rather than seeing

them enhances the capability of the computer-

based system and gives the user the feeling of so

called “Immersion”. Figure 8 shows the flowchart

of the haptic feedback algorithm.

Figure 8: The Haptic Feedback Algorithm.

Figure 9 shows the haptic feedback system

designed, implemented and tested with the

computer-based simulation system.

Figure 9: The Haptic Feedback System.

7 SIMULATIONS AND RESULTS

Techniques of performing wrist arthroscopy have

been developed to evaluate and treat various wrist

disorders, such as scaphoid fractures. For

example, the dorsal percutaneous approach is a

very efficient way in treating displaced proximal

pole scaphoid fractures in many clinical and

operating rooms. This technique allows for faster

rehabilitation without restriction once CT scan

confirms a solid union.

Percutaneous arthroscopically assisted internal

fixation by a dorsal approach may be considered

in all acute scaphoid fractures selected for

surgical fixation (Rettig and Raskin, 1999). The

dorsal guide wire permits dorsal and volar

implantation of a cannulated screw along the

central axis of the scaphoid (Wozasek and Moser,

1991). The surgical technique described in (Slade

and Jaskwhich, 2001) uses the Standard Acutrak

screw. This screw is a headless, cannulated,

tapered screw with a graduated thread pitch to

provide inter-fragmentary compression without

hardware protrusion. This technique permits the

percutaneous reduction and rigid internal fixation

of proximal pole fractures.

First, the wrist is flexed and pronated for the

scaphoid to appear as a cylinder. The center of the

cylinder is the location for guide wire placement.

Then, the guide wire is driven dorsal to volar

A VIRTUAL REALITY SIMULATOR FOR TRAINING WRIST ARTHROSCOPIC SURGERY

through the center of the scaphoid. The wire exits

at the base of the thumb. Figure 10 (a) shows the

real placement of the guide wire during the

surgery. Real figures of the operation are taken

from a real surgery done by Dr. Joseph F. Slade

and distributed by ACUMED. On the other hand,

figure 10 (b) shows the same process done

virtually using the proposed VR simulation

system

Figure 10 (a): Real Operation.

Figure 10 (b): Virtual Operation.

After this step, a hand-held cannulated reamer

is placed over the guide wire and is used to

prepare the scaphoid. The scaphoid is reamed to

fit the length of the screw. Then the screw is

selected and is advanced with a cannulated driver

to the level of the reamed scaphoid. Figure 11 (a)

shows the real insertion of the screw in the

scaphoid while figure 11 (b) shows the virtual

operation of the same process.

Figure 11 (a): Real Operation.

Figure 11 (b): Virtual Operation.

8 CONCLUSIONS

VR surgical simulators play a very important role in

the practice of surgery for medical education and

training. An innovative application is the hand

surgery, especially wrist arthroscopy, which has

proven to be an extremely valuable tool in both

diagnosis and therapy. This paper presents a

functional prototype of a VR training system for

simulating wrist arthroscopy. Segmentation of CT

images and 3-D virtual model of the wrist of a

patient are shown. Algorithms that model objects

using the CH approaches and simulate real time

exact CD for solid objects are presented. Also, a

force feedback device coupled with a haptic

simulation algorithm is incorporated with the

system. Finally, a virtual simulation of dorsal

percutaneous scaphoid fixation is shown. This leads

in the development of a system that is used to

simulate wrist arthroscopic surgery procedures in a

VE. Medical students can learn the required basic

skills and then perform the training procedure on

real patients. This low cost system is safe, flexible

and can provide the students with precise and

detailed information for training and educational

HEALTHINF 2009 - International Conference on Health Informatics

purposes with the same benefits as professional

devices.

ACKNOWLEDGEMENTS

This research is funded by a grant from the Lebanese

National Council of Scientific Research (CNRS-L).

REFERENCES

Goebel, M., 1993. (Ed.): Virtual Reality, Computers and

Graphics, Special Issue. vol.17, 6.

Medical Simulation System, Immersion Corporation

[online]. www.immersion.com/medical/products

Haisman, J., Bush, M., Wolfe, S., 2005. “Wrist

Arthroscopy: Standard Portals and Arthroscopic

Anatomy”, Journal of the American Society for

Surgery of the Hand, vol. 5, Issue 3, pp. 175-181.

LapSIM, Surgical Science Products, [online].

http://www.surgical-science.com

LapMentor, Simbionix Company Products, [online].

http://www.simbionix.com/LAP_Mentor.html

Szekely, G., Brechbuhler, C., Dual, J., et al. 2000. “Virtual

Reality-Based Simulation of Endoscopic Surgery”.

Teleoperators and Virtual Environments, 9(3), pp.

310-333.

Procedicus VIST, Procedicus MIST, Mentice Products,

[online]. http://www.mentice.com

UroMentor, Simbionix Company Products, [online].

http://www.simbionix.com/URO_Mentot.html

GI Mentor II, Simbionix Products [online].

http://www.simbionix.com/GI_Mentor.html

Bro-Nielsen, M., Tasto, J., Cunningham, R., and Merril,

G., 1999. “PreOp endoscopic simulator: A PC-Based

Immersive Training System for Bronchoscopy”,

Medicine meets virtual reality, MMVR, vol. 7, pp. 76-

82.

Heng, P., Cheng, Ch., Wong, T., Xu, Y., Chui, Y., Chan

K., and Tso, S., 2004. “A Virtual-Reality Training

System for Knee Arthroscopic Surgery”, IEEE

Transactions on Information Technology in

Biomedicine, Vol. 8, No. 2, pp. 217-227.

Bayonat, S., Garcia, M., Mendoza, C., and Fernandez,

J.M., 2006. “Shoulder Arthroscopy Training System

with Force Feedback”, IEEE conference on Medical

Information Visualization, MedVis, pp. 71-76.

Couprie, M., Najman, L., and Bertrand, G., 2005. “Quasi-

linear algorithms for the topological watershed”.

Journal of Mathematical Imaging and Vision, Special

Issue on Mathematical Morphology. Vol. 22, Issue 2 -

3, pp. 231-249.

Daragon, X., Couprie, M., and Bertrand, G., 2003.

“Discrete Frontiers”, Discrete geometry for computer

imagery, LNCS, Springer Verlag, Vol. 2886, pp. 236-

245.

Yaacoub, F., Hamam, Y., Abche, A., and Fares, C., 2006.

“Convex Hull in Medical Simulations: A New Hybrid

Approach”, 32

Annual Conference of IEEE

Industrial Electronics Society, IECON'06, pp. 3308–

3313.

Yaacoub, F., Hamam, Y., and Abche, A., 2007. “Collision

Detection in Computer Simulation for Wrist

Arthroscopy Surgery Training”, IEEE International

Conference on Computer as a Tool, EUROCON'07,

pp. 2088–2095.

Slade, J., Jaskwhich, D., 2001. “Percutaneous Fixation of

Scaphoid Fractures”, Hand clinics, vol. 17(4), pp. 553-

574.

Rettig, M., Raskin, K., 1999. “Retrograde Compression

Screw Fixation of Acute Proximal Pole Scaphoid

Fractures”, Journal of Hand Surgery, vol. 24 pp.

1206-1210.

Wozasek, G., Moser, K., 1991. “Percutaneous Screw

Fixation of Fractures of The Scaphoid”, Journal of

Bone Joint Surgery, vol. 73, pp. 138-142.

ACUMED, Innovative Orthopedic Implants and

Accessories, [online]. http://www.acumed.net

A VIRTUAL REALITY SIMULATOR FOR TRAINING WRIST ARTHROSCOPIC SURGERY