MEASURING THE FORCES APPLIED TO A VIRTUAL

REALITY LAPAROSCOPIC SURGICAL SIMULATOR WITH

QUANTUM TUNNELLING COMPOSITE SENSORS

I. W. Mack, K. McMenemy, R. S. Ferguson

The Queen’s University of Belfast, Belfast, United Kingdom

S. Potts, A. Dick

The Royal Belfast Hospital for Sick Children, Belfast, United Kingdom

Keywords: Laparoscopic simulator, low cost, PIC microcontroller, quantum tunnelling composite, force feedback.

Abstract: Abdominal surgery performed by laparoscopy requires a very high degree of skill in the surgeon. This skill

level can only be acquired through practice and training. However, a virtual reality (VR) training simulator

commands a high price. There is no reason for a VR simulator to be expensive, as a modern personal

computer can produce high-quality graphics. If all that was required was good graphics, every surgeon

could have a training tool within their laptop computer. What is missing is suitable low-cost human

interface hardware – the equivalent of the computer game ‘joystick’. This paper presents a design for a low

cost device to address this issue. In order to provide force feedback, the forces exerted on the surgical

instruments have to be measured by sensors mounted at various points. The sensors are constructed from

quantum tunnelling composite pills which measure the pressure applied to them by the surgeon. The force

feedback is produced by small direct current motors. The low cost design has been tested by both specialist

laparoscopic surgeons and non surgical personnel to assess its benefits in training at different levels of

expertise. A preliminary qualitative report is given which documents the findings from these initial tests.

1 INTRODUCTION

Laparoscopic, or minimally invasive, or keyhole

surgery is well established among surgeons as a

technique used to carry out surgery through small

incisions as compared to larger incisions required for

traditional surgery. However, while a singer, for

example, can practice before a concert, a surgeon

does not always have available the means to sharpen

his/her skills before performing an operation.

The quality of the training equipment to which a

surgeon has access largely depends on the amount of

money a hospital has available for that purpose. At

the low cost end of the market, a plastic box with a

fabric top represents the body, with holes through

which real surgical instruments are pushed to

operate on rubber body parts. A camera looks

through the side of the box and displays the

‘operation’ on an ordinary video monitor.

At the top end of the market the laparoscopic

surgical trainer can be a pair of pseudo surgical

instruments with force feedback and virtual reality

software running on a computer which displays a

depiction of the inside of a virtual patient and VR

tool tips. The LapVR system from the Immersion

Corporation allows the surgeon to learn essential

skills through virtual reality training. The VEST

systems from Select-IT VEST Systems AG allow for

training in gall bladder removal and surgery in

gynaecology. The SurgicalSim Education Platform

from Simsurgery can be used for a wide range of

training scenarios including trocar placement,

camera navigation, suturing and knot tying.

VR training systems like these, and the one

shown in Figure 1, can cost many thousands of

dollars. However, the cost of such simulators need

not be prohibitive, as personal computer (PC) game

technology can deliver excellent, high quality, real

time interactive 3D graphics that can easily meet any

W. Mack I., McMenemy K., S. Ferguson R., Potts S. and Dick A. (2008).

MEASURING THE FORCES APPLIED TO A VIRTUAL REALITY LAPAROSCOPIC SURGICAL SIMULATOR WITH QUANTUM TUNNELLING

COMPOSITE SENSORS.

In Proceedings of the First International Conference on Biomedical Electronics and Devices, pages 29-34

DOI: 10.5220/0001046700290034

 SciTePress

requirements of a VR laparoscopic simulator. If it

was only an issue of graphics, then every surgeon

could have a training and practicing tool on their

laptop PC. It is suitable human interface hardware,

the equivalent of the computer game ‘joystick’ or

‘flight simulator yolk’ that is missing.

Figure 1: A VR laparoscopic surgery application

displaying a representation of the abdomen and two simple

instruments.

When interacting with a 3D virtual environment the

experience not only depends on the type of device

used to provide the human interaction, but also on

external factors such as feedback, whether this is

visual, audio or tactile. Other factors also have to be

considered, such as the number of degrees of

freedom of the user interaction device, as well as

subjective and ergonomic aspects. For example,

(Kontarinis and Howe, 1995) demonstrated that high

frequency vibrations played a significant role in

manipulation tasks. (Lindeman, Sibert and Hahn

1999) showed that adding passive-haptic feedback to

precise manipulation tasks appreciably improved

user performance. (Lindeman, Templeman, Sibert

and Cutler, 2002) demonstrated that adding

vibrotactile feedback to visual and auditory feedback

improved the user experience of virtual

environments.

Force reflecting interfaces such as the

PHANTOM Desktop Device from Sensable

Technologies provide useful feedback, but their

utilization is limited by their cost and the fact that it

is difficult to customize them to fulfill a specific

task.

Vibrating motors, like the ones in mobile

telephones, can be used to provide low-cost

vibrotactile feedback as demonstrated by (Cheng,

Kazman and Robinson, 1996). Indeed vibration

elements have been attached to a standard computer

mouse by (Hughes and Forrest, 1996) to provide

tactile feedback.

This research reflects the belief that integrated

tactile and visual feedback must be implemented in

surgical simulators. The visual feedback is readily

available in real time and at low cost using advanced

computer graphics. The tactile feedback is much

more difficult to integrate into the simulator at low

cost. This paper aims to resolve this issue by

documenting the development of a low cost haptic

device for use in a surgical simulator. Following on

from this introduction, section 2 will outline the

basic design of the mechanical hardware. Sections 3

and 4 describe the electronic system and interface

protocols. The quantum tunnelling sensors are

introduced in section 5 and section 6 illustrates how

the concept was initially tested. Section 7 will then

present conclusions from our work to date.

2 THE SURGICAL TOOLS

This section outlines the design and development of

low-cost haptic devices for interaction with a

surgical simulator.

2.1 The Mk1 Surgical Instruments

The hand grips, and spatial positions used in the

pseudo-instruments, were constructed to match

dimensions taken from actual surgical instruments.

The instruments (the surgical tools) were mounted

on pillars attached to a base. They could rotate from

the horizontal to point down by 45 degrees, and

from facing forward to point inwards by 45 degrees.

These instruments had no electronics attached, as

they were produced solely to verify that the

measurements taken had been translated into an

accurate 3D model. Following initial field trials it

was decided to make some alterations to the position

of the pillars because, during an operation the tool-

tips mostly remain within a 4cm diameter spherical

volume. These changes had the advantage of

allowing more space at the tool-tip end to mount

some electronics in the Mk2 instruments.

2.2 The Mk2 Surgical Instruments

From our earlier research, Mack, Ferguson, Potts

and McMenemy (2006), and discussions with

practicing surgeons it was decided that each joystick

or surgical tool should have six degrees of freedom.

That is, they should allow for movement in the X, Y

and Z axes, rotation about the Z-axis, rotation of the

BIODEVICES 2008 - International Conference on Biomedical Electronics and Devices

tool-tip about the Z-axis, and the opening and

closing of the tool-tip. A Binary-Coded Decimal

(BCD) thumbwheel switch was used to simulate the

actual method of rotating the tool-tip. Five small

rotary potentiometers were used to detect movement

in the X, Y and Z axes, rotation about the Z axis and

the opening and closing of the tool-tip.

For increased stability the instruments were

mounted on a heavy wooden base as shown in

Figure 2.

Figure 2: A pair of Mk2 instruments mounted on a

wooden base.

3 THE USB INTERFACE

This section outlines the design and development of

the Universal Serial Bus (USB) which was chosen to

interface the instruments to the PC.

3.1 The Electronics

The 16C765 USB microcontroller from Microchip

was selected for the interface because Microchip’s

MPLAB and Crownhill’s Proton Development Suite

provide a stable and comprehensive Integrated

Development Environment (IDE). The PICs are

cheap and employ re-usable re-programming

technology, and require little in the way of

additional external components to implement a

working circuit. They have low power requirements

and can therefore draw their power from the

Universal Serial Bus itself.

Members of the Human Interface Device (HID)

Class such as USB keyboards and mice are low-

speed devices, and use interrupt data transfer.

The maximum possible transfer rate of data for

this combination is 8 bytes per 10 milliseconds.

This is quite sufficient for joysticks and similar

devices such as the custom-made Mk2 pseudo

surgical instruments because each surgical

instrument’s interface has only to send 6 bytes of

data, one byte for each potentiometer and one byte

for the thumbwheel switch on the pseudo surgical

instrument.

The PIC was initially configured as a Human

Interface Device without force feedback for the Mk2

joystick. Descriptor details for a HID device can be

found in our earlier paper (Mack et al., 2006).

With the addition of force feedback, each

interface would have to send force data to the PC

and receive force feedback data from the PC. It was

apparent that a HID class interface could not handle

the data rates required. For a Mk3 joystick another

method would have to be found to send and receive

larger data bursts.

Other classes of USB devices were investigated

to determine if any would be suitable for use in a

force feedback interface. The Communications

Device Class (CDC) specification from USB.org

indicated that it could be used for bulk data transfer.

However, the 16C765 could only operate as a

HID class device, so the 18F4550 was selected as it

could operate as a Communications Class Device.

A PCB very similar to the one used for the Mk2

joysticks was designed for the new microcontroller

on the Mk3 joysticks. Two of these circuit boards

are required, one for each surgical instrument, and

are mounted underneath the base of the instrument.

3.2 The PIC Descriptors

The Universal Serial Bus interface uses a serial

protocol, and depending on how it is configured, can

be low, full or high speed. The maximum data rates

are 800 bytes per second for low speed, 1.2

Megabits per second for full speed and 53 Megabits

per second for high speed.

When a USB interface is connected to a PC a

procedure called enumeration takes place. During

enumeration the interface must send descriptors to

the PC which completely define the USB device’s

capabilities and how the device will be used.

MEASURING THE FORCES APPLIED TO A VIRTUAL REALITY LAPAROSCOPIC SURGICAL SIMULATOR

WITH QUANTUM TUNNELLING COMPOSITE SENSORS

If a USB device can be incorporated into a standard

USB device class then there is a good chance that it

can be made to work using the standard device

drivers included with the Windows operating

system. Configuring the interfaces as members of

the Communication Device Class results in a

twofold advantage. Firstly, the Proton+ compiler is

able to use customizable descriptors provided by

Microchip which allows the Communications

Device to emulate RS232 serial protocol over a USB

connection. Secondly, the device enumerates as a

COM port on the PC and allows the use of a

standard Windows device driver to establish

communications with the device.

Although the descriptors define the interfaces as

standard COM ports, this is only used as a

convenient way to input data to and output data from

the PC application via the Universal Serial Bus.

Details of the descriptors used in the Mk2 and

Mk3 surgical instruments are explained in detail in

internal documents entitled, “Descriptors required

for a HID USB Interface”, and, “Descriptors

required for a CDC USB Interface”, respectively.

4 THE PIC PROGRAM

However, to have any purpose, the USB devices

must each run an application program.

The surgical instruments were originally

developed without force feedback, and only fed

position information to a Windows VR application,

which can be seen in Figure 1. A different PIC

application was developed for use with an

experimental rig to test the setup for force feedback

in one degree of freedom.

The data flow for one degree of freedom can be

seen in Figure 3. The force feedback in the surgical

instruments will be provided by small DC motors,

which will provide a more tactile feel for the

surgeon. A potentiometer at each pivot point

provides position data. The force exerted by the

user is measured by the use of pressure sensors

produced from Quantum Tunnelling Composite

(QTC) pills manufactured by Peratech.

Excluding the thumbwheel switch input, which is

not suitable for force sensing or force feedback, four

sets of data are required for the operation of each

degree of freedom. Position data has to be sent from

the joystick to the PC application, as does data

regarding the force exerted by the user in two

directly opposing directions. Force feedback data

has to be sent from the PC application to the pseudo

surgical instruments.

Figure 3: Data flow for one degree of freedom.

Small DC motors have been used in the

development process. The motors have to operate in

both forward and reverse, and this can be achieved

by the use of an H-bridge circuit. The motor torque

can be controlled by using Pulse Width Modulation

(PWM) to vary the average current. A motor can be

controlled by varying the duty cycle and/or the

frequency of the PWM pulse train applied to it.

Some PICs, such as the 16F777 used here, have

hardware PWM channels which allow the pulse train

to be produced in the background while the program

is executing other instructions.

It was decided to produce separate modules

which could control three degrees of freedom, with

each module being controlled by a 16F777 PIC. The

PIC also has the nine Analogue-To-Digital (A/D)

converters required to sample three position and six

force sensors. Each module sends data to the USB

interface PIC and receives data from it to control the

torque on three force feedback motors.

At this stage of the research it was decided to

work on only one degree of freedom on one surgical

instrument.

The applications for the USB and 16F777

module PICs were written in Basic and compiled

using the Proton Development Suite.

BIODEVICES 2008 - International Conference on Biomedical Electronics and Devices

5 THE QTC SENSORS

When deformed, Quantum Tunnelling Composites

transform from a near perfect insulator to a

conductor similar to metal. This transformation can

be as a result of compression, stretching or twisting

the composite.

Each QTC pill is sandwiched between two

electrodes, and the user force applied

perpendicularly to the flat face of the pill, as shown

in Figure 4.

The circuit for measuring user force is a simple

potential divider with the sensor in series with a

current limiting resistor. Under no pressure the

sensor appears open circuit, while under pressure it

appears short circuit.

Figure 4: A QTC sandwich sensor.

6 TESTING THE CONCEPT

Figure 5 shows the rig used to evaluate the force

feedback setup. For the sake of clarity it is shown

before wiring.

A simple front-end program, written in Visual

Basic (VB) is used to control and display the output

from the experimental test rig.

Figure 5: The force feedback test rig.

Four bytes of data specify the motor behaviour. The

application program allows forward or reverse

motion to be selected. It is also possible to

interactively control the PWM signal sent to the

motor through the simulated RS232 protocol over

the USB interface to the 18F4550 PIC. The

18F4550 sends the data to the 16F777, to configure

the H-bridge and hence control the DC motor, thus

providing force feedback.

The tests proved that a single personal computer

could act as output generator for force specification

and simultaneously acquire and act upon signals

from the pressure sensors in real-time.

7 CONCLUSIONS

This paper presented a design for, and practical

realization of, a low-cost laparoscopic surgical

training tool that offers the possibility of

incorporating force sensing through the novel use of

quantum tunnelling composite force sensors.

Several electronic interfaces have been

developed, culminating in a fully implemented USB

design that emulates a traditional PC COM port. A

test rig for a one degree of freedom force feedback

axis has been produced which sends position and

force data via one of these USB interfaces to a driver

program in a host PC. Data from the driver program

has been returned to the test rig to control a force

feedback motor. It is the ultimate aim of this

research to have twelve such motors controlled in

the final version of the surgical instruments.

Two instruments give a surgeon the opportunity

to practice realistic procedures, and our design offers

the possibility of highly accurate force sensing and

feedback in the future. The tight interaction between

force sensing and feedback afforded by the quantum

tunnelling sensors will alleviate some of the

problems that arise in conventional haptic rendering

applications, for example the need for a very high

sampling rate in the servo loop.

When combined with the realistic graphics that

current Graphics Processing Unit (GPU) based

rendering hardware offers it will be possible to offer

a very effective training package that could be made

widely available.

Two surgeons and two non-specialists tested the

Mk2 instruments in conjunction with the virtual

reality software and they were very favourably

received. Both groups found the instruments easy to

use, with the surgeons able to perform simulated

surgery on a gall bladder. With force sensing, force

feedback and collision detection algorithms,

surgeons will be able to practice realistic surgical

procedures with a good level of authenticity.

The development of force feedback systems that

are actually used by surgeons requires close

MEASURING THE FORCES APPLIED TO A VIRTUAL REALITY LAPAROSCOPIC SURGICAL SIMULATOR

WITH QUANTUM TUNNELLING COMPOSITE SENSORS

teamwork between medical staff and researchers. It

is hoped that the development of this VR trainer will

result in a laparoscopic simulator which will be

acknowledged by the medical profession and lead to

an enhancement in the safety of patient care.

The original aim of making a pair of low-cost

pseudo surgical instruments for a VR laparoscopic

simulator is well on the way to a successful

conclusion. The component cost of the Mk2

instruments is approximately 0.4% of the cost of the

Virtual Endoscopical Surgery Training (VEST)

simulator without force feedback from Select-IT

VEST Systems AG. The Mk3 training instruments,

when they have force sensing and force feedback

implemented will have a projected component cost

of 1.1% of the cost of the VEST force feedback

system. While our costs do not include the cost of

manufacturing the instruments they compare very

favourably with commercial equipment. Indeed we

could include a laptop computer with pre-installed

laparoscopic simulation software, which could also

be used independently as an ordinary PC, and still

cost only 1.5% and 1.7% of the commercial non

force feedback and force feedback systems

respectively.

REFERENCES

Cheng, L.-T., Kazman, R., & Robinson, J. (1996).

Vibrotactile Feedback in Delicate Virtual Reality

Operations. Paper presented at Fourth ACM

International Conference On Multimedia. Boston,

MA.

Hughes, R., & Forrest, A. (1996). Perceptualisation Using

a Tactile Mouse. Paper presented at Visualization ’96.

San Francisco, CA.

Kontarinis, D., & Howe, R. (1995). Tactile Display of

Vibratory Information in Teleoperation and Virtual

Environments. Presence: Teleoperators and Virtual

Environments, 4(4), 387-402.

Lindeman, R., Sibert, J., & Hahn, J. (1999). Towards

Usable VR: An Empirical Study of User Interfaces for

Immersive Virtual Environments. Paper presented at

ACM CHI ’99. Pittsburgh, PA.

Lindeman, R., Templeman, J., Sibert, J., & Cutler, J.

(2002). Handling of Virtual Contact in Immersive

Virtual Environments: Beyond Visuals. Virtual

Reality; 6(3), 130-139.

Mack, I.W., Ferguson, R.S., Potts, S., & McMenemy, K.

(2006). Adding tactile realism to a virtual reality

laparoscopic surgical simulator with a cost effective

human interface device. Paper presented at Electronic

Imaging 2006. San Jose, CA.

BIODEVICES 2008 - International Conference on Biomedical Electronics and Devices