Development of a Multispectral Gastroendoscope to Improve the

Detection of Precancerous Lesions in Digestive Gastroendoscopy

Sergio Ernesto Martinez Herrera

, Yannick Benezeth

, Matthieu Boffety

, François Goudail

Dominique Lamarque

, Jean-François Emile

and Franck Marzani

Université de Versailles Saint Quentin en Yvelines, Versailles, France

Le2i,Université de Bourgogne, Dijon, France

Institut d’Optique Graduate School, Palaiseau, France

1 STAGE OF THE RESEARCH

The actual stage of research is the beginning of the

second year of the PhD thesis. The duration of the

thesis is three years. The first year has begun with

the state of the art involving two main parts. The

first one focused on medical aspects related to the

development and staging of stomach cancer. The

second part was oriented to the actual technology

and image processing techniques which are used to

help in the diagnosis of malignancies in the stomach.

The review of the state of the art mainly focused on

the study of multispectral imaging. An overview of

the state of the art is presented in section 4. Then, a

multispectral endoscope prototype has been

developed; this system is described in section 5. It is

based on the use of a filter wheel to modify the

regular white light of a gastroendoscopic system.

Afterwards, a set of image pre-processing techniques

has been developed to improve the usability of the

multispectral images obtained by the prototype.

These techniques are also described in section 5.

Finally, in section 6 are presented the future work

and the expected outcome.

2 OUTLINE OF OBJECTIVES

In order to improve the diagnosis during

gastroendoscopy, practitioners need additional

information from the tissue characteristics in a non-

invasive, efficient and accurate way.

The actual technology used to perform

gastroendoscopy is mainly based on the visual

exploration under white light; an illustration is

presented in figure 1a. Unfortunately, it is often

difficult to visualize malignancies in the tissue with

this technology. Practitioners with all degrees of

experience, including those with many years of

practice mention this situation.

The diagnosis of gastric pathologies is performed

based on biopsy acquisition and its histological

analysis, which is the microscopic evaluation of

tissue. This is considered to be the most reliable

technique for diagnosis. Unfortunately, the

collection is difficult because in many cases there

are no macroscopic differences between healthy and

wounded tissue. In consequence, practitioners

usually collect biopsies randomly. This situation

leads to the acquisition of tissue without any

pathology, which produces undesirable false

negative diagnosis.

Figure 1: a) Colour image under white light. b), c), d)

Monoband images which highlight different features.

Considering that the gastric pathologies are

related to modifications in the properties of the

tissue, there is an important need to measure these

variations. Based on previously successful

applications, we believe that multispectral imaging

can help in the identification of early stages of

Martinez Herrera S., Benezeth Y., Boffety M., Goudail F., Lamarque D., Emile J. and Marzani F..

Development of a Multispectral Gastroendoscope to Improve the Detection of Precancerous Lesions in Digestive Gastroendoscopy.

 2014 SCITEPRESS (Science and Technology Publications, Lda.)

gastric cancer, even if these lesions present subtle

colour and morphological differences in comparison

with healthy tissues using conventional white light

illumination.

In summary, there are two main objectives in this

PhD thesis. The first one is the design and

development of a prototype capable to acquire

multispectral images of the stomach during

gastroendoscopy. The system must be compatible

with the actual systems used in gastroendoscopy.

The second objective is to propose tools and

methods to identify cancerous tissue at an early

stage. This second part is not presented here. We

present in the remainder of the document, the

proposed acquisition system and the image pre-

processing techniques implemented.

3 RESEARCH PROBLEM

The first challenge of the PhD is the acquisition of

multispectral images during gastroendoscopy. Some

recent works have focused on the analysis of the

reflectance from gastric tissue using spectroscopy

(Bashkatov et al., 2007). On the other hand, other

recent works were focused on the analysis of

multispectral images of gastric tissue from ex-vivo

samples (Galeano et al., 2012); (Kiyotoki et al.,

2013). To the best of our knowledge, there is no

work on the analysis of pre-cancerous lesions using

in-vivo multispectral images of the stomach tissue

acquired during gastroendoscopy. Consequently, the

first research problem to solve is the development of

a system to acquire the data, which is not a trivial

task due to the inherent constrains from the working

environment.

The second problem to solve is the registration

between successive monoband images. Due to the

configuration of the acquisition system, the

monoband images that compose the multispectral

images are acquired sequentially and present a

shifting. It is necessary to register the monoband

images to form the multispectral image. This

registration is highly complicated because many

assumptions, upon which the majority of the state of

the art techniques rely, are not respected for our

data. These methods are usually based either on the

use of anatomical references, texture features or the

underlying assumptions to the use of optical flow,

where there is a small displacement of a constant

intensity between two images. In our particular case,

the images obtained are not compliant with these

fundamental assumptions, for instance the gastric

tissue is highly homogeneous with subtle texture in

some cases, it is in constant movement (non-rigid

deformations) and the environment is moist (large

variations in photometric properties). Moreover,

monoband images of the same area at two different

wavelengths present strong differences. In figure 1 is

shown an example of these difficulties with 3

monoband images from a gastric multispectral

image.

The two above problems are detailed in this

document. We can also mention the difficulty to

extract information from these images. The first

problem will be given by the quality of the images.

The methods to detect pre-cancerous lesions should

be sufficiently robust to operate on noisy images.

Then, the methods should be fast enough to identify

in real time tissue which is more likely to develop

cancer, in order to help the practitioner during the

gastroendoscopy. These issues will be a major part

of the work during the second and third year of the

PhD thesis.

Before introducing the acquisition system and

the image processing techniques, we present in the

following section the state of the art.

4 STATE OF THE ART

Nowadays, the majority of gastroendoscopy imaging

devices provide colour images acquired under white

light. Some systems have been developed that

increase the visualization of the lesions from a

macroscopically point of view. These systems can

be classified in two main categories. The first one

takes advantage of an external agent, for instance a

dye is used to highlight specific features of the

lesions. The main technique in this category is

chromoendoscopy (Kida et al., 2003).

The second class of systems increases the

spectral resolution of the images in order to enhance

the visualization of the tissue. The Narrow band

Imaging (NBI proposed by Olympus) or Multi Band

Imaging (FICE proposed by Fuji) belongs to this

category. For these systems, a false colour image is

formed with 2 or 3 monoband images at a specific

wavelength (Wong Kee Song et al., 2008

). These

techniques can be considered as a virtual version of

chromoendoscopy.

Even if they are limited to a few bands, these

techniques show the potential of multispectral

imaging for gastroendoscopy. A multispectral image

is formed by monoband images taken at different

wavelength. This technique has an important

advantage since it provides spatial and spectral

information. This situation leads to use image and

VISIGRAPP2014-DoctoralConsortium

signal processing algorithms for data treatment.

There are different examples of successful

implementations of this technique oriented to

medical applications. For instance, it has been used

to increase the proportion of anomalies found in

skin, but also to characterize and delimitate lesions

(Tomatis et al., 2005

)

. Recent approaches have

showed that it is possible to retrieve biological

parameters from the tissue under controlled

acquisition conditions (Jolivot et al., 2011).

Multispectral imaging presents some

disadvantages, for instance the amount of memory

required for a single image (width x height x total of

wavelengths). In consequence, the computational

cost increases significantly. Furthermore, the images

are acquired typically sequentially; this can be

problematic in case of non-static scenarios.

After reviewing the actual technologies to detect

pre-cancerous lesions, as well as the advantages and

disadvantages of multispectral imaging, we

introduce in the following section the acquisition

prototype and the algorithms used.

5 METHODOLOGY

In this section we describe the development of the

multispectral prototype, as well as the pre-

processing image techniques implemented.

5.1 Prototype of the Acquisition System

Multispectral imaging acquires information in two

spatial dimensions (width and height) and one

spectral dimension (wavelength). The acquisition is

performed in most of the cases through two

dimensions at the time, creating two options (Simon

et al., 2013); (Grahn and Geladi, 2007).

The first one acquires one spatial and the spectral

dimension; this option demands some kind of

motion to scan the other spatial dimension.

The second one acquires the spatial dimensions

whereas the remaining spectral dimension is

scanned. In practice, this option can take advantage

of a CCD camera to acquire monoband images at

different wavelengths. Therefore, this configuration

is selected over the other due to its compatibility to

use the camera from the current gastroendoscopes.

Physically, there are two main options to scan

the spectral dimension as is presented in figure 2.

The first option is to illuminate the scene with a

series of specific wavelengths. This can be achieved

for instance by using a tunable light source or

filtering the light from a single light source. This last

option is commonly used in practice through a filter

wheel.

The second option is to filter the incoming light

from the scene to the sensor, which can also be

achieved by placing a filter wheel in front of the

camera or by using a multispectral camera.

Figure 2: Techniques to acquire multispectral images. a)

Single wavelength illuminates the scene. b) Filtering the

light arriving to the camera.

For reasons of simplicity and compatibility with

the actual gastroendoscopic systems, we decided to

modify the source of light of the regular

gastroendoscopes. The final configuration is

presented in figure 3, where the source of light is a

Xenon lamp, filtered by a filter wheel. This light is

transmitted through the gastroendoscope (Olympus

Exera II) to illuminate the stomach. Then, the

camera from the gastroendoscope is used to capture

the image which is finally transmitted to a computer

connected to the gastroendoscopic station.

Figure 3: Multispectral acquisition prototype.

The filter wheel includes six filters in the visible

range from 440 to 640 nm with an equidistant

spectral separation. This option was selected over

the others due to the wide range of available filters

that facilitates the customization. Moreover the cost

is reasonable and the light power allows strong input

ranges in comparison with the wavelength generator

light source. The output of this system is a video

sequence at 25fps with a resolution of 640x480

pixels, from which we can extract the multispectral

images.

The video is first deinterlaced using the widely

known algorithm of Yadif (Hegenbart et al., 2013).

DevelopmentofaMultispectralGastroendoscopetoImprovetheDetectionofPrecancerousLesionsinDigestive

Gastroendoscopy

Then, the sequences of monoband images that

formed the multispectral images are extracted from

the video. The speed of the filter wheel is configured

in order to obtain 4 monoband images from each

wavelength. The first and fourth are the transition

between two filters and are discarded. The second

and third images can be used to generate the

multispectral image; in our application, we use the

third image.

The moist environment of the stomach produces

areas of specular reflection in the images, these

regions are easily removed using a threshold.

The acquisition time of a multispectral image is

approximately one second. Because the stomach is

in constant movement, the monoband images

acquired sequentially present a shifting. This shifting

can be reduced increasing the acquisition speed, but

it is limited to the speed of the filter wheel and the

frame rate of the gastroendoscopic camera. In the

following section is addressed this shifting issue.

5.2 Image Registration

This stage is crucial for the image analysis, since it

allows the superposition of the monoband images.

The first step is a pre-processing step in order to

enhance the contrast information. The algorithm of

contrast limited adaptive histogram equalization is

used in each monoband image.

Then, an affine model is used to model the

transformation between consecutive monoband

images. The parameters of the transformation are

computed using the hierarchical motion-based

estimation (Bergen et al., 1992). The transformation

matrix has six degrees of freedom, which covers the

relative small variations caused in the image by the

movement of the endoscope and the tissue during

the acquisition.

Figure 4: Virtual white light image computed from a)

original monoband images and b) registered monoband

images.

In order to minimize the cumulative error, we

visualize the six monoband images as a sequence,

where the center image is used as a common image

for two subsequences.

The fourth monoband image is assumed to be the

center image and functions as reference for the

registration of the other images. Then, to explain the

procedure, we use the smaller subsequence as an

example (images fourth, fifth and sixth); the fifth

monoband image is registered to the reference

image. Then, the sixth image is registered to the fifth

image producing a temporal image; later on, we

apply to this image the transformation before

estimated to register the fifth image with the

reference image. If there where further images, this

procedure can be iteratively repeated until all the

images from the subsequence are registered. Then,

the same procedure is applied to the other

subsequence.

Finally, when all the transformations are known,

they are applied to the original data to produce a

new set of six monoband images. In order to

highlight the advantages of the registration, figure 4

shows two images whose simulate an endoscopic

image acquired under white light. It is clear that the

figure 4b generated from the registered images is

sharper in comparison with 4a.

5.3 Normalization

The fact that the distance and the orientation

between the tissue and the camera are not constant,

has an important impact in the amplitude of the

estimated spectrum as is shown in figure 5a.

We believe that the shape of the spectrum is

more significant than the amplitude to differentiate

precancerous lesions; therefore, a normalization step

is clearly necessary in order to accurately identify

malignancies in gastric tissue. In this sense, the Area

Under the Curve (AUC) is selected as a spectral

normalizer function. Figure 5b presents the spectra

after normalization, where the spectral shape

remains, facilitating its comparison. This

characteristic is highly desired in the input data for

classification algorithms.

The multispectral images acquired with the

prototype and the described treatment produce

promising spectra. These findings lead the path to

the second and third year of PhD in order to propose

methods to identify pre-cancerous lesions at an early

stage.

6 EXPECTED OUTCOME

The three years of PhD thesis are expected to

VISIGRAPP2014-DoctoralConsortium

produce two main results.

The first one is a multispectral acquisition

system for gastroendoscopy. This system is being

designed to be compatible with the actual acquisition

systems used in gastroendoscopy.

The acquired multispectral image during

gastroendoscopy leads to the second outcome, which

is the identification of cancerous lesions at an early

stage. The research is oriented to the proposal and

development of tools and methods oriented to

identify pre-cancerous lesions. These methods are

expected to be robust to noise due to the acquisition

conditions and fast enough, in order to recognize in

real time the tissue, which is more likely to develop

cancer. These algorithms will be a major part of the

work during the second and third year of the PhD

thesis.

Figure 5: Spectrum from healthy tissue, a) original

spectrum, b) normalized spectrum.

REFERENCES

Bashkatov, A. N., Genina, E. A., Kochubey, V. I.,

Gavrilova, A. A., Kapralov, S. V., Grishaev, V. A.,

Tuchin V. V., 2007. Optical properties of human

stomach mucosa in the spectral range from 400 to

2000 nm: Prognosis for gastroenterology. In Medical

Laser Application, 22(2), p95-104.

Bergen, J. R., Anandan, P., Hanna, K. J., Hingorani, R.,

1992. Hierarchical model-based motion estimation. In

ECCV’92, 588, p237-252.

Galeano, J., Jolivot, R., Benezeth, Y., Marzani, F., Emile,

J.-F., Lamarque, D., 2012. Analysis of Multispectral

Images of Excised Colon Tissue Samples Based on

Genetic Algorithms. In int. conf. on Signal Image

Technology & Internet Based Systems (SITIS), 25-29

Nov, Naples, Italy, pp. 833-838.

Grahn, H. & Geladi, P., 2007. Techniques and

Applications of Hyperspectral Image Analysis, West

Sussex: Wiley. P1-13.

Jolivot, R., Vabres, P., Marzani, F., 2011. Reconstruction

of hyperspectral cutaneous data from an artificial

neural network-based multispectral imaging system. In

Computerized Medical Imaging and Graphics, 35(2),

p85-88.

Hegenbart, S., Uhl, A., Wimmer, G., Vecsei, A., 2013. On

the effects of de-interlacing on the classification

accuracy of interlaced endoscopic videos with

indication for celiac disease. In Computer-Based

Medical Systems (CBMS), 2013 IEEE 26th

International Symposium, 20-22 June, Porto, Portugal,

pp. 137-142.

Kida M., Kobayashi K., Saigenji K., 2003. Routine

chromoendoscopy for gastrointestinal diseases:

indications revised. In Endoscopy, 35(7), p590-596.

Kiyotoki, S., Nishikawa, J., Okamoto, T., Hamabe, K.,

Saito, M., Goto, A., Fujita, Y., Hamamoto, Y.,

Takeuchi, Y., Satori, S., Sakaida, I., 2013. New

method for detection of gastric cancer by

hyperspectral imaging: a pilot study. In Journal of

Biomedical Optics, 18(2), p26010.

Simon C., Mansouric, A., Marzani, F., Booch, F., 2013.

Integration of 3D and multispectral data for cultural

heritage applications: survey and perspectives. In

Image and Vision Computing, 31(1), p91-102.

Tomatis, S., Carrara, M., Bono A., 2005. Automated

melanoma detection with a novel multispectral

imaging system: results of a prospective study. In

Physics in Medicine and Biology, 50(8), p1675-1687.

Wong Kee Song, L. M., Adler, D. G., Conway J. D.,

Diehl, D. L., Farraye, F. A., Kantsevoy, S.V., Kwon,

R., Mamula, P., Rodriguez, B., Shah, R. J., Tierney,

W. M., 2008. Narrow band imaging and multiband

imaging. In Gastrointestinal Endoscopy, 67(4), p581-

589.

DevelopmentofaMultispectralGastroendoscopetoImprovetheDetectionofPrecancerousLesionsinDigestive

Gastroendoscopy