Mobile-based Risk Assessment of Diabetic Retinopathy using a
Smartphone and Adapted Ophtalmoscope
Sim
˜
ao Felgueiras, Jo
˜
ao Costa, Jo
˜
ao Gonc¸alves and Filipe Soares
Fraunhofer Portugal AICOS, Rua Alfredo Allen 455/461, 4200-135 Porto, Portugal
Keywords:
Diabetic Retinopathy, Retinal Image Acquisition, Automated Detection, Exudates, Microaneuryms, Decision
Support System.
Abstract:
The large prevalence of diabetes in the global population is associated with an increasing number of Diabetic
Retinopathy cases. This disease is associated with a progressive risk of blindness, due to physiological changes
that affect the retina. Since most of the progression is asymptomatic and late stage damage is often irreversible,
there is a large incentive to implement effective methodologies that allow large scale screening of the diabetic
population. In this work, a research study of a mobile approach for the assessment of Diabetic Retinopathy
was conducted, by analyzing 80 patients already being followed for ophthalmological care. A smartphone-
based fundus imaging system was used to acquire images of the retina during the normal clinical workflow in
a Central Hospital in Portugal. Relevant images were automatically analyzed by a Decision Support System
(DSS) based on computer vision methods. The results were obtained for ground-truth correlation as well
as time impact of this novel system. Our conclusions support that the DSS is highly sensitive in detecting
pathological information on images, after a dedicated quality image filtering, and the acquisition procedure
has minimal adverse impact in the clinical setting.
1 INTRODUCTION
The increasing prevalence of diabetes in the popula-
tion is associated with several health issues, one of
which is the development of Diabetic Retinopathy.
This disease causes several cumulative lesions to the
patient retina through microvascular changes and, if
left untreated, may lead to blindness.
Diabetic retinopathy affects a large proportion of
the diabetic population: around 40% of all the people
with type 2 diabetes in the United States suffer from
some stage of the disease (Cheung et al., 2010), and a
particularly problematic aspect is that its progression
is mostly asymptomatic until the later stages, often
already associated with some degree of vision loss.
Even though effective treatment is possible in
early stages of the disease, vision loss is often irre-
versible, motivating the need for implementation of
screening programs. In these screening programs, im-
ages of the patient retinas are acquired by trained per-
sonnel and analyzed by experts in opthalmology, but
this requires the use of expensive equipment to per-
form retinal image acquisition as well as the manual
time-consuming analysis of those images.
In order to tackle the limitations associated with
traditional retinal imaging equipment, several mobile
solutions have been proposed. Volk inView is a re-
cent development which allows low cost acquisition
of retinal images with iPhone devices, but it is not
suitable for non-mydriatic acquisition, thus requiring
the administration of pharmacological agents for en-
abling mydriasis.
More recently, a solution for telemedicine screen-
ing was developed to detect retina diseases with a
portable non-mydriatic fundus camera. However, it
only performs image acquisition and the camera is
incorporated into the device. By having an embed-
ded camera, these kind of approaches do not take ad-
vantage of the widely available and expansible smart-
phones with increasingly enhanced cameras (Jin et al.,
2017). Besides reducing the cost for retinal image
acquisition, the use of smartphone exploits the con-
siderable computing power of these devices to per-
form automatic analysis of the acquired retinal im-
ages. Other solutions are the Volk Pictor and Pictor
Plus cameras, which allow non-mydriatic acquisition
but at very high costs (Zhang et al., 2015).
In a previous research work by (Costa et al.,
2016), it was proposed a methodology for mobile-
based computer-aided detection of Diabetic Retinopa-
168
Felgueiras, S., Costa, J., Gonçalves, J. and Soares, F.
Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope.
DOI: 10.5220/0006599701680175
In Proceedings of the 11th International Joint Conference on Biomedical Engineering Systems and Technologies (BIOSTEC 2018) - Volume 5: HEALTHINF, pages 168-175
ISBN: 978-989-758-281-3
Copyright © 2018 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
thy signs in retinal images. This approach was tested
using images from tabletop fundus cameras while
achieving a sensitivity of 87% in the detection of Di-
abetic Retinopathy, running in smartphones with rel-
atively low computational requirements and fast pro-
cessing.
In this work, we introduce a smartphone compat-
ible system for acquisition of retinal images which,
by building upon the work of (Costa et al., 2016),
allows an end-to-end low cost solution for Diabetic
Retinopathy screening and detection, without requir-
ing pupil dilation. The main goal is to evaluate auto-
matically the quality of the acquired images obtained
during the normal clinical workflow and, at the same
time, measure the performance of a Decision Support
System (DSS) on the previously filtered images.
2 METHODOLOGY
2.1 Retinal Image Acquisition Device
In order to acquire retinal images without using cum-
bersome equipment, a mobile prototype was devel-
oped, based in the commercial Welch Allyn PanOp-
tic ophthalmoscope, which provides a relatively large
Field of View of 25
. The ophthalmoscope enables
easy access to the patient retina without requiring
pupil dilation, which is of major importance in a
screening setting, even if not necessarily so in spe-
cialized ophthalmological care.
Welch Allyn commercializes a mechanical
adapter for the iPhone 4/4s smartphone, but the An-
droid ecosystem was preferred for our work, due to
the higher flexibility offered. As such, we developed
a 3D printed mechanical case (see Figure 1) for the
Samsung S6 smartphone, which aligns the optical
axis of the ophthalmoscope with the smartphone
camera.
The unmodified ophthalmoscope uses a standard
halogen lamp for illumination of the retina, whose in-
tensity is manually adjusted through a rheostat. Since
the retina has low light reflectance, it is generally de-
sirable to use a relatively high light intensity in or-
der to obtain images with good quality. However, this
makes the examination process extremely uncomfort-
able for the patient, since he needs to withstand this
light for one or two minutes while the operator aligns
and acquires the retinal images. This also hinders the
operator, since alignment is more difficult due to the
inevitable blinking of the patient and pupil constric-
tion (or myosis). Our preliminary experiments re-
vealed that it was very difficult to achieve a satisfy-
ing trade-off between image quality and patient com-
fort/operator ease of use and so an alternative was re-
searched.
The alignment of the ophthalmoscope with the pa-
tient retina is crucial for obtaining meaningful images
for analysis, but image quality is not a priority for this
task. As such, it is possible to improve patient com-
fort by using a very low light intensity, offsetted by
using a high camera sensor sensitivity (ISO). Since
image acquisition is very brief, taking less than 50
milliseconds, we can minimize exposure of the pa-
tient to the high light intensity (see Figure 2 and 3).
However, the process of using different light in-
tensities for alignment and acquisition is not feasible
with the manual light intensity adjustment through the
rheostat. As such, we developed a Custom Light Con-
troller hardware module, directly powered and con-
trolled by the smartphone through USB On-The-Go
(OTG). The Custom Light Controller adjusts the light
output of a white high powered LED, at the command
of a Android smartphone application. This applica-
tion manages the process of acquiring the retinal im-
ages, by accommodating several camera parameters
such as focus, sensor exposure time, sensitivity, etc.
The applications sets the light output extremely low
while the user is aligning the ophthalmoscope, and,
when the user presses a physical button associated
with the Custom Light Controller, an order to acquire
an image is triggered, and the application synchro-
nizes the camera shutter with a high intensity light
pulse, similar to a flash.
Figure 1: Assembled EyeFundusScope prototype.
Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope
169
Figure 2: Example of an EyeFundusScope acquisition pro-
cedure.
Figure 3: Example of an acquired image.
2.2 Quality Image Algorithm
The quality of the acquired retinal image is the main
focus of mobile image acquisition systems (Jin et al.,
2017) for ocular diseases. Therefore, it is important
to verify if the EyeFundusScope prototype acquires
an image with enough quality and details about the
presence of Diabetic Retinopathy. In order to evalu-
ate the images collected in this study, a Quality Image
Algorithm was implemented with the goal of select-
ing images with good quality for further processing,
as shown in Figure 4. This algorithm is not only based
on image luminosity and color, but also in entropy and
most relevantly in the tissue area. If the area is higher
than a predefined minimum value, the current image
is further analyzed by the next stages of quality eval-
uation. The minimum area value is an absolute value
and it was calculated using the dimensions of the full
image capture. To determine the tissue area, the algo-
rithm calculates the median filter of the image using
the red channel and applies a region threshold on the
median filtered image. All these transformations are
done with downsized images, with the objective of re-
ducing computational costs.
Figure 4: Images with good quality, according to the Qual-
ity Image Algorithm, despite minor reflections.
2.3 Panoramic Image
To increase the field-of-view obtained by the pro-
posed acquisition system using the commercial oph-
thalmoscope, which can be a problem for clinical
use when compared with the gold standard of table-
top fundus cameras, the EyeFundusScope software
application performs stitching of the acquired im-
ages. There are two main steps when generating a
panoramic image, one of which is to extract the blood
vessels from the green channel and use them for im-
age registration. The other step aims to create the mo-
HEALTHINF 2018 - 11th International Conference on Health Informatics
170
saic image with the original images and the registra-
tion calculated with the blood vessels. This allows the
stitching of various images from different quadrants
of the eye to get a greater field-of-view. The objective
is to provide to the prototype operator the most com-
plete and easy to use information from the acquisition
that is performed, as shown in Figure 5.
Figure 5: Example of a stitched image, as a result of merg-
ing two overlapping retinal images.
2.4 Decision-Support System
After the first phases of image acquisition and qual-
ity filtering, the Decision-Support System is exe-
cuted, aiming at the separation between pathology-
free and positive cases. When Diabetic Retinopa-
thy is detected earlier, the success of the treatment
will increase. Therefore, in early stages, an auto-
mated computer-aided tool may increase the success
of the diagnosis, particularly in remote scenarios. The
Decision-Support System receives as input the re-
sults from microaneurysms detection and from the
decision-tree that classifies the exudates, based on
the approaches developed in (Costa et al., 2016) and
(Felgueiras et al., 2016). Having this information, the
DSS outputs if the image is pathology-free or not.
2.4.1 Microaneurysms Detection
Microaneurysms formation is the first evident (and
visible) sign of Diabetic Retinopathy. Histologically,
microaneurysms present a loss of capillary cells (the
perycites) on its walls. This leads to acellular cap-
illaries, thus enabling the pouching of these capillar-
ies, which originate the microaneurysms. The mech-
anisms that causes the decellularization are not well
understood, but they include release of a vasoprolifer-
ative factor and an increase in capillary pressure (Adal
et al., 2014).
Automated microaneurysm detection on the ac-
quired retinal images follows a similar approach to
(Costa et al., 2016). This approach uses the character-
istics of microaneurysms, such as small size and low
grayscale intensity to extract them from the rest of
the image. An example of detected microaneurysms
is shown in Figure 6.
Template matching using an inverted 2D Gaussian
kernel is applied to the green channel of the image
and the result is thresholded, with each component
in the resulting binary image constituting a detected
microaneurysm. In order to account for variation in
microaneurysm size, several σ values are used for the
Gaussian kernel.
Since retinal vessels are a common source of false
positives, the microaneurysm extraction method also
relies on the segmentation of vessels in the image, in
order to exclude microaneurysms located on those re-
gions. As a first step, the green channel of the origi-
nal image is subtracted to its median filtered version,
which is followed by a top hat transform. The result
of this operation is thresholded, and small connected
components are discarded as noise.
Figure 6: Retinal image with microaneurysms.
2.4.2 Exudate Detection
There are two types of exudates, the soft exudates,
also known as cotton wool spots, and the hard exu-
dates. Soft exudates are accumulations of axioplasm
and hard exudates accumulations of lipid and protein
in the retina (Ravishankar et al., 2009). Their typical
Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope
171
characteristics are bright, reflective, white or cream
colored lesions found in the retina (Ravishankar et al.,
2009). These lesions indicate increased vessel perme-
ability and an associated risk of retinal edema. If they
appear close to the macula center, they are considered
as sight threatening lesions. Exudates are often asso-
ciated with microaneurysms.
EyeFundusScope performs automatic detection
of soft and hard exudates based on the work of
(Felgueiras et al., 2016) comprising image pre-
processing, feature extraction and candidate classifi-
cation.
Figure 7: Retinal image with exudates. Right image high-
lights the area where the exudates are located.
3 STUDY SETUP
The ophthalmology department at Hospital Santo
Ant
´
onio, Centro Hospitalar do Porto, Portugal, hosted
this study with patients in a real scenario. Written in-
formed consents were obtained from the patients as
well as the approval from the institution’s research
ethics committee. The main objective of the study
was the validation of the mobile acquisition in di-
abetic patients without intrusive procedures. In the
sense, the acquisition was planned for the duration of
2 minutes per eye. Some patient data had to be reg-
istered: age, type and duration of diabetes, and pres-
ence of Diabetic Retinopathy according to a previous
diagnosis.
An Android application was developed to allow
the enhanced hardware control of the camera. Ad-
ditionally, the application was responsible for the
anonymization of the data by generating a unique ran-
domized key for each patient. The images acquired
are divided in two groups, images from the left and
the right eye. A form to insert information about the
patient status was part of the acquisition workflow.
To perform image acquisition the room should
comply with some basic guidelines. The luminosity
should be low and a chair for the patient should be
available. The acquisition should also follow a pro-
tocol. The patient should sit on the chair, remove the
glasses if necessary, and look to a fixed point. While
acquiring images the operator will try to obtain im-
ages around the macula region. After performing this
action on both eyes the acquisition is ended.
The acquisitions took place after the ophthalmolo-
gist appointment and the specialist rated the acquired
images in terms of quality and gave a diagnosis about
Diabetic Retinopathy.
4 RESULTS
A total of 80 patients were analyzed, 68 of which
suffer from Diabetic Retinopathy. Diabetes type 2
is the predominant condition, and there is no infor-
mation about 4 patients. The acquired database in-
cludes 16 patients with pharmacologic dilation of the
pupil. Based on clinical expert analysis, no cases
were found with the presence of soft exudates. Only
4 cases do not have been analyzed by the Doctor on
both eyes, due to physical conditions of the patients.
The youngest patient acquired was 43 years old and
the oldest was 85 years old. The longest acquisition
with EyeFundusScope took 9 minutes and the fastest
one 4 minutes, being the average 5 minutes per case.
Photocoagulation treatment was already applied
to 30 patients, 12 of which had visible marks that
are noticeable in the acquired images. Analysing
these acquisitions, 7 exams have hard exudates and
4 present microaneurysms. From these photocoagu-
lation cases, 10 have good quality and no presence of
lesions on the acquired quadrants of the eye.
From the complete dataset, 31 acquisitions pre-
sented bad quality, 23 due to acquisition conditions
and 8 due to wrong prototype configuration. The
database has 13 exams where the acquisition difficul-
ties were originated from various patient conditions.
Problems like cataracts, retinal detachment, strabis-
mus and intermittent eyelids movement have a nega-
tive impact on the acquisition process.
The quality image algorithm evaluated the ac-
quired images from all acquisitions, resulting in a to-
tal of 277 good quality images for classification pur-
poses using the DSS.
As expected, the images considered as having bad
quality by the Quality Image Algorithm were auto-
matically excluded. Figure 8 represents the number
of cases with medical diagnosis for which at least
one image was selected after quality filtering. The
class of Other Problems is related with other eye prob-
lems presented on patient eyes, as already mentioned
above.
HEALTHINF 2018 - 11th International Conference on Health Informatics
172
Exudates
Microaneurysms
Photocoagulation
Other Problems
No Lesion
0
5
10
Number of cases
Figure 8: Medical diagnosis for the patients from which the
images were selected after quality filtering.
4.1 Quality of the Acquisitions
From a specialist perspective, a good image should al-
ways contain retinal tissue with minimal reflections,
vessels, macula and optic disc. However, during the
analysis of the exams, some images were found con-
taining hard exudates and vessels, but without mac-
ula or optic disc. For this reason, the quality met-
ric that provided the best results was the tissue area.
The other group of metrics already mentioned in sec-
tion 2.2, ensures that the acquired image has relevant
structures to be detected, as shown in Figure 9.
Figure 9: Image with vessels and hard exudates only.
4.2 Performance of the Detection
Algorithms
The exudates detection was tested on the selected im-
ages by the quality image algorithm and a classifi-
cation accuracy of 91% was achieved using the DSS
(see Table 1).
Table 1: Exudates classification obtained.
Predicted
Exudates
Predicted
No Exudates
Exudates 31 11
No Exudates 12 223
From the quality image algorithm analysis, 277
images were considered adequate, 235 of which with-
out the presence of lesions. A Sensitivity of 74% and
a Specificity of 95% was achieved by the proposed
system on a per image-based analysis, as shown in
Table 2. Without considering the 16 images acquired
with a dilated pupil (removing mydriatic cases) as part
of the dataset, the Specificity is 95% and the Sen-
sitivity is reduced to 67%. Therefore, it is relevant
to state that the performance of EyeFundusScope is
naturally affected in non-mydriatic cases, due to the
reduced pupil size which complicates pupil and oph-
talmoscope alignment. However, the results show no
variance in the high values of Specifity performance
which is relevant for screening use cases. The sys-
tem has the advantage of acquiring multiple images
and the Quality Image Algorithm works as a filter to
achieve the best images. For this reason, it is feasible
to acquire images with relevant information without
pupil dilation.
Table 2: Classification Performance.
Full
dataset
Non-mydriatic
dataset
Sensitivity 74% 67%
Specificity 95% 95%
Before this study, the EyeFundusScope automated
screening algorithms were tested with Messidor and
E-Ophta databases. Knowing if the analyzed eye is
the left or the right, we can predict if the macula is to
the right or left of the optic disc, since anatomically
it is located temporally. In order to detect the op-
tic disc, EyeFundusScope uses a template matching
algorithm, using a circular shape as a kernel. Since
the macula is usually located at a distance of approx-
imately 2.5 times the diameter of the optic disc, and
knowing the macula direction, it is possible to locate
the macular region. If the acquired image contains
the temporal quadrants of the retina and the optic disc
is visible, the developed methodology is then able to
find the location of the macula, as shown in Figure 10.
Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope
173
Figure 10: The probabilistic risk zone near to the macula.
4.3 Clinical Impact
As found in this study, the adverse impact of inte-
grating the retinal image acquisition with the Eye-
FundusScope prototype is minimal. The appointment
generally last from 10 to 15 minutes per patient and
the EyeFundusScope acquisition has an average du-
ration of 5 minutes, but without requiring specialist
time.
The obtained images have enough information to
provide feedback about Diabetic Retinopathy. How-
ever, sometimes it is complicated to provide feedback
about all quadrants of the eye. Regarding this aspect,
the solution needs to be improved in order to cover all
the retinal regions.
The pupil dilation improves the quality and the
area acquired in each image, but the objective of the
solution is to avoid requiring this procedure. To en-
able a similar result without performing dilation will
allow more flexibility for the retinal image acquisi-
tion, since the patient will have full visual capabilities
after the examination, being able to walk without as-
sistance and even drive on its own.
5 CONCLUSIONS
In this paper, a mobile approach is evaluated for
fundus image acquisition, together with quality and
pathology assessment, by software suitable for run-
ning in smartphone devices. The time taken for
the proposed acquisition procedure in clinical context
was very low, even facing old adults with severe opti-
cal conditions, which fits the use case of a mobile sys-
tem in remote places with operation by non-experts in
opthalmology.
The Quality Image Algorithm works as a filter and
ensures each exam performed has acceptable images,
so that the detection algorithms are able to extract
valuable information.
The classification results has shown that the usage
of EyeFundusScope may contribute for the reduction
of the time-consuming task of image interpretation by
specialists.
This study provided valuable data to improve the
research in the scope of the EyeFundusScope soft-
ware, hardware and trigger new acquisition perspec-
tives and use cases.
In the future, the authors intend to improve the
classification performance by employing state-of-art
Deep Learning approaches to the problem of Dia-
betic Retinopathy risk assessment, considering the
specificities of smartphone-based image acquisition.
Moreover, improving the usability of operation with
minimal training of the health professional is ex-
pected by leveraging real-time quality image assur-
ance to provide feedback to the prototype operator,
improving human-machine interaction.
ACKNOWLEDGEMENTS
We would like to acknowledge the financial support
obtained from North Portugal Regional Operational
Programme (NORTE 2020), Portugal 2020 and the
European Regional Development Fund (ERDF) from
European Union through the project Symbiotic tech-
nology for societal efficiency gains: Deus ex Machina
(DEM), NORTE-01-0145-FEDER-000026.
A special acknowledgement to Doctor Ant
´
onio
Friande from Hospital Santo Ant
´
onio, Centro Hospi-
talar do Porto, Portugal, for providing the access to
patients, anonymised data and diagnosis.
HEALTHINF 2018 - 11th International Conference on Health Informatics
174
REFERENCES
Adal, K. M., Sidib, D., Ali, S., Chaum, E., Karnowski,
T. P., and Mriaudeau, F. (2014). Automated detection
of microaneurysms using scale-adapted blob analysis
and semi-supervised learning. Computer Methods and
Programs in Biomedicine, 114(1):1–10.
Cheung, N., Mitchell, P., and Wong, T. Y. (2010). Diabetic
retinopathy. The Lancet, 376(9735):124 – 136.
Costa, J., Sousa, I., and Soares, F. (2016). Smartphone-
based decision support system for elimination of
pathology-free cases in diabetic retinopathy screen-
ing.
Felgueiras, S., Costa, J., Soares, F., and Monteiro, M. P.
(2016). Cotton wool spots in eye fundus scope.
Jin, K., Lu, H., Su, Z., Cheng, C., Ye, J., and Qian, D.
(2017). Telemedicine screening of retinal diseases
with a handheld portable non-mydriatic fundus cam-
era. BMC Ophthalmology, 17.
Ravishankar, S., Jain, A., and Mittal, A. (2009). Auto-
mated feature extraction for early detection of diabetic
retinopathy in fundus images. In IEEE Conference
on Computer Vision and Pattern Recognition, 2009.
CVPR 2009, pages 210–217.
Zhang, W., Nicholas, P., Schuman, S., Allingham, M.,
Faridi, A., Tushar, S., Cousins, S. W., and Prakala-
pakorn, S. G. (2015). Screening for diabetic retinopa-
thy using the hand-held pictor camera. Investigative
Ophthalmology & Visual Science, 56(7):1426–1426.
Mobile-based Risk Assessment of Diabetic Retinopathy using a Smartphone and Adapted Ophtalmoscope
175