UAV Technology Integration for Remote Sensing Image Analysis
Marco Piragnolo and Francesco Pirotti
CIRGEO, Interdepartmental Research Center of Geomatics, University of Padua, Viale dell'Università,
16 35020 Legnaro, Italy
1 RESEARCH PROBLEM
In this paper, we focus on a multilevel remote
sensing framework to integrate information obtained
through UAV images, satellite images from Sentinel
I and II, radiometric analysis, and spatial
information in order to derive informative maps to
be used for educated decision support. The goal is
the answer to the following questions:
1. Could the classic techniques of remote
sensing be used to extract suitable land use
maps – suitable in terms of classification
accuracy – also for the very high resolution
UAV images?
2. Which methods are optimal to analyze UAV
images and which benefits could be achieved
through the use of more sophisticated
techniques, such as the integration of multi-
source spatial data to add to the feature
vector?
3. Could information from UAV images be
merged with data from satellite images in the
same area, in order to achieve better results?
2 OUTLINE OF OBJECTIVES
Based on the above research questions, the specific
objectives of this project are:
1. To test standard classification methods of
remote sensing to UAV multispectral images.
2. To integrate spatial and morphological
information of objects to the machine
learning methods applied for classification
3. To test advanced classifiers to UAV only and
to UAV and satellite data integrated together.
3 STATE OF THE ART
In the last years there was a growing demand for
innovative tools to monitor geomorphological
aspects for environmental analyses, land use,
fragmentation of habitats and risk assessment
(Piragnolo et al., 2014a; Piragnolo et al., 2014b; Van
Asselen et al., 2013) in particular in rural areas
which, in many cases, have proved to be of strategic
importance to national and regional economy
(Marsden, 2010; van Eupen et al., 2012).
Recently, unmanned aircraft vehicles (UAVs)
have seen great attention from the scientific
community. There are many aspects regarding this
attention, the main one is the prospect to get high-
resolution data “on demand” quickly at a relatively
low cost. The technology in terms of cost and
availability follows the typical development curve:
the prices and weight of the components have
decreased, data accuracy has increased, and all with
a lower power demand, or a constant power and
greater durability of the apparatus as a whole. The
market has come at a point where the cost for the
apparatus, with RGB or multispectral sensors,
becomes accessible to amateur users and to a large
audience. Research fields are cultural heritage,
archaeology, 3D survey, environmental, forestry and
precision agriculture (Berni, 2009; Haarbrink and
Koers, 2006; Herwitz, 2004; Hunt, 2010; Lelong,
2008; Remondino et al., 2011).
Software for image processing is playing a key
role in the diffusion of UAV technology. Since the
accuracy of the positioning systems and orientation
is not comparable to the classical systems of aerial
photogrammetry, software would compensate this
limit with a massive use of image matching and
structure from motion (SfM) techniques. These
techniques, coupled with computer vision
algorithms, have led to the development of various
software for photogrammetric processing available
with commercial licenses and Open Source licenses
(Remondino, 2012). Several authors (Grenzdörffer
et al., 2008; Sona et al., 2014) have reviewed these
new technology and they have reported some
problems in photogrammetric, radiometric aspects
and data size:
1. Photogrammetric problems concern the
limited size and quality of the sensor in the
12
Piragnolo, M. and Pirotti, F.
UAV Technology Integration for Remote Sensing Image Analysis.
In Doctoral Consortium (DCGISTAM 2016), pages 12-19
camera mounted on the UAV; i.e. missing
information regarding the internal orientation,
distortion of frames, overlapping of frames,
low precision of GPS-INS, high number of
ground control point (GCP) required.
2. Radiometric problems are related to image
interpretation, correct use of radiometric
information, new techniques for the
processing of Multispectral Data and
calculation of derived index (Honkavaara et
al., 2012, Torres-Sanchez et al., 2014).
3. Sensors with high spatial and temporal
resolution produce massive data size which
increases exponentially (Zaslavsky, 2013).
Data size and processing time can be related
to the Big Data paradigm: Big Data not only
relates to physical storage, but also to the
velocity of acquisition and variability of
number of files, tables, records and
processing time (Singh, 2012).
Photogrammetric techniques will be used to
obtain the basic data. The evaluation and
improvement of the accuracy of the
photogrammetric survey will be studied marginally
as it has to be taken into account to provide the
spatial error budget. In literature, many authors have
proposed new frameworks, GIS environments and
objects algorithms in order to solve problems of size
and scalability of dataset (Baumann, 2014; Lin et al.,
2013; Peña et al., 2013; Zhao and He, 2009).
Radiometric analyses for segmentation and
classification for GIS environment are the issues that
will be considered in this study.
4 METHODOLOGY
The issues that will be considered are related to
analysis in GIS environment thus with full spatial
support like image interpretation, spectral
information, the calculation of derived indices and
the integration of other spatial data (data fusion).
UAV data will be collected in test areas where
ground information is acquired from experts
assigning agricultural classes depending on crop
type and yield. These data will be analysed in order
to understand whether the classic techniques of
remote sensing could be applied - i.e. minimum
distance, maximum likelihood algorithms (Richards,
2006) and spectral angle mapping SAM (Kruse,
1993) – to correctly return the class of the area.
Whether new techniques are necessary and which
benefits could be achieved through the use of more
advanced techniques, such as the integration of
spatial data to increase the number of features
describing significantly the phenomena, which we
want to model. The integration of information
obtained through photogrammetric methods and
remote sensing, such as Sentinel-2 data, might
improve the quality of derived products such as land
use maps. The accuracies of the classification
methods will be evaluated by weighing both the
feature information from the reflectance from the
spectral bands (optical information), and the
information on the spatial proximity between classes
or morphological information of the objects; spatial
and morphological information is the third
dimension obtained by photogrammetric technique
(Dalponte et al., 2008). A first example of feature
vector with elements that will be tested is [b1, b2,
b3, b4, b5, H, P] where bx are the bands of
wavelength increasing from blue to near infrared, H
refers to height from the ground, and P refers to
slope. Standard classifiers and sophisticated
classifiers such as support vector machines (SVM)
(Melgani and Bruzzone, 2004) and Random Forest
(Brieman 2001) will be tested.
Considering the continuous use of multiband
UAV digital images, it is necessary to structure data
and to apply a harmonious management. It is
important to manage the "raw" data, and information
obtained from the various stages of the processing,
to define the standard products; these data must be
kept for further analysis.
5 EXPECTED OUTCOME
5.1 Multilevel Remote Sensing
Framework
The expected outcome is to set a procedure for
classification and relative algorithms for integrating
satellite and UAV data with other spatial
information. The best algorithms in term of
performance could be integrated in a multilevel
remote sensing framework. The framework could
integrate the information obtained through
photogrammetric methods and remote sensing
techniques (Figure 1). A first classification at
smaller scale will be executed on satellite images.
Classification results and accuracies will be
evaluated using a control dataset which consists of
an independent classification. In case of errors a
deeper analysis at larger scale will be necessary, e.g.
using aerial or drones orthopotos.
UAV Technology Integration for Remote Sensing Image Analysis
13
Figure 1: Multilevel framework.
Figure 2: Classification map of land use produced by
random forest algorithm.
In Figure 2 we present an initial classification of
a test area. It is located at south-east of city of
Padova, in Italian Veneto Region. The classification
is based on Sentinel II images using random
Random Forest algorithm.
Figure 3 shows the UAV image of the test area
flown with a drone. The overlap shows a
disagreement between Urban class of classification
(red pixels) and crops that can be recognized in
UAV orthomosaic.
Figure 3: Testing area was flew by drone.
The final classification will be cross validated
using a ground-truth dataset acquired by a team of
professionals working in the field of land-use maps.
The expected outcome is a robust procedure to
integrate UAV and satellite data to support decision
procedures mainly, but not limited to, the field of
agricultural crop administration.
DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management
14
5.2 UAV Fly Test
Testing area is located in Legnaro inside Campus of
Agripolis of University of Padova, at south-east of
city of Padova, in Italian Veneto Region. It measures
242 meters width, 508 meters height extension, and
area is twelve hectares. It was chosen because it
contains heterogeneous crops, not flat
geomorphology, and ground truth is well known. In
November 2015 eighteen ground control points
(GCP) were put in the area, and the coordinates were
collected with GPS in Real Time Kinematic. The
root mean square error of measures is between 0.008
and 0.011 centimetres. Then the area was flown by
Agency of Veneto Region for payment in
Agriculture (AVEPA), with eBee UAV, Figure 4.
Figure 4: Position of the GCP in the testing area.
Ebee UAV was equipped with three Sensefly
cameras, Red Green Blue (RGB), Near Infrared
(NIR) and multispectral. RGB camera model was
WX. NIR camera model was S110 NIR with three
bands, green with central wavelength at 550 nm, red
with central wavelength at 625 nm, near infrared
with central wavelength at 850 nm. Multispectral
camera model was multiSPEC 4C with four bands,
green with central wavelength at 550 nm, red with
central wavelength at 660 nm, Red edge with central
wavelength at 735 nm, near infrared with central
wavelength at 790 nm. RGB and NIR camera images
had pixel size of 4.5 centimeters. Multispectral camera
images had pixel size of 18 centimeters. All images
were processed with photogrammetric software Agisoft
Photoscan, and then orthorectified. The error calculated
by Photoscan is 0.396 pixel (Table 1). Single band
orthomosaic were exported as GeoTIFF file.
Table 1: GCP errors calculated with Photoscan.
GCP XY
error
(m)
Z
error
(m)
Error
(m)
Proj. Error
(pix)
1 0.0198 0.0002 0.0198 86 0.3340
2 0.0291 -0.0089 0.0304 83 0.3630
3 0.0286 0.0074 0.0295 75 0.3180
4 0.0260 -0.0109 0.0281 92 0.4170
5 0.0156 0.0233 0.0280 106 0.3440
6 0.0331 -0.0307 0.0452 102 0.3180
7 0.0498 -0.0051 0.0500 91 0.4220
8 0.0237 -0.0394 0.0460 109 0.3430
9 0.0193 -0.0069 0.0205 91 0.5160
10 0.0324 0.0783 0.0848 81 0.3980
11 0.0115 -0.0082 0.0141 85 0.4350
12 0.0316 0.0116 0.0336 88 0.3780
13 0.0111 -0.0116 0.0160 116 0.4260
14 0.0480 0.0392 0.0620 84 0.3470
15 0.0267 -0.0550 0.0611 100 0.4100
16 0.0562 0.0613 0.0832 89 0.4590
17 0.0467 -0.0005 0.0467 78 0.3540
18 0.0300 -0.0135 0.0329 45 0.5410
Tot 0.0198 0.0002 0.0456 0.3960
5.3 Classification
In the previous step we have orthorectified nine
bands. Then we have selected seven bands in order to
have continuous spectrum coverage without overlaps
(Table 2), and we uploaded the images in QGis.
Table 2: Bands selected for the classification test.
Band Camera Wavelength nm
Blue RGB 450
Green multiSPEC 4C 550
Red NIR 625
Red multiSPEC 4C 660
Red
Edge
multiSPEC 4C 735
Nir multiSPEC 4C 790
Nir NIR 850
UAV Technology Integration for Remote Sensing Image Analysis
15
We used Semi-Automatic Classification Plugin
Version 4.9. To test two algorithms, Minimum
Distance and Maximum Likelihood, we chose four
classes that are, 1 - urban, 2 - ploughed land, 3-
crops and 4- vegetation, and we identified regions of
interest (ROI) using the specific tool. Minimum
Distance classification is shown in Figure 5.
Figure 5: Classification with Minimum Distance
algorithm.
In order to asses the classification accuracy a
comparison ROI was created and it was used to
calculate error matrix (Table 3) and Kappa index.
Kappa index for Minimum Distance classfication is
0.64. Then we applied the same procedure for
Maximum Likelihood algorithm. Figure 6 shows the
classification map, and Table 4 shows error matrix.
Kappa index for Maximum likelihood is 0.92.
Figure 6: Classification with Maximum likelihood
algorithm.
DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management
16
Table 3: Error matrix for Minimum Distance
classification.
Reference
Class 1 2 3 4 Tot.
1 32718 4313 0 479 37510
2 10779 389257 2276 0 402312
3 877 53722 32506 29239 116344
4 0 0 6793 50438 57231
Tot. 44374 447292 41575 80156 613397
Table 4: Error matrix for Maximum likelihood
classification.
Reference
Class 1 2 3 4 Tot.
1 42746 306 0 0 43052
2 1438 442206 6342 0 449986
3 0 4610 27994 842 33446
4 190 170 7239 79314 86913
Tot. 44374 447292 41575 80156 613397
5.4 Conclusion
This work is preliminary analysis to explore the
potentiality of Satellite images coupled with UAV
images. We have defined a procedure for integrating
satellite and UAV data, and we have tested two
classic remote sensing algorithms, Minimum
distance and Maximum likelihood with UAV data.
Images were collected with eBee drone, using with
different sensors. Then they were orthorectified and
classified in four classes, urban, ploughed land,
crops and vegetation. The accuracy of classification
was estimated with K index. Maximum Likelihood
got 0.91, while Minimum Distance got 0.64. In
literature Maximum Likelihood algorithm is one of
the most popular classifiers used in remote sensing
from satellite. In this preliminary test with images
from drone, Maximum Likelihood algorithm gives
better result than Minimum Distance classifier. In
Figure 7 we can see two comparisons between the
algorithms and ground truth. On left images,
Minimum Distance algorithm classifies trees as
buildings, while Maximum Likelihood assigns trees
to vegetation class. On right images Minimum
Distance Algorithm produces confused
classification. Maximum Likelihood is more precise,
but it mixes crops and vegetation.
Figure 7: Comparison between classifications obtained
two Minimum Distance and Maximum Likelihood
algorithms.
6 STAGE OF THE RESEARCH
At the moment the research is at initial phase as the
project started a few months ago. In this contribution
we want to present the research question and the
methods which will be tested in the project.
REFERENCES
Baumann, P., 2014. Spatio-Temporal Big Data the
rasdaman approach, p.32. Available at:
UAV Technology Integration for Remote Sensing Image Analysis
17
http://2014.ogrs-community.org/2014_workshops/Ras
daman/BigDataRasdamanWorkshop.pdf.
Berni, J., Zarco-Tejada, P. J., Suarez, L., Fereres, E., 2009.
Thermal and narrowband multispectral remote sensing
for vegetation monitoring from an unmanned aerial
vehicle. IEEE Transactions on Geoscience and
Remote Sensing, 47, pp.722–738.
Breiman, L., 2001. Random forests. Machine Learning,
45, 5–32. doi:10.1023/A:1010933404324.
Coppa, U., Guarnieri, A., Pirotti, F., and Vettore, A., 2009.
Accuracy enhancement of unmanned helicopter
positioning with low-cost system. Applied Geomatics,
1(3), pp.85–95.
Dalponte, M., Bruzzone, L., Gianelle, D., Member, S.,
2008. Fusion of Hyperspectral and LIDAR Remote
Sensing Data for Classification of Complex Forest
Areas. IEEE Transactions on Geoscience and Remote
Sensing, 46(5), pp.1416–1427.
Grenzdörffer, G., Engel, A., Teichert, B., 2008. The
photogrammetric potential of low-cost UAVs in
forestry and agriculture. International Archives of
Photogrammetry Remote Sensing and Spatial
Information Sciences, 1, pp.1207–1213. Available at:
http://www.isprs.org/proceedings/XXXVII/congress/1
_pdf/206.pdf.
Haarbrink, R., Eisenbeiss, H., 2008. Accurate DSM
production from unmanned helicopter systems.
International Archives of Photogrammetry Remote
Sensing and Spatial Information Sciences, Vol.
XXXVI, pp.1259–1264. Available at: http://www.isp
rs.org/proceedings/XXXVII/congress/1_pdf/214.pdf.
Herwitz, S.R., Johnson, L.F., Dunagan, S.E., Higgins,
R.G., Sullivan, D. V., Zheng, J., Lobitz, B.M., Leung,
J.G., Gallmeyer, B. a., Aoyagi, M., Slye, R.E., Brass,
J. a., 2004. Imaging from an unmanned aerial vehicle:
Agricultural surveillance and decision support.
Computers and Electronics in Agriculture, 44(1),
pp.49–61.
Honkavaara, E., Kaivosoja, J., Mäkynen, J., Pellikka, I.,
Pesonen, L., Saari, H., Salo, H., Hakala, T., Marklelin,
L., Rosnell, T., 2012. Hyperspectral Reflectance
Signatures and Point Clouds for Precision Agriculture
By Light Weight UAV Imaging System. ISPRS
Annals of Photogrammetry, Remote Sensing and
Spatial Information Sciences, I-7(September), pp.353–
358.
Hunt, E.R., Dean Hively, W., Fujikawa, S.J., Linden, D.S.,
Daughtry, C.S.T., McCarty, G.W., 2010. Acquisition
of NIR-green-blue digital photographs from unmanned
aircraft for crop monitoring. Remote Sensing, 2(1),
pp.290–305.
Kruse, F. A., Lefkoff, A.B., Boardman, K.B. Shapiro,
A.T., Barloon, P.J., Goetz, A.F.H., 1993. The Spectral
Image Processing System (SIPS) - Interactive
Visualization and Analysis of Imaging Spectrometer
Data. Remote Sensing of Environment, 44, pp.145–
163. doi:10.1016/0034-4257(93)90013-N.
Lelong, C.C.D., Burger, P., Jubelin, G., Roux, B., Labbé,
S., Baret, F., 2008. Assessment of unmanned aerial
vehicles imagery for quantitative monitoring of wheat
crop in small plots. Sensors, 8(5), pp.3557–3585.
Lin, F.-C., Chung, L.-K., Ku, W.-Y., Chu, L.-R., Chou,
T.-Y., 2013. The Framework of Cloud Computing
Platform for Massive Remote Sensing Images.
Advanced Information Networking and Applications
(AINA), 2013 IEEE 27th International Conference on
,
pp.621–628. Available at: http://ieeexplore.ieee.org/xp
ls/icp.jsp?arnumber=6531812.
Marsden, T., 2010. Mobilizing the regional eco-economy:
evolving webs of agri-food and rural development in
the UK. Cambridge Journal of Regions, Economy and
Society, 3 (2 ), pp.225–244. Available at: http://cjres.
oxfordjournals.org/content/3/2/225.abstract.
Melgani, F. and Bruzzone, L., 2004. Classification of
hyperspectral remote sensing images with support
vector machines. Geoscience and Remote Sensing,
IEEE Transactions on, 42(8), pp.1778–1790.
Peña, J.M., Torres-Sánchez, J., de Castro, A.I., Kelly, M.,
López-Granados, F., 2013. Weed Mapping in Early-
Season Maize Fields Using Object-Based Analysis of
Unmanned Aerial Vehicle (UAV) Images. PLoS ONE,
8(10), pp.1–11.
Piragnolo, M., Pirotti, F., Vettore, A., Salogni, G., 2014a.
ANTHROPIC RISK ASSESSMENT ON
BIODIVERSITY. ISPRS - International Archives of
the Photogrammetry, Remote Sensing and Spatial
Information Sciences, XL-5/W3, pp.21–26.
doi:10.5194/isprsarchives-XL-5-W3-21-2013.
Piragnolo, M., Pirotti, F., Guarnieri, A., Vettore, A.,
Salogni, G., 2014b. Geo-Spatial Support for
Assessment of Anthropic Impact on Biodiversity.
ISPRS International Journal of Geo-Information, 3,
pp.599–618. doi:10.3390/ijgi3020599.
Remondino, F., Barazzetti, L., Nex, F., Scaioni, M.,
Sarazzi, D., 2011. UAV photgrammetry for mapping
and 3D modeling current status and future
perspectives. International Archives of the
Photogrammetry, Remote Sensing and Spatial
Information Sciences, XXXVIII (September), pp.14–
16.
Remondino, F., Pizzo, S. Del, 2012. Low-cost and open-
source solutions for automated image orientation–a
critical overview. Progress in Cultural Heritage
Preservation, pp.40–54. Available at: http://link.spring
er.com/chapter/10.1007/978-3-642-34234-9_5.
Richards, J. A., Jia, X., 2006. Remote Sensing Digital
Image Analysis: An Introduction. Berlin, Germany:
Springer.
Singh, S., Singh, N., 2012. Big Data analytics. 2012
International Conference on Communication,
Information and Computing Technology (ICCICT),
pp.1–4. Available at: http://ieeexplore.ieee.org/lpdocs
/epic03/wrapper.htm?arnumber=6398180.
Sona, G., Pinto, L., Pagliari, D., Passoni, D., Gini, R.,
2014. Experimental analysis of different software
packages for orientation and digital surface modelling
from UAV images. Earth Science Informatics, 7(2),
pp.97–107.
DCGISTAM 2016 - Doctoral Consortium on Geographical Information Systems Theory, Applications and Management
18
Torres-Sánchez, J., Peña, J.M., de Castro, a. I., López-
Granados, F., 2014. Multi-temporal mapping of the
vegetation fraction in early-season wheat fields using
images from UAV. Computers and Electronics in
Agriculture, 103, pp.104–113.
Van Asselen, Sanneke, and Peter H. Verburg. 2013. Land
Cover Change or Land-Use Intensification: Simulating
Land System Change with a Global-Scale Land
Change Model. Global Change Biology, 19, pp.3648–
3667. doi:10.1111/gcb.12331.
Van Eupen, M., Metzger, M.J., Pérez-Soba, M., Verburg,
P.H., van Doorn, a., Bunce, R.G.H., 2012. A rural
typology for strategic European policies. Land Use
Policy, 29(3), pp.473–482. Available at:
http://dx.doi.org/10.1016/j.landusepol.2011.07.007.
Zaslavsky, A., Perera, C., Georgakopoulos, D., 2013.
Sensing as a Service and Big Data, in: arXiv Preprint
arXiv:1301.0159. Bangalore.
Zhao, W., Ma, H. and He, Q., 2009. Parallel K-Means
Clustering Based on Map Reduce. Cloud Computing,
5931, pp.674–679. Available at: http://link.springe
r.com/chapter/10.1007/978-3-642-10665-1_71.
UAV Technology Integration for Remote Sensing Image Analysis
19