A MOBILE BROWSER FOR GEO-REFERENCED IMAGES

USING AN ACCELEROMETER-BASED COMPASS

Francesco Massidda, Roberto Manca

Geoweb and Mobile Experience Lab, Sardegna Ricerche, 09010 Pula , CA - Italy

Davide Carboni

CRS4, Parco Scientifico e Tecnologico Sardegna Ricerche, 09010 Pula CA - Italy

Keywords: Mobile Photo Browsing, GPS, Compass, Accelerometers, Geo-Browsing, Maps.

Abstract: In this paper a new mobile browser for geo-referenced pictures is introduced. Based on common embedded

GPS and accelerometer sensors, the implemented mobile browser is able to show tagged photos on the web,

depending on the direction the user is facing to, allowing a positional-dependent touristic, commercial or

cultural preview of our cities. A novel compass-simulator developed using build-in accelerometers data

samples represents.

1 INTRODUCTION

One of the most promising usage of user generated

content and metadata is the geo-referencing of

multimedia objects in e-learning, environmental

protection, tourism, culture and other application

fields. Among the others, pictures perfectly meet the

requirement of digital geo-referenced resources

because photos always refer to well known places

and times in/at which they have been taken.

Furthermore, the release of worldwide known

programming tools like the Google Maps API has

fostered the development of a lot of geographical

web applications and mashups (ProgrammableWeb).

Browsing large collections of geo-referenced

photos is an emerging topic in research (Carboni et

al., 2006). Some works focus on automatic

categorization and classification of images (Sarvas

et al., 2004) to improve usability and to provide the

ability to capture images and events without the

worries of adding tags and metadata. In

(Shneiderman et al., 2004) a combination of photos

annotation, browsing and sharing is proposed with

the aim of supporting exploratory search. In World

Explorer (Ahern et al., 2007) an aggregated

representation of a place is performed getting geo-

referenced photos from Flickr and displaying labels

on the map/aerial photo to relate a place with a

concept. The project “Degree Confluence”

represents an attempt to tie images to the space (The

degree confluence project).

Furthermore, even more mobile devices are able

to browse digital image libraries, thanks to their

large displays and internet connection capabilities

(Bluetooth, Wi-Fi and 3G). However, traditional

mobile picture browsers (Fig. 1), tend to use a static

approach that is not suitable for large datasets,

particularly when geo-referencing data are attached.

In this paper, authors present a novel picture

browsing method, based on embedded GPS and

acceleration sensors in order to create a dynamic

browser able to collect, catalogue and view images

from public web sites like panoramio.com,

according to the geographical orientation of users

carrying their mobile phones.

(a) (b) (c)

Figure 1: Mobile picture browsers (a), (b) use static

navigation tools or smarter and dynamic approach (Nokia

N95 mobile phone) (c).

431

Massidda F., Manca R. and Carboni D.

A MOBILE BROWSER FOR GEO-REFERENCED IMAGES USING AN ACCELEROMETER-BASED COMPASS.

DOI: 10.5220/0001840704310434

In Proceedings of the Fifth International Conference on Web Information Systems and Technologies (WEBIST 2009), page

ISBN: 978-989-8111-81-4

2 MOBILE PICTURE BROWSING

An interesting approach to the visualization of geo-

referenced pictures on desktop has been recently

proposed by the authors (Carboni et al., 2008).

furthermore, effective mobile-embedded

technologies, like a digital compass should allow

new scenarios and new interaction modalities for

portable devices. Unfortunately, at the time we are

writing the paper, mobile terminals with a digital

compass embedded are very few: the Nokia 6210

Navigator and the old Nokia 5140i while, mobile

phones with accelerometers sensors are, instead,

very common Accelerometers have been used in the

recent literature in many research and development

fields: in medical applications, in the automation of

the vehicular navigation, for shock protection

systems, for Robots’ guide automation and, finally,

for human assistance in hazarding and dangerous

working environments. In mobile applications the

accelerometers are used to increase or improve the

interaction or to speed up the textual input (Sung-Do

Choi et al., 2006), (Wigdor et al., 2003).

3 THE

ACCELEROMETER-BASED

COMPASS SIMULATOR

To provide a view of surrounding points of interest

based both on heading and position, the proposed

system must be connected with the GPS system

which main objective is to provide position and not

to provide heading. Nevertheless, if a GPS sensor is

moving fast enough, it is possible to sample two

positions after a short time interval to determine the

heading. Unfortunately this approach is not viable

for slow pedestrian mobility. Using acceleration

sensors to determine heading is a theoretical

solution: the double integral of acceleration gives the

space associated with a movement. This

computation is not practicable on a mobile terminal

and the implementation must take into account a

model for user movements in order to manage the

problem like a pattern recognition problem. To make

this model as simple as possible we restrict the set of

actions to:

• a 90° clockwise body rotation

• a 90° counter-clockwise body rotation

• a pause of stillness

Thus, the problem is now how to recognize in

which of the three actions the actual movement falls.

Noise always affects the samples from the sensors.

We can divide noise into two categories: the first

cause of noise is the user that cannot move exactly

how the algorithm expects especially when the user

is still. Stillness is not real stillness and little

oscillations are perceived by the sensors, so it is

difficult to determine if the body is rotating slowly

or if simply the user believes to be still. The second

cause of noise is in the sensor itself. Even if the

sensor is really still, an offset is often visible.

(a) (b)

Figure 2: Reference coordinate systems for users (a) and

accelerometers phone geometry (b).

(a)

(b)

(c)

(d)

(e)

Figure 3: Samples from x-acceleration in different states:

mobile terminal still (a), still but with offsets in one or

more directions (b), with little oscillations (c), counter-

clockwise (d) and clockwise movement (e).

First of all, lets define a common reference

coordinate system as depicted in Fig. 2. It is possible

to monitor and record acceleration data from a

mobile terminal with sensor embedded: Fig. 3 shows

some graphs related to the output data from the x-

WEBIST 2009 - 5th International Conference on Web Information Systems and Technologies

432

component of the acceleration,. It can be observe

that there is a strong correlation between output data

and one of the five possible observation patterns

described above.

Using the chosen reference coordinate system,

and simplifying the problem using only the x-axis

data, it is possible to create a mathematical model

that describes each expected and possible state.

According to Fig. 3.(d) and Fig. 3.(e) and

generalizing them, it is possible to create the relative

(ideal) pattern functions of the x-axis data for a

counter-clockwise and a clockwise rotation of the

body with the following formulas:

)1cos()1sin(

ttbttaS

leftx

⋅⋅+⋅⋅=

(1)

)1cos()1sin(

ttdttcS

rightx

⋅⋅−⋅⋅−=

(2)

For . Where is the interval size of

the movement done by the user. At the same time, it

is possible to define a formula for the still state:

[

,tt

=Δ

]

()

++≡tS

still

(3)

Where the last two term are noise contribution

respectively from offsets and user oscillations. Once

defined the parameters, learned for each user in

order to better fit the observed samples with one of

the possible states, the system is ready to recognize

movements. Samples are filtered by a set of matched

filters. Matched filters are obtained by correlating

the discretized known template functions with the

observed signal in order to detect the presence of the

template in the observation. Using a Maximum-A-

Posteriori (MAP) criterion, the most probable state

is chosen.

4 THE MOBILE PROTOTYPE

After the definition of both, the mathematical model

describing the codebook of allowed movements and

a MAP decision criterion, a prototype has been

planned.

Despite of the achievable accuracy (in example

increasing the number of cardinal points or using the

three axis acceleration data), a basic prototype has

been implemented. In particular, only the four main

cardinal points have been considered (N,O,S,E).

The idea was that once started the application-

browser, a set of pictures are always shown in the

display. Which set of pictures must be drawn

depends on the current GPS position and the current

heading of the user. An embedded image set is

available but the application is able to connect and

browse geo-tagged pictures directly from panoramio

database, all over the world. Every time the user

changes its geographical orientation, the system

draws (90° clockwise body rotation or 90° counter-

clockwise body rotation) or not (pause of stillness)

other five pictures, choosing them with respect to the

estimated direction.

In order to assign different importance levels to

the different areas closest to the user’s location, a

geographical map is considered and centered in the

user position. The map is divided into four areas,

starting from the central position (P

GPS

) estimated by

the GPS tracker (Fig. 4). Then, each area is further

divided into three different sub-regions A, B, C. If

almost five pictures exist in the A sub-region

corresponding to the direction the user is facing to,

these photos are displayed. Otherwise, remaining

images are taken from the B sub-region and so on.

Figure 4: The partitioning scheme for the geographical

maps.

Figure 5: A snapshot of the application on a Nokia N95.

When the user starts using the application, and

sets the initial point, the pictures are displayed and

the user can start rotating and looking photos

representing places and things around him. A

snapshot of the running prototype is shown in Fig 5.

The Prototype application is freely available and

downloadable in the authors’ website.

A MOBILE BROWSER FOR GEO-REFERENCED IMAGES USING AN ACCELEROMETER-BASED COMPASS

433

5 EXPERIMENTAL TESTS

AND RESULTS

In order to assess the compass simulator efficiency

and the effectiveness of the mobile geo-browser, two

preliminary tests have been developed. We used ten

beta testers in each test: in the first experimental test

(Table1), two different sessions were developed,

using 40 and 80 accelerometer data samples per

second. In both sessions, the software recorded

computing times. In the second experimental test

(Table 2), users were asked to answer a brief

questionnaire with scores ranging from 1 (the worst)

to 5 (the best).

Table 1: Hit Percentage and computational times for the

accuracy tests. Mean times are computed in seconds.

Beta

Testers

% Hit

(40 samples)

Mean

Times

% Hit

(80 samples)

Mean

Times

User #1 93 0.604 97 0.837

User #2 91 0.663 98 0.903

User #3 88 0.597 98 0.881

User #4 90 0.548 95 0.688

User #5 95 0.615 97 0.753

User #6 90 0.771 98 0.899

User #7 93 0.659 97 0.932

User #8 95 0.570 99 0.775

User #9 92 0.588 94 0.874

User #10 89 0.656 97 0.829

Table 2: Results of subjective tests.

Beta

Testers

It is useful? It works? It is fast enough?

User #1 4 4 5

User #2 5 4 5

User #3 3 3 4

User #4 4 5 5

User #5 4 4 4

User #6 5 5 5

User #7 5 4 4

User #8 4 2 4

User #9 5 4 4

User #10 4 4 4

In Table 1, success hit percentages and

computing times for the first experimental test are

reported. Clearly, for all the users the hit percentage

was always greater than 94% when 80 samples were

used. Mean elaboration times increase with the

number of samples but they still remain acceptable

also for the 80 samples session.

The subjective experimental test, despite of its

preliminary characteristic, seams to show a

generalized positive assessment (Table 2).

6 CONCLUSIONS

In this paper a new mobile browser for geo-

referenced pictures has been proposed. Accuracy

results show that the accelerometers-based

simulation technology works already well and users

find very interesting and useful this type of

applications. In order to make the prototype even

more functional and effective, further developments

are under study and planning, like the increasing of

the accelerometer data rate, and the use of samples

from all the three axis sensors. Next generation

mobile terminals will be used to allow the real-time

processing of this bigger amount of data. Extra

capabilities like a map zoom-in/out function will be

embedded in order to better define the areas

interested by the selection of the images displayed

on the mobile terminal.

REFERENCES

ProgrammableWeb - Mashups, APIs, and the Web as

Platform (http://programmableweb.com).

Sarvas, R., Herrarte, E., Wilhelm, A., and Davis, M. (June

06-09, 2004). Metadata creation system for mobile

images, Proc. of the 2nd Int. Conference on Mobile

Systems, Applications, and Services Boston, MA,

USA, MobiSys '04. ACM Press, 36-48. New York, NY.

Shneiderman, B., Bederson, B. B., and Drucker, S. M.

(Apr 04, 2004). Find that photo!: interface strategies to

annotate, browse, and share, Commun. ACM, 49, 69-71.

Ahern, S., Naaman, M., Nair, R., and Yang, J. H. (June

18-23, 2007). World explorer: visualizing aggregate

data from unstructured text in geo-referenced

collections, In Proceedings of the 2007 Conference on

Digital Libraries Vancouver, Canada, JCDL '07. ACM

Press, 1-10. New York, NY,.

The degree confluence project. (http://www.confluence.org).

Carboni, D., Sanna, S., and Zanarini, P (September 12-15,

2006). GeoPix: image retrieval on the geo web, from

camera click to mouse click, In Proceedings of the 8th

Conference on Human-Computer interaction with Mobile

Devices and Services, Helsinki, Finland. MobileHCI '06,

vol. 159. ACM Press, 169-172. New York, NY.

Sung-Do Choi, Lee, A.S. Soo-Young Lee (August, 2006),

On-Line Handwritten Character Recognition with 3D

Accelerometer, 2006 IEEE International Conference

on Information Acquisition, (pp. 845-850). Weihai.

Daniel Wigdor, Ravin Balakrishnan. TiltText: Using Tilt

for Text Input to Mobile Phones (2003), Department of

Computer Science, University of Toronto

(www.dgp.toronto.edu).

D. Carboni, V. Marotto, F. Massidda, and P. Zanarini

(September 10, 2008). Fractal Browsing of Large

Geo-Referenced Picture Sets, 2nd International

Workshop on Distributed Agent-based Retrieval

Tools. Cagliari, Italy.

WEBIST 2009 - 5th International Conference on Web Information Systems and Technologies

434