Examining the Privacy Vulnerability Level of Android Applications

Georgia M. Kapitsaki

and Modestos Ioannou

Department of Computer Science, University of Cyprus, Aglantzia, Cyprus

Keywords:

Mobile Applications, Privacy, Privacy Vulnerabilities, Android Permissions.

Abstract:

Mobile applications are often granted access to various data available on the mobile device. Android appli-

cations provide the notion of permissions to let the developers deﬁne the data their applications require to

function properly. However, through accessing these data, applications may gain direct or indirect access to

sensitive user data. In this paper, we address the detection of privacy vulnerabilities in mobile applications

in Android via an analysis that is based mainly on the use of Android permissions. Different aspects of the

application are analyzed in order to draw conclusions offering an aggregated view of permission analysis in

the form of a penalty score, a feature that is missing in previous approaches that analyze permission use in

Android. Our work is supported by a web application prototype of App Privacy Analyzer that allows users

to upload an application and view the respective analysis results comparing them with other applications up-

loaded in previous uses of the system. This approach can be useful for security and privacy analysts and

developers that wish to examine the privacy vulnerability level of their Android applications, but also for end

users with technical expertise. We have used the tool for the analysis of 800 Android applications and are

discussing the results the observed permission use.

1 INTRODUCTION

The Android OS uses the notion of permissions in or-

der to let developers deﬁne the data their application

requires to function properly. Android permissions

have been investigated in many previous works in or-

der to recommend appropriate applications that con-

sider user’s security requirements (Zhu et al., 2014)

or analyze code use in terms of permissions (Wang

et al., 2015a).

Although a plethora of previous works along with

ofﬁcial documents on using permissions exist, per-

mission misuse is still often observed in Android ap-

plications. Tools that assist developers, and security

and privacy analysts in evaluating the level of privacy

protection in the Android applications they are devel-

oping or using is a required step towards understand-

ing the implications of privacy protection in mobile

applications and can assist in preventing relevant mis-

use. The introduction of the EU General Data Pro-

tection Regulation (GDPR) has rendered the need of

introducing more elaborate analysis techniques even

more important, as end users need to be well informed

on the collection of their personal data (T

abus¸c

a et al.,

2018).

https://orcid.org/0000-0003-3742-7123

Having as motivation the above, in this work, we

introduce a process for the analysis of privacy vulner-

abilities in Android applications with a focus on per-

missions. The process is based on the analysis of the

Application Package Kit (APK) ﬁle of the application.

Although similar tools are available (e.g. Exodus Pri-

vacy

), we integrate different mechanisms in one ap-

proach that also assigns an aggregated penalty score

in each application, allowing a comparative view in

respect to other relevant applications that are avail-

able in the system. Based on this approach, we have

analyzed 800 Android applications. This way we are

drawing some conclusions on the current permission

use in Android applications that are discussed in this

work. The resulting tool, App Privacy Analyzer, is

available as a prototype web application and also of-

fers statistics on privacy vulnerabilities considering

the applications that have been analyzed so far.

The contribution of this work is threefold:

1. We introduce an aggregated score calculation for

privacy vulnerabilities based on a combination of

different approaches of code analysis.

2. We offer an online web application that is open

for use, providing access to its source code.

https://reports.exodus-privacy.eu.org/trackers/

Kapitsaki, G. and Ioannou, M.

Examining the Privacy Vulnerability Level of Android Applications.

DOI: 10.5220/0007955100340045

In Proceedings of the 15th International Conference on Web Information Systems and Technologies (WEBIST 2019), pages 34-45

ISBN: 978-989-758-386-5

3. We perform an analysis on existing applications

to see where they stand in respect to this scoring

maintaining a repository with these data.

We argue that our approach offers a useful mecha-

nism for security and privacy analysts and developers

to assess the vulnerability level of their applications,

whereas it can also be used by end users that have

some technical knowledge in order to understand the

privacy level of applications they are using or intend

to use.

The rest of the paper is structured as follows. Sec-

tion 2 provides background information by describ-

ing the use of Android permissions. The introduced

approach is presented in detail in section 3. Section

4 gives some implementation details, and Section 5

introduces the results of the application analysis per-

formed by App Privacy Analyzer discussing the main

conclusions drawn. Threats to validity are brieﬂy dis-

cussed in Section 6. Section 7 is dedicated to the pre-

sentation of previous works in the area and ﬁnally,

Section 8 concludes the paper outlining directions of

future work.

2 ANDROID PERMISSIONS

Android has moved from ask-on-install (AOI) model

to ask-on-ﬁrst-use (AOFU) model regarding permis-

sions providing more freedom to users. Applications

of Android version 5.1.1 or lower ask the user, if she

accepts all permissions associated with an application

in order to be able to use it. Otherwise, the installa-

tion of the application cannot proceed. More recent

versions (since Android version 6.0) inform the user

about the need to access a speciﬁc resource after ap-

plication installation and upon execution. If the user

accepts, the preference is saved for subsequent uses

by the same application. Otherwise, the application

will ask the same question the next time the resource

needs to be accessed. If the user however, selects the

Never ask again option, the question will not appear

again, but the user will not be able to use the appli-

cation functionality related with the permission. An-

droid uses the following permission categories:

• Normal: refers to permissions requesting access

to resources that do not affect the system or other

applications. Since Android 6.0, access to normal

permissions is provided by default. The user of

the application is not asked about them, but she

can view them before installing the application.

• Signature: these permissions are provided to the

application on its installation, only if the certiﬁ-

cate of the application requesting the permission

matches the certiﬁcate of the application that de-

ﬁnes this permission.

• SignatureOrSystem: similar to signature permis-

sions, but used by vendors that distribute devices

with applications embedded in the Android oper-

ating system.

• Dangerous: these permissions are considered un-

safe, since the resources they access may affect

user privacy or device data, and may even affect

other installed applications. GPS and camera ac-

cess belong to this category.

• Custom: these permissions are deﬁned by the ap-

plication developer and refer to application re-

sources and not system resources.

Custom permissions are not used further in

our work. Since a large number of permissions

exist, Android places each permission within a

group inside the main permission category (e.g.

WRITE CALENDAR and READ CALENDAR are

placed in the CALENDAR group). Requests for dan-

gerous permissions are based on the permission cat-

egory. For instance, if the application asks permis-

sion for READ CALENDAR and it then requires also

WRITE CALENDAR, permission will be granted

without asking the user again, since the two belong

to the same group. This is not the case however, for

the other permission categories.

Android permissions have disadvantages that have

been studied in the past. For instance, some permis-

sion categories are too vague to allow users to make

meaningful decisions (Felt et al., 2011b), and many

applications declare permissions they are not actually

using in their code (Felt et al., 2011a). Moreover,

permissions along with sandboxing are not adequate

measures to prevent attacks in Android applications.

Privilege escalation attacks are possible due to tran-

sitive permission usage that permits non-privileged

applications to invoke higher-privileged applications

that do not have sufﬁcient protection mechanisms in

their interfaces (Davi et al., 2010). A previous work

has identiﬁed whether applications are over privi-

leged or follow the least privilege principle based on

a combination of runtime information and static anal-

ysis (Geneiatakis et al., 2015). Permission usage re-

lates though to the application functionality and these

aspects, i.e. sensitive information transmission and

application functions, should be viewed together, as

addressed in the DroidJust tool (Chen and Zhu, 2015).

Examining the Privacy Vulnerability Level of Android Applications

App Privacy Analyzer

APK

AndroidManifest.xml

analysis

Permission detection

and permission

groups

Library analysis

Permissions API calls

Report and

statistics

Preprocessing phase Calculation phase

Malware detection

Tracker detection

VirusTotal analysis

storage

Score calculation

Figure 1: Steps of Android App Privacy Analyzer.

3 APPLICATION ANALYSIS

PROCESS

3.1 Preprocessing Phase

The steps of the analysis performed by App Privacy

Analyzer in order to infer the privacy vulnerability

level of Android applications are depicted in Figure 1.

Initially, the APK is decompiled and the Android-

Manifest.xml ﬁle is detected. The following attributes

are extracted with some being privacy-relevant and

other being generic and used only for information pur-

poses:

• label: the application name.

• package: corresponds to an application unique

identiﬁer, referred to as Google Play ID. It usu-

ally corresponds to the project’s directory struc-

ture (e.g. com.example.android.app.diary).

• versionName and versionCode: version of the ap-

plication (name and number respectively).

• minSdkVersion and targetSdkVersion: version of

the lowest possible API and the targeted API re-

spectively.

• allowBackup: determines whether application

data can be backed up and restored (true/false

value). Although this functionality can be use-

ful, it also brings some security implications for

application data.

• debuggable: indicates whether the application can

be debugged during execution on user’s device

(true/false value). It is also considered a vulner-

ability, as it can expose data, and should be exam-

ined during penetration testing.

During the manifest analysis, the hash of the ap-

plication based on SHA-256 (that is still considered

secure) is also calculated (Gilbert and Handschuh,

2003). This is required in order to be able to uniquely

identify applications, even when application repack-

aging has been performed, and avoid performing the

analysis on an application that has already been pro-

cessed.

In the next step (Permission detection and permis-

sion groups) we focus on permissions the application

uses. Declared permissions can be extracted from

AndroidManifest.xml, whereas used permissions are

detected based on source code analysis. App Pri-

vacy Analyzer matches the declared permissions of

the manifest ﬁle with whether they are actually used

in the application source code. The following permis-

sion groups are formed as a result:

• Declared: permissions that are declared in An-

droidManifest.xml;

• Declared and used: permissions that are declared

in the manifest and are actually used in the appli-

cation;

• Not declared but used: permissions identiﬁed in

the source code but not in the manifest;

• Declared but not used: permissions that are de-

clared in the manifest ﬁle but are not present in

the source code.

In the subsequent step of Library analysis, App

Privacy Analyzer extracts from the source code third

party libraries used by the application. Based on this,

we also ﬁnd the permissions that are declared in the

third party libraries used by the application, in or-

der to detect permissions used via these libraries in-

directly, as they are also important for the user. The

following data are collected for each detected library:

library name, match ratio, package name, library per-

missions list, popularity, library type (e.g. ads) and

library website.

Used permissions (from the declared and used

and the not declared but used groups) are matched

with the method they are used in, pointing to the call

that is related with a permission (Permissions API

calls step). This is performed via static code analy-

sis. We then detect the speciﬁc places at the source

code, where this permission call is made, in order to

be able to indicate this information to the user (path

to source code ﬁle and method name).

3.2 Calculation Phase

After the preprocessing phase, we calculate the vul-

nerability level of Android applications using a com-

bination of techniques. Initially, a machine learn-

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

Table 1: Comparison of classiﬁcation algorithms for mal-

ware detection.

Algorithm Accuracy (%)

Naive Bayes 90.201

Bagging 91.709

k-Nearest Neighbor (kNN) 93.216

Stochastic Gradient Descent (SGD) 92.965

Locally Weighted Learning (LWL) 91.709

ing approach to malware detection that relies on per-

missions is adopted (Lopez and Cadavid, 2016). A

dataset

of 398 Android applications was used to train

a classiﬁer that characterizes an application as mal-

ware or non-malware. We used 10-fold cross vali-

dation to compare different classiﬁcation algorithms

and k-Nearest Neighbor was adopted, as it demon-

strated the best accuracy, although it is only slightly

better than Stochastic Gradient Descent (Table 1). We

are using this technique only partially in order to de-

termine whether an application can be regarded as

malware, as we observed that permission-based only

analysis is not sufﬁcient to characterize an application

as malware.

Tracker use in the application is also examined via

tracker libraries (Tracker detection step). Identiﬁed

libraries are matched against the tracker list provided

by Exodus Privacy. Tracker detection is indicated to

the user but it is not further considered in the calcula-

tion of the vulnerability level. We ﬁnally perform an

external additional malware analysis via VirusTotal

Based on the analysis of the previous steps, we

provide a consolidated view of privacy vulnerabili-

ties of the application. We calculate an aggregated

score, that allows the comparison of applications. We

use the following process for the score calculation. A

penalty score is assigned to the application for each

characteristic that is considered relevant to a privacy

vulnerability. The scoring used for this purpose is

shown in Table 2 and has been created from empir-

ical observations and by experimenting with different

scoring, but can be adapted in the future. For permis-

sions, the penalty score indicated is applied for each

permission independently (e.g. if an application has

three declared and used dangerous permissions, the

respective penalty score is 30). We also add a low

penalty score to applications that declare permissions

they do not use. Applications are penalized more,

when they are using permissions they do not declare

with a different penalty score for each permission

category (Dangerous, Signature/SignatureOrSystem,

Normal).

https://www.kaggle.com/xwolf12/

datasetandroidpermissions

https://www.virustotal.com

We explain below the rationale of choosing each

property that should be penalized:

• Dangerous, Signature/SystemOrSignature Per-

mission Declared but not Used: Although some

applications may keep these permissions as de-

clared for future needs, this leads to misinforming

the user and should be avoided. The penalty score

applied is very low.

• Dangerous, Signature/SystemOrSignature Per-

mission Declared and Used: many applications

require permissions of these categories for their

functionality. However, their use should be kept

to the minimum level, and for this reason the ap-

plications are penalized.

• Dangerous, Signature/SystemOrSignature, Nor-

mal Permission not Declared but Used: in the case

of dangerous permissions, this is a severe vulner-

ability case, as the user is unaware of the use of

the permission. A high penalty score is assigned.

• APK is Debuggable: debuggable Android appli-

cations pose a severe risk, as it allows sensitive

device data to be extracted. As this may even

contain data for banking transactions, the penalty

score applied is high.

• APK Allows Backup: application backup data can

be easily modiﬁed

, and for this reason a penalty

score is applied. However, the score is lower than

in the case of debuggable property, as it is less

critical.

• APK Classiﬁed as Dangerous: it is based on the

machine learning analysis. A high penalty score is

provided, as the classiﬁcation is based on the ex-

istence of permissions in applications. However,

less emphasis is put in this analysis in compari-

son to VirusTotal, as we encountered many false

positives (e.g. Facebook, Messenger, Instagram

applications were characterized as malware).

• APK Regarded Malware by VirusTotal: VirusTo-

tal is currently considered the most popular web-

site for multi-antivirus scanning, aggregating an-

tivirus products and online scan engines. The

penalty score is high due to the signiﬁcance of

characterizing an application as malware in this

tool and is assigned, when at least 30% of the

VirusTotal engines detect the application as mal-

ware. However, if an application is characterized

as malware by VirusTotal, but does not contain a

large number of permissions (e.g. dangerous per-

missions), it is not indicated as a high risk appli-

cation (based on the scores described later in the

section).

https://securitygrind.com/exploiting-android-backup/

Examining the Privacy Vulnerability Level of Android Applications

The maximum penalty score is 100. Although the

total of some applications may surpass this score (e.g.

when there are seven not declared but used dangerous

permissions), this was used as an upper limit so that

the application is not penalized more.

Based on the total penalty score of an application,

we place each application in one of the following risk

levels. The penalty score is not adequate to charac-

terize an application as risky, but we argue that it pro-

vides a measure for security and privacy analysts to

understand where their applications stand and decide

whether additional examination is required, whereas

it can assist end users providing them comparative in-

formation on an application they are using or are con-

sidering to use:

• no risk (score between 0 and 24) corresponding

to applications that have at most two dangerous

permissions or a larger number of permissions of

other categories;

• low risk corresponding to a low number of per-

missions (score 25-49);

• average risk (score 50-79) including applications

that are usually characterized as malware and use

an average number of permissions;

• high risk for applications that require further in-

vestigation (score 80-100).

4 IMPLEMENTATION

The App Privacy Analyzer tool has been implemented

as a web application using the Spring

and Vaadin

frameworks, employing Java and web technologies,

whereas Python has been used for application anal-

ysis purposes, since many of the existing tools we

have relied on are implemented in Python. In order

to extract information from AndroidManifest.xml ﬁle,

we have used the AndroGuard

reverse engineering

tool. For the quantitative permission analysis from

the source code we rely on PermissionChecker that

forms part of RiskInDroid (Risk Index for Android)

tool and may be used for research purposes (Merlo

and Georgiu, 2017). RiskInDroid uses machine learn-

ing techniques to calculate a numeric risk value for

Android applications between 0 and 100. For ﬁnding

third party libraries used in the applications, including

tracker libraries, we use LibRadar

(Ma et al., 2016).

For the permission-related methods, we are employ-

ing Pscout (Permission Scout) that performs static

https://spring.io/

https://vaadin.com/

https://github.com/androguard/androguard

https://github.com/pkumza/LibRadar

analysis in order to extract a permission speciﬁca-

tion using light-weight call-graph analysis to Android

API (Au et al., 2012). Pscout lists the permissions that

every Android API call requires and provides these

permissions used in a CSV (Comma-Separated Val-

ues) ﬁle. In our work, it is used to detect permission

method calls, in order to identify where a permission

call is performed. We have employed the permission

datasets provided by Pscout for Android APIs 9, 10,

14-19 and 21-22, in order to cover method changes

between these versions, but the appropriate version is

used for each analyzed application.

Application analysis results are stored in a ded-

icated database, so that existing APKs can be di-

rectly retrieved without analyzing the same applica-

tion again. Applications are uniquely identiﬁed us-

ing the hash created in the preprocessing phase. A

prototype implementation of App Privacy Analyzer is

available and its source code is available in the respec-

tive repository on GitHub

. For the machine learn-

ing algorithms, we used the corresponding implemen-

tations in WEKA (Waikato Environment for Knowl-

edge Analysis) (Hall et al., 2009).

Portions of the screenshots of the online appli-

cation are shown in Figure 2 and Figure 3 with the

main data of an example application and its aggre-

gated score, and the detected permissions in used li-

braries and permission calls, respectively. Data col-

lected from the application for information purposes,

e.g. label, minSdkVersion are also shown to the user.

Regarding VirusTotal, the users can visit the respec-

tive analysis in the link provided (visible in Figure 2),

whereas in the example provided there is the indica-

tion that 43 engines were detected by VirusTotal in

this application.

5 RESULTS AND DISCUSSION

There is no ground truth for Android permission use

in applications and for this reason it is not feasible

to compare the results of our approach to a baseline,

in terms of penalty scores used. As aforementioned,

we have experimented with different values for the

penalty scores before establishing the current form.

Regarding the independent steps, we are using exter-

nal tools to guide our analysis and we are relying on

their accuracy. However, we do provide a compar-

ative view of applications analyzed by App Privacy

Analyzer, and we compare our results with VirusTo-

tal. We collected a number of Android applications

https://github.com/CS-UCY-SEIT-lab/

AndroidAppPrivacyAnalyzer

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

Figure 2: Basic data of analyzed application.

Table 2: Penalty scores for application properties.

Property Score

Dangerous permission declared but not used 3

Signature/SystemOrSignature declared but not used 1.5

Dangerous permission declared and used 10

Signature/SystemOrSignature permission declared and used 5

Dangerous permission not declared but used 15

Signature/SystemOrSignature permission not declared but used 3.75

Normal permission not declared but used 7.5

APK is debuggable 8

APK allows backup 3

APK classiﬁed as dangerous 15

APK regarded malware by VirusTotal 20

Figure 3: Permissions calls locations in analyzed application.

from the F-Droid

catalogue of Android applications

and from APKPure

with a total of 800 Android

applications, since both F-Droid and APKPure pro-

vide access to the APK ﬁle. These applications dif-

https://f-droid.org

https://apkpure.com/

fer from the dataset of malware

applications that

we have used in the malware detection step. The ap-

plications belong to different categories, range from

minSdk version 1 to 24 and targetSdk 1 to 28 with

most belonging to minSdk version 16 (19.1%) and

https://github.com/ashishb/android-malware

Examining the Privacy Vulnerability Level of Android Applications

targetSdk version 25 (19.9%). The dataset is avail-

able on GitHub

Table 3 shows the 10 dangerous permissions most

usually employed in the analyzed Android applica-

tions. The last column shows the positions of the per-

mission in the list of the 40 most risky permissions

according to (Wang et al., 2014). The three numbers

correspond accordingly to the position of the permis-

sion in the list of most risky permissions considering

three different machine learning techniques for cal-

culation: Correlation Coefﬁcient (CorrCoef), ranking

with mutual information and t-test. Permissions with

no numbering do not appear in the lists. This may

be attributed to the fact that Android permission cat-

egories have been changed, since this previous publi-

cation. Table 4 shows the top 10 found permissions

regardless of the category. The most usual dangerous

permission, i.e. WRITE EXTERNAL STORAGE, is

in the tenth position. Normal permissions that were

considered dangerous in previous Android versions

appear in the list, e.g. INTERNET.

Regarding applications that use permissions they

do not declare, this was observed in 9.6% of cases.

Most permissions are declared but not used (36.67%)

and 18.3% are declared and used, which is a rela-

tively low percentage. A large number of permissions

comes from libraries used (35.44%) indicating that

caution is needed when using third party libraries, as

it may not be feasible to control the permissions they

are using. Among the applications, allowBackup is

set in 84.75%, posing a threat for most applications,

but only 1.5% are indicated as debuggable indicating

that this is a vulnerability identiﬁed by most develop-

ers. The average penalty score is 44.69 with 39.625%

applications belonging to the no risk range, 19.875%

to the low risk and 3.75% to the average risk range.

Our tool characterized 187 of applications as vul-

nerable. This corresponds to 23.375% of applications

being assigned to the high risk level. Among them,

14.5% reach a penalty score of 95 and above. We

manually analyzed some of the applications charac-

terized as risky. As many applications have a large

number of functionalities, the number of permissions

they use increases. For example, many applications

request storage access, camera access (mostly for QR

scan purposes) and location access. We are not con-

sidering the number of functionalities the application

offers in our analysis, as we rely on the APK ﬁles,

but future research work could perform further source

code analysis or combine the information for the ap-

plication description in Google Play to draw conclu-

sions about the complexity of functionality offered

by the application and its connection with permission

use. Via manual analysis we also detected applica-

tions that requested permissions, such as access to

make calls or send SMS, with no relevant functional-

ity to explain these requests and therefore, their char-

acterization as vulnerable is justiﬁed. Their results

can be considered close to the results of the AVC Un-

Droid

tool, where 31.6% of applications that have

been submitted for analysis are characterized as mali-

cious.

The applications that are characterized as most

vulnerable (aggregated penalty score above 95) are

usually linked with the following dangerous permis-

sions:

• ACCESS COARSE LOCATION

• ACCESS FINE LOCATION

• WRITE EXTERNAL STORAGE

• READ PHONE STATE

• and READ EXTERNAL STORAGE

Different trackers have also been found in these

applications, e.g. Google Analytics, Google Ads,

Amazon Advertisement. Note that the trackers list we

are using consists of 152 trackers. The analysis re-

sults provide a good indication for vulnerabilities, but

further examination is required in order to draw more

concrete conclusions for each application.

We also investigated whether the number of dan-

gerous permissions found in an application is related

to whether the application uses permissions that it

does not declare. We ran a two independent sam-

ples t-test. The difference was statistically signiﬁcant.

We observed that applications with at least one not

declared but used permission (regardless of the per-

mission group, i.e. declared, declared but not used,

not declared but used, permission coming from a li-

brary) have a larger number of dangerous permissions

(t = −6.708, p = 0.000). The descriptive statistics are

displayed in Table 5. This result justiﬁes the use of

these parameters in the penalty score calculation.

Finally, we compared our results with VirusTotal.

We compared the penalty score with the number of

engines of VirusTotal that identiﬁed the application as

malware. We found that 47.1% of applications identi-

ﬁed as malware by at least one VirusTotal engine were

placed in the high risk category (in 121 applications

at least 1 engine detected the application as malware).

There were many cases with a high penalty score that

were not identiﬁed by VirusTotal: 72.5% of applica-

tions in the high risk category were not detected by

any VirusTotal engine. This difference is expected, as

the aim of VirusTotal is malware detection. May ap-

plications are privacy risky, but do not contain viruses.

http://undroid.av-comparatives.info

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

Table 3: Frequency of top dangerous permissions.

Permission Frequency Position in (Wang et al., 2014)

WRITE EXTERNAL STORAGE 63 19, 13, 19

ACCESS COARSE LOCATION 36.75 -

READ PHONE STATE 36.125 7, 4, 3

ACCESS FINE LOCATION 32.875 39, 38, 39

READ EXTERNAL STORAGE 31.5 8, 5, 5

CAMERA 17.75 31, 22, 31

GET ACCOUNTS 16.875 -

READ CONTACTS 11.875 17, 17, 17

RECORD AUDIO 9.625 -

RECEIVE SMS 9.125 2, 2, 10

Table 4: Frequency of top permissions.

Permission Frequency (%)

INTERNET 88.5

DUMP 81.375

INTERACT ACROSS USERS FULL 81

INTERACT ACROSS USERS 81

WAKE LOCK 78.5

VIBRATE 68.875

BACKUP 65.875

ACCESS NETWORK STATE 63.625

MANAGE APP TOKENS 63.375

WRITE EXTERNAL STORAGE 63

Table 5: Group statistics for t-test comparing not declared

but used with dangerous permissions.

Permission N Mean number Std.

of dangerous deviation

permissions

≥1 not declared but 522 6.77 5.903

used

none not declared but 277 3.53 7.479

used

6 THREATS TO VALIDITY

In relevance to external validity, referring to the ex-

tent we can generalize our ﬁndings, we present re-

sults based on the analyzed applications that form a

relatively small dataset, but we do not expect large

deviations for a larger number of applications (Yin,

2013). A further limitation is that we are considering

the current handling of permissions by Android. If

this changes in the future, our approach will have to

be adapted.

Regarding internal validity, we rely in some steps

of our approach on existing source code analysis tools

and assume that the results they provide are accurate.

For instance, we use LibRadar to detect tracker li-

braries, but if they are not adequately detected, our

approach is also affected.

Construct validity, referring to whether a test mea-

sures the intended construct, may be affected by the

values chosen for the penalty scores for the aggre-

gated score calculation. In order to minimize this ef-

fect, we have adapted the penalty scores based on ob-

servations from our initial experiments and empirical

analysis in previous works. We do not expect to be

affected by conclusions validity, as the analysis per-

formed employs simple statistics.

7 RELATED WORK

Many previous works have focused on malware of

Android applications, but are not further analyzed as

they have a different target, although some of them

make use of permissions (Lindorfer et al., 2015; Yang

et al., 2015; Idrees et al., 2017). Regarding permis-

sion analysis, one work examines the risks associated

with Android permissions, both on the level of indi-

vidual permissions and on the level of a group of col-

laborative permissions (Wang et al., 2014). This way

permissions are ranked with respect to their risk with

READ SMS and RECEIVE SMS obtaining the high-

est risk scores.

Suggestions as to how a permission system should

be designed are made in (Rosen et al., 2013). Dis-

advantages found are that permissions are not ﬁne-

grained enough, that there should be a differentiation

between actions performed by users and actions per-

formed in the background, and that the differences be-

tween Android application code and third-party code

should be examined. The authors also create a knowl-

edge base of privacy-relevant API calls and use it to

produce application behavior proﬁles. Although it

would have been useful to utilize this knowledge base

also in the framework of our work, it seems that it is

no longer available. Improvements on how permis-

sions are handled in terms of user interactions have

been investigated in a usability approach (Micinski

et al., 2017). The authors have measured how user in-

teractions and sensitive resource use are related in ex-

isting applications and have then performed an online

Examining the Privacy Vulnerability Level of Android Applications

survey to see how users view interactions with mobile

applications in terms of connecting user actions in an

application with access to sensitive resources

In order to assist permission management from

the user’s perspective, in (Wijesekera et al., 2017)

the authors propose an approach that uses context to

adapt user decisions for access to resources. Initially,

the authors use a ﬁeld study to analyze the contex-

tual aspects of user privacy decisions to regulate ac-

cess to sensitive resources. At a later stage, privacy

decisions are made by the proposed system without

user’s intervention. Another approach ﬁnds the bal-

ance between using ads in mobile applications and

free use (Leontiadis et al., 2012). On the one hand,

the user is interested in maintaining her privacy and,

on the other hand, developers would like to maxi-

mize their advertisement revenue through user pro-

ﬁling. The authors propose a market-aware privacy

protection framework based on a feedback loop that

adjusts privacy protection level on mobile phones in

response to advertisement generated revenue.

Another group of previous works focus either on

tracking information ﬂow or providing permission

controls at ﬁner levels of granularity. In (Zimmeck

et al., 2017), the authors introduce a system that

checks data practices of Android applications against

privacy requirements using their privacy policies and

the legislation. They have found that in a large num-

ber of applications there are potential inconsistencies

between what the applications states to do and what

the code of the application does.

Works that are closer to our approach perform

static analysis that can be performed on mobile ap-

plications for various purposes, as described in a pre-

vious survey (Li et al., 2017). Information ﬂows

that may violate privacy relying on the applica-

tion’s Dalvik bytecode are examined in the theoreti-

cal approach of a security type system in (Mann and

Starostin, 2012). The type system uses a privacy pol-

icy that deﬁnes method and ﬁeld signatures as in-

put. Methods in the application are then examined

to see whether they respect the speciﬁed signatures.

Dynamic analysis is used in (Wang et al., 2015a) in

conjunction with static analysis to determine the pre-

cise set of permissions that an application needs to

run correctly in relevance to the declared permissions.

In this approach, permission usage information is ini-

tially extracted from API invocations, and then a dy-

namic testing technique monitors the runtime behav-

ior of the application.

Other approaches focus on inferring the permis-

sions an API needs. The ApMiner approach is able

to learn from the usage of APIs and permissions in

other applications published in a marketplace (Karim

et al., 2016). Using this information, it can help the

development of new applications by recommending

the permissions to be added in the application taking

into account the APIs it uses. This work is further

enhanced in (Bao et al., 2016), where the authors in-

troduce an approach based on collaborative ﬁltering.

This approach is based on the assumption that appli-

cations that share similar features, based on the APIs

they use, share also similar permissions.

Going one step further, the work in (Wang et al.,

2015b) employs text mining techniques to infer the

purpose of permission use. It is based on decom-

piled code to extract application-speciﬁc (e.g. calls

to APIs, use of Intent, Content Provider) and text-

based (e.g. words extracted from package names,

class names and other sources) features, and has

been used in the case of contacts and location per-

missions, i.e. READ CONTACT LIST and AC-

CESS FINE LOCATION.

VetDroid is based on dynamic analysis for

identifying sensitive behaviors of Android applica-

tions (Zhang et al., 2014). It observes how appli-

cations use permissions to access system resources

and how this information is then used in the appli-

cation. It has been used to facilitate malware analy-

sis. RiskMon shares some similarities with our ap-

proach, since it assigns a risk score to each applica-

tion (Jing et al., 2015). This is performed on every

access attempt on sensitive information and at the end

a cumulative score is calculated for each application.

Based on this, it performs automated permission re-

vocation if needed. An extension of the Android OS

is provided as a proof-of-concept implementation of

RiskMon.

Some online tools offer users the possibility to

analyze Android applications. Exodus Privacy

de-

tects ads, tracking, analytics, whereas it also reports

suspicious network trafﬁc. Imported trackers are de-

tected based on a tracker list. Concerning permis-

sions, Exodus analyzes AndroidManifest.xml and re-

ports indicated permissions. DNS, UDP, HTTP and

HTTPS trafﬁc is supported as detected with dynamic

application analysis using a simulated Android de-

vice. AVC UnDroid

analyzes the APK of uploaded

applications and uses different tools: Buster Sandbox

Analyzer

(evaluates malware suspicions based on

process behavior), ssdeep

(computes context trig-

gered piecewise hashes or fuzzy hashes and detects

potentially risky applications) (Kornblum, 2006), AP-

KTool

(for performing reverse engineering on the

APK) and security engines. There is an image indica-

http://bsa.isoftware.nl

https://ssdeep-project.github.io/ssdeep

https://ibotpeaches.github.io/Apktool

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

Table 6: Online privacy analysis tools comparison.

Property Exodus Privacy AVC UnDroid App Privacy Analyzer

tracker support 3 7 3

trafﬁc analysis support 3 7 7

declared permissions detection 3 3 3

used permissions detection 7 3 3

permission level indication 3 3 3

permission call location 7 7 3

indication

actions/intents indication 7 3 via VirusTotal

malware detection 7 partial 3

adware SDK detection 7 3 7

statistics most frequent trackers submissions per country most frequent permissions

clean vs. malicious APKs most frequent dangerous permissions

analyzed APKs number analyzed APKs number

analyzed APKs release dates declared vs. used permissions

ad-supported APKs uses per permission

malware families declared vs. used per permission

adware SDKs average penalty score

tools used tracker list Buster Sandbox Analyzer, ssdeep, AndroGuard, PermissionChecker,

APKTool, security engines LibRadar, Pscout, VirusTotal

input Google Play URL APK ﬁle APK ﬁle

tion of how dangerous an application is, but it is not

indicated how this is calculated. Malware detection

information does not appear in the analysis report.

In regard to previous works, we employ different

mechanisms in order to offer a consolidated view of

the application. We introduce an aggregated vulner-

ability score based on the independent examinations

providing a summary of the analysis report. To the

best of our knowledge, this is the ﬁrst work that intro-

duces this construct that allows users to compare the

application with other analyzed applications. Table 6

summarizes the comparison of our approach with Ex-

odus Privacy and AVC UnDroid that are the closest

existing approaches. In comparison to these tools,

we provide more statistics concerning permission use,

but do not offer adware information, since it is not di-

rectly related to privacy, unless adware is classiﬁed

as spyware. The Exodus tool detects trackers based

on the detection of code signatures in the application

(that refer to tracker packages) and their matching to

predeﬁned tracker lists. Instead, we use LibRadar to

detect tracker libraries, but utilize also the tracker list

of Exodus. We cover the main capabilities offered by

AVC UnDroid, but use AndroGuard instead of AP-

KTool for reverse engineering purposes. Permission

call location indication is also a feature that is miss-

ing from existing approaches, but it is useful for easily

identifying the permission call inside the application

source code, especially for large applications.

8 CONCLUSIONS

In this paper, we have presented our approach on an-

alyzing Android applications for privacy vulnerabili-

ties. The approach employs different steps, wherein

an aggregated score based on application characteris-

tics as detected in these steps is calculated for each

application. We described the results of using the tool

on 800 Android applications. The approach can be a

useful tool for privacy and security analysts and de-

velopers in identifying issues that need further exami-

nation and for users that can get an overview of issues

in applications they are using or intend to use.

As future work, we intend to study how the use

of common libraries affects permission analysis for

a given application, as a previous work has identi-

ﬁed that this may lead to inaccurate results, if analysis

mechanisms do not consider library code that can be

considered noise for the application (Li et al., 2016).

We will examine the consideration of more charac-

teristics in the calculation of the aggregated penalty

score, considering also the compliance of the An-

droid application with EU GDPR. We also intend to

put the user in the loop of privacy handling providing

more user-friendly ways for presenting to the users in-

formation on the privacy level of applications focus-

ing on users with limited or non technical expertise,

and users of younger ages. Finally, we intend to add

more steps in our analysis. Dynamic code analysis

tools can be utilized for this purpose, such as the Mo-

bile Security Framework (MobSF

), Dynamic Exe-

cutable Code Analysis Framework (DECAF) (Hen-

derson et al., 2014) and DroidBox

https://github.com/MobSF/Mobile-Security-

Framework-MobSF

https://github.com/pjlantz/droidbox

Examining the Privacy Vulnerability Level of Android Applications

ACKNOWLEDGMENT

This work was partially funded by the European Com-

munity CEF-TC-2018-1 - Safer Internet (grant agree-

ment number INEA/CEF/ICT/A2018/1614254) CY-

berSafety II (http://www.cybersafety.cy/) project.

REFERENCES

Au, K. W. Y., Zhou, Y. F., Huang, Z., and Lie, D. (2012).

Pscout: analyzing the android permission speciﬁca-

tion. In Proceedings of the 2012 ACM conference on

Computer and communications security, pages 217–

228. ACM.

Bao, L., Lo, D., Xia, X., and Li, S. (2016). What permis-

sions should this android app request? In 2016 In-

ternational Conference on Software Analysis, Testing

and Evolution (SATE), pages 36–41.

Chen, X. and Zhu, S. (2015). Droidjust: Automated

functionality-aware privacy leakage analysis for an-

droid applications. In Proceedings of the 8th ACM

Conference on Security & Privacy in Wireless and

Mobile Networks, WiSec ’15, pages 5:1–5:12, New

York, NY, USA. ACM.

Davi, L., Dmitrienko, A., Sadeghi, A.-R., and Winandy, M.

(2010). Privilege escalation attacks on android. In in-

ternational conference on Information security, pages

346–360. Springer.

Felt, A. P., Chin, E., Hanna, S., Song, D., and Wagner, D.

(2011a). Android permissions demystiﬁed. In Pro-

ceedings of the 18th ACM Conference on Computer

and Communications Security, CCS ’11, pages 627–

638, New York, NY, USA. ACM.

Felt, A. P., Greenwood, K., and Wagner, D. (2011b). The

effectiveness of application permissions. In Proceed-

ings of the 2Nd USENIX Conference on Web Applica-

tion Development, WebApps’11, pages 7–7, Berkeley,

CA, USA. USENIX Association.

Geneiatakis, D., Fovino, I. N., Kounelis, I., and Stir-

paro, P. (2015). A permission veriﬁcation approach

for android mobile applications. Comput. Secur.,

49(C):192–205.

Gilbert, H. and Handschuh, H. (2003). Security analysis

of sha-256 and sisters. In International workshop

on selected areas in cryptography, pages 175–193.

Springer.

Hall, M., Frank, E., Holmes, G., Pfahringer, B., Reutemann,

P., and Witten, I. H. (2009). The weka data min-

ing software: an update. ACM SIGKDD explorations

newsletter, 11(1):10–18.

Henderson, A., Prakash, A., Yan, L. K., Hu, X., Wang, X.,

Zhou, R., and Yin, H. (2014). Make it work, make it

right, make it fast: Building a platform-neutral whole-

system dynamic binary analysis platform. In Proceed-

ings of the 2014 International Symposium on Software

Testing and Analysis, ISSTA 2014, pages 248–258,

New York, NY, USA. ACM.

Idrees, F., Rajarajan, M., Conti, M., Chen, T. M., and

Rahulamathavan, Y. (2017). Pindroid: A novel an-

droid malware detection system using ensemble learn-

ing methods. Computers & Security, 68:36 – 46.

Jing, Y., Ahn, G. J., Zhao, Z., and Hu, H. (2015). Towards

automated risk assessment and mitigation of mobile

applications. IEEE Transactions on Dependable and

Secure Computing, 12(5):571–584.

Karim, M. Y., Kagdi, H., and Penta, M. D. (2016). Min-

ing android apps to recommend permissions. In 2016

IEEE 23rd International Conference on Software

Analysis, Evolution, and Reengineering (SANER),

volume 1, pages 427–437.

Kornblum, J. (2006). Identifying almost identical ﬁles us-

ing context triggered piecewise hashing. Digital in-

vestigation, 3:91–97.

Leontiadis, I., Efstratiou, C., Picone, M., and Mascolo, C.

(2012). Don’t kill my ads!: Balancing privacy in an

ad-supported mobile application market. In Proceed-

ings of the Twelfth Workshop on Mobile Computing

Systems & Applications, HotMobile ’12, pages

2:1–2:6, New York, NY, USA. ACM.

Li, L., Bissyand

e, T. F., Klein, J., and Traon, Y. L. (2016).

An investigation into the use of common libraries in

android apps. In 2016 IEEE 23rd International Con-

ference on Software Analysis, Evolution, and Reengi-

neering (SANER), volume 1, pages 403–414.

Li, L., Bissyand, T. F., Papadakis, M., Rasthofer, S., Bartel,

A., Octeau, D., Klein, J., and Traon, L. (2017). Static

analysis of android apps: A systematic literature re-

view. Information and Software Technology, 88:67 –

95.

Lindorfer, M., Neugschwandtner, M., and Platzer, C.

(2015). Marvin: Efﬁcient and comprehensive mobile

app classiﬁcation through static and dynamic analy-

sis. In Computer Software and Applications Confer-

ence (COMPSAC), 2015 IEEE 39th Annual, volume 2,

pages 422–433. IEEE.

Lopez, C. C. U. and Cadavid, A. N. (2016). Machine learn-

ing classiﬁers for android malware analysis. In Com-

munications and Computing (COLCOM), 2016 IEEE

Colombian Conference on, pages 1–6. IEEE.

Ma, Z., Wang, H., Guo, Y., and Chen, X. (2016). Libradar:

fast and accurate detection of third-party libraries in

android apps. In Proceedings of the 38th Interna-

tional Conference on Software Engineering Compan-

ion, pages 653–656. ACM.

Mann, C. and Starostin, A. (2012). A framework for static

detection of privacy leaks in android applications. In

Proceedings of the 27th annual ACM symposium on

applied computing, pages 1457–1462. ACM.

Merlo, A. and Georgiu, G. C. (2017). Riskindroid: Machine

learning-based risk analysis on android. In IFIP In-

ternational Conference on ICT Systems Security and

Privacy Protection, pages 538–552. Springer.

Micinski, K., Votipka, D., Stevens, R., Koﬁnas, N.,

Mazurek, M. L., and Foster, J. S. (2017). User in-

teractions and permission use on android. In Proceed-

ings of the 2017 CHI Conference on Human Factors

in Computing Systems, CHI ’17, pages 362–373, New

York, NY, USA. ACM.

WEBIST 2019 - 15th International Conference on Web Information Systems and Technologies

Rosen, S., Qian, Z., and Mao, Z. M. (2013). Appproﬁler:

A ﬂexible method of exposing privacy-related behav-

ior in android applications to end users. In Proceed-

ings of the Third ACM Conference on Data and Appli-

cation Security and Privacy, CODASPY ’13, pages

221–232, New York, NY, USA. ACM.

abus¸c

a, A., T

abus¸c

a, S.-M., Garais, G. E., and En

aceanu,

A. (2018). Mobile apps and gdpr issues. Journal of In-

formation Systems & Operations Management, 12(1).

Wang, H., Guo, Y., Tang, Z., Bai, G., and Chen, X. (2015a).

Reevaluating android permission gaps with static and

dynamic analysis. In 2015 IEEE Global Communica-

tions Conference (GLOBECOM), pages 1–6.

Wang, H., Hong, J., and Guo, Y. (2015b). Using text min-

ing to infer the purpose of permission use in mobile

apps. In Proceedings of the 2015 ACM International

Joint Conference on Pervasive and Ubiquitous Com-

puting, UbiComp ’15, pages 1107–1118, New York,

NY, USA. ACM.

Wang, W., Wang, X., Feng, D., Liu, J., Han, Z., and Zhang,

X. (2014). Exploring permission-induced risk in an-

droid applications for malicious application detection.

IEEE Transactions on Information Forensics and Se-

curity, 9(11):1869–1882.

Wijesekera, P., Baokar, A., Tsai, L., Reardon, J., Egelman,

S., Wagner, D., and Beznosov, K. (2017). The fea-

sibility of dynamically granted permissions: Aligning

mobile privacy with user preferences. In Security and

Privacy (SP), 2017 IEEE Symposium on, pages 1077–

1093. IEEE.

Yang, W., Xiao, X., Andow, B., Li, S., Xie, T., and Enck,

W. (2015). Appcontext: Differentiating malicious and

benign mobile app behaviors using context. In 2015

IEEE/ACM 37th IEEE International Conference on

Software Engineering, volume 1, pages 303–313.

Yin, R. K. (2013). Case study research: Design and meth-

ods. Sage publications.

Zhang, Y., Yang, M., Yang, Z., Gu, G., Ning, P., and Zang,

B. (2014). Permission use analysis for vetting unde-

sirable behaviors in android apps. IEEE Transactions

on Information Forensics and Security, 9(11):1828–

1842.

Zhu, H., Xiong, H., Ge, Y., and Chen, E. (2014). Mobile app

recommendations with security and privacy aware-

ness. In Proceedings of the 20th ACM SIGKDD In-

ternational Conference on Knowledge Discovery and

Data Mining, KDD ’14, pages 951–960, New York,

NY, USA. ACM.

Zimmeck, S., Wang, Z., Zou, L., Iyengar, R., Liu, B.,

Schaub, F., Wilson, S., Sadeh, N., Bellovin, S. M.,

and Reidenberg, J. (2017). Automated analysis of pri-

vacy requirements for mobile apps. In Proceedings of

the Network and Distributed System Security (NDSS)

Symposium, volume 2017.

Examining the Privacy Vulnerability Level of Android Applications