Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for

Hybrid Systems

Marco Anisetti

1 a

, Claudio A. Ardagna

1 b

, Nicola Bena

and Ernesto Damiani

2 c

Dipartimento di Informatica, Universit

a degli Studi di Milano, Milan, Italy

Artiﬁcial Intelligence and Intelligent Systems Institute (AIISI), Khalifa University, Abu Dhabi, U.A.E.

Keywords:

Assurance, Hybrid System, Security, Virtual Private Network.

Abstract:

Security assurance provides a wealth of techniques to demonstrate that a target system holds some non-

functional properties and behaves as expected. These techniques have been recently applied to the cloud

ecosystem, while encountering some critical issues that reduced their beneﬁt when hybrid systems, mixing

public and private infrastructures, are considered. In this paper, we present a new assurance framework that

evaluates the trustworthiness of hybrid systems, from traditional private networks to public clouds. It imple-

ments an assurance process that relies on a Virtual Private Network (VPN)-based solution to smoothly integrate

with the target systems. The assurance process provides a transparent and non-invasive solution that does not

interfere with the working of the target system. The performance of the framework have been experimentally

evaluated in a simulated scenario.

1 INTRODUCTION

We live in a pervasive and connected society, where

users as well as enterprises are engaging with digi-

tal technologies to carry out day-to-day activities and

business processes. Recent years have been charac-

terized by a continuous and fast evolution of commu-

nication and computation technologies towards pub-

lic infrastructures, moving from service-based archi-

tectures to the cloud, and more recently to microser-

vices and Internet of Things (IoT). At the same time,

the importance of private infrastructures has a come-

back pushed by an increasing need of data protec-

tion, which resulted in new regulations such as the

General Data Protection Regulation (GDPR) in Eu-

rope. In this complex scenario, made of hybrid sys-

tems mixing public and private infrastructures, new

concerns emerged undermining the users’ perceived

trust (e.g., (Teigeler et al., 2017)), as well as their con-

ﬁdence in the security of the overall systems.

The problem of guaranteeing the trustworthiness

of such systems has been extensively studied by the

research community in the last couple of decades, to

the aim of fully unleashing their potential and foster

widespread adoption. Security assurance stands out

https://orcid.org/0000-0002-5438-9467

https://orcid.org/0000-0001-7426-4795

https://orcid.org/0000-0002-9557-6496

as the way to gain justiﬁable conﬁdence that IT sys-

tems will consistently demonstrate one or more secu-

rity properties, and operationally behave as expected,

despite failures and attacks (Anisetti et al., 2017). It

implements processes and techniques, based on audit,

certiﬁcation, compliance, supporting the assessment

and veriﬁcation of a target system behavior against

security properties and requirements (Ardagna et al.,

2015). Assurance solutions have been recently ap-

plied to service-based systems, including cloud and

IoT systems (Ardagna et al., 2015; Baldini et al.,

2016), introducing new frameworks addressing pecu-

liar requirements such as scalability, multi-layer eval-

uation, and continuous monitoring. Although they

provide outstanding beneﬁts on the perceived trust,

they lack of generality and cannot be easily adapted to

current scenarios, where different services deployed

on hybrid public and private infrastructures are com-

posed at run time. Many of these solutions are in

fact ad hoc (Cheah et al., 2018; Elsayed and Zulk-

ernine, 2018), meaning they cannot handle a modern

IT system as a whole. Moreover, existing assurance

techniques, and corresponding frameworks, require

some effort for being integrated with the target sys-

tem, interfering with its normal operation (e.g., per-

formance), and introducing not-negligible (monetary

and business) costs.

Anisetti, M., Ardagna, C., Bena, N. and Damiani, E.

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems.

DOI: 10.5220/0009822600980109

In Proceedings of the 17th International Joint Conference on e-Business and Telecommunications (ICETE 2020) - SECRYPT, pages 98-109

ISBN: 978-989-758-446-6

In this paper, we aim to ﬁll in these gaps by

proposing a new assurance framework enabling a cen-

tralized security assurance targeting both public and

private infrastructures, including public and private

cloud as well as traditional private systems. It im-

plements an assurance process that relies on a Virtual

Private Network (VPN)-based solution for a smooth

integration with the target system, minimizing the in-

terferences of the framework on the target system

functioning. Our contribution is threefold. We ﬁrst

deﬁne the requirements a security assurance frame-

work and corresponding process have to fulﬁll in

our scenario made of hybrid systems. We then pro-

pose a novel VPN-based assurance framework and

corresponding process addressing these requirements.

To this aim, we introduce several modiﬁcations to a

standard VPN conﬁguration, including the so-called

Server-side NAT, Client-side NAT, and a custom pro-

tocol to resolve conﬂicts between the networks in the

VPN. We ﬁnally propose an experimental evaluation

of our framework and comparison with the state of the

art according to the identiﬁed requirements.

The remaining of this paper is organized as fol-

lows. Section 2 deﬁnes an assurance process and

identiﬁes the requirements it has to fulﬁll. Section 3

presents our assurance framework. Section 4 de-

scribes the VPN-based approach at the basis of our

assurance process in Section 5. Section 6 presents

an experimental evaluation of the framework perfor-

mance in a simulated scenario. Section 7 proposes a

comparison with the state of the art according to the

identiﬁed requirements. Section 8 draws our ﬁnal re-

marks.

2 ASSURANCE REQUIREMENTS

The advent and success of cloud computing and Inter-

net of Things (IoT) are radically changing the shape

of distributed systems. Hybrid systems, building on

both private and public technologies, introduce new

requirements and challenges on security assurance

techniques, which must take a step forward for be-

ing applicable to modern architectures. In particular,

the deﬁnition of new assurance processes is crucial

to ﬁll in the lack of trustworthiness that is one of the

main hurdles against the widespread diffusion of such

systems.

Despite targeting complex systems, a security as-

surance process should be lightweight and not in-

terfere with the normal operation of the system un-

der veriﬁcation. The need of a lightweight process

is strictly connected to its psychological acceptabil-

ity (Saltzer and Schroeder, 1975), meaning that ﬁnal

users are more willing to accept to perform assur-

ance activities that preserve the behavior of the sys-

tem and do not increase overall costs. In fact, al-

though the undebatable advantages given by a con-

tinuous evaluation of system security, users are recal-

citrant with respect to a process perceived as heavy

and costly (West, 2008).

Cost management and optimization are the foun-

dation of assurance adoption. Costs refer to monetary

costs in terms of additional human and IT resources,

as well as performance and business costs in terms

of overhead, latency, and reliability. Monetary costs

include the need of highly specialized personnel, on

one side, and resources allocated and paid on demand

on the other side, which are spent to manage non-

functional aspects of the system often considered as

superﬂuous. Performance costs include the need of

continuously verifying the security status of a system.

They intrinsically introduce a not-negligible overhead

and latency, an assurance process has to cope with.

Assessment activities are only viable if they take re-

source demands under control, avoiding scenarios in

which they become a source of attack. Business costs

are partially overlapped with performance costs and

model how much assurance activities interfere with

the normal operations of a business process. On one

side, changes required to the system to connect an as-

surance process should be reduced to the minimum,

and mostly work at the interface level. On the other

side, an assurance process cannot threat itself the sys-

tem. For example, run-time veriﬁcation of a system

security status cannot increase the risk of system un-

availability by performing penetration testing on the

production system. A good balance between active

and passive testing/monitoring should be provided.

We identify the main requirements an assurance

process has to satisfy (MUST/SHOULD) to address

the peculiarities of modern systems, as follows.

Transparency: it MUST not interfere with the nor-

mal operation of the business processes, being

transparent to the ﬁnal user of the system where

the assurance process is performed.

Non-invasivess: it MUST require the least possible

set of changes to the target system.

Safety: it MUST not introduce (or at least minimize)

new risks on the target system.

Continuity: it SHOULD provide a continuous pro-

cess, verifying the status of security while the sys-

tem is operating and evolving.

Lightness: it SHOULD be lightweight and cope with

systems having limited resources.

Adaptivity: it SHOULD be dynamic and incremen-

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

tal to adapt to changes in the system under veriﬁ-

cation and its environment.

Such requirements should be supported by a cen-

tralized framework tuning each aspect of the assur-

ance evaluation. The framework itself has its own

requirements (Anisetti et al., 2016), which are sum-

marized in the following.

Evidence-based Veriﬁcation: it SHOULD imple-

ment a veriﬁcation built on evidence collected on

the target system, to get the real picture of its se-

curity status.

Extensibility: it MUST inspect hybrid targets, from

traditional private networks to public clouds, as

well as hybrid clouds and IoT.

Multi-layer: it SHOULD assess system security

at different layers, from network protocols to

application-level services.

Scalability: it SHOULD support a scalable process,

able to manage an increasing number of assurance

processes and evaluations.

Generally speaking, an assurance framework

MUST at least implement a process that has the low-

est possible impact on the target resources and nor-

mal system activities (transparency), do not modify

the current ICT infrastructure or at least require very

few modiﬁcations (non-invasiveness), do not affect

security by introducing new risks (safety), while be-

ing generic enough to address peculiarities of hybrid

systems (extensibility).

3 ASSURANCE FRAMEWORK

We extend our assurance framework in (Anisetti et al.,

2018), designed for the assessment of cloud systems,

to satisfy the requirements in Section 2 and address

the peculiarities of modern environments. The frame-

work in (Anisetti et al., 2018), in fact, does not fully

satisfy property extensibility and needs to be modi-

ﬁed to target those private deployments not directly

reachable from the outside (e.g., traditional private

corporate networks and private clouds). The sim-

plest solution of moving assurance controls to the

private network, directly connecting them to the tar-

get system, is not viable because it would cause

the violation of different requirements in Section 2.

For instance, it would affect properties transparency

and non-invasiveness, since one or more backdoors

should be coded in the target system, also affecting

property safety. It would also interfere with property

adaptivity constraining the ability of adapting the pro-

cess to changes in the system. It would also increase

costs, violating property lightness.

The framework in this paper aims to provide a

lightweight solution based on Virtual Private Network

(VPN) that embraces peculiarities of distributed sys-

tems, including cloud, microservices, and traditional

private networks. In particular, it adopts a layer-3

VPN that connects the framework with the private

deployments under veriﬁcation (i.e., the target net-

works).

The architecture of the new assurance framework

is presented in Figure 1, adding three components to

the one in (Anisetti et al., 2018): VPN Server, VPN

Client, and VPN Manager. In a nutshell, different

VPN Servers are installed within the framework, each

one responsible to handle isolated VPN tunnels with

client devices placed in the target networks. A sin-

gle VPN connection consists of a VPN Client directly

connected to the target network, and a VPN Server in-

stalled in the framework. The framework manages

an assurance process (Section 5) that consists of a

set of evaluation rules (evaluations in the following).

Each evaluation is a Boolean expression of test cases,

which are evaluated on the basis of the evidence col-

lected by probes and meta-probes. Probes are self-

contained test scripts that assess the status of the given

target by collecting relevant evidence on its behavior.

They return as output a Boolean result indicating the

success or failure of the test case. Meta probes are de-

ﬁned as probes collecting meta-information, such as

the response time of a service. The framework com-

ponents are summarized in the following.

Execution Manager manages the assurance pro-

cess. Upon receiving an evaluation request, it se-

lects the relevant probes and executes them. There

are two types of Execution Managers: one tar-

geting public clouds (Public Execution Manager),

and one targeting private deployments (Private

Execution Manager). The only difference be-

tween them is the way in which trafﬁc is routed

to the destination.

Evidence Analyzer produces the overall result of an

evaluation by collecting the results of the single

test cases composing the evaluation, and validat-

ing them against the Boolean expression of the

evaluation.

Evidence Database stores the results of probe exe-

cution, including both the collected evidence and

the Boolean results.

Dashboard is the user interface used to conﬁgure

new evaluations and access their results.

VPN Server is a dedicated VM running the VPN

software. It handles several VPN tunnels, one for

each private network, which are strictly isolated.

SECRYPT 2020 - 17th International Conference on Security and Cryptography

100

DashboardEvidence Analyzer Evidence DB

Private Execution

Manager

Public Execution

Manager

Private network

Private target

Target service

Target

platform

Public

cloud

Public

cloud

Cloud API

VPN

Cloud API

(Meta)-

Probe

(Meta)-

Probe

VPN Manager

VPN Server

VPN Client

Figure 1: Our framework architecture. Double line rectangles highlight new components.

It acts as a default gateway for multiple Private

Execution Managers.

VPN Client is physically located into the target net-

work. It establishes a VPN connection with the

VPN Server in the framework, traversing the ﬁre-

wall protecting the private network.

VPN Manager is a REST API service that manages

the automatic conﬁguration of the VPN. It auto-

matically generates conﬁguration ﬁles and han-

dles all activities needed to manage VPN connec-

tions.

VPN Client and VPN Server are the stubs mediat-

ing the communication between the target system and

the framework, respectively. They act as intermedi-

aries supporting protocol translation and VPN work-

ing, and interacting with the VPN Manager for the

channel conﬁguration.

Example 3.1 Let us consider an assurance evalua-

tion targeting a public website composed of two test

cases chained with a logic AND: i) a test case eval-

uating compliance against Mozilla best practices for

websites and ii) a test case evaluating the proper con-

ﬁguration of HTTPS. The Execution Manager man-

ages the assurance process as follows. Two probes

are executed to collect the evidence needed to evalu-

ate the two test cases, producing two Boolean results.

Those results are then evaluated by the Evidence Ana-

lyzer according to the evaluation formula, a conjunc-

tion (AND) of test cases i) and ii). As such, the overall

evaluation is successful if and only if both test cases

succeed.

4 VPN-BASED METHODOLOGY

The assurance methodology implemented by the

framework in Figure 1 builds on Virtual Private Net-

work (VPN) to address the must-have requirements

(i.e., transparency, non-invasiveness, safety, and ex-

tensibility) in Section 2. The goal is to provide

a VPN-based solution that smoothly integrates our

framework with the target system. In the following

of this section, we brieﬂy present the basis of VPN

and discuss the reasons why it cannot be used as is to

achieve our requirements; we then describe our VPN-

based solution and how it differentiates from a com-

mon VPN conﬁguration.

4.1 VPN in a Nutshell

Virtual Private Network (VPN) stands for a set of

technologies used to build overlay networks over

the public network. It provides hosts with remote

access to a corporate network, or connects several

geographically-distributed networks like they are sep-

arated by one router (Alshalan et al., 2016).

In this paper, we focus on Site-to-Site VPN, where

several networks are connected using the VPN –

rather than connecting one single host with a remote

network. Usually there is one host per network con-

nected to the VPN that acts as a VPN gateway, me-

diating trafﬁc between internal hosts within its net-

work and other networks. It routes trafﬁc coming

from internal hosts to the other VPN gateways and

back. VPN gateways are called either VPN clients or

VPN servers, where servers can handle connections

to multiple clients, while a client establishes a single

tunnel with a server.

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

101

Private network

Private target

(host)

Private target

(private

cloud)

VPN Server

VPN

Client

Gateway

VPN

VPN Manager

Framework

Figure 2: Architecture of VPN-Based Solution.

VPNs usually combine a virtual network interface

card (virtual NIC) and a socket-like connection. A

virtual NIC is a NIC that has no physical correspon-

dence, and is associated with a userspace process –

in this case the VPN software. Packets sent by such

process to its virtual NIC are received by the oper-

ating system (OS), and further processed just like a

real network packet. At the same time, the OS can

send packets to it, and the VPN software, through its

NIC, will be the receiver. The socket-like connection

is used to transmit packets between VPN gateways us-

ing a cryptographic VPN protocol. The virtual NIC

is used to send and receive packets coming from and

whose destination is the host’s network.

Virtual NICs of the same VPN have IP addresses

belonging to the same subnet, called VPN subnet.

When the operating system of the VPN gateway han-

dles a packet whose destination is a host in the VPN

subnet, it sends the packet to the local virtual NIC,

like a normal routing operation. Two sets of routing

rules have to be deﬁned: i) on each network, a rule on

the default gateway that speciﬁes to route trafﬁc for

other networks to the local VPN gateway; ii) on each

VPN gateway, a rule that speciﬁes to route trafﬁc for

other networks to the local virtual NIC.

Our ﬁnal goal is to realize a Site-to-Site VPN be-

tween the framework and the private targets. How-

ever, a traditional VPN implementation does not per-

mit to address many of the requirements in Section 2.

The aforementioned routing rules, in fact, must be

installed on both sides of the communication. Set-

ting up these routes on the targets’ default gateways

requires access to the devices to alter their conﬁg-

urations. This violates properties non-invasiveness,

transparency, and safety. Our approach avoids this,

by adding several conﬁgurations on top of a standard

VPN setup.

4.2 VPN-based Solution

Figure 2 presents the VPN-based solution at the ba-

sis of our assurance framework. This solution is

composed of two main parts: i) the mapping be-

tween the assurance framework and the private target

(framework-to-system mapping), ii) the management

of IP conﬂicts between the framework network and

the private networks (conﬂict-resolution protocol).

4.2.1 Framework-to-System Mapping

The mapping between the assurance framework and

the private target builds on two components: VPN

Client and VPN Server.

VPN Client establishes a VPN connection with the

server, exposing its network to the framework. It re-

alizes a Client-side NAT that avoids setting up routing

rules on the target network. The issue is that packets

generated by the framework and injected by the VPN

Client into the target network have a source IP address

belonging to the framework network. As such, re-

sponses to such packets would be routed to the target

network default gateway (because they appertain to a

different network than the current one) instead of the

VPN Client. To address this, we propose a lightweight

approach based on network address translation (NAT),

which does not require to conﬁgure default gateways.

Once packets are received by the VPN Client from the

framework through the VPN, it translates their source

IP address in the VPN Client IP address. Since this be-

longs to the same subnet of the target hosts, no routes

need to be conﬁgured. Responses can directly reach

the VPN Client, where the destination IP address of

the packets is translated back. We implemented this

address translation with nftables, available in Linux-

based operating systems.

VPN Server handles VPN tunnels with several

clients; each tunnel is isolated to each other. It imple-

ments a Server-side NAT, to provide higher dynam-

ics. There are two problems behind Server-side NAT,

both involving routing conﬁguration. On one side,

VPN Clients need to know the network IP address

of the framework (Section 4.1), on the other side,

these routes must be known a priori, an assumption

not trivial in our scenario. The network IP address

of the framework, in fact, can change, for example,

if the framework moves to a different cloud provider

or for security reasons. We address the aforemen-

tioned problems by setting up different NAT rules on

SECRYPT 2020 - 17th International Conference on Security and Cryptography

102

INPUT

s ∈ S: VPN Server

: network to map

OUTPUT

: mapped version of n

MAP NET

available nets ← db query select(s);

if length(available nets) != 0 then

pair ← havailable nets[0], n

db query insert(pair);

else Error();

return pair;

INPUT

. j: j-th IP address ∈ network n

OUTPUT

. j: j-th corresponding

IP address ∈ network n

MAP IP

← net id(n

. j);

host id ← host id(n

, n

. j);

← get corresponding net(n

);

. j ← build address(n

, host id);

return n

. j;

INPUT

.k: k-th IP address ∈ network n

OUTPUT

.k: k-th corresponding

IP address ∈ network n

REMAP IP

← net id(n

.k);

host id ← host id(n

, n

.k);

← get corresponding net(n

);

.k ← build address(n

, host id);

return n

.k;

Figure 3: IP Mapping: Pseudocode.

the VPN Server. They modify packets coming from

the framework just before being received by the vir-

tual NIC of the VPN software. These rules change

the source IP address of packets by replacing it with

the virtual NIC IP address of the server. Thus, packets

received by a VPN Client have a source IP address be-

longing to the current VPN subnet. Then, correspond-

ing responses generated by the target hosts, after the

application of Client-side NAT, have a destination IP

address appertaining to the VPN subnet. Recalling

that a VPN Client knows how to handle packets gen-

erated – or appearing to be generated – directly from

the VPN subnet, the VPN Client OS can route those

packets to the local virtual NIC, without additional

conﬁgurations. They are then received by the VPN

software and ﬁnally sent to the server. Server-side

NAT is implemented as a set of nftables rules on the

VPN Servers.

4.2.2 Conﬂict-resolution Protocol

A mandatory requirement for a Site-to-Site VPN is

that the network IP addresses of each participating

network must be non-conﬂicting. Guaranteeing this

assumption is necessary to allow a single VPN server

to connect multiple networks together – in our case

to allow a single VPN Server to handle several tar-

get networks. In corporate VPNs, it is trivial to assert

this property, since the networks are under the con-

trol of the same organization. This assumption is not

valid in our scenario, where two target networks could

have the same network IP address, or a target network

could conﬂict with the framework one. We propose

an approach called IP Mapping to solve this issue.

IP Mapping is based on the the concept of map-

ping the original network to a new one, called mapped

network and guaranteed to be unique. Each IP ad-

dress of the original network is translated into a new

one, belonging to the corresponding mapped net-

work. This translation is reversible, and the mapped

address is speciﬁed by the framework as the target

when executing a new evaluation. IP Mapping is re-

alized through three functions whose pseudocode is

described in Figure 3. The overall protocol is com-

pletely transparent to the ﬁnal user and works as fol-

lows. For simplicity, we consider an evaluation com-

posed of a single test case.

First, when a new target network is being regis-

tered, the function map net is invoked by the frame-

work, to obtain a non-conﬂicting version of the orig-

inal target network. The pair horiginal, mappedi

is saved into the database. This function is offered

through a REST API by VPN Manager.

When a user issues a new evaluation, it enters

the original target IP address. The framework calls

map ip to obtain its mapped version, and builds the

corresponding test case using this IP address as des-

tination. The test packets are then sent through the

VPN. Function map ip is offered by VPN Manager.

The VPN Client receives the packets and calls

remap ip to get the original version of the destina-

tion IP address of the packets. This address is then set

as the destination address: packets can now be sent to

the target.

When corresponding responses reach back the

VPN Client, the latter invokes map ip to obtain the

mapped version of the current IP source address; the

result is set as the new IP source address. This sec-

ond translation is issued in order to re-apply IP Map-

ping and let packets becoming correct responses to

the ones generated by the framework. Finally, they

are sent along the VPN and reach the framework.

Functions map ip and remap ip are implemented

by a set of NAT rules using nftables.

The soundness of the overall VPN setup passes

from IP Mapping, which, using the terminology in

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

103

Table 1: Comparison of a standard layer-3 VPN and a layer-3 VPN with our modiﬁcations on top.

Standard layer-3 VPN Our approach

Client-side requiring conﬁguration Yes No (Client-side NAT)

Server network known a priori Yes No (Server-side NAT)

Conﬂicting networks Not allowed Allowed (IP Mapping)

Address conﬂict resolution Manual Automatic (VPN Manager)

Plug-and-play integration No Yes

Figure 3, must support the following properties.

1. Mapping uniqueness: let A ⊆ n

× S; ∀ a

, a

∈

A, (a

.s = a

.s ∧ a

6= a

) ⇒ (a

.n 6= a

.n)

2. Mapping correctness: ∀ n

∀ address

∈ n

remap ip(map ip(address)) =

map ip(address)

−1

3. Implementation correspondence: ∀ n

, ∀ address

∈ n

, map ip

(address) = map ip

(address)

The ﬁrst property expresses that no conﬂicts can

happen, that is, two mapped networks with the same

network IP address attached to the same VPN Server

cannot exist. The second property expresses the re-

versibility of the translation process. It guarantees

that a response to mapped packets generated by the

framework is correct, that is, the source IP address of

a response is equal to the destination IP address of a

request. The third property expresses the need of hav-

ing two implementations of map ip (as a REST API

or NAT rule) with the same behavior. We note that

the pseudocode shown in Figure 3 is a possible im-

plementation of the three functions.

Table 1 summarizes the differences between a

standard VPN and the one described in this paper.

Our solution does not require any conﬁgurations on

the target network, thanks to Client-side NAT; it also

does not require to know the network IP address of

the framework, thanks to Server-side NAT. Moreover,

the networks participating in the VPN can have con-

ﬂicting IP addresses, which are automatically disam-

biguated by IP Mapping and VPN Manager. To con-

clude, our solution allows a plug-and-play integration

between the framework and the target network.

5 ASSURANCE PROCESS

The overall assurance process is composed of three

phases: i) connection setup, ii) assurance request

(Figure 4(a)), iii) assurance response (Figure 4(b)).

We note that, in Figure 4, we denote Mapped Address

and Original Address as MA and OA, respectively.

Connection Setup. It starts with the user register-

ing the net ID of a new network in the framework.

The framework calls the REST API map net and re-

trieves the mapped version of the input network. As

described in Section 4.2.2, this mapping is stored in

the framework database and triggers the creation of

a new VPN Client. VPN Manager also conﬁgures

VPN Server to support connections from that client.

The client device is then moved into the correct loca-

tion and connected to the server. For the sake of dis-

cussion, we consider the following sample network

conﬁguration: framework net ID 192.168.1.0/24,

target net ID 192.168.50.0/24, mapped target net

ID 192.168.200.0/24, and VPN subnet net ID

10.7.0.0/24; we also consider an evaluation with a

single test case.

Assurance Request. The assurance request in Fig-

ure 4(a) starts with the user submitting an evaluation

request to the framework (Step (1) in Figure 4(a)),

specifying the IP address of the target

(192.168.50.100 in our example). The frame-

work then calls the VPN Manager REST API map ip

(Step (2)), obtaining the mapped version of target ad-

dress (192.168.200.100). Private Execution Man-

ager executes the probe corresponding to the re-

quested test case against the mapped IP address. The

packets generated by the probe are then sent to the

VPN Server. Upon receiving them, VPN Server ap-

plies Server-side NAT (Step (3)), which changes the

source IP address of the packets to its virtual NIC

address (10.7.0.1). Modiﬁed packets are then sent

through the VPN, ﬁnally reaching VPN Client. At

this point, VPN Client executes function remap ip

(Step (4)), which replaces the destination IP address

of the packets with their original version. It then ap-

plies Client-side NAT (Step (5)), which changes the

source IP address from the VPN Server virtual NIC

address to its IP address (192.168.50.30). Finally,

test packets reach their target.

Assurance Response. The assurance response pro-

cess in Figure 4(b) starts when the test target sends

back response packets to the VPN Client. This phase

applies the assurance request steps in the reverse or-

der, to correctly forward responses to the probe. VPN

Client ﬁrst executes the reverse of Client-side NAT,

by replacing the destination IP address of the pack-

ets with the VPN Server virtual NIC (Step (1) in Fig-

SECRYPT 2020 - 17th International Conference on Security and Cryptography

104

Framework network: 192.168.1.0/24

Target network: 192.168.50.0/24

Mapped target network: 192.168.200.0/24

VPN subnet: 10.7.0.0/24

Execution Manager

192.168.1.25

IP PACKET

SRC:192.168.1.25

DST:192.168.200.100

NEW TEST

TO: 192.168.50.100

NEW TEST

TO: 192.168.200.100

(2) map_ip(192.168.50.100)

IP PACKET

SRC:10.7.0.1

DST:192.168.200.100

(1) Test creation

IP PACKET

SRC:10.7.0.1

DST:192.168.200.100

IP PACKET

SRC:10.7.0.1

DST:192.168.50.100

IP PACKET

SRC:192.168.50.30

DST:192.168.50.100

TEST TARGET

OA: 192.168.50.100

MA: 192.168.200.100

192.168.1.30

Virtual NIC: 10.7.0.1

(4)remap_ip(192.168.200.100)(5)client-side NAT

Target network

Framework network

(3) server-side NAT

VPN Server

VPN Client

OA: 192.168.50.30, MA:192.168.200.30, Virtual NIC: 10.7.0.2

(a) Assurance request

Execution Manager

192.168.1.25

IP PACKET

SRC:192.168.200.100

DST:192.168.1.25

IP PACKET

SRC:192.168.200.100

DST: 192.168.1.25

IP PACKET

SRC:192.168.200.100

DST:10.7.0.1

IP PACKET

SRC:192.168.50.100

DST:10.7.0.1

IP PACKET

SRC:192.168.50.100

DST:192.168.50.30

TEST TARGET

OA: 192.168.50.100

MA: 192.168.200.100

192.168.1.30

Virtual NIC: 10.7.0.1

(2)map_ip(192.168.50.100)

(1)reverse client-

side NAT

Target network

Framework network

(3) reverse server-side

NAT

VPN Server

VPN Client

OA: 192.168.50.30, MA: 192.168.200.30, Virtual NIC: 10.7.0.2

(4) evaluate, save

and show results

(b) Assurance response

Figure 4: Packet ﬂow: (a) assurance request, (b) assurance response.

ure 4(b)). It then applies map ip to change the source

IP address (192.168.50.100) with the correspond-

ing mapped version (192.168.200.100) (Step (2)).

Next, packets are forwarded to the VPN Server. Upon

their reception, VPN Server applies the reverse of

Server-side NAT (Step (3)). This step changes the des-

tination address of the packets from the address of the

VPN Server virtual NIC (10.7.0.1) to the address

of the Execution Manager (192.168.1.25). Finally,

packets reach the probe, and results are evaluated and

stored (Step (4)).

It is important to note that the involvement of users

in this process is very limited; users need only to pro-

vide the IP address of the target. The framework then

manages IP Mapping and performs every step in a

transparent way, guaranteeing transparency and non-

invasiveness of the process, avoiding any conﬁgura-

tions on the target system.

6 EXPERIMENTAL EVALUATION

We provide a performance evaluation of our frame-

work and corresponding assurance process.

6.1 Settings

Our framework has been realized on top of Open-

VPN, a ﬂexible and open-source VPN solution that

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

105

permits to tune every aspect of a VPN tunnel. In par-

ticular, we conﬁgured a layer-3 VPN, using TCP as

the encapsulating protocol, to maximize the probabil-

ity of traversing ﬁrewalls in the path from the frame-

work to the target system. Client-side NAT, Server-

side NAT and IP Mapping have been implemented

as NAT rules with nftables. Execution Manager and

VPN Server have been installed in two virtual ma-

chines (VMs) running the operating system CentOS 7

x64, both equipped with 1 CPU and 4 GBs of RAM.

VPN Server runs OpenVPN version 2.4.6 and nfta-

bles version 0.8. Both VMs have been deployed on a

Dell PowerEdge M360 physical host that features 16

CPUs Intel

 Xeon

 CPU E5-2620 v4 @ 2.10 GHz

and 191 GBs of RAM.

The target system has been deployed on AWS EC2

and was composed of two virtual machines t2.micro,

both with 1 vCPU and 1 GB of RAM. The ﬁrst one,

VPN Client, with operating system Ubuntu 18.04 x64,

OpenVPN version 2.4.7, and nftables version 0.8. The

second one, test target, with operating system Ubuntu

16.04 x64 offering WordPress version 5.2.2.

We ﬁnally setup two experiments with the goal of

computing the overhead of our solution, which vary

the target system deployment: i) public deployment

exposing the target on the public network, ii) private

deployment using the approach in Section 4.2. We

run the same evaluations measuring the overhead our

approach adds on top of public deployment veriﬁca-

tion.

6.2 Performance and Discussion

We executed the following evaluations against the two

deployments of the target system.

• Infowebsite that extracts as much information as

possible from a target website. It is denoted as E1

in Figure 5.

• Observatory-Compliance that checks whether a

website has implemented common best prac-

tices, such as HTTPS redirection and cross-site-

scripting countermeasures. It is denoted as E2 in

Figure 5.

• SSH-Compliance that checks the compliance of

a SSH conﬁguration against Mozilla SSH guide-

lines. It is denoted as E3 in Figure 5.

• TLS-strength that evaluates whether the TLS

channel has been properly conﬁgured, such as

avoiding weak ciphers and older versions of the

protocol. It is denoted as E4 in Figure 5.

• WordPress-scan that scans the target WordPress-

based website looking for WordPress-speciﬁc vul-

nerabilities. It is denoted as E5 in Figure 5.

E1 E2 E3 E4 E5

Execution Time (s)

Evaluation

Public Target

Private Target

Figure 5: Execution times of evaluations E1 – Infowebsite,

E2 – Observatory-Compliance, E3 – SSH-Compliance, E4 –

TLS-strength, E5 – WordPress-scan.

We chose these evaluations to maximize test cov-

erage and diversity, from the evaluation of web re-

sources (Infowebsite, Observatory-Compliance), to

the evaluation of protocol conﬁgurations (SSH-

Compliance, TLS-strength) and speciﬁc applications

(WordPress-scan). Each evaluation was executed 10

times and the average time was computed. Figure 5

presents the average execution time of evaluations

E1–E5. It shows that, as expected, the execution time

in the private scenario is higher than the same in the

public scenario, with an overhead varying between

≈0.3s and 2s.

More in detail, evaluation E1 (Infowebsite) experi-

enced a very low overhead, less than a second. Eval-

uation E2 (Observatory-Compliance) experienced an

overhead of approximately 1 second. Evaluations

E3, E4 and E5 (SSH-Compliance, TLS-strength, Word-

Press-scan, resp.) experienced a higher overhead,

approximately 2 seconds, increasing execution time

from ≈2s to ≈4s for E3, from ≈8s to ≈10s for E4

and from ≈12s to ≈14s. Overall, the increase in the

execution time was globally under control, never ex-

ceeding 2 seconds. This overhead can be tolerated in

all scenarios supporting requirements in Section 2.

To conclude, there is a subtlety to consider when

an assurance process for hybrid systems is concerned:

the accuracy of the retrieved results. There could be

some cases in which the evidence collected by a probe

on a public endpoint is different from the one col-

lected by the same probe on a private endpoint. For

instance, evaluation E1 (Infowebsite), in the private

scenario, failed to discover the version of the target

WordPress website. This was due to a partial incom-

patibility between the probe implementation and our

VPN-based solution. Being our approach probe-inde-

pendent, this issue can be solved by reﬁning the probe

associated with Infowebsite. In our experiments, eval-

SECRYPT 2020 - 17th International Conference on Security and Cryptography

106

Table 2: Comparison of frameworks for security assurance with the one in this paper.

References Transparency Non Invasiveness Safety Continuity Lightness Adaptivity

(Alcaraz Calero and Aguado, 2015) 3 ∼ ∼ 3 7 3

(Cheah et al., 2018) ∼ 7 ∼ 7 7 7

(Ciuffoletti, 2016) 3 ∼ 3 3 3 ∼

(De Chaves et al., 2011) ∼ ∼ 3 3 7 ∼

(Elsayed and Zulkernine, 2018) ∼ 3 ∼ 3 7 ∼

(Jahan et al., 2019) 7 7 ∼ 3 7 3

(Ouedraogo et al., 2010) 7 7 7 3 7 7

(Povedano-Molina et al., 2013) 3 ∼ 7 ∼ ∼ 7

(Wu and Marotta, 2013) 7 7 7 7 3 3

(Anisetti et al., 2018) 3 ∼ ∼ 3 ∼ ∼

This paper 3 3 ∼ 3 ∼ 3

(a) Process requirements

Reference Evidence-based veriﬁcation Extensibility Multi-layer Scalability

(Alcaraz Calero and Aguado, 2015) ∼ 7 3 ∼

(Cheah et al., 2018) 3 7 ∼ 7

(Ciuffoletti, 2016) ∼ 7 ∼ 3

(De Chaves et al., 2011) ∼ ∼ ∼ ∼

(Elsayed and Zulkernine, 2018) ∼ 7 ∼ 3

(Jahan et al., 2019) 7 7 7 7

(Ouedraogo et al., 2010) 7 ∼ ∼ ∼

(Povedano-Molina et al., 2013) ∼ 7 3 3

(Wu and Marotta, 2013) 7 ∼ ∼ 7

(Anisetti et al., 2018) 3 7 3 ∼

This paper 3 ∼ 3 ∼

(b) Framework requirements

uation E1 was the only experiencing such problem,

while the other evaluations have been able to collect

the same evidence in both private and public deploy-

ments. We leave the analysis of this issue for our fu-

ture work.

7 COMPARISON WITH

EXISTING SOLUTIONS

Many solutions for security assurance have been pre-

sented in literature, moving from software-based sys-

tems (Herrmann, 2002) to service-based environ-

ments (Ardagna et al., 2015), providing certiﬁcation,

compliance, and audit solutions based on testing and

monitoring. We analyzed the main assurance frame-

works and processes, which can be classiﬁed accord-

ing to the following categories: monitoring-based,

test-based and domain-speciﬁc. Table 2 provides a

comparison of these frameworks, including the one in

this paper, with respect to requirements in Section 2.

Monitoring-based Frameworks. (Aceto et al.,

2013) provided a comprehensive survey of assurance

solutions based on monitoring. They ﬁrst consid-

ered property intrusiveness, which is similar to our

requirements transparency and non-invasiveness, and

found that many commercial monitoring tools do not

address such property. They then considered require-

ment lightness, because monitoring tends to be ex-

pensive in term of resource consumption. Two mon-

itoring frameworks have been presented in (Alcaraz

Calero and Aguado, 2015; De Chaves et al., 2011),

both building on monitoring tool Nagios. Due to the

intrinsic nature of monitoring, these frameworks can

easily satisfy the requirement continuity. Moreover,

the work in (Alcaraz Calero and Aguado, 2015) can

achieve a very good adaptivity and offers a monitor-

ing platform both for cloud providers and users. Nev-

ertheless, they require a signiﬁcant effort in terms of

setting up the monitoring infrastructure and resources

for maintaining it, thus violating requirements non-

invasiveness and lightness. Framework in (De Chaves

et al., 2011) has also proven to suffer of extensibil-

ity and scalability issues (Taherizadeh et al., 2018).

(Povedano-Molina et al., 2013) described a monitor-

ing framework called DARGOS, built with scalabil-

ity and ﬂexibility in mind. Being fully distributed, it

supports scalability and can be enriched with more

sensors. However, being speciﬁcally tailored for the

cloud, it cannot be easily adapted to other scenarios.

(Ciuffoletti, 2016) presented a novel approach, where

a simple, cloud-independent API-based solution has

been used to conﬁgure monitoring. Such cloud-ag-

nosticism is realized through an OCCI (Open Cloud

Computing Interface) extension, designed towards

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

107

Monitoring-as-a-Service. The author also proposed

how the cloud providers should implement such func-

tionalities in their backends.

Test-based Frameworks. (Wu and Marotta, 2013)

presented a work-in-progress testing-based frame-

work that instruments client binaries to perform cloud

testing. The main issue is that binaries instrumen-

tation may not be always feasible, and might also

introduce undesired behavior in modiﬁed programs.

As such, the framework fails to satisfy requirements

transparency, non-invasiveness, and safety. (Oue-

draogo et al., 2010) presented a framework that uses

agents to perform security assurance, although agents

themselves need to be properly secured. (Green-

berg et al., 1998) claimed that, to protect hosts from

agent misuse or attacks, several techniques need to

be properly employed. Agents also pose a mainte-

nance problem: they have to be kept updated, and

things can only become worse as the number of

agents increases. Also, they introduce substantial

costs since they need to be physically installed on

each host/device to be assessed and coordinated, in-

troducing not-negligible network trafﬁc. For these

reasons, the agent-based framework in (Ouedraogo

et al., 2010) does not satisfy requirements trans-

parency, non-invasiveness and safety. (Jahan et al.,

2019) discussed MAPE-SAC, a conceptual approach

for security assurance of self-adaptive systems, where

the system itself changes, and security requirements

must adapt to these changes. While it is not possi-

ble to completely evaluate our requirements due to

the lack of a real, implemented framework, MAPE-

SAC fulﬁlls requirements adaptivity and continuity.

A different solution has been given in our work

in (Anisetti et al., 2018), where we described the

cloud-ready framework brieﬂy summarized at the be-

ginning of Section 3. As already discussed, the pro-

posed approach is based on probes and meta-probes,

and fails to address requirements extensibility, non-

invasiveness, and partially, safety.

Domain-speciﬁc Frameworks. The most recent cat-

egory of assurance frameworks. (Elsayed and Zulk-

ernine, 2018) described a distributed framework for

monitoring cloud analytics applications, based on an-

alyzing logs produced by such applications. The

proposed approach requires very few conﬁgurations

at the cloud side, and can be offered through the

Security-as-a-Service paradigm. (Cheah et al., 2018)

considered the automotive world, where cases are

generated after evaluating the severity of threats.

Threats are found through threat modeling and con-

ﬁrmed with a penetration testing. The usage of pene-

tration testing violates requirement non-invasiveness.

Often, being tailored for a speciﬁc domain, solutions

in this category cannot claim property extensibility.

To conclude, the comparison in Table 2 shows

that the existing frameworks (and corresponding pro-

cesses) do not even come close to addressing the re-

quirements in Section 2. In general, existing solutions

mainly target continuous evaluation and multi-layer

infrastructures, as well as transparency and adaptiv-

ity, failing to achieve non-invasiveness, safety, light-

ness, and extensibility. The framework in this paper,

instead, provides a ﬁrst boost in this direction ad-

dressing, at least partially all requirements in Table 2.

Following the comparison therein, this paper leaves

space for future work. We will ﬁrst aim to extend our

framework towards Big Data and IoT environments,

further improving extensibility, lightness, and scala-

bility. We will also focus on strengthening the safety

of the framework and its components, for example the

Execution Manager that can easily become a single

point of failure/attack.

8 CONCLUSIONS

Security assurance is increasingly adopted as the solu-

tion to verify whether a distributed system holds some

security properties and behaves as expected. Cur-

rent approaches and, when available, frameworks of-

ten target a speciﬁc system lacking extensibility and

have a not-negligible impact on the functioning of the

system target of the veriﬁcation. These issues rep-

resent big hurdles towards assurance adoption, espe-

cially when hybrid systems are considered. In this

paper, we presented a VPN-based assurance frame-

work that smoothly integrates with hybrid systems,

from private networks to public clouds. The frame-

work implemented an assurance process with limited

impact and costs on the target system, while provid-

ing a safe and scalable approach. The present work

leaves space for further work. First, our VPN-based

approach can be extended to increase extensibility, ad-

dressing more domains and their peculiarities (e.g.,

IoT). Second, it can be reﬁned to increase the safety

of the framework and its core components (i.e., Exe-

cution Manager, VPN Server).

ACKNOWLEDGMENTS

Research supported, in parts, by EC H2020 Project

CONCORDIA GA 830927 and Universit

a degli Studi

di Milano under the program “Piano sostegno alla

ricerca”.

SECRYPT 2020 - 17th International Conference on Security and Cryptography

108

REFERENCES

Aceto, G., Botta, A., de Donato, W., and Pescap

e, A.

(2013). Cloud monitoring: A survey. Computer Net-

works, 57(9):2093 – 2115.

Alcaraz Calero, J. M. and Aguado, J. G. (2015). Monpaas:

An adaptive monitoring platformas a service for cloud

computing infrastructures and services. IEEE TSC,

8(1):65–78.

Alshalan, A., Pisharody, S., and Huang, D. (2016). A

survey of mobile vpn technologies. IEEE COMST,

18(2):1177–1196.

Anisetti, M., Ardagna, C., Damiani, E., and Gaudenzi, F.

(2016). A certiﬁcation framework for cloud-based ser-

vices. In Proc. of ACM SAC, Pisa, Italy.

Anisetti, M., Ardagna, C., Damiani, E., and Gaudenzi, F.

(2017). A semi-automatic and trustworthy scheme for

continuous cloud service certiﬁcation. IEEE TSC.

Anisetti, M., Ardagna, C., Damiani, E., Ioini, N. E., and

Gaudenzi, F. (2018). Modeling time, probability, and

conﬁguration constraints for continuous cloud service

certiﬁcation. Computers & Security, 72:234 – 254.

Ardagna, C., Asal, R., Damiani, E., and Vu, Q. (2015).

From security to assurance in the cloud: A survey.

ACM CSUR, 48(1):2:1–2:50.

Baldini, G., Skarmeta, A., Fourneret, E., Neisse, R., Leg-

eard, B., and Le Gall, F. (2016). Security certiﬁcation

and labelling in internet of things. In 2016 IEEE 3rd

World Forum on Internet of Things (WF-IoT).

Cheah, M., Shaikh, S. A., Bryans, J., and Wooderson, P.

(2018). Building an automotive security assurance

case using systematic security evaluations. COSE,

77:360 – 379.

Ciuffoletti, A. (2016). Application level interface for a

cloud monitoring service. CS&I, 46:15 – 22.

De Chaves, S. A., Uriarte, R. B., and Westphall, C. B.

(2011). Toward an architecture for monitoring private

clouds. IEEE Comm. Mag., 49(12):130–137.

Elsayed, M. and Zulkernine, M. (2018). Towards security

monitoring for cloud analytic applications. In Proc. of

IEEE BigDataSecurity/HPSC/IDS 2018, Omaha, NE,

USA.

Greenberg, M. S., Byington, J. C., and Harper, D. G.

(1998). Mobile agents and security. IEEE Comm.

Mag., 36(7):76–85.

Herrmann, D. (2002). Using the Common Criteria for IT

security evaluation. Auerbach Publications.

Jahan, S., Marshall, A., and Gamble, R. (2018). Self-

adaptation strategies to maintain security assurance

cases. In Proc. of IEEE SASO 2018, Trento, Italy.

Jahan, S., Pasco, M., Gamble, R., McKinley, P., and Cheng,

B. (2019). Mape-sac: A framework to dynamically

manage security assurance cases. In Proc. of IEEE

FAS*W 2019.

Ouedraogo, M., Mouratidis, H., Khadraoui, D., and Dubois,

E. (2010). An agent-based system to support assur-

ance of security requirements. In Proc. of SSIRI 2010,

Singapore.

Povedano-Molina, J., Lopez-Vega, J. M., Lopez-Soler,

J. M., Corradi, A., and Foschini, L. (2013). Dargos: A

highly adaptable and scalable monitoring architecture

for multi-tenant clouds. FGCS, 29(8):2041–2056.

Saltzer, J. H. and Schroeder, M. D. (1975). The protection

of information in computer systems. Proceedings of

the IEEE, 63(9):1278–1308.

Taherizadeh, S., Jones, A. C., Taylor, I., Zhao, Z., and

Stankovski, V. (2018). Monitoring self-adaptive appli-

cations within edge computing frameworks: A state-

of-the-art review. JSS, 136:19 – 38.

Teigeler, H., Lins, S., and Sunyaev, A. (2017). Chicken

and egg problem: What drives cloud service providers

and certiﬁcation authorities to adopt continuous ser-

vice certiﬁcation? In Proc. of WISP 2017, Seoul,

South Korea.

West, R. (2008). The psychology of security. Commun.

ACM, 51(4):34–40.

Wu, C. and Marotta, S. (2013). Framework for assess-

ing cloud trustworthiness. In Proc. of IEEE CLOUD

2013, Santa Clara, CA, USA.

Stay Thrifty, Stay Secure: A VPN-based Assurance Framework for Hybrid Systems

109