DECIDE: DevOps for Trusted, Portable and Interoperable

Multi-Cloud Applications towards the Digital Single Market

Juncal Alonso, Leire Orue-Echevarria, Marisa Escalante and Gorka Benguria

Tecnalia Research & Innovation, Parque Científico y Tecnológico de Bizkaia, Edificio 700, 48160 Derio, Bizkaia, Spain

Keywords: Self-adaptation, Intermediation and Federation of Heterogeneous Resources, Run-Time Monitoring,

Deployment Simulation, Multi-Cloud Architectural Patterns.

Abstract: The main objective of the DECIDE action is to provide a new generation of multi-cloud service-based

software framework, enabling techniques and mechanisms to design, develop, and dynamically deploy

multi-cloud aware applications in an ecosystem of reliable, interoperable, and legal compliant cloud

services. Three use cases will be conducted to validate the proposed approach.

1 INTRODUCTION

The digital transformation from product to service

economy means changes in the companies’

operating environment: they need to transform into

service providers from product providers and be able

to flexibly change their role in the value chain and

markets. In order to be able to foster the change, the

companies IT infrastructure needs to be more

flexible. Cloud services enable this to some degree,

but as such create dependency to external partners

for a company. In a world where new players come,

others disappear, and conditions are continuously

changing, how can the companies be sure that the

architectural decisions that were taken in the past

continue to be the best one? For example, while

developing or migrating a web site, an organization

can decide to build it in a dedicated internal

computer, build it as an instance in a shared internal

computer, build it in a dedicated external computer,

or even build it as an instance in a shared external.

The decision on using one, another, or several

approaches simultaneously is driven by certain

evaluation criteria (e.g. profitability, reliability,

performance, security, legal or even ecological

aspects). Cloud services providers (CSPs)

themselves may fail too, so for the greatest measure

of protection possible, an enterprise may wish to

embark upon a multi-cloud strategy. There are

several multi-cloud solutions available for solving

specific problems, but to date, little attention has

been paid to distributing the cloud risk, and

managing multiple clouds from a single technology

platform. Working with many CSPs means

managing multiple relationships. Currently, most

companies are already in the process of negotiating

multiple contracts with multiple CSPs. That involves

activities such as to analyze their service level

agreements, manage multiple payments, storing

passwords, and so on. Those cumbersome activities

lead to question whether it would be possible to

unify those efforts somehow in order to maximize

both the efficiency and effectiveness of the usage of

services coming from different CSPs. One way to

achieve that is through Cloud Service Broker (CSB).

Gartner defines a cloud services brokerage as a

third-party software that adds value to cloud services

on behalf of cloud service consumers (Gartner,

2016). The major benefit of using CSBs is that these

allow organizations to focus on other critical day-to-

day business needs instead of in how to manage

multiple CSPs. As expressed by Gartner, a viable

intermediator and federator of cloud services can

make it less expensive, easier, safer (also in legal

terms), interoperable and more productive for

companies to discover, aggregate, consume and

extend cloud services, particularly when they span

multiple, diverse cloud services providers in

different EU Member States (Gartner, 2012).

1.1 Multi-Cloud Applications: Main

Challenges and Unsolved Issues

In the context of this paper, a multi-cloud native

application is an application distributed over

heterogeneous cloud resources whose components

Alonso, J., Orue-Echevarria, L., Escalante, M. and Benguria, G.

DECIDE: DevOps for Trusted, Portable and Interoperable Multi-Cloud Applications towards the Digital Single Market.

DOI: 10.5220/0006292403970404

In Proceedings of the 7th International Conference on Cloud Computing and Services Science (CLOSER 2017), pages 369-376

ISBN: 978-989-758-243-1

369

are deployed on different CSPs and still, they all

work in an integrated way and transparently for the

end-user. There are several reasons for deploying an

application in a multi-cloud architecture, the most

important ones being: non-compliance of the CSPs

to the agreed SLAs, avoidance of vendor lock-in,

increasing reliability or improving other QoS

concerns such as increasing performance or security,

and finally, reducing costs. The application types

that would benefit the most from such a multi-cloud

approach are on the one hand, those that are critical

to the business and that need to respond efficiently

to the user’s needs in terms of performance,

reliability and security and on the other hand,

complex applications whose components need to be

distributed over different cloud providers due to

their specific needs and requirements. However, any

application offered as SaaS can benefit from a multi-

cloud architecture. Currently, this is solved by

deploying the same application on several cloud

providers following a master-slave or active-passive

approach. This, however, poses also several risks,

since the synchronization of all the data is critical for

a correct functioning of the application if no data

loss is wanted. The multi-cloud approach presented

in this paper avoids synchronization risks and

guarantees the fulfillment of the application

providers’ requirements, which can range from

maintaining a constant cost structure to a certain

response time, security issues or a certain

performance level.

Developing, deploying and operating multi-

cloud applications present the following challenges:

 Applications need to be responsive to

hybrid/multi-cloud model scenario, in which

an application that is executing in a private

cloud bursts into a public cloud when the

demand for computing capacity spikes. This

implies that the application architecture shall

be re-designed to be “multi-cloud” aware

simplifying the cloud application assembly

and the deployment process. A possible

solution to this problem is to deploy

components of the same application on

multiple clouds distributing the workload

among them, so as to continuously guarantee

the average QoS requirements, dynamically

allocate resources and automatically

reconfigure the deployment configuration

when a Non-Functional Requirement (NFR)

value is not met. Multi-cloud application

solutions have to deal with a set of Non-

Functional Property (NFP) (e.g. performance,

security, availability, governance etc.) of the

individual components given their

complexity and distribution as well as with

the overall NFP of the application including

the communications and the data flow

between components. Even if each CSP

offers means and controls for measuring

NFP, the multi-cloud application has to

ensure integrated non-functional

characteristics across the whole composition.

Therefore, the overall non-functional

characteristics depend on the NFP of the

application components, which in turn

depend on the NFP offered by the cloud

resources they exploit.

 Means shall be provided to manage and

assess cloud deployment alternatives to better

support our cloud re-deployment decisions.

This implies profiling and classifying

application components and cloud nodes, as

well as analyzing and simulating the behavior

of the application under stressful conditions

to support the deployment decision making

process considering additional factors such as

NFR, namely performance, availability,

localization, cost, or risks associated with the

change of cloud resources. Multi-cloud has

value only when the right providers are

selected, whether public or private (combined

into different cloud deployment models), to

meet functional and NFR.

 Existing cloud services shall be made

available dynamically, broadly and cross

border, so that software providers can re-use

and combine cloud services, assembling a

dynamic and re-configurable network of

interoperable, legally secured, quality

assessed (against SLAs) single and composite

cloud services, facilitating the free flow of

data and therefore impacting in the Digital

Single Market (DSM).

1.2 DevOps for Multicloud

Applications: Progress beyond the

State of the Art

1.2.1 DevOps

DevOps refers to the emerging professional

movement and philosophy that advocates for a

collaborative working relationship between

Development and IT Operations, lowering barriers

and silo-based teams, resulting in the fast flow of

planned work, while simultaneously increasing the

reliability, stability, resilience of the production

CLOSER 2017 - 7th International Conference on Cloud Computing and Services Science

370

environment. This is often called the “DevOps

Paradox” (Edwards, 2015): “Going faster brings

higher quality, lower costs, and better outcomes”.

Organizations such as Etsy, Netflix, Facebook,

Amazon, Twitter and Google, but also ING, or

Target, by applying the DevOps philosophy, they

have been able to achieve levels of performance that

were unthinkable even five years ago. DevOps

pivots around three axes (UpGuard, 2016):

processes, people and technology. From the people

perspective, DevOps symbolizes a cultural change

where collaboration and cooperation are key pillars,

and this often results in an increased understanding

to prioritize requests that the business needs. From

the processes perspective, DevOps advocates for a

more agile change processes, with an increased rate

of acceptance for new features, improved quality in

software developments, a decrease in number of

incidents per release and an increased time to market

and velocity to pass from development to

production. Finally, from the technology point of

view, DevOps results in an application with a

reduced number of defects and therefore with more

quality, and in an increased deployment of features.

The authors have analyzed several DevOps

solutions existing on the market and in open source

communities, such as The Eclipse Cloud

Development project (Eclipse Cloud Development;,

2015), IBM Bluemix DevOps Services (IBM

BlueMix, 2016), or Puppet+MCollective (Puppet

labs, 2013) and Terraform (Terraforms, 2015)

among others. From this analysis the authors

conclude that the most of the current so-called

“DevOps” solutions focus mostly on Continuous

Delivery (CD – often also named continuous

deployment in literature), while simultaneously

applying practices like Continuous Integration (CI),

Continuous Quality (CQ). CD practices allow for

automated deployment, while CQ and CI practices

allow for errors to be caught in an earlier phase of

the development cycle which are thus cheaper to

solve and with less rework, accompanied by a

configuration management system. However, the

tools hereby presented while aiming at cloud

deployments, they do not fully support the multi-

cloud approach.

1.2.2 Deployment Simulation

There are two main sources in the state of the art

related to this topic, namely the model based testing

and the cloud resource simulation.

The use of models for designing, simulating and

testing such systems is currently one of the strongest

industrial trends with significant impact on the

overall development and quality assurance

processes. Most of developments in that area, such

as Fokus!MBT (Fraunhofer Fokus, 2015) have been

mainly focused on the testing of the correctness of

the model and not so much in the determination of

the non-functional characteristics of the resulting

system. However, some works have been carried out

on the simulation of performance metrics for grid

computing (Li, 2009); with can be used as a starting

point for the definition of strategies for cloud

resources performance simulation.

In the domain of cloud resource simulation, the

most relevant input is Cloudsim (CLOUDS

Laboratory, University of Melbourne, 2012). It

provides a generalized and extensible simulation

framework that enables seamless modelling,

simulation, and experimentation of emerging Cloud

computing infrastructures and application services.

The usage of Cloudsim is based on coding the use

cases in Java, and the provided example mainly

make reference to IaaS infrastructures.

On other hand, there are several tools in the

market that allow comparing IaaS services, such as

CloudHarmony (CloudHarmony, n.d.), or

Cloudorado (Cloudorado, 2016). There are also tools

such as Cloudsleuth (Cloudsleuth, 2016) that allow

comparing the performance of IaaS and PaaS

providers, but are mostly focused in monitoring

response-time and availability. However, all these

decision support type tools usually ignore the

complexity of multi-cloud environments, where

combinations of cloud services need to be evaluated

as well as non-functional properties.

1.2.3 Modeling of Dynamic and

Reconfigurable Multi-Cloud

Applications

Multi-Cloud is often defined as the serial or

simultaneous use of services from diverse providers

to execute an application (Petcu, 2013). At business

level, Hybrid Cloud is the term commonly used.

Gartner (Mazzuca, 2015) defines hybrid Cloud as

the coordinated use of cloud services across isolation

and provider boundaries among public, private and

community service providers, or between internal

and external cloud services. A number of scenarios

demonstrate these serial or simultaneous interactions

among hybrid heterogeneous private and public

clouds and across all cloud layers (IaaS/PaaS/SaaS)

(ETSI, 2013). Multi-cloud applications engineering

as they are understood by the authors of this paper,

with the application components distributed across

DECIDE: DevOps for Trusted, Portable and Interoperable Multi-Cloud Applications towards the Digital Single Market

371

heterogeneous cloud resources and still seamlessly

interoperating in a single whole, is not a common

practice yet.

1.2.4 Automatic Re-Deployment

The automatic re-deployment of multi-cloud

applications implies several technical issues like 1)

supporting different cloud service interfaces (for

distributing the application workload on different

cloud platforms), 2) monitoring the application

components (for verifying the metrics), and 3)

implement strategies for migrating workload

(namely applications or parts of applications) from

one cloud platform to another.

Cloud computing applications can be monitored

with a very broad set of monitoring solutions that

range from services made available by Software as a

Service (SaaS) providers (e.g. AppDynamics

(APPDYNAMICS, 2016), BMC’s TrueSight Pulse

Monitoring-as-a-Service (BMC, 2016), New Relic

(New Relic, 2016), RackSpace’s Cloud Monitoring

(Rackspace, 2015), etc.), services provided by the

platform hosting the application (e.g. OpenStack

Monasca (OpenStack, 2016), AWS CloudWatch

(Amazon, 2016)) and tools implemented as stand-

alone software packages (e.g. Nagios (Nagios,

2016), Zabbix (Zabbix, 2016)).

Adapting cloud applications for restoring the

expected working conditions can be interpreted and

implemented in several ways. Some cloud platforms

(e.g. Google Cloud Platform (Google Cloud

Platform, 2016) and AWS (Elastic Cloud Gate,

2014)) provide functions for periodically monitoring

the status of a VM and, in case of unavailability, for

restarting it. For sophisticated conditions, it is

possible to adapt the application taking advantage of

a fundamental capability of cloud computing which

is named elasticity.

1.2.5 Cloud Service Brokers

“Cloud Marketplaces” are emerging to offer a

mixture of service management, cloud deployment

automation and application assembly, often in multi-

cloud environments. Cloud providers such as

Amazon WS (Amazon, 2016), HP (HPE, 2016) or

IBM (IBM, 2016) have already launched their own

cloud marketplace services. At the same time, both

commercial solution providers (such as Appcara

AppStack (Appcara, 2015) and Jamcracker Service

Delivery Network (Jamcracker Platform, 2016) and

Open Source initiatives (Ubuntu Juju (Ubuntu,

2016)) are developing solutions that enable the

creation of customized Cloud marketplaces.

2 DECIDE: AN ADVANCED

MULTI-CLOUD SERVICES

BASED FRAMEWORK

2.1 DECIDE Framework: Proposed

Approach

DECIDE focuses on a novel concept of multi-cloud

applications. The application of this new concept

encompasses several challenges that pave the way for

the proposed innovations and for improving the

competitive advantage of DECIDE partners compared

to what the market offers today. The design of

efficient multi-cloud application requires a set of

established architectural patterns. DECIDE will

develop architectural patterns and modeling practices

focused on the description of the system architecture

in terms of cloud resource dependencies as well as in

terms of NFR of the system as a whole. DECIDE will

provide the architect with suggestions on which

pattern has to be applied, how, when and the potential

trade-offs, for multi-cloud applications deployed on

heterogeneous cloud providers configuring different

cloud layers (as in an IoT environment), making use

of heterogeneous resources and services, in some

cases, provided by the DECIDE Advanced Cloud

Service intermediator.(ACSmI) ACSmI will be user

and resource-centric, searching always for the best

opportunistic choices while fulfilling the requirements

set by the user. Moreover, ACSmI will develop its

activities in a context which will be legally secured by

adequate innovative contractual and policy solutions

and will foster cross-border interoperability. The

DECICE OPTIMUS simulation tool will provide on

one hand, the automation of the provisioning

resources and deployment scripts for multi-cloud

native applications based on the modeling of cloud

resources at multiple cloud layers (IaaS, PaaS) and of

multiple CSPs, and on the other, it will profile and

classify the application components, which will be

used to simulate the application behavior under

certain conditions.

Multi-cloud applications demand supporting

tools, such as DevOps, not only for their design,

development and deployment but also for their

operation. In order to remain sustainable, a cloud

based application cannot stop its operation and it is

expected that it is self-adaptive with respect to the

new topology needed to fulfill the users’ requirements

at all times. DECIDE aims to offer a dynamic

monitoring of NFRs as set by the user or potential

SLA violations, which will trigger the self-

adaptability and reconfiguration of the application at

CLOSER 2017 - 7th International Conference on Cloud Computing and Services Science

372

run-time through the DECIDE ADAPT multi-cloud

application self-adaptation tool. It will pro-actively

adjust the running configuration of the application

based on measurements that are derived from the

dynamic monitoring activities of both the application

and the NFPs of the CSPs and cloud offerings where

the application is deployed and making use of.

The assembling of the mentioned novel

components along with other DevOps natural

components (such as continuous integration (CI),

continuous quality (CQ), and continuous delivery

(CD)) will set up a DevOps framework for develop-

ment and operation of multi-cloud native applications

in compliance with the DevOps paradigm.

DECIDE workflow starts from the design of the

multi-cloud native application that is sensitive to the

changeable situation in a multi-cloud based

environment. For that, developers establish a set of

quantitative (i.e. Mean Time Between Failures

(MBTF)), availability, response time, lag, cost,

throughput) and qualitative (i.e. security, location,

financial, low / high technological risk) NFPs that the

application must comply with and uses DECIDE’s

ARCHITECT tool to support the design and

development process of the distributed application

and its components through the architectural

implementation of patterns and the recommendations

derived from the tool on which patterns to apply in

which components. Qualitative NFR will only be

applicable for the selection of the cloud services,

while quantitative will be used for monitoring and

simulation purposes. After the application of this

initial set of multi-cloud based architectural patterns,

the developer follows with the implementation

process (following the CQ, CI and again continuous

architecting DevOps approach). For the

implementation (continuous development, CI, CQ,

CD), DECIDE will integrate open environments such

as Eclipse, Git, Puppet, Chef, Docker, Jenkins, and

Vagrant, among others.

As a next step, DECIDE will support the selection

of the deployment topology and the underlying

selection of the most suitable cloud services through

the OPTIMUS simulation tool. The OPTIMUS tool

will base the simulation on the profiling of the

different components to be considered: profiling of

the multi-cloud application, profiling of the cloud

services to be used (data bases, processing clusters,

etc.), profiling of the communications between nodes,

and profiling of external services to be used by the

multi-cloud application. For the modeling of the

profiling information so that it can be processed,

represented and used, existing technologies such as

CloudML and OpenTosca will be evaluated.

Optimization algorithms such as genetic algorithms,

Harmony search, or Dandelion codes will be used by

OPTIMUS to provide a set of potential combination

of cloud services that fulfills the established user

requirements. Along with these simulation results,

OPTIMUS will provide the developer with

information about the required changes in the

application structure/schema/code to achieve the

required configuration deployment and the

technological risk that each of these configurations

imply (low technological risk or high technological

risk), i.e. moving from an IaaS to a PaaS, move from

one PaaS to another like OpenShift vs. Cloudfoundry.

Once the application is implemented and the

cloud services are selected, the developer needs to

define the service level agreement that the application

will offer to end-users (Multi-Cloud SLA - MCSLA).

This MCSLA will be influenced by the SLAs of the

underlying (combination of) cloud services to be

contracted. DECIDE Multi-cloud native applications

DevOps framework will support the definition of

these composite MCSLAs (Multi Cloud Service

Level Agreement) and the corresponding SLOs

(Service Level Objectives) of the application and the

dependencies and needs on the underlying

(combination of) cloud services in a machine-readable

format for the representation. This composite

MCSLA will be then assessed at run time to check if

it is being accomplished.

To finish this first cycle of the development

phase, the developer will select the deployment

scripts based on the selected configuration from the

simulation phase through the continuous deployment

supporting tools and the architectural patterns for

deployment. Each deployment configuration will be

stored in the multi-cloud native application controller,

maintaining the current deployment configuration

situation as well as the historic of the previous

deployment configuration used, so that they can be

checked in the re-deployment phase.

Once the application has been developed, the

operation phase starts. The application owner

contracts the corresponding (combination of) cloud

services (accomplishing the required MCSLAs) and

deploys the application over different clouds using the

ADAPT continuous deployment tool.

During the application operation phase, the

DECIDE self-adaptation application provisioning tool

will continuously monitor and assess the fulfillment

of the established NFR and MCSLA. If a violation of

any of the former metrics occurs, the self-adaptation

tool through the ACSmI will assess the operation of

the (combination of) cloud services selected and

discard those that are affecting the MCSLA. If the

DECIDE: DevOps for Trusted, Portable and Interoperable Multi-Cloud Applications towards the Digital Single Market

373

application configuration has been established as of

low technological risk, the multi-cloud application

will be self-adaptive and it will be redeployed

automatically, following a new deployment

configuration. In case the application has been

identified as high technological risk, once it has

identified the aspects that are affecting the

malfunctioning of the application, it will alert the

operator and using the OPTIMUS tool it will look for

new (combination of) cloud services to set up a new

deployment schema. The DECIDE application

controller and the continuous deployment supporting

tools will support the selection of the new deployment

scripts (based on the architectural patterns for

deployment), and thus semi-automatically re-deploy

it. DECIDE will also support parallel re-deployment

strategies definition and multiple cloud layers. In this

case the new operation phase will start again

contracting the new services and deploying the new

scripts into the new configuration of cloud services.

2.2 DECIDE: Proposed Technical

Architecture

Figure 1 presents the proposed technical architecture

for DECIDE components.

3 DECIDE TESTS BEDS

The resulting assets of this approach will be

validated in several test beds. They have been

selected bearing in mind that the main beneficiaries

of this framework are, namely developers and

operators of applications that need to be legally

aware and compliant, and need to fulfill high

demanding requirements of performance, availa-

bility and reliability, without reaching high costs.

3.1 Case Study 1: High Availability

High availability (HA) is usually offered in the same

ISP, as a high cost dedicated hardware solution

distributed, when available, in different Datacenters

or in different ISPs using different control panel and

services. The main aim of this use case is to provide

a low-cost HA service that users can manage

through a single control panel. Through DECIDE

ACSmI, any customer will be able to add

redundancy to their system in order to eliminate a

single point of failure. Developers will also be able

to detect failures so as to maintain availability across

the different cloud platforms.

3.2 Case Study 2: e-Health

The second validation case comes from a cloud

solutions provider for health data based applications

and digital services. This provider is based on

England even though it provides the services in

England and the UK, it hosts European wide Clinical

Data Entry Tools, as well as applications that have

users all over Europe. Currently the problems faced

around hosting health data across Europe are:

Figure 1: DECIDE Technical Architecture.

CLOSER 2017 - 7th International Conference on Cloud Computing and Services Science

374

(1) Legal Compliance, (2) Resilience, and (3)

Performance. Often there is a compromise to be

made in one of these areas to facilitate the

implementation of services using health data In

order to solve such challenges, it is of paramount

importance to simulate potential multi-cloud

deployments that comply with the legal, cross-

border and interoperability requirements associated

with e-health data services.

3.3 Case Study 3: Network

Management

Network management is a key aspect of the

operation of the telecom networks as performed by

the Telecom Operators and other players. The

network is crucial to every business, optimizing

network performance effectively is necessary to

achieve a competitive business society. Current

network management tools are limited to the

collection and presentation of a summary of the

status of the network but do not meet present

industry demands. There is a need for a very

dynamic Development and Operation environment

(DevOps) where maintenance activities, Long-term

Network Configuration and Planning as well as

updates of the operation tools and rules have to

implemented and modified continuously with the

business and operation feedback. This need to be

updated on-the-fly but using specific tools to

promote a culture of agile team work between

development, test and operation.

4 FUTURE WORK

Following the approach proposed, next steps will

deal with the specification of the technical design of

the different tools as well as the implementations of

the first prototypes.

These prototypes will be tested into real

industrial use cases in the context of the DECIDE

research action.

5 CONCLUSIONS

This paper presents how DECIDE action provides

a novel DevOps multi-cloud application framework,

enabling techniques and mechanisms to architect,

develop, and dynamically deploy multi-cloud aware

applications in an ecosystem of reliable,

interoperable, and legal compliant cloud services.

DECIDE will support software development

companies in:

1. Enhancing their (multi cloud applications)

development and operations processes,

2. Improving the developers’ and operators’

productivity,

3. While ensuring the application

maintainability, Quality of Experience

(QoE) and Quality of Service (QoS) in its

whole life,

4. Decreasing the time-to-market.

The next activities include the actual

implementation of all components described in this

paper and the validation in the three use cases. For

that, the authors will follow

an iterative and

incremental approach based on SCRUM and in

alignment with a DevOps philosophy. The

prioritisation of the functionalities will come from

the priority analysis of functional requirements and

the use cases.

ACKNOWLEDGEMENTS

The project leading to this paper has received

funding from the European Union’s Horizon 2020

research and innovation programme under grant

agreement No 731533.

REFERENCES

Amazon, 2016. Amazon CloudWatch. [Online]

Available at: https://aws.amazon.com/cloudwatch/

?nc1=h_ls[Accessed 17 November 2016].

Amazon, 2016. AWS Marketplace. [Online]

Available at: https://aws.amazon.com/marketplace

[Accessed March 2016].

Appcara, 2015. Appcara- AppStack. [Online]

Available at: http://www.appcara.com/products/

appstack/[Accessed March 2016].

APPDYNAMICS, 2016. APPDYNAMICS. [Online]

Available at: https://www.appdynamics.com/

[Accessed 17 November 2016].

BMC, 2016. TrueSight Pulse Monitoring-as-a-Service.

[Online] Available at: http://www.bmc.com/

truesightpulse[Accessed 17 November 2016].

CloudHarmony, n.d. CloudHarmony. [Online]

Available at: https://cloudharmony.com/directory

[Accessed 17 November 2016].

Cloudorado, 2016. Cloudorado. [Online]

Available at: https://www.cloudorado.com/

[Accessed 17 November 2016].

CLOUDS Laboratory, University of Melbourne, 2012.

CloudSim: A Framework for Modeling and

Simulation of Cloud Computing Infrastructures and

DECIDE: DevOps for Trusted, Portable and Interoperable Multi-Cloud Applications towards the Digital Single Market

375

Services. [Online] Available at: http://www.cloudbus

.org/cloudsim/[Accessed 17 November 2016].

Cloudsleuth, 2016. Cloudsleuth. [Online] Available at:

https://cloudsleuth.net[Accessed November 2016].

CloudSurfing, 2016. CloudSurfing. [Online] Available at:

https://www.cloudsurfing.com/[Accessed 17

November 2016].

Eclipse Cloud Development;, 2015. Eclipse Cloud

Development. [Online] Available at: http://www.

eclipse.org/ecd/[Accessed 17 November 2016].

Edwards, D., 2015. DevOps paradox: Going Faster Brings

Higher Quality, Lower Costs, & Better Outcomes, s.l.:

s.n.

Elastic Cloud Gate, 2014. EC2 Monitoring - Always On -

Elastic Cloud GateElastic Cloud Gate. [Online]

Available at: http://blog.ecloudgate.com/ec2-

monitoring-always-on/

[Accessed 17 November 2016].

ETSI, 2013. Cloud Standards Coordination - Final Report,

s.l.: s.n.

Fraunhofer Fokus, 2015. Fokus!MBT Features. [Online]

Available at: http://www.fokusmbt.com/fmbt/

key_features[Accessed 17 November 2016].

Gartner, 2012. The Business Landscape of Cloud

Computing. [Online] Available at: https://www.

cloudeassurance.com/white-papers/Gartner_Daryl_

Plummer_and_Financial_Times_e-Book_The_

Business_Landscape_of_Cloud_Computing.pdf

[Accessed 17 November 2016].

Gartner, 2016. Cloud Services Brokerage. [Online]

Available at: http://www.gartner.com/it-glossary

/cloud-services-brokerage-csb/ [Accessed 17

November 2016].

Google Cloud Platform, 2016. Setting Instance Scheduling

Options - Compute Engine. [Online] Available at:

https://cloud.google.com/compute/docs/instances/setti

ng-instance-scheduling-options[Accessed 17

November 2016].

HPE, 2016. HEP Helion. [Online] Available at:

http://www8.hp.com/us/en/cloud/helion-overview.

html[Accessed March 2016].

IBM BlueMix, 2016. IBM BlueMix DevOps services.

[Online] Available at: hub.jazz.net[Accessed 2017

November 2016].

IBM, 2016. IBM Cloud. [Online] Available at:

https://www.ibm.com/marketplace/cloud/us/en-us?

lnk=mp[Accessed March 2016].

Jamcracker Platform, 2016. Jamcracker Services Delivery

Network. [Online] Available at: http://www.

jamcracker.com/jamcracker-services-delivery-

network-jsdn[Accessed March 2016].

Li, H. a. R. B., 2009. Model-based simulation and

performance evaluation of grid scheduling strategies.

In: Future Generation Computer Systems. s.l.:s.n., pp.

460-465.

Mazzuca, J., 2015. Survey Analysis: Cloud Adoption

Across Vertical Industries Exhibits More Similarities

Than Differences, s.l.: Anderson.

Nagios, 2016. Industry Standard In IT Infrastructure

Monitoring. [Online] Available at: https://www.

nagios.com[Accessed 17 November 2016].

New Relic, 2016. Application Performance Monitoring

Features. [Online] Available at: http://

newrelic.com/application-monitoring[Accessed 17

November 2016].

OpenStack, 2016. Monasca- OpenStack. [Online]

Available at: https://wiki.openstack.org/wiki/Monasca

[Accessed 17 November 2016].

Petcu, D., 2013. Multi-Cloud: Expectations and Current

Approaches. New York, s.n.

Puppet labs, 2013. MCollective Introduction:

Programmatic Execution of Systems Administrations

Actions - Puppet Labs. [Online] Available at:

https://puppetlabs.com/mcollective[Accessed March

2016].

Rackspace, 2015. Rackspace Monitoring. [Online]

Available at: https://www.rackspace.com/cloud/

monitoring[Accessed 17 November 2016].

Terraforms, 2015. Terraform. [Online] Available at:

https://www.terraform.io/[Accessed April 2016].

Ubuntu, 2016. Ubuntu Juju. [Online] Available at:

http://community.ubuntu.com/[Accessed March 2016].

UpGuard, 2016. The four prerequisites for DevOps

success, s.l.: s.n.

Zabbix, 2016. The Enterprise-class Monitoring Solution

for Everyone. [Online] Available at: http://www.

zabbix.com[Accessed March 2016].

CLOSER 2017 - 7th International Conference on Cloud Computing and Services Science

376