Return on Cybersecurity Investment in Operational Technology Systems:
Quantifying the Value That Cybersecurity Technologies Provide after
Roger A. Hallman
, Maxine Major
, Jose Romero-Mariona
, Richard Phipps
, Esperanza Romero
and John M. San Miguel
Thayer School of Engineering, Dartmouth College, Hanover, New Hampshire, U.S.A.
Naval Information Warfare Center Pacific, San Diego, California, U.S.A.
{esperanza.romero, john.m.sanmiguel}
Cybersecurity Investment, Acquisition, Decision Support, Return on Cybersecurity Investment.
Appropriate cybersecurity investment is a challenge faced by both private and public organizations. This chal-
lenge includes understanding the actual vulnerabilities of an organization’s networked systems, as well as the
cost of a successful cyber attack on those systems. On top of this, an organization’s cybersecurity acqui-
sition workforce must be able to discern reality from the marketing hype that is produced by cybersecurity
sales forces. This paper builds upon earlier work which developed a cybersecurity acquisition decision sup-
port mechanism (Romero-Mariona. et al., 2016). In particular, cybersecurity technology evaluation results
are extended to assist organizations to define a Return on Cybersecurity Investment. This new capability is
instantiated within the context of networked critical infrastructure and industrial control systems.
Cybersecurity is a complex challenge for many or-
ganizations, ranging from individuals surfing the In-
ternet to public and private organizations. Many or-
ganizations, both public and private, have fallen vic-
tim to cyber attacks (Davis, 2015; Cieply and Barnes,
2015). Moreover, as Internet-connected systems are
integrated more and more into the average person’s
daily life, cyber attacks have become a normal aspect
of modern life for many people. Many attacks are of
little to no consequence, however many attacks end up
becoming high-visibility and costly cyber incidences,
which may cause lasting damage to the organization.
There are a variety of reasons that an organization’s
network may be susceptible to a cyber attack, ranging
from personnel with poor online practices to systems
that do not patch vulnerabilities in a timely manner.
Cybersecurity investment is another challenge for or-
ganizations. Indeed, “cybersecurity investment” deals
with multiple challenges including underinvestment,
lifecycle management, etc.
Cybersecurity for organizations that own and
operate cyber-physical systems—specifically indus-
trial systems, critical infrastructure, or other legacy
systems—is an especially challenging problem due
to the fact that these systems often comprise of
components that were not designed with security
or modern interconnectedness in mind. For coun-
tries in conflict with an unscrupulous adversary, net-
worked critical infrastructure systems can make an
enticing target. Indeed, cyber attacks against crit-
ical infrastructure are well known to lead to se-
vere consequences (Liang et al., 2016), as en-
ergy system disruption can lead to critical sys-
tems (e.g.,waste processing, hospital/medical sys-
tems, traffic lights,refrigeration/storage systems, etc.)
failing. Even in countries that are not in active con-
flict, many critical infrastructure systems are known
to be vulnerable to cyber attack and adversarial actors
may have already compromised those systems (US-
CERT, 2018).
To address these challenges, we developed the Re-
silient Critical Infrastructures through Secure and Ef-
ficient Microgrids (ReCIst) project, funded through
the United States’ Office of Naval Research Energy
System Technology Evaluation Program (ESTEP)
to develop a decision-support capability that provides
visibility into the true costs of introducing cyberse-
curity solutions to industrial power grids. The de-
Hallman, R., Major, M., Romero-Mariona, J., Phipps, R., Romero, E. and Miguel, J.
Return on Cybersecurity Investment in Operational Technology Systems: Quantifying the Value That Cybersecurity Technologies Provide after Integration.
DOI: 10.5220/0009416200430052
In Proceedings of the 5th International Conference on Complexity, Future Information Systems and Risk (COMPLEXIS 2020), pages 43-52
ISBN: 978-989-758-427-5
2020 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
cision of what cybersecurity solution would be best
is ultimately a financial decision, therefore we devel-
oped a return on investment model to help acquisi-
tion workers navigate the costs of their own facilities
in comparison with the costs and benefits associated
with a virtual marketplace of potential cybersecurity
solutions. Our main contributions in this paper are as
We build upon earlier work, which developed a
decision support framework for a cybersecurity
acquisition workforce (Romero-Mariona. et al.,
2016) by feeding technology evaluations into a
framework for determining a Return on Cyberse-
curity Investment (ROCI);
We describe an instantiation of the ROCI model
to quantify the effects of cybersecurity investment
for critical infrastructure.
To the best of our knowledge, ours is the first cyberse-
curity investment framework that attempts to quantify
a return on investment for the critical infrastructure
The remainder of this paper is organized as fol-
lows: Section 2 provides surveys previous work on
cybersecurity investment strategies and cybersecurity
economics. Our cybersecurity technology evaluation
and decision support framework is reviewed in Sec-
tion 3. Section 4 describes the Return on Cybersecu-
rity Investment model within the context of industrial
control systems and networked critical infrastructure
while Section 5 demonstrates the feeding of technol-
ogy evaluations to determine a ROCI for the adoption
of a suite of cybersecurity technologies. Concluding
remarks and ongoing/planned work is found in Sec-
tion 6.
The challenges of optimizing an organization’s cyber-
security investment is reasonably well studied. Some
organizations (e.g., large companies, governments,
etc.) may be able to support a specialized cybersecu-
rity workforce which serves as a center of institutional
knowledge and can provide direction or assistance on
the issue throughout the organization. If in-house cy-
bersecurity expertise cannot be maintained, cyberse-
curity strategies for investment and network manage-
ment must rely on other knowledge bases (e.g., inter-
nal personnel with some level of cybersecurity knowl-
edge, vendor sales teams, etc.). Beyond the initial in-
vestment in cybersecurity technology, there are a host
of other issues which must be taken into considera-
tion. Some of these issues include workforce develop-
ment and continued certifications and lifecycle man-
agement for integrated cybersecurity acquisitions.
2.1 Cybersecurity Investment Strategies
Gordon and Loeb developed a cybersecurity invest-
ment model that has become the seminal work on the
subject (Gordon and Loeb, 2002). The Gordon and
Loeb model deals with information technology rather
than operational technology systems, specifically fo-
cusing on protecting information sets. The model
shows that organizations should not necessarily fo-
cus cybersecurity investments on the most vulnerable
information sets, which may be exorbitantly expen-
sive to protect, instead investing on the protection of
information sets with intermediate-level vulnerabili-
ties. Additionally, the model gives instruction on op-
timization, suggesting that an organization only needs
to spend a small fraction of the expected loss from a
cyber attack.
Further work from Gordon, Loeb, et al., build
upon their original model. (Gordon et al., 2014a)
present a discussion on the different cybersecurity in-
vestment strategies that for-profit firms may pursue,
as opposed to strategies pursued by non-profit orga-
nizations or governments. (Gordon et al., 2014b)
takes external factors into account for determining
how much an organization should optimally spend
on cybersecurity investment. In particular, taking ex-
ternalities into account, they showed that an optimal
cybersecurity investment increases by at most 35%
over the earlier investment model. Analysis on in-
formation sharing between organizations and the ef-
fect on cybersecurity investment—specifically avoid-
ing underinvestment—is covered in (Gordon et al.,
Cavusoglu, Mishra, and Raghunathan (Cavusoglu
et al., 2004) present an explicit outcome-based cyber-
security investment model that attempts to calculate
a return on security investment for IT systems. Their
model utilizes Bayesian modeling and a game theo-
retic approach to inform which parts of a network are
vulnerable to attack and how much manual monitor-
ing should be implemented in order to minimize se-
curity cost. The model is applied to suites of cyber-
security technologies and optimized to direct specific
As with the Cavusoglu model, Quantitative Eval-
uation of Risk for Investment Efficient Strategies
(QuERIES) (Carin et al., 2008), proposed by Carin,
Cybenko, and Hughes, adopted a game theoretical ap-
proach to provide a computational approach to cyber-
security risk assessment. QuERIES was meant to deal
with protecting IT systems that hold critical intellec-
COMPLEXIS 2020 - 5th International Conference on Complexity, Future Information Systems and Risk
tual property (IP) in which the loss of a single IP copy
would prove catastrophic (e.g., weapon designs held
in a military’s network, personally identifiable infor-
mation). Specifically, the QuERIES methodology is
used for assessing the efficacy of network protections
to prevent reverse engineering attacks. Given a model
of an organization’s security strategy, reverse engi-
neering attack graphs are built and represented as Par-
tially Observable Markov Decision Processes. Given
these models, QuERIES provides optimal cybersecu-
rity policies and investment.
Different organizations will have hold a variety
of risk management strategies, (Huang et al., 2008)
consider a cybersecurity investment strategy for risk-
averse firms. This strategy is similar to the risk-
neutral model (Gordon and Loeb, 2002), though it in-
corporates a different set of assumptions and bound-
ary conditions. Under the assumption that not all
cybersecurity risks are worth defending against, this
strategy proposes some minimum possible loss as a
trigger for cybersecurity investment, though the op-
timal level of cybersecurity investment does not nec-
essarily increase with a firm’s risk aversiveness. in-
terestingly, they show that optimal investment will in-
crease vulnerability.
Moore, Dynes, and Chang (Moore et al., 2015) in-
terviewed security managers and executives (usually
a Chief Information Security Officer or Chief Infor-
mation Officer) across a series of sectors, including
finance, healthcare, retail, and government organiza-
tions. These interviews demonstrated that most orga-
nizations rely heavily on frameworks to inform cyber-
security investment decision making
. Moreover, due
to this reliance on process-oriented frameworks (Joint
Task Force and Transformation Initiative, 2013), in-
vestment metrics that consider outcomes, such as Re-
turn on Investment, are not calculated.
Cybersecurity is a growing concern for governments
around the world. The United States Government,
for example, is on the receiving end of more than
100,000 daily cyber attacks (Maloney, 2016). Fur-
thermore, Cybersecurity investment strategies must
account for the reality that cybersecurity technology
is a fast-growing market and constantly evolving to
counter threats to information and operational sys-
tems. In spite of the fact that it spends billions of
dollars annually on cybersecurity (Morgan, 2016), the
United States Government lacks a standardized and
repeatable methodology for evaluating cybersecurity
technologies and informing investment strategies. As
a consequence of this lack of standardization and re-
peatability, the acquisition process inevitably leads to
duplicated efforts on the part of technical and acquisi-
tion personnel. Moreover, lessons learned within one
sector of the Government are not easily shared with
others, which may lead to multiple agencies adopt-
ing a cybersecurity technology that fails to meet their
needs. In certain sectors of the government, where
personnel are rotated through on a regular basis, cy-
bersecurity policies and products may be changed
with each shift in project supervision. A practical and
important consequence of this is that cybersecurity
acquisition decisions will be made by security non-
experts. If an expert in a security topic leaves a team,
their institutional knowledge on that topic may be lost
to current personnel. Knowledge that was common
sense in previous decision-making efforts is not obvi-
ous to new team members.
We developed a standardized and repeatable
methodology for evaluating cybersecurity technolo-
gies, the DoD-Centric and Independent Technology
Evaluation Capability (DITEC) (Romero-Mariona,
2014), to address these challenges
. DITEC pro-
vides a bulwark against the loss of institutional cyber-
security knowledge that is endemic to organizations
where personnel are regularly rotated through, thus
mitigating the risk of cybersecurity investment deci-
sions being made solely by non-experts based on a
sales presentation (Moore et al., 2015). Moreover,
even when technical experts are given input in the
cybersecurity acquisition process, they are likely to
have biases that may not ideally suit the organiza-
tion’s local requirements. DITEC standardizes the
cybersecurity acquisition process in part by institut-
ing guidelines and frameworks, e.g., National Insti-
tute of Standards and Technology (NIST) Cyberse-
curity Framework, the DoD 8500 Series Information
Assurance (IA) Controls
, to establish the types of
procedures, controls, threats, and features that pro-
vide the test cases for which cybersecurity technolo-
gies would be evaluated. A market survey was con-
ducted to learn what technologies were available for
acquisition and to classify them into a three-tiered cat-
egorization based on their capabilities. Technology
evaluation metrics and a scoring algorithm were de-
veloped by creating a taxonomy which matched tech-
nologies and test cases, allowing users to evaluate and
Note that while DITEC is oriented for government use
cases, it can easily be adapted to other organization models.
Return on Cybersecurity Investment in Operational Technology Systems: Quantifying the Value That Cybersecurity Technologies Provide
after Integration
make high-level comparisons of multiple technolo-
gies against one another.
3.1 DITEC Components
DITEC has three main components:
The DITEC Process is used to evaluate a specific
cybersecurity technology to determine how well it
meets DoD/Navy needs;
DITEC Metrics measure how well each technol-
ogy meets the specified needs across 125 different
test cases;
The DITEC Framework provides the format nec-
essary to compare and contrast multiple technolo-
gies of a specific cybersecurity area.
Cybersecurity technologies are rated by metrics on
three levels of granularity. The highest level of granu-
larity is the Capability level. There are 10 such Capa-
bilities, which correspond to very broad ability cate-
gories (e.g., ‘Protect’, ‘Respond’, ‘Operations’, ‘Life-
cycle Management’). A middle level of granularity,
Sub-Capability, narrows the focus from the Capabil-
ity level (e.g., from ‘Protect’‘—-‘Cryptographic Sup-
port’, ‘Lifecycle Management’—-‘Cost of Extended
Vendor Support’). Finally, the Sub-Capability Ele-
ments level includes very specific test cases (e.g. Does
the technology offer whitelisting?).
3.2 DITEC+
DITEC+ (Romero-Mariona et al., 2015) expands
upon the initial DITEC proof of concept to implement
the scalability required for further development and
adoption, creating an enterprise-ready tool to aid in
the acquisition process. Improvements on the initial
DITEC concept include:
Process–DITEC+ prescribes addi-
tional/customizable steps for focused evaluations
pertaining to specific stakeholders and offers
those steps as a library of evaluation guidelines;
Metrics–DITEC+ revises and improves on the
DITEC Metrics module in order to enable tech-
nologies to receive a “score” based on their eval-
uation performance against the metrics, providing
the ability to apply “weights” to each evaluation
per specific items of interest identified during the
process, as well as support for prioritizing results
based on a variety of different aspects;
Framework–DITEC+ leverages the existing
DITEC Framework but ensures that it is ready
for enterprise-wide use, supporting multiple
users and evaluations by adding robustness to the
database and evaluation algorithms.
These and other improvements enable a
DoD/Navy-centric, cost-effective, streamlined
evaluation of various cybersecurity technologies,
defined by a process that is standardized, flexible,
repeatable, scalable, and granular metrics (devel-
oped in-house with subject matter expert support).
Additionally, DITEC+:
Supports multiple and concurrent users and tech-
nology evaluations;
Provides the ability to compare various cyberse-
curity technology evaluations;
Integrated CAULDRON (Jajodia et al., 2011),
a network vulnerability mapping tool developed
new metrics for measuring differences across
evaluations and technologies while estimating the
level of cybersecurity provided;
Developed new metrics for measuring differences
across evaluations and technologies while esti-
mating the level of cybersecurity provided;
Developed a new ranking/prioritization mecha-
nism of evaluated technologies based on user pref-
Recognizing that personnel at different levels of
an organization would have competing priorities, the
User Priority Designation (UPD) was developed to
view evaluations in light of priorities. For example,
an agency’s comptroller may place a heavier priority
on the lifetime cost of a product, where a network ad-
ministrator would be more concerned with the ability
to install a vendor update with minimal system down-
time. Using DITEC+’s UPD tool, technology evalu-
ations can be viewed by cybersecurity professionals
and by management, allowing all stakeholders within
the agency to project how various technologies and
products will meet their needs according to differing
criteria. Finally, to assist cybersecurity non-experts
who have a role in the acquisition decision making
process, we invented a recommender system which
helps to match users to the technology (or suite of
technologies) which best match their needs (Romero-
Mariona et al., 2016).
3.3 Applying DITEC+ to Critical
The Cyber-SCADA Evaluation Capability (C-SEC)
is an instantiation of DITEC+, designed specifically
to test and evaluate the suitability of cybersecu-
rity technologies to networked critical infrastructures,
specifically Supervisory Control and Data Acquisi-
tion (SCADA) systems (Romero-Mariona. et al.,
2016). Security has not traditionally been a concern
COMPLEXIS 2020 - 5th International Conference on Complexity, Future Information Systems and Risk
for SCADA systems because different manufactur-
ers employed diverse protocols; however as protocols
have become standardized this is no longer the case.
Specifically, C-SEC utilizes the tools and processes
of DITEC+ and applies them to a SCADA test bed,
into which cybersecurity products are integrated and
evaluated. The system receives a baseline vulnerabil-
ity scan prior to product integration as well as a post-
integration scan, the scans are then compared to deter-
mine the level of effectiveness based on an expected
number of reduced vulnerabilities. Integration into
the test bed system gives insight into how the prod-
uct under evaluation (PuE) may affect other compo-
nents in an operational environment. It also gives red
team personnel the opportunity to stress test the test
bed system with the PuE. To further demonstrate the
effectiveness of the C-SEC approach, C-SEC On The
Move mobilizes the DITEC+ infrastructure and the
SCADA System scanner into a package that can be
taken to SCADA system sites to run diagnostic scans,
map vulnerabilities, and make recommendations for
the acquisition and placement of cybersecurity prod-
The decision-making capabilities provided by
DITEC+ and C-SEC’s Technology Matching Tool
(TMT) recommendation only serve to inform cus-
tomers of the suitability of a solution based purely on
the capabilities of that solution. Because of this, often
the top ranked cyber solution recommendations in the
C-SEC capability consistently include Security In-
formation and Event Management (SIEM) solutions,
SIEM capabilities typically include a broad spectrum
of robust capabilities and consequently tend to be
the most expensive. This can result in significant
financial impact to not only procure, but to train and
educate staff and weather downtime of industrial
services during installation and routine maintenance.
In the end, the decision to procure a cyber solution
is a financial decision, where the decision to include
cyber comes down to justifying the cost to procure,
integrate, and maintain against the financial risks of a
cyber attack.
The ReCIst capability improves on the previous
cyber solution recommendation models by bringing
the financial cost of integrating and maintaining a so-
lution in line with the end user’s financial picture. By
focusing on the “is it worth it” aspect of cyber solu-
tions, our Return on Cyber Investment (ROCI) model
depicts how financial factors can measure the finan-
cial impact of a cyber attack, and how a cyber solution
can mitigate - or contribute to - the financial fallout
from a cyber attack.
The key features of ROCI borrow from typical re-
turn on investment (ROI) modeling.
Costo fCountermeasure
ALE = (NumberO f PotentialIncidentsPerYear)
In the ROCI model, the ultimate return value to
determine whether or not procuring a cyber solution
would result in a net gain or loss for an organiza-
tion is calculated as the annual difference between
costs associated with cyber attacks minus the costs of
those same attacks, now mitigated by a cyber solu-
tion. Startup, training, and all up front costs, as well
as annual maintenance for the cyber solution are in-
cluded in the cost for cyber, as well as any impact to
service, energy expenditure in particular. One of the
main reasons organizations are hesitant to adopt cy-
ber security technology in industrial systems is due
to the availability of the industrial services taking the
highest priority at all times. Taking systems offline to
install and troubleshoot a cyber solution, and adding
responsibilities for staff to maintain and audit this new
solution, can be difficult to justify without financial
The full ROCI model is depicted in Figure 1 (Ma-
jor et al., 2020). This model is depicted as a series
of modular arithmetic calculations based off inputs
which are sourced from customers, vendors, and re-
search. Although many of the more abstract research
values are still currently being discovered (e.g., aver-
age financial loss tied to social damage as a result of
a publicized hack), even with a best guess, in the Re-
CIst software application a customer can evaluate the
potential financial impact by adjusting risk inputs to
determine a possible best or worst case scenario.
The ROCI model requires inputs of specific inte-
ger values from either the customer, the vendor, or
publicly available data. The distinction is not made
clearly at this time, as the source for these values may
differ for each user of the ReCIst tool. Inputs include
organization-specific values such as the hourly cost
for services to be offline, which is expected during in-
stallation, testing, and maintenance. Labor costs are
part of the model, including installation, testing, and
maintenance, as well as training, troubleshooting, and
Return on Cybersecurity Investment in Operational Technology Systems: Quantifying the Value That Cybersecurity Technologies Provide
after Integration
Figure 1: The Full Return on Cybersecurity Investment Model.
COMPLEXIS 2020 - 5th International Conference on Complexity, Future Information Systems and Risk
even the costs for auditing and forensics when a cy-
ber attack does occur. The most difficult values to
input into this model are those which can predict the
likelihood and frequency of a cyber attack, as well as
the scope of damage and recovery from such an at-
tack. Other questions, which are more personal and
customized to the organization include the financial
costs associated with reputation damage when a leak
is publicized.
These inputs are fed into a series of simple arith-
metic calculations which in turn are used to calculate
the ROCI score:
before security
cost after security
cost attack
cost human injury
cost system damages
cost system downtime
cost social damages
cost incident response
cost investigation
cost security
cost initial
cost init training
cost installation
cost annual maint
cost security repairs
cost security replacement
cost energy consumption
benefit security (currently simplified to a per-
centage of impact to each of the cost attack val-
It should be noted that the ROCI model is only used
to calculate costs that can be directly attributed to to
a cyber attack. Industrial systems often experience
outages, downtime, and exhibit unexpected behav-
ior due to normal events, such as equipment failure,
mis-calibrations and human error, and even wildlife
and other acts of nature. Intelligent industrial attacks
which aim for low-key degradations of service - not
openly destroy or cause outages - may not be diag-
nosed as a cyber attack for a period of time, if at all.
While the ROCI model does not attempt to account
for costs associated with variations in service that may
or may not be part of normal industrial operations,
it is possible that future versions of cyber investment
models for industrial systems may use existing finan-
cial forecasting to supplement the ROCI value for so-
lutions that detect or deter some percentage of these
types of attack.
In the ReCIst tool, the ROCI model improves the C-
SEC and DITEC+ recommendation capabilities to cy-
ber solutions that also have the best ROCI value in ad-
dition to meeting customer needs. This integration is
addressed in the following section.
The ReCIst capability integrates technology rec-
ommendations based on the user’s technological
needs, but then prioritizes the recommendations based
on the best value ROCI score, presenting the top 3 to
the user. See Figure 2.
Although this concept is simple in implementation
- the top three technologies with the least financial
impact are recommended - all the information gained
can then be used to allow a user to re-prioritize and
gain new recommendations based on the which as-
pects of each solution have the most financial im-
pact. This gives the customer data which will em-
power them to make even better decisions based on
their technological and financial goals.
Figure 2: ROCI Impact Comparison for Two Products.
For example, ROCI data collected allows ReCIst
to provide additional calculations for aspects such as
total annual labor or system downtime costs. The Re-
CIst tool presents this information to the end user via
a number of graphics. For example, in Figure 3, the
first and third solutions are fairly well-balanced across
categories, but the hardware cost is high, and energy
costs low on the third option compared to the first so-
lution, where energy costs are very high, and the hard-
ware cost is average. If the customer is able to afford
Return on Cybersecurity Investment in Operational Technology Systems: Quantifying the Value That Cybersecurity Technologies Provide
after Integration
Figure 3: Radar Chart Depicting Cost Category Differences for Three Different Products.
a high up-front cost, but needs to keep recurring costs
such as energy low, then the third option might be a
better solution. If a given solution is only expensive
because of initial startup and training fees, but if this
product requires low maintenance fees and has a low
failure rate, they may be willing to budget for a high
startup cost, knowing the solution will pay for itself
in the long run. While ROCI and the ReCIst capabil-
ity focus on the first year of a solution - the decision
to procure and the costs associated with the first year
of a solution’s life, future versions of this work will
aim to incorporate forecasting, such as can be done in
a fast-moving landscape of technological innovation
and escalating cyber threats.
Users may also be able to mitigate some of these
costs in-house. For example, if 100% system down-
time is too financially impactful for installation and
maintenance, but the customer has a strategy in mind,
which involves only taking half of the systems of-
fline at any given time, minimizing service disrup-
tions, then they know that while their labor rates may
rise during these times, the service impact to their cus-
tomers is minimal, and in the end, they will gain en-
hanced cyber protections with minimal impact to rep-
utation and customer turnover. These numbers can
empower end users to make these kinds of decisions,
whereas previously they might have been difficult to
Cybersecurity is a complex and difficult challenge
for organizations with Internet-connected systems.
These complexities and difficulties are multiplied
when legacy systems or operational technologies such
as electrical generation and distribution infrastructure
are Internet-connected, as earlier designers and engi-
neers did not foresee modern interconnectedness or
security risks. We introduce ReCIst, which integrates
the output of our earlier work in cybersecurity tech-
nology evaluation and decision support frameworks
to quantify a return on investment for cybersecurity
technologies, specifically in the critical infrastructure
sector. This is, to the best of our knowledge, the first
attempt at examining the return on cybersecurity in-
vestment for critical infrastructure or other publicly-
owned/managed, critical systems.
Our ReCIst capability aims to ease the burden of
cyber solution decision-making for microgrid facility
managers by producing a Return on Cyber Investment
(ROCI) model, which considers not only the costs as-
sociated with a cyber solution, but how that solution’s
impact on the costs associated with a cyber attack af-
fect the bottom line. The model is designed to be ex-
panded, so that follow-on research and cost complexi-
ties can further optimize the decision making process.
Ongoing work includes:
The inclusion of new inputs, such as results from
a vulnerability scanner (e.g., Nessus) which are
factored into a cyber vulnerability score;
The inclusion of a recommender system (Romero-
Mariona et al., 2016) which can assist an organi-
zation’s acquisition workforce in optimizing their
Generalize the ROCI model beyond the current
critical infrastructure instantiation, thus enabling
the model’s application to model the benefits of
cybersecurity technology to all manner of net-
worked systems;
Improve the ROCI model to include the nuance
of how specific cyber capabilities mitigate specific
certain cyber threats or risks, so that solution ca-
pabilities are more accurately tied to financial gain
or loss;
COMPLEXIS 2020 - 5th International Conference on Complexity, Future Information Systems and Risk
Incorporate external impacts on ROCI such as cy-
ber insurance;
Forecasting long-term investment impacts which
plan beyond the first year of cyber solution adop-
tion, including equipment failure rates, annual
maintenance, degradation of performance and
technological upgrades;
Model attack diversity and intended consequences
that may map to military or government sites (do
attack statistics show that different exploits are
utilized for target-specific campaigns, such as data
exfiltration compared to reduced mission effec-
Evaluating risk and risk management is another
complex field of study. In the traditional two-axis
management view of risk, risk calculation depends
on two independent variables: consequence (the im-
pact of a successful attack), and probability (the like-
lihood that the attack will be successfully executed).
Currently, the ROCI model conflates these two vari-
ables, an action that may reduce the granularity of the
model’s analysis. In the future, ROCI may be im-
proved by separating these axes, according to the tra-
ditional model of risk estimation. Furthermore, While
cybersecurity risk has been modeled in the traditional
risk management matrix (Collard et al., 2016), other
research shows that these risk matrices are not effec-
tive (Thomas, 2013). Various research methods also
exist to show that decisions can be optimized by in-
corporating risk (Hubbard and Seiersen, 2016). By
capturing the nuances of risk and the methods used
to manage the risk, the Return-on-Investment mod-
els and recommendations would be even more robust,
giving system managers a greater amount of knowl-
edge with where and how to improve their system
from cyber attacks in a cost-efficient and effective
Roger A. Hallman is supported by the United States
Department of Defense SMART Scholarship for Ser-
vice Program, funded by USD/R&E (The Under Sec-
retary of Defense-Research and Engineering), Na-
tional Defense Education Program (NDEP) / BA-1,
Basic Research.
Bergner., S. and Lechner., U. (2017). Cybersecurity ontol-
ogy for critical infrastructures. In Proceedings of the
9th International Joint Conference on Knowledge Dis-
covery, Knowledge Engineering and Knowledge Man-
agement - Volume 2: KEOD,, pages 80–85. INSTICC,
Bodin, L. D., Gordon, L. A., Loeb, M. P., and Wang,
A. (2018). Cybersecurity insurance and risk-sharing.
Journal of Accounting and Public Policy, 37(6):527–
Carin, L., Cybenko, G., and Hughes, J. (2008). Cybersecu-
rity strategies: The queries methodology. Computer,
Cavusoglu, H., Mishra, B., and Raghunathan, S. (2004). A
model for evaluating it security investments. Commu-
nications of the ACM, 47(7):87–92.
Cieply, M. and Barnes, B. (2015). Sony at-
tack, first a nuisance, swiftly grew into a
firestorm. The New York Times, page A1.
Collard, G., Disson, E., Talens, G., and Ducroquet, S.
(2016). Proposition of a method to aid security clas-
sification in cybersecurity context. In 2016 14th An-
nual Conference on Privacy, Security and Trust (PST),
pages 88–95. IEEE.
Davis, J. H. (2015). Hacking exposed 21 million in u.s.,
government says. The New York Times, page A1.
Gordon, L. A. and Loeb, M. P. (2002). The economics of in-
formation security investment. ACM Transactions on
Information and System Security (TISSEC), 5(4):438–
Gordon, L. A., Loeb, M. P., and Lucyshyn, W. (2014a). Cy-
bersecurity investments in the private sector: the role
of governments. Geo. J. Int’l Aff., 15:79.
Gordon, L. A., Loeb, M. P., Lucyshyn, W., and Zhou, L.
(2015). The impact of information sharing on cyber-
security underinvestment: A real options perspective.
Journal of Accounting and Public Policy, 34(5):509–
Gordon, L. A., Loeb, M. P., Lucyshyn, W., Zhou, L., et al.
(2014b). Externalities and the magnitude of cyber se-
curity underinvestment by private sector firms: a mod-
ification of the gordon-loeb model. Journal of Infor-
mation Security, 6(01):24.
Huang, C. D., Hu, Q., and Behara, R. S. (2008). An eco-
nomic analysis of the optimal information security in-
vestment in the case of a risk-averse firm. Interna-
tional journal of production economics, 114(2):793–
Hubbard, D. W. and Seiersen, R. (2016). How to measure
anything in cybersecurity risk. John Wiley & Sons.
Jajodia, S., Noel, S., Kalapa, P., Albanese, M., and
Williams, J. (2011). Cauldron mission-centric cy-
ber situational awareness with defense in depth. In
2011-MILCOM 2011 Military Communications Con-
ference, pages 1339–1344. IEEE.
Return on Cybersecurity Investment in Operational Technology Systems: Quantifying the Value That Cybersecurity Technologies Provide
after Integration
Joint Task Force and Transformation Initiative (2013). Se-
curity and privacy controls for federal information
systems and organizations. NIST Special Publication,
Liang, G., Weller, S. R., Zhao, J., Luo, F., and Dong, Z. Y.
(2016). The 2015 ukraine blackout: Implications for
false data injection attacks. IEEE Transactions on
Power Systems, 32(4):3317–3318.
Major, M., Romero-Mariona, J., Phipps, R., Tacliad,
F., Slayback, S. M., Romero, E., and Hallman,
R. A. (2020). Towards quantifying energy resiliency
through return on cyber investment modeling. In
HICSS Symposium on Cybersecurity Big Data Ana-
Maloney, M. (2016). Pentagon’s dc3i memo acknowledges
thousands of cyber breaches that compromised dod
systems and commits to new cyber culture. Govern-
Moore, T., Dynes, S., and Chang, F. R. (2015). Iden-
tifying how firms manage cybersecurity investment.
Available: Southern Methodist University. Available
at: http://blog. smu. edu/research/files/2015/10/SMU-
IBM. pdf (Accessed 2015-12-14), 32.
Morgan, S. (2016). Worldwide cybersecurity spending in-
creasing to $170 billion by 2020.
Omerovic., A., Vefsnmo., H., Erdogan., G., Gjerde., O.,
Gramme., E., and Simonsen., S. (2019). A feasibil-
ity study of a method for identification and modelling
of cybersecurity risks in the context of smart power
grids. In Proceedings of the 4th International Con-
ference on Complexity, Future Information Systems
and Risk - Volume 1: COMPLEXIS,, pages 39–51. IN-
STICC, SciTePress.
Romero-Mariona, J. (2014). Ditec (dod-centric and inde-
pendent technology evaluation capability): a process
for testing security. In 2014 IEEE Seventh Interna-
tional Conference on Software Testing, Verification
and Validation Workshops, pages 24–25. IEEE.
Romero-Mariona, J., Hallman, R., Kline, M., Palavicini, G.,
Bryan, J., San Miguel, J., Kerr, L., Major, M., and Al-
varez, J. (2015). An approach to organizational cy-
bersecurity. In International Workshop on Enterprise
Security, pages 203–222. Springer.
Romero-Mariona., J., Hallman, R. A., Kline, M., Miguel,
J. S., Major, M., and Kerr, L. (2016). Security in the
industrial internet of things - the c-sec approach. In
Proceedings of the International Conference on Inter-
net of Things and Big Data - Volume 1: IoTBD,, pages
421–428. INSTICC, SciTePress.
Romero-Mariona, J., Kerr, L., Hallman, R., Coronado, B.,
Bryan, J., Kline, M., Palavicini, G., Major, M., and
San Miguel, J. (2016). Tmt: Technology matching
tool for scada network security. In 2016 Cybersecurity
Symposium (CYBERSEC), pages 38–43. IEEE.
Sedgewick, A. (2014). Framework for improving critical
infrastructure cybersecurity, version 1.0. Technical re-
Thomas, P. (2013). The risk of using risk matrices. Master’s
thesis, University of Stavanger, Norway.
US-CERT (2018). Alert (ta18-074a): Russian gov-
ernment cyber activity targeting energy and other
critical infrastructure sectors.
Woods, D. and Moore, T. (2019). Does insurance have a
future in governing cybersecurity? IEEE Security and
Privacy Magazine.
COMPLEXIS 2020 - 5th International Conference on Complexity, Future Information Systems and Risk