Design and Implementation of a Test Tool for PSD2 Compliant Interfaces

Gloria Bondel

, Josef Kamysek

and Markus Kraft

and Florian Matthes

Technical University of Munich, Faculty of Informatics, Garching, Germany

msgGillardon AG, Munich, Germany

Keywords:

PSD2, XS2A, Test Tool.

Abstract:

The Revised Payment Services Directive (PSD2) forces retail banks to make customer accounts accessible

to TPPs via standardized and secure ”Access to Account” (XS2A) interfaces. Furthermore, banks have to

ensure that these interfaces continuously meet functional and performance requirements, hence testing is very

important. A known challenge in software testing is the design of test cases. While standardized speciﬁcations

and derived test cases exist, the actual implementations of XS2A interfaces often deviate, leading to the need to

adapt existing or create new test cases. We apply a design science approach, including ﬁve expert interviews,

to iteratively generate a concept of a test tool that enables testing of several XS2A interface implementations

with the same set of test cases. The concept makes use of ﬁles mapping deviations between the standardized

speciﬁcation and the implemented interfaces. We demonstrate the concept’s feasibility by implementing a

prototype and testing its functionality in a sandbox setting.

1 INTRODUCTION

The European Union published the Revised Payment

Services Directive (Directive (EU) 2015/2366, ab-

breviated PSD2) to address risks resulting from new

online and mobile services that emerged in retail

banking. One measure prescribed by the PSD2 is

the realization of standardized and secure ”Access

to Account” (XS2A) interfaces enabling TPPs to ac-

cess customer accounts, given the customers’ pro-

vided their consent (Scheja and Machielse, 2019;

BG, 2018). However, while the PSD2 and the ad-

ditionally published regulatory technical standards

(RTS) require banks to implement an XS2A interface,

they do not specify a technical solution (Scheja and

Machielse, 2019).

The prescription for implementing secure XS2A

interfaces does not only lead to additional effort for

the bank implementing the interface but also for TPPs

who consume it. The TPPs need sufﬁcient time to

adapt their systems and continuously well perform-

ing interfaces to prevent disruption of their business.

Hence, the regulations deﬁne strict testing require-

ments for XS2A interfaces. Even now, after the tran-

sition phase (14.09.2019) has passed, the banks have

to ensure that the interface continuously meets func-

tional and performance requirements.

A major challenge in software testing is test case

generation, which is perceived as being a tedious and

erroneous process (Arcuri, 2019). In the context of

the PSD2 regulation, industry standardization institu-

tions design XS2A interface speciﬁcations and derive

test cases from these speciﬁcations. Thus, it would

be intuitive that banks can leverage the existing test

cases if they adapt the standard. However, banks usu-

ally implement XS2A interface standards with devi-

ations. These deviations lead to the need to create

new test cases or adapt the existing test cases for each

bank. To address this issue, we aim to answer the fol-

lowing research question: ”How can we design and

implement a test tool for XS2A interfaces exploiting

existing sets of test cases?”

We apply a design science research approach

(Hevner et al., 2004) to answer this research ques-

tion and conducted ﬁve expert interviews to iteratively

design the artifact. As a result, we present a test

tool concept that enables testing interfaces that devi-

ate form standards with one set of test cases. We show

our concept’s feasibility by implementing a prototype

and testing its functionality in a sandbox setting. The

test tool enables banks to save time on the design of

test cases and provides automatic testing facilities to

TPPs. Furthermore, we contribute to the scientiﬁc and

practical community by presenting a concept for efﬁ-

ciently testing RESTful Web APIs based on a stan-

dard that could also be transferred to other ﬁelds with

emerging interface standards.

Bondel, G., Kamysek, J., Kraft, M. and Matthes, F.

Design and Implementation of a Test Tool for PSD2 Compliant Interfaces.

DOI: 10.5220/0010439502490256

In Proceedings of the 23rd International Conference on Enterprise Information Systems (ICEIS 2021) - Volume 2, pages 249-256

ISBN: 978-989-758-509-8

249

2 FOUNDATIONS

We will introduce the background of the PSD1 and

PSD2, the importance of testing in the context of

PSD2, and the challenges of testing Web APIs.

2.1 Regulation of the European

Payment Market

The European Union introduced the ﬁrst Payment

Service Directive (Directive 2007/64/EC, abbreviated

PSD1) in 2007 with the goal to increase competition

between banks and payment service providers, lead-

ing to a broader choice of convenient, efﬁcient, and

secure cross-border payment services for end-users

within Europe. However, after the PSD1 became

effective, Third-Party Payment Service Providers

(TPPs) emerged and started offering new online and

mobile payment services (Cortet et al., 2016; Bram-

berger, 2019). More speciﬁcally, the regulators iden-

tiﬁed three services which are (1) the creation of an

integrated view of customer accounts even if differ-

ent banks manage these accounts, (2) the initiation

of a payment from an existing account, and (3) the

conﬁrmation that a certain amount of funds are avail-

able (EU, 2015). These new services have in common

that the TPPs do not operate customer accounts them-

selves. Instead, they access the customer’s account

operated by a bank to enable their new services. Due

to missing regulations and standards, only fragmented

interfaces to securely access these accounts existed

(Scheja and Machielse, 2019). Hence, TPPs often

resorted to screen-scraping online banking websites

using the end-users online banking login credentials

(Scheja and Machielse, 2019; Cortet et al., 2016).

Addressing the resulting security risks in the

evolving online payment services industry, in 2015,

the European Union published the Revised Payment

Services Directive (Directive (EU) 2015/2366, abbre-

viated PSD2) replacing the PSD1. The overall goal

of the PSD2 is to foster innovative online and mo-

bile payment services while at the same time ensuring

customer protection (Bramberger, 2019) (Zachariadis

and Ozcan, 2017). The PSD2 deﬁnes several mea-

sures of which we focus on the PSD2’s prescription

to allow TPPs standardized and secure access to cus-

tomer payment accounts (BG, 2018).

The PSD2 requires that banks and TPPs im-

plement secure communication standards to enable

TPPs to access customer accounts securely (EU,

2015). However, the PSD2 does not detail such stan-

dards or any speciﬁc technical solutions (Scheja and

Machielse, 2019). Instead, the PSD2 commissioned

the European Banking Authority (EBA) to issue reg-

ulatory technical standards (RTS) (Art. 98 PSD2).

The RTS, published in the ﬁnal version in March

2018, mandates that banks that provide online bank-

ing have to provide an ”Access to Account” (XS2A)

interface. An XS2A interface enables TPPs to request

information on payment accounts and to initiate pay-

ment orders securely (Art. 30 RTS). Although more

speciﬁc than the PSD2, the RTS still only provide a

high-level deﬁnition of the interface functionality and

do not detail any technical speciﬁcation (Scheja and

Machielse, 2019). Nevertheless, all regulated entities

have to comply with the RTS provisions and provide

an XS2A interface 18 months after the RTS was pub-

lished, i.e., latest on the 14.09.2019.

The development of detailed technical standards

is thus left to the banking industry (Scheja and

Machielse, 2019). Several standardization bodies

formed and have since provided technical speciﬁca-

tions of the XS2A interface, e.g., the Berlin Group

or the Open Banking Initiative

. Among these, the

Berlin Group is one of the major standardization bod-

ies since it created an XS2A interface speciﬁcation

in cooperation with 52 European banking entities, in-

cluding banks, banking associations, payment asso-

ciations, payment schemes, and interbank processors

active in the EU (Scheja and Machielse, 2019). The

XS2A interface speciﬁcation of the Berlin Group is

named the Berlin Group Next-GenPSD2 Framework

which deﬁnes the XS2A interface as a RESTful Web

API (BG, 2018).

2.2 Testing of XS2A Interfaces

Testing plays an essential role during the transition

to XS2A interfaces as well as afterwards. The RTS

prescribe functional and performance testing. Func-

tional requirements are mentioned in the RTS but not

speciﬁed in detail (Art. 30 RTS). Banks are instead

encouraged to follow standards to realize these func-

tional requirements (EBA, 2018b). Regarding perfor-

mance testing, banks need to prove that the XS2A

interfaces perform as good as the old interfaces, i.e.,

online banking websites. Performance testing is nec-

essary since latency in the TPP’s product due to in-

ferior interface performance could lead to dissatisfac-

tion and loss of TPPs’ and banks’ customers.

During the transition to the standardized XS2A in-

terfaces, testing played an essential role. Timely ac-

cess to XS2A testing and production environments al-

lowed TPPs to explore the new interfaces and to adapt

their systems before access to old interfaces was re-

voked. Also, the RTS introduced a mandatory contin-

https://www.berlin-group.org/

https://www.openbanking.org.uk/

ICEIS 2021 - 23rd International Conference on Enterprise Information Systems

250

Figure 1: Timeline of the PSD2 regulation.

gency measure, often referred to as fall back rule (Art.

33 RTS) (EBA, 2018b; EBA, 2018a) to enforce the

compliance with functional and performance require-

ments. The fall back rule prescribes that banks have

to enable TPPs to access customer payment accounts

not just through the XS2A interface, but also through

the customer-facing interface, i.e., the online bank-

ing website. However, the banks have to adjust the

customer-facing interfaces to comply with the same

security requirements as the XS2A interface, leading

to additional implementation effort for banks. An ex-

emption of the fall back rule is possible, if a bank

meets strict testing and reporting requirements (Art.

33 RTS). These requirements entail that banks have

to provide a test sandbox for connection and func-

tionality testing with mock data to certiﬁed TPPs as

well as appropriate documentation of the XS2A inter-

face half a year before the API goes into production

(14.03.2019) (Art. 30(5), 30(3) RTS). The productive

API has to be accessible for testing three months be-

fore the RTS becomes effective (14.06.2019). Dur-

ing that time, the bank has to promote the testing

of the XS2A interface by TPPs and address raised

issues (EBA, 2018b). After the testing period, the

banks have to report the number of TPPs that tested

the API, the issues raised, and measures to address

them and certain performance KPIs (EBA, 2018b).

An overview of the timeline is provided in Fig. 1.

After the transition period the essential role of

testing is not diminished. A bank has to continuously

ensure that the interface still meets the functional and

performance requirements, especially if the interface

implementation evolves. A failure to meet the re-

quirements, even after the end of the transition phase,

leads to the need to implement the fall back solution.

2.3 Challenges of Testing

Software development is a complex and error-prone

process that can lead to defects or unexpected behav-

ior in software systems or components. Thus, test-

ing activities are an essential part of software devel-

opment. Testing aims at evaluating the properties

of software and ensuring they meet the user’s needs

(ISO, 2013). Therefore, testing can prevent negative

impacts of releasing erroneous software, e.g., security

threats, losses, or a bad reputation (ISO, 2013).

As part of the test design and implementation, a

tester creates a test case for a test item. The test item

is the ”work product that is an object of testing” (ISO,

2013), e.g., the software component to be tested. The

test case is the ”set of test case preconditions, inputs

[. . . ] and expected results, developed to drive the ex-

ecution of a test item to meet test objectives” (ISO,

2013). The test execution can be performed manually

or automated with the help of a test tool.

In this research paper, the test item is a XS2A

interface of a bank. According to the Berlin Group

speciﬁcation, the XS2A interface is a RESTful Web

API. A Web API makes functionality or data available

via endpoints, that can be accessed over a network

using the HTTP protocol (Bermbach and Wittern,

2016). For each endpoint, the API designers deﬁne

operations and parameters of a successful API call.

REST is an architectural pattern that prescribes cer-

tain constraints to Web API design (Fielding, 2000).

In this context, a test case is one HTTP request to a

RESTful Web API.

A major challenge of testing is test case genera-

tion. Test case generation is tedious and error-prone

(Arcuri, 2019). Several approaches to enable auto-

mated test case generation for RESTful Web APIs

have been previously presented, e.g., search-based

testing (Arcuri, 2019) or test case generation based

on a formal model (Fertig and Braun, 2015).

The approach presented in this research paper cir-

cumvents the challenge of test case generation by en-

Design and Implementation of a Test Tool for PSD2 Compliant Interfaces

251

abling banks to test an XS2A interface by exploit-

ing preexisting test cases provided by standardiza-

tion bodies. Thus, banks do not have to create and

maintain their test cases. To the best of the authors

knowledge, no scientiﬁc paper addressing the testing

of XS2A interfaces have been previously published.

3 RESEARCH APPROACH

We apply a design science research approach since it

ensures relevance and rigor while focusing ”on cre-

ating and evaluating innovative IT artifacts that en-

able organizations to address important information-

related tasks” (Hevner et al., 2004). We describe the

research approach following the seven guidelines for

design science as presented by (Hevner et al., 2004).

Design as an Artifact. The goal of a design science

research approach is to create a purposeful artifacts

that address an organizational problem (Hevner et al.,

2004). The artifact presented in this research paper is

a concept of a prototypical implementation of a test

tool that enables the testing of several standardized

XS2A interfaces with a single set of test cases, even

if the XS2A implementations deviate from the stan-

dard. The organizational issue that the prototype ad-

dresses is the regulatory need to test XS2A interfaces

properly. The need to test XS2A interfaces is not lim-

ited to the PSD2 transition phase in 2019, but regres-

sion testing is also valuable afterwards since the in-

terface implementation is continuously evolving. Fur-

thermore, the concept could be applied to other ﬁelds

with emerging interface standards, e.g., in the health

sector with the Fast Healthcare Interoperability Re-

sources (FHIR) standard (Kasthurirathne et al., 2015).

Problem Relevance. In design science, the de-

veloped artifact has to solve a signiﬁcant business

problem. Our attention has been brought to test-

ing of XS2A interfaces through discussions with in-

dustry experts specializing in XS2A implementation

projects. The implementation and testing of XS2A

interfaces is mandatory for retail banks but the ambi-

tious timeline for implementing and testing the inter-

faces is a challenge. Also, when evolving the inter-

face, regression testing is necessary. Our artifact en-

ables banks to save time on the design of test cases

and to provide automatic testing facilities to TPPs.

The relevance of the problem is further highlighted

by the emergence of several start-ups and consult-

ing companies that specialize in providing testing so-

lutions for XS2A interfaces, e.g., adorsys

, Forge-

https://adorsys.com/de/produkte/xs2a-compatibility-

test-kit/

Rock

, and Tieto

Design Evaluation. We use two evaluation methods

to show the ”utility, quality, and efﬁcacy” (Hevner

et al., 2004) of our design artifact for addressing the

problem of testing XS2A interfaces. First, we con-

ducted ﬁve interviews with experts for XS2A inter-

faces and API testing to iteratively evaluate and ad-

vance the artifacts. The experts have different roles

including a product owner, a head of department,

a test manager, a software architect, and a start-up

founder and are employed at different types of or-

ganizations including a retail bank, banking service

providers, and IT service providers. Secondly, we

show the feasibility of our concept by implementing

a prototype and testing its functionality in a sandbox

setting.

Research Contributions. The research contribution

is the artifact itself, i.e., a concept and a proof of con-

struction of a test tool for testing XS2A interfaces us-

ing a predeﬁned set of test cases. The contribution is

novel since the current practice is that banks design

their own test cases and build their own test tools,

which is error-prone and costly. Thus, with the test

tool, testing XS2A interfaces can be realized more ef-

ﬁciently. Furthermore, the concept can be transferred

to other areas with standardized interfaces.

Research Rigor. Research rigor means that ”The ar-

tifact itself must be rigorously deﬁned, formally repre-

sented, coherent, and internally consistent.” (Hevner

et al., 2004). Thus, we describe the artifact as well as

any assumptions and limitations in sections 4 and 5.

Design as a Search. The design science process is

”essentially a search process to discover an effective

solution to a problem” (Hevner et al., 2004). We de-

signed the test tool concept and prototype between

December 2019 and April 2020. During that time, we

conducted expert interviews to evaluate and advance

the artifacts. Furthermore, we regularly presented our

intermediate results to our main industry partner and

discussed improvement potentials.

Communication of Research. The results of design

science need to be communicated to a technical and

managerial audience (Hevner et al., 2004), hence we

motivate the research endeavor understandably and

provide information to reconstruct the artifact in ap-

propriate settings.

https://www.forgerock.com/open-banking-sandbox

https://openbanking.api.tieto.com/

ICEIS 2021 - 23rd International Conference on Enterprise Information Systems

252

Figure 2: Test tool concept.

4 CONCEPT OF A XS2A

INTERFACE TEST TOOL

In this section we present the test tool concept and the

roles involved in operating the test tool.

4.1 Test Tool Concept

We assume that standardized interface speciﬁcations

and associated test cases exist. One or several banks

have implemented an XS2A interface guided by the

standardized interface speciﬁcation, but the actual im-

plementations diverge from the speciﬁcation. The

goal is to nevertheless test the API implementations

with the predeﬁned set of test cases. Thus, two chal-

lenges arise that have to be addressed.

First, the structure of valid API calls can differ be-

tween the implemented API and the test cases. Our

concept addresses this issue with mapping ﬁles, that

map the structure of each implemented API endpoint

with the structure deﬁned in the standardized speci-

ﬁcation. A mapping ﬁle describes how a standard-

ized test case is modiﬁed by the test tool to generate

a valid API request to a speciﬁc interface implemen-

tation. The mapping ﬁles are an input to the test tool

and can easily be adapted. Hence, the test tool itself

does not have to be modiﬁed if a bank changes its API

implementation or a new API is tested.

The second challenge is that the banks’ systems

store different data. If a tester executes a predeﬁned

test case without considering the deviation in (test)

data, a test case will fail even if the bank implemented

the API functionality correctly. We address this issue

by designing the test cases in a data-driven way, i.e.,

the data is separated from the test case structure and

can easily be adapted for each API implementation.

In summary, we present a test tool concept for test-

ing APIs which use the same standardized speciﬁca-

tion as a starting point but deviate with regards to the

implemented endpoint structure and the test data (see

Fig. 2). A standardization body prescribes the API

speciﬁcation and the derived test cases. The test cases

with the corresponding test data and the mapping ﬁles

are inputs to the test tool. The test tool uses the in-

puts to generate an API call taking into account the

speciﬁcs of the actually implemented API. The test

tool then makes the API call to the implemented API

and receives a response. The tool evaluates if the re-

sponse matches the expected response and creates a

test report. The concept addresses the differences in

the endpoint structure with a mapping ﬁle and the dif-

ferences in the test data with a data-driven testing ap-

proach. Thus, the test tool itself does not have to be

adapted if an API implementation and the test data

changes or if a new API is tested.

4.2 Involved Roles

In this subsection, we present ﬁve relevant roles for

operating the test tool. The ﬁrst role is the standards

provider, who creates a standard, usually in cooper-

ation with several interested parties, e.g., the Berlin

Group. The second role is the test case provider,

who derives a set of test cases from an XS2A stan-

dard. This role can be taken by an industry standard-

ization body with expertise in testing, e.g., the NISP.

The test tool provider maintains the test tool. The

test tool provider’s tasks are to adapt the test tool in

case standards change and to add new standards if

they emerge. This role should be taken by an inde-

pendent party.

The mapping ﬁle provider needs to know the

XS2A interface standard and the implemented XS2A

Design and Implementation of a Test Tool for PSD2 Compliant Interfaces

253

interface. This role could be taken by an external

party as long as the bank provides good API docu-

mentation. However, since a banks XS2A develop-

ment team knows the actual implementation best and

has studied the standardized speciﬁcation, this team is

particularly suitable to create the mapping ﬁle.

Finally, the test tool user utilizes the tool to test an

XS2A interface. The test tool user needs to have the

right test data for a speciﬁc XS2A interface. Thus, the

test tool user could be a bank that wants to test their

own interfaces. Also, a TPP with an eIDAS certiﬁcate

and access to test data could use the test tool to verify

an XS2A interface functionality.

5 PROTOTYPICAL

IMPLEMENTATION

In this section, we present a prototypical implemen-

tation of the concept of a test tool that enables testing

of different XS2A interfaces. Furthermore, we report

on the challenges that we encountered.

5.1 Test Tool Implementation

Assumptions. First, we want to disclose the un-

derlying assumptions of the prototype. We choose

the Berlin Group Next-GenPSD2 Framework as the

XS2A interface standard since it is a dominant stan-

dard used by many banks in Europe. Also, it is freely

accessible. The Berlin Group provides a detailed

XS2A interface speciﬁcation, including the paths of

all endpoints and the methods, parameters, and pos-

sible responses for each endpoint. The Berlin Group

standard deﬁnes JSON or XML as the data exchange

format in the API request and response bodies. In our

prototype, we choose the data exchange format JSON,

since it is the most common media type used in Web

APIs (Arcuri, 2019). We use the well-recognized li-

brary REST-assured

to execute test cases.

Inputs of the Test Tool. A test tool user has to pro-

vide three types of inputs for the test tool to work.

These inputs are the test cases, the mapping ﬁles, and

a certiﬁcate. We will describe each of these inputs.

Test Cases. The test tool user has to provide the

test cases derived from the standardized XS2A inter-

face speciﬁcation. The NISP

derived test speciﬁ-

cations from the Berlin Group XS2A speciﬁcation

however, these test speciﬁcations are not publicly ac-

http://rest-assured.io/

https://nisp.online/

https://berlingroup.stackstorage.com/s/1FBrOlC7Iquz

G35B

cessible. Also, no information regarding the format

of these test cases is available. Hence, we deﬁne test

speciﬁcations using a JSON schema.

The test case contains test metadata, all data nec-

essary for generating the API request to be tested,

and the expected API response. The test metadata in-

cludes a unique test ID, a test name, and a descrip-

tion. Each test case holds all information for one API

call, but the tester can indicate that several test cases

belong together in the metadata. The test tool then

executes these test cases sequentially. Being able to

execute test cases sequentially can be necessary for

XS2A interfaces since an interaction often consists of

ﬁrst verifying if a client authorized an action and then

performing said action, e.g., triggering a payment.

Additionally, the test case holds all information to

create a valid API call according to the Berlin Group

speciﬁcation, including the endpoint, method and pa-

rameters structure and names. Also, the test case con-

tains the expected response to the API request, i.e., a

speciﬁc status code. This information enables the test

tool to evaluate if the test was successful or not.

Mapping Files. The test service user has to pro-

vide two mapping ﬁles for each test case, a header

and a body payload mapping ﬁle. The mapping ﬁles

have two purposes. First, they provide speciﬁcs for

accessing a speciﬁc bank’s interface, e.g., the host

and port. Secondly, they account for the differences

between the standardized API speciﬁcation and the

actual API implementation. The mapping ﬁles are

JSON ﬁles that use a speciﬁc notation to map vari-

ations in the naming or structure of the implemented

API and the standardized API speciﬁcation. Thus, the

mapping ﬁles are basically JSON API adapters.

At this point, we will shortly describe the nota-

tion used to map two JSON structures. JSON deﬁnes

two data structures, which are objects and arrays. In

our notation, we characterize an object with its name

and a subsequent dot (.). The string after the dot is

the name of the object, array or key of a key/value

pair located inside the object. An exception is the

highest-level object, the JSON object, that is indicated

by a dot preceding the ﬁrst element inside the object.

An array is described with a subsequent colon (:) fol-

lowed by a number indicating the position of the rel-

evant value inside the array. The number is followed

by a dot and the name of the object, array, or key of

a key/value pair located at that position in the array.

To make the notation clear, we present the following

example.

".customer.accounts:0.balance"

In this example, the object ”customer” is located

in the JSON object. The object ”customer” contains

ICEIS 2021 - 23rd International Conference on Enterprise Information Systems

254

an array ”accounts”, and a key/value pair with the key

“balance” is located at position 0 within the array.

We also provide an example of the mapping be-

tween two JSON schemas. The schema reﬂecting the

standardize API speciﬁcation is placed on the left of

the colon with whitespaces around it. The schema re-

ﬂecting the actual implementation of the API is on the

right side of the colon.

".customer.acc:0.balance" :

".customer.acclist:2.balance"

The mapping in this example accounts for differ-

ences in the naming within the JSON schemas, i.e.,

the array is named ”acclist” instead of ”acc”. In the

prototype, this would reﬂect a difference in the nam-

ing of endpoints or parameters. Also, the key/value

pair with the key “balance” is stored in the third in-

stead of the ﬁrst position in the array, which accounts

for a change in the structure of the JSON schemas.

".customer.acc:0.balance" :

".creditcard.customer.acc"

Also, the notation can handle more severe differ-

ences in the structures of the JSON schema as shown

in this last example. Instead of a nested structure with

an object, an array, and a key, the implemented API is

reﬂected in a JSON schema nesting two objects and a

key.

Certiﬁcate. Finally, the test tool user needs to

provide her ”Electronic Identiﬁcation, Authentication

and Trust Services (eIDAS)” certiﬁcate in every API

request. The eIDAS certiﬁcate veriﬁes a tester’s iden-

tity and ensures that the respective national bank-

ing regulation institution approved the tester. A re-

quest to an XS2A interface without an eIDAS certiﬁ-

cate will invariably fail. In Germany, an organiza-

tion or tester has to undergo a complicated registration

process with the Bundesanstalt f

ur Finanzdienstleis-

tungsaufsicht (BaFin) to receive the eIDAS certiﬁcate

(EBA, 2018c).

Test Tool Execution. Finally, we describe the process

of the test tool execution. First, the test tool checks

if the test service user provided all necessary inputs,

i.e., the test case, the mapping ﬁles, and a certiﬁcate.

Secondly, the test tool transforms the generic test case

speciﬁcation into a test case that accounts for a bank’s

API’s speciﬁcs using the mapping ﬁles. Thirdly, the

tool generates an API request from the transformed

test case and routes it to the respective banks’ API.

Once the test tool receives the response, the response

is modiﬁed back from the speciﬁc bank API response

to the generic Berlin Group speciﬁcation. Finally, the

test tool compares the API response message to the

expected response in the test case derived from the

Berlin Group speciﬁcation and determines if the test

was successful.

5.2 Limitations of the Test Tool

Overall, we implemented and tested the prototype

successfully, thus providing a proof of construc-

tion. However, we also encountered some challenges

which lead to limitations. First of all, the test tool

can currently only test XS2A interfaces based on the

Berlin Group standard. Furthermore, the test tool

is limited to testing the eight API endpoints neces-

sary for the account information service (AIS) use

case. The AIS use case entails the creation of account

holder consent and retrieval of information on the bal-

ance and transactions of the account, e.g., to create a

service providing an integrated view of the account

holder’s accounts operated by different banks.

Furthermore, since we did not have access to the

NISP test cases, we derived test cases for the Berlin

Group speciﬁcation ourselves.

Another challenge is that we could not access

XS2A test sandboxes or productive interfaces of

banks. The access to these sandboxes and productive

interfaces is limited to TPPs with a valid eIDAS cer-

tiﬁcate, i.e., TPPs approved by the respective national

banking regulation institution. Since we could not ob-

tain such a certiﬁcate, we resort to testing the test tool

using a publicly available sandbox of a ﬁnancial ser-

vices provider

. The sandbox is based on the Berlin

Group standard but slightly differs in some points.

Finally, the last challenge that we want to report is

the handling of the customer authentication process.

Except for a few exceptions, most use cases consist

of two steps. First, an account owner has to provide

her consent on an action performed on her account.

Afterward, the respective operation can be execute,

e.g., a payment is triggered. As part of giving con-

sent, the customer has to be authenticated by a strong

customer authentication (SCA) approach (EU, 2015).

Even though many different approaches are deﬁned,

in most cases, a combination of a login and a One

Time Password (OTP), e.g. a TAN, is used. Automat-

ing this process in a test tool is complex.

In summary, we had to make some assumptions

and limit the test tool’s scope, mainly due to a lack of

access to speciﬁc resources.

https://adorsys.com/de/produkte/xs2a-sandbox/

Design and Implementation of a Test Tool for PSD2 Compliant Interfaces

255

6 CONCLUSION

The PSD2 prescribes banks to provide secure ac-

cess to customer accounts via XS2A interfaces that

continuously meet predeﬁned functional and perfor-

mance requirements. Thus, testing is essential. A

known challenge in software testing is creating test

cases since it is tedious and error-prone(Scheja and

Machielse, 2019). For XS2A interfaces, standardiza-

tion bodies have designed speciﬁcations and already

derived test cases from these speciﬁcations. However,

the standards are not legally binding, and the actual

implementations of the XS2A interfaces often deviate

from the standardized speciﬁcation. Thus, banks have

to adapt their test cases or create new ones. To address

this issue, we address the research question ”How can

we design and implement a test tool for XS2A inter-

faces exploiting existing sets of test cases?”

We answered the research question by applying

a design science research approach including inter-

views with ﬁve experts to create a concept and a pro-

totypical implementation of a test tool for XS2A in-

terfaces. We addressed the issue of interface imple-

mentations diverging from standardized implementa-

tion using mapping ﬁles. We use a simple notation to

map changes between the structure and naming of the

speciﬁcation and the implementation. Furthermore,

we introduce relevant roles for operating the test tool.

A bank beneﬁts from the artifact since it can use a pre-

deﬁned and collaboratively developed set of test cases

for testing its XS2A interface instead of having to de-

sign test cases, which is an effortful and error-prone

task. Also, the bank can use an existing test tool and

provide access to the test tool to TPPs. Finally, the

concept can be applied to other ﬁelds with emerging

interface standards.

Several limitations of the tool have been men-

tioned in section 5.2. Thus, our future work will focus

on evaluating the test tool in a real-world setting, i.e.,

using the test cases created by the NISP and testing

the tool with a productive XS2A interface. Further-

more, we will extract general guidelines and ﬁndings

on testing standardized interfaces and evaluate them

by transferring them to other industries with interface

standards.

REFERENCES

Arcuri, A. (2019). Restful api automated test case genera-

tion with evomaster. ACM Transactions on Software

Engineering and Methodology (TOSEM), 28(1):1–37.

Bermbach, D. and Wittern, E. (2016). Benchmarking web

api quality. In International Conference on Web Engi-

neering, pages 188–206. Springer.

BG (2018). Joint initiative on a psd2 compliant xs2a in-

terface - nextgenpsd2 xs2a framework - operational

rules. Technical report, Berling Group. Technical Re-

port.

Bramberger, M. (2019). Open Banking: Neupositionierung

europ

aischer Finanzinstitute. Springer.

Cortet, M., Rijks, T., and Nijland, S. (2016). Psd2: The

digital transformation accelerator for banks. Journal

of Payments Strategy & Systems, 10(1):13–27.

EBA (2018a). Commission delegated regulation (eu)

2018/389 of 27 november 2017 supplementing direc-

tive (eu) 2015/2366 of the european parliament and of

the council with regard to regulatory technical stan-

dards for strong customer authentication and common

and secure open standards of communication (text

with eea relevance). Ofﬁcial Journal of the European

Union.

EBA (2018b). Final report - guidelines on the condi-

tions to beneﬁt from an exemption from the contin-

gency mechanism under article 33(6) of regulation

(eu) 2018/389 (rts on sca & csc). Technical report,

EBA. Technical Report.

EBA (2018c). Opinion of the european banking authority

on the use of eidas certiﬁcates under the rts on sca and

csc. Technical report, European Banking Authority.

Opinion.

EU (2015). Directive (eu) 2015/2366 of the european par-

liament and of the council of 25 november 2015 on

payment services in the internal market, amending di-

rectives 2002/65/ec, 2009/110/ec and 2013/36/eu and

regulation (eu) no 1093/2010, and repealing directive

2007/64/ec (text with eea relevance). Ofﬁcial Journal

of the European Union.

Fertig, T. and Braun, P. (2015). Model-driven testing of

restful apis. In Proceedings of the 24th International

Conference on World Wide Web, pages 1497–1502.

Fielding, R. T. (2000). Rest: architectural styles and the de-

sign of network-based software architectures. Techni-

cal report, University of California. Doctoral disserta-

tion.

Hevner, A. R., March, S. T., Park, J., and Ram, S. (2004).

Design science in information systems research. MIS

quarterly, pages 75–105.

ISO (2013). Iso/iec/ieee 29119-1: Software and systems

engineering — software testing — part 1: Concepts

and deﬁnitions — ﬁrst edition 2013-09-01. Technical

report, ISO.

Kasthurirathne, S. N., Mamlin, B., Kumara, H., Grieve, G.,

and Biondich, P. (2015). Enabling better interoper-

ability for healthcare: lessons in developing a stan-

dards based application programing interface for elec-

tronic medical record systems. Journal of medical sys-

tems, 39(11):182.

Scheja, O. and Machielse, W. (2019). The nextgenpsd2

framework in a pan-european psd2 account access

context. Journal of Payments Strategy & Systems,

13(1):54–65.

Zachariadis, M. and Ozcan, P. (2017). The api economy and

digital transformation in ﬁnancial services: The case

of open banking. Technical report, SWIFT Institute.

Working Paper.

ICEIS 2021 - 23rd International Conference on Enterprise Information Systems

256