A Tool for Developing and Evaluating Methods (TDEM)

Murad Huseynli, Michael Chima Ogbuachi and Udo Bub

Faculty of Informatics, E

otv

os Lor

and University (ELTE), P

azmany P

eter s

any 1/C, H-1117 Budapest, Hungary

Keywords:

Information Systems, Digital Innovation, Design Science Research, Method Engineering, Method Design,

Method Evaluation, Process Evaluation, Microservices.

Abstract:

Digital Innovation (DI) is the use of digital technologies during the process of innovation or as the result of in-

novation. While we see a huge amount of interest in DI research from IS scholars, there is still scarcity in terms

of using Design Science Research (DSR) for the engineering of DI, speciﬁcally from a method perspective and

a possible integration of both ﬁelds. In this paper, we propose a tool for developing and evaluating methods to

build method design theories, not just in the DI ﬁeld, but also generally applicable to method engineering in

other ﬁelds as well. Thus, we contribute systematically to the body of knowledge for information systems with

a tool that shall be an effective assistant in method engineering. The tool builds on the eight components of a

design theory ﬁrst introduced by (Gregor and Jones, 2007) and conceptualized by (Offermann et al., 2010a)

for the artifact type “method”. Finally, we analyze the role and utility of the tool in detail, concentrating on a

method for the engineering of DI, in a DI project related to the microservices architecture, and lastly, we show

generalizability of the tool in terms of design process evaluation, not speciﬁcally following a DSR paradigm.

1 INTRODUCTION

Design science research is generally applied to classes

of artifacts, including algorithms, computer or hu-

man interfaces, design methods and techniques, and

process models. It is primarily applied in engineer-

ing and computer science, but is not limited to these

ﬁelds and can be found in many other disciplines and

ﬁelds. Since the research paradigm of design science

focuses on the creation of an innovative artifact, it

contributes to the ﬁeld of digital innovation from both

a research and practical perspective. Digital innova-

tion describes the use of digital technologies as an aid

during the innovation process, or as an outcome of

innovation. Digital innovation is increasingly becom-

ing a dominant research focus in the ﬁeld of informa-

tion systems. Although the pursuit of innovation is

crucial for forward-looking companies (Bub, 2018),

little attention is paid to the systematic development

of digital innovation in related ﬁelds such as IS and

computer science, which play a key role in digital

transformation. Innovation plays an important role in

economic progress and consequently, in human well-

being (Baumol, 2002). Information technologies are

developing at a rapid pace, so digital innovation based

on digital technologies has become the focus of at-

tention. Basically, digital innovation combines phys-

ical and digital entities and creates new products on

this basis (Hevner and Gregor, 2020). As digitized

products and services are among the key drivers of

innovation, digital innovation is now practiced by a

growing number of companies (Berntsson-Svensson

and Taghavianfar, 2015). At its core, digital innova-

tion is about novel digital technologies, entrepreneur-

ship and innovation management, and digitization of

information and services.

At its essence, DSR in Information Systems (IS)

is about DI (Hevner and Gregor, 2020). While this

much synergy exists between both ﬁelds, not many

studies have been conducted regarding method engi-

neering for DI, using DSR. This gap was the moti-

vation for this research. As we looked into existing

literature we found that we could not identify a scien-

tiﬁc tool that allowed researchers to perform proper

method engineering, not only in DI, but also in any

other IS ﬁelds where DSR methodologies can make a

huge impact (considering research processes and out-

comes). DSR is a generally accepted paradigm of In-

formation Systems (IS) research (Hevner et al., 2004;

Peffers et al., 2008). Among the artifact types that

are subject to design in IS, “method” is of high rel-

evance and subject to intensive research work in the

IS community (Hevner et al., 2004; Bucher and Win-

ter, 2008; Offermann et al., 2010a). Reviews of ex-

Huseynli, M., Ogbuachi, M. and Bub, U.

A Tool for Developing and Evaluating Methods (TDEM).

DOI: 10.5220/0011077700003179

In Proceedings of the 24th International Conference on Enterprise Information Systems (ICEIS 2022) - Volume 2, pages 543-552

ISBN: 978-989-758-569-2; ISSN: 2184-4992

543

isting literature (Huseynli et al., 2021) showed that

there is still scarcity regarding methods for the engi-

neering of DI using DSR. (Bub, 2018) takes the ini-

tiative for the development of such a method and puts

the basis for it. This effort, though, needs further val-

idation and improvement. Designing such a method

requires a thorough, comprehensive knowledge base

with both scientiﬁc rigor and practical relevance. Re-

searchers face challenges in identifying academic in-

sight and progress in comparison to the state of the

art, due to the complex nature and diverse ways of

describing method artifacts. While it is unavoidable

to put much effort into identifying how to design and

evaluate methods, to support these activities, we in-

troduce a tool for use in method development and

evaluation. Our proposal is scientiﬁcally based on

IS Design Theories (Gregor, 2006) and preliminary

practical considerations (Bub, 2020). In their seminal

work, (Gregor and Jones, 2007) have identiﬁed eight

components for an IS Design Theory: “Purpose and

Scope”, “Constructs”, “Principles of Form and Func-

tion”, “Artifact Mutability”, “Testable Propositions”,

“Justiﬁcatory Knowledge”, “Principles of Implemen-

tation”, and “Expository Instantiation”. Their work

targets components of a Design Theory of Design and

Action, meant to be independent of any artifact types

and that require further detailing for concrete imple-

mentation (i.e. to deﬁne method artifacts). (Offer-

mann et al., 2010a) have built on their work and have

proposed detailed advice on how to describe methods

in a scientiﬁc manner, by leveraging the aforemen-

tioned components to build method Design Theories.

We further took the identiﬁed concepts from both re-

search works and built a tool for developing and eval-

uating methods based on them. We abbreviate this

tool as TDEM (A Tool for Developing and Evaluat-

ing Methods) and we illustrate the utility of TDEM

with a thorough analysis.

We structured the overall process as follows. The

concept of TDEM for Design Science in IS is intro-

duced and its components are thoroughly explained.

Subsequently, an analysis of the potential uses of

TDEM is provided and two aspects are investigated:

the analytical and ideational points of view. For the

ﬁrst one, an example is shown by providing a brief

study (evaluation) on a scientiﬁc paper, while for

the latter, a design project of DI in microservices

was developed and is introduced, which discusses

how TDEM participates in the ideation process and

helps establish scientiﬁc knowledge in Design Sci-

ence Research. Finally, we show the generalizability

of TDEM on a non-DSR paper in terms of process

evaluation.

2 BACKGROUND

In this section, we closely look into DI, DSR, and

Method Engineering from a theoretical point of view.

2.1 Digital Innovation

Digital technologies transform organizations world-

wide to a great extent. DI can be seen as the appli-

cation of digital technologies to business problems.

(Hevner et al., 2019) explains digital innovation as

the appropriation of digital technologies during the

process of and as the outcome of innovation. DI en-

compasses several directives just like developing a

novel technology or technology strategy in a business

context, adopting, and implementing technology re-

sources, and digitizing work activities within an or-

ganization. Organizations that innovate or transform

come across with several beneﬁts, such as streamlin-

ing business processes by automating and integrating

different technological artifacts and cutting costs as a

result of having digital solutions, having a better re-

turn on investment, and boosting revenue streams.

2.2 Design Science Research

Design Science Research (DSR) has seen a vast

amount of interest in Information Systems (IS), as

an outcome-based research methodology that seeks

to create novel technological artifacts. In its essence,

Design Science (DS) pursues exploring, comprehend-

ing, and rectifying potential components, constructs,

and methods to create and position original artifacts

which are capable of solving an organizational prob-

lem in a natural setting (Baskerville, 2008). DSR

aims to further improve the designed artifacts through

its iterative approach, focusing on their development

and evaluation. The output artifact resulting from

DSR can be software, an algorithm, a mathemati-

cal equation, frameworks, process models, and so on.

DSR outputs can be categorized in constructs, mod-

els, methods, implementations, architectures, design

principles, frameworks, instantiations, and theories as

described by researchers (Chatterjee, 2015; Vaishnavi

and Kuechler, 2004). The all-round learning and es-

tablishment of knowledge, where it is essential to de-

liver innovative artifacts, is at the heart of Design Sci-

ence Research. The actual creation process generally

also covers the improvement of design artifacts and

processes in their setting. As the iterative evaluation

is the main part of DSR, it provides a ﬁner explica-

tion of the problem and provides feedback to improve

the quality and efﬁcacy of the artifact and its design

process.

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

544

2.3 Method Engineering

Initially, (Bergstra et al., 1985) came up with the term

“Method Engineering” (ME), then (van Slooten and

Brinkkemper, 1993) and (Brinkkemper, 1996) fur-

ther supported the idea of ME. ME in IS focuses

on designing, constructing, and evaluating methods,

procedures, tools, and techniques to enhance infor-

mation systems development (Brinkkemper, 1996).

While computer engineering deals with hardware and

ﬁrmware aspects, ME focuses on methods, proce-

dures, and techniques in all engineering activities,

since ME can relate to other research areas including

project management, software conﬁguration manage-

ment, software engineering environments, and soft-

ware process modelling (Brinkkemper, 1996). In

addition, methods are the fundament of goal-driven

and tactical actions in various application ﬁelds. A

Method incorporates several parts that make the in-

formation systems development process more man-

ageable (Sunyaev et al., 2008). There are several

concepts regarding methods, such as Method Chains

and Alliances (Nilsson, 1999), which are necessary

when performing integration of methods and meth-

ods parts. Among the latter, we have: Method Com-

ponents (Karlsson and Wistrand, 2003), which are de-

ﬁned as being part of a method, consisting of concept,

notation, and process; Method Fragments (Harmsen

et al., 1994), which are building blocks of methods;

Method Chunks (Ralyt

e and Rolland, 2001), which

are the combination of the existing process and prod-

uct fragments.

3 A TOOL FOR DEVELOPING

AND EVALUATING METHODS

(TDEM)

(Gregor and Jones, 2007) established a design the-

ory meant for information systems research, consti-

tuted by eight components. These eight components

were taken by (Offermann et al., 2010a) and reﬁned,

characterizing an implementation that was speciﬁc for

methods. In fact, it is important to remember that the

original components were meant to deﬁne a process

or structure for building Design Theory, which means

they were meant for a much more generic scope. The

aforementioned reﬁnements added important opera-

tional detail in the structure, mostly in two forms: at-

tributes of a component and evaluation criteria.

The attributes give added focus and structure and

help deﬁne the elements that are most important for

the deﬁnition of a method. A simple way of men-

tally visualizing these, which is particularly close to

people working in Information Systems, would be by

thinking of them like the attributes that deﬁne a class

in Object Oriented Programming (OOP), which are

the elements that constitute the internal properties of

a type, thus characterize it (where a class is one of the

eight components by (Gregor and Jones, 2007)). And

again, as with the OOP class attributes, these can also

be either singletons or collections of elements (some

attributes may be speciﬁed by a plurality of elements,

depending on the type of method that is being de-

signed). See Figure 1 for the structure of TDEM.

The evaluation criteria constitute an inbound ana-

lytical tool for the process and are expressed in the

form of questions. Each component has its set of

questions (criteria), which allow a segmented study

of the method that is being created, speciﬁc for each

component. The questions are meant to be able to ad-

mit both Boolean (of the yes/no type) and complex

answers. This level of detail, considering segmenta-

tion and structure of the questions, is very important,

as it makes it possible to understand if the process

for building a method is being utilized “correctly”, or

at least in a coherent manner. These questions were

selected through a hermeneutical analysis of the com-

ponents and revisited several times (Offermann et al.,

2010a). They are posed in a way that would push the

user to deeply assess the strategy that is being used to

structure the method.

So, to brieﬂy recapitulate, the process to structure

a method should include the use of these eight compo-

nents of Design Theory, tailored to be used within the

area of Design Science Research. Each of these com-

ponents is structured in such a way that they have at-

tributes, to deﬁne them, and evaluation criteria, which

help make an in-depth analysis of the method that is

being built.

We gather all this important preparatory work as

input for creating a new tool, TDEM. This can be fun-

damentally explained as a one-pager tool that gives

space to all the augmented versions of the eight com-

ponents of Design Theory, as explained in the previ-

ous paragraphs. This characterization as a tool makes

it easier to understand the usage of the components

and provides an easy way to arrange, also at a graph-

ical level, such an abstract analytical process. Specif-

ically, this can be seen as a tool to perform an in-

depth study of a method and the artefacts that are pro-

duced as a result of the Design Science Research pro-

cess. Further in this section, the eight components are

shown in detail.

Purpose and Scope - This component describes

what a method is meant to achieve and what area of a

research ﬁeld in Design Science it should cover. Pur-

pose statements should provide information about the

A Tool for Developing and Evaluating Methods (TDEM)

545

A Tool for Developing and Evaluating Methods (TDEM): <Name of the Method>

Purpose and Scope

 Project type

 Project context

 Lifecycle coverage

 Activity coverage

Evaluation criteria: Given a problem, is the purpose described in a way to tell if the method solves the problem? Is the product of the method clear? Given a problem context, is the scope described in a way to tell if

the context falls within the scope? Are all relevant dimensions specified within the scope? Are the covered lifecycle phases, roles and activities specified?

Constructs

 Method specific constructs, meta-model

 Output-specific constructs

 Concepts in the application context

Evaluation criteria: Are terms and concepts presented to describe the method? ... describe the resulting

structure? ... describe the application context? ... describe the purpose and scope? ... create the testable

propositions? Are the concepts introduced comprehensively in one location and derived systematically?

Testable Propositions

 Utility statement about theory

 Optional: truth statements about match between method design and requirements of

method output

Evaluation criteria: Are testable propositions concerning the method utility given? Are the

propositions sufficiently well operationalised to allow testing by other scientists?

Principles of Form and Function

 Description of the method (according to the meta-model selected above)

Evaluation criteria: Is the method described extensively enough to be useful and transferable? For each

possible role, is it clear which activities have to be done in what order?

Justificatory Knowledge

 Design theories that this method is based on

 Theories about the application context

 Other aspects of interest

Evaluation Criteria: Are theories presented for key method features not covered by testable

propositions? ... to support the testable propositions? ... regarding the method product?

... regarding the method setting?

Artifact Mutability

 Foreseeable changes (which part of the method, method base or method instance, kind of change)

 Optional: support for change offered by method design paradigm (i.e. situational, method rationale)

Evaluation criteria: Is the method described extensively enough to understand the functioning of each

component? Are conditions for changes described? Does it become clear, which parts change and what they

change into? Are method tailoring mechanisms defined?

Principles of Implementation

 Tailoring/assembly advice

 Advice regarding introduction into real-life setting

Evaluation criteria: Is the tailoring/assembly advice comprehensive with regard to the combinatorial

possibility? Does the advice for introducing the method cover

different settings within the scope, or is at least clear for which it applies?

Expository Instantiation

 Example of an instantiated method

 Report on a real-life case study in which an instantiated method has been enacted

Evaluation criteria: Does the example cover a case within the scope and is it covering the main concepts of the method? Is the example specific enough to be illustrative but not too idiosyncratic to be

incomprehensible?

Figure 1: TDEM based on (Offermann et al., 2010a; Gregor and Jones, 2007).

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

546

type of output that the usage of a method should yield,

the properties of the output, and some statements re-

garding the method itself. On top of that, the scope of

the method is also described from an external and an

internal point of view.

Scientiﬁc Role: Purpose and scope give the basis

for the scientiﬁc signiﬁcance of a theory or method, as

they provide a clear vision of the end goal for which

they are being developed, and describe what the re-

search area should “look like”. This strongly con-

tributes to providing the generalisability that should

be typical of a proper method or theory. When the

scope is well deﬁned, it is easier to generalize the va-

lidity of the scientiﬁc process.

Constructs - Constructs are properties that can

be identiﬁed either from the method itself, from the

structure of its output, or the context of the enactment

of such method. One of the concepts that concretely

participate in the constructs is that of Method itself, as

methods can be generated from other method models

or meta-models. Other constructs might come from

product classes or justiﬁcatory knowledge as well.

Scientiﬁc Role: The constructs do not provide in-

formation that would directly help the evaluation of a

theory or method, but they do help in setting the fun-

dament for the description of all the other components

of the process.

Principles of Form and Function: This compo-

nent describes what the abstract architecture of the

artefact should look like, especially when the arte-

fact is a system. In case the artefact is a method,

then there is no speciﬁcation for different architec-

tural models, but situational context is considered. A

general method is described, which can be seen as

a set of individual solutions. The principles of form

and function establish a strict relation between the el-

ements of the method and the meta-model that deﬁned

the internal structures. The way such elements are de-

scribed must comply with the representation coming

from the metamodel and having justiﬁcatory knowl-

edge would be an added point.

Scientiﬁc Role: This component is strictly neces-

sary for a method to be useful. In fact, it provides

the information that allows the understanding of the

purpose of the method, which is necessary for its co-

herent use.

Artefact Mutability: Mutability during a re-

search process can happen both on the design stage of

the method, or on a particular instance of the method

itself. For methods, speciﬁcally, two types of mutabil-

ity are identiﬁable. The ﬁrst kind identiﬁes the muta-

bility, changes on the method itself, that the designer

can predict. The second kind identiﬁes the mutability

that can be expected and is described in the original

method design, but happens during the instantiation

of the method.

Scientiﬁc Role: This is another component that

greatly helps increase the possibility of generalization

of a method, such as the principles of form and func-

tion. Understanding what changes (and in what man-

ner) a method can undergo also helps to understand

whether it is possible to transfer it across research do-

mains.

Testable Propositions: Here it is possible to

make an analysis and have a complete view of what

are the statements made about the method or arte-

fact of the process in general, that can be measured

and tested. For simplicity, an overview of what these

propositions are can be obtained using the following

three questions: What are the statements about? What

is asserted by the statement, utility or truth? How

complete should the collection of testable proposi-

tions be?

Scientiﬁc Role: Testable propositions have the

purpose of supporting the validity of the method, es-

pecially considering its utility statement. As in (Of-

fermann et al., 2010a), “. . . a method is valid if it is

useful in respect to its purpose.”. Of course, an ob-

jective and testable analysis can only be made if the

hypotheses are made in such a way that they can be

reproduced by someone else.

Justiﬁcatory Knowledge: This component is

meant to clearly state all the possible theories sup-

porting the method, the product, and the context in

which the method is being applied. As in (Gregor,

2009), there are different kinds of theory that can be

selected, based on speciﬁc situations.

Scientiﬁc role: Justiﬁcatory knowledge is meant

to support the possibility of transferring a method into

different research domains and to prove its validity. It

provides elements that can be used to repeat a proof

of validity for the method in different contexts.

Principles of Implementation: This component

helps explore what the necessary steps are during the

implementation phase, which usually imply adapting

a method to the speciﬁc scope (situation or organiza-

tion) it is being used in. This is also known as tailor-

ing process.

Scientiﬁc role: These principles help again sup-

port the statements about the utility of a method and

deﬁne how such a method is transferable into different

scopes.

Expository Instantiation: This component is

meant to show a real-life application of the method

that is being designed. This can be represented either

as a description of the method applied in a speciﬁc

context or as information about the execution of the

method.

A Tool for Developing and Evaluating Methods (TDEM)

547

Scientiﬁc role: These principles, as the previous

one, also support transferability and generalization,

but provide more concrete information about the re-

sults coming from an actual tentative instantiation and

use of the method.

4 USAGE DUALITY OF TDEM

In this section, we explore the possibilities that

TDEM offers in Design Science. Speciﬁcally, two

distinct modes are taken into consideration, which we

named the analytical (evaluation) and the ideational

(development) processes. Both processes are ex-

plored and illustrated by giving a speciﬁc example for

each.

4.1 TDEM as an Analytical Tool

From what concerns the analytical point of view,

TDEM proved to be a tool that is effective in the

evaluation of artifacts of Design Science, speciﬁcally

methods. In an earlier work, we explored a poten-

tial usage of TDEM as a framework component for

process evaluation (Ogbuachi et al., 2021). In this

subsection, the evaluation process is shown through

the analysis of a method for the engineering of DI by

(Bub, 2018), which describes design activities, design

outcomes, and the necessary roles for organizations

that deliver innovation. The essential construct is the

process model, which shows the sequence of design

activities, essentially concentrating on method engi-

neering.

The analysis conducted on the approach using

TDEM, revealed several interesting matters, which

are illustrated here for each of the eight components

of Design Science Research.

Purpose and Scope: The research was carried out

within the scope of method engineering for digital in-

novation. The purpose of the paper is to develop a

method for the engineering of DI and integrating a

DSR process into the process model.

Constructs: Fundamental constructs can be eas-

ily identiﬁed from the method, such as an enhanced

meta-model for situational method engineering, a

stage-gate-oriented idea-to-launch process model, a

stage-gate-oriented process model integrated with a

DSR process, and a combined process model.

Testable Propositions: The utility of the pro-

posed method is shown in the paper, through a de-

scription of how the method is applied in a university-

industry co-innovation lab. The research process and

implementation are also described.

Principles of Form and Function: The process

model is clearly described in the paper. Design activ-

ities are in sequential order and they have to be per-

formed “from left to right”, completing each stage one

after the other.

Justiﬁcatory Knowledge: Method design theo-

ries are presented, such as theories about the applica-

tion context (including innovation management) and

theories about DSR.

Artifact Mutability: After application in the

original method setting, situational tailoring for

project types (and similar adaptations) is encouraged

for other settings, since the number of gates for inno-

vation processes can be adjusted to speciﬁc needs of

the respective enterprise.

Principles of Implementation: The method re-

quires an already implemented idea-to-launch pro-

cess, or willingness to implement it at the enterprise.

Expository Instantiation: The paper is illustrat-

ing a case study in which an instantiated method has

been enacted.

4.2 TDEM as an Ideation Tool

TDEM worked well also as a creational tool for De-

sign Science, thus representing a sort of duality if

compared to the analytical approach that was de-

scribed in the previous section. This new point of

view is illustrated here, by a study that was conducted

in a DI internal project, from the structuring of its fun-

damental parts to the veriﬁcation of its completion.

The project aimed at representing the subdivision of

a simple monolithic application into a structure that

tried to replicate the microservices architecture.

The project is based on the Stencil Parallel Pat-

tern. This pattern suits speciﬁcally areas like image

processing. In Stencil, every step of computation is

done starting from a memory address (or an array in-

dex) and operating on offsets from that address. The

result is then saved to an output array, speciﬁcally to

an index that corresponds to the starting memory lo-

cation for the input. The image processing program

consists of a set of algorithms that run consecutively

on a given image dataset, using this pattern.

The pipeline is constituted by execution units

(functions) that are executed in sequence. Given

the structure of the application, the plan consisted in

analysing what the functional units were, that could

be deﬁned independently from all the others, and

transforming those into actual applications that would

run as small services.

As mentioned earlier, in this speciﬁc case the

functional units are the functions that implement the

image processing algorithms. Those functions are

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

548

therefore encapsulated into ﬁne-grained services that

are able to retrieve an image as an input, process it

by applying one speciﬁc ﬁlter to the image, and give

back an output as a result, having only a few mi-

nor requirements on the input image. Each service

is able to operate independently from who sends the

input, thus fulﬁlling one of the core characteristics of

a microservices-based architecture.

It is important to remember, though, that in this

speciﬁc example the functionality of the full mono-

lithic application should also be preserved, after the

transition to a microservices-based architecture. De-

spite the structure being very different now from an

architectural point of view, the original functionality

could still be preserved by keeping the temporal point

of view unchanged. By this, we mean that the orig-

inal order of operations was sequential in the origi-

nal monolithic application, as it is typical of the gen-

eral imperative programming approach, but that same

order could be replicated in the new architecture as

well, by enforcing the sequentiality of the operations

through a structured series of service calls and data

transfers. The intended sequence then looks as shown

in Figure 2.

Image Upload

Edge

Detection

ASCII

Service

Endpoint

Grayscale

Resize

Brightness

Correction

Output

Service

Figure 2: Sequential order of operations.

Therefore, the initial image is now uploaded to an

endpoint of the service, which acts as the only com-

ponent the user is allowed to directly interact with.

This endpoint should then send the image that was

uploaded by the user to the ﬁrst step of the image ﬁl-

tering process, which is the Grayscale function. After

the execution is complete, the output gray image is

then sent to the next component, Resize, and so on,

until the last one, ASCII text-based rendering, com-

pletes its elaboration and returns the image as a ﬁle

to the user, as a response from the same aforemen-

tioned service endpoint. In our set-up, each func-

tion provides an input to the next one, completing

the pipeline. By restructuring the application in a

microservice architecture, we explored the possibility

of enforcing synchronous behavior (originating from

the monolithic application) in an environment (in this

case, the network) that is inherently asynchronous.

Having a guideline to deﬁne the development process,

helped us identify the potential issues (such as com-

munication issues, segregation of data, scalability of

workload, etc.) that could arise when building a ser-

vice that strongly relies on asynchronous interactions.

As before, an analysis based on the eight compo-

nents of TDEM is provided, with the different focus

presented in this section.

Purpose and Scope: The purpose of the project

can be explained easily as the creation of a method to

deﬁne microservices from an existing monolithic ap-

plication. Trying to identify the context, the best op-

tion is software engineering, speciﬁcally image pro-

cessing and networking. The deﬁnition of a lifecycle

coverage process was not entirely deﬁned, as it was

partly out of the scope of the process (only the design

phase is considered). While structuring the method,

we described the purpose in a way that can tell if the

method solves the proposed problem, and it does, es-

pecially the problem of splitting a pre-existing appli-

cation into smaller parts. The approach we adopted is

general enough to be adaptable to the speciﬁc prob-

lem.

Constructs: The construct that comes as an out-

put of this project is a modelling approach to the def-

inition of a microservice from a pre-existing applica-

tion. Thus, the main construct is a purely theoretical

high-level approach. The application context would

be Autonomous processing services and Microservice

candidates. We described the basic concepts needed

to present the method, the resulting structure, and the

application context.

Testable Propositions: As a utility statement, we

agreed that this approach should allow an easier def-

inition for the subdivision of components when cre-

ating a new microservice system by splitting existing

systems and, in this speciﬁc case, the method tries to

fulﬁl completely the problem it was used for. The

work that has been done should be enough informa-

tion to allow at least the unit testing of components,

in most cases, testable propositions about the method

A Tool for Developing and Evaluating Methods (TDEM)

549

utility are also given.

Principles of Form and Function: The method

described in this paper is meant to be a starting point

for the creation of microservices based on the seg-

mentation of an application. The method we pro-

duced was extensively described, and the amount of

information provided is enough to understand that it

is adaptable. For each role in the project, the neces-

sary activities and their order were clearly speciﬁed.

In fact, the method is described with step-by-step clar-

ity (keeping the descriptions very high-level).

Justiﬁcatory Knowledge: The proposed method

is based on Domain Driven Design and Microser-

vices. It is still general, but the instantiation is im-

age processing as a domain. The main theory is that

monolithic, computationally expensive applications

can be split into smaller parts, to which we added at-

tention to the fact that the knowledge base of the ap-

plications can be singular or plural, both before and

after the splitting process, and both approaches could

actually be beneﬁcial (indirectly), especially while

considering efﬁciency.

Artifact Mutability: Considering the foreseeable

changes, we estimated that, because of the generic

nature of this method, which is also based on very

simple patterns, such changes are minor and would

not extensively change the deﬁned structure. More

speciﬁcally, the foreseeable changes are mostly based

on small adaptations to the application scopes. The

only method tailoring mechanism that was deﬁned

was for the given example, trying to put it in a more

generic context.

Principles of Implementation: About the tailor-

ing or assembly advice we provided, most of it is

about the deﬁnition of “autonomous entities”, and

mostly about the use in the suggested scope, with-

out excluding the consideration of several other areas.

Particular attention has been put in making it clear for

the proposed scope.

Expository Instantiation: The method is instan-

tiated only within the provided example. There is no

report of the sort of real-life case study in the paper.

Mostly the use we made was for laboratory experi-

ments in a controlled environment. The provided ex-

ample deﬁned the operational scope and it is speciﬁc

enough to be illustrative about the applied method.

5 GENERALIZABILITY OF

TDEM

TDEM can also be used as a tool for evaluating pro-

cesses, which are not being necessarily developed us-

ing DSR paradigm. We show this through the analysis

of a design approach structured by O’Connor, Elger,

and Clarke in (O’Connor et al., 2016), which is meant

to help enhance development (especially in IT-related

ﬁelds) by taking into consideration the so-called situ-

ational context of a project.

As described in the paper, situational context can

be of utmost importance when designing a process.

It can be deﬁned as the set of elements that may

have a direct inﬂuence on a project and its outcome,

from the Enterprise level to the actual development

phases. In particular, (O’Connor et al., 2016) show

that the elements identiﬁed as components of this con-

text are the personnel, requirements, application (the

software project itself), technology, organization, op-

eration, management, and business. The ﬁrst four el-

ements are the ones having the highest impact on the

design stages. The personnel is identiﬁed as the non-

managerial characters participating ﬁrst-hand in the

development of the software application (the software

engineers), while the requirements are all the imple-

mentation needs and rules as deﬁned in the software

engineering standard ISO 12207 (Singh, 1996), which

can be characterized by the commissioner or the de-

velopment team. The technology that is available in

the company affects directly the capabilities of the

aforementioned personnel and the required develop-

ment time.

The analysis conducted on this approach using

TDEM, revealed several interesting matters, which

are illustrated here for each of the eight components.

Purpose and Scope: The type, context, life-

cycle, and activity coverage of the project could

be explained fairly easily. The project was car-

ried out within the scope of software engineering for

microservices, and the lifecycle management corre-

sponds to what is often seen in the paradigm known

as “agile development”. The purpose of the paper was

not to show a method, speciﬁcally, but rather showing

through a concrete example how situational context

affects project management.

Constructs: The “products” of the project can be

plainly pointed out. Speciﬁcally, the output-speciﬁc

construct here is a framework for situational analy-

sis which is fundamental to understand the necessary

adaptation steps required for general methods, when

applying them to any concrete project. One notable

remark is that the use of terms and concepts in the pa-

per has been made in such a way that allows the entire

framework to be easily understandable, to the point

that an entire section was dedicated to the deﬁnition

of concepts and their utility.

Testable Propositions: The utility of the pro-

posed theory is illustrated across the paper and can

be summarized with one proposition: the suggested

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

550

analysis should provide a better view of how to anal-

yse the effects of context on development. Although

there are no speciﬁcally deﬁned measures, the infor-

mation provided by the authors includes the results of

actual tests conducted in an enterprise.

Principles of Form and Function: This paper

provides a framework to analyze the environment in

which a microservices-based software system is be-

ing developed, to easily adapt generic methods to the

needs of speciﬁc cases. The suggested approach is de-

scribed very extensively and the amount of informa-

tion provided is enough to understand how the adapta-

tion process works. The framework is described with

precise detail and great clarity (as mentioned earlier,

a use case example is also shown).

Justiﬁcatory Knowledge: This component helps

identify some important properties regarding the

background knowledge needed to justify the theories

that are discussed in the paper. Starting from the de-

sign theories, which are based on the agile and lean

development principles and the microservices archi-

tecture, it is understandable that the majorly observed

scope is that of web applications and full-stack opera-

tions. Other aspects of interest are the precision of the

framework and the possibility of extending it. Theo-

ries supporting the testable propositions and regarding

the product of the research are also presented.

Artifact Mutability: Foreseeable changes are to

be taken into consideration. In fact, because of the

generic nature of this framework, which is also based

on very simple discernment strategies, it is correct

to say that the foreseeable changes are minor and

wouldn’t extensively change the deﬁned structure of

the framework itself. It can also be added that the

foreseeable changes are mostly situational, based on

adaptations to the application scopes, and they are

supposed to be like that. The framework resulting

from this work is described extensively to allow its

use as a method component and, since change is con-

sidered to be a fundamental factor through the whole

process, tailoring mechanisms have been deﬁned for

the given example and for more generic contexts.

Principles of Implementation: It would not be

wrong to afﬁrm that the paper is meant to empha-

size and prove the importance of tailoring as a prin-

ciple. Some advice regarding the introduction of the

approach into real-life settings is also included, to-

gether with an extensive example conducted in a real

company. Generally speaking, the advice is mostly

clear for the setting in which it is applied, but also

helps a general understanding for an adaptation into

different scopes.

Expository Instantiation: An expository instan-

tiation is not missing, as the approach has been instan-

tiated in the provided example, which is practically an

enactment within a real-life case study. The example

covers the main concepts illustrated by the proposed

approach and tackles the issues of a case that fully

belongs to the predeﬁned scope.

6 CONCLUSION AND FUTURE

WORK

There exists a huge amount of interest in DI re-

search from IS scholars. As of our previous research

(Huseynli et al., 2021), we found out that there is

still scarcity in terms of using Design Science Re-

search (DSR) for the engineering of DI, especially

from a method perspective, and a possible integration

of both ﬁelds. While getting motivation for develop-

ing a method for the engineering of DI, using and in-

tegrating DSR, we came across early research works

- (Gregor and Jones, 2007) and (Offermann et al.,

2010a) - and based on the concepts illustrated in both

works, we proposed a tool for developing and evaluat-

ing methods which we called TDEM. We showed the

utility of TDEM as an evaluation tool by analysing

a paper from the DI-DSR research area and also il-

lustrated its generalizability capabilities in non-DSR

works, on a paper based on continuous software engi-

neering from a microservices perspective. TDEM is

also useful as a development tool for methods, and we

described this through a DI internal project. We be-

lieve that TDEM is a useful contribution to the body

of knowledge for information systems and shall be

utilized while developing and evaluating methods.

Our work focuses on the very important artifact

type “method”. (March and Smith, 1995) identify

four artifact types from design science: constructs,

models, methods, and instantiations. Moreover, (Of-

fermann et al., 2010b) identiﬁed eight different arti-

fact types from an extensive literature review of de-

sign science publications: Systems Design, Method,

Language/Notation, Algorithm, Guideline, Require-

ments, Pattern, Metric. In future work, we would like

to develop similar tools also for artifact types other

than methods, possibly starting from system design.

REFERENCES

Baskerville, R. (2008). What design science is not. Euro-

pean Journal of Information Systems, 17:441–443.

Baumol, W. J. (2002). The free-market innovation machine

- analyzing the growth miracle of capitalism. Journal

of Economics, 82:93–97.

Bergstra, J., Jonkers, H., and Obbink, J. (1985). A software

A Tool for Developing and Evaluating Methods (TDEM)

551

development model for method engineering. Esprit,

84:84–94.

Berntsson-Svensson, R. and Taghavianfar, M. (2015). Se-

lecting creativity techniques for creative require-

ments: An evaluation of four techniques using cre-

ativity workshops. 2015 IEEE 23rd International Re-

quirements Engineering Conference (RE), pages 66–

75.

Brinkkemper, S. (1996). Method engineering: engineer-

ing of information systems development methods and

tools. Inf. Softw. Technol., 38:275–280.

Bub, U. (2018). Towards an integrated method for the

engineering of digital innovation and design science

research. In European Conference on Advances in

Databases and Information Systems, pages 327–338.

Springer.

Bub, U. (2020). Methodenvergleich und methodenentwick-

lung mit einem canvas. Wirtschaftsinformatik & Man-

agement, 12:146–157.

Bucher, T. and Winter, R. (2008). Dissemination and im-

portance of the ”method” artifact in the context of

design research for information systems. In Vaish-

navi, V. and Baskerville, R., editors, Proceedings of

the Third International Conference on Design Sci-

ence Research in Information Systems and Technology

(DESRIST 2008), pages 39–59, Atlanta, GA. Georgia

State University.

Chatterjee, S. (2015). Writing my next design science re-

search master-piece: But how do i make a theoretical

contribution to dsr? In ECIS 2015 Completed Re-

search Papers.

Gregor, S. (2009). Building theory in the sciences of the

artiﬁcial. In Proceedings of the 4th International Con-

ference on Design Science Research in Information

Systems and Technology, DESRIST ’09, New York,

NY, USA. Association for Computing Machinery.

Gregor, S. D. (2006). The nature of theory in information

systems. MIS Q., 30:611–642.

Gregor, S. D. and Jones, D. (2007). The anatomy of a design

theory. J. Assoc. Inf. Syst., 8:19.

Harmsen, F., Brinkkemper, S., and Oei, J. L. H. (1994). Sit-

uational method engineering for informational system

project approaches. In Methods and Associated Tools

for the Information Systems Life Cycle.

Hevner, A. R. and Gregor, S. D. (2020). Envisioning en-

trepreneurship and digital innovation through a design

science research lens: A matrix approach. Information

& Management, page 103350.

Hevner, A. R., March, S. T., Park, J., and Ram, S. (2004).

Design science in information systems research. MIS

Q., 28:75–105.

Hevner, A. R., vom Brocke, J., and Maedche, A. (2019).

Roles of digital innovation in design science research.

Business & Information Systems Engineering, 61:3–8.

Huseynli, M., Podder, I., Ogbuachi, M. C., and Bub, U.

(2021). Digital innovation and design science re-

search: The state-of-the-art review and integrated in-

novation strategies framework. In Proceedings of the

10th Jubilee Interdisciplinary Doctoral Conference

2021, pages 355 – 368.

Karlsson, F. and Wistrand, K. (2003). Method components

- extending the vision. In Proceedings of the 26th In-

formation Systems Research Seminar in Scandinavia

(IRIS 26), Haikko Manor, Finland.

March, S. T. and Smith, G. F. (1995). Design and natural

science research on information technology. Decis.

Support Syst., 15:251–266.

Nilsson, A. G. (1999). The business developer’s toolbox:

Chains and alliances between established methods. In

Perspectives on Business Modelling, pages 217–241.

Springer.

O’Connor, R. V., Elger, P., and Clarke, P. M. (2016). Ex-

ploring the impact of situational context — a case

study of a software development process for a mi-

croservices architecture. 2016 IEEE/ACM Interna-

tional Conference on Software and System Processes

(ICSSP), pages 6–10.

Offermann, P., Blom, S., Levina, O., and Bub, U. (2010a).

Vorschlag f

ur komponenten von methodendesignthe-

orien. WIRTSCHAFTSINFORMATIK, 52:287–297.

Offermann, P., Blom, S., Sch

onherr, M., and Bub, U.

(2010b). Artifact types in information systems design

science - a literature review. In DESRIST.

Ogbuachi, M. C., Podder, I., Bub, U., and Huseynli, M.

(2021). A framework for quantiﬁable process im-

provement through method fragments in situational

method engineering. In 2021 3rd International Con-

ference on Advanced Information Science and System

(AISS 2021), AISS 2021, New York, NY, USA. Asso-

ciation for Computing Machinery.

Peffers, K., Tuunanen, T., Rothenberger, M. A., and Chat-

terjee, S. (2008). A design science research method-

ology for information systems research. Journal of

Management Information Systems, 24:45 – 77.

Ralyt

e, J. and Rolland, C. (2001). An approach for method

reengineering. In International Conference on Con-

ceptual Modeling, pages 471–484. Springer.

Singh, R. (1996). International standard iso/iec 12207 soft-

ware life cycle processes. Software Process: Improve-

ment and Practice, 2:35–50.

Sunyaev, A., Hansen, M., and Krcmar, H. (2008). Method

engineering: A formal description. In ISD.

Vaishnavi, V. K. and Kuechler, W. L. (2004). Design Sci-

ence Research in Information Systems. Ais, pages 1–

45.

van Slooten, K. and Brinkkemper, S. (1993). A method en-

gineering approach to information systems develop-

ment. In Information System Development Process.

ICEIS 2022 - 24th International Conference on Enterprise Information Systems

552