Incremental Bidirectional Transformations: Evaluating Declarative and
Imperative Approaches using the AST2Dag Benchmark
Matthias Bank, Sebastian Kaske, Thomas Buchmann and Bernhard Westfechtel
Chair of Applied Computer Science I, University of Bayreuth, Universit
¨
atsstrasse 30, 95440 Bayreuth, Germany
Keywords:
Model-Driven Development, Model Transformations, Bidirectional Transformations, Incremental
Transformations, BX, Benchmark, Evaluation, QVT-R, BXtend.
Abstract:
Model transformation are the core of model-driven software engineering. Typically an initial model is refined
throughout the development process using model transformations to derive subsequent models until eventually
code is generated. In round-trip engineering processes, these model transformations are performed not only
in forward, but also in backward direction. To this end, bidirectional transformation languages provide a
single transformation definition for both directions. This paper evaluates the transformation languages QVT
Relations (QVT-R) which allows to specify incremental bidirectional transformations declaratively at a high
level of abstraction and BXtend - a framework for procedural specification of both forward and backward
transformation in a single rule set. Both languages have been used to implement the AST2Dag transformation
example. The benchmarx framework was used for a quantitative and qualitative evaluation of the obtained
results.
1 INTRODUCTION
Model-Driven Software Engineering (MDSE)
(Schmidt, 2006) aims at increasing the productivity
of software engineers by replacing low-level program
code with high-level models. Typically, modeling
languages are defined with the help of metamodels,
e.g., the Meta Object Facility standard MOF (Object
Management Group, 2016b) provided by the Object
Management Group (OMG). A subset of MOF
called Essential MOF (EMOF) is available, which
has been implemented in the Eclipse Modeling
Framework (EMF) (Steinberg et al., 2009). In the
context of model transformations, a vast variety of
languages and tools has been developed (Czarnecki
and Helsen, 2006). They differ with respect to the
underlying computational paradigm (procedural,
functional, rule-based, object-oriented), application
and scheduling strategies, directionality (uni- vs.
bidirectional), support for in-place or out-place, batch
or incremental transformations, etc.
This paper focuses on bidirectional transforma-
tions. It compares two languages that have been de-
signed for bidirectional transformations: QVT-R (Ob-
ject Management Group, 2016a), a declarative lan-
guage in which transformations are defined by re-
lations between source and target model elements,
and BXtend (Buchmann, 2018), an object-oriented
framework for bidirectional transformations in which
transformations are programmed in Xtend. In QVT-
R, it is possible to define a bidirectional transforma-
tion by a single set of relations, while both trans-
formation directions have to be specified separately
in BXtend. The comparison is performed with the
help of a benchmark which we implemented on top
of the Benchmarx framework for evaluating bidirec-
tional transformations (Anjorin et al., 2017b; Anjorin
et al., 2019). The evaluation is performed with re-
spect to the following properties: size of the transfor-
mation definitions, functionality (measured in terms
of passed test cases), and performance (run time).
For our comparison, we selected the AST2Dag
transformation case that was described first in (West-
fechtel, 2018) and was added later to the bx wiki
for bidirectional transformations
1
. An abstract syn-
tax tree represents an arithmetic expression as a tree,
while a directed acyclic graph representation of the
same expression includes each subexpression only
once. Even though both models are equivalent rep-
resentations of the same information, the AST2Dag
case proved to be challenging for bidirectional model
transformation languages and tools.
1
http://bx-community.wikidot.com
Bank, M., Kaske, S., Buchmann, T. and Westfechtel, B.
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark.
DOI: 10.5220/0009206602490260
In Proceedings of the 15th International Conference on Evaluation of Novel Approaches to Software Engineering (ENASE 2020), pages 249-260
ISBN: 978-989-758-421-3
Copyright
c
2020 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
249
2 BIDIRECTIONAL
TRANSFORMATIONS
2.1 Overview
A broad range of different languages and tools ad-
dressing model transformations has been developed
in the past decade. They differ with respect to com-
putational paradigm (rule-based, functional, object-
oriented, procedural), directionality (unidirectional
vs. bidirectional), support for in-place, out-place,
incremental or batch transformations, etc. Numer-
ous model transformation languages emerged accord-
ingly, ranging from general ones, like ATL (Jouault
et al., 2008), graph-based languages like Henshin
(Arendt et al., 2010) or eMoflon (Anjorin et al., 2011)
to domain-specific languages such as, e.g., the Ep-
silon family of transformation languages (Rose et al.,
2014) or EMG (Popoola et al., 2016).
The vast majority of transformation languages
supports only unidirectional batch transformations
with accompanying fairly well developed tool sup-
port. However, bidirectional transformations are re-
quired in a variety of use cases of industrial relevance.
For example, round-trip engineering integrates for-
ward and reverse engineering into a coherent process
which needs to be supported by bidirectional transfor-
mation tools: After transforming a source model into
a target model, additions or modifications of the tar-
get model may be necessary. Consequently, changes
to the source model need to be propagated in a way
which allows to retain the modifications of the target
model. These change propagations call for incremen-
tal rather than batch transformations. Additionally,
changes to the target model may have to be propa-
gated back to the source model, resulting in bidirec-
tional transformations. Furthermore, integration of
heterogeneous tools calls for data converters which
ideally perform lossless transformations in all direc-
tions.
In the light of this diversity, the OMG issued
the QVT standard (Queries, Views, and Transforma-
tions) which defines a family of model transforma-
tion languages located at different levels of abstrac-
tion (Object Management Group, 2016a). The most
high-level language defined in the QVT standard is
QVT Relations (QVT-R). QVT-R provides a language
for the declarative specification of transformations
between MOF-based models. Furthermore, QVT-R
reuses the Object Constraint Language (OCL), an
expression language for MOF models (Object Man-
agement Group, 2014). QVT-R addresses a wide
spectrum of model transformation scenarios, includ-
ing enforcing and checking, unidirectional and bidi-
rectional, batch and incremental, and n:1 transforma-
tions. However, bidirectionality is not a unique fea-
ture of QVT-R (Czarnecki et al., 2009). The most
prominent competitors are languages and tools based
on triple graph grammars, which were introduced
much earlier (Sch
¨
urr, 1994) and have been developed
further in numerous research contributions (K
¨
onigs
and Sch
¨
urr, 2006; Becker et al., 2007). In contrast
to the grammar-based approach, QVT-R follows a re-
lational paradigm where a transformation is specified
by a set of relations among patterns to be instantiated
in the participating models.
2.2 Languages and Tools
In the following, we describe the languages and tools
used for our AST2Dag benchmark.
QVT-R. While QVT-R provides many interesting
and promising features, the language has not been
adopted widely. This stands in sharp contrast to other
OMG standards such as MOF and OCL. To the best of
our knowledge, currently no tool is available that fully
conforms to the standard defined by the OMG. For
the benchmark, we used medini QVT
2
, which runs
only under old Eclipse versions, but still can be down-
loaded from the cited web site. The tool QVT Declar-
ative
3
is under development, but supports the QVT-R
standard only partially so far.
Medini QVT conforms to the QVT-R standard
syntactically, but implements a deviating semantics.
In medini QVT, a persistent trace is stored which
records relations between source and target elements.
The trace is used to support incremental transforma-
tions. When source elements have been inserted, they
are transformed because they do not appear in the
trace so far. When source elements have been deleted,
deletion is propagated to target elements via the trace
elements. Finally, certain changes to source elements
(modifications of attribute values) may be propagated
to target elements such that consistency is restored.
BXtend. The BXtend (Buchmann, 2018) frame-
work was created as a result to a case study in round-
trip engineering between UML class models and Java
source code. Approaches using TGGs (Buchmann
and Westfechtel, 2016) and QVT-R (Greiner and
Buchmann, 2016) clearly revealed the drawbacks of
highly declarative approaches, when right-hand sides
of rules contain variability in creation patterns (e.g.,
2
http://projects.ikv.de/qvt
3
https://projects.eclipse.org/projects/modeling.mmt.qvtd
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
250
it is not possible to specify rules between abstract su-
perclasses and create the corresponding subtypes dur-
ing rule execution). This leads to massive copying
and pasting similar rules. As a consequence BXtend
uses an object-oriented approach, where the frame-
work infrastructure comprising abstract rules, generic
creation of source and target elements and handling
the correspondence model is generated automatically.
The transformation developer only needs to specify
transformation patterns for forward and backward di-
rections respectively using the Xtend programming
language
4
, a fully Java compatible, less verbose lan-
guage which comes with high-level constructs such as
powerful lambda expressions.
3 THE TRANSFORMATION
CASE: EXPRESSION TREES
AND DAGS
3.1 Description
We study a problem from compiler construction:
Given an abstract syntax tree of an expression, a di-
rected acyclic graph (dag) is constructed which con-
tains common subexpressions only once (Figure 1).
From the dag created by this folding transformation,
the original tree may be reconstructed by unfolding
the dag. In contrast to a case study performed ear-
lier (Westfechtel and Buchmann, ), where both source
and target model contained different information, the
same information is represented in different ways
here. A pair of a tree and a DAG is consistent, if
traversing both the tree and the DAG in the same or-
der (e.g., inorder) results in the same expression rep-
resented by the data structure.
Figure 2 displays the metamodels for trees and
dags. In the tree metamodel, left and right are con-
tainment references from operators to left and right
operands, respectively. In contrast, these references
do not own the containment property in the dag meta-
model because an expression may occur at multiple
locations. Thus, in the dag representation of an ex-
pression all nodes are immediate components of the
root object of the dag model, and the hierarchy of
expressions is represented by non-containment refer-
ences.
This transformation case is highly problematic for
rule-based and grammar based approaches and poten-
tially highlights current limitations. While the tree is
best constructed in a top-down way by unfolding the
4
https://www.eclipse.org/xtend/
a 10 b 10 a 10 b 10
/ /
+ +
*
Expressiontree
a b 10
/
+
*
Expressiondag
Figure 1: Tree and dag for (a/10 + (b-10))*(a/10 + (b-10)).
Model
Expression
Operator
op :ArithmeticOperator
<<enumeration>>
ArithmeticOperator
Add
Subtract
Multiply
Divide
1
model
0..1
expr
Operand
Variable
name :String
Number
value :Integer
left
leftInverse
right
right
Inverse
1
1
0..1
0..1
Expressiontrees
Model
Expression
Operator
op :ArithmeticOperator
<<enumeration>>
ArithmeticOperator
Add
Subtract
Multiply
Divide
1..*
model
1
expr
Operand
Variable
name :String
Number
value :Integer
left
leftInverse
right
right
Inverse
1
1
0..*
0..*
Expressiondags
Figure 2: Metamodels for Expression Trees and Dags.
DAG, the DAG is best constructed bottom-up by fold-
ing the tree.
Let us briefly summarize the formal properties
of forward and backward transformations: Trees and
dags are equivalent representations of expressions.
Both the forward and the backward transformation are
total, deterministic, injective, and surjective.
3.2 Benchmarx
In order to evaluate the solutions provided in this pa-
per, we use the benchmarx framework (Anjorin et al.,
2017b; Anjorin et al., 2019). It is designed as a
generic framework for benchmarking bx tools and
uses JUnit test cases designed as synchronization di-
alogs. Such a synchronization dialog always starts
from the same agreed upon initial consistent state, and
then applies a sequence of edits to the source and tar-
get models. After the edits, the transformation is ex-
ecuted and only the resulting models are directly as-
serted by comparing them with predefined expected
versions. In this paper, we provide a benchmark for a
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark
251
specific bx problem (Ast2Dag) by providing a suite of
test cases which are used to evaluate the solutions to
the transformation problem specified in QVT-R and
BXtend. The test suite contains test cases for both
forward and backward direction as well as batch and
incremental tests for each direction. Batch test cases
always start with an empty model and an edit delta
which creates the starting state of the source model
of the transformation (AST model for forward direc-
tion, DAG model for the backward direction). After
the model state is established, the transformation is
started and the obtained result is compared with a pre-
defined expected state. Incremental test cases on the
other hand start with an initial consistent state of both
models. Afterwards a series of edit deltas is applied
to one of those models and the transformation is re-
run. The test result is obtained by comparing the ac-
tual state of the models with the expected one. Please
note that the incremental test cases also check if the
propagated change is minimal.
3.3 Test Cases
Batch Forward. The batch forward test cases com-
prise the following scenarios:
Initialization: An empty model with the root ele-
ment Model is created.
Single Expression: This test case creates an expres-
sion with a single number in an empty AST. The
expected result is an expression DAG with a cor-
responding single number.
Complex Expression: Extension of the previous test
case: a complex expression tree is created, which
is transformed to a corresponding DAG.
Multiple Expressions: An expression tree with mul-
tiple expressions is created. Expressions appear
more than one time. The corresponding DAG
should contain multiple references to correspond-
ing expression elements.
Multiple Complex Expressions: Analogously to
the previous test case multiple expressions over
several hierarchy layers which now have iden-
tical subtrees are created. The corresponding
DAG should contain multiple references to
corresponding elements.
Batch Backward. Analogously to the forward test
cases, the batch backward test cases address the fol-
lowing scenarios:
Single Expression: An expression with a single
number in an empty DAG is created. The ex-
pected result is a single number expression in the
corresponding AST.
Multiple Expressions: Multiple expressions are cre-
ated in the DAG, its references are single valued.
The expected result are multiple expressions in
the corresponding AST.
Multiple Complex Expressions: Multiple expres-
sions in the DAG which have multi-valued
references. The corresponding unfolding oper-
ation should create multiple occurrences of the
expressions in the AST.
Multiple Expressions with Identical Subtrees:
Similar to the previous test case, but containing
expressions that have identical subtrees.
Incremental Forward. The incremental test cases
start with a consistent pair of AST and DAG mod-
els. Afterwards, the AST is modified and the changes
need to be propagated to the DAG.
Incremental Inserts: Leaves are replaced with a
complex expression in an existing AST (which
has been transformed to a corresponding DAG in
a previous step). The expected result is a corre-
sponding DAG.
Incremental Deletions: Nodes from an existing
AST are removed and replaced with a number.
Please note, that inner nodes are removed, as re-
moving only leaves would result in an invalid
AST. Children of deleted nodes are removed re-
cursively. The expected result is an accordingly
updated DAG.
Incremental Modifications: An existing AST is
modified. A modification is either changing an
attribute of a node or switching between variable
and number. The expected result is a correspond-
ing DAG.
Incremental Modifications Resulting in Deletions:
An existing AST is modified. The modification
leads to merging of elements in the DAG.
Stability: This test re-runs the transformation after
an idle source delta has been applied. The ex-
pected result is an unchanged target DAG.
Incremental Backward. Incremental backward
test cases also start with a consistent pair of DAG
and AST models. After a change to the DAG,
corresponding updates are propagated to the AST.
Incremental Inserts: New nodes are inserted into
the DAG. Changes are propagated to the AST.
Incremental Deletions: Nodes are deleted from the
DAG. The resulting AST must be valid, i.e. Chil-
dren of affected AST nodes need to be deleted re-
cursively.
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
252
Incremental Modificactions: An existing DAG is
modified. A modification comprises changes of
node values and switching the type of a node
(variable and number). The corresponding AST
needs to be updated accordingly.
Stability: The stability of the transformation is
tested. An idle delta to the DAG is propagated
to the AST, resulting in an unchanged AST.
4 SOLUTIONS
4.1 QVT-R
We failed in the construction of a single bidirectional
transformation. Instead, we developed separate uni-
directional transformations operating bottom-up and
top-down, respectively. The solution was described
in (Westfechtel, 2018) and is summarized briefly be-
low to make this paper self-contained. Please note the
separate specification of both transformation direc-
tions renders round-trip engineering impossible be-
cause forward and backward transformation employ
different trace models.
The forward transformation employs backward
chaining of relations through relation calls in when
clauses (Listing 1). The relations Variable2Variable
and Number2Number collapse multiple occurrences
of the same atomic operand. Collapsing of multiple
occurrences works recursively since it also applies to
operators. The key declarations in lines 2–4 ensure
that only one element per key value is created in the
dag model.
In contrast, the backward transformation makes
use of forward chaining through relation calls in
where clauses (Listing 2). The specification contains
three top-level relations which handle three cases of
expression root nodes (variable, number, and opera-
tor). Starting from the root, the expression tree is built
top-down. For each occurrence of a subexpression,
exactly one relation call is executed. For each type of
nested operand, there are two relations transforming
it as a left or right operand of the enclosing operator,
respectively.
4.2 BXtend
The BXtend solution to our example scenario con-
sists of four rule classes, one for each concrete class
in the respective source and target models (Model,
Number, Operator, and Variable). Within those rule
classes bodies for the automatically generated method
stubs for sourceToTarget() and targetToSource() are
supplied by the user, specifying transformation pat-
terns for each execution direction separately. Since
the transformation problem describes folding/unfold-
ing the respective data structures, a n:1 correspon-
dence model is required. Since the correspondence
model is shared between both transformation direc-
tions, round-trip engineering may be supported (in
contrast to the solution in medini QVT).
Since the transformation developer has full con-
trol over rule orchestration, we specified a bottom-
up traversal of the AST for the forward direction
and a top-down creation of the AST for the back-
ward direction. Listing 3 depicts a cutout of the
rule class responsible for handling numbers. The
method sourceToTarget() (lines 3-25) is called from
the framework whenever a forward transformation is
executed. We iterate over all instances of Number
in the model and check if there is already a corre-
spondence model element available (line 6). If no
correspondence model element is found the method
addToTargetElem is called. Since a folding of multi-
ple AST elements with same values to a single DAG
element is performed in this transformation direction,
the method findTargetElem is called to retrieve match-
ing DAG elements. If no matching element is found, a
new target model element dag.Number is created (line
30). Otherwise the current source model element is
added to the list of matching elements (line 33). The
incremental behavior of the transformation is realized
in lines 11-18. A check is performed, if all source cor-
respondences which contain the current source model
element contain the same value attribute (line 11-12).
If the current source value was updated, this check
fails and the transformation proceeds in line 19, where
the source element is removed from the correspon-
dence list and addToTargetElem is called to establish
a correct matching.
Listing 4 depicts the method targetToSource from
the rule class responsible for transforming Operators.
Since the corresponding AST is constructed in a top-
down way, an ordering of the elements is required
(lines 4-9). An Xtend switch statement is used to map
the operator types of both models. Since variables and
numbers may only occur at the leaves of the AST, the
hierarchy of operators is established by traversing the
left and right references of the current element (lines
33-47) and creating or updating corresponding AST
elements.
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark
253
Listing 1: Transformation of trees to dags.
1 transformation as t2 da g ( as tM o d e l : ast , d ag Mo de l : dag ) {
2 key da g :: V ar ia bl e { m odel , na me };
3 key da g :: N u mb er { mo del , v al ue };
4 key da g :: O pe ra to r { m odel , op , left , r ig ht };
5 top relation M od e l 2 Mo de l { .. .}
6 relation E x pr e ss i on 2 Ex p r e s s i on {
7 checkonly domain a st Mo de l a s t_ ex p re s si on : a st :: Ex pr e ss io n {} ;
8 enforce domain d ag Mo de l d a g_ ex p re s s i on : d ag :: Ex pr e ss io n {} ; }
9 top relation V a r i ab l e2 V ar i a b le {
10 na me : S t ri ng ; a s t : a st : : M od el ; dag : da g :: M o de l ;
11 checkonly domain a st Mo de l a s t_ va r ia bl e : as t :: V ar ia bl e { na me = na me };
12 enforce domain d ag Mo de l d a g_ va r ia bl e : da g :: V ar ia bl e {
13 mo de l = dag ,
14 na m e = na me };
15 when { ast = r oot ( a st _ va ri a bl e ) . m od el ;
16 M od el 2 M o de l ( ast , dag ); }
17 where {
18 if d a g _ va r i a bl e . o cl Is U nd e f i ne d () then false
19 else E x pr e ss i on 2 Ex p r e s s i on ( a st _v ar ia bl e , d ag _v a ri ab l e )
20 endif; }
21 }
22 top relation N um b er 2N u mb er {. .. }
23 top relation O p e r at o r2 O pe r a t or {
24 as t_ op : a s t :: A r it h me ti c Op e ra t or ; a s t : a st : : M od el ; dag : da g :: M o de l ;
25 checkonly domain a st Mo de l a s t_ op e ra to r : as t :: O pe ra to r {
26 op = as t_o p , l eft = a s t_ le ft : a st : : E x p r es si o n {} ,
27 ri gh t = as t_ r ig ht : a st : : E x p r es si o n {}
28 };
29 enforce domain d ag Mo de l d a g_ op e ra to r : da g :: O pe ra to r {
30 mo de l = dag , o p = c o r r e s p o n d i n g D a g O p e r a t o r ( as t_ op ) ,
31 le f t = d ag _l ef t : dag :: E xp re ss i on {} ,
32 ri gh t = da g_ r ig ht : d ag : : E x p r es si o n { }
33 };
34 when { ast = ro ot ( a st _o p er at o r ) . m od e l ;
35 M od el 2 M o de l ( ast , dag );
36 E xp r es s io n 2E x pr e s s io n ( as t_ lef t , d ag _l ef t ) ;
37 E xp r es s io n 2E x pr e s s io n ( a s t_ ri ght , d a g_ ri gh t ) ; }
38 where { if d a g_ op e ra to r . o cl I sU nd e fi n e d () then false
39 else E x pr e ss i on 2 Ex p r e s s i on ( a st _o pe ra to r , d ag _o p er at o r )
40 endif; }
41 }
42 ...
5 EVALUATION
5.1 Size of the Transformation
Specification
Both solutions presented in this paper support a tex-
tual concrete syntax with which transformation defi-
nitions can be specified. Thus, a quantitative impres-
sion of the size of the transformation definitions can
be obtained by counting the number of lines of code
(excluding empty lines and comments), the number
of words (character strings separated by whitespace)
in these lines, and the number of characters in these
words. The values obtained for these metrics are de-
picted in Table 1. In general, the solution in QVT-R is
expected to be shorter for three reasons: (1) It is pos-
sible to define both transformation directions simulta-
neously by a single relational specification. (2) The
specification is declarative, at a high level of abstrac-
tion. (3) Incremental consistency restoration need
(and can) not be specified at all. In the AST2Dag case,
the first reason does not hold because even in QVT-R
both transformation directions have to be defined sep-
arately. Even then, we expect savings for the other
reasons. In contrast, the size metrics for the solutions
in QVT-R and BXtend are in the same order of mag-
nitude. Thus, BXtend’s procedural approach does not
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
254
Listing 2: Transformation of dags to trees.
1 transformation da g2 as t ( da gM o d e l : dag , a st Mo de l : ast ) {
2 top relation R o o t V a r ia b le 2 Va r ia b le {. .. }
3 relation L e ft V ar i ab l e2 V a r i a b le {
4 nam e : St ri ng ;
5 checkonly domain d ag Mo de l d a g_ op e ra to r : da g :: O pe ra to r {
6 le f t = d ag _ v a ri ab l e : dag :: Va r i a bl e {
7 nam e = na me }} ;
8 enforce domain a st Mo de l a s t_ op e ra to r : as t :: O pe ra to r {
9 le f t = a st _ v a ri ab l e : ast :: Va r i a bl e {
10 nam e = na me }} ;
11 }
12 relation R i gh t Va r ia b le 2 Va r ia b l e {. .. }
13 top relation R o o t Nu m be r 2N u m b er {. .. }
14 relation L e ft N u m be r 2N u mb e r { . .. }
15 relation R i gh t N u mb e r2 N um b er {. .. }
16 top relation R o o t O p e ra t or 2 Op e ra t or {
17 da g_ op : d a g :: A r it h me ti c Op e ra t or ;
18 as t_ op : a st : : A r it h me t ic O p e ra t or ;
19 checkonly domain d ag Mo de l da g : d a g :: M ode l {
20 ex pr s = da g _o pe r at or : d ag :: Op er at or {
21 op = d ag _o p } };
22 enforce domain a st Mo de l as t : a s t :: M ode l {
23 ex p r = a st _ o p er at o r : ast :: Op e r a to r {
24 op = a st _o p } };
25 when { d a g _o p e r at o r . l ef tI nv er se -> is Em pt y () ;
26 d ag _o p er at o r . r ig ht In ve rs e -> is Em pt y ( ) ; }
27 where { as t_ op = c or r e s p o n d i n g A s t O p e r a t o r ( d ag _o p ) ;
28 L ef t Va r ia b le 2 Va r i a bl e ( d ag _o pe r a t or , a s t _o p e r at o r ) ;
29 L ef t Nu mb e r2 N um b er ( d ag _o pe ra t o r , a st _o p er at o r ) ;
30 L ef t Op e ra t or 2 Op e r a to r ( d ag _o pe r a t or , a s t _o p e r at o r ) ;
31 R ig h tV a ri a bl e 2V a ri a bl e ( da g_ op er at or , a s t_ o p e ra to r );
32 R ig h tN u m b er 2 Nu m be r ( da g_ op er at or , as t _ o pe r a t or ) ;
33 R ig h tO p er a to r 2O p er a to r ( da g_ op er at or , a s t_ o p e ra to r ); }
34 relation L e ft O pe r at o r2 O p e r a t or {. .. }
35 relation R i gh t Op e ra t or 2 Op e ra t o r {. .. }
36 }
increase the size of the transformation definition.
Table 1: Size of the Transformation Definitions of Both So-
lutions.
medini QVT BXtend
Lines of code 353 282
Number of words 702 832
Number of characters 7576 9974
5.2 Qualitative Analysis
In order to perform a qualitative analysis, test cases
for the different transformation directions have been
specified and executed for both batch and incremen-
tal mode of operation. We assume a test case to be
passed, if the resulting model matches a predefined
expected model state. The BXtend solution is able
to pass all tests specified in Section 3.3. The QVT-
R solution passes batch forward and backward tests
and also incremental backward tests, but it fails in two
incremental forward tests: incremental modifications
and incremental modifications resulting in deletions.
Expressiontree
*
+
a 5
+
a 7
Expressiondag
*
+
a 5
+
7
(1)
Expressiontree
*
+
a 5
+
a 5
(2)
Expressiondag
*
+
a 5
+
5
Expressiondag
*
+
a 5
(3)
(3)
Actualresult Expectedresult
Figure 3: Test Case failing in Medini QVT.
The failures are due to the fact that the incremental
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark
255
Listing 3: Transformation of trees to dags.
1 class N u m be r 2N u mb e r extends E le m 2E l em {
2
3 override def s o u rc e To T a r ge t () {
4 so u rc eM o de l . al l Co n te n ts .filter(typeof( N u mbe r ) )
5 .forEach[ n |
6 val co r r = n. g et C or rM o de l E l em as Mu l ti E le m
7 if( c orr == null) {
8 n . a dd To T ar g e t El e m
9 } else {
10 val t = corr . t a rg et E le m en t as dag . Nu m ber
11 if( c orr . s o ur ce E le m en t s .forall[ i t instanceof N umb e r &&
12 ( it as Numb e r ) . v a lue == n . v a lue ] ) {
13 val n e wT a rg e t = n . fi n d T ar g et E le m
14 if( ne w Tar g et ! = null)
15 ( n e wTa r ge t . g e tC o rr Mo d el E l e m as M ul t iEl e m ) . so u rc eE l em e nt s += n
16 else
17 t . v a lue = n . va l ue
18 }
19 if(t . v alu e ! = n . v alu e ) {
20 c o rr . s o ur c eE l e m en t s -= n
21 n . a dd To T ar g e t El e m
22 }
23 }
24 ]
25 }
26
27 def private a d d To T ar ge t El e m ( ast . Nu m ber e ) {
28 var n e wT a rg e t = e . fi n d T ar g et E le m
29 if( ne w Tar g et = = null) {
30 n ew T ar g et = c r e a te T a r ge t E l em e n t ( d a g . D ag P ac k ag e . eI N ST A NC E . n um b e r ) as d a g . N um b e r
31 }
32 val n e wCo r r = ne w Ta r ge t . ge t O r Cr ea t e C or rM o d e lE l e m en t ( ru l eID ) as M u lt i El e m
33 ne w Cor r . so u r c eE l em e nt s += e
34 ne w Ta r ge t . v alu e = e. v a l ue
35 ne w Ta r ge t . m ode l = e. m o d el . g e tC or r Mo d e l El e m . t a r g et E le m en t as dag . Mo d el
36 el em e nt s To Co r r . put ( ne w Co r r )
37 }
38
39 def private f i n dT a rg e t E le m ( a s t . N um b e r e) {
40 (e. m o del . g et C or r Mo de l El em . t ar g et E le m e n t as dag . M o d el ) . e xpr s
41 .filter(typeof( dag . Num b er ) ) .findFirst[ it . va l u e == e . va l ue ]
42 }
43
44 ...
45 }
behavior built into medini QVT deviates from the re-
quired behavior. One of the failing test cases is illus-
trated in Figure 3. In an initial batch transformation
(1), an expression tree for (a+5)*(a+7) is transformed
successfully into a dag, in which variable a is shared.
In step (2), the number 7 is replaced with the number
5. Since both operands of the multiply operator are
the same, the incremental transformation (3) should
result in the dag displayed at the right-hand side. In
contrast, medini QVT merely propagates the modi-
fication of the attribute value, without checking for
common subexpressions. The second test case fails
for the same reason, but even results in a dag which
represents a different expression than the tree. These
failures are symptoms of a general problem with the
design of QVT-R (note that this is not a problem spe-
cific to medini QVT): In QVT-R, relations between
source and target elements are specified declaratively;
incremental behavior is provided “for free”. However,
the provided behavior may deviate from the required
behavior. In contrast, in BXtend incremental behav-
ior has to be specified explicitly, but can tailored to
specific requirements. For example, when a number
is changed, it is checked whether the same number is
already present in the dag. In that case, the dag is re-
organized such that the number is present only once;
this reorganization is applied recursively updwards in
the dag.
5.3 Performance Analysis
In order to evaluate the efficiency and scalability of
the resulting transformation with respect to increasing
model size, two experiments were conducted in both
forward and backward directions resulting in four sets
of measurements: (1) batch transformations in for-
ward and backward directions, and (2) incremental
transformations in forward and backward directions.
The batch transformations test how the solutions scale
when creating the corresponding opposite model of
increasing size, starting with an AST with only one
single expression up to a tree with a depth of 20 hier-
archy layers. The incremental transformations apply
the exact same edit to master models of increasing
size. After correspondence has been established, one
single value of a leaf is changed. The time required
to locate and propagate this change to the dependent
model is measured. The tests were performed on the
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
256
Listing 4: Transformation of dags to trees.
1 class O p e ra t o r 2O p er at o r extends E le m 2E l em {
2 override def t a r ge t To S o u rc e () {
3 var List < d a g . O p erat o r > p re O rd e r = new A r ray L i st < da g . O p e rato r >() ;
4 for (var it = ta r ge t Mo d el . a ll C on t en t s .filter(typeof( dag . Op e ra t or ) ) ; it . h asN e xt ( ) ; ) {
5 val dag . O p era t or o p = it . n e x t () ;
6 if ( op . l e ft I nv e rs e . e mpt y & & o p . r i g h tI n ve r se . e m pty ) {
7 p re O rd e r += op ;
8 }
9 }
10 if ( p re O rd e r . size == 0) {
11 return;
12 } else if ( p r eO r der . s i ze > 1) {
13 throw new As s er t i o nE r ro r (" Da g h a s m u lti p le r oot e le m en t s .") ;
14 }
15
16 var List < O p erat o r > pre O rd e r S rc = new Arra y L ist < O p erat o r >() ;
17 val c o rr R oot = p r eO r de r . g e t (0) . g e t O rC r e a te Co r r M od el E l e me n t ( r u leI D ) as M ul t iE l em ;
18 val s r cRo o t = co r rRo o t . g e t O rC r e a te S o u rc eE l em ( s o u rc e Pa c ka ge . op e r ator , [ tru e ] ) as Ope r at o r ;
19 pr e Or de r Sr c . a d d ( s rc R o ot ) ;
20 sr c Roo t . m ode l = p r eO r de r . g e t (0) . mo d e l . c o rr M od e lE l e m . so u rc e El em e nt as Mode l ;
21
22 whi l e (! p r eOr d er . e m pty ) {
23 val dag . O p er a t or cu r re n t = pre O rd e r . re m o ve ( 0) ;
24 val O p er a tor c u rr e nt S rc = p r eO r de r Sr c . re m o ve ( 0 ) ;
25 switch ( c ur r ent . op ) {
26 case ADD : c u rr e nt S rc . op = a s t . A ri th m et ic O pe ra t or . A D D
27 case D I VID E : c ur r en t Sr c . op = as t . A r i th m e t ic O p e ra t o r . D IVI D E
28 case M U LT I PLY : c ur r en t Sr c . op = ast . Ar it h me ti c Op er a to r . MU L TI P LY
29 case S U BT R ACT : c ur r en t Sr c . op = ast . Ar it h me ti c Op er a to r . SU B TR A CT
30 }
31 if ( c ur r ent . r i ght instanceof d a g . O pe r at o r ) {
32 p re O rd e r . a dd (0 , c ur r ent . r i ght as d a g . O pe r at o r );
33 val c o rr R ig h t = pr e O rd e r . get (0 ) . g e t Or C r e at eC o r r Mo d e l E l em e n t ( rul e ID ) as Mu l ti E le m ;
34 val s r cR i ght = c o rr R ig h t . g e t O rC r e a te S o u rc e E l em (
35 s ou rc e Pa c ka g e . op e r ator , [ e | ( e as O pe r at o r ). r ig h tI n ve r s e = = c u rr e nt S rc ] ) as O pe r at o r ;
36 p re O rd er S rc . add (0 , sr c Ri g ht ) ;
37 c ur r en t Sr c . rig h t = s rc R igh t ;
38 }
39 if ( c ur r ent . l e f t instanceof dag . O p era t or ) {
40 p re O rd e r . a dd (0 , c ur r ent . l e f t as dag . O p era t or ) ;
41 val c o rr L eft = p r eO r de r . g e t (0) . g e t O rC r e a te Co r r M od el E l e me n t ( r u leI D ) as M ul t iE l em ;
42 val s r cLe f t = co r rLe f t . g e t O rC r e a te S o u rc eE l em (
43 s ou rc e Pa c ka g e . op e r ator , [ e | ( e as O pe r at o r ). l ef t In v er s e == c ur r en t Sr c ]) as Op e rat o r ;
44 p re O rd er S rc . add (0 , sr c Le f t );
45 c ur r en t Sr c . left = sr c Le f t ;
46 }
47 }
48 }
49 ...
50 }
same machine and in isolation for each solution. A
standard machine with an Intel Core i7-4770 CPU
was used, running at 3.40 GHz, with 16 GB of DDR3
RAM and with Microsoft Windows 10 64-bit as oper-
ating system. We used Java 1.8.0 191, Eclipse Neon
(4.6.3), and EMF version 2.12.0 to compile and exe-
cute the Java code for the scalability test suite. Each
test was repeated 5 times and the median measured
time was computed. The four measurement results are
depicted in Fig. 4, 5, 6, and 7. A figure is composed
of two plots a plot with linear/linear scale to the
left, and a plot with log/log scale to the right. While
the linear plot provides a realistic impression for the
actual complexity of each solution, the logarithmic
one zooms into finer details for smaller models and
zooms out for larger models, allowing to qualitatively
present large differences in runtime. The BXtend so-
lution outperforms the Medini QVT solution in ev-
ery benchmark. The gap between both solutions be-
comes clearly evident in the Figures showing linear/-
linear scale. Medini QVT shows sharp increases in
runtime kicking in already at relatively small models
of around 10k elements, while the BXtend solution is
able to handle signifcantly larger models.
6 RELATED WORK
While a broad range of languages and tools exist for
model transformations in general and bidirectional
transformations in particular, thorough evaluations of
different approaches applied to various transforma-
tion problems are still missing in the literature. The
transformation tool contest (TTC)
5
, aims at bringing
together tool developers in a competition-like annual
event. Several predefined transformation problems
are given which have to be solved by the participants
and their respective tools. However, to the best of our
knowledge so far only one case for bidirectional trans-
formations has been posed and solved, so far (see be-
5
https://www.transformation-tool-contest.eu/
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark
257
# model elements
time in s
0
1000
2000
3000
4000
5000
50000 100000 150000 200000 250000 300000 350000
BXtend Medini QVT
Batch Forward (lin - lin scale)
# model elements
time in s
0.1
1
10
100
1000
10 100 1000 10000 100000
BXtend Medini QVT
Batch Forward (log - log scale)
Figure 4: Forward Batch Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).
# model elements
time in s
0
5000
10000
15000
25000 50000 75000 100000 125000 150000 175000
BXtend Medini QVT
Incremental Forward (lin - lin scale)
# model elements
time in s
0.01
1
100
10000
10 100 1000 10000 100000
BXtend Medini QVT
Incremental Forward (log - log scale)
Figure 5: Forward Incremental Transformation: Linear/linear Scale (Left) and Log/log (Right).
# model elements
time in s
0
10000
20000
30000
40000
50000 100000 150000 200000 250000 300000 350000
BXtend Medini QVT
Batch Backward (lin - lin scale)
# model elements
time in s
0.01
1
100
10000
10 100 1000 10000 100000
BXtend Medini QVT
Batch Backward (log - log scale)
Figure 6: Backward Batch Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).
# model elements
0
2500
5000
7500
10000
12500
25000 50000 75000 100000 125000 150000 175000
BXtend Medini QVT
Incremental Backward (lin - lin scale)
# model elements
time in s
0.01
1
100
10000
10 100 1000 10000 100000
BXtend Medini QVT
Incremental Backward (log - log scale)
Figure 7: Backward Incremental Transformation: Linear/linear Scale (Left) and Log/log Scale (Right).
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
258
low).
The benchmarx (Anjorin et al., 2017b) frame-
work is an important step towards providing a unified
platform for assessing and benchmarking incremental
model transformations. For the transformation tool
contest 2017, a bidirectional transformation problem
has been accepted, which addresses the transforma-
tion of family and person databases (Anjorin et al.,
2017a). A wide range of solutions was submitted,
which allowed for a comparison of seven significantly
differing tools (Anjorin et al., 2019). An additional,
more detailed comparison of BXtend and QVT-R ap-
plied to the Families to Persons case is described in
(Westfechtel and Buchmann, 2018).
The transformation problem discussed in this pa-
per differs significantly from the Families to Persons
case. While the latter involves information loss and
asymmetric specifications of both transformation di-
rections, the AST2Dag benchmark is concerned with
a symmetric transformation problem without any loss
of information. Both the AST and the Dag represent
the same information in different ways. Transforma-
tions in both directions require restructuring (folding
in forward direction and unfolding in backward di-
rection). Interestingly, currently there is no bidirec-
tional language or tool known that is able to solve
this bidirectional transformation problem in a sin-
gle specification (please recall that our solutions in
BXtend and QVT-R both require two unidirectional
transformations that are specified separately). Thus,
the AST2Dag benchmark constitutes a real challenge
for bidirectional languages and tools.
7 CONCLUSION
In this paper we provide an evaluation of declarative
and imperative approaches to solve a bidirectional
transformation problem. The problem is taken from
compiler construction and addresses expressions rep-
resented as abstract syntax trees (AST) and directed
acyclic graphs (DAG). Incremental and bidirectional
transformations between the two data structures rep-
resented as EMF models have been created using
QVT-R as a representative of declarative approaches
and BXtend, an imperative framework specifically de-
signed for incremental bidirectional transformations.
The experiments revealed that we failed to specify
a single bidirectional transformation with QVT-R.
Rather, two unidirectional transformations were re-
quired. As a consequence, the quantitative analysis
shows that the declarative approach does not provide a
benefit in terms of size of the transformation descrip-
tion. The qualitative analysis is done using Bench-
marx, a framework for evaluating bx tools. A set of
test cases for each transformation direction and for
each mode of operation had to be solved with each
tool. While the BXtend solution was able to pass all
test cases, the QVT-R solution failed in two incremen-
tal forward ones. Furthermore, the performance tests
revealed that the QVT-R solution shows significant in-
creases in runtime for even smaller models, while the
BXtend solution scales much better with increasing
model sizes.
As pointed out in the section on related work,
the AST2Dag case constitutes a challenge for bidi-
rectional transformation languages and tools. To gain
further insights from this case, it would highly desir-
able to have this case implemented in further bidirec-
tional tools (again using the benchmarx framework).
REFERENCES
Anjorin, A., Buchmann, T., and Westfechtel, B. (2017a).
The Families to Persons case. volume 2026 of CEUR
Workshop Proceedings, pages 27–34, Marburg, Ger-
many.
Anjorin, A., Buchmann, T., Westfechtel, B., Diskin, Z., Ko,
H.-S., Eramo, R., Hinkel, G., Samimi-Dehkordi, L.,
and Z
¨
undorf, A. (2019). Benchmarking bidirectional
transformations: theory, implementation, application,
and assessment. Software and Systems Modeling.
Anjorin, A., Diskin, Z., Jouault, F., Ko, H., Leblebici, E.,
and Westfechtel, B. (2017b). Benchmarx reloaded:
A practical benchmark framework for bidirectional
transformations. In Eramo, R. and Johnson, M., ed-
itors, Proceedings of the 6th International Workshop
on Bidirectional Transformations co-located with The
European Joint Conferences on Theory and Prac-
tice of Software, BX@ETAPS 2017, Uppsala, Swe-
den, April 29, 2017., volume 1827 of CEUR Workshop
Proceedings, pages 15–30. CEUR-WS.org.
Anjorin, A., Lauder, M., Patzina, S., and Sch
¨
urr, A. (2011).
Emoflon: leveraging EMF and professional CASE
tools. In Heiß, H., Pepper, P., Schlingloff, H.,
and Schneider, J., editors, Informatik 2011: Infor-
matik schafft Communities, Beitr
¨
age der 41. Jahresta-
gung der Gesellschaft f
¨
ur Informatik e.V. (GI), 4.-
7.10.2011, Berlin, Deutschland (Abstract Proceed-
ings), volume 192 of LNI, page 281. GI.
Arendt, T., Biermann, E., Jurack, S., Krause, C., and
Taentzer, G. (2010). Henshin: Advanced Concepts
and Tools for In-Place EMF Model Transformations.
In Petriu, D. C., Rouquette, N., and Haugen, Ø., ed-
itors, Proceedings 13th International Conference on
Model Driven Engineering Languages and Systems
(MODELS 2010), Part I, volume 6394 of Lecture
Notes in Computer Science, pages 121–135, Oslo,
Norway. Springer-Verlag.
Becker, S. M., Herold, S., Lohmann, S., and Westfechtel,
B. (2007). A graph-based algorithm for consistency
Incremental Bidirectional Transformations: Evaluating Declarative and Imperative Approaches using the AST2Dag Benchmark
259
maintenance in incremental and interactive integration
tools. Software and Systems Modeling, 6(3):287–316.
Buchmann, T. (2018). Bxtend - A framework for (bidi-
rectional) incremental model transformations. In
Hammoudi, S., Pires, L. F., and Selic, B., edi-
tors, Proceedings of the 6th International Confer-
ence on Model-Driven Engineering and Software De-
velopment, MODELSWARD 2018, Funchal, Madeira
- Portugal, January 22-24, 2018., pages 336–345.
SciTePress.
Buchmann, T. and Westfechtel, B. (2016). Using triple
graph grammars to realise incremental round-trip en-
gineering. IET Software, 10(6):173–181.
Czarnecki, K., Foster, J. N., Hu, Z., L
¨
ammel, R., Sch
¨
urr, A.,
and Terwilliger, J. F. (2009). Bidirectional transforma-
tions: A cross-discipline perspective. In Paige, R. F.,
editor, Proceedings of the Second International Con-
ference on Theory and Practice of Model Transforma-
tions (ICMT 2009), volume 5563 of Lecture Notes in
Computer Science, pages 260–283, Zurich, Switzer-
land. Springer-Verlag.
Czarnecki, K. and Helsen, S. (2006). Feature-based sur-
vey of model transformation approaches. IBM Sys-
tems Journal, 45(3):621–646.
Greiner, S. and Buchmann, T. (2016). Round-trip engineer-
ing UML class models and java models: A real-world
use case for bidirectional transformations with QVT-
R. International Journal of Information System Mod-
eling and Design (IJISMD), 7(3):72–92.
Jouault, F., Allilaire, F., B
´
ezivin, J., and Kurtev, I. (2008).
ATL: A model transformation tool. Sci. Comput. Pro-
gram., 72(1-2):31–39.
K
¨
onigs, A. and Sch
¨
urr, A. (2006). Tool integration with
triple graph grammars - a survey. In Heckel, R., ed-
itor, Proceedings of the School of SegraVis Research
Training Network on Foundations of Visual Modelling
Techniques (FoVMT 2004), volume 148 of Electronic
Notes in Theoretical Computer Science, pages 113–
150, Dagstuhl, Germany. Elsevier Science.
Object Management Group (2014). Object Constraint Lan-
guage Version 2.4. Needham, MA, formal/14-02-03
edition.
Object Management Group (2016a). Meta Object Facility
(MOF) 2.0 Query/View/Transformation Specification
Version 1.3. Needham, MA, formal/16-06-03 edition.
Object Management Group (2016b). OMG Meta Object Fa-
cility (MOF) Core Specification Version 2.5.1. Need-
ham, MA, formal/19-10-01 edition.
Popoola, S., Kolovos, D. S., and Rodriguez, H. H. (2016).
EMG: A domain-specific transformation language for
synthetic model generation. In Gorp, P. V. and Engels,
G., editors, Theory and Practice of Model Transfor-
mations - 9th International Conference, ICMT 2016,
Held as Part of STAF 2016, Vienna, Austria, July 4-5,
2016, Proceedings, volume 9765 of Lecture Notes in
Computer Science, pages 36–51. Springer.
Rose, L. M., Kolovos, D. S., Paige, R. F., Polack, F. A. C.,
and Poulding, S. M. (2014). Epsilon flock: a model
migration language. Software and System Modeling,
13(2):735–755.
Schmidt, D. C. (2006). Guest editor’s introduction: Model-
driven engineering. IEEE Computer, 39(2):25–31.
Sch
¨
urr, A. (1994). Specification of graph translators with
triple graph grammars. In Mayr, E. W., Schmidt, G.,
and Tinhofer, G., editors, Graph-Theoretic Concepts
in Computer Science, 20th International Workshop,
WG ’94, Herrsching, Germany, June 16-18, 1994,
Proceedings, volume 903 of Lecture Notes in Com-
puter Science, pages 151–163. Springer.
Steinberg, D., Budinsky, F., Paternostro, M., and Merks,
E. (2009). EMF Eclipse Modeling Framework. The
Eclipse Series. Addison-Wesley, Upper Saddle River,
NJ, 2nd edition.
Westfechtel, B. (2018). Case-based exploration of bidirec-
tional transformations in QVT Relations. Software
and Systems Modeling, 17(3):989–1029.
Westfechtel, B. and Buchmann, T. Incremental bidirectional
transformations: Comparing declarative and procedu-
ral approaches using the families to persons bench-
mark. In Damiani, E., Spanoudakis, G., and Maci-
aszek, L. A., editors, Evaluation of Novel Approaches
to Software Engineering - 13th International Con-
ference, ENASE 2018, Funchal, Madeira, Portugal,
March 23-24, 2018, Revised Selected Papers, pages
98–118.
Westfechtel, B. and Buchmann, T. (2018). Incremental bidi-
rectional transformations: Comparing declarative and
procedural approaches using the families to persons
benchmark. In Damiani, E., Spanoudakis, G., and
Maciaszek, L. A., editors, Evaluation of Novel Ap-
proaches to Software Engineering - 13th International
Conference, ENASE 2018, Funchal, Madeira, Portu-
gal, March 23-24, 2018, Revised Selected Papers, vol-
ume 1023 of Communications in Computer and Infor-
mation Science, pages 98–118. Springer.
ENASE 2020 - 15th International Conference on Evaluation of Novel Approaches to Software Engineering
260