LPO Proofs in Two Educational Contexts

Engelbert Hubbers

a

Institute for Computing and Information Sciences, Radboud University, Nijmegen, The Netherlands

Keywords:

Term Rewriting Systems, Lexicographical Path Order, Tree Style Proof, Fitch Style Proof, Digital Exams.

Abstract:

The purpose of this paper is twofold. First, it introduces several styles for constructing and writing down

mathematical proofs for a speciﬁc technique used in theoretical Computing Science. Second, an inventory

of pros and cons of these proof styles is made in two educational contexts, namely whether the proof styles

help students in understanding the proofs, and whether the proof styles are practical in both written and digital

exams. It turns out that there is no clear winner in both contexts, but the newly introduced so-called shufﬂed

Fitch style is the most practical choice.

1 INTRODUCTION

For several years the author has been involved in

teaching an introductory course on Term Rewriting

Systems (TRS) and λ-calculus to second-year Com-

puting Science students. With respect to the TRS

part, basically three topics are discussed: reduction,

termination, and conﬂuence. For the termination

part, two proof methods are introduced: the seman-

tical method of polynomial monotonic interpretations

(Terese, 2003), and the syntactical method of lexico-

graphic path order, or LPO in short, (Kamin and Levy,

1980; Dershowitz, 1982; Terese, 2003).

After introducing these methods and having the

students do a homework assignment about this topic,

the author always asks which method is preferred

by the students. And typically, they vote like 90%

in favor of the polynomial monotonic interpretations.

Even though they know that one of the beneﬁts of the

LPO method is that they can build the requested order

on the ﬂy: even if students don’t have a clue about a

useful order, they can simply start applying the rules

and then derive such an order. At the exam, often the

TRS to be studied is chosen in such a way that both

techniques can be used for proving termination of the

system. And again, most students opt for the polyno-

mial monotonic interpretations. For instance, in the

exam of 2020, 34 out of 42 students chose the method

of polynomial monotonic interpretations. However,

every now and then there is an exam where students

are enforced to apply the LPO method, and this usu-

ally leads to bad results. Students seem to get lost in

a

https://orcid.org/0000-0002-6182-6493

all the nested applications of the deﬁnition of >

lpo

as

given below in Deﬁnition 1.

In order to solve this problem a few different styles

of visualizations of such an LPO proof are presented

in this paper. After describing some beneﬁts and

drawbacks with respect to the educational objective of

letting the students understand the proofs better, it is

also discussed whether these proof styles can be used

easily in both written and digital exams.

Because this paper is not about comparing the

two different methods for proving termination, but

speciﬁcally about improving the didactics for the LPO

method, we do not explain the details for the polyno-

mial monotonic interpretations method.

2 THE LPO THEORY

The method of LPO was introduced in (Kamin and

Levy, 1980). Later on, many slightly different vari-

ants have been published, for instance in (Dershowitz,

1982; Baader and Nipkow, 1998; Terese, 2003). Ac-

tually, some of these versions are considered more

practical than Deﬁnition 1 below, however, this pa-

per is about the author’s course which happens to use

course notes (Zantema, 2014) with the following def-

inition:

Deﬁnition 1 (>

lpo

). Let > be an order on the set of

function symbols. Let f and g be function symbols.

And let t

1

, . . . , t

n

, u, u

1

, . . . , u

m

be terms. Then

f (t

1

,...,t

n

)>

lpo

u if and only if

1. there exists i ∈ {1,...,n} such that

270

Hubbers, E.

LPO Proofs in Two Educational Contexts.

DOI: 10.5220/0010399402700278

In Proceedings of the 13th International Conference on Computer Supported Education (CSEDU 2021) - Volume 1, pages 270-278

ISBN: 978-989-758-502-9

Copyright

c

2021 by SCITEPRESS – Science and Technology Publications, Lda. All rights reserved

(a) t

i

= u, or

(b) t

i

>

lpo

u

or

2. u = g(u

1

,...,u

m

) and for all i ∈ {1, . . . , m} it holds

that f (t

1

,...,t

n

)>

lpo

u

i

and either

(a) f > g, or

(b) f = g and (t

1

,...,t

n

) >

lex

lpo

(u

1

,...,u

m

).

Rule 2b refers to >

lex

lpo

, which is deﬁned as follows:

Deﬁnition 2 (>

lex

lpo

). Let t

1

, . . . , t

n

, u

1

, . . . , u

m

be

terms. Then (t

1

,...,t

n

) >

lex

lpo

(u

1

,...,u

m

) if and only

if

1. n = m, and

2. there exists i ∈ {1,...,n} such that

(a) t

i

>

lpo

u

i

and

(b) for all j ∈ {1, . . . , i − 1} it holds that t

j

= u

j

.

And this is the main theorem behind the LPO method:

Theorem 3. Let R be a TRS. If ` >

lpo

r for all ` → r

in R, then R is terminating.

So in order to prove that ` >

lpo

r for some ` → r

in R, students have to apply Rules 1b, 2a and 2b re-

peatedly until they end up with a proof obligation

for which they can apply one of the closing rules,

Rules 1a, 2a and 2b.

As a running example in this paper, we will apply

the technique on the two-parameter version of the so-

called Ackermann function (Ackermann, 1928; P

´

eter,

1935; Robinson, 1948):

Example 4. Consider the TRS R deﬁned by these

rules:

A(0,n) → s(n)

A(s(m),0) → A(m,s(0))

A(s(m),s(n)) → A(m,A(s(m),n))

Show that R is terminating by using the lexicographic

path order given that A > s > 0. Because of Theo-

rem 3 we know that it sufﬁces to show that:

1. A(0,n)>

lpo

s(n)

2. A(s(m),0)>

lpo

A(m,s(0))

3. A(s(m),s(n))>

lpo

A(m,A(s(m),n))

3 PROOF STYLES

We now continue by presenting proofs of termination

of the TRS R from Example 4 using different styles.

3.1 Natural Language Style Proofs

In the literature, LPO proofs are usually given in

the form of natural languages proofs, for instance in

the classic works (Baader and Nipkow, 1998) and

(Terese, 2003). Proofs in natural language can be

quite diverse with respect to their formal counterparts.

Because the ﬁrst claim is intrinsically easy to prove,

even a very informal usage of natural language will

sufﬁce. However, a proof for the second claim in an

informal style, will be more difﬁcult to follow:

Claim A(s(m),0)>

lpo

A(m,s(0)) holds be-

cause we can apply Rule 2b. This is allowed

because both the left-hand side term and the

right-hand side term start with the same func-

tion symbol A, in combination with the fact

that A(s(m),0)>

lpo

m, A(s(m),0)>

lpo

s(0),

and (s(m), 0) >

lex

lpo

(m,s(0)). The ﬁrst of these

three facts follows directly from Rule 1a. The

second fact follows from Rule 2a, because

. . . (and so on)

Although this kind of prose is not wrong, students

are advised to use a more formalized approach us-

ing explicit references when writing natural language

proofs. This is what that could look like for the proof

of the third claim:

1. Claim A(s(m),s(n))>

lpo

A(m,A(s(m),n))

holds because we can apply Rule 2b.

Therefore, we have to prove these claims:

• (a): A(s(m), s(n)) and A(m, A(s(m), n))

start with the same function symbol,

• (b): A(s(m),s(n))>

lpo

m,

• (c): A(s(m),s(n))>

lpo

A(s(m),n), and

• (d): (s(m),s(n)) >

lex

lpo

(m,A(s(m),n)).

2. Claim (a) holds because both terms start

with A.

3. Claim (b) holds because we can apply

Rule 1b. Therefore, we have to prove a sin-

gle claim:

• (b.a): s(m) >

lpo

m.

4. Claim (b.a) follows from applying Rule 1a.

5. Claim (c) holds because we can apply

Rule 2b. Therefore, we have to prove three

claims:

• (c.a): A(s(m),s(n))>

lpo

s(m),

• (c.b): A(s(m),s(n))>

lpo

n, and

• (c.c): (s(m),s(n)) >

lex

lpo

(s(m),n).

6. Claim (c.a) follows directly from Rule 1a.

7. And so on for claims (c.b), (c.c), and (d).

This proof is written in the order that these kind of

proofs are typically created, the so-called top-down

LPO Proofs in Two Educational Contexts

271

1a-i

f (t

1

,...,t

n

)>

lpo

t

i

t

i

>

lpo

u

1b-i

f (t

1

,...,t

n

)>

lpo

u

f (t

1

,...,t

n

)>

lpo

u

1

... f (t

1

,...,t

n

)>

lpo

u

m

2a- f > g

f (t

1

,...,t

n

)>

lpo

g(u

1

,...,u

m

)

f (t

1

,...,t

n

)>

lpo

u

1

... f (t

1

,...,t

n

)>

lpo

u

m

(t

1

,...,t

n

) >

lex

lpo

(u

1

,...,u

m

)

2b- f

f (t

1

,...,t

n

)>

lpo

f (u

1

,...,u

m

)

t

i

>

lpo

u

i

lex-i

(t

1

,...,t

i−1

,t

i

,t

i+1

,...,t

n

) >

lex

lpo

(t

1

,...,t

i−1

,u

i

,u

i+1

,...,u

n

)

Figure 1: Derivation rules corresponding to >

lpo

and >

lex

lpo

.

construction: starting with the main complex goal, di-

viding it up into simpler goals step by step. However,

writing out such a proof in this order in a clear way, is

more difﬁcult, due to the nesting of rule applications,

which sort of enforces to use complex references like

(c.b). It turns out to be more easy to write out a proof

starting with the simpler statements and building up to

the ﬁnal complex conclusion, the so-called bottom-up

presentation.

1. From Rule 1a it follows that s(m) >

lpo

m.

2. From claim 1 and Rule 1b it follows that

A(s(m),s(n))>

lpo

m.

3. From the application of Rule 1a it follows

that A(s(m),s(n))>

lpo

s(m).

4. From Rule 1a it follows that s(n) >

lpo

n.

5. From claim 4 and Rule 1b it follows that

A(s(m),s(n))>

lpo

n.

6. From Rule 1a it follows that s(n) >

lpo

n.

7. From claim 6 and the deﬁnition of >

lex

lpo

it

follows that (s(m),s(n)) >

lex

lpo

(s(m),n).

8. From 3, 5, 7 and Rule 2b, it follows that

A(s(m),s(n))>

lpo

A(s(m),n).

9. And so on for the remaining steps. . .

3.2 Tree Style Proofs

Students taking this particular course, all followed a

different course about logic where the theory of nat-

ural deduction was the main topic. In particular, the

proofs in that course were presented in Gentzen tree

style. Students do not really like the fact that these

proofs tend to become quite wide, but they do like

the fact that the structure of the proof is completely

clear. So why not introduce tree style proofs for LPO

as well? Note that this has been done in the past as

well, for instance in (Cichon and Marion, 2000).

The deﬁnitions are quite naturally transformed

into derivation rules:

Deﬁnition 5. Let > be an order on the set of function

symbols. Let f and g be function symbols. And let

t

1

, . . . , t

n

, u, u

1

, . . . , u

m

be terms. Then the derivation

rules corresponding to Deﬁnition 1 and Deﬁnition 2

are given in Figure 1.

These rules require some explanation:

• In Rule 1a, the ‘there exists i ∈

{

1,...,n

}

’ is made

explicit in the name of the rule. Maybe a more

natural alternative would have been:

t

i

= u

1a

f (t

1

,...,t

n

)>

lpo

u

However, in the proofs that would lead to rather

trivial proof obligations of the form u = u, which

would need an additional reﬂexivity axiom to ac-

tually close the branch. (Note that this approach is

taken in (Cichon and Marion, 2000).) Therefore,

we just include the index i in the name of the rule

and replace the u below the line by t

i

.

• In Rule 1b we do write t

i

explicitly above the line,

because in this case there may still be a complex

proof needed above this rule.

• In Rule 2a the characterizing part is the proof obli-

gation f > g. This could have been made explicit

by adding it above the line, but then we would

need another axiom to close this branch. And by

listing it in the name of the rule, we still have a

clear place in the proof where we know that we

can only apply this rule, if the given order indeed

implies that f > g. Note that if g happens to be a

function symbol with no arguments, then Rule 2a

introduces no new proof obligations and acts in

fact like an axiom.

• In Rule 2b we do not write the proof obligation

f = g above the line, but enforce this already

below the line by replacing the original g by f ,

which implies that you can only apply this rule if

CSEDU 2021 - 13th International Conference on Computer Supported Education

272

Proof of A(s(m),s(n))>

lpo

A(m,A(s(m),n)):

1a-1

s(m)>

lpo

m

1b-1

A(s(m),s(n))>

lpo

m T

1

1a-1

s(m)>

lpo

m

lex-1

(s(m),s(n)) >

lex

lpo

(m,A(s(m),n))

2b-A

A(s(m),s(n))>

lpo

A(m,A(s(m),n))

where T

1

is an abbreviation for:

1a-1

A(s(m),s(n))>

lpo

s(m)

1a-1

s(n)>

lpo

n

1b-2

A(s(m),s(n))>

lpo

n

1a-1

s(n)>

lpo

n

lex-2

(s(m),s(n)) >

lex

lpo

(s(m),n)

2b-A

A(s(m),s(n))>

lpo

A(s(m),n)

Figure 2: Tree style termination proof for Example 4.

the leading function symbols are the same. Just

like in Rule 2a, if f is a function symbol with no

arguments, this rule actually operates like an ax-

iom.

• Also in the rule for the deﬁnition of >

lex

lpo

we en-

force the obligation that n = m by replacing the

original m by n below the line. In addition, we

also do not include the proof obligations t

1

= u

1

,

. . . , t

i−1

= u

i−1

above the line, but enforce these

already below the line by replacing u

1

,...,u

i−1

explicitly by t

1

,...,t

i−1

.

As indicated before, if we want to prove that the TRS

from Example 4 is terminating, we have to prove three

claims. Due to space constraints in this paper, we only

provide the tree for the proof of the third claim in Fig-

ure 2. Note that we used an abbreviation T

1

for a sub-

proof, because otherwise the proof would be too wide

to ﬁt. The students of this course are used to this kind

of abbreviations.

Note that the bottom-up presentation in natural

language given before, exactly coincides with the left

branch and the T

1

branch!

Because the trees with their subtrees clearly re-

semble the proof obligations from Deﬁnition 1 and

Deﬁnition 2, the conclusion is that the tree style

proofs really help in clarifying the structure of the

proof.

3.3 Fitch Style Proofs

So the introduced tree style proof is clear and under-

standable to students, mainly because they have seen

proof trees before. However, it has the problem that

the proofs can typically become very wide. And on

paper ‘wide’ means trouble. But proofs that become

‘tall’ usually cause less trouble. So it seems reason-

able to use the same solution that is common to natu-

ral deduction proofs in general: use Fitch notation.

There are several slightly different versions of this

type of proofs, but in general they are all called Fitch

style. See (Pelletier, 1999a) for an overview. What

is the general idea? Proofs are linear, numbered lists

of propositions, starting with assumptions at the top

and conclusions at the bottom. For every (intermedi-

ate) conclusion, it is written explicitly which rule is

applied and on which (lower) line numbers. In case

a temporary assumption is made, this will be visual-

ized by creating a new ‘box’ with this assumption at

the top and the new goal again at the bottom of this

‘box’. This ‘box’ deﬁnes the scope of the new as-

sumption: it is only valid inside its own ‘box’. The

main difference between the variants of Fitch proofs

is the way the ‘boxes’ are drawn. Sometimes, they

are drawn like real boxes, sometimes only with long

lines, sometimes only with short hooks. However, in

the situation of LPO proofs, it doesn’t really matter

which speciﬁc visualization is used for ‘boxes’, be-

cause there are never temporary assumptions intro-

duced, and hence there are never ‘boxes’ used in the

proof!

The proof for the third claim, which can be found

in Figure 3, looks pretty simple in this plain Fitch

style. In fact, it is a slightly more formal version of the

bottom-up presentation in natural language that we

saw before. Note that we didn’t optimize this proof.

Because Fitch proofs can use any proposition that is

both in scope and written above it, we could have op-

timized the proof a bit by removing duplicate propo-

sitions. For instance, in the proof in Figure 3 lines 4

and 6, and lines 1 and 9 are the same. So we could

have removed lines 6 and 9 and change the references

in lines 7 and 10 to 4 and 1 respectively. This opti-

mization was not done in order to keep the relation

with the tree style proofs more clear.

However, although these Fitch representations are

fairly simple, the proofs do have some drawbacks.

LPO Proofs in Two Educational Contexts

273

1. s(m) >

lpo

m 1a-1

2. A(s(m), s(n))>

lpo

m 1b-1 1

3. A(s(m), s(n))>

lpo

s(m) 1a-1

4. s(n) >

lpo

n 1a-1

5. A(s(m), s(n))>

lpo

n 1b-2 4

6. s(n) >

lpo

n 1a-1

7. (s(m), s(n)) >

lex

lpo

(s(m),n) lex-2 6

8. A(s(m), s(n))>

lpo

A(s(m),n) 2b-A 3, 5, 7

9. s(m) >

lpo

m 1a-1

10. (s(m), s(n)) >

lex

lpo

(m,A(s(m),n)) lex-1 9

11. A(s(m),s(n))>

lpo

A(m,A(s(m), n)) 2b-A 2, 8, 10

Figure 3: Linear Fitch style proof of the third claim

A(s(m),s(n)) >

lpo

A(m,A(s(m), n)).

These drawbacks are related to the reasons why the

author typically prefers tree style proofs when con-

structing them on the blackboard during lectures, al-

though a study on elementary logic text books (Pel-

letier, 1999b) showed that out of 33 books, only four

of them use tree style proofs. First, note that proof

trees can easily be created on the blackboard. For

proofs in Fitch style this is more difﬁcult, because you

have to guess in advance how much vertical space is

needed to complete the subproofs. Second, you can

only ﬁll in the reference numbers when the full proof

is ﬁnished, because they typically change when cre-

ating the proof. So at the end you have to be very

precise in ﬁnding the correct lines to reference, which

can easily go wrong because the clear structure of the

tree style proofs is lost. So for longer proofs, it can be

difﬁcult to ﬁnd the proper references at the end.

This problem of lack of clear structure can eas-

ily be solved by adjusting the presentation by using

‘nested hooks’, for indicating the branches of the orig-

inal tree. The result for the proof of the third claim is

displayed in this so-called ‘hooked Fitch style’ in Fig-

1.

s(m)>

lpo

m 1a-1

2. A(s(m),s(n))>

lpo

m 1b-1 1

3.

A(s(m),s(n))>

lpo

s(m) 1a-1

4.

s(n)>

lpo

n 1a-1

5. A(s(m),s(n))>

lpo

n 1b-2 4

6.

s(n)>

lpo

n 1a-1

7. (s(m),s(n)) >

lex

lpo

(s(m),n) lex-2 6

8. A(s(m),s(n))>

lpo

A(s(m),n) 2b-A 3, 5, 7

9.

s(m)>

lpo

m 1a-1

10. (s(m),s(n)) >

lex

lpo

(m,A(s(m), n)) lex-1 9

11. A(s(m),s(n))>

lpo

A(m,A(s(m), n)) 2b-A 2, 8, 10

Figure 4: Fitch style proof of the third claim

A(s(m),s(n)) >

lpo

A(m,A(s(m), n)) with hooks for

subproofs.

ure 4. Now that the structure is clear again, it is much

easier to check the reference numbers, because for

single references they simply refer to numbers higher

in the current hook, and for compound rules, they al-

ways refer to the last lines of all the hooks above on

the current level.

The third variant of a Fitch style proof that is intro-

duced, tries to solve the problems that students have

with the two variants that are already mentioned be-

fore. In the linear Fitch style, the proofs are con-

structed top-down, but presented bottom-up. In ad-

dition, students ﬁnd it difﬁcult to get the reference

numbers right, especially without the hooks. This so-

called ‘top-down Fitch style’ overcomes both prob-

lems by using nested hooks again but this time in a

top-down presentation. See Figure 5. Unfortunately,

it doesn’t solve the vertical space guessing problem.

A(s(m),s(n)) >

lpo

A(m,A(s(m), n))

Apply rule 2b-A; to prove:

A(s(m),s(n)) >

lpo

m

Apply rule 1b-1; to prove:

s(m)>

lpo

m

Apply rule 1a-1; done.

A(s(m),s(n)) >

lpo

A(s(m),n)

Apply rule 2b-A; to prove:

A(s(m),s(n)) >

lpo

s(m)

Apply rule 1a-1; done.

A(s(m),s(n)) >

lpo

n

Apply rule 1b-2; to prove:

s(n)>

lpo

n

Apply rule 1a-1; done.

(s(m),s(n)) >

lex

lpo

(s(m),n)

Apply rule lex-2; to prove:

s(n)>

lpo

n

Apply rule 1a-1; done.

(s(m),s(n)) >

lex

lpo

(m,A(s(m), n))

Apply rule lex-1; to prove:

s(m)>

lpo

m

Apply rule 1a-1; done.

Figure 5: Top-down Fitch style proof of the third claim with

hooks instead of references. Because the visualization of

the nesting is more important than the actual claims, we

took the liberty to reduce the size a bit.

The last variant that is presented here, was brought

to the author’s attention by Cynthia Kop when dis-

cussing this paper. And because her method scores

very well on the evaluation criteria introduced later on

in Section 5, it was decided to include it as well, al-

though it was not used by the author yet in his course.

In this paper this method is referred to as the ‘shufﬂed

Fitch style’. The reason for this name will become

clear after seeing an example.

So how does it work? First, one writes down

the original proof obligation on the ﬁrst line, with-

out any references behind it. Then, one selects the

ﬁrst line that has no proof justiﬁcation behind it, one

applies the appropriate rule on it, and one writes the

CSEDU 2021 - 13th International Conference on Computer Supported Education

274

1. A(s(m), s(n))>

lpo

A(m,A(s(m), n)) 2b-A 2, 3, 4

2. A(s(m), s(n))>

lpo

m 1b-1 5

3. A(s(m), s(n))>

lpo

A(s(m),n) 2b-A 6, 7, 8

4. (s(m), s(n)) >

lex

lpo

(m,A(s(m), n)) lex-1 5

5. s(m) >

lpo

m 1a-1

6. A(s(m), s(n))>

lpo

s(m) 1a-1

7. A(s(m), s(n))>

lpo

n 1b-2 9

8. (s(m), s(n)) >

lex

lpo

(s(m),n) lex-2 9

9. s(n) >

lpo

n 1a-1

Figure 6: Shufﬂed Fitch style proof of the third claim.

new proof obligations under the existing list. The

name of the applied rule (in this case 2b-A), together

with the newly generated line numbers (2, 3, and 4)

can be written down immediately, because they won’t

change anymore. And this process is repeated. So in

the next step the second proof obligation is the ﬁrst

without a proof justiﬁcation. So the appropriate rule

is applied, a single new proof obligation is added, and

the proof justiﬁcation (1b-1 5) is written down. This

process continues until all proof obligations have a

proof justiﬁcation. See Figure 10 for the full construc-

tion.

Once the total proof is ﬁnished, the ﬁrst thing that

is noticeable is that the proof is two steps shorter than

the previous proofs. This is due to the fact that reusing

lines 5 and 9 comes natural in this method, whereas in

the previous methods it would have been possible to

do that as well, but in a less natural way. For instance,

in tree style it is very uncommon to reuse results from

a different branch. The second noticeable thing is that

it really looks like the Fitch proof in Figure 3, except

for the fact that the order is no longer enforced by the

structure of the proof, but by the order in which partic-

ular proof obligations are justiﬁed. When introduced

above, the method stated that each time the ﬁrst obli-

gation without a justiﬁcation should be taken care of,

but that was an arbitrary choice for ease of explain-

ing. The last line, or a random line would have also

worked. This results in a shufﬂed order of a normal

Fitch proof, and hence the name.

4 PROOFS IN DIGITAL EXAMS

In the previous sections, the styles presented were

mainly introduced with the focus on the clarity and

usability when being written down on paper. And for

this it doesn’t really matter whether it is written on pa-

per as part of a homework assignment or as part of an

exam. They are all usable in these situations, although

the more formal notations are easier to check for ac-

tual correctness. The question, however, is whether

these styles are also usable in digital exams.

A few years ago digital exams were introduced

at the author’s institute. One of the reasons for this

was the increasing number of students, making it ever

more work to grade exams. And having a system

that provides a good way of at least partial automatic

grading of student submissions saves a lot of time.

There are several environments for organizing digital

exams, for instance Inspera Assessment (Nordic As-

sessment Innovators, 2020), RemindoToets (Paragin,

2020), TestVision (Teelen, 2020), Cirrus Assessment

(Cirrus BV, 2020), and WISEﬂow (UNIwise, 2020).

The system currently in use at the author’s institution

is the Cirrus Assessment software. Therefore, the rest

of this paper is about usability in Cirrus, but presum-

ably the results will also hold for other digital exami-

nation systems.

Cirrus Assessment is cloud based: it can be used

from campus, but also from home. It used to be

that all exams at the author’s institute were taken in

on campus lecture rooms in order to have controlled

circumstances. However, due to the COVID19 pan-

demic, many exams are nowadays taken at home as

well. In this paper it is not discussed which measures

are taken to control the circumstances when the stu-

dents are taking the exam at home. But it is impor-

tant to stress that due to this pandemic, many courses

that were scheduled to have a regular written exam on

campus, now had to be converted to a digital exam. So

there is a need for dealing with mathematical proofs

like the ones in this paper in digital assessments.

The Cirrus software allows several types of ques-

tions that allow automatic grading, like multiple-

choice questions, multiple-response questions, select

from a list, ﬁll in a blank, or even matching ques-

tions. However, the system was clearly not created

with mathematical exams in mind. Although for some

mathematical questions it is really well possible to re-

design them a bit in order to check the same learning

objective as before, but now in a way that can be au-

tomatically graded, for many other questions that is

just not possible. Recently, Cirrus has added a par-

ticular ‘mathematical question’ that can be automat-

ically graded, based on the platform Sowiso (Sowiso

BV, 2020). This platform is connected to the com-

puter algebra system Maxima (Shelter, 1982), which

allows for randomization and complex evaluation of

the student’s answers. However, also this type of

question is not very suitable for many mathematical

problems. In particular, questions where diagrams

or ﬁgures should be created to answer the question

are difﬁcult to answer. Fortunately, Cirrus imple-

LPO Proofs in Two Educational Contexts

275

Figure 7: Tree style proof for the third claim created in Cirrus.

mented two question types that can be used for this

kind of questions, namely the ‘essay question’ and

the ‘ﬁle response question’. The last one basically

transforms the exam into a regular written exam, be-

cause students can write things on paper, take a pic-

ture of it, and upload it into Cirrus as an answer. This

works pretty well for ‘at home’ exams, but it usually

cannot be used for ‘on campus’ exams, because the

controlled circumstances disallow the usage of smart-

phones at all and there is no alternative available for

scanning student’s notes. Therefore, the focus in this

paper is about the usability of the ‘essay question’ for

submitting LPO proofs.

4.1 Natural Language Style Proofs in

Cirrus

When answering an ‘essay question’, students get a

‘rich text editor’ with the usual features like font se-

lection, markup like bold, italics, superscript and sub-

script, and lists, both with and without numbers. It

won’t be a surprise that this editor is suitable for typ-

ing proofs in natural language. Basically, the only

change is that it is probably wise to replace math-

ematical notation like >

lpo

and >

lex

lpo

by ‘>lpo’ and

‘>lpolex’ respectively, although even this can be ar-

ranged using the superscript and subscript options.

4.2 Tree Style Proofs in Cirrus

It may be a bit surprising, but within this editor

it is pretty well doable to create the trees in plain

ASCII. Again, replacing >

lpo

and >

lex

lpo

by ‘>lpo’ and

‘>lpolex’ respectively, makes it easier. In Figure 7

the tree for the proof of the third claim is included

as it is created inside of Cirrus. The main trick in

creating these trees easily within Cirrus is putting the

editor into a mono-spaced font. The fact that these

trees on paper are created bottom-up, is no problem,

because it is easy to insert new lines above the current

one. Aligning the different branches properly is also

easily established by inserting the proper amount of

spaces or dashes. So it is deﬁnitely doable to create

tree style proofs like this in Cirrus. A clear drawback

is that it takes quite some time.

4.3 Fitch Style Proofs in Cirrus

Whereas writing down Fitch proofs on paper has the

problems of guessing the amount of vertical space for

the subproofs and getting all the reference numbers

properly, in the rich text editor this is actually no prob-

lem at all. This is because of the automatically num-

bered lists. Simply start with the ﬁnal goal as the ﬁrst

item in a numbered list. In the proof of the third claim

this goal is proved by applying Rule 2b, which gives

three new proof obligations. Insert those above the

current goal and now the proof has four lines each

with their own unique label. In addition temporary

references I, II, and III can be added to the last line

because Rule 2b depends on those references. Those

temporary labels should also be added to the proof

obligations that they correspond to. As long as the

proof isn’t ﬁnished, the actual labels may still change

and therefore these temporary labels and references

are used. Once the proof is ﬁnished, all temporary

references can be replaced by the actual labels cor-

responding to the temporary labels. And of course,

all temporary labels can be removed. This process

is partially shown in Figure 8. Note that this exam-

ple provides also temporary labels for non-branching

steps. Of course, more experienced students will no-

tice that these are not really needed, because the ﬁnal

reference will always be one line above in the situa-

tion where we do not optimize for having the same

proof obligations more than once.

Note that the Fitch style proof with hooks is not

easy to create in the rich text editor. If normal dashes

are used for showing the hooks, then the automatic

numbering is broken. However, it can be done by us-

ing the ‘underline’ option of the editor, but that takes

deﬁnitely more effort. We do not include the result in

CSEDU 2021 - 13th International Conference on Computer Supported Education

276

Figure 8: Step by step creation of a Fitch style proof for

the third claim in Cirrus. When the proof is complete all

temporary roman references can be replaced by the ﬁnal line

numbers. The ﬁnal result is an ASCII version of the proof

in Figure 3.

Figure 9: Partial Top-down Fitch style proof for the third

claim created in Cirrus.

this paper.

In contrast to this, the top-down Fitch style from

Figure 5 turns out to be pretty easy to format in the

rich text editor, because vertical and horizontal bars

can be typed directly in the editor. In addition, the

vertical space guessing problem is solved, because it

is easy to insert more space later on. The result is

presented in Figure 9.

However, the easiest method is probably the shuf-

ﬂed Fitch proof. Figure 10 indicates how such a proof

can be created step by step. Note that we have intro-

duced a shortcut here to immediately add the proof

justiﬁcation to a newly generated proof obligation

if this does not require any new proof obligations,

which is typically the case if we can prove a claim

by Rule 1a.

5 CONCLUSION

This paper is concluded by evaluating the proposed

proof styles with respect to the criteria that are impor-

tant in several educational contexts:

1. Do the proofs have an understandable structure

for the students?

Figure 10: Step by step creation in Cirrus of a shufﬂed Fitch

style proof for the third claim.

2. Can the proofs be constructed top-down in a sim-

ple way on paper?

3. Can the proofs be presented top-down in a simple

way on paper?

4. Can the proofs be constructed top-down in a sim-

ple way in a rich text editor?

5. Can the proofs be presented top-down in a simple

way in a rich text editor?

As can be seen in Table 1, there is no deﬁnitive ‘best

method’ that scores a plus on all criteria, but the top-

down Fitch style and the shufﬂed Fitch come close.

The only problem for the ﬁrst one is the necessary

Table 1: Proof styles related to criteria.

Styles

Criteria

1 2 3 4 5

Natural language − ± + ± +

Tree + + − + −

Linear Fitch ± ± − + −

Hooked Fitch + ± − − −

Top-down Fitch + ± + + +

Shufﬂed Fitch ± + + + +

LPO Proofs in Two Educational Contexts

277

vertical space guessing when doing the proof on pa-

per. And the only problem for the latter one is the

lack of structure in the proof. However, the shufﬂed

Fitch style has an additional beneﬁt that was not men-

tioned before: it is completely natural to write a sin-

gle proof in this style starting with all three claims for

the full proof, which is not the case for the top-down

Fitch style! The method will work in the same easy

way. So even though the shufﬂed Fitch style does not

clearly align with the nesting structure of the proof

method in general, it is very easy to present to stu-

dents as an algorithm. Therefore, this is probably the

best and certainly the most practical method for Com-

puting Science students to come up with a good, easy

to create, and easy to check LPO proof! If, on the

other hand, the focus is on the structure of the proof,

then the tree style proofs are probably the best pick,

especially for this group of students that has not seen

Fitch proofs before.

Note that this paper was speciﬁcally about LPO

proofs and about the digital exam environment Cir-

rus. However, the styles can be applied for basically

all mathematical proofs that rely on a series of ap-

plications of clear rules, and also in all other digital

assessment environments that have a reasonable rich

text editor.

6 FUTURE WORK

Because the idea for writing this paper only came up

after the exam and resit of the last time the course was

taught, there was no formal experiment conducted to

support the conclusions in Table 1, but instead gen-

eral impressions from the lectures, the input at the

exam (out of the eight students using LPO, one had

basically nothing, one tried a very informal natural

language proof, four had a formalized natural lan-

guage proof, one had a top-down Fitch style without

the lines, and one had both an informal natural lan-

guage proof and a top-down Fitch style without the

lines because he considered his own natural language

proof not clear enough), and private conversation with

students. In the next round of the course, also the

shufﬂed Fitch method will be introduced and students

will probably speciﬁcally be asked to use several LPO

proof visualizations in the homework and at the exam.

In addition, the question came up to give formal

semantics for the shufﬂed Fitch method and prove

that the method is sound and complete for this spe-

ciﬁc case where no normal ‘boxes’ are needed by

lack of real assumptions. It is also interesting to

check whether this shufﬂed Fitch method also works

for proofs that do use local assumptions. Using the

current informal semantics presented in Section 3.3,

this doesn’t seem likely because claims inside ‘boxes’

cannot typically be used outside these ‘boxes’.

ACKNOWLEDGEMENTS

The author thanks Cynthia Kop for comments on a

preliminary version and in particular for showing him

the so-called ‘shufﬂed Fitch style’ proof, which ac-

tually is the most innovative and promising method

discussed in this paper.

And the author thanks his wife for encouraging

him to ﬁnally write down some of his ideas forthcom-

ing from his teaching in this paper.

REFERENCES

Ackermann, W. (1928). Zum Hilbertschen Aufbau der

reellen Zahlen. Mathematische Annalen, 99(1):118–

133.

Baader, F. and Nipkow, T. (1998). Term rewriting and all

that. Cambridge University Press.

Cichon, E. and Marion, J.-Y. (2000). The Light Lexico-

graphic Path Ordering. CoRR, cs.PL/0010008.

Cirrus BV (2020). Cirrus Assessment. https://

cirrusassessment.com/.

Dershowitz, N. (1982). Orderings for term-rewriting sys-

tems. Theor. Comput. Sci., 17:279–301.

Kamin, S. and Levy, J.-J. (1980). Two Generalizations of

the Recursive Path Ordering. Technical report, Uni-

versity of Illinois, Urbana/IL.

Nordic Assessment Innovators (2020). Inspera Assessment.

https://www.inspera.com.

Paragin (2020). RemindoToets. https://www.paragin.nl/

remindotoets/.

Pelletier, F. J. (1999a). A Brief History of Natural Deduc-

tion. History and Philosophy of Logic, 20(1):1–31.

Pelletier, F. J. (1999b). A History of Natural Deduction and

Elementary Logic Textbooks.

P

´

eter, R. (1935). Konstruktion nichtrekursiver Funktionen.

Mathematische Annalen, 111(1):42–60.

Robinson, R. (1948). Recursion and Double Recur-

sion. Bulletin of the American Mathematical Society,

54:987–993.

Shelter, W. (1982). Maxima, a Computer Algebra System.

http://maxima.sourceforge.net/.

Sowiso BV (2020). SOWISO: A secure online testing and

learning environment for higher education math and

statistics. https://sowiso.nl/en.

Teelen (2020). TestVision. https://www.testvision.nl/en/.

Terese (2003). Term rewriting systems, volume 55 of Cam-

bridge tracts in theoretical computer science. Cam-

bridge University Press.

UNIwise (2020). WISEﬂow. https://www.uniwise.co.uk/

wiseﬂow.

Zantema, H. (2014). Berekeningsmodellen. Course notes.

CSEDU 2021 - 13th International Conference on Computer Supported Education

278