Computer Supported Argumentation Learning: Design of a Learning

Scenario in Academic Writing by Means of a Conjecture Map

Michael Burkhard

, Sabine Seufert

, Reto Gubelmann

, Christina Niklaus

and Patcharin Panjaburee

Institute for Educational Management and Technologies, University of St.Gallen,

St. Jakob-Strasse 21, 9000 St.Gallen, Switzerland

Institute of Computer Science, University of St. Gallen, Rosenbergstrasse 30, 9000 St. Gallen, Switzerland

Faculty of Education, Khon Kaen University, 123 Thanon Mittraphap, Mueang Khon Kaen, Thailand

Keywords: Argumentation Learning, Academic Writing, Learning Design, Higher Education, Natural Language

Processing, Argument Mining.

Abstract: In academic writing, the competency to argue is important. However, first-year students often have difficulties

to construct good arguments. Advances in natural language processing (NLP) have made it possible to better

analyze the writing quality of texts. New tools have emerged which can give students individual feedback on

their texts and the structure of their arguments. While the use of these argumentation learning support tools

can help create better texts, using them in an academic context also carries risks. Learning scenarios are

needed that promote argumentation competency using argumentation tools while also making students aware

of their limitations. To address this issue, this paper investigates how a learning design with an argumentation

learning support tool can be developed to increase the argumentation competency of first-year students. The

conjecture-mapping technique was used, to visualize our assumptions and illustrate the developed learning

design. As part of a first design cycle, the learning design was tested with 80 students in seven academic

writing classes at the University of St.Gallen in Switzerland. Preliminary findings suggest that the learning

design might be helpful to improve the argumentation competency as well as the data-literacy of students (in

relation to argumentation tools). However, further research is necessary to confirm or reject our hypotheses.

1 INTRODUCTION

The competence of being able to argue is important

in everyday life (Scheuer et al., 2010, p. 2) as well as

in a scientific context (Jonassen & Kim, 2010, p.

440). Argumentation provides means by which we

engage in the rational resolution of issues, questions,

disputes, and problem solving (Jonassen & Kim,

2010, p. 439). In academic writing, argumentation is

one of several important competencies to acquire

(Seufert & Spiroudis, 2017, p. 5; Becker-Mrotzek &

Schindler, 2007). However, for students in their first

year, uncertainty appears to be particularly high

because students are still in transition from high

school to university (Vedral & Ederer-Fick, 2015;

Seufert et al., 2021). Students often lack the

requirements to write research papers for academic

writing (Kruse & Chitez, 2014).

To offer students more guidance in developing

their academic writing skills, text production and

feedback on produced texts can be an important

element of teaching (Seufert & Spiroudis, 2017).

However, for the teacher it is often difficult or at least

very time consuming to provide individual feedback

to each student (Jeong et al., 2019).

As a possible solution, since the 1990’s many

software tools have emerged that aim to support

argumentation (see e.g., Scheuer et al., 2010). Such

tools often have the capability to visualize arguments

graphically and point out missing connections

(Scheuer et al., 2010, p. 12). In this way, these tools

can provide individual feedback to each student. Due

to the advances of artificial intelligence (AI) and

natural language processing (NLP) it has become

possible to better analyze the writing quality of texts

(Crossley, 2020). In the context of academic writing,

new support tools have emerged (Rapp & Kauf, 2018;

Strobl et. al., 2019; Burkhard et al., 2022). A very

powerful recent tool is chatGPT (see ChatGPT Pro,

2023) that can compose entire texts and also support

argumentation (if you ask the chatbot to do so). With

ChatGPT, artificial intelligence (AI) has now made

Burkhard, M., Seufert, S., Gubelmann, R., Niklaus, C. and Panjaburee, P.

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map.

DOI: 10.5220/0011984100003470

In Proceedings of the 15th International Conference on Computer Supported Education (CSEDU 2023) - Volume 1, pages 103-114

ISBN: 978-989-758-641-5; ISSN: 2184-5026

 2023 by SCITEPRESS – Science and Technology Publications, Lda. Under CC license (CC BY-NC-ND 4.0)

103

its way into education in schools and universities. The

power of this AI tool has led to widespread concerns

that learners are using it to plagiarize assignments by

creating essays or exam papers rather than taking the

time to develop their own arguments (see e.g.,

(Sharples, 2022; Marche, 2022; Heilweil, 2022).

For students to make sense of the complementary

strengths of such tools (and also to better understand

their limitations), the requirements for developing

good arguments are likely to increase. To this end, it

might be useful to develop learning designs that use

argumentation tools to foster argumentation

competency, but at the same time make students

actively aware of the problems and limitations that

come with the use of these tools. Considering the

identified research desideratum, the following

research question should be addressed:

How can a learning design with an argumentation

learning support tool be developed to increase the

argumentation competency of first-year students?

The objective of the paper at hand is therefore to

develop a learning design by using the argumentation

support tool Artist to foster argumentation

competency of first-year students. The tool Artist is

an adapted version based on the tool Argumentation

Learning developed at the University of St.Gallen by

Wambsganss et al. (2020). The created learning

design was tested with 80 students of the University

of St.Gallen during a first design cycle in the fall term

of 2022. Following an educational design research

(EDR) approach by McKenney and Reeves (2018),

the goal is to contribute to theory and practice

simultaneously.

From a theoretical perspective, the conjectures we

have derived about the learning design can be a

starting point for further research and discussion, as

they highlight the complexity and multiple demands

of technology applications in real classrooms

(compared to a laboratory setting). From a practical

perspective, the paper can serve as a guideline for

other researchers who want to implement similar

projects and explore the potential of the technology in

more detail. It further contributes to a better

understanding on how argumentation learning can be

designed and implemented in the context of academic

writing.

To this end, the paper is structured as followed:

Section 2 lays the foundation for our design by

elaborating on the theoretical background of

argumentation competency for academic writing and

how it can be fostered. Section 3 provides information

about the applied research design and the methods

used. Section 4 describes the learning design to foster

argumentation competency of first-semester students.

Section 5 gives insights into the testing of the learning

design and critically reflects on the chosen approach.

Section 6 concludes with some final remarks.

2 THEORETICAL

BACKGROUND

2.1 Argumentation Competency

for Academic Writing

Argumentation can be defined “as the valid

combination between claims and premises” (Rapanta

et al., 2013, p. 483). In the philosophy of logic,

"validity" is used in different ways, depending on the

specific relationship between premise(s) and claim

(Gubelmann et al., 2022). With a deductively valid

inference, it is not logically possible that the premise

is true, while the conclusion is wrong. Deductively

valid inferences then divide into inferences that are

deductively valid due to the form of premise(s) and

conclusion. Such formally valid inferences are the

domain of formal logics. Other inferences are

deductively valid due to the content, or meaning, of

premise and claim. They are usually called materially

valid. In addition to deductively valid inferences,

there are defeasible valid ones, where the truth of the

premise(s) gives reason to accept the truth of the

conclusion without guaranteeing it. Many everyday

inferences are of this sort, variously called inductions

or abductions (inferences to the best explanation). An

overview on this terminology is given in Figure 1.

Figure 1: Kinds of valid inferences. Source: Gubelmann et

al. (2022).

For education, argumentation competence is

considered important, because it is associated with

higher-order thinking, helps students to connect

information across contexts, separates relevant from

irrelevant information and increases the ability of

students to explain their knowledge (Rapanta et al.,

2013, p. 484). Being able to argue is important in

everyday life (Scheuer et al., 2010, p. 2) as well as in

a scientific context (Jonassen & Kim, 2010, p. 440).

Fostering argumentative activities incorporated in

CSEDU 2023 - 15th International Conference on Computer Supported Education

104

learning environments support productive thinking as

well as conceptual change (Jonassen & Kim, 2010, p.

439). In the context of academic writing,

argumentation is an important – though not the only

– competency to acquire (Seufert & Spiroudis, 2017,

p. 5; Becker-Mrotzek & Schindler, 2007). By

explicitly supporting their claims with premises,

students examine and reveal their assumptions about

knowledge domains, which likely leads to a more

relativistic, differentiated view (Jonassen & Kim,

2010, p. 440).

In education, argumentation can be approached

from two different perspectives: 1) the arguing to

learn approach and 2) the learning to argue

approach (Jonassen & Kim, 2010; Rapanta et al.,

2013, p. 486). In the 1) arguing to learn approach,

“learning emerges as a natural result of an

argumentative intervention” (Rapanta et al., 2013, p.

486). An example for arguing to learn in the context

of academic writing would be, if students peer-review

each other’s drafts, critically discuss and argue about

their texts and learn from that interaction. In the 2)

learning to argue approach, this relationship is

reversed. In this approach, the focus lies on

argumentation itself, how it can be fostered as well as

its benefits. An example for learning to argue in the

context of academic writing would be if students

learn more about the logic of text structures in order

to create more convincing arguments. This paper

investigates the topic from the second perspective and

adopts a 2) learning to argue approach.

Rapanta et al. (2013, pp. 489-491) further

distinguish between 1) argument as form, 2)

argument as strategy and 3) argument as goal. From

the perspective of 1) argument as form, arguments are

primarily investigated as products consisting of

different forms of premise-claim statements. From

the perspective of 2) argument as strategy, the focus

of interest lies “in the procedure of the argument

exchange” (Rapanta et al., 2013, p. 491). As

arguments are often embedded in a dialogical context,

arguments are analyzed from a strategic view based

on different argumentative moves (Rapanta et al.,

2013, p. 490). Finally, from the perspective of 3)

argument as goal, the focus of interest lies on the

overall discursive process, which traditionally has

been persuasion (Walton, 1989; Rapanta et al., 2013,

p. 491). The critical discussion in general or the

negotiation of content to reach consensus might be

other goals of argumentation (Baker, 1999; Rapanta

et al., 2013, p. 491).

In this paper, we primarily adopt the view of 1)

argument as form, which can be represented by

Toulmin’s argument pattern (TAP) (Rapanta et al.,

2013, p. 489). TAP (see Figure 2) is a prominent

model of rhetorical argumentation developed by

Figure 2: Revised version of Toulmin’s argument pattern (TAP). Source: Own representation based on Toulmin (2003, p. 97)

and Amhag (2011, p. 4).

Qualifier (Q)

Related to the claim and indicates the degree

of strength in the claim of using peculiar

comments

Data (D)

Information which the claim is based

(previous research, personal experience,

common sense or statements) and are used

as evidence to support this claim

Claim (C)

Assertions about what exists or the

justification of the norms or values

that people hold or desire for

acceptance of the claim

Warrant (W)

Explicit of implicit argument that explains

the relationship between data and claim

“Because” / “Since”

Backing (B)

Connected directly to the

warrant, with often implicit

motives underlying underwriting

and claims

“Because of” / “On account of”

Rebuttal (R)

Connected to the qualifier with the

statements of facts that either

contradict the claim, data or rebuttal

or qualify an argument

“But”

“Unless”

“Therefore”

“Hence”

Mandatory elements

Optional elements

“Probably” / “Maybe”

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map

105

Toulmin in 1958 (Jonassen & Kim, 2010, p. 440).

According to Toulmin (2003, pp. 89-100), the

argumentation model consists of the elements claim

(C), data (D), warrant (W), qualifier (Q), backing (B)

and rebuttal (R). As depicted in Figure 2, an arguer

justifies a claim (C) by a fact (D) which both are linked

through a warrant (W), that explains the relationship

between the fact (D) and the claim (C) (Amhag, 2011,

p. 4). Additional optional elements can be added, such

as the qualifier (Q), which indicates the degree of

strength of the relationship through words such as

“probably” or “maybe” (Amhag, 2011, p. 4). Other

optional elements are the rebuttal (R), which

relativizes existing statements using words such as

“but” or “unless”; as well as the backing (B), which is

linked to the warrant (W) and states further implicit

motives and assumptions (Amhag, 2011, p. 4).

In addition to the TAP, there exist also simplified

argumentation models, which usually only consist of

a claim (C) and one or multiple premises (P) (see e.g.,

Stab & Gurevych, 2014; Wambsganss et al., 2020).

Figure 3 illustrates such a simplified model, where

two premises (in the TAP they were called data (D)

and warrant (W)) are combined to justify the claim

that “Marie Curie is mortal” (see Figure 3).

Our argumentation support tool Artist will rely on

the simplified model based on claims and premises

(see Figure 3) to focus on the most important aspects

and make the system as robust as possible. However,

for the overall learning design, we will use the TAP

(see Figure 2) as a reference framework to highlight

contents that our tool Artist cannot currently cover.

These contents such as the Backing (B) can then be

discussed verbally by the teacher during the learning

scenario to show students the current limitations of

our tool and to sensitize them to other important

aspects of argumentation.

Figure 3: Argument pattern with claim and premise(s).

Source: Own representation based on Toulmin (2003, p.

100) and Stab & Gurevych (2014, p. 1503).

2.2 Fostering Argumentation

Competency

Regarding a learning to argue approach, how can

argumentation competency be fostered? According to

Jonassen and Kim (2010, pp. 444-454), various

methods can be used for developing argumentation

competency in the classroom as well as in other

learning environments.

First, Jonassen and Kim (2010, p. 445) consider it

essential that students engage in meaningful, project-

based or problem-based learning tasks. In their view,

a good learning environment confronts students with

a puzzling claim or solution they must resolve.

Second, counterarguments should be created by

the students in order to better understand opposing

positions and adopt a less self-centred more holistic

perspective of a given topic (Jonassen & Kim, 2010,

p. 445).

Third, scaffolding elements can be used to stimulate

students’ thinking processes by asking topic relevant

questions (Jonassen & Kim, 2010, p. 446). The concept

of scaffolding goes back to the work of Wood et al.

(1976), who defined scaffolding as a “process that

enables a child or novice to solve a problem, carry out

a task or achieve a goal which would be beyond his

unassisted efforts” (Wood et al., 1976, p. 90).

Scaffolding can occur through different channels such

as hints, prompts, illustrations, or the provision of

feedback (Duffy & Azevedo, 2015). Graphical

argumentation aids are widely used (see e.g., Kirschner

et al., 2003) to visualize arguments to improve their

construction (Jonassen & Kim, 2010, p. 448).

In addition, since the 1990’s many software tools

have emerged that aim to support argumentation (see

e.g., Scheuer et al., 2010). Such tools often have the

capability to visualize arguments graphically and

point out missing connections (Scheuer et al., 2010,

p. 12). These tools can be used to support argument

analysis as well as argument generation (Scheuer et

al., 2010, p. 13). Depending on the use case, different

kind of feedback mechanisms such as immediate

system feedback, on-demand feedback, summative

system feedback or moderator-driven feedback may

be appropriate to support the learner (Scheuer et al.,

2010, p. 28).

Due to the advances of artificial intelligence (AI)

and natural language processing (NLP) it has become

possible to better analyze the writing quality of texts

(Crossley, 2020). In the context of academic writing,

new support tools have emerged (Rapp & Kauf, 2018;

Strobl et. al., 2019; Burkhard et al., 2022). For

example, the scientific writing assistant is able to

provide feedback on the overall students’ text

structure (Turunen, 2013). The tool can draw

attention to the fact that certain elements that occur in

the text are not mentioned in specific sections (e.g.,

the abstract); or that some passages (e.g., introduction

section) might be relatively too long or too short in

comparison to the rest of the text (Turunen, 2013).

CSEDU 2023 - 15th International Conference on Computer Supported Education

106

The tool AcaWriter provides students with a

reflective report on their inserted text, highlighting

rhetorical moves that are usually used to construct

convincing texts (Knight et al., 2020; University of

Technology Sydney, 2019). For example, the tool

indicates if background information and previous

literature on the topic seems to be missing; or if the

topic is only treated in a very one-sided way (Knight

et al., 2020, p. 153).

Another tool that directly supports argumentation

is the application Argumentation Learning by the

research group around Wambsganss et al. (2020). In

a first step, students can insert their own texts. The

tool then analyzes the logical structure of the text by

identifying argument components (claims and

premises), as well as the relationships between pairs

of argumentative units (in the same logic as depicted

in Figure 3). Moreover, a set of summary quality

scores are assessed (readability, coherence and

persuasiveness). The results are presented in a

learning dashboard through (i) in-text highlighting of

the argument components; (ii) graph visualization of

the argumentation structure; and (iii) bar-chart

visualization of the three quality dimensions. In that

way, students receive immediate and personalized

feedback that supports them in iteratively improving

their argumentation if needed.

The study by Wambsganss et al. (2020) showed in

a laboratory experiment that students working with

the tool Argumentation Learning were able to write

“more convincing texts with better formal quality of

argumentation” compared to students using a

traditional discussion scripting approach based on

Stegmann et al. (2012) (Wambsganss et al., 2020, p.

1). In addition, design principles related to the design

and development of an argumentation feedback tool

(e.g., to provide the learning tool as a web-based

application, to provide the learning tool with a visual

argumentation and discourse feedback on written or

spoken information) have been worked out by

Wambsganss et al. (2020, p. 5).

Data-rich environments require a certain level of

data-literacy to realize their potential in and out of the

classroom (Wasson et al., 2016). At the same time,

when working with data-rich environments, students

can train their data-literacy competency. Data-literacy

can involve many different aspects such as the analysis

and interpretation of data, understanding problems

when using data or the critical reflection about data in

general (Bonikowska et al., 2019). Only if students can

interpret the data provided by the learning environment

(e.g., data visualization, feedback metrics),

argumentation support tools can develop their full

potential and increase learning gains.

3 RESEARCH

DESIGN & METHODS

In this paper, in the context of a 4-year project funded

by the Swiss National Science Foundation (SNSF),

we will build on the tool Argumentation Learning

created by the research group around Wambsganss et

al. (2020) with the goal of further adapting and

improving it to the context of an actual classroom.

Compared to a laboratory setting, other contextual

factors need to be considered such as the course

syllabus or adjusting the learning content to the level

of the students. The goal is to create a meaningful

teaching and learning scenario using the

argumentative writing tool Artist.

As methodological foundation for the design and

development of the teaching scenario, we follow the

educational design research (EDR) approach by

McKenney and Reeves (2018). EDR has two goals it

simultaneously tries to achieve: On the one hand,

EDR makes contributions to theory as it helps to

improve the theoretical understanding, which e.g., in

the form of guidelines can serve as a building block

for the design of future interventions. On the other

hand, EDR makes also contributions to practice, as it

addresses the problem at hand and provides maturing

interventions (McKenney & Reeves, 2018, p. 86).

EDR consists of three main processes: 1) analysis and

exploration, 2) design and construction, as well as 3)

evaluation and reflection (McKenney & Reeves,

2018, p. 77).

The first EDR process, 1) analysis and exploration

was covered in chapter 2, which introduced important

concepts related to argumentation competency for

academic writing and how it can be fostered. In

addition to that, features of state-of-the-art writing

tools in higher education were analyzed and compared,

whose findings have been published in a previous

paper (see Burkhard et al., 2022).

Building on this knowledge, the second EDR

process 2) design and exploration will be addressed in

chapter 4. In this chapter, the design of the teaching and

learning scenario with the argumentative writing

support tool Artist will be presented. To illustrate our

assumptions, structures, processes, and the expected

dependencies, we will use the conjecture mapping

technique by Sandoval (2014), which can be used to

conceptualize educational design research (see e.g.,

Moser et al., 2021; Boelens et al., 2020; Wozniak,

2015).

The third EDR process 3) evaluation and

reflection is discussed in chapter 5. In this chapter, the

take aways and lessons learned from an initial testing

of the designed learning scenario in seven academic

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map

107

writing classes (in total 80 first semester students) at

the University of St.Gallen will be described.

4 ARTIFACT DESCRIPTION:

LEARNING DESIGN TO

FOSTER ARGUMENTATION

COMPETENCY

Figure 4 shows the conjecture map of the designed

artifact, a learning design to foster argumentation

competency of first-semester university students.

From left to right, Figure 4 is arranged into the four

components high level conjectures (what are the

overall assumptions?), embodiment (what materials,

tasks and structures are needed for the learning

design?), mediating processes (how does

embodiment lead to observable interactions and

artifacts?) as well as outcomes (what are the desired

learning outcomes?).

4.1 High Level Conjectures

In the previous sections 1 and 2, we have already

described the high level conjectures necessary for our

scenario. Based on the overall goal to develop

argumentation competency for academic writing (I.),

we use a problem-based learning approach (II.) as

well as a computer-supported argumentation learning

tool (Artist) (III.) to foster learning. As important

design principles, we rely on the learning-to-argue-

approach (V.) (see Jonassen & Kim, 2010) as well as

on elements that have been found to characterize

good argumentation learning tools (VI.) (see Scheuer

et al., 2010; Wambsganss et al., 2020, p. 5).

4.2 Embodiment

Regarding the embodiment of the designed learning

scenario, students have access to the web-based

learning tool Artist (1). Figure 5 shows the user

interface of the learning tool. Among other things,

students can load predefined examples or generate

their own texts. By clicking a button, the

argumentative discourse structure of students' text is

mined (using pre-trained classifiers) and the scores

for the quality dimensions are calculated. Based on

the results of the argument analysis process, the

student receives visual feedback through a graphical

representation of their arguments. In addition,

students are provided with lecture slides (2) depicting

key argumentation concepts similar to the one in

Figure 3.

Figure 4: Conjecture map of the designed learning scenario.

CSEDU 2023 - 15th International Conference on Computer Supported Education

108

Figure 5: User interface of the learning tool Artist.

Note: The tool Artist is an adapted version based on the tool Argumentation Learning developed at the University of St.Gallen

by Wambsganss et al. (2020). The tool Artist was tested with German-speaking students in the German language. For the

purpose of illustration, Figure 5 is displayed in English.

Based on the provided lecture slides (2), the

teacher in a first step clarifies fundamental concepts

(3) such as the distinction between claims and

premises or the use of indicators to construct

arguments (e.g., through words like “because”,

“therefore” etc.). We consider this step important to

bring everyone on the same level and to avoid

conceptual misunderstandings. After that, students

are advised to use Artist. In the sense of a problem-

based learning approach (II), the students are

confronted with multiple broken examples that are

not working properly inside the Artist tool (4).

Students are given the task of correcting and

improving the incorrect examples by analyzing the

examples with the tool and adjusting them as they see

fit. While correcting the flawed examples, students

are required to back up both arguments as well as

counterarguments on the same topic in order to adopt

a less self-centred and more holistic perspective.

Figure 6 shows such a broken example (left side of

Figure 6), that had to be fixed (right side of Figure 6).

After students have become familiar with the

Artist tool by solving multiple predefined examples

on a given topic (4), the students are given the task of

creating their own conclusion on the topic and

justifying it (5). After that, a classroom discussion

between the teacher and the students takes place,

where the experiences made with Artist are critically

reflected and limitations pointed out (6). Overall, the

embodiment (see Figure 4) can be characterized by a

mixture of individual exercises with Artist as well as

classroom discussions (7), in which the teacher takes

on the role of an instructor and moderator of

classroom interaction (8).

4.3 Mediating Processes

As mediating processes (see Figure 4), we can

observe the students' use of the Artist dashboard (a.),

how they experiment with strategies to fix the broken

examples (b. & e.) as well as the generation of their

own texts inside the Artist tool (c.) to improve their

text quality (g.). In addition, the teacher can observe

student motivation (c.) (e.g., during classroom

discussions) as well as student autonomy (d.) (e.g.,

measured by how many times students need

assistance while working with Artist).

4.4 Outcomes

Regarding the outcomes (see Figure 4), on the one

hand, we expect that students have improved

argumentation competency. Because students are

guided in their argument creation by the Artist tool,

we expect them to create arguments with better

quality (i.). In addition, as during the classroom

discussions experiences made with the tool are

critically reflected and argumentative concepts

investigated, we expect that students will have an

improved content knowledge about argumentation

(ii.). On the other hand, students may also have

improved data-literacy due to the participation in the

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map

109

Figure 6: Problem-based learning environment: fixing broken examples.

Note: The tool Artist was tested with German-speaking students in the German language. For the purpose of illustration,

Figure 6 is displayed in English.

learning scenario. By working with the tool as well as

participating in the classroom discussions, students

get a better understanding of the capabilities and

limitations of text analysis (iii.) and improve their

understanding on how to interpret data (iv.) (e.g.,

understanding data visualization and feedback

metrics in general, applying them to their own text).

5 EVALUATION & REFLECTION

As part of an initial EDR design cycle, the learning

design described in the previous section was tested in

the fall term of 2022 at the University of St. Gallen

with seven academic writing classes. The learning

design was tested with German-speaking students in

the German language. Since a total of 80 students

participated in the learning design, this corresponds

to a class size of 10-15 students per class.

In a first phase, the teacher clarified fundamental

argumentation concepts. For this purpose, lecture

slides were used. Students learned about the

difference between claims and premises as well as

indicators (e.g., “because”, “as a result”) to construct

arguments. Since students in the first semester are

very heterogeneous in terms of their prior knowledge,

this approach seemed meaningful to establish a

common ground (e.g., regarding the terminology used

to describe arguments). During this phase of around

five to ten minutes, students seemed motivated and

had only few comprehension questions.

In a second phase, the teacher shared the link to

Artist. The students were given the task to solve

within Artist the predefined examples about the topic

of “public surveillance”. In the process, students had

to correct and solve one example related to the pro-

arguments and one example related to the contra

arguments. After that, students were given the task by

the teacher to generate with Artist their own

conclusion about the topic and to justify it. During

this phase of around ten minutes, the teacher walked

around the classroom and observed the students'

CSEDU 2023 - 15th International Conference on Computer Supported Education

110

behavior. Because all examples (as well as sample

solutions if needed) were directly available in Artist,

students could work independently and had only few,

if any, questions.

In a third phase, students were asked to complete

a short survey to stimulate reflection on the tool usage

and to get formative feedback for future improvement

of the Artist tool. In three open ended questions,

students were asked 1) what they liked about the tool,

2) what features should be improved, and 3) to state

reasons why they would or would not use the tool

Artist as a support to write their own seminar paper.

Overall, the received feedback was mixed. 25 out

of 80 students (31%) found the tool useful and would

like to use it for writing their own seminar paper.

These students thought the tool provided valuable

feedback. 15 out of 80 students (19%) would not use

the tool. These students mostly stated that they found

the tool confusing in general, or that they were able

to construct better arguments themselves in a more

time efficient way without using the tool. 40 out of 80

students (50%) were still undecided and answered

with maybe or probably. These students often thought

that the tool could be valuable to get a second opinion,

but that they sometimes had difficulties to understand

why the tool made certain recommendations. Due to

that fact, they found it difficult to fully rely on it.

After the survey, a classroom discussion took

place, where the experiences made with Artist were

critically reflected and limitations pointed out by the

teacher. On the one hand, working with Artist can be

valuable for first-year students, as it helps to

understand the basic concepts of argumentation such

as claims and premises. It also encourages thinking

about meaningful text structure as well as the use of

indicators to make an argument explicit. On the other

hand, argumentation is often much more complex as

it involves more than just claims and premises (see

Toulmin’s argument pattern in Figure 2). For

example, arguments often involve further backing

(B), implicit motives underlying the premises and

claims (Amhag, 2011, p. 4). Such implicit motives are

often not mentioned, or one is not even explicitly

aware of them. Particularly first-year students have

trouble understanding their own positionality and

thus their implicit assumptions about the world

(Holmes, 2020). Writing positionality statements

with students may be helpful to make implicit

assumptions about the world explicit (Robinson &

Wilson, 2022, pp. 10-16) to adopt a less self-centric,

more holistic argumentation perspective. Such

content is currently not included in Artist, but could

be added to the learning environment in a next step.

As an additional limitation of the tool, the applied

machine learning approach used by the tool to create

the feedback recommendations was mentioned to the

students. Since machine learning approaches today

are often a black-box, it is difficult or even impossible

to interpret why certain recommendations were made

(Zornoza, 2020). Even though it is explained within

Artist how the displayed metrics are (roughly)

calculated and therefore attempted to create a certain

transparency, not every recommendation made by the

tool is comprehensible down to the last detail due to

the applied machine learning approach. Therefore,

students must critically question the

recommendations they receive by the tool and

strengthen in this way their data-literacy competency.

Figure 7: The three different phases of the learning

scenario.

Overall, we believe that the testing of the learning

design as part of a first EDR design cycle has been

mostly successful. However, our designed learning

scenario is subject to several limitations. First, the

learning design (and its tool Artist) is still in a

development phase and has therefore been tested with

only seven classes whose teachers possessed a certain

affinity for technology and were already familiar with

the interface of the Artist tool. To test external

validity, a larger sample size would be desirable. In

handling Artist, additional new teachers may need to

be instructed. Second, students only worked with

Artist for a relatively short period of time of around

ten to fifteen minutes because the tool was used in the

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map

111

context of a normal classroom lesson to learn basic

argumentation competency. In a next step, it would

be interesting to investigate the use over a longer time

period and with longer texts to see if Artist is not only

helpful for learning basic argumentation competency

but can support students also in their daily text writing

(e.g., for writing a seminar paper). Third, no control

group design was used, making it difficult to draw a

definitive conclusion about the learning outcomes

achieved. However, consistent with the EDR

approach and the goal of obtaining formative

feedback as part of an initial design cycle, this

limitation was deliberately accepted.

In a next step, the goal will be to integrate more

elements of the learning scenario into the Artist tool.

For example, the clarification of fundamental

argumentation concepts (3), undertaken by a teacher

in our learning scenario, could be outsourced directly

to Artist as part of an enhanced onboarding process.

In the sense of a self-learning environment, this

would allow students to work more independently

with Artist.

5 CONCLUSION & OUTLOOK

This paper investigated how learning designs with

argumentation learning support systems can be

developed to increase argumentation competency of

first-year university students. Building on literature

about argumentation competency and how it can be

fostered (see section 2), the conjecture mapping

technique of Sandoval (2014) was used, to illustrate

our assumptions as well as the expected conjectures.

The designed learning scenario has the dual goal of

fostering argumentation competency as well as data-

literacy of students. Although the preliminary

feedback received from the classes is promising,

further iterative EDR cycles of development are

needed to improve our learning design and to evaluate

it for its learning effects.

From a theoretical perspective, the conjectures we

have derived about the learning design can be a

starting point for further research and discussion, as

they highlight the complexity and multiple demands

of technology applications in real classrooms

(compared to a laboratory setting). From a practical

perspective, the paper can serve as a guideline for

other researchers who want to implement similar

projects and explore the potential of the technology in

more detail. It further contributes to a better

understanding on how argumentation learning can be

designed and implemented in the context of academic

writing.

The designed learning scenario with Artist shows

that writing tools can be used to support and relieve

the teacher in the classroom. While using digital tools

for education does not mean that fewer teachers are

needed (Dillenbourg, 2016), the role of the teacher

may evolve and change.

Although writing and argumentation tools can

support us in our writing and even are able to create

whole texts for us (such as chatGPT), argumentation

competency will – in our view – remain of critical

importance. Only if we understand what determines

good arguments and can critically reflect on them, we

will be able to make sense of the recommendations of

such tools and adapt them to our needs. The GPT-3

language model (underlying model of ChatGPT)

provides developers with a playground for prototypes

to create training systems like Artist. However, once

GPT-3 is used for an extended period of time, usage

fees apply. The vision for the use of AI in academic

writing could be to build an ecosystem of available

tools for students and teachers in a digitally protected

educational space. However, it is still an open

question whether it makes sense to work with and

build upon GPT-3 or to continue to use and develop

smaller, open-source language models for this

targeted purpose of argumentation.

ACKNOWLEDGEMENTS

We would like to thank the Swiss National Science

Foundation (SNSF) for the grant to support our

project Next Generation of Digital Support for

Fostering Students’ Academic Writing Skills: A

Learning Support System based on Machine Learning

(ML), a collaboration project between the University

of St.Gallen in Switzerland and the Mahidol

University in Thailand.

We further would like to thank the research team

around Thiemo Wambsganss, Christina Niklaus,

Matthias Cetto, Matthias Söllner, Siegfried

Handschuh and Jan Marco Leimeister, who created

the tool Argumentation Learning on which our tool

Artist is based on.

REFERENCES

Amhag, L. (2011). Students’ argument patterns in

asynchronous dialogues for learning. Research

highlights in technology and teacher education, 3, 137-

144.

Baker, M. J. (1999). Argumentation and constructive

interaction. Foundations of argumentative text

CSEDU 2023 - 15th International Conference on Computer Supported Education

112

processing, 5, 179-202. University of Amsterdam

Press.

Becker-Mrotzek, M., & Schindler, K. (2007). Texte

schreiben. Kölner Beiträge zur Sprachdidaktik, 7-26.

Gilles & Francke Verlag.

Boelens, R., De Wever, B., & McKenney, S. (2020).

Conjecture mapping to support vocationally educated

adult learners in open-ended tasks. Journal of the

Learning Sciences, 29(3), 430-470. https://doi.org/

10.1080/10508406.2020.1759605

Bonikowska, A., Sanmartin, C., & Frenette, M. (2019).

Data literacy: What it is and how to measure it in the

public service. Statistics Canada, Analytical Studies

Branch. https://www150.statcan.gc.ca/n1/pub/11-633-

x/11-633-x2019003-eng.htm

Burkhard, M., Seufert, S., Panjaburee, P., Pichitpornchai,

C., & Niklaus, C. (2022). Micro-and Macro-Level

Features of NLP-Based Writing Tools in Higher

Education. Proceedings of the 30th International

Conference on Computers in Education (ICCE 2022).

Kuala Lumpur, Malaysia.

ChatGPT Pro. (2023, January 23). ChatGPT: The Most

Advanced AI Chatbot in 2022. https://chatgpt.pro/

Crossley, S. A. (2020). Linguistic features in writing quality

and development: An overview. Journal of Writing

Research, 11(3), 415-443. https://doi.org/10.17239/

jowr-2020.11.03.01

Dillenbourg, P. (2016). The evolution of research on digital

education. International Journal of Artificial

Intelligence in Education, 26(2), 544-560.

https://doi.org/10.1007/s40593-016-0106-z

Duffy, M. C., & Azevedo, R. (2015). Motivation matters:

Interactions between achievement goals and agent

scaffolding for self-regulated learning within an

intelligent tutoring system. Computers in Human

Behavior, 52, 338-348. https://doi.org/10.1016/j.chb.

2015.05.041

Gubelmann, R., Niklaus, C., & Handschuh, S. (2022). A

Philosophically-Informed Contribution to the

Generalization Problem of Neural Natural Language

Inference: Shallow Heuristics, Bias, and the Varieties

of Inference. In Proceedings of the 3rd Natural Logic

Meets Machine Learning Workshop (NALOMA III), 38-

50, Galway, Ireland. Association for Computational

Linguistics..

Heilweil, R. (2022, December 7). AI is finally good at stuff,

and that’s a problem. Vox. https://www.vox.com/

recode/2022/12/7/23498694/ai-artificial-intelligence-

chat-gpt-openai

Holmes, A. G. D. (2020). Researcher Positionality--A

Consideration of Its Influence and Place in Qualitative

Research--A New Researcher Guide. Shanlax

International Journal of Education, 8(4), 1-10.

Jeong, H., Hmelo-Silver, C. E., & Jo, K. (2019). Ten years

of Computer-Supported Collaborative Learning: A

meta-analysis of CSCL in STEM education during

2005–2014. Educational research review, 28, 100284.

https://doi.org/10.1016/j.edurev.2019.100284

Jonassen, D. H., & Kim, B. (2010). Arguing to learn and

learning to argue: Design justifications and guidelines.

Educational Technology Research and Development,

58(4), 439-457. https://doi.org/10.1007/s11423-009-

9143-8

Kirschner, P. A., Buckingham-Shum, S. J., & Carr, C. S.

(2003). Visualizing argumentation: Software tools for

collaborative and educational sense-making. Springer.

Knight, S., Shibani, A., Abel, S., Gibson, A., Ryan, P.,

Sutton, N., Wight, R., Lucas, C., Sándor, Á., Kitto, K.,

Liu, M., Vijay Mogarkar, R., & Buckingham Shum, S.

(2020). Acawriter: A learning analytics tool for

formative feedback on academic writing. Journal of

Writing Research, 12(1), 141-186. https://doi.org/10.17

239/jowr-2020.12.01.06

Kruse, O., & Chitez, M. (2014). Schreibkompetenz im

Studium: Komponenten, Modelle, Messung. In

Dreyfürst, S. & Sennewald, N. (Eds.). Grundlagentexte

zur Theorie, Didaktik und Beratung, 107-126. Verlag

Barbara Budrich.

Marche, S. (2022, December 6). The College Essay Is

Dead. The Atlantic. https://www.theatlantic.com/

technology/archive/2022/12/chatgpt-ai-writing-college

-student-essays/672371/

McKenney, S., & Reeves, T. (2018). Conducting

educational design research (2nd ed.) Routledge:

Oxon.

Moser, L., Guggemos, J., & Seufert, S. (2021). Improving

a MOOC to foster information literacy by means of a

conjecture map. International Journal of Learning

Technology, 16(1), 65-86. https://doi.org/10.1504/

IJLT.2021.115470

Rapanta, C., Garcia-Mila, M., & Gilabert, S. (2013). What

is meant by argumentative competence? An integrative

review of methods of analysis and assessment in

education. Review of Educational Research, 83(4), 483-

520. https://doi.org/10.3102/0034654313487606

Rapp, C., & Kauf, P. (2018). Scaling Academic Writing

Instruction: Evaluation of a Scaffolding Tool (Thesis

Writer). International Journal of Artificial Intelligence

in Education, 28, 590–615. https://doi.org/10.1007/

s40593-017-0162-z

Robinson, O., & Wilson, A. (2022). Practicing and

Presenting Social Research. The University of British

Columbia, Open UBC. https://pressbooks.bc

campus.ca/undergradresearch/

Sandoval, W. (2014). Conjecture Mapping: An Approach

to Systematic Educational Design Research, Journal of

the Learning Sciences, 23(1), 18-36.

https://doi.org/10.1080/10508406.2013.778204

Scheuer, O., Loll, F., Pinkwart, N., & McLaren, B. M.

(2010). Computer-supported argumentation: A review

of the state of the art. International Journal of

Computer-supported collaborative learning, 5(1), 43-

102. https://doi.org/10.1007/s11412-009-9080-x

Seufert, S., Burkhard, M., & Handschuh, S. (2021).

Fostering Students' Academic Writing Skills: Feedback

Model for an AI-enabled Support Environment. In T.

Bastiaens (Ed.), Proceedings of Innovate Learning

Summit 2021, 49-58. Online, United States: Association

for the Advancement of Computing in Education

Computer Supported Argumentation Learning: Design of a Learning Scenario in Academic Writing by Means of a Conjecture Map

113

(AACE). https://www.learntechlib.org/primary/p/220

270/

Seufert, S., & Spiroudis, E. (2017). Wissenschaftliche

Schreibkompetenzen in der Studieneingangsphase

fördern: Erfahrungen mit einem Rückmelde- und

Bewertungsraster (Rubric). Zeitschrift Schreiben, 1-9.

Sharples, M. (2022, May 17). New AI tools that can write

student essays require educators to rethink teaching

and assessment. The London School of Economics and

Political Science (LSE). https://blogs.lse.ac.uk/

impactofsocialsciences/2022/05/17/new-ai-tools-that-

can-write-student-essays-require-educators-to-rethink-

teaching-and-assessment/

Stab, C., & Gurevych, I. (2014). Annotating argument

components and relations in persuasive essays.

Proceedings of COLING 2014, the 25th international

conference on computational linguistics: Technical

papers, 1501-1510.

Stegmann, K., Wecker, C., Weinberger, A., & Fischer, F.

(2012). Collaborative argumentation and cognitive

elaboration in a computer-supported collaborative

learning environment. Instructional Science, 40(2),

297-323. https://doi.org/10.1007/s11251-011-9174-5

Strobl, C., Ailhaud, E., Benetos, K., Devitt, A., Kruse, O.,

Proske, A., & Rapp, C. (2019). Digital support for

academic writing: A review of technologies and

pedagogies. Computers & Education, 131, 33–48.

https://doi.org/10.1016/j.compedu.2018.12.005

Toulmin, S. E. (2003). The uses of argument, Updated

Edition. Cambridge University Press.

Turunen, T. P. (2013). Introduction to Scientific Writing

Assistant (SWAN)- Tool for Evaluating the Quality of

Scientific Manuscripts [Master thesis, School of

Computing, University of Eastern Finland]. SWAN.

https://cs.joensuu.fi/swan/publications.html

University of Technology Sydney. (2019, May 11). Using

the AcaWriter platform [Video]. YouTube.

https://www.youtube.com/watch?v=CHNLZrc1YWk

&t=16s

Vedral, J., & Ederer-Fick, E. (2015). Schreibforschung und

Schreibdidaktik: Modelle und Theorien

wissenschaftlichen Schreibens. In Egger, R.,

Wustmann, C., & Karber, A. (Eds.). Forschungs-

geleitete Lehre in einem Massenstudium: Bedingungen

und Möglichkeiten in den Erziehungs- und

Bildungswissenschaften, 217-238. Springer Verlag.

Walton, D. N. (1989). Informal logic: A handbook for

critical argument. Cambridge University Press.

Wasson, B., Hansen, C. & Netteland, G. (2016). Data

literacy and use for learning when using learning

analytics for learners. CEUR Workshop Proceedings,

1596, 38–41. http://hdl.han-dle.net/11250/2429460

Wambsganss, T., Niklaus, C., Cetto, M., Söllner, M.,

Handschuh, S., & Leimeister, J. M. (2020). AL: an

adaptive learning support system for argumentation

skills. Proceedings of the 2020 CHI Conference on

Human Factors in Computing Systems, 1-14.

https://doi.org/10.1145/3313831.3376732

Wood, D., Bruner, J. S., & Ross, G. (1976). The role of

tutoring in problem solving. Journal of Child

Psychology and Psychiatry, and Allied Disciplines,

17(2), 89–100. https://doi.org/10.1111/j.1469-

7610.1976.tb00381.x.

Wozniak, H. (2015). Conjecture mapping to optimize the

educational design research process. Australasian

Journal of Educational Technology, 31(5).

https://doi.org/10.14742/ajet.2505

Zornoza, J. (2020, April 15). Explainable Artificial

Intelligence: Discover one of the biggest trends in

Machine Learning and AI. Towards data science.

https://towardsdatascience.com/explainable-artificial-

intelligence-14944563cc79

CSEDU 2023 - 15th International Conference on Computer Supported Education

114