Towards Tool Support for Team Awareness in Collaborative

Argumentation

Maria Fysaraki

1,2

and Heinrich Hussmann

Department of Psychology, University of Munich (LMU), Leopoldstr. 44, Munich, Germany

Media Informatics Group, University of Munich (LMU), Amalienstr. 17, Munich, Germany

Keywords: Team Awareness, Collaborative Argumentation, Multiple Case Study.

Abstract: Supporting collaborative argumentation in higher education is a challenging task, and raising the awareness

of students on the relational space of their collaboration may be the key to simplifying it. In this paper, we

explore the role of team awareness for enhancing the quality of collaborative argumentation. Ten groups of

master students from Media Informatics (n= 28) participated in a multiple-case study for arguing on ill-

structured problems where they received different team awareness prompts embedded in collaboration scripts

, argued for solving the problems and built argument maps. The initial analysis of the argument maps indicates

that the social awareness script has a moderately higher impact on the quality of argumentation compared to

the behavioural awareness script.

1 INTRODUCTION

Collaborative argumentation is regarded as a highly

effective instructional strategy for higher education

but it is also linked to big challenges for students who

need to take care of the collaboration dynamics in the

group while struggling with learning how to argue

and arguing for learning at the same time (von

Aufschnaiter et al., 2008). Collaboration scripts can

facilitate the argumentation processes (Stegmann et

al., 2007). When combined with task awareness

support (i.e. information on group members’ prior

and current knowledge) scripts can help students

acquire deeper understanding of domain concepts

(Gijlers et al., 2013). A recent stream of studies has

focused on the facilitation of group awareness and

argumentation scripts with tools in online learning

situations (Tsovaltzi et al., 2014). However, little is

known as to how we can raise team awareness in

collocated collaboration through pedagogical scripts

and combine them with tools for enhancing

argumentation in classroom.

In a first -small scale- study we investigate how

different team awareness prompts influence the

quality of collaborative argumentation when

embedded in a collaborative argumentation script.

To meet this need we created two variations of the

same pedagogical face-to-face macro-script

combining argument scaffold elements with different

team awareness prompts. The first script variation

included behavioural awareness prompts for

informing students about their activities in the group

(i.e. reminders for performing participation check,

performance comparisons and coordination checks)

(Janssen et al., 2011). The second one included social

awareness prompts for informing students about the

functioning of the group as perceived by their

collaborators (i.e. reminders for assigning roles,

keeping an open mind and being friendly in the group,

openly evaluating their performance) (Phielix et al.,

2011). In comparing the different awareness oriented

argumentation script variations over four sessions of

75 minutes each for collaborative argumentation we

want to shed light on the effects of different team

awareness prompts for enhancing (a) team awareness

processes and (b) quality of collaborative

argumentation and (c) the relation between different

team awareness processes and the quality of

collaborative argumentation.

At first, we present the background theories for

the design of the script variations and their link to

tools for collaborative argumentation. We then

continue with the multiple case study design and the

analysis of the argumentation maps and of the post-

study feedback survey. We conclude with a

discussion on the initial findings of the first study and

their connection to the upcoming study on a tool for

facilitating collaborative argumentation with a focus

Fysaraki, M. and Hussmann, H.

Towards Tool Support for Team Awareness in Collaborative Argumentation.

In Proceedings of the 8th International Conference on Computer Supported Education (CSEDU 2016) - Volume 1, pages 507-512

ISBN: 978-989-758-179-3

507

on enhancing awareness of collaboration (under

implementation).

2 BACKGROUND

In order to ground this research within the wider

context of technology-enhanced argumentation, we

present in this section some interesting research

findings on supporting (a) team awareness and (b)

scientific reasoning and argumentation which were

taken into account for designing the awareness

oriented argumentation scripts. Lastly, we explain the

rationale behind combing scripts with an argument

mapping tool for ensuring equal support for building

arguments to all participants in the study.

2.1 Team Awareness Prompts

Beers et al., (2007) indicated that supporting

awareness of students’ activities in the relational

space of collaboration results in improvement in the

quality of collaborative processes (e.g.

communication of concepts) in the content space of

collaboration. Team awareness relates to awareness

of expected behaviors in the group- behavioral

awareness (Janssen et al., 2011), and to awareness of

students functioning in the group as perceived by

their collaborators in the relational space of

collaboration– social awareness (Phielix et al., 2011).

On the other hand, task awareness relates to cognitive

information about the knowledge of group members

on the content space of collaboration (Janssen et al.,

2011).

The effects of scripting collaboration and task

raising awareness in collaborative learning settings

have been investigated by Gijlers et al., in 2013. They

experimented with elementary students and provided

them with awareness support and scripting separately.

Afterwards, they compared the students in both

conditions with respect to their ability to facilitate

knowledge construction and discourse quality in a

computer supported collaborative drawing scenario.

Both forms of collaboration support -awareness and

scripting managed to facilitate students’ learning

processes and outcomes.

In this study, we seek to expand the research on

team awareness prompts for collaboration. In our

study, students communicate face to face and the

prompts in the script encourage a desired mode of

communication.

The team awareness prompts are combined here

with thought provoking questions and friendly

reminders. Prompts are meant to trigger the

discussion on the group around not only the

information about the individual learner’s context,

but also on the context of the whole learning team

thus supporting the members’ fruitful interaction in

the group on the relational level of collaboration. By

embedding two different sets of awareness prompts

(behavioural versus social) in the same basic script

for collaborative argumentation and comparing them

in the two conditions we want to shed light on the role

of different team awareness prompts in enhancing

collaborative argumentation.

2.2 Scaffold for Argument Building

Most argumentation scripts rely on Toulmin’s model

(Toulmin, 2003) for argumentation, where the

emphasis is on the identification of structural

elements of single arguments (e.g. claims, rebuttals

and backing, etc.). The Toulmin model was used here

as a basis for designing the argument scaffold for both

variations of awareness oriented scripts. Students

were also familiarized with the basics of the Toulmin

model during a training session prior to our study.

Toulmin’s logic provides a useful framework with

which a student can construct and deconstruct an

argument.

Taking a step further from the Toulmin’s model

for practical reasoning to the direction of promoting

high-quality forms of argumentation we built on the

eight epistemic activities of scientific reasoning and

scientific argumentation in higher education (SRA) as

defined by Fischer et al., in 2014 for designing the

awareness oriented argumentation scripts. These

epistemic activities include; „problem identification,

questioning, hypothesis generation, construction and

redesign of artefacts, evidence generation, evidence

evaluation, drawing conclusions as well as

communicating and scrutinizing scientific reasoning

and its results”(Fischer et al., p. 29). Regarding this

study we focused on enhancing specific SRA skills

for constructing artefacts, drawing conclusions and

communicating reasoning. We support students in

constructing artefacts and drawing conclusions with

the concise and intriguing problem cases and with the

argument scaffold in the scripts. For facilitating the

communication of reasoning processes, we employ

argument mapping techniques and a software tool in

conjunction with the scripts.

2.3 Argument Mapping and Scripts

Argument maps (also known as argument diagrams)

refer to external knowledge representations that can

help students to structure visually their arguments

CSEDU 2016 - 8th International Conference on Computer Supported Education

508

(Scheuer et al., 2013). They allow for increased

clarity and reflection on the strength of one’s

argument by identifying the key components of an

essay or a report. They also contribute towards

making a convincing argument and can be an

effective way to improve general critical thinking

skills. Argument maps can be drawn either on paper

or in sophisticated technological environments and

have some basic theory and visual conventions for

constructing and modifying a map as well as for

communicating its contents efficiently (i.e. arrows

between claims and warrants) (van Gelder, 2013).

Computer-based argument mapping is a rapidly

progressing field in computer supported learning

research has been found to enhance student critical

thinking (Twardy, 2004). Scheuer et al., (2013) have

successfully tested the combination of argument

diagramming and scripted interfaces for having

synergistic effects in promoting high-quality

argumentation. They compared students in an

argument mapping environment with versus without

a script. They found out that scripting could

additionally enhance argumentative quality of the

discussions.

Taking on their approach, we provide structuring

on the epistemic level via the argument scaffolds in

the scripts and the web based argument mapping tool

Rationale® and combine it with different awareness

prompts for supporting forms of social interaction on

the relational level of collaboration. The Rationale ®

argument mapping tool allows for visualizing and

organizing arguments and supporting or objecting

them with logic and evidence. The use of this specific

system for argument mapping is supported by

literature on computer-supported argument

visualization (Davies, 2009). Moreover, it is in line

with the purpose of this small-scale study for creating

awareness oriented argumentation scripts to be tested

on software tools for collaborative argumentation.

3 METHODS

This study employs a multiple case study design (Yin,

2009), in which each of the ten groups of Media

Informatics master students (n = 28, in ten groups of

three or two) is conceptualized as a ‘case’. In a

separate meeting prior to the study students were

trained in the use to the argument mapping tool

Rationale® and practiced arguing for solving ill-

structured problems based on the Toulmin model and

the conventions of the argument mapping tool.

Students’ main task was to argue for and agree on the

best solution to the problem and then transfer their

arguments into a joint argument map using the online

argumentation mapping tool Rationale® while

collaborating (see Figure 1).

Figure 1: Example argument map in Rationale®.

At the beginning of each session, students were

provided with the problem case, the task description

and the theory at hand. The problem cases were built

to match the contents of a masters’ class on

"Multimedia-Based Learning Environments" and

dealt with topics such as constructivist theory and

cognitive load of animations (see Figure 2).

Figure 2: Problem case script card on “animations and

cognitive load”.

Following, students received help for building

their arguments on the map. The argumentation part

of the scripts was divided in two subtasks and was

presented in two cards with thought provoking

questions for triggering the argumentative

collaboration on them. The cards were also enhanced

with additional argument building help in the form of

sentence openers. Furthermore, counter argument

support and support for creating backup of claims and

bringing examples was included in the cards (see

Table 1). Students had twenty minutes for working

uninterrupted on each subtask from the

argumentation script cards.

Towards Tool Support for Team Awareness in Collaborative Argumentation

509

Table 1: A script card with argument scaffold for both

conditions.

Questions: In the context of understanding

„centrifugal force „would you

argue that the animation or the

static picture is better in

reducing cognitive load?

Sentence openers:

One argument against could be:

because:

for example:

Prompts:

Please justify your approach by

means of examples and back it

up with the theory at hand.

Either before or after the students had worked on

the argumentative subtasks, they received cards with

social or behavioural awareness prompts on paper

depending on their condition and they had to discuss

them in the group for five minutes (see Table 2). Half

of the groups argued on ill-structured problems

following the behavioural awareness scripts

(behavioural awareness script condition) and the

other half following a social awareness script (social

awareness script condition). The awareness script

cards included one or two questions for the whole

group and a prompt for discussing them. Some

prompts included also suggestions for coordinating

the group work and friendly reminders for the value

of the prompts. The script cards in the Behavioural

Awareness Script condition prompted students to

perform participation checks and coordination checks

throughout the collaboration, and evaluate the

participation and coordination efforts at the end of

every session. Whereas, the script cards in Social

Awareness Script condition prompted students to

assign roles at the beginning of every session, keep an

Table 2: Examples of cards with team awareness prompts

from both conditions.

Social awareness prompt

card for evaluating

performance.

Behavioural awareness

prompt card for

coordination check.

How would you

evaluate your

performance as

writer, reviser or

controller?

Discuss this in the

group.

Reassign the roles

amongst you if needed.

Is the problem case clear

to all of you? Discuss any

ambiguities in the group.

Create a plan for the next

steps for solving the

problem.

Remember: Achieving a

common understanding

and following your plan

will benefit your

collaboration.

open mind and be friendly throughout the

collaboration, and finally evaluate each other for the

performance of the roles.

The video recordings from the collaborative

argumentation sessions, as well as the argument

maps, produced throughout each session are being

analysed using qualitative methods. Finally, students’

feedback on their experience with the argumentation

and awareness script parts as well as with Rationale®

as collected in post-study survey is analysed

qualitatively.

3.1 Preliminary Analysis of Results

For deciding on the level of collaborative

argumentation (low, medium or high,) we coded the

argument maps (element-wise) with respect to criteria

of formal correctness and evidence sufficiency (see

Table 3). The argument maps were examined against

model solutions for each of the ill-structured

problems and with respect to the conventions of the

argument mapping tool. The coding schemas

included the categories of formal completeness (i.e.

one reason with two co-premises, full declarative

sentences) and evidence sufficiency (i.e. correct and

relevant evidence from text, from personal experience

or other scientific sources) of arguments. In the initial

analysis argument maps from the first session were

compared to the ones from the fourth session (20

maps from 10 groups) and examined for any changes

in the quality of collaborative argumentation in the

two conditions.

The quality of team awareness processes is

currently being examined using content analysis

(Krippendorff, 1989). We code for students’

references to the prompts and their application (e.g.

engagement in mutual performance monitoring)

based on video segments from the five minutes before

and after introducing each of the team awareness

prompts to the groups in both conditions.

Table 3: Changes in the levels of quality of arguments

between the first and the fourth session for collaborative

argumentation.

Formal

completeness

Evidence

Sufficiency

BAS

MediumÆHigh LowÆ High

SAS

LowÆHigh LowÆMedium

Additionally, we have analyzed qualitatively the

answers of the students from an obligatory but

anonymized post-study feedback survey on their

experience with the argument mapping system and

CSEDU 2016 - 8th International Conference on Computer Supported Education

510

the script. With respect to the most and least helpful

awareness prompts in the script of the BAS condition

we observed the following patterns in the answers to

the open end questions. Students expressed

themselves positively about the coordination checks

throughout the collaboration. Creating and following

a plan for the collaboration made them “think about

why we weren't as successful as we wished and

"forced" us to think about how to change it.” The

prompts for running a participation check in the group

received mixed comments. One student stated: “we

did not discuss it much in the group, but it helped me

personally to reflect whether I am rather quiet today”.

Lastly, the prompts for evaluating the participation

and coordination efforts at the end of every session

were not perceived well by the BAS students. e.g. “It

feels wrong to compare yourself to your teammates

while they sit around you”.

Students in the SAS condition gave their own

feedback on the most and least useful awareness

prompts of their script. They commented positively

on the prompt for assigning roles and agreed that “it

gave the collaboration a good structure and everyone

knew what to do or what tasks to push”. However,

they did not refer positively to the prompt for

discussing and evaluating their performance as

writers, revisers or controllers in the group e.g.

“reassigning the roles amongst you, if needed -

because the all participants contributed in the same

way to all of the roles.” Furthermore, students often

stated in their answers that the prompts for keeping an

open mind and being friendly throughout the

collaboration helped them “get different minds

together”.

Regarding the timing of the appearance of the

prompts, students in both conditions referred to it as

“rather disruptive” for the collaboration. The time

assigned for working on the prompts (5 minutes for

each prompt) was found to be “more than enough” in

most cases. Most of students’ statements about the

time assigned for working on the argumentation tasks

were in line with this one: “20 minutes is enough to

sketch out some pros and cons for the argument”. The

students in both conditions were also asked about the

use of the additional argument scaffold provided by

the script. Most students agreed that sentence openers

and the thought provoking questions were helpful to

them but not the counter argument support or the

support for creating backup of claims and bringing

examples.

Finally, most students in both conditions reported

positive tendencies for future use of the Rationale®

system. However, many of them criticized the system

for the fact that “only one person can work with the

mapping tool at a same time. “

4 FINDINGS

The initial analysis of argument maps from the first

and the fourth session for collaboration between the

two conditions showed that groups in both conditions

increased the levels of formal completeness and

evidence sufficiency of their arguments between the

sessions. A closer look to formal completeness levels

of arguments indicates that the social awareness

script condition (SAS) did moderately better in raising

their FC levels from low or medium to high than the

groups in the behavioural awareness script condition

(BAS). However, the patterns change slightly in

favour of the groups in the BAS condition in the case

of evidence sufficiency levels of arguments when

comparing the two conditions with respect to their ES

levels in the first and the fourth session. When

accounting for both the levels of formal completeness

and evidence sufficiency as an indicator of the quality

of collaborative argumentation, we observe that SAS

condition did better in all groups.

The analysis of the post-study feedback survey

showed that students in the BAS condition liked

particularly the prompts for achieving a common

understanding prior to arguing. However, they

reported feelings of uneasiness when it came to

evaluating the participation and coordination efforts

at the end of every session. In the SAS condition,

students found the prompt for assigning roles in the

group of practical use for organizing the group

workflow but they did not like evaluating their

performance openly in the group. Students’ opinions

on the timings of the script were rather positive. The

argument scaffold was only partially used (i.e.

thought provoking questions). Finally, students

acknowledged the benefits of working with an

argument mapping tool but pointed out the technical

difficulties of working with Rationale® on one

computer in group. Currently, we are working on the

content analysis for the quality of team awareness

processes from the videos.

5 CONCLUSIONS

The moderate improvement in the quality of

collaborative argumentation in the social awareness

script condition indicates that information about the

functioning of the group as perceived by their

collaborators could be more helpful for enhancing

Towards Tool Support for Team Awareness in Collaborative Argumentation

511

collocated collaborative argumentation in higher

education than information about activities in the

group. A further analysis of the argument maps from

all sessions across conditions may be needed for

explaining the low levels of evidence sufficiency and

highlight the sophistication of students’ arguments

over time. In this respect, the ongoing content

analysis of the group discussions on the different

team awareness prompts might shed light on the

relation between the team awareness processes and

the quality of collaborative argumentation. The direct

feedback of students on the experience with of the

awareness and argumentation script parts in both

conditions will complement the main analysis.

Our expectation from this small scale qualitative

study is to gain useful insights on the importance of

different kinds of team awareness for improving the

quality of arguments. These insights will be used for

creating a balanced awareness oriented CSCL script

for a collaborative argumentation tool (currently

under implementation) in a follow up intervention

study. The goal of this study is to identify how

information about social and behavioral issues of

collaboration can be effectively transformed to

features of a system for collaborative argumentation

mapping (Rationale®). The system will run on

interactive tablet displays and browser technologies

and will aim at facilitating collocated collaborative

argumentation with argument mapping in higher

education settings.

REFERENCES

Davies, W. M. (2009). Computer-assisted argument

mapping: a rationale approach. Higher Education,

58(6), 799-820.

Fischer, F., Kollar, I., Ufer, S., Sodian, B., Hussmann, H.,

Pekrun, R., ... & Strijbos, J. W. (2014). Scientific

reasoning and argumentation: Advancing an

interdisciplinary research agenda in education.

Frontline Learning Research, 2(3), 28-45.

Gijlers, H., Weinberger, A., van Dijk, A. M., Bollen, L., &

van Joolingen, W. (2013). Collaborative drawing on a

shared digital canvas in elementary science education:

The effects of script and task awareness support.

International journal of computer-supported

collaborative learning, 8(4), 427-453.

Janssen, J., Erkens, G., & Kirschner, P. A. (2011). Group

awareness tools: It’s what you do with it that matters.

Computers in Human Behavior, 27(3), 1046-1058.

Krippendorff, K. (1989). Content analysis. In E. Barnouw,

G. Gerbner, W. Schramm, T. L. Worth, & L. Gross

(Eds.), International encyclopedia of communication

(Vol. 1, pp. 403-407). New York, NY: Oxford

University Press. Retrieved from http://repository.upen

n.edu/asc_papers/226.

Phielix, C., Prins, F. J., Kirschner, P. A., Erkens, G., &

Jaspers, J. (2011). Group awareness of social and

cognitive performance in a CSCL environment: Effects

of a peer feedback and reflection tool. Computers in

Human Behavior, 27, 1087–1102.

Scheuer, O., McLaren, B. M., Weinberger, A., & Niebuhr,

S. (2014). Promoting critical, elaborative discussions

through a collaboration script and argument diagrams.

Instructional Science, 42(2), 127-157.

Stegmann, K., Weinberger, A., & Fischer, F. (2007).

Facilitating argumentative knowledge construction

with computer-supported collaboration scripts.

International journal of computer-supported

collaborative learning, 2(4), 421-447.

Toulmin, S. (2003). The uses of argument. Cambridge

University Press.

Tsovaltzi, D., Puhl, T., Judele, R., & Weinberger, A.

(2014). Group awareness support and argumentation

scripts for individual preparation of arguments in

Facebook. Computers & Education, 76, 108-118.

Twardy, C. (2004). Argument maps improve critical

thinking. Teaching Philosophy, 27(2), 95-116.

van Gelder, T. (2013) Argument mapping. In Pashler, H.

(ed.), Encyclopedia of the Mind. Thousand Oaks, CA:

Sage.

Von Aufschnaiter, C., Erduran, S., Osborne, J., & Simon,

S. (2008). Arguing to learn and learning to argue: Case

studies of how students' argumentation relates to their

scientific knowledge. Journal of Research in Science

Teaching, 45(1), 101-131.

Yin, R. K. (2013). Case study research: Design and

methods. Sage publications.

CSEDU 2016 - 8th International Conference on Computer Supported Education

512