Supporting Educators to Design Collaborative Learning Scenarios
Edmar Welington Oliveira
and Marcos Roberto da Silva Borges
Departament of Computer Science, Federal University of Juiz de Fora, Juiz de Fora, Brazil
TECNUN, University of Navarra, Donostia/San Sebastián, Spain
Keywords: Collaborative Learning, Learning Scenarios, Learning Design.
Abstract: Collaborative learning (CL) processes are not always effective and the inadequate design of CL scenarios is
one of the main causes of its unsuccess. Designing CL scenarios is a complex task, since it involves countless
requirements and constraints that affect learning process and, hence, the learning outcomes. Consequently,
CL scenarios are usually inappropriately structured. This study is particularly interested in the complexities
inherent to the process of designing CL scenarios. Its objective is to propose a framework, composed of a
conceptual metamodel and a computational tool, in order to support/guide educators throughout the process.
The proposed framework was evaluated through a case study with 22 professors of a federal university. The
results showed that with the framework it is possible to expose educators to design parameters in a way that
they can effectively and systematically be specified.
There is an increasing interest in providing learners
with collaborative learning (CL) scenarios in order to
support them acquire and develop their knowledge
and skills (Isotani et al., 2013). However, the simple
fact of putting learners working together does not
ensure effective learning (Weinberger et al., 2009).
stressing the potential of such scenarios show
that the chance for having meaningful and also lasting
learning diminishes considerably when they are not
appropriately designed (Isotani et al., 2010). In fact,
the inadequate design of CL scenarios is one of the
main causes of unsuccessful group learning (Strijbos
et al., 2004; Dillenbourg, 2002).
Designing CL scenarios is a complex task, since
it involves countless requirements and constraints
(King, 2014). Well-designed CL scenarios must be
structured based on learners’ learning characteristics
and needs, and considering the necessity of guidance
learners’ actions and interactions. Moreover, they
must be structured in a way that enables educators to
perform monitoring, analysis and evaluation of the
learning process accurately – mainly considering the
individually. In a broader sense, the difficulty
is to transform all aforementioned issues into
elements that structure a scenario. Therefore, due to
the complexities inherent to the process of designing
CL scenarios, educators do not perform a thorough
planning. The process is particularly challenging for
(but not limited to) novice educators – since, in most
cases, they do not have all necessary knowledge and
experience (Isotani et al., 2013). Consequently, such
scenarios are improperly structured (Barkley, 2014;
Höver and Mühlhäuser, 2014).
In this paper, the research problem refers to the
lack of adequate design of CL scenarios. The goal is
to provide
an infrastructure able to guide and support
educators in the process of designing such scenarios.
A design framework was implemented as a proposed
solution to the presented problem. A case study was
carried out in order to evaluate the framework.
The following sections initially discuss some
approaches to structuring collaborative learning
scenarios. Then, in section 3, a set of design
principles is presented defined in
order to guide
educators when designing CL scenarios. Section 4
presents some results of our previous research.
Section 5 presents the framework, describing its
content and implementation. Section 6 presents the
evaluation of the framework, carried out through the
aforementioned case study. The results are presented
in section 7. Finally, section 8 presents the
conclusions of this study.
Oliveira, E. and Borges, M.
Supporting Educators to Design Collaborative Learning Scenarios.
DOI: 10.5220/0010458700950103
In Proceedings of the 13th International Conference on Computer Supported Education (CSEDU 2021) - Volume 2, pages 95-103
ISBN: 978-989-758-502-9
2021 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
Many approaches have been proposed in order to
support educators in structuring CL scenarios. One of
them refers to the concept of collaborative scripts.
a collaborative
script (CS) corresponds
to a set of instructions related to how members of a
group work should interact, collaborate and solve a
specific problem (O’Donnel and Dansereau, 1992).
So, a CS is a collaborative instructional scenario that
organizes the activities and actions inherent to the
learning process.
Another approach, specifically within the CSCL
community, consists of CSCL patterns (an initiative
particularly aimed at the reuse of artefacts – i.e.,
practices, techniques etc.). Formally, a pattern is a
strategy of organizing information about a common
and its solution, enabling it to be repeatedly
used. CLFP
patterns (Hernández, 2006) are examples
of collaborative learning
patterns. Specifically, they
aim to
document techniques to organize the sequence
of collaborative activities included in CL scenarios.
In line with these patterns, there are the
collaborative learning techniques
(CoLTs). Barkley et
al. (2014) present several of these techniques,
describing them as artefacts that provide instructions
for engaging the learners in CL processes.
Another approach consists of instructional
models, defined as a set of activities (previously
sequenced) to be followed by educational designers for
the design of CL scenarios. Most of the existing models
similar elements, varying in the number and/or
terminology of their phases. ADDIE (Jonassen et al.,
1999) is an example of an instructional design model.
Despite several approaches, all of them have
limitations. Regarding collaborative scripts: (a) there
is no reference model for their specification; and (b)
there is also a
lack of guidance about how educators
should specify the script elements. The limitations of
collaborative patterns are: (a) they have instructions,
activities etc. predefined therefore, imposing a
limitation in relation to
their flexibility (be
according to educators’ needs and intentions), and (b)
there is a considerable number
of patterns (thus,
imposing on educators the need to know several
patterns in order to be able to choose those that best
suit one’s needs). Finally, regarding the design
models, it is observed that (a) they do not include
specific rules, guidelines and
instructions (they only
present general guidelines thus, not defining how
the activities they establish should be carried out).
Collaborative learning flow patterns
complexity of designing CL scenarios,
computational tools capable of supporting it become
essential. Challco et al. (2016) performed an analysis
of some of these tools. Despite the diversity of tools,
there are also limitations in them. In fact, in general,
they (a) do not
provide any guidance to the educators
(i.e. how the design process should be carried out),
they are restricted
models, limiting
their flexibility. Therefore, it is observed that all the
aforementioned approaches have limitations related
to the ability to guide and support educators through
the design
process, as well as
to their flexibility.
Although there are many proposals to the problem of
inadequate design of CL scenarios, all of them have
limitations (mainly
regarding the support provided to
educators when carrying out a design). Due to these
limitations, and considering our interest in providing
educators with proper support/guidance, we directed
efforts in a previous study (Oliveira and Borges, 2019)
towards identifying the (most) relevant elements to the
design process. From these
elements, a set of
design principles was specified
(i.e. recommendations
the purpose of guiding educators throughout the
process), presented in table 1.
Table 1: Design principles.
Considering the learning objective (work), it is
recommended that its specification occurs not only at
the level of the entire work and its phases, but also at
the level of its activities. In fact, when the
are defined for the work activities, learners have the
chance to better manage their achievement. Besides, it
is possible for the educator to carry out assessment and
monitoring processes in a more precise way, and also
to have a clearer perception about the specific purposes
of these activities.
CSEDU 2021 - 13th International Conference on Computer Supported Education
Regarding the learners' learning objectives, it is
recommended that they be specified for the learners
(individually). In this way, enabling the educator to
monitor and assess the learning of each learner in a
more precise way.
Regarding the formation of group, in
order to
avoid inappropriate composition of groups, the
participation of the educator is recommended (with
the educator taking responsibility for the process or
guiding the learners through it).
Regarding the specification of activities, it is
recommended that they be defined with a high level
of granularity (i.e., more specific activities). This not
only facilitates the specification of their parameters
(objectives, deadlines etc.), but also enables the
educators to perform assessment and/or monitoring
processes more accurately.
Regarding the suitability of activities, although
it is important to design the activities according to
of groups or classes (educational
needs, level of knowledge etc.), it is recommended
that they be designed considering individual learners’
characteristics and needs.
Regarding the division of activities among group
members, the active participation of the educator is
recommended in order to avoid that the process
occurs improperly (for
example, learners being
responsible for activities they consider easier)
unequally (i.e., few members being responsible for
most of the tasks).
Concerning the collaborative process, it is also
recommended that educators guide it. This
understanding is based on the premise that learners
working in
groups in a free collaboration
always provide satisfactory results; in general,
they do not have an accurate understanding of the
collaborative process, neither of how to behave and
learn in environments that
About the development monitoring, although
is essential to monitor the development of the work
and its phases, it is recommended that it is carried out
at the activity level (making it possible to obtain more
precise information that is essential to the
management of the process, by both learners and
Regarding the
learning monitoring, it should be
performed through careful observation. Besides, it is
important to previously specify goals, analysis points
etc. In addition, it should be done at the level of the
learners, so that the educator is clear about the
individual evolution of each one throughout the
educational process.
As to the
activities guidance, it is recommended
that educators support/guide learners to accomplish
the activities, since in general they do not carry out
collaborative tasks in an adequate way. It is assumed
that poor learner guidance during a learning process
is significantly less effective than a specifically
designed guidance. Therefore, the educator should
guide learners on the organization/management of
their actions during the development of activities
thereby, supporting them to achieve the learning
It is also recommended that educators provide
support material. Although it is important that the
material assists students in the development of the
work (and its phases), it is recommended to provide
materials that assist students in the development of
each task thus, helping them in a more specific way.
Regarding the assessment of learning, although
it is important that it is carried out at the level of the
groups, as well as based on the final results developed
by them, it is recommended that
it is carried out at the
student level (individually) and throughout the work
3.1 Analysis Framework
In order to enable the analysis of the design of CL
scenarios implemented by educators in an
instructional environment, an analysis framework
based on the aforementioned design principles – was
developed in our previous work. For each design
principle, three alternatives were specified (a, b and
c); for
each one, a valuation score was defined, with
the scores “0” (does not meet the principle), “0,5”
(partially meets the principle), “1” (fully meets the
principle). Table 2 presents a piece of the developed
Table 2: Part of the analysis framework.
Supporting Educators to Design Collaborative Learning Scenarios
The aforementioned framework was used to analyze
the results of an exploratory study, also presented in
our previous work. The objective was to analyze the
adherence of the design carried out by educators to
the 12
design principles. The study observed whether
and how educators of higher education (particularly,
in the computer science domain) design group work
scenarios while teaching undergraduate courses.
Personal and semi-structured interviews were
conducted with 30 professors from the computer
science department of a federal university. The
sample was composed of 27 professors with doctor’s
degree and 3 with master’s degree. Regarding the
teaching experience (in years) in higher education, 6
professors had less than 5 years, 7 had between 5 and
10, 5 had between 11 and 15, 8 had between 16 and
20, 2 had between 21 and 30, and 2 had more than 31.
Of the 30 educators, 22 answered they use group
work practices while teaching an undergraduate
course. Thus, according to their answers to the
interview, a score (0,0,5 or “1) was obtained for
each of the 12 design principles. For each educator,
the scores were added up, obtaining a total score,
which represents the level of adherence of the design
implemented by the educator. In order to assess the
level of adherence, a classification score (presented in
Table 3a) was defined. Table 3b presents the
educators’ classification. The 22 educators are
identified as P1 to P22.
Table 3: Educators’ design adherence.
The results show that, of the 22 educators, 18
implement an insufficient design; 3 implement a
regular design; and 1 educator implements a design
whose adherence level is null. It was also observed
that of a possible maximum score of 12, the best-
evaluated educators achieved only 4.5. The results
expose a set of deficiencies regarding the design of
group work scenarios implemented by the 22
educators. In fact, they do not specify several of the
important elements to the design process. The results
reinforce the necessity of providing educators with
useful and proper support/guidance when designing
CL scenarios, helping them to both understand these
elements and specify them.
study is to guide
and support educators to appropriately design CL
scenarios, the solution consists of a design
framework, which is composed of (i) a design
metamodel and (ii) a computational tool. The first one
refers to a structure that conceptualizes the domain of
CL scenarios design, based on the design principles
mentioned in section 4. The second one consists of a
computational infrastructure to support the
metamodel, making possible its use by the educator
when carrying out a design. Basically, the workflow
consists of specifying the elements of the metamodel,
according to the educator's instructional intentions,
generating a particular CL scenario that expresses
5.1 Design Metamodel
The design metamodel includes a set of basic
concepts – which characterize the basic structure of a
group work –, and a set of concepts based on the
design principles. Figure 1 presents it.
The design metamodel is composed of 14
concepts or macroelements. The basic structure of
a group
work (project) comprises the macroelements:
Project, Section, Practice and Module. According to
the metamodel, a project
is composed of modules
(topics explored in the project) and sections, in which
practices (development of learning activities) are
performed by learners. Practices can be sequenced
and/or related by the educator through an activity
Learners or groups perform practices, and
learners can play specific roles. Learning objects can
be used in order to support the execution of practices.
CSEDU 2021 - 13th International Conference on Computer Supported Education
Figure 1: Design metamodel.
practices or sections. Evaluation activities can be
defined to enable the assessment of practices or
sections. Educators can assist the development of
practices or sections through support activities.
Contingency activities correspond to actions/tasks to
be performed as a result of some specific condition
identified by the educator, through a monitoring or
evaluation activity.
The definition of the aforementioned elements was
based on the design principles. As an example, the
macroelements proposed in order to meet the Learning
Objective (project) principle were: Project, Section and
Learning Activity. All of them comprise parameters
that enable the educator to specify the general
objectives of the work, the objectives of its sections
and the objectives of each learning activity. Figure 2
shows the parameters of the Project macroelement.
Figure 2: Parameters of the project microelement.
Besides the basic
parameters (light gray), Module
and Section macroelements integrate the
macroelement, indicating that a project has modules
and sections.
For the specification of the design metamodel,
XML language was used. It provides a free and
practical strategy for the definition of information, as
well as its
representation in hierarchical structures,
adding semantic levels capable of conferring
significant power of expression. The option for XML
also considered the fact that it is a technology-
independent language – therefore, there is no need of
a specific software to read and interpret XML files. A
fragment of the XML Project macroelement is
presented in figure 3.
Figure 3: XML Fragment of the project microelement.
5.2 Computational Tool
The computational tool aims to provide support for
the use of the design metamodel, guiding the educator
with regard to the analysis and specification of its
elements. Therefore, the development of the tool was
carried out in order to support the educator in
specifying the 12 design principles.
The tool provides a set of functionalities, grouped
in 6 categories: create (create/register elements in a
project), edit (complement and/or modify information
of elements registered in the project), view (visualize
information of elements registered in a project and
relationships between elements), include (class of
actions that enables educators to relate elements; for
instance, include learning object in a section), assign
(relate elements in a project for instance, assign role
to learners) and delete (class of actions that enables
the educator to delete an element of the project or
“disconnect” related elements).
For the development of the tool, object oriented
programming paradigm and PHP programming
language were adopted. Regarding the architecture,
Supporting Educators to Design Collaborative Learning Scenarios
the tool was structured as a monolithic Web
application, using a multilayer architecture. Figure 4
presents a screen (in Portuguese) of the tool related to
the creation of practices.
The evaluation aimed to analyze the CL scenario
design process carried out by educators in their face-
to-face undergraduate courses (in the computer
science domain). The purpose was not to evaluate if
the design supports the educator in conducting the
learning process or if it promotes learners’ learning.
Actually, the goal was to evaluate the perception of
educators regarding the guidance and support
provided by the framework and the adherence of the
design implemented by them to the design principles.
Although it is important to analyze the effectiveness
of the design in supporting educators in conducting
the learning process, as well as fostering the
development of learners’ learning, these are issues to
be addressed in future works.
This study adopted the case study as research
methodology. According to Yin (2015), a case study
consists of an empirical investigation that studies a
contemporary phenomenon in depth and in its real-
world context. The case study is presented as the
preferred method in situations in which it is necessary
to answer questions such as "how"
or "why", when the
researcher has little or no control over the
investigated event and when the focus of the study is
a contemporary phenomenon.
Particularly, our study proposes to investigate
how the design of collaborative learning scenarios
can be carried out to promote its adherence to the
design principles. In addition, given the
characteristics of the event of interest, the researcher
has no control over it. In fact, the study focuses on
investigating real situations of design of CL scenarios
by educators. In this context, considering that design
is a task inherent to the educator, as well
as the
environment in which he/she is inserted, the
researcher has no interference or control over the
studied event.
6.1 Case Study
For this study, the approach of multiple case studies
was adopted, with multiple units of analysis defined
for them. The choice was appropriate since it makes
it possible to carry out a comparative analysis of the
cases, allowing a broad view of the studied
The definition of the cases, as well as of the units
of analysis (or educators), was based on the results
obtained from the exploratory study carried out in our
previous research (Oliveira and Borges, 2019). For
this study, three cases were defined. Case 1 includes
the educators whose designs were not adherent to the
design principles (when compared to the results of
other educators). Case 2 comprises the educators
whose designs were moderately adherent. Case 3
includes the educators whose designs were adherent
to the design principles.
The definition of the units of analysis was based
on the scores (level of adherence) presented in table
3. Three units of analysis were selected for each case.
This number was based on Yin (2015) – according to
in projects of multiple cases, it is necessary
Figure 4: Creation of a practice.
CSEDU 2021 - 13th International Conference on Computer Supported Education
to have at least two individual units in each case. The
selected units are presented in Table 4.
Table 4: Selected units of analysis.
The data collection protocol of the case study
comprises 3 phases. Phase 1 consists of analyzing the
results of the exploratory study (performed in our
previous research). Phase 2 consists of the design of
CL scenarios. Particularly in this phase, the educators
of cases 1, 2 and 3 use the proposed framework to
design CL scenarios. Before this phase 2, the
presented a workshop in order to explain
the case study as well as the computational tool to the
educators. All educators had six weeks to complete
the design. At the end of phase 2, the researcher
performed an analysis of the design carried out by the
educators in order to evaluate its adherence to the
design principles. Phase 3 consists of a personal and
semi-structured interview with each educator. The
purpose of this interview
was to analyze the
perception of the educators in relation to the use of
the proposed framework.
Table 5 presents the comparative results between the
3 units of analysis (P18, P19 and P22) in case 1.
These results show that all educators achieved a
higher level of adherence (total score) when
comparing the scores related to the design without
using the framework and using it. Therefore, the
framework was able to support all educators in
carrying out a CL design more adherent to the design
Table 5: Adherence level comparison – case 1.
Table 6 presents the comparative results between
the 3 units of analysis (P14, P16 and P20) in case 2.
Similarly to case 1, all educators achieved a higher
level of adherence (comparing the scores associated
to the design without using the framework and using
it). Also for this case 2, the framework was able to
support all educators in carrying out a CL design
more adherent to the design principles.
Table 6: Adherence level comparison – case 2.
Table 7 presents the comparative results between
the three units of analysis (P2, P6 and P17) in case 3.
Similarly to cases 1 and 2, all educators achieved a
higher level of adherence. Therefore, the framework
was also able to support all educators in carrying out
a CL design more adherent to the design principles.
Table 7: Adherence level comparison – case 3.
Table 8 presents the comparative results among
the cases 1, 2 and 3.
Table 8: Adherence level comparison – cases 1, 2 and 3.
It is possible to observe, for all cases (1, 2 and 3),
an improvement in terms of adherence to the design
principles – when comparing the scores of educators
without using the framework and using it. For case 1,
the total score (sum of the scores of educators P18,
P18 and P22) increased from 2.5 to 18.5. For case 2,
Supporting Educators to Design Collaborative Learning Scenarios
the score increased from 7 to 19.5. For case 3, the
score increased from 10.5 to 19.5.
These results show that the framework had a
greater impact in case 2 (lower level of adherence)
and less impact in case 3 (greater level of adherence).
It is noted, therefore, that in the cases in which
educators presented more expressive difficulties in
terms of design (without the use of the
the proposed framework presented greater support
capability (to carry out a design more adherent to the
design principles). On the other hand, the lower this
difficulty for educators (analyzing the cases, as a
whole), the lower the support capability provided to
them by the framework.
Table 9 presents the results for two of the many
questions present in the interview carried out in phase
3 of the data collection protocol. Question "a" asked
the educator if the framework provides adequate
support for the design of CL scenarios. Question "b"
asked whether the educator, based on the experience
of using the framework, changed his/her perception
of how to design CL scenarios.
Table 9: Answers to (some) interview questions.
The results show that, for question "a", all
educators answered that the framework provided
moderate to high support and guidance in order to
assist them in specifying the design elements that is,
in designing CL scenarios. Similarly, for question
“b”, all educators answered that the experience of
using the proposed framework was able to change
their view regarding how to design CL scenarios, in a
positive way. For all these educators, the framework
made it possible to specify several design elements
that they did not consider when planning their group
work scenarios.
Careful planning is essential to the effectiveness of
collaborative learning processes. However, the
planning process of CL scenarios is complex;
therefore, they are usually inappropriately and
inefficiently structured, making it difficult for
students to achieve learning objectives.
Previous studies indicate the need to provide
with useful and proper support and
guidance, exposing them to parameters and processes
that should be accounted in the CL scenarios design
process. In this study,
it was investigated how
educators perform designing of CL
teaching an undergraduate course in the computer
science domain. A design infrastructure was
developed and evaluated through a case study with 22
professors of a federal university.
The results showed that the framework was able
to support and guide all educators in carrying out a
collaborative learning design more adherent to a set
of design principles (a set of recommendations with
the purpose of guiding educators throughout the CL
design process).
As future work, it is intended to analyze whether
and how the framework is able to support educators
in conducting the learning process. Moreover, it is
proposed to investigate the impact of the framework
on learners’ learning.
Barkley, E. F., Major, C. H., K., Cross, K. P. (2014)
Collaborative Learning Techniques: A Handbook for
College Faculty. 2
Edition. Jossey-Bass, San
Challco, G. C.; Bittencourt, I. I.; Isotani, S. (2016).
Computer-based systems for automating instructional
design of collaborative learning scenarios: a
systematic literature review. International Journal of
Knowledge and Learning, [S.l.], v. 11, n. 4.
Dillenbourg, P. (2002). Over-scripting CSCL: The risks of
blending collaborative learning with instructional
design. Three worlds of CSCL. Can we support CSCL?
Heerlen: Open University Nederland, 61–91.
Hernández Leo, D. et al. (2006). Collage: a collaborative
learning design editor based on patterns. Educational
Technology & Society, [S.l.], v. 9, n. 1, 58-71.
Höver, K. M., Mühlhäuser, M. (2014). Can We Use S-BPM
for Modeling Collaboration Scripts? Communications
in Computer and Information Science. 174-187.
Isotani, S., Mizoguchi, R., Isotani, S., Capeli, O. M.,
Isotani, N., Albuquerque, A. R., Jaques, P. (2013). A
Semantic Web-based authoring tool to facilitate the
planning of collaborative learning scenarios compliant
with learning theories. Computer & Education, 63,
Isotani, S., Mizoguchi, R., Inaba, A., Ikeda, M. (2010). The
foundations of a theory-aware authoring tool for CSCL
design. Computers & Education, v. 54, n. 4, 809-834.
CSEDU 2021 - 13th International Conference on Computer Supported Education
Jonassen, D. H.; Tessmer, M.; Hannum, W. H. (1999). Task
analysis methods for instructional design. Mahwah:
Lawrence Erlbaum Associates.
King A. (2007). Scripting Collaborative Learning
Processes: A Cognitive Perspective. Fischer F., Kollar
I., Mandl H., Haake J. M. (eds) Scripting Computer-
Supported Collaborative Learning. Computer-
Supported Collaborative Learning, v. 6. Springer,
Boston, MA.
Oliveira, E. W.; Borges, M. R. S. (2019). How Educators
Design Group Learning Scenarios in Higher
Education? 18th International Conference on
Information Technology Based Higher Education and
Training, Magdeburg.
O’Donnel, A. M.; Dansereau. (1992). Scripted cooperation
in student dyads: a method for analyzing and enhancing
academic learning and performance. In: Herts-
Lazarowitz, R.; Miller, N. (Ed.). Interaction in
cooperative groups: the theoretical anatomy of group
learning. Cambridge: Cambridge University Press, 120-
Strijbos, J. W., Martens, R. L., Jochems, W. M. G. (2004).
Designing group based learning: six steps to designing
computer-supported group based learning. Computers
& Education, 42, 403–424.
Weinberger A., Kollar I., Dimitriadis Y., Mäkitalo-Siegl,
K., Fischer F. (2009). Computer-Supported
Collaboration Scripts. Balacheff N., Ludvigsen S., de
Jong T., Lazonder A., Barnes S. (eds) Technology-
Enhanced Learning. Springer, Dordrecht.
Yin, R. K. (2015). Case study research: design and
methods. 5
Supporting Educators to Design Collaborative Learning Scenarios