Analysing the Use of Worked Examples and Tutored and Untutored

Problem-Solving in a Dispositional Learning Analytics Context

Dirk T. Tempelaar

, Bart Rienties

and Quan Nguyen

Maastricht University, School of Business and Economics, PO Box 616, 6200 MD Maastricht, The Netherlands

Open University U.K., Institute of Educational Technology, Walton Hal, Milton Keynes, MK7 6AA, U.K.

Keywords: Blended Learning, Dispositional Learning Analytics, Learning Strategies, Multi-modal Data, Prediction

Models, Tutored Problem-Solving, Untutored Problem-Solving, Worked Examples.

Abstract: The identification of students’ learning strategies by using multi-modal data that combine trace data with

self-report data is the prime aim of this study. Our context is an application of dispositional learning

analytics in a large introductory course mathematics and statistics, based on blended learning. Building on

previous studies in which we found marked differences in how students use worked examples as a learning

strategy, we compare different profiles of learning strategies on learning dispositions and learning outcome.

Our results cast a new light on the issue of efficiency of learning by worked examples, tutored and untutored

problem-solving: in contexts where students can apply their own preferred learning strategy, we find that

learning strategies depend on learning dispositions. As a result, learning dispositions will have a

confounding effect when studying the efficiency of worked examples as a learning strategy in an

ecologically valid context.

1 INTRODUCTION

For many decades, research into student learning

tactics and strategies has primarily relied on self-

reports or think-aloud protocols, open to the bias

often present in self-reported perceptions, or

excluding naturalistic contexts from the analysis

(Azevedo, Harley, Trevors, Duffy, Feyzi-Behnagh,

Bouchet, et al., 2013; Gašević, Jovanović, Pardo, &

Dawson, 2017; Gašević, Mirriahi, Dawson, &

Joksimović, 2017). The increasing use of blended

learning and other forms of technology-enhanced

education gave way to measure revealed learning

strategies by collecting traces of students’ learning

behaviours in the digital learning platforms. This

new opportunity of combining trace data with self-

report data has boosted empirical research in

learning tactics and strategies. Examples of such are

Azevedo et al. (2013), and research by Gašević and

co-authors (Gašević, Jovanović, et al. 2017;

Gašević, Mirriahi, et al., 2017).

This type of research aims to investigate

relationships between learning strategies measured

by trace data, learning approaches measured by self-

reports, and academic performance as learning

outcomes. For instance, Gašević, Jovanović et al.

(2017) finds that learning strategies are related to

deep learning approaches, but not to surface learning

approaches. In the experimental study Gašević,

Mirriahi, et al. (2017), the role of instructional

conditions and prior experience with technology-

enhanced education is investigated. However, most

of these studies do not take individual differences

into account, as expressed in Gašević, Mirriahi, et al.

(2017, p. 216): ‘Future studies should also account

for the effects of individual differences -e.g.,

motivation to use technology, self-efﬁcacy about the

subject matter and/or technology, achievement goal

orientation, approaches to learning, and

metacognitive awareness’.

Our paper aims to contribute to this lack of

empirical work incorporating individual differences,

by addressing students’ learning strategies within a

dispositional learning analytics context. The

Dispositional Learning Analytics (DLA)

infrastructure, introduced by Buckingham Shum and

Crick (2012), combines learning data (generated in

learning activities through technology-enhanced

systems) with learner data (student dispositions,

values, and attitudes measured through self-report

surveys). Learning dispositions represent individual

difference characteristics that impact all learning

294

Tempelaar, D., Rienties, B. and Nguyen, Q.

Analysing the Use of Worked Examples and Tutored and Untutored Problem-Solving in a Dispositional Learning Analytics Context.

DOI: 10.5220/0006760202940301

In Proceedings of the 10th International Conference on Computer Supported Education (CSEDU 2018), pages 294-301

ISBN: 978-989-758-291-2

processes and include affective, behavioural and

cognitive facets (Rienties, Cross, & Zdrahal, 2017).

Student’s preferred learning approaches are

examples of such dispositions of both cognitive and

behavioural type.

The current study builds on our previous DLA-

based research (Nguyen, Tempelaar, Rienties, &

Giesbers, 2016; Tempelaar, Cuypers, Van de Vrie,

Heck, & Van der Kooij, 2013; Tempelaar,

Mittelmeier, Rienties, & Nguyen, 2017; Tempelaar,

Rienties, & Giesbers, 2015; Tempelaar, Rienties, &

Nguyen, 2017). One of our empirical findings in

these studies was that traces of student learning in

digital platforms show marked differences in the use

of worked examples (Nguyen et al., 2016;

Tempelaar, Mittelmeier, et al., 2017; Tempelaar,

Rienties, & Nguyen, 2017). The merits of the

worked examples principle Renkl (2014) in the

initial acquisition of cognitive skills are well

documented. The use of worked solutions in multi-

media based learning environments stimulates

gaining deep understanding (Renkl, 2014). When

compared to the use of erroneous examples, tutored

problem-solving, and problem-solving in computer-

based environments, the use of worked examples

may be more efficient as it reaches similar learning

outcomes in less time and with less learning efforts.

The mechanism responsible for this outcome is

disclosed in Renkl (2014, p. 400): ‘examples relieve

learners of problem-solving that – in initial cognitive

skill acquisition when learners still lack

understanding – is typically slow, error-prone, and

driven by superficial strategies. When beginning

learners solve problems, the corresponding demands

may burden working memory capacities or even

overload them, which strengthens learners’ surface

orientation. … When learning from examples,

learners have enough working memory capacity for

self-explaining and comparing examples by which

abstract principles can be considered, and those

principles are then related to concrete exemplars. In

this way, learners gain an understanding of how to

apply principles in problem-solving and how to

relate problem cases to underlying principles’.

Following research by McLaren and co-authors

(McLaren, van Gog, Ganoe, Karabinos, & Yaron,

2016; McLaren, van Gog, Ganoe, Yaron, &

Karabinos, 2014), we extend the range of preferred

learning strategies taken into account to include,

beyond worked-examples, the tutored and untutored

problem-solving strategies. In the tutored problem-

solving strategy, students receive feedback in the

form of hints and an evaluation of provided answers,

both during and at the end of the problem-solving

steps. In untutored problem-solving, feedback is

restricted to the evaluation of provided answers at

the end of the problem-solving steps (McLaren et

al., 2014, 2016).

Evidence for the worked examples principle is

typically based on laboratory-based experimental

studies, in which the effectiveness of different

instructional designs is compared (Renkl, 2014).

McLaren and co-authors take the research into the

effectiveness of several learning strategies a step

into the direction of ecological validity, by choosing

for an experimental design in a classroom context,

assigning the alternative learning approaches

worked-examples, tutored and untutored problem-

solving, and erroneous examples as the conditions of

the experiment (McLaren et al., 2014, 2016). In our

research, we increase ecological validity one more

step by offering a digital learning environment that

encompasses all learning strategies of worked-

examples, tutored and untutored problem-solving,

and observing the revealed preference of the

students in terms of learning strategy they apply. In

this naturalistic context, the potential contribution of

LA-based investigations is that we can observe

students’ revealed preferences for a specific learning

strategy, how these preferences depend on the

learning task at hand, and how these preferences link

to other observations, such an individual difference

characteristics. By doing so, we aim to derive a

characterization of students who actively apply

worked examples or tutored problem-solving, and

those not doing so. In line with contemporary

research into learning strategies applying trace data

(Gašević, Jovanović, et al. 2017; Gašević, Mirriahi,

et al., 2017), we adopt two research questions: 1)

how does the choice for learning strategy relate to

learning dispositions? and 2) how does the learning

strategy of using worked examples or tutored

problem-solving relate to learning outcomes?

2 METHODS

2.1 Context of the Empirical Study

This study takes place in a large-scale introductory

mathematics and statistics course for first-year

undergraduate students in a business and economics

programme in the Netherlands. The educational

system is best described as ‘blended’ or ‘hybrid’.

The main component is face-to-face: Problem-Based

Learning (PBL), in small groups (14 students),

coached by a content expert tutor. Participation in

tutorial groups is required. Optional is the online

Analysing the Use of Worked Examples and Tutored and Untutored Problem-Solving in a Dispositional Learning Analytics Context

295

component of the blend: the use of the two e-

tutorials SOWISO (https://sowiso.nl/) and

MyStatLab (MSL) (Nguyen et al., 2016; Tempelaar

et al., 2013; Tempelaar et al., 2015; Tempelaar,

Rienties, et al., 2017). This design is based on the

philosophy of student-centred education, placing the

responsibility for making educational choices

primarily on the student. Since most of the learning

takes place during self-study outside class through

the e-tutorials or other learning materials, class time

is used to discuss solving advanced problems. Thus,

the instructional format shares most characteristics

of the flipped-classroom design. Using and

achieving good scores in the e-tutorial practice

modes is incentivized by providing bonus points for

good performance in quizzes that are taken every

two weeks and consist of items that are drawn from

the same item pools applied in the practising mode.

This approach was chosen to encourage students

with limited prior knowledge to make intensive use

of the e-tutorials.

The subject of this study is the full 2016/2017

cohort of students (1093 students). A large diversity

of the student population was present: only 19%

were educated in the Dutch high school system,

against 81% being international students, with 50

nationalities present. A large share of students was

of European nationality, with only 3.9% of students

from outside Europe. High school systems in Europe

differ strongly, most particularly in the teaching of

mathematics and statistics. Therefore, it is crucial

that this introductory module is flexible and allows

for individual learning paths. Students spend on

average 24 hours in SOWISO and 32 hours in MSL,

which is 30% to 40% of the available time of 80

hours for learning both topics.

2.2 Instruments and Procedure

Both e-tutorial systems SOWISO and MSL follow a

test-directed learning and practising approach. Each

step in the learning process is initiated by a question,

and students are encouraged to (attempt to) answer

each question. If a student does not master a

question (completely), she/he can either ask for hints

to solve the problem step-by-step, or ask for a fully

worked example. After receiving feedback, a new

version of the problem loads (parameter based) to

allow the student to demonstrate his/her newly

acquired mastery. Students’ revealed preferences for

learning strategies are related to their learning

dispositions, as we demonstrated in previous

research (Nguyen et al., 2016; Tempelaar et al.,

2017, 2017b) for the use of worked-examples in

SOWISO), and the use of worked-examples in MSL

(Tempelaar, 2017). This study extends Nguyen et al.

(2016) and Tempelaar et al. (2017, 2017b) by

investigating three learning strategies in the

SOWISO tool: worked examples, and supported and

tutored problem-solving.

Figure 1 demonstrates the implementation of the

alternative learning strategies students can opt for a

sample exercise:

• Check: the untutored problem-solving

approach, offering only correctness

feedback after problem-solving;

• Hint: the tutored problem-solving approach,

offering feedback and hints to assist the

student in the several problem-solving

steps;

• Solution: the worked-examples approach;

• Theory: asking for a short explanation of

the mathematical principle.

Figure 1: Sample of SOWISO exercise with the options

Check, Theory, Solution, and Hint.

Our study combines trace data of the SOWISO e-

tutorial with self-report survey data measuring

learning dispositions. Trace data is both of product

and process type (Azevedo et al., 2013). SOWISO

reporting options of trace data are very broad,

requiring making selections from the data. First, all

dynamic trace data were aggregated over time, to

arrive at static, full course period accounts of trace

data. Second, from the large array of trace variables,

a selection was made by focusing on process

variables most strongly connected to alternative

CSEDU 2018 - 10th International Conference on Computer Supported Education

296

Table 1: Descriptive statistics of four times four sub-samples of students achieving at least 70% mastery level.

Group N Mastery #Attempts #Hints #Examples

HintsQ1&ExamplesQ1 72 96.2% 340 4.3 38.7

HintsQ1&ExamplesQ2 41 97.8% 418 4.8 88.7

HintsQ1&ExamplesQ3 59 98.2% 534 4.2 139.9

HintsQ1&ExamplesQ4 59 97.7% 729 4.5 157.7

HintsQ2&ExamplesQ1 64 97.5% 367 20.8 41.9

HintsQ2&ExamplesQ2 55 99.2% 432 21.8 85.3

HintsQ2&ExamplesQ3 39 99.7% 520 21.0 134.5

HintsQ2&ExamplesQ4 45 98.8% 758 20.3 269.9

HintsQ3&ExamplesQ1 53 97.0% 370 57.4 48.5

HintsQ3&ExamplesQ2 53 98.8% 433 53.8 88.6

HintsQ3&ExamplesQ3 56 99.3% 528 56.7 136.1

HintsQ3&ExamplesQ4 52 99.5% 786 59.5 275.4

HintsQ4&ExamplesQ1 33 99.2% 486 135.8 49.1

HintsQ4&ExamplesQ2 60 98.4% 476 137.9 89.7

HintsQ4&ExamplesQ3 61 99.3% 587 157.2 140.8

HintsQ4&ExamplesQ4 58 99.4% 764 174.9 249.1

Total 860 98.4% 530 58.1 135.5

learning strategies. In total, five trace variables

were selected:

• Mastery in the tool, the proportion of

exercises successfully solved as product

indicator;

• #Attempts: total number of attempts of

individual exercises;

• #Hints: the total number of Hints called for.

• #Examples: total number of worked-out

examples called;

To disentangle the effects of learning intensity

from learning strategy, we restricted the sample to

those students who have been very active in the e-

tutorial and achieved at least a 70% mastery level

(that is, successfully solved at least 162 of the 231

exercises): 860 of the 1080 students. Rather than

applying advanced statistical techniques to create

different profiles of using worked examples, as in

Gašević and coauthors (Gašević, Jovanović, et al.

2017; Gašević, Mirriahi, et al., 2017) or our previous

research (Nguyen et al., 2016; Tempelaar, et al.,

2017, 2017a), we applied quartile splits: the selected

students were split into four equal-sized groups

according to intensity of using worked examples, as

well the intensity of using hints. Table 1 provides

descriptive statistics of these four times four sub-

samples.

The operationalization of revealed preferences

for learning strategies follow these quartile splits.

The revealed preference for the worked-examples

strategy is operationalized as students calling for a

lot of examples, ending up in the higher quarters of

the quartile-split for examples. The revealed

preference for the tutored problem-solving strategy

is operationalized as calling for a large number of

hints, thus ending in the higher quarters of the

quartile-split for hints. As is clear from Table 1,

revealed preferences for learning strategies are not

disjunct. A student can combine the strategies of

worked-examples and tutored problem-solving,

calling for an above-average number of hints as well

as above average number of examples.

The strategy of untutored problem-solving is a

necessary component of any of the revealed

preferences, since students can only build mastery

through untutored problem-solving, and the students

included in this analysis all obtained high mastery

levels.

Mastery level is indeed invariant over groups,

and never below 96%: there is a large majority of

students in all sub-samples that reached full mastery.

There is considerable variation in the use of hints,

and the use of examples. The use of hints and

examples seems only weakly associated, except for

the quarter of students using most hints: HintsQ4. In

that quarter, the use of hints and examples is

positively correlated (in HintsQ4, the correlation of

hints and examples equals 0.23, against 0.07 in all

four quarters).

In this study, we will focus on a selection with

regard to the self-report surveys measuring student

learning dispositions. More than a dozen were

administered, ranging from affective learning

emotions to cognitive learning processing strategies:

• Epistemological self-theories of intelligence;

• Epistemological views on role effort in

learning;

• Epistemic learning emotions;

Analysing the Use of Worked Examples and Tutored and Untutored Problem-Solving in a Dispositional Learning Analytics Context

297

• Cognitive learning processing strategies;

• Metacognitive learning regulation strategies;

• Subject-specific (math & stats) learning

attitudes;

• Academic motivations;

• Achievement goals;

• Achievement orientations;

• Learning activity emotions;

• Motivation & Engagement wheel;

• Cultural intelligence;

• National cultural values; and

• Help-seeking behaviour

Due to lack of space, we refer to previous

research for the description of the self-report

instruments measuring learning dispositions

(Nguyen et al., 2016; Tempelaar, Mittelmeier, et al.,

2017; Tempelaar et al., 2015; Tempelaar, Rienties,

et al., 2017). The description of the research

outcomes will focus on specific aspects of learning

dispositions: learning approaches, anxiety and

uncertainty as aspects of students’ attitudes and

learning emotions.

Course performance data is based on the final

written exam, as well as the three intermediate

quizzes. Quiz scores are averaged, and both exam

and quiz are decomposed into two topic scores,

resulting in MathExam, StatsExam, MathQuiz and

StatsQuiz.

3 RESULTS

3.1 Previous Research

In previous research (Nguyen, 2016; Tempelaar,

2017; Tempelaar et al., 2017a, 2017b), we

investigated the role of worked examples in LA

applications and found that a range of dispositions

predict the use of worked examples as a learning

strategy. Demographic variables, student learning

approaches, learning attitudes and learning emotions

influenced the use of worked examples, with effect

sizes up to 7% for individual dispositions. In our

profiling study (Tempelaar et al., 2017) we found

that the use of worked examples and the total

number of attempts to be the two variables shaping

most of the characteristic differences between

different profiles in the use of the e-tutorial. The use

of hints did not strongly contribute to the creation of

the student use profiles. As a consequence, we

expect dispositions to play a less strong role in the

explanation of the use of hints as a learning strategy

than it has in the explanation of the use of worked

examples. This expectation does indeed come true,

and in the reporting of the empirical outcomes in the

next sections, we will focus on the cases where

dispositions matter in the explanation of both

learning strategies, leaving out the cases where the

impact is primarily on the use of worked examples,

that are described in previous research (Nguyen,

2016; Tempelaar, 2017; Tempelaar et al., 2017,

2017a, 2017b).

3.2 Demographics

Demographic variables have no practical

significance in the explanation of the use of hints:

gender and international status have statistically

non-significant relationships with the intensity of

use of hints. Math prior education has a marginally

significant effect with limited size (p-value=.04, eta

squared=1.2%). Differences in national cultural

values follow this pattern, with the single exception

that students from cultures that assign greater value

to long-term orientation tend to apply the learning

strategy of using hints more often than students from

other cultures (p-value=.004, eta squared=1.2%).

3.3 Learning Approaches

Although the use of learning strategies is the explicit

focus of learning approaches frameworks, the

learning strategy of supported problem solving by

using hints was not adequately captured in our

learning approaches instrument. Hence, we found no

differences in learning approaches for the use of

hints. In the use of worked examples, there were

significant differences for both the deep and

stepwise processing strategies, and self-regulation of

learning.

3.4 Prior Knowledge

Differences induced by different levels of prior math

schooling are enlarged in the first measurement of

cognitive type: the math entry test, administered at

the very start of the course. The score of diagnostic

test was strongly associated with the intensity of

using hints, and the use of worked examples.

Significance levels are .006, <.001 and .018 for the

hints effect, the examples effect, and the interaction

effect, respectively, with a total effect size of eta

squared=11.9%. Figure 2 provides a graphical

illustration of the effects in the several quarters,

where we applied reversed scaling to the several

quarters to facilitate readability. Students with the

highest prior knowledge levels tend to use fewer

CSEDU 2018 - 10th International Conference on Computer Supported Education

298

hints and fewer examples than the other students.

However, there was more consistency in the pattern

for the use of examples, than that for the use of

hints: in the quarter of students with the highest

intensity of using examples, both the Q1 and Q4

quarters of hint use demonstrate low levels of prior

knowledge.

Figure 2: Quarter differences for prior math knowledge, as

measured by a diagnostic test (reversed scaling).

3.5 Learning Attitudes

Since learning attitudes as Affect and Cognitive

Competence are associated with levels of prior

knowledge, it is to be expected that the intensity of

use of both learning strategies is associated with

learning attitudes. That was indeed the case: Affect

(p-value hints<.001, p-value examples<.001, total

eta squared=8.9%), Cognitive competence (p-value

hints<.001, p-value examples<.001, total eta

squared=8.8%) demonstrated clear linear effects in

the absence of interaction effects. Value and Interest

had no role in explaining difference in strategy use,

whereas the NoDifficulty variable was only weakly

associated with both strategies (p-value hints=.034,

p-value examples=.008, total eta squared=2.9%),

and the Effort variable is associated with only the

examples strategy (p-value hints=.461, p-value

examples=.002, total eta squared=3.5%). Figure 3

provides a graphical presentation for the case of

Affect. As in the previous figure, we see that the

highest levels of Affect are to be found in the group

of students who use both hints and examples least

frequently and that intensive use of both strategies is

associated with low levels of Affect.

Figure 3: Quarter differences for learning attitude Affect

(reversed scaling).

3.6 Epistemic Emotions

Epistemic emotions demonstrated group differences

for the negative emotions Confusion (p-value

hints=.004, p-value examples<.001, total eta

squared=7.0%) and Frustration (p-value hints<.001,

p-value examples<.001, total eta squared=6.4%).

Frustration was one of the few disposition variables

that was associated with the use of hints (partial eta-

squared=2.7%) more than with the use of examples

(partial eta-squared=2.4%). Epistemic Enjoyment

makes an even stronger case: here the only

significant relationship is with the use of hints (p-

value hints=.001, p-value examples=.083, total eta

squared=4.8%). Figure 4 demonstrates the

association for Epistemic Frustration, Figure 5 that

for Epistemic Enjoyment.

Figure 4: Quarter differences for Epistemic Frustration

(non-reversed scaling).

Analysing the Use of Worked Examples and Tutored and Untutored Problem-Solving in a Dispositional Learning Analytics Context

299

Figure 5: Quarter differences for Epistemic Enjoyment

(reversed scaling).

3.7 Learning Outcomes

In the main outcome variable of the learning

process, Math Exam or the achievement in the math

section of the final written exam, only associations

with the learning strategy of using examples can be

found, be it that the interaction term is significant (p-

value hints=.106, p-value examples<.001, p-value

interaction=.013, total eta squared=10.9%). Figure 6

provides a graphical description: math exam score is

generally increasing for less intensive use of

examples, but the pattern is not identical for all

quarters of hint use intensity. Specifically, in

students in the third quarter of hint use intensity, the

use of examples and performance seem fairly

unrelated.

Figure 6: Quarter differences for Math Exam score

(reversed scaling).

4 DISCUSSION AND

CONCLUSION

Existing studies into the efficiency of alternative

learning strategies, both in lab settings (Renkl, 2014)

and in classroom settings (McLaren et al., 2014,

2016), point in the direction of worked-examples

being superior to tutored and untutored problem-

solving. These are generic conclusions, which do not

differentiate between types of academic tasks or

types of students. The main contribution of this

study was the emphasis on the individual

differences: learning dispositions make a difference,

academic tasks make a difference. Allowing for

individual differences and task differences also

changes the first order conclusions.

Regarding the first research question, we found

that students who had less prior knowledge sought

more support from both worked-examples and hints.

Similarly: students who experienced more negative

epistemic emotions such as confusion and

frustration, examples of mal-adaptive dispositions,

sought more support from both worked-examples

and hints. Students who scored higher in the prior

knowledge test usually took on the task by

themselves without seeking help from hints or

examples. At the same time, students who used

fewer hints and worked examples scored higher on

the math exam (second research question). This

implies that worked-examples are only superior to

tutored and untutored problem-solving when the

latter two learning strategies are not sufficient to

achieve proficiency. The initial acquisition of

complex knowledge is an example of such a context.

In cases the cognitive challenges of the learning

tasks are less, this superiority may break down, and

worked-examples may be less efficient learning

strategies than problem-solving approaches.

Transferring the findings of the Renkl (2014) and

the McLaren et al. (2014, 2016) studies to our

context, suggests that the superiority of the worked-

examples strategy may be the result of the tasks

offered to the participants of these studies to be of

such type that students in their studies had little or

no prior knowledge. Our context has been different:

given the wide variety of the tasks and the large

diversity in prior knowledge of students, there exists

a wide range of relevant prior knowledge levels for

any task at hand. In such a context, where students

are expected to demonstrate mastery, mastery that

can only be acquired in the untutored problem-

solving mode, the use of examples and hints is

inevitably a roundabout route, adding inefficiency to

the most direct way to mastery. That route of using

CSEDU 2018 - 10th International Conference on Computer Supported Education

300

tutored problem-solving and worked-examples is

taken by students who assessed the direct way of

untutored problem-solving to be -still- impassable,

explaining the relationship with prior knowledge.

The current study has focussed on individual

differences between students in their preference for

learning strategies, and the relationship with learning

dispositions. In future research, we intend to

additionally include the task dimension, by

investigating student preference for learning

strategies as a function of both individual differences

in learning dispositions and task characteristics.

REFERENCES

Azevedo, R., Harley, J., Trevors, G., Duffy, M., Feyzi-

Behnagh, R., Bouchet, F., et al. (2013). Using trace

data to examine the complex roles of cognitive,

metacognitive, and emotional self-regulatory

processes during learning with multi-agents systems.

In R. Azevedo & V. Aleven (Eds.), International

handbook of metacognition and learning technologies,

427–449. Amsterdam, The Netherlands: Springer.

Buckingham Shum, S. & Deakin Crick, R. (2012).

Learning Dispositions and Transferable Competencies:

Pedagogy, Modelling and Learning Analytics. In S.

Buckingham Shum, D. Gasevic, & R. Ferguson (Eds.).

Proceedings of the 2nd International Conference on

Learning Analytics and Knowledge, 92-101. ACM,

New York, NY, USA. DOI:

10.1145/2330601.2330629.

Gašević, D., Jovanović, J., Pardo, A., & Dawson, S.

(2017). Detecting learning strategies with analytics:

Links with self-reported measures and academic

performance. In Journal of Learning Analytics, 4(1),

113–128 DOI: 10.18608/jla.2017.42.10.

Gašević, D., Mirriahi, N., Dawson, S., & Joksimović, S.

(2017). Effects of instructional conditions and

experience on the adoption of a learning tool

Computers in Human Behavior, 67, 207–220. DOI:

10.1016/j.chb.2016.10.026.

McLaren, B. M., van Gog, T., Ganoe, C., Karabinos, M.,

& Yaron, D. (2016). The efficiency of worked

examples compared to erroneous examples, tutored

problem solving, and problem solving in classroom

experiments. Computers in Human Behavior, 55, 87-

99. DOI: 10.1016/j.chb.2015.08.038.

McLaren, B.M., van Gog, T., Ganoe, C., Yaron, D. &

Karabinos, M. (2014) Exploring the assistance

dilemma: Comparing instructional support in

examples and problems. In S. Trausan-Matu et al.

(Eds.) Proceedings of the Twelfth International

Conference on Intelligent Tutoring Systems (ITS-

2014). LNCS 8474. (pp. 354-361). Springer

International Publishing Switzerland.

Nguyen, Q., Tempelaar, D.T., Rienties, B., & Giesbers, B.

(2016). What learning analytics based prediction

models tell us about feedback preferences of students.

In Amirault, R., & Visser, Y., (Eds.). (2016). e-

Learners and Their Data, Part 1: Conceptual,

Research, and Exploratory Perspectives. Quarterly

Review of Distance Education. 17(3).

Renkl, A. (2014). The Worked Examples Principle in

Multimedia Learning. In R. E. Mayer (Ed.), The

Cambridge Handbook of Multimedia Learning, 391-

412. Cambridge, UK: Cambridge University Press.

Rienties, B., Cross, S., & Zdrahal, Z (2017). Implementing

a Learning Analytics Intervention and Evaluation

Framework: What Works? In B. Kei Daniel (Ed.), Big

data and learning analytics: Current theory and

practice in higher education, 147–166. Cham:

Springer International Publishing. DOI: 10.1007/978-

3-319-06520-5_10.

Tempelaar, D. T., Cuypers, H., Van de Vrie, E., Heck, A.,

Van der Kooij, H. (2013). Formative Assessment and

Learning Analytics. In D. Suthers & K. Verbert (Eds.),

Proceedings of the 3rd International Conference on

Learning Analytics and Knowledge, 205-209. New

York: ACM. DOI: 10.1145/2460296.2460337.

Tempelaar, D. (2017). How dispositional Learning

Analytics helps understanding the worked-example

principle. In D. G. Sampson, J. M. Spector, D.

Ifenthaler, & P. Isaias (Eds.), Proceedings 14th

International Conference on Cognition and

Exploratory Learning in Digital Age (CELDA 2017),

pp. 117-124. International Association for

Development of the Information Society, IADIS Press.

Tempelaar, D.T., Mittelmeier, J., Rienties, B., & Nguyen,

Q. (2017). Student profiling in a dispositional learning

analytics application using formative assessment,

Computers in Human Behavior (in press). DOI:

10.1016/j.chb.2017.08.010.

Tempelaar, D. T., Rienties, B., & Giesbers, B. (2015). In

search for the most informative data for feedback

generation: Learning Analytics in a data-rich context.

Computers in Human Behavior, 47, 157-167. DOI:

10.1016/j.chb.2014.05.038.

Tempelaar, D.T., Rienties, B., & Nguyen, Q. (2017a).

Towards actionable learning analytics using

dispositions. IEEE Transactions on Education, 10(1),

6-16. DOI:10.1109/TLT.2017.2662679.

Tempelaar, D.T., Rienties, B., & Nguyen, Q. (2017b).

Adding dispositions to create pedagogy-based

Learning Analytics. Zeitschrift für

Hochschulentwicklung, ZFHE, 12(1), 15-35.

Analysing the Use of Worked Examples and Tutored and Untutored Problem-Solving in a Dispositional Learning Analytics Context

301