Designing a Mediation Vocabulary for Authoring Learning Analytics

Patricia Charlton, Sokratis Karkalas and Manolis Mavrikis

London Knowledge Lab, Institute of Education, University College London, London, U.K.

Keywords: Education, Designing a Mediation Vocabulary, Knowledge Representation, Feedback, Learning Analytics,

Knowledge Integration.

Abstract: This paper provides a knowledge representation process for authoring of learning experiences that capture

feedback designed in the context of learning environments. The paper reports on a year long study with

designers who are creating mathematical teaching and learning resources as part of an EU project (M C

Squared). In this paper we examine the knowledge representation process we used in design and creation of

a mediation vocabulary. The model to be designed has to provide different layers of ‘knowledge integration’

and thus offers insights into the importance of knowledge mediation in the emergence of new learning

environments and experiences. Hence, authoring of designs and feedback through use of ontologies to form

part of the annotating of the learning activities. The annotations form part of the context to be used as part of

the learning analytics.

1 INTRODUCTION

The M C Squared EU (http://www.mc2-project.eu)

project is researching into and creating digital

teaching and learning resources for secondary school

mathematics. The core focus of the project is to

investigate and evaluate social creativity and

creative mathematical thinking (Bokhove et al.,

2014).

Part of the objective of the project is to support

authoring of the activities by the designers and

teachers. Authoring learning activities is not new

and there are many tools and attempts to support

this. However, there are number of problems with

authoring systems (a) they only work for very

specific tasks, (b) the tools often require

considerable technical knowledge, such as the

teacher/designer needing to program complex rules

and (c) they burden the teacher with extra work load

that seldom provides the desired insights.

The increase in the use of e-learning activities

has brought with it the logging of data resulting in

the development of learning environments and tools

to support learning analytics.

The definition of learning analytics set out in the

call for papers of the first international Conference

on Learning Analytics and Knowledge (LAK 2011)

and adopted by the Society for Learning Analytics

Research (SoLAR):“Learning analytics is the

measurement, collection, analysis and reporting of

data about learners and their contexts, for purposes

of understanding and optimising learning and the

environments in which it occurs.”

The definition is typically coupled with two

assumptions: that learning analytics makes use of

machine-readable data, and that its techniques can

be used to handle data in ways that would not be

practicable to deal with manually.

The rationale behind the authoring tools for

feedback and data analytics is that both teachers and

learners require support from different perspectives.

While students require support when interacting

with the learning activities, teachers need to know

when and how to intervene as well as how the

learning activities are being used. Lastly both

teachers and designers can benefit from the

availability of data as it provides potential to lead to

evidence about the student’s learning and eventually

redesign of teaching and learning activities based on

this evidence.

The use of an ontology was designed to reduce

the overhead in authoring of activities and to

potentially improve the value of learning analytics

through the added context. The approach we use to

build the ontology draws from previous experience

of the Learning Designer project (Laurillard et al.,

2013). In the learning designer project we developed

an ontological model to automate the annotation of

learning designs.

Charlton, P., Karkalas, S. and Mavrikis, M..

Designing a Mediation Vocabulary for Authoring Learning Analytics.

In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management (IC3K 2015) - Volume 2: KEOD, pages 223-230

ISBN: 978-989-758-158-8

223

This study provides the design process used to

develop an ontology to model the vocabulary for

authoring feedback to students. The analytics engine

uses the same vocabulary terms set by the designers

when providing results back to the designer.

In this paper we examine the knowledge

representation process used in the design and

creation of a mediation vocabulary. The model to be

designed has to provide different layers of

‘knowledge integration’ and thus offers insights into

the importance of knowledge mediation in the

emergence of new learning environments and

experiences. This approach addresses authoring of

learning activities. The ontology represents the

vocabulary terms used for both authoring learning

activities and for viewing the results of student

engagement.

2 BACKGROUND

2.1 Learning Design and Instructional

Design Theory

The development of “Learning Design” has links

with Instructional Design theory. The recognition of

the need to make theoretical findings readily

available to practitioners led to extensive work on

Instructional Design Theory (Reigeluth, 1999),

which attempted to make learning theories more

operational. However, the later focus on

“constructivist” theories of learning presented more

of a challenge to an operational approach.

Learning Design emerged as the realisation that

the constructivist pedagogical theories were not

easily embedded in the practice of teaching

(Jonassen, 1994). The emphasis on what learners

were doing, and how to support their activities, was

much less constrained by constructivism, and

therefore created a degree of uncertainty about the

way it would work in specific contexts. This

dependence on the context in which learning takes

place required an approach to teaching based on

design principles rather than pre-defined

instructional sequences (Oliver et al., 2002). There

have been attempts to offer “toolkits” or software to

enable ease of entry into pedagogic design and

support non-specialists in engaging with learning

theories. Despite the effort, existing e-learning

systems and authoring tools have limitations in

respect of support provided and usability. They do

not accommodate the needs of teachers who

increasingly look for more intelligent services and

support when designing instruction in order to avoid

cognitive overload (Mizoguchi and Bourdeau,

1999).

In previous research, the authors had found for

designers of learning, developing a tool that

supported learning design vocabulary mediated the

process of authoring designs, sharing their designs

more effectively and adopting and modifying

designs by others (Charlton et al., 2012; Laurillard et

al., 2013). A key finding of the Learning Designer

project (Charlton et al., 2012) was the representation

of learning design knowledge as an explicit

vocabulary supported the creation of learning

designs by designers. The vocabulary is an

approximation of the concepts used by designers.

The knowledge constructs of a learning design used

the shared vocabulary that acted as mediation of

knowledge between the designers. The vocabulary

captured meaning that was relevant to the designers.

This reduced the burden of design sharing and thus

facilitates design re-use.

2.2 Use of Ontologies in Annotation

Ontologies are one of the most important

technologies proposed in the context of the semantic

web. A frequent use of an ontology integrated into

science systems is to support formal information

retrieval of domain concepts and related content.

One of the most successful projects in use of

ontologies in science is the Gene Ontology project

(http://www.geneontology.org). It develops and uses

a set of structured, controlled vocabularies for

community use in annotating genes, gene products

and sequences.

The mediation vocabulary use in the design of

learning activities in authoring feedback is similar to

both the Gene Ontology project and the learning

designer project. It is the annotation use of mapping

a term in text selected by the designer to the

corresponding concept in the ontology. For the M C

Squared project ontologies are created and go a step

further. The annotations formed from the design

form part of the student’s learning context that is

shared back to the designers that now includes the

student’s use.

The students’ interaction creates a change to the

sequences and is in the student’s learning pathway.

The learning designer project and M C Squared

project differ from the Gene Ontology project is the

artefact itself. The instance of every use creates

another artefact (in this case the use of a c-book

creates new insights about the c-book) to be

evaluated within the context of the original design. It

is a dynamic changing artefact and the annotations

KEOD 2015 - 7th International Conference on Knowledge Engineering and Ontology Development

224

Figure 1: The mediation vocabulary for authoring feedback.

structured by the ontology model of the original

artefact provides a reference point for comparison.

So how the activities are used, revealed through

these annotations of design and student usage unveil

the authored analytics of learning in context and thus

offer a potential paradigm shift for both design and

use of learning analytics.

2.3 Design Considerations

While the ontology model designed and used to

automate the annotation process of learning

activities is not necessarily complex from an AI

perspective, other aspects required in order to

incorporate its use effectively are not so

straightforward. Essentially, neither a pure

knowledge engineering nor a software engineering

approach is sufficient in the creation of a mediation

vocabulary. In fact designers and developers of an

ontology for education systems and AI driven

education systems rarely consider this role. The

models are either used only as part of design

(implementation is never used) or are hidden only

given partial access to the model, if at all. Other

education systems that enable rich authoring may

expect designers to understand too much technical

aspects of the tool e.g. formulating rules and

program procedures and content management focus

(e.g.VLEs) This layer of technical engagement

distracts the designer from focusing on creating

learning activities. This increases barriers to

pedagogical annotation and reduces the possibility

of sharing these activities in communities.

Before going into details about the methodology

used and findings from the study we examine

aspects of the project context that needs to be

considered.

2.3.1 Domain of Design

While the learning activities are about mathematics

in secondary school the domain focus is about

creative mathematical thinking. The designers are

creating activities that they categorise as potentially

fostering creative mathematical thinking by the

learners. The design of the authoring vocabulary of

the activities will be formed around a set of formal

creative concepts.

Similar to the learning designer the vocabulary

needs to be ‘good enough’ and map to the designer’s

internal model of design concepts to facilitate the

authoring of creative mathematical thinking for their

students.

2.3.2 Learning Analytics Platform

Integration

Current platforms with learning analytics

functionality only support the common form of

analytics that of task completion analysis, correct

answers and time spent on task. They do not support

‘dynamic’ context of student feedback that we are

referring to and the authoring of this feedback

(Charlton et al., 2013).

The mediation vocabulary will need to facilitate

the integration of ‘relevant data’ between the

different components used when designing learning

activities, support the feedback process, form part of

annotating the student’s pathway and be part of the

Backend

database

Feedback

ontology

For

media on

vocabulary

In‐memory

database

reasoner

Wrapper

“annotated

learning

pathway”

authored

learning

analy cs

Ac vity

Wrapper

annotated

feedback

ac vi es

Wrapper

annota on

widgets’

feedback

Page

wrapper

(hub)

annota on

concepts

and

instances

C‐book

page

In‐memory

database

reasoner

Wrapper

“annotated

learning

pathway”

authored

learning

analy cs

Wrapper

annotated

feedback

ac vi es

Wrapper

annota on

widgets

feedback

Page

wrapper

(hub)

annota on

concepts

and

instances

Teacher

tools

Widget

annota on

Student

Teacher

Learning

Analy cs

Designing a Mediation Vocabulary for Authoring Learning Analytics

225

data queried for retuning the analytics visually. This

reflects the dilution of the distinction between data

management and data analysis in order to

contextualise the learning experience effectively.

2.3.3 C-book and Vocabulary of Creativity

A core concept of the project is the ‘c-book’. The c-

book is a set of learning activities for learning about

mathematics. A c-book is formed as a digital

interactive book. The ‘c’ stands for creativity. All c-

books are designed using social creativity process

and share an aim of fostering creative mathematical

thinking (CMT) of students (Ruthven, 2008). The

shared vocabulary across the project and community

of interests is focussed on social creativity and

creative mathematical thinking (Silver, 1997; Leikin,

2009). The core model is on supporting the

annotation of learning activities with creativity

concepts defined, understood and shared by the

community.

The M C Squared system (Karkalas et al., 2015)

is based on a generic framework that enables

seamless integration of complex learning objects

with e-book platforms. Authors can use the system

to dynamically query learning objects, identify

elements of interest and configure data logging,

learning analytics and intelligent support.

In Figure 1 we illustrate extending M C Squared

architecture to include mediation vocabulary for

authoring learning analytics and student feedback

through the annotation process. It is distributed

across the c-book components.

The pilot study investigated authoring

vocabulary requirements for the designers using a

design-based methodology. To capture the insights

and expert design knowledge we used ‘role play’ -

thinking aloud strategy to uncover the designer’s

knowledge (deGroot, 1995). Here we report on the

methodology used to determine the vocabulary and

provide a simple example of it being used to author

feedback for students and its use in learning

analytics.

3 METHODOLOGY

The c-book resources have been created to work

both in the classroom and as online independent

learning resources. When designing a c-book a

designer has available many ‘widgets’. A widget is a

distributed set of rich resources about mathematics.

A c-book has access to set of these resources. The

research study worked with four communities of

interest that are participating in the EU project and

creating different c-books based on different areas of

mathematics.

To design the vocabulary required understanding

the designers conceptual requirements and for them

to make their tacit knowledge explicit. The principal

being that if the designers had designed the feedback

with concepts that made sense to them then it was

more likely the results returned in the same context

would be of value. An iterative approach has been

used, using different design artefacts to share ideas

about authoring learning analytics, using

storyboards, knowledge elicitation templates of

different artefacts that included online interviews,

face-to-face workshops and partner meetings.

Role-play technique was used during an

intensive face-to-face workshop. Two specific

methods were adapted to facilitate knowledge

capture from the designers. The first is the use of

value creation stories, which draws on the work by

Wenger (Wenger et al., 2008). These are usually

used after engagement with a community tool or

exchange to capture the value added experience to

the users. The templates have been adapted as part

of a knowledge engineering design process to

capture key points by the designers e.g. using

‘AHA’ moments in conjunction with thinking aloud.

‘AHA’ moments are moments of sudden realization,

inspiration, insight, recognition, or comprehension.

This fits well within the context of creativity context

used for designing the learning activities. Designers

were being asked to ‘imagine’ how they would

respond if the student was in the room with them.

How and why they would respond to a particular

context would form part of the model.

The second approach uses peer review process as

a role-play activity. This provided the designers the

opportunity to: (a) Explain their perspective of

feedback given as a student, designer or learning

analytics about a particular c-book, (b) Give details

of the type of feedback that they felt would be

beneficial depending what role they played and (c)

Provide example concepts of creativity to author

student feedback and identify creative mathematical

thinking in students.

These two approaches meant that the designers

did not focus on mathematical detail but on the

learning experience for the potential students and

what kind of authoring would help the students, as

well as themselves.

KEOD 2015 - 7th International Conference on Knowledge Engineering and Ontology Development

226

4 MODEL AND KNOWLEDGE

REPRESENTATION

It is possible to currently add feedback in the c-book

design environment. The current feedback is specific

to a particular mathematical concept and cannot be

used across a variety of learning resources about

pedagogical value and creativity.

Using learning about co-ordinates activity a

simple authoring example of creative mathematical

thinking in Figure 2 is provided. The student is

given pathways to explore the learning resources. In

this example if the student is struggling with the

current task then the student is offered a different

style of interaction to facilitate the elaboration

process of thinking. The feedback needs to be

created by the designer and authored as content for

the purposes of elaboration. The content can then be

annotated with these concepts and can later be

explored in context with other data by the designer.

Figure 2: Authoring feedback example.

The flexibility concept here uses a simple timing

constraint to challenge the student. A more elaborate

activity could be created e.g. enabling the students to

create their own travel challenges and the answers to

these.

The concept of creative thinking through fluency

is when engaging fluently and keeping pace with the

task. The student can engage in elaboration or

challenging activities at any point in the learning

pathway.

4.1 Ontology Annotation Model

Using the above example we illustrate the use of the

ontology model in the creation of learning pathways.

We are using protégé to design and test out the

vocabulary before integrating this into the learning

analytics authoring of c-books platform. As a

mediation vocabulary it needs to serve a number of

requirements. Figure 2 indicates the authoring

potential of the c-books. This is where the model

functions to support designers to author feedback by

extending a maths activity. The author/teacher is still

designing but the ontology feature adds the

annotations automatically when feedback is

explicitly added. When a student uses the feedback

or an implicit condition is reached (designed by the

designer) then a feedback pathway is created.

There are two key points when the feedback is

triggered and contributes explicitly to the learning

pathway.

The first point is when the student actively

decides to engage with an elaboration activity or a

flexibility activity. The other is when the students

perceived behaviour authored by the designer is

recognised and feedback is automatically triggered.

Figure 3: Example of annotated student’s learning

pathway.

In the example given about authoring, the student

who encounters ‘elaboration’ feedback, in contrast

to the students’ who requests more challenging

activities, will be engaged in further details and

query about a specific question e.g. exploring the

cheapest flight or ranking the flights further. These

are key data points to be mapped to the common

analytics that takes place. It is not expected that a

student will engage only with one type of feedback

so the learning pathway for any one student may

have a number of different types of authored

feedback. However, this permits a number of

interesting explorations into the data about the

students to be reflected on as the student’s learning

pathway is now annotated with these concepts (see

figure 3). For example, more insight may be possible

if when seeing the results of students who performed

well in later tasks about coordinates, one can explore

which feedback did they seem to benefit from. When

misconceptions prevail, which feedback was used or

not used. Did the results of the student’s interactions

match the expectations of the designer? The learning

earning

path

ays

choices

made

available

the

student

during

certain

ac vi es

would

explore

further

know

what

doing

would

something

challenging

Authoring

feedback

during

design

Authoring

op ons

Se ngs

Set

Fluency

[

]

Set

Flexibility

[

Set

Elabora on

[X]

elabora on

context

then

reveal

what

answer

the

student

has

given

e.g.

“You

have

given

cheapest

answer”

now

see

you

can

…

Set

limit

Co-ordinates

Activity

Student

interaction

Feedback ontology

concepts

Advances

activities

Concept instance

Creates

Own example

Completion results in

Learning Outcome

Instance

Designing a Mediation Vocabulary for Authoring Learning Analytics

227

analytics tool can now use this knowledge to provide

this feedback to the designers.

5 FINDINGS

Using one of the activities in the c-book called

coordinates we illustrate the process used and a

summary of the initial findings. The c-book unit

‘Coordinates and points’ targeting first-year

secondary school students (12-13 years old),

presents an introduction to the Cartesian coordinate

system and the notion of ordered pairs. An objective

of the c-book is to help students understand points

on a graph. Firstly, students are asked to reason

qualitatively about the meaning of points plotted on

a graph. Secondly, a quantitative notion is

introduced in order to establish connections between

points and to get a preliminary notion of functions.

The students are encouraged to understand

different representations and to interpret the

knowledge that is presented. One of the challenges

for students is to remember and understand the

Cartesian representation (x,y) and the importance of

order. Also, there is the visual representation of

information and categorisation challenge.

General advice and guidance requirements about

supporting the use of the activities emerged. The

designers as students working on a ‘c-book’ that

wasn’t their design encountered challenges. In

certain activities the recognition of correct or

incorrect answers led to the designers articulating

that feedback operates on several levels and this

needs to be recognised and handled. For example at

the c-book level, hinting to a student ‘look at this

other page in a c-book’, another example at

presenting the scores, results, activity used, and

maybe expectations. Also, evaluating if students

correctly use representations etc. This experience for

the designers and knowledge engineers leads to

requirements of the mathematical widgets

capabilities in supporting such knowledge. While

pivotal but for most technical designers an almost

obvious requirement in designs of systems but

difficult to articulate this requirement in such a

complex system.

Reflective points for the designers emerged

requiring the creation of student exploration space/

to work through an answer and not just gaming the

system. Feedback trigger rules could be designed

and further requirements/dependencies of widgets

capabilities where considered (a) Registering

attempts, time and determine what type of feedback

to present based on this, (b) Use delayed feedback,

so allow students to do what they want to do, to

allow for ‘gradual insight’ and (c) After X attempts

enable ‘automatic’ interoperability between widgets:

show where points placed.

Classification of misconception and forming

elaboration pathway for exploration with the

student: (a) For example having a completely wrong

set of coordinates is not the same as mixing up x and

y coordinates, (b) Scaffold the feedback: provide

different levels of feedback e.g. an example, a mode,

an open question (“look at the x and y’s more

carefully”) and (c) Exploration to deeper knowledge

that supports pedagogical completeness and

soundness e.g. After the graph there needs to be a

question about the relationship.

In any activity a student can move between a

state of being (a) over-challenged (e.g. stuck) (b) in

flow or (c) under-challenged (e.g. because the

activity is too easy). The learning pathways emerged

identifying the students demonstrating creative

thinking through elaboration, flexibility and fluency.

This is a key observation and finding. For example,

in the coordinates book the students are given a table

to interact with and to plot co-ordinates. It is

assumed to be prior knowledge. If this is not correct

then maybe feedback referring to where this is

explained and to explain what the student doesn’t

know. This is an elaboration process. At this point

we see that the designers are now mapping ‘an idea’

of designing feedback to match the learner’s

possible ‘creative’ state of learning. Thus to enable

the learning to move on if a student is in an

elaborative learning state then the feedback to

engage with the learner is designed to develop

elaborative learning.

In another example c-book adaptivity was

required to reveal hidden pages, revealed if the

student is struggling or create an extension option

for students who are finding this too easy. The

feedback was designed with both elaborative and

flexibility types of feedback to bridge knowledge

gaps. The extension activities would be other pages

of a c-book revealed under the right conditions.

This led to the designers thinking about self-

reporting by student (for example opinion usefulness

of tool) can be compared with actual user logs etc.

This reflective log provides elements of ‘value

creation stories’. While this is a qualitative

expression of knowledge the design of a quantitative

led inquiry provides three points of data analytics

that the designer can reflect on (a) the learning

pathway (authored through the feedback process)

taken by the student, (b) the actual data from the

tasks completed and (c) the student’s view of their

KEOD 2015 - 7th International Conference on Knowledge Engineering and Ontology Development

228

progress.

Requirements emerged for feedback triggers.

Also, feedback needs to be designed to pose

questions rather than fixed answers. The questions

form part of the feedback leading students through

different learning pathways.

From the initial analysis of the data we define

three of the concepts from a pedagogical perspective

and learning pathway. These definitions may form a

different focus from that of the creativity

perspective:

Elaboration this we define as providing more

detail about the current context and is seen as the

‘easier’ entry point into a problem space. This

pathway is used when a student’s conversation is

frozen (Holmberg, 1983). Elaboration uses the

current context and encourages the student to expand

and investigate the problem space.

Fluency is the production of ideas, alternatives

or solutions. It has been shown that the more ideas

we produce, the more likely we are to find a useful

idea or solution. Here when fluency is in action there

is potential retention of previous

concepts/knowledge that is to be applied to the

problem space. Not only knowing what to use and

draw-upon but also how to apply the knowledge to

this context. This is when the learning pathway is

going in the right direction/as planned by the

designer.

Flexibility is especially important when logical

methods fail to give satisfactory results. Thus a

pathway that starts with elaboration may result in

flexibility occurring and the student uses a novel

(unexpected or not taught or a collection of usual

techniques) to resolve the problem.

6 CONCLUSIONS

Both the design of the conceptual framework to

capture the vocabulary and the use of it to

contextualise learning pathways through feedback

are novel. The added value of ontology driven

education tools through the annotation process can

add context to the artefacts, in this case c-books.

What is challenging in creating AI and learning

analytic solutions in this area is lack of methodology

that identifies the boundaries of the knowledge

representation otherwise the task is too complex.

The complexity arises from the knowledge

integration within systems that are intricate pieces of

software dealing with large sets of structured and

unstructured data. For example in this project there

are complex mathematical widgets that perform

multiple levels of computation supporting the design

of learning activities. Thinking of the knowledge

integration as a mediation task rather than exposing

in depth software operations of the widgets provides

a design of loose software coupling. This in software

engineering is done through for example, APIs.

However, the API level for many designers to use

would require too much technical knowledge. An

API definition is too fine grained and does not

operate at the design level for authoring learning

activities. However, enabling a knowledge

integration of the widgets feedback capabilities is

‘closer to’ the right level for the designers. It

requires ‘extension’ to the software of each widget’s

API but this is relatively minor at a technical level.

Another feature is the authoring of learning

analytics. For example we chose one aspect of

authoring, that of student feedback. Student

feedback has a large body of knowledge and like

creativity or the mathematical widgets we have to

limit the degree of knowledge to what has meaning

to the designers, what will add value to the process

of design, use and reflection. We used the designers

to guide this aspect. Expecting the designers to

author everything would be tedious and no doubt

stop the designers from focusing on their core task

of creating c-books. It is important that the authoring

process brings value in design as well as the use of

the design and the analytics to follow.

We have benefitted from combining two

methodologies of design: knowledge engineering

and design-based method. For example knowledge

representation looks for the explicit concepts ‘for

ontology commitment’ and design-based

methodology uses ‘thinking aloud’ and value

creation stories to express concepts of importance.

Using a vocabulary that is ‘subject matter

agnostic’ in education terms (not knowledge

representation terms) and foster a space to reflect,

such as creativity to author feedback may itself offer

insights into the design process. Creativity concepts

in this context may facilitate the design of more

appropriate activities and feedback. The use of a

digital mediation space illustrates the potential for

designing explicit knowledge in less well defined

domains. A key contribution to how to design

ontologies for education and creativity by combining

knowledge elicitation and design-based methods.

Finally, this design process was to support the

authoring of learning analytics. Does the result of

such a process provide insight for the designers? The

evaluation of embedding the ontology and testing

with designers the automated process is the next

step. It will no doubt reveal more about designing a

Designing a Mediation Vocabulary for Authoring Learning Analytics

229

mediation vocabulary and the potential of the

combined methodologies of design-based and

knowledge representation to advance the scaling up

and re-use of learning activities.

ACKNOWLEDGEMENTS

The research leading to these results has received

funding from the European Union Seventh

Framework Programme (FP7/2007-2013) under

grant agreement N0610467 - project ”M C

Squared”. This publication reflects only the author’s

views and the EU is not liable for any use that may

be made of the information contained therein.

REFERENCES

Bokhove, C., Jones, K., Charlton, P., Mavrikis, M. and

Geraniou, E., (2014). Authoring your own creative,

electronic book for mathematics: the MC-squared

project. In, Keith Jones, Christian Bokhove, Geoffrey

Howson and Lianghuo Fan (eds.). Proceedings of the

International Conference on Mathematics Textbook

Research and Development (ICMT-2014).

Southampton, GB, University of Southampton, 547-

552.

Charlton, P., Manolis, M, Katsifili. D., 2013. “The

Potential of Leaning Analytics and Big data”, in

Adriane, 2013,

Charlton P., Magoulas G., Laurillard, D., 2012. Enabling

Creative Learning Design through Semantic Web

Technologies, Journal of Technology, Pedagogy and

Education July 2012.

deGroot, A. D., 1965 Thought and Choice in Chess. The

Hague, the Netherlands: Mouton. 1969 Methodology:

Foundations of Inference and Research in the

Behavioral Sciences. New York and the Hague, the

Netherlands: Mouton.

Holmberg, B. (1983). Guided didactic conversation in

distance education. In D. Sewart, D. Keegan, and B.

Holmberg (Eds.), Distance education: International

perspectives (pp. 114-122). London: Croom Helm.

Jonassen, D. H., Thinking technology: Toward a

constructivist design model, Educational Technology,

34(2), 34-37, 1994.

Karkalas, S., Bokhove, C., Charlton, P., & Mavrikis, M.

(2015, June). Towards Configurable Learning

Analytics for Constructionist Mathematical e-Books.

In AIED (Vol. 2015).

LAK (2011) Learning Analytics and Knowledge,

http://solaresearch.org/

Laurillard, D., Charlton, P., Dimakopoulos, D.,

Ljubojevic, D., Magoulas, G., Masterman, E., Pujadas,

R., Whitley, E.A., & Whittlestone, K. (2013) A

constructionist learning environment for teachers to

model learning designs. JCAL.

Leikin, R. (2009). Exploring mathematical creativity using

multiple solution tasks. In R. Leikin, A. Berman, and

B. Koichu (Eds.), Creativity in mathematics and the

education of gifted students (pp. 129-145). Rotterdam,

the Netherlands: Sense Publishers.

Merrill, M. D. Instructional Design Theory, Englewood

Cliffs, NJ, Educational Technology Publication, 1994..

Mizoguchi, R., and Bourdeau, J Using ontological

engineering to overcome common AI-ED problems.

International Journal of Artificial Intelligence in

Education, 11, 107-121, 2000.Paradigm of

Instructional Theory, vol. II., Mahwah, NJ, Lawrence

Erlbaum Associates, 1999.

Oliver, R., Harper, B., Hedberg, J., Wills, S. and

Agostinho, S., Formalising the description of learning

designs. In A. Goody, J. Herrington and M. Northcote

(Eds), Quality conversations: Research and

Development in Higher Education, vol. 25, Jamison,

ACT, HERDSA, 2002.

Reigeluth, C. M.. What is Instructional-Design Theory and

How Is It Changing? In C.M. Reigeluth (Ed),

Instructional-Design Theories and Models: A New

Paradigm of Instructional Theory, vol. II., Mahwah,

NJ, Lawrence Erlbaum Associates, 1999.

Wenger Etienne, Trayner Beverly, de Laat Maarten (2008)

Promoting and assessing value creation in

communities and networks: a conceptual framework

http://wenger-

trayner.com/documents/Wenger_Trayner_DeLaat_Val

ue_creation.pdf.

Ruthven, K. (2008). Mathematical technologies as a

vehicle for intuition and experiment: A foundational

theme of the International Commission on

Mathematical Instruction, and a continuing

preoccupation. International Journal for the History of

Mathematics Education, 3(2), 91-102.

Silver, E. A. (1997). Fostering creativity through

instruction rich in mathematical problem solving and

problem posing. ZDM, 3, 75-80.

KEOD 2015 - 7th International Conference on Knowledge Engineering and Ontology Development

230