ExperD: Web-based Support for Laboratory Class Workflow Design

and Execution

Koos van der Kolk

, Gerrit Beldman

, Rob Hartog

, Gerard Moerland

and Harry Gruppen

Laboratory of Food Chemistry, Wageningen University, P.O. Box 8219, Wageningen, The Netherlands

Wageningen MultiMedia Research Centre, Wageningen University, Hollandseweg 1, Wageningen, The Netherlands

Keywords: Laboratory Class, Electronic Performance Support, Web-based.

Abstract: The design, use, and evaluation of a web-based experiment designer, ExperD, are described. ExperD

supports students in designing a research strategy for their laboratory class. Next, ExperD supports students

in their actual laboratory class work by showing them which experiments they have to carry out, and what

the relation is between experiments. The use of ExperD was evaluated in the 2009 and 2011 editions of a

Food Chemistry course at Wageningen University in The Netherlands. The evaluations showed that students

(n = 60 and 98) find ExperD helpful and that teachers see the ExperD as a valuable addition to the

laboratory class. Usage logs show that students used the tool throughout the entire laboratory class.

Furthermore, the ExperD proved to be a promising research tool for monitoring both student design

activities as well as student actual lab work activities.

1 INTRODUCTION

Laboratory classes are an essential part of chemistry

education. With respect to the work presented in this

article, we focus on two challenges in laboratory

class education. Firstly, skills related to designing

experiments are often undertrained in laboratory

classes (Bennet and O’Neale, 1998; Domin, 1999).

Secondly, students can experience working memory

overload in laboratory classes (Johnstone, 1997). If a

problem requires the learner to have too many

chunks of information in his or her working memory

simultaneously, this memory becomes ‘overloaded’.

Working memory overload hampers both the

problem solving process and learning (Kirschner

2002; Sweller et al., 1998). In practice, this will lead

to less effective and less efficient laboratory classes

(Johnstone, 1997). These two challenges were also

recognized in the B.Sc. ‘Food Chemistry’ laboratory

class at the Wageningen University. In the

remainder of this article we first describe this

laboratory class in more detail, followed by the aim

and methodology of the research presented here.

Next, we describe the web-based design tool ExperD

and two case studies in which ExperD was

evaluated. Finally, we discuss the results of this

evaluation.

1.1 The B.Sc. ‘Food Chemistry’

Laboratory Class

The eight week morning course ‘Food Chemistry’ (6

ECTS credits) at Wageningen University in the

Netherlands consists of four parts. In the first three

weeks students attend lectures, practice the theory

using digital exercises and perform self-study. The

next three weeks are spent in a laboratory class.

Next a self-study week is scheduled followed by an

exam in the 8

week of the course. During the

laboratory class students should:

 Learn to work together in small groups.

 Acquire hands-on experience with common

food chemistry research methods.

 Learn to design a research strategy.

Students work in small groups of 2-3 students.

Each group is given an agricultural material (e.g.

barley) and investigates major chemical changes

during simulated processing (e.g. beer brewing)

during the 3 weeks. For example, in the case of

barley, groups mimic the first steps in beer brewing

on a bench scale and are asked to investigate what

happens to the major carbohydrates and proteins.

Groups are guided through the investigation by 15-

19 assignments. They design their research strategy

by relating these assignments to common food

chemistry experiments. There is a many-to-many

van der Kolk K., Beldman G., Hartog R., Moerland G. and Gruppen H..

ExperD: Web-based Support for Laboratory Class Workﬂow Design and Execution.

DOI: 10.5220/0004349800250032

In Proceedings of the 5th International Conference on Computer Supported Education (CSEDU-2013), pages 25-32

ISBN: 978-989-8565-53-2

 2013 SCITEPRESS (Science and Technology Publications, Lda.)

relationship between assignments and experiments:

Assignments relate to multiple experiments and

experiments relate to multiple assignments.

Assignments as well as experiments can take more

than one day to complete. The groups should make a

time schedule of their laboratory work and distribute

tasks among group members. Once a group has

completed the formulation of their research strategy

they have to present their set-up to a teacher. The

teacher provides feedback such as pointers to

inconsistencies or inefficiencies.

In general, teachers of the Food Chemistry

laboratory class were not satisfied with the research

strategies student groups came up with. Many

groups made unclear designs, others just made a list

of experiments and assignment numbers (see Figure

1).

Figure 1: Example of research strategy design made by

students.

As a consequence of the unclear designs, teachers

often had to spend quite some time on figuring out

what students meant, and felt it was difficult to give

sufficient adequate feedback. In defense of the

students it can be argued that they did not receive

training nor guidance in making clear research

strategies. We therefore felt that there was an

opportunity to improve the laboratory class by

offering students support in designing research

strategies.

Teachers also observed that the majority of the

students were ‘just carrying out a list of

experiments’ during the laboratory class. So most

students did not know why they were carrying out a

particular experiment, nor the relation of that

particular experiment with the research strategy as a

whole. With (Johnstone, 1997) we attribute this

behavior - at least partly - to an overloading of

working memory. We further hypothesize that this

overloading was related to the research strategies

that they had designed and in particular to the

chaotic nature of the formulation of these strategies.

This reinforced our belief that offering support in

making a clear research strategy could improve the

laboratory class.

1.2 Aim of this Research

The aim of this design oriented research was to

address the opportunity described in the above

section. As workflows of experiments are not an

uncommon format for food chemists to present their

research strategies, e.g. (Christiaens et al., 2012; De

Roeck et al., 2008; Chassaigne et al., 2007), the

basic idea was to provide a web based tool that

would support students in designing a workflow of

experiments. Teacher-student and student-student

interactions could then benefit from the standardized

representation of the workflow designs.

Additionally, the workflow could function as a

scaffold during laboratory work, as it would give

students a clear view on the relation between

experiments and insight in their progress. To our

knowledge, such a tool for chemistry laboratory

classes does not exist yet.

The following research question was leading

during the research: Is it possible to design, realize

and implement a web based experiment workflow

design tool that

1. students find helpful?

2. teachers find valuable?

3. students really use during the laboratory class?

4. serves as a research tool for monitoring student

design activities and student progress during

the laboratory class?

1.3 Research Method

Design oriented research aims at the generation of

knowledge by designing a new artefact (Busstra,

2008; Österle et al., 2010). This model focusses on

sharing knowledge with respect to sensible goals in

a well specified real university context, providing

arguments why these goals make sense

and demonstrating how they can be achieved in that

context (Hartog et al., 2010). The goals are

formulated in terms of testable design requirements,

which are used to evaluate the realized and

implemented artefact (Verschuren and Hartog 2005).

For the design we chose the satisficing strategy, a

strategy that tries “to meet criteria for adequacy,

rather than identifying an optimal solution”

(Jonassen, 2008) Our design requirements are listed

in Table 1. From now on we will refer to the realized

design by its name 'ExperD'.

The ExperD would have to be implemented in an

existing educational setting. This implied that it

should fit the existing infrastructure and some

already available web based resources. In particular,

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

Table 1: ExperD design requirements. Evaluation questions use a five-point Likert scale (1 = agree, 5 = disagree) for

response. Our satisficing criterion is that we consider the design requirements to be met when at least 80% of the students

rate an item as 4 or 5.

Design requirement How to determine whether the design requirement is met.

r1. According to the students

ExperD should be helpful

Student questionnaire questions/statements:

a. in general

q1. "I found it useful to design a scheme."

q2. "I would like to have such an ExperD in other laboratory classes."

b. in order to work efficiently q5. "The ExperD helped our group to work efficiently."

c. by giving them the

overview

q6. "The ExperD helped me to figure out what I could expect during the

laboratory class."

q7. "The ExperD helped me to have the overview during the laboratory class."

d. by being easy to use

q3. "The ExperD was easy to use."

q4. "The ExperD was self-explanatory."

q8. “It was easy to distribute tasks using the ExperD’s user interface”

r2. Be really used by groups

during their practical work

Usage logging: 80% of the groups should be updating their experimental

workflow during the first two weeks of the laboratory class.

r3. Be appreciated by the teachers. Teacher interviews

r4. Serve as a monitoring tool for

design activities.

Teacher interviews / Usage logging

ExperD would make use of desktop computers that

are present on the student laboratory benches.

Moreover, ExperD should become part of the

content management system Drupal™ 6, which is

used by the Laboratory of Food Chemistry to deliver

and manage their e-learning resources. Thirdly, the

ExperD should be integrated with the web based

laboratory manual developed earlier (Kolk et al.,

2011).

1.4 ExperD

Taking into account the design requirements from

Table 1 and a set of design and usability

recommendations (Mayer, 2009), a web-based

environment for the design of an experimental

workflow (ExperD) was realized. The user interface

of the ExperD consists of five main elements: 1) a

main bar with available experiments, 2) a workflow

view containing 3) one or more experiments, 4) a

dialog window to edit the properties of the selected

experiment (Figure 2) and 5) a time planner (Figure

6). These user interface elements can be configured

depending on the characteristics of the course. In the

remainder of this section ExperD’s user interface

elements will be discussed as they were configured

for the course ‘Food Chemistry’.

With ExperD, students design a research strategy

in the form of a workflow of experiments. In the

‘Food Chemistry’ course, they do this by choosing

one of the assignments from the available

experiments and adding the appropriate experiments

to the workflow (Figures 3 and 4). Students connect

those experiments of which samples should be

transferred from one experiment to the next. For

example: They connect the experiment 'Get starch

solution' to the experiment 'Hydrolyze starch with

enzymes' because the sample obtained in the former

experiment is used in the latter experiment. Next,

students describe the sample in chemical/physical

terms by selecting one or more properties from a list

with properties (see Figure 5). For example: Does

the sample contain carbohydrates, fats, proteins; is

the sample solid or liquid? To support the design

process, the ExperD gives feedback on the

properties selected by the students. For example: the

experiment 'Grind sample' does not expect a liquid

sample, so if students try to connect an experiment

to 'Grind sample' having a liquid sample, the ExperD

gives a warning message (Figure 3). Because the

feedback is based on the properties of the ingoing

samples – and not on the upstream experiments

providing these samples – teachers do not have to

adjust the feedback of existing experiments when

they add or remove experiments. Besides describing

the sample properties, students can enter other data

for each experiment.

They can enter for what assignments/research

questions they need the experiment, what the

experiment’s purpose is, which group member is

going to carry out the method, what the results are

and when the method will be carried out. The

scheduling of methods is done in a ‘Gantt chart’ like

manner (Figure 6): Students drag and drop, stretch

and shrink the experiments on a horizontal time axis

to obtain a time planning. Lastly, an experiment in

the ExperD can be linked (Figure 7) to a learning

object in a web based laboratory manual (Kolk et al.,

2011).

ExperD:Web-basedSupportforLaboratoryClassWorkflowDesignandExecution

Figure 2: Overview of the ExperD’s user interface. Students design their experimental workflow by selecting experiments

from the Main bar (1). The experiments (3) are added to the Workflow canvas (2) and students can connect them, or change

their properties using a dialog (4).

Figure 3: The ExperD’s feedback system is based on the student-defined properties of the samples going from one method

to another.

Figure 4: An experiment in the ExperD is displayed as a

block with ingoing and outgoing samples. For example:

the experiment ‘Get protein solubility vs pH’ has one

ingoing sample ‘In’ and two outgoing samples ‘Pellet’ and

‘Supernatant’.

1.5 Two Case Studies

The ExperD was implemented and evaluated in the

2009/2010 (further referred to as ‘2009’) and

2011/2012 (‘2011’) editions of the course ‘Food

Chemistry’. The set-up of the laboratory class did

not significantly change between these two editions.

There were differences between the versions of the

ExperD software used. The version of ExperD that

was used in 2009 did not yet include a time planning

module. This came only available in 2011. In 2009

students had to save the workflow manually a few

times a day. In 2011 this workflow saving was

Figure 5: The properties dialog as configured in the course

‘Food Chemistry’. In this dialog students can view/edit the

properties of the selected experiment.

automated: any change to the workflow was

instantly saved. In 2011 ExperD failed to provide

feedback due to a technical problem.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

Figure 6: Overview of the time planning module with the ExperD (introduced in 2011). Each experiment is in the workflow

represented by a horizontal bar in the time planning. The position and the length of this bar represent the starting time and

the duration of the experiment. The shaded area in the time planner means the past, the white area the future. Each student

in a group has a color (red, orange, green) and these colors are used throughout the user interface to show which method is

assigned to which student(s). Icons are used to show the method’s status: Whether it is ‘in progress’ or it has been

‘finished’.

Figure 7: ExperD is linked to a web based laboratory manual. Students can view the online manual of a particular method

by clicking on the ‘View in lab manual’ link in the properties dialog.

In both editions of the course, students designed a

concept workflow on the first day of the laboratory

class. The teachers then gave oral feedback on the

workflows, after which students made some

adjustments. Students used the workflow throughout

the remainder of the laboratory class, e.g. to see

what experiments they scheduled for a particular

day, to enter results, to update it, etc.

In 2009 (n=60) and in 2011 (n=98) the ExperD

was evaluated by the students by means of a

questionnaire, which they had to fill in after the

laboratory class ended. In 2009, teachers of this

laboratory class (excluding those who supervised the

class for the first time, n=4) were interviewed by one

of the authors a few weeks later. The 2011 teachers

(n=6) were asked to comment on the conclusions of

the 2009 interviews.

1.6 Collected Data

The results of the questionnaire are listed in Table 2.

The most important outcomes from the teacher

interviews from the 2009 case study were:

t1. The teachers find the ExperD a valuable

addition to the laboratory class. It especially

helped teachers in discussions with students

during the laboratory class, because both they

and the students could easily indicate certain

points in a standardized workflow. All groups

did forget to include one or more experiments in

their initial workflow designs.

t2. Some teachers had indications that their

students had more overview during the

laboratory class than in previous years. For

example: They recalled several occasions where

ExperD:Web-basedSupportforLaboratoryClassWorkflowDesignandExecution

students themselves found out that they could

combine the samples for certain analyses. The

teachers did not recall that this occurred in

previous years.

t3. Some student groups seemed to have stopped

thinking about the laboratory class design after

they finished designing it. When asked ‘why are

you doing this experiment?’, the answer these

groups gave was: ‘Because it is in the scheme’.

t4. The ExperD allows groups to make a ‘perfect’

separation of tasks. ‘Perfect’ in the sense that

students did not know what experiments other

group members were doing. Within groups

‘specialists’ arose, who did all analyses of a

specific kind, often without knowing anything

about the samples they had to analyze.

These outcomes were confirmed by the 2011

teachers of the course.

In Figure 8 the percentage of groups updating

and using the ExperD are plotted against time. The

method status (whether a method was ‘in progress’

or ‘finished’) was kept up to date by 90% of all

groups during the laboratory class.

In Figure 9 the ExperD usage and webLM usage

are plotted per group. Between groups we found

substantial differences in the intensity in which the

ExperD was used (Figure 9), the most active group

generating 11 times as much updates as the least

active group.

1.7 Discussion

In the introduction we mentioned several challenges

for our laboratory class, which were operationalized

in a set of design requirements (Table 1). We will

discuss whether these design requirements have

been met, and come up with some recommendations

to improve the ExperD.

1.7.1 Requirement 1 and 2: The ExperD

should be Helpful for and used by

Students

Students found it useful to make a design with the

ExperD on beforehand (q1 in Table 2). Surprisingly,

the 2009 students seem to find it more useful to

design a scheme than the 2011 students. We have no

explanation for this difference, but the design

requirement r1 was met in both cases. A large

majority (84-86%) of the students would like to see

the ExperD to be available in other laboratory

classes (q2). Students also indicate that the ExperD

helped them to work efficiently (q5). Although this

self-reporting has some value (e.g. with regard to

student motivation), ‘working efficiently’ should be

further operationalized in a follow-up study to make

more objective claims. A similar conclusion can be

drawn for ‘the ExperD gives students the overview’

(requirement 2c): We have indications that the

ExperD gives students the overview (q7, t2), but

also indications that point otherwise (t4). Although

the students find the tool easy to use (q3, q8), the

result for q4 “The ExperD is self-explanatory” is

still unsatisfactory. This could be improved by

offering students an interactive tutorial before they

start designing, or by giving inline hints when they

use the ExperD for the first time (e.g. a textbox near

the main bar: ‘Click on a method to add it to the

workflow’, followed by a textbox near the added

method: ‘Click on a method to see its properties’,

etc.).

Table 2: Questionnaire results of the 2009 (n=60) and 2011 (n=98) case studies. For each question two result rows are

shown: the upper one being the results of 2009, the lower one the results of 2011.

# Question Answers (%)

1=disagree, 5=agree

% 4+5

1 2 3 4 5

I found it useful to design a scheme.

0 3 0 30 67 97

1 5 14 49 32 81

I would like to have such an ExperD in other laboratory classes.

0 2 15 37 47 84

0 2 12 48 38 86

The ExperD was easy to use.

3 3 7 53 33 86

0 7 11 59 22 81

The ExperD is self-explanatory.

2 5 25 58 11 69

0 8 15 67 10 77

The ExperD helped our group to work efficiently.

0 3 3 58 36 94

0 3 13 58 26 84

The ExperD helped me to figure out what I could expect during the

laboratory class.

2 2 10 64 22 86

0 1 10 75 14 89

The ExperD helped me to have the overview during the laboratory class.

0 0 9 64 28 92

0 1 3 49 48 97

It was easy to distribute tasks using the ExperD’s user interface.

3 3 19 41 34 75

1 2 16 59 22 81

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation

Figure 8: Percentage of groups updating and using their ExperD worklow at least once during the 2010 laboratory class.

Groups making at least one change to their workflow are considered to be an ‘updating’ group for that day. Some groups

did not update the workflow for one day, but did update it the next. Because we assume that these students did use the

workflow in between (for viewing only), these groups are considered to be ‘using’ the ExperD on both days.

Figure 9: The ExperD and web lab manual (webLM) usages per 2011 group. To obtain the ExperD activity values, the

number of laboratory methods changed in the workflow during the laboratory class was summed per group. The webLM

usage was determined as described previously (Kolk et al. 2011). For one group, webLM usage data became unusable

because of a problem in the logging software (the other groups were unaffected by this problem).

The majority (>80%) of the groups continued using

their experimental workflow during the first 10 days

of the laboratory class (Figure 8). The usage declines

in the second and third weeks, most likely because

laboratory class workflows did not need to be

adjusted anymore and because groups finished their

experiments. Earlier we expected that there would

be ‘computer minded’ groups, which would use both

the ExperD and webLM intensively, and less

‘computer minded’ groups, which would avoid using

both tools. Our results indicate that this is not the

case.

1.7.2 Requirement 3: The ExperD should

be Appreciated by Teachers

In general, the teachers find ExperD a valuable

addition to the laboratory class, as it helped them in

their discussions with students (t1). However,

teachers were somewhat unpleasantly surprised by

the extent to which the ExperD enabled students

within a group to work independently from each

other (t5). It can be argued though, that ExperD

made a ‘weakness’ of the laboratory class set-up

apparent. Namely, that it is possible for a student

group to solve the assignments and obtain a

sufficient mark for the laboratory class without the

ExperD:Web-basedSupportforLaboratoryClassWorkflowDesignandExecution

student group members knowing what the others are

doing.

Teachers observed that all student groups did

forget to include one or more experiments (t2).

Letting the ExperD check for ‘childless’

assignments (i.e. assignments without methods

linked to them) or ‘orphan’ methods (i.e. methods in

the workflow without assignments linked to them)

could prevent these kind of mistakes in the

workflows.

1.7.3 Requirement 4: The ExperD

should Serve as a Monitoring Tool

for Design Activities

Figure 8 and Figure 9 show possible usages of

monitoring student design activities. Because each

update to the workflows is saved instantly, teachers

can monitor student design activities in real time

from their own computer. This can help them e.g. in

finding groups that are struggling to make progress

during the laboratory class. Student groups have the

possibility of changing the ‘status’ of an experiment

in the workflow. For groups using this feature - 90%

of all groups - a chart could be developed, in which

group progress is plotted against time. This gives

teachers a quick indication of how groups are

performing in the laboratory class. Finally, the data

generated by ExperD allows for replaying the

workflow design process and reconstructing how

groups progressed through the laboratory class.

Analysing this process might be useful to find the

problems students have with designing workflows of

laboratory classes in general. It can also be used by

teachers to detect difficult or unclear assignments

and other bottlenecks in a specific laboratory class.

1.8 Concluding Remarks

The leading research question in this research was:

Is it possible to design, realize and implement a web

based experimental workflow design tool, which

students find helpful, which teachers find valuable,

which students really use and which can serve as a

research/monitoring tool? In other words, we aimed

to falsify the hypothesis that it is not possible to

design, realize and implement such a tool. We

believe that the case studies in which ExperD was

used falsify this hypothesis and thus provide a proof

of feasibility. ExperD is a highly-valued tool, used

intensively by a large majority of the students within

our laboratory class, and might be of use for both

teachers and researchers. Since the 2009 evaluation,

ExperD has also successfully been introduced to the

laboratory classes of an interdisciplinary B.Sc. level

course ‘Food Related Allergies and Intolerances’

and a M.Sc. level course ‘Food Ingredient

Functionalities’. We are currently in consultation

with other chair groups at Wageningen University to

investigate how to implement ExperD in their

laboratory education.

REFERENCES

Bennet, S. W. & O’Neale, K., 1998. Skills development

and practical work in chemistry. University Chemistry

Education, 2(2), pp.58–63.

Busstra, M. C., 2008. Design and evaluation of digital

learning material for academic education in Human

Nutrition,

Chassaigne, H. et al., 2007. Investigation on sequential

extraction of peanut allergens for subsequent analysis

by ELISA and 2D gel electrophoresis. Food

Chemistry, 105(4), pp.1671–1681.

Christiaens, S. et al., 2012. In situ pectin engineering as a

tool to tailor the consistency and syneresis of carrot

purée. Food Chemistry, 133(1), pp.146–155.

Domin, D. S., 1999. A Content Analysis of General

Chemistry Laboratory Manuals for Evidence of

Higher-Order Cognitive Tasks. Journal of Chemical

Education, 76(1), p.109.

Hartog, R., Beulens, A. & Tramper, J., 2010. Faculty-

based design-oriented research on digital learning

materials: Defining project goals. Global Learn Asia

Pacific 2010, 2010(1), pp.1418–1427.

Johnstone, A. H., 1997. Chemistry teaching—science or

alchemy? Journal of Chemical Education, 74(3),

pp.262–268.

Jonassen, D. H., 2008. Instructional Design as Design

Problem Solving: An Iterative Process. Educational

Technology, 48(3), pp.21–26.

Kirschner, P. A., 2002. Cognitive load theory:

implications of cognitive load theory on the design of

learning. Learning and Instruction, 12(1), pp.1–10.

Kolk, K. van der et al., 2011. Students Using a Novel

Web-Based Laboratory Class Support System: A Case

Study in Food Chemistry Education. J. Chem. Educ.,

89(1), pp.103–108.

Mayer, R. E., 2009. Multimedia Learning 2nd ed.,

Cambridge University Press.

Österle, H. et al., 2010. Memorandum on design-oriented

information systems research. European Journal of

Information Systems, 20(1), pp.7–10.

De Roeck, A. et al., 2008. Effect of high pressure/high

temperature processing on cell wall pectic substances

in relation to firmness of carrot tissue. Food

Chemistry, 107(3), pp.1225–1235.

Sweller, J., van Merrienboer, J. & Paas, F., 1998.

Cognitive architecture and instructional design.

Educational Psychology Review, 10(3), pp.251–296.

Verschuren, P. & Hartog, R., 2005. Evaluation in design-

oriented research. Quality and Quantity, 39(6),

pp.733–762.

CSEDU2013-5thInternationalConferenceonComputerSupportedEducation