ViPS

An Intelligent Tutoring System for Exploring and Learning Physics through

Simple Machines

Lakshman S. Myneni and N. Hari Narayanan

Intelligent & Interactive Systems Research Laboratory, Computer Science and Software Engineering Dept.

Auburn University, Auburn, AL 36849, U.S.A.

Keywords: Intelligent Tutoring System, Learning, Misconceptions, Physics, Simulation.

Abstract: Students tend to retain naive understandings of concepts such as energy and force even after completing

physics lessons in science classes. We developed a learning environment called the Virtual Physics System

(ViPS) to help students master these concepts in the context of pulleys, a class of simple machines that are

difficult to construct and experiment with in the real world. Several features make ViPS unique: it combines

simulation and tutoring, it customizes tutoring to address common misconceptions and it employs a peda-

gogical strategy that identifies student misconceptions and guides students in solving problems through vir-

tual experimentation. This paper describes the ViPS system and studies in which we evaluated its efficacy.

Our results indicate that ViPS is effective in helping students learn and overcome their misconceptions.

1 INTRODUCTION

Computers have been extensively used in education

since the sixties (Martin and Mitrovic, 2001).

Teachers and students use computers in all aspects

of education such as researching, organizing infor-

mation, and doing or grading homework. At the

present time, it is hard to imagine a modern educa-

tion without computers. The use of computers can be

beneficial for teachers and learners. Intelligent Tu-

toring Systems (ITS) exemplify this benefit, by

tracking a student’s progress and providing tailored

feedback and hints along the way. By collecting

information on a particular student’s performance

and modelling that student’s progress, an ITS can

make inferences about a student’s strengths and

weaknesses, and can suggest additional work. This

paper presents the design and evaluation of an intel-

ligent simulation and tutoring system called the Vir-

tual Physics System (ViPS) for exploring and learn-

ing physics concepts within the context of a particu-

lar class of simple machines.

Our research is part of a large multi-university

project to investigate the teaching and learning of

physics concepts in middle schools. We have seen

that it is difficult for a teacher to track the progress

of students individually in a class with many stu-

dents. A teacher may not always know who is hav-

ing difficulty during group work in the class or labo-

ratory, may not be able to tell why a student is hav-

ing difficulties, and may not have enough time to

look into every student’s needs in a large class. A

tutoring system coupled with an experimentation

and simulation environment, on the other hand, will

be able to track each individual student’s problem

solving activities, such as the construction of a sim-

ple machine, the running of it, and solving problems

based on such simulations, and provide individual-

ized feedback. This is the primary motivation behind

the development of ViPS. One goal of middle school

science instruction is to inculcate deep knowledge of

fundamental physics concepts such as energy, force,

work and mechanical advantage in students through

hands-on work with simple machines like inclined

planes, levers and pulleys. However, learning about

simple machines, especially pulley systems, is in

itself a challenging task for many students. In addi-

tion, teachers face the difficulty of helping students

abstract what is learned in the context of hands-on

work to a more general understanding of physics

concepts. Several design and project-based ap-

proaches have integrated software and innovative

curricula to address this. Though students have ex-

hibited better understanding following such inter-

ventions, there is still room for improvement as mis-

S. Myneni L. and Narayanan N..

ViPS - An Intelligent Tutoring System for Exploring and Learning Physics through Simple Machines.

DOI: 10.5220/0003924700730082

In Proceedings of the 4th International Conference on Computer Supported Education (CSEDU-2012), pages 73-82

ISBN: 978-989-8565-06-8

 2012 SCITEPRESS (Science and Technology Publications, Lda.)

conceptions regarding important physics concepts

persist often into college years.

We addressed this problem through a two-

pronged approach: (1) by making it easier for stu-

dents to construct, simulate and experiment with

simple machines in a virtual environment, and (2) by

integrating a tutoring component with the simulation

component. We chose one class of simple machines,

pulley systems, as the domain for the tutoring and

simulation environment because students generally

find pulleys harder to understand than simpler ma-

chines like inclined planes. Another reason for this

choice is that complex pulley setups (e.g., those

involving compound pulleys with multiple grooves

or many movable pulleys) are so difficult to cor-

rectly build and experiment with in the real world

within the limited class time available that teachers

tend to limit hands-on activities to very simple set-

ups only. Furthermore, there are experimental setups

such as those with no friction that are impossible to

construct and test in the real world. The rest of this

paper is structured as follows. Section 2 discusses

research literature that forms the background of our

work. Section 3 describes the architecture of the

simulation and tutoring system ViPS. Section 4 pre-

sents empirical evidence for the efficacy of ViPS

and section 5 concludes the paper.

2 BACKGROUND

Tutoring is an instructional activity known to im-

prove student learning. For instance, Reiser, Ander-

son and Farrell (1985) reported that students work-

ing with private tutors could learn material four

times faster than students who attended traditional

classroom lectures, studied textbooks and worked on

homework alone. When a human tutor is not availa-

ble, the next best option maybe an Intelligent Tutor-

ing System (ITS). An ITS is a computer-based in-

structional system that has knowledge bases for

instructional content and teaching strategies. It at-

tempts to acquire and use knowledge about a stu-

dent‘s level of mastery of topics in order to dynami-

cally adapt instruction. Anderson & Skwarecki

(1986) reported that an ITS is a cost-effective means

of one-on-one tutoring to provide novices with the

individualized attention needed to overcome learn-

ing difficulties. ITS have been built for various do-

mains such as mathematics, medicine, engineering,

public services, computer science, etc. (Ritter el al.,

2007). The potential of ITS for helping students

learn is well recognized.

Another learning activity that is beneficial is

problem solving through experimentation. It is a

hands-on activity that involves designing and build-

ing an experimental setup, letting it perform its func-

tion and collecting data from it in order to solve a

problem, to better understand the underlying phe-

nomena or to test a scientific hypothesis. Computer

modelling and simulation often take the place of

physical manipulation in this learning activity. Many

researchers have described the affordances and limi-

tations of problem solving using physical manipula-

tives and computer simulations in science education

research (de Jong and Van Joolingen, 1998);

(Finkelstein, et al., 2005); (Triona, et al., 2005).

Zacharia and Anderson (2003) investigated the ef-

fects of interactive computer-based simulations,

presented prior to inquiry-based laboratory experi-

ments, on students’ conceptual understanding of

mechanics. They found that the use of simulations

improved students’ ability to generate predictions

and explanations of the phenomena in the experi-

ments. Triona and colleagues (2005) investigated

how physical and virtual manipulatives affected

student learning about mousetrap cars. Students used

either physical or virtual manipulatives to design

their cars. The physical and virtual treatments

showed the same effectiveness in helping students

design cars. Finkelstein and co-workers (2005)

looked at how students learned about electrical cir-

cuits differently with virtual or physical manipula-

tives. The simulations used by the students were

similar to the physical materials, except that the

simulations showed electron flow within the circuit,

which the physical materials could not. They re-

ported that the students who had used virtual ma-

nipulatives, i.e. the simulations, scored better on an

exam and were able to build physical circuits more

quickly than students who used physical manipula-

tives. Zacharia et al., (2008) looked at physical and

virtual manipulatives in the context of heat and tem-

perature. One group of students used physical ma-

nipulatives, while other group of student used physi-

cal manipulatives followed by virtual ones. Students

who worked with physical followed by virtual ma-

nipulatives performed better on a conceptual test

than students who only used the physical manipula-

tives. The authors’ conclusion was that one reason

for the addition of simulation increasing student

learning was that simulations could be manipulated

more quickly than physical setups.

Our research combines these two strands of tu-

toring and experimentation by designing and testing

a system, ViPS, that has both intelligent tutoring and

virtual experimentation capabilities. ViPS is able to

provide guided tutoring to a student as he or she

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

solves physics problems involving pulleys. ViPS

also allows the student to construct, run and collect

data from complex as well as simple pulley setups.

The interfaces of ViPS have been designed in

congruence with the Cognitive Theory of Multime-

dia Learning (Mayer, 2009), and its tutoring em-

ploys the instructional technique of Coached Prob-

lem Solving (Conati et al., 1997). Furthermore, ViPS

is designed to detect and help address the following

misconceptions regarding pulleys that students

commonly exhibit (see Table 1).

Table 1: Different misconceptions tutored by ViPS.

Definition

Misconception 1

The more pulleys there are in a setup, the

easier it is to pull to lift a load.

Misconception 2

The longer the string in a pulley setup, the

easier it is to pull to lift a load.

Misconception 3

Pulling upwards is harder than pulling

downwards.

Misconception 4

Having more pulleys in a pulley setup

reduces the amount of work.

Misconception 5

Size (radius) of pulleys in a pulley setup

affects the amount of work.

Misconception 6

Improper understanding of force and

work.

ViPS detects which of these misconceptions a

student has by asking the student to solve a set of

problems at the beginning. This involves answering

questions about pulley setups after constructing and

running them in the simulation environment. Based

on this, ViPS constructs a student model. This

model, which is continually updated throughout the

tutoring session, is used for generating additional

problems for the student to solve, and for providing

hints and other kinds of feedback based on the stu-

dents’ knowledge state. As far as we know, ViPS is

the first system to integrate a virtual experimentation

environment with a tutoring component specifically

tailored to address student misconceptions.

3 DESCRIPTION OF VIPS

ViPS provides a student with an interactive simula-

tion and tutoring environment in which pulley setups

can be created and simulated. Components required

for pulley setups can be created and manipulated

using a drag and drop interface. Students are asked

by ViPS to solve problems in this environment by

creating and running pulley simulations. As a stu-

dent is working towards a solution, the system keeps

track of his or her actions and provides feedback to

help the student make progress.

The architecture of ViPS, implemented in Java,

is shown in Figure 1. It consists of a graphical user

interface that manages interaction with students, a

simulation module that creates and simulates the

pulley setups built by students, a feedback module

that generates appropriate messages for the student,

a knowledge evaluator that evaluates the knowledge

of the student, a tutor module that tutors the student

for misconceptions, a student model that includes

the history of student interactions and various meas-

ures of student performance, a domain knowledge

model that represents domain knowledge, a database

of problems, and a procedural knowledge model that

represents student solution paths within individual

problems. We briefly describe these components

below (more details may be found in Myneni, 2011).

3.1 Graphical User Interface

The graphical user interface is responsible for all the

interactions with the student. This interface is di-

vided into two main parts: a tabbed work area for

creating pulley setups and solving problems, and an

object pallet for selecting the components required

to create a pulley setup. A snapshot of the interface

can be seen in Figure 2. Using this interface, stu-

dents can create a pulley setup by dragging the re-

quired components from the object pallet on to the

work area and clicking on the thread button. Stu-

dents can also interactively manipulate various pa-

rameters of the components, like the size of a pulley,

value of the load etc. A problem is given to the stu-

dent in the form of textual and pictorial representa-

tions (see Figure 3). The student is asked to solve the

problem by creating the setups required to answer

the question, running the simulations and comparing

the simulation outputs of the setups created. The

problems in ViPS were designed and checked by

experienced physics educators. Currently, ViPS

contains ten problems per misconception (60 in to-

tal, with more to be added in future) in its database.

A web-based interface is available to teachers and

experts to add or modify the problems. The reason

ViPS poses problems to a student is to first identify

his/her misconceptions and then to address them

through coached problem solving.

3.2 Knowledge Evaluator

When a student first initiates ViPS, a pre knowledge

test, in the form of problems to solve (see Figure 2

for an example), is given. Once the student finishes

the test, his/her answers are evaluated by the knowl-

edge evaluator to estimate the student’s initial

ViPS-AnIntelligentTutoringSystemforExploringandLearningPhysicsthroughSimpleMachines

Figure 1: Architecture of ViPS. Figure 2: ViPS work area.

Figure 3: ViPS problem pane. Figure 4: ViPS simulation window.

knowledge level and to identify the misconceptions

he/she might have so that a subsequent sequence of

problems can be generated for the student to solve in

a tutoring session. Similarly, a post knowledge test

is given to the student after the completion of a tu-

toring session, and the answers are evaluated by the

knowledge evaluator to determine the student’s post

knowledge level and the status of each misconcep-

tion identified from the pre-test.

After helping students to clear a particular mis-

conception that the system is currently addressing

through problem solving or tutoring, a follow up test

is given in order to estimate the knowledge acquired

by the students from problem solving or tutoring.

After the students exits the tutor module, a post

knowledge test is also given to evaluate the status of

all misconceptions detected from the pretest. Results

from the post knowledge test are used to determine

whether a student retained the acquired knowledge

through the end of the session. The knowledge eval-

uator is responsible for evaluating the students’

knowledge retention.

3.3 Simulation Module

The simulation module is responsible for creating

and simulating the setups created by the student. In

particular, it provides a platform for running simula-

tions of setups that are difficult or impossible to

create in the physical world, such as running a simu-

lation with zero friction or running a simulation with

quintuple pulleys. The outputs generated by the

simulation include graphs and real time values of

variables like force, work done, potential energy,

friction and mechanical advantage (see Figure 4).

The student uses the simulation module to run the

different pulley setups he/she creates during problem

solving. The domain knowledge of the simulation

model regarding possible or valid pulley setups is

represented in the form of a Bayesian Belief Net-

work. This network is used by ViPS to 1) find all

possible setups that can be created using components

that an individual student has created on the work

area, 2) find components for creating a valid setup

that are missing from the work area, and 3) generate

dynamic hints regarding pulley setups based on stu-

dent actions. ViPS generates all possible setups that

the student may possibly have in mind, based on the

components that the student created in the work

area. This setup inference process is illustrated by

the example in Figure 5. Figure 5(a) shows the part

of the network corresponding to a single compound

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

Figure 5: Setup inference using Bayesian Network.

pulley setup with extra pulley (C5). Initially the

probability of this setup is zero or false (indicated by

the orange bar in the figure). As the student creates a

fixed pulley (SFP) in the work area, this evidence

updates the network as shown in Figure 5(b). The

probability of SFP is updated to 1 or true (indicated

by the blue bar in the SFP box), and this results in an

increase of the probability of C5. There is a further

increase in the probability of the setup C5 when the

student now adds a second pulley and a load (proba-

bility increases from 31% to 71%, indicated by the

length of the blue bar in the C5 box) (Figure 5(c)).

The probability for C5 increases to 99% upon the

addition of a movable pulley to the existing setup

(Figure 5(d)). This example shows how ViPS infers

the intended setup if all the required components for

that particular setup are present in the work area.

It is possible that the components assembled by a

student do not lead to a unique pulley setup, and

instead can be used to produce several possible set-

ups. If this happens, ViPS infers and displays a list

of possible setups based on the probabilities of creat-

ing each setup as determined by the Bayesian net-

work, and ranked by an algorithm that we devel-

oped. This algorithm uses four attributes to rank

order possible setups: (1) the number of components

needed by a setup that are missing from the work

area; (2) the number of grooves in each pulley in the

setup; (3) the total number of components in the

setup; and (4) the number of times this setup was

created by the student previously. The rank of each

setup is determined based on these attributes. For

example, a possible setup with fewer number of

components missing from the work area will have a

higher rank than a setup with more missing compo-

nents. In other words, higher the number of missing

components, the lower the rank. Similarly, the algo-

rithm ranks simpler setups containing fewer number

of components and pulleys with fewer grooves

higher than more complex setups, and also ranks

setups that the student had previously created higher.

Once the rank ordering is complete, the system dis-

plays the top five setups to the student. Then the

student is asked about which of these setups most

closely match his or her intention. Based on the

students’ selection, the simulation module generates

dynamic hints to guide the student towards the com-

pletion of the intended setup in the work area.

3.4 Student Model

The student model includes information about each

individual student’s interactions with the system, pre

and post knowledge levels and misconceptions (as

identified from the tests), and the past problem solv-

ing behavior of the student. A Bayesian inference

network is used to update the student model

(Mislevy and Gitomer, 1996); (Conati et al., 1997).

A classical approach on how people forget is

based on research conducted by Herman Ebbing-

haus, and appears in reprinted form in (Ebbinghaus,

1998). Ebbinghaus’ empirical research led him to

create a mathematical formula that calculates an

approximation of how much may be remembered by

an individual sometime after he or she stops a learn-

ing activity (equation 1).

b = 100 *k/ (log t)

+ k (1)

Where:

• t is the time in minutes counting from one minute

before the end of the learning activity,

• b is an estimate of the amount remembered from

the learning activity after time t, and

• c and k are two constants with predetermined

values k=1.84 and c=1.25.

In the student model of ViPS, the Ebbinghaus calcu-

lations have been used as the basis for finding out

how much tutoring content or content learned from

problem solving is retained by the student. After

solving problems related to each misconception or

after tutoring the student for a particular misconcep-

tion, a follow-up test with three questions is given to

the student and based on the responses, the students’

initial memory state is calculated using equation 2.

ViPS-AnIntelligentTutoringSystemforExploringandLearningPhysicsthroughSimpleMachines

X%= b/100 * RQ (2)

Here b is Ebbinghaus’ power function result calcu-

lated using equation 1 with t=0. However, equation

2 has a new factor, called the Response Quality

(RQ). This is used to individualize equation 1 to the

particular circumstances of each student by taking

into account his or her answers to the follow-up

questions. RQ is the number of correct responses to

the follow-up questions asked after the completion

of problem solving or tutoring for a particular mis-

conception.

Once the tutoring for all the misconceptions a

student might have is completed, or after all the

problems have been solved, the student is given a

post knowledge test that is used to estimate the sta-

tus of each misconception detected by ViPS from

the pre knowledge test. This test is also used to cal-

culate the knowledge retained by the student, using

equations 1 and 2. The difference between the initial

(immediately after a problem solving or tutoring

session for a misconception) and final (after all mis-

conceptions have been addressed) retention levels

gives us an estimate of content that is not retained by

the student. This information is used by the system

to decide whether to re-tutor the student.

3.5 Tutor Module

The Tutor module of ViPS is based on the Cognitive

Theory of Multimedia Learning (Mayer, 2009) and

Vygotsky’s theory of learning. The tutor module is

responsible for overseeing the process of tutoring the

student for the misconceptions he/she might have,

and it is also responsible for overseeing the process

of problem solving by using the information gener-

ated by the student model to select and present ap-

propriate problems. It uses a decision algorithm to

determine the level of coaching to provide, and in-

teracts with the feedback module to generate appro-

priate hints. The design of tutoring content, text and

pictures, adhered to the principles stated in the Cog-

nitive Theory of Multimedia Learning, and the feed-

back generated by the tutor module is based on the

Zone of Proximal Development (ZPD) component

of Vygotsky’s theory of learning.

The tutor’s decision to tutor or not depends on

the student’s response to the problems he or she has

been given to solve. For every problem, the student

has to enter his prediction (P), actual answer (A) and

answer to a follow-up (FU) question. Based on these

answers (correct answer: T and wrong answer: F)

the problem is classified into one of the categories as

shown in Table 2.

Table 2: Students’ problem solving classification truth

table.

(P) (A) (FU) Classification

T T T R+

T T F R-

T F T W+

T F F W

F T T R-

F T F R

F F T W+

F F F W-

The problem is classified as successfully solved

(true), if the outcomes are R+, R-, and R or else it is

classified as incorrectly solved (false) (W+, W-, W).

The tutor module presents two or three problems per

misconception to determine whether a student has

that misconception or not. It uses these problem

outcomes to decide whether to tutor the student for

the current misconception or move on to evaluate

the next misconception using another set of three

problems. Table 3 shows the tutor action truth table.

For example, if the student solves the first two prob-

lems correctly, he or she is determined not to have

the corresponding misconception, so the tutor will

move on to the next misconception (Table 3, row 1).

If the student solves the first problem correctly but

errs in the second one, the tutor will present a third

problem and depending on its outcome will either

move to the next misconception (Table 3, row 2) or

start tutoring actions to clear the current misconcep-

tion (Table 3, row 3). Tables 2 and 3 together illus-

trate the tutor module decision tree. The student is

given up to four attempts to solve a given problem.

Figure 6 shows the flowchart of the tutoring process.

3.6 Feedback Module

The feedback module is responsible for generating

feedback messages for the students. This module

produces the following four types of feedback:

1. The student creates a setup by dragging compo-

nents onto the work area and clicking the thread

button. If any constraint violations that can lead to

impossible or invalid pulley setups are detected,

ViPS generates a feedback known as setup feedback.

2. The student creates a set of valid components in

the work area, but has no idea of what to do next,

i.e., how to thread a string through the pulleys to

complete the setup construction. In these circum-

stances, ViPS can deliver feedback about the next

moves the student has to make. This is known as

threading hint feedback.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

Table 3: Tutor action truth table.

Problem 1 Problem 2 Problem 3 Tutor Action

T(R+, R-, R) T(R+, R-, R) N/A Next Misc

T(R+, R-, R) F(W+,W-,W) T(R+, R-, R) Next Misc

T(R+, R-, R) F(W+,W-,W) F(W+,W-,W) Tutor Action

F(W+,W-,W) T(R+, R-, R) T(R+, R-, R) Next Misc

F(W+,W-,W) T(R+, R-, R) F(W+,W-,W) Tutor Action

F(W+,W-,W) F(W+,W-,W) N/A Tutor Action

Figure 6: ViPS tutoring process flowchart.

3. After creating and simulating one or more setups,

the student submits his/her problem solutions. The

system evaluates this and generates messages known

as problem feedback.

4. ViPS can coach students when needed during the

process of problem solving, and this is known as

problem hint feedback.

4 EVALUATION OF VIPS

We conducted evaluation studies of ViPS at one

university with 12 engineering majors enrolled in

their first physics course, and at another university

with 210 pre-service elementary teachers enrolled in

a physics course. Since ViPS is intended for even-

tual middle school use, our iterative design approach

to ViPS involves the following stages: (1) initial

design; (2) usability test of the initial design with the

target middle school population; (3) redesign; (4)

evaluation with more advanced (i.e. college) stu-

dents regarding the usefulness and usability of the

system; (5) redesign; and (6) deployment in middle

schools for final evaluations. Myneni (2011) pro-

vides details of the initial design (stage 1) and us-

ability testing with middle school students (stage 2),

which showed that the interface was usable, but also

revealed problem areas that were then corrected in

redesign (stage 3). Myneni (2011) also describes the

evaluation studies (stage 4). The analysis of data

generated from these studies revealed that ViPS is

effective in helping students learn and is also well

perceived by students. Below, we present just a

summary of this evaluation.

4.1 Experimental Procedure

A total of 220 students, 12 engineering majors from

one university and 208 pre-service elementary

teachers from another university, took part in the

evaluation studies. Experimental subjects with dif-

fering backgrounds were recruited so that we could

explore the extent to which ViPS would help both

students with more advanced physics knowledge

(engineering majors) and those who have less exper-

tise (elementary teachers). Twelve participants from

the first university and 50 from the second were

assigned to one experimental condition: the Virtual-

Only condition in which participants constructed

pulley systems using ViPS and solved problems

ViPS-AnIntelligentTutoringSystemforExploringandLearningPhysicsthroughSimpleMachines

related to misconceptions. However, data from two

participants at the first and 3 participants at the se-

cond university could not be used for analysis be-

cause of gaps in collected data. Hundred and fifty

eight participants from the second university were

randomly assigned to two experimental conditions:

(1) the Physical-Virtual condition (PV) in which

participants worked in groups of two, first with

physical pulleys and next with ViPS, to solve prob-

lems related to one misconception, and (2) the Vir-

tual-Physical condition (VP) in which participants

worked in groups of two, first with ViPS and then

with physical pulleys, to solve problems related to

the same misconception. There was no Physical-

Only condition because learning data for physical

pulleys could be obtained from the PV and VP

groups. All students answered a usability question-

naire at the end of their sessions, which was used to

assess user satisfaction with the system.

Study Procedure for Virtual-Only Condition

• Pre-Test: In a pre-test, the participants were

asked to individually answer 18 questions related to

pulley systems in ViPS in order to measure their

prior knowledge.

• Problem Solving: The participants individually

solved problems related to the misconceptions that

were identified from the pre-test, and underwent

tutoring as needed using ViPS.

• Post-Test: In a post-test, the participants were

asked to individually answer 18 questions (same

questions as pre-test, but displayed in different or-

der) related to pulley systems in ViPS in order to

measure their knowledge. ViPS used this test to

detect any remaining misconceptions.

• Usability Survey: All participants were asked to

fill out a usability survey to measure their overall

satisfaction in using ViPS.

Study Procedure for Physical-Virtual and Virtual-

Physical Conditions

• Pre-Test: In a pre-test, the participants were

asked to individually answer 18 questions related to

pulley systems on paper in order to measure their

prior knowledge.

• Group Assignment: Participants were paired and

pairs were randomly assigned to either the PV

Group or the VP Group.

• Problem Solving: Each group solved problems

related to one misconception (“the more pulleys

there are in a setup, the easier it is to pull to lift a

load”) using either actual pulleys or ViPS depending

on their assignment to the PV or VP condition.

• Mid-Test: In a mid-test, the participants were

asked to individually answer 18 questions related to

pulley systems (same questions as pre-test, but pre-

sented in different order) on paper in order to meas-

ure their knowledge after solving problems using

either actual pulleys or ViPS.

• Problem Solving: Each group then solved prob-

lems related to the same misconception using either

ViPS or actual pulleys depending on their assign-

ment to the PV or VP condition.

• Post-Test: In a post-test, all the participants were

asked to individually answer 18 questions related to

pulley systems (same questions as pre-test and mid-

test, but presented in different order) on paper in

order to measure their knowledge after solving prob-

lems using actual pulleys and then ViPS or vice

versa.

• Usability Survey: All participants were asked to

fill out a usability survey to measure their overall

satisfaction in using ViPS.

4.2 Log Analysis

The interactions between ViPS and the students have

been comprehensively logged. From these log files,

several features have been extracted and compared

using linear regression.

4.2.1 Pre-test Scores

Linear regression found a significant negative corre-

lation (see Table 4) between pre-test score and learn-

ing gain in Virtual Only, Virtual (VP), and Physical

(PV) groups. It is not surprising that these correla-

tions are strong as many of the students have low

pre-test scores.

Table 4: Correlation between pre-test score and learning

gain.

N R p Std Beta

Virtual Only 57 0.66 0.001 -0.664

Virtual (VP) 80 0.66 0.001 -0.633

Physical (PV) 78 0.62 0.001 -0.620

4.2.2 Problems Solved

Linear regression found a significant positive corre-

lation (N=57, R=0.756, R

=0.571, p=0.03, Standard-

ized Beta=0.792) between learning gain and number

of problems solved in the Virtual Only group. On

average, each student solved 8 problems while

working with the ViPS tutor. The other two groups

(PV and VP) were excluded from this analysis as

they solved problems related to only one misconcep-

tion.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation

4.2.3 Number of Simulations Created

Linear regression found a positive correlation be-

tween learning gain and number of simulations cre-

ated, but the value of p was not statistically signifi-

cant (N=57, R=0.039, R

=0.002, p=0.830). On aver-

age, each student created 14 simulations.

4.2.4 Problem Time

Figure 7 shows the average time taken to solve the

three problems in each misconception category (see

Table 1). Repeated measures ANOVA revealed an

overall significant difference in the average time

taken to solve the three problems while working

with ViPS (F(1,140)=9.1, p<0.02). The time re-

quired to solve a problem decreased significantly as

students moved to subsequent problems in the same

misconception category. This suggests that students

took more time to solve the first problem in a mis-

conception category as they were seeing a problem

related to that misconception for the first time, but

that they were faster at solving subsequent problems.

Figure 7: Average time taken to solve three problems in

each misconception category.

4.2.5 Misconceptions

Figure 8 shows the detected frequency of each mis-

conception. The most common misconception

among all the students who participated in the eval-

uation experiments is Misconception 2 (see Table 1)

followed by Misconception 1 and Misconception 4.

Out of all the students, 60 exhibited all the six mis-

conceptions. That such misconceptions persist in

college students is an interesting finding. A paired-

sample t-test comparing the number of misconcep-

tions identified in the pre-test and post-test in the

Virtual-Only group found a significant reduction in

number of misconceptions (t(54)=16.6, p=0.001).

On average, each student exhibited five misconcep-

tions after pre-test and two misconceptions after

post-test. The number of misconceptions decreased

significantly after working with ViPS.

Figure 8: Frequency of misconceptions.

5 CONCLUSIONS

In this paper, we presented an intelligent simulation

and tutoring system called ViPS for learning physics

concepts through exploring a class of simple ma-

chines. ViPS is innovative in several ways. First,

ViPS employs the Coached Problem Solving ap-

proach (Conati et al, 1997) to detect and effectively

tutor for common student misconceptions regarding

physics concepts exemplified in pulley systems.

ViPS is able to dynamically infer valid pulley setups

from the components that a student selects and

places on the workspace, and to adaptively generate

hints based on student actions. Second, ViPS is a

new tool for virtually experimenting with – creating,

exploring and simulating – pulley setups, which are

hard to build and manipulate in the physical world.

Third, the graphical interface of ViPS is designed

according to the Cognitive Theory of Multimedia

Learning (Mayer, 2009) in order to help students

connect abstract and difficult concepts of physics

with representations at a more tangible level. Fourth,

ViPS brings together the concepts of virtual experi-

mentation and intelligent tutoring in one platform.

Data analyses from ViPS evaluations support

these conclusions: (1) the less prior knowledge a

student has, the more he or she learns from ViPS; (2)

amount of learning is directly related to the number

of problems a student solves and the number of

simulations he or she runs; (3) the more a student

works with ViPS, the faster he or she is able to solve

problems; and (4) ViPS is able to reduce the number

of misconceptions students commonly exhibit.

One limitation of this work is that the system is

yet to be field-tested with the target population of

middle school students. An early study did show that

ViPS-AnIntelligentTutoringSystemforExploringandLearningPhysicsthroughSimpleMachines

middle school students found the system to be us-

able (Myneni, 2011). Results from evaluation with

college students also give us confidence that the

system will be effective in middle schools. It is in-

teresting to note that we identified an average of five

misconceptions in college students, even though

middle school curricula in physics are generally

thought to address and remedy such misconceptions.

ViPS was indeed successful in remedying many of

these misconceptions in college students. Neverthe-

less, this finding needs to be re-evaluated in middle

schools. Therefore, our future work is on fielding

ViPS in middle schools and evaluating it further.

REFERENCES

Anderson, J. R. and Skwarecki, E., (1986). The automated

tutoring of introductory computer programming.

Communications of the ACM, 29, 842-849.

Conati, C., Gertner, A., VanLehn, K. and Druzdzel, M.,

(1997). On-line student modelling for coached prob-

lem solving using Bayesian Networks. Proc. 6

Int’l

Conf. on User Modeling (pp. 231-242), Springer.

de Jong, T., and van Joolingen, W. R., (1998). Scientific

discovery learning with computer simulations of con-

ceptual domains. Review of Educational Research, 68,

179-202.

Ebbinghaus, H., (1998). “Classics in Psychology, 1885:

Vol. 20, Memory”, R.H. Wozniak (Ed.), Thoemmes

Press, 1998.

Finkelstein, N. D., Adams, W. K., Keller, C. J., Kohl, P.

B., Perkins, K. K. and Podolefsky, N. S., (2005).

When learning about the real world is better done vir-

tually: A study of substituting computer simulations

for laboratory equipment. Physical Review Special

Topics – Physics Education Research, 1(1), 010103.

Lewis, J. R., (1995). IBM computer usability satisfaction

questionnaires: Psychometric evaluation and instruc-

tions for use, International Journal of Human-

Computer Interaction, 7, 57-78.

Martin B. and Mitrovic A., (2001). Easing the ITS bottle-

neck with constraint-based modelling. New Zealand

Journal of Computing, 8(3), 38–47.

Mislevy, R. J. and Gitomer, D. H., (1996). The role of

probability-based inference in an intelligent tutoring

system. User-Modeling and User Adapted Interaction,

5, 253-282.

Myneni L. S., (2011). An Intelligent and Interactive Simu-

lation and Tutoring Environment for Exploring and

Learning Simple Machines. Unpublished doctoral dis-

sertation, Auburn University.

Reiser, B. J., Anderson, J. R. and Farrell, R. G., (1985).

Dynamic student modeling in an intelligent tutor for

LISP programming. Proc. Int’l Joint Conferences on

Artificial Intelligence (pp. 8-14).

Ritter, S., Anderson, J., Koedinger, K. and Corbett, A.,

(2007). The Cognitive Tutor: Applied research in

mathematics education. Psychonomics Bulletin & Re-

view, 14(2), 249-255.

Mayer, R. E., (2009). Multimedia Learning. New York:

Cambridge University Press.

Triona, L. M., Klahr, D. and Williams, C., (2005). Point or

click or building by hand: Comparing the effects of

physical vs. virtual materials on middle school stu-

dents’ ability to optimize an engineering design, Proc.

Annual Conference of the Cognitive Science Soci-

ety (pp. 2202-2205).

Zacharia, Z. and Anderson, O. R., (2003). The effects of

an interactive computer-based simulation prior to per-

forming a laboratory inquiry-based experiment on stu-

dents’ conceptual understanding of physics. American

Journal of Physics, 71(6), 618-629.

Zacharia, Z. C., Olympiou, G. and Papaevripidou, M.,

(2008). Effects of experimenting with physical and

virtual manipulatives on students' conceptual under-

standing in heat and temperature. Journal of Research

in Science Teaching, 45, 1021–1035.

CSEDU2012-4thInternationalConferenceonComputerSupportedEducation