EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES
IN EDUCATION OF ENGINEERS
Gabrielle Hansen-Nygård, Kjetil L. Nielsen, John B. Stav, Trond M. Thorseth and Ketil Arnesen
Sør-Trøndelag Univsity College, Faculty of Technology, E. C. Dahls gt. 2, 7004 Trondheim, Norway
Keywords: Student response systems, Mobile learning, Mobile technology, Smartphone’s, Assessment by use of mobile
devices.
Abstract: This article reports experiences achieved during the development of open, online Student Response Services
(SRS), and the emerging extension of these SRS into assessment services carried out on modern mobile
devices. The evaluation results obtained from engineering classes in Norway that have used the online SRS,
show very good results. The online SRS is flexible, intuitive, easy and fast to use. They may be used
together with any kind of software. It is also reported how teachers are going to use a forthcoming and
extended version of the SRS, as a tool for verification or elaborative feedback immediately after completion
of tests and exams. This may be done for single students or groups of students. The development of the
SRS and the forthcoming assessment system is co-funded by the EU-Commission through the Lifelong
Learning Programme.
1 INTRODUCTION TO MOBILE
RESPONSE TECHNOLOGY
Mobile learning is dramatically shaping the nature of
teaching, learning, and social interaction.
Assessment methods, however, is still frequently
often done by use of traditional evaluation methods,
though students and teachers may integrate mobile
technology and learning both in and out of the
classroom, due to portability of mobile devices and
their ability to connect to Internet almost anywhere.
They are ideal as a store of reference materials,
learning experiences, and general-use tools for
fieldwork.
Mobile technology provides a new active
collaborative learning approach, which let students
set the in-class terms for discussion in order to get a
clearer view of their instant knowledge and
perception. The new mobile technology based online
Student Response Services (SRS) (EduMecca, 2010)
are flexible, intuitive, easy and fast to use. The SRS
may be used together with any kind of software
systems, and for in-class, laboratory and distance
training purposes. Use of SRS easily display
complex interrelationships between opinions in
class, including most relevant or marginal choices,
or what is most difficult to understand. It
interconnects the teacher and student in a new way
that by highlighting learning impact, which it is
impossible to do so quickly by using traditional
methods. The tools were during 2009 and 2010
tested and validated in training of staff from higher
education and industry in several European
countries, e.g. Norway, UK, Sweden, Hungary,
Romania, Slovenia and Slovakia. The evaluations
show excellent feedback from students and teachers.
This paper reports experiences obtained and
discusses recent developments that bring mobile
learning to the next level by using open web-based
solutions for cheap Smartphone’s in education,
keeping in mind the pedagogical challenges in the
new mobile learning environment. Mobile devices
can be used almost anywhere, they are perfect
platforms for situated and context based learning
activities, where real life is used to provide stimuli
and activity for learning. They are constructed for
use in vocational education and training, and in
higher education courses. This is achieved through
easy and flexible integration with interactive touch
screen blackboards by utilizing Flash in combination
with AIR and FLEX technology. The European
Commission cofounded the R&D during 2008-10 as
a pilot project under the KA3-ICT program
(EduMecca, 2010).
At the end, the article outlines new, emerging
mobile learning assessment solutions (Done-it,
2011) that are based up on latest mobile technology.
383
Hansen-Nygård G., L. Nielsen K., B. Stav J., M. Thorseth T. and Arnesen K..
EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES IN EDUCATION OF ENGINEERS.
DOI: 10.5220/0003473003830391
In Proceedings of the 3rd International Conference on Computer Supported Education (ATTeL-2011), pages 383-391
ISBN: 978-989-8425-50-8
Copyright
c
 2011 SCITEPRESS (Science and Technology Publications, Lda.)

This will extend the usability of the SRS by
construction an assessment solution for mobile
devices.
2 ONLINE RESPONSE
TECHNOLOGY SOLUTIONS
The new SRS is based up on XML-based standards
and web authoring facilities for the contents
available on web pages, by providing XML-based
universal notation and interface including
visualization of scientific and engineering drawings
and graphs. The search facilities retrieve the
postulates of the instructor through a service-
oriented architecture that integrates semantic web
into the system for retrieval of information from the
knowledge base system. Closed solutions like for
instance iTunes are avoided by using open web-
based solutions.
The decision process solution system is open and
flexible in order to achieve maximum
interoperability. The easiest teacher lead SRS
knowledge cycle includes, based up on results from
Norway:
1. Teacher starts the SRS for mobile devices
when he/she is ready to teach, and the students
enter the 3 digit session code just before the
lecturing starts. The sesson codes allocates the
class to one lecture room, whereby several
neighbooring classrooms may to use the same
WI-FI network in paralell.
2. Teacher present new material from the
curriculum
3. Students are presented a conseptual quiz and
asked to discuss with each other for 2-3
minutes
4. The teacher starts a voting session by using a a
web interface on the digital blackboard
5. Student casts individual votes using the
handheld units.
6. The vote is closed and results are presented on
the blackboard (immediately or when the
teacher decides)
7. The teacher comments on the various
alternatives and highlight the correct one,
explaining thoroughly why it’s the correct one
and why the others are incorrect.
8. Go back to point 3 and repeat.
A lesson consists often of 2 lectures, each lasting
about 45 minutes. During each 45 minutes period the
students are usually presented for at maximum of 2
conceptual questions. In order to start a polling
session (which usually lasts 30 seconds), a “ticking
clock” is used to shift the students attention away
from discussion and over to the voting session in
progress.
Several trials of the SRS have been completed in
Norway and Sweden. Four trials have been
performed internal at Sør-Trøndelag University
College (HiST), in Trondheim in 2010. In addition
to the internal testing at HiST, the SRS have also
been tested by external users both in Norway and
Sweden. In connection with these, numerous
surveys, in form of questionnaire and interviews,
have been conducted. The surveys consist of a
mixture of written questionnaire where informants
read the questions and indicate their response on a
form, and focus group interviews with end-users. A
focus group interview constitutes a form, with a
group interview where the conservation and
discussion process is essential. Unlike more
conventional forms of interviews, where those
conducting the interviews take the role of
interviewers, a focus group interviewer takes on the
role of a discussion moderator, that is, a moderator
who organizes discussion within the groups. One of
the main advantages of focus group interviews is
that, if properly managed, it can be extremely
dynamic (Bergh 2007).
The interviews were analyzed using type of
analysis called grounded theory, a method for the
analysis of qualitative data that has wide acceptance
in social science (Johannesen, Tufte & Kristoffersen
2004). Grounded theory is an appropriate direction
for the analysis of topics such as personal
experience, opinions, feelings and attitudes
(Charmaz 2001). The aim of the current evaluation
was to bring out the experiences, views and opinions
expressed by our students in relation to the use of
SRS in engineering education. Grounded theory was
considered to be an appropriate too for achieving
that goal. This method provides specific procedures
for the analysis of data, where data is coded in the
steps:
• Line- by- line coding
•
Focused coding
• Categorization
3 EVALUATION RESULTS
OBTAINED IN ENGINEERING
EDUCATION IN NORWAY
The first testing of SRS for mobile devices was done
over a period of 5 weeks in the autumn 2009 (Stav
CSEDU 2011 - 3rd International Conference on Computer Supported Education
384

2010) in civil engineering classes. The second
testing of the SRS was done in the autumn 2009, in a
preparatory class for engineering in the subject of
physics and lasted over a period of 4 weeks. The
evaluation was carried out at the end of the 4-week
testing period, and consisted of two focus group
interviews and a questionnaire. Six students
participated in the interviews, with three students in
each group. The interviews were analysed using the
analytical approach called grounded theory. The
questionnaire was conducted online, through Google
docs, by using the “redirect” function in the response
system. A total of 57 students participated (39 boys
and 17 girls). The analysis resulted in three
categories: Feedback on own learning (A),
Increased engagement (B) and Group discussion
(B). These categories are made up of and represent
the students' main experiences in relation to the use
of student response system in class.
A. Feedback on Own Learning
1. Immediate feedback on their learning: Use of
SRS provides students with valuable feedback on
their learning and progression. To answer the quiz
questions and receive immediate feedback was a
way for students to test themselves on the fly. They
got an immediate feedback on whether they had
understood what the teacher had tried to convey,
whereby they immediately tested their knowledge in
practice. Two of the students had the following to
say about the feedback that SRS gave them;
Per: "You get a feedback on how well you have
understood the topic. For if you have selected the
correct alternative to a quiz, you get feedback that
you’ve understood this subject. You get feedback
that you are able to use the formulas and laws – yes,
the material that the teacher has presented."
Ole: "Yes, you get a feedback on whether you
have understood it. Yes, it’s all about your own
learning. You get to see if you've learned
something."
For students is feedback an important part of
their learning. Feedback tells them how they are
doing in their own learning process. The feedback
activities these students get are usually related to
various types of tests and assignments. It is rarely
provided any kind of feedback activity during the
actual teaching time. The only opportunities the
students have to receive feedback are by raising their
hand and either ask or answer questions from the
teacher. Whether students will receive feedback
during the lessons depends, in other words, on them
self, and whether they take the initiative and actually
reply to or ask questions. From the student side, it
too often ends up with very little feedback, because
they find it very uncomfortable to raise their hand
and talk loud over dozens other students. When
asked whether the teacher included some feedback
activities during their lessons, one of the students
said; "No, the teachers may ask, "do you
understand?” and then they just look sheepishly at
us and move on. None of us that dare to raise our
hand and respond. In that sense, it’s our own
responsibility, but no, I certainly don’t. Feedback
activities are normally not included, which is a bit of
a shame."
Students desire something they call constructive
feedback in their academic life. This is feedback that
in addition to providing them with a pointing stick if
they're on the right track or not, explain why
something is right or possibly wrong. From the
student side, this is feedback that gives them
something concrete to work with; they get feedback
on what they are possibly struggling with and what
they need to focus at. Without such feedback
students feel that they only have themselves to deal
with, making it difficult for them to know where
they stand in relation to their own learning and
progression. One of the students said it quite clearly;
"Without constructive feedback, how can we really
improve?"
2. Teacher's explanation afterwards: the key to
students' understanding: In relation to students'
desire for more constructive feedback, SRS came in
as a long awaited breath of teaching. Firstly, the
system gave them an immediate feedback on their
polling, in that they got to see if they had voted right
or wrong, as well as the teacher went through each
option after the vote and explained thoroughly why
they were correct or incorrect. For students, the
teacher's explanation was perceived as a constructive
feedback, and was further highlighted as critically
important in relation to their experience of learning.
In short, it was here that the learning came into play.
Through the teacher's explanation the students get an
understanding of why the alternatives were correct
or incorrect. One thing is to cast a vote that turns out
to be right or wrong; another thing is to really
understand why it is right or wrong. If they achieve
such an understanding; they feel they have really
learned some of the quiz questions. Especially the
wrong options are highlighted as important to get a
thorough explanation around. For students, this is all
about giving those who have answered incorrectly a
chance to understand why they answered incorrectly.
One of the student groups explained it this way;
Emma: "Those of us who got the answer wrong
have to be given to chance to understand that we
were wrong. Some part of the class usually got it
wrong, and then it must be explained in such a way
that we can understand where we went wrong.
EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES IN EDUCATION OF ENGINEERS
385

Because we obviously don’t know if an option is
wrong – otherwise we wouldn’t have voted for it! "
Lise: "Yes, I feel it gives me a chance to
understand what the subject is really about."
The second group had the following to say;
Per: "There's a reason why people have
answered incorrectly, it's because they have
misunderstood something, and then they have to be
explained why the answer was wrong."
Ole: "Yeah I think the explanation from the
teacher is very important. I think it is necessary that
he explains why he uses certain laws, or other parts
of the curriculum, and that he shows us why it is
right or wrong. "
Jens: "Spending some time to explain or discuss
the different options and how to the right answer in
this way is; well, I feel that the quiz becomes a bit
useless if you don’t do that - if you don’t spend
enough time on it and do it thoroughly. The quiz
then becomes – maybe not useless, but the quiz has a
much greater effect on learning if you get an
explanation why the answers are right or wrong. "
3. For improvement – how to get a better feedback
on the “actual understanding”?: From the student
side, there is little doubt that the use of SRS can
provide them with valuable feedback on their
learning, particularly if the teacher gives them a
thorough explanation after the vote. At the same
time, however, they leave no doubt that the SRS
may have a much greater learning potential than was
used in their teaching.
Traditional teaching where planned for the test
group of students. They usually have two or three
lessons after each other. During trials of the SRS
they usually got the first quiz question a little off in
the first teaching hours (often after about 20-25
minutes), the next question came either later in the
same teaching hour or a bit out in the next one. The
times they had three hours in a row, they had no quiz
questions in the last hour. The teacher started the
lesson by introducing a new part of the curriculum.
Afterwards the students got a quiz questions based
on what they had recently been presented.
According to students, this was a straightforward
way to implement SRS in teaching, as they due to
the teacher's explanation after the vote felt that they
learned something from it. In relation to test their
understanding, however, this was no optimal
solution. The quiz questions were introduced too
early. Whether you test comprehension or not, is
according to the students depending on the time the
quiz question is being asked. It is also depends on
the available time they have for learning before the
question is asked. In other words, if a teacher wants
to use SRS to test students' comprehension, students
must first be given time to work with the curriculum
and acquire the academic requirements they need to
answer the quiz question. I they get the quiz
questions too early in the teaching hour it is not
certain that students have got these prerequisites.
Thus, instead of testing their understanding through
the use of SRS, the teacher gives them an
understanding through a thorough explanation after
the voting.
Group 1:
Per: “I would like to get a quiz at the end of the
day too, in order to check if we’ve really understood
it. After we’ve worked with the exercises for a period
of time, and had time to process the material.”
Jens: “Yes, I agree.”
Per: “That would give a very good indication as
to whether you’ve understood something or not.
That would be a proper test!”
Ole: “Then we would have worked with it for a
bit, and then we’ll get to see if we’ve understood it.”
Group 2:
Emma: “I somehow ... need time to understand
it, in a way. Sometimes I think that the quiz
questions seemed to come too early for me, in a way
... There were times when I just made a guess. I had
somehow not received the scientific basis for
properly discussing it. I felt that it was a bit
unnecessary.”
Ingrid: “Yes, we’d almost have to lie ahead, if we
are to do it that way. The questions tend to be from
the new subject area that we’ve just been through.
So really, it might be best if he took us through the
curriculum first, and included questions at the end of
the class.”
Lise: “Yes, to see that people had actually …
understood it.”
The students point out that in many ways it is up
to the teacher how he/she wants to use the SRS. The
teacher may use it to give the student an
understanding, by giving them a quiz question
followed by a thorough explanation, or he/she may
just test their understanding. According to the
students, we had in no way selected to use the SRS
in an incorrect way. They just pointed out that to
really test their understanding, they must first
possess an understanding of the curriculum thought,
and it is rarely in place after only 20 minutes. It
comes usually after the material has decreased
slightly, which usually happens when they have
worked a bit with it through exercises. If we will test
their understanding it is at this point they should get
quiz questions.
B. Increased Engagement
1. Finally there's something else!: Use of SRS is
CSEDU 2011 - 3rd International Conference on Computer Supported Education
386

a great way for the students to engage in the
teaching process. By answering the questions they
participate actively in the production of educational
content, and their involvement increases. In contrast
to ordinary classes, where students feel they have
more than enough to listen and take notes, use of
SRS introduces a break where they get time to think
about, resonate around the educational material and
process, and simultaneously test themselves. From
the student side, it is impossible to maintain the
concentration in one or several hours. It is simply
too much information to absorb at once, whereby it
is easy to lose the concentration and the motivation
drops. Using the SRS, however, stop the regular
teaching process in the class and the students may
recover during a short time frame where something
else happens! Students welcome a distraction
introduced by SRS, instead of just sitting and
receiving information. They become activated and
motivation increases. One group of students
discusses the increasing involvement in relation to
the use of SRS in the following manner;
Per: "You participate, yes, you are active in that
you work with quiz questions and talk to other
students. I was more engaged when using the
system. But it has an effect on motivation as well. I
think it's a bit exciting, a bit of fun, and it made
classes more fun."
Ole: "It was like a small activity in the middle of
the lecture, which restored my motivation when I
started to doze off. I think, the lecture, well, there’s
too much information at once, you cannot keep up.
So it’s refreshing that you get to think for yourself,
even if it’s just for a short period, and get to answer
questions.”
Jens: "Yes, I felt that the class got a motivational
boost, and became more active. It is definitely one
thing that helps to maintain interest during classes!
The tests are seen as a bonus, "soon a quiz will come
and then I may test myself to see if I’ve understood
it." I see nothing but positive aspects with SRS."
Per: "Me too. I’m normally not very active, so it
was fun to join in and participate."
Jens: "Yes it was very, very positive, a real
bonus."
Ole: "Yeah, well, you get a break from the usual
lecture."
2. Anonymity: the magic key: Although SRS
offers students a much sought-after break from the
ordinary teaching process in classes, it is according
to the students the way the system allows them to
participate that is the main reason behind the
increase in their commitment. SRS offers students
something that ordinary education is missing: a
chance to participate anonymously. In contrast to
ordinary classes, where students' participation
usually involves raising their hands with oral
responds, they may use the SRS to answer questions
without that answers are traced back to them. What
they respond, and if they answer correctly, it is only
they themselves that know. The students explain that
anonymity is a crucial role in relation to their
participation and usage of SRS. In short, anonymous
responses made it sure that they participate. One of
the students describes the role of anonymity in the
following manner;
"It ensures that everyone participates! Everyone
may provide his or her vote and their “voice” will
be heard (pause), that will never happen in a normal
class. It [the anonymity] was the key factor, which
convinced me to attend, no doubt. "
The most common option, and often the only
one, where students may have the opportunity to
participate actively in a lesson is by raising your
hand to either answer or ask a question. From too
many students point of view, this is not a
particularly attractive opportunity. Far too many find
it uncomfortable showing off by raising their hand
and talk. They are afraid to make fools of
themselves, either by asking a stupid question or
answer incorrectly. They have all experienced to
wonder of something but not asked any question, or
to avoid answering questions from the teacher,
especially if they are not sure about the correct
answer. The fear of exposing themselves to the class
prevents them simply from active participation in the
class. Students define their own role in an ordinary
teaching as a spectator, and not a participant! The
usage of the SRS reduces the threshold for active
participation significantly. By answering the quiz by
using SRS, their anonymous participation was
placed in safe limits. Everyone could answer without
having any fear of dumb out towards fellow
students. Use of SRS was a new way for students to
solve tasks, which resulted in the response from the
entire class, versus the usual few.
3. Ultimately: learning at all: From the student
side, engagement and learning are flip sides of the
same coin. One does not exist without the other!
Commitment is a prerequisite for obtaining good
learning. In addition it provides a better experience
of learning. Being involved is described in this
context as being mentally present. The student’s
concentration and attention are sharpened, and they
find it easier to absorb information. The use of the
SRS helped to initiate such processes among the
students. By getting the opportunity to think for
them selves, discuss with the person sitting next by,
answer questions and receive responses, the students
experienced increased engagement. They were
activated and felt that the concentration, which often
disappears during traditional teaching hours, were
EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES IN EDUCATION OF ENGINEERS
387

awakened. Four of the students had this to say about
both the commitment and learning, and SRS's role in
relation to commitment;
"As far as learning is concerned, unless I’m
somehow engaged, I simply drop out and start
thinking about other things. So for my learning, I
need to be engaged during lessons to be able to
absorb information. "
"You learn a lot better when you are engaged.
You are more “on the ball", because you get to
conclude with different things – pulling different
cords in a way – you think more actively about
things, you work better with the material, and you
learn better! "
"I think it was good, because you are a bit like,
“aha, now we’re going to run a quiz”, and you get a
bit excited like that. You never as much attention as
when you get the quiz up on the board. So I think,
for me there was more involvement. You were more
into it! "
"Let me put it this way: at least I woke up"
C. Talk with the Person Sitting Next by
1. Group Discussion: a good way to learn: Before
each vote using the SRS, the students were
encouraged to consult the persons next by discuss
the quiz question and its alternative in a few
minutes. An encourage that no doubt was taken
seriously by students. From their side, to collaborate
with other students is an effective way to learn.
Hearing others' perspectives, opinions and
viewpoints are highlighted as important to get
deeper into curriculum and achieve a better
understanding. The discussions gave them, in other
words a professional benefit. Students described the
discussion as follows;
Group 1:
Per: "I think I learned something from them,
absolutely ..."
Ole: "It's always nice to get the opinion from the
person sitting next to you"
Jens: "Yes, when you see the questions, you form
an opinion that goes one way, and then along comes
the person next to you with a different opinion. Thus,
you get input from somebody who may think in a
completely different way, and you just realize, "I
never thought about that”. Yes, you get a chance to
discuss what the correct option is."
Group 2:
Lise: "It's very nice to be given the opportunity
to speak with someone, especially since we’re
covering subject areas that are new to us. It’s good
to hear what others think, and together try to
achieve a common understanding."
Emma: "Yeah I think it worked really well. We
tried to reach an agreement on the correct answer.
So, if we disagreed there would be a very good
discussion. You knew that both sides couldn’t
possibly be right, so you’d turn the material a bit
upside down and discuss it. Very good. "
2. Targeted discussion, they discuss not only to
discuss: The discussion with the person sitting next
bye was perceived as a valuable element in relation
to the use of SRS. One reason for this is that the
discussion had a clear goal, it should end in a vote
that would give students feedback on their learning.
They discussed, with other words, not only to
discuss, they discussed to be better understand and
to cast their vote. The goal was to find the correct
answer that would further give them a positive
feedback, and the discussion raised the chances of
achieving that goal. The feedback students receive
when using the SRS was thus a "carrot" that
motivated them to participate actively in the
discussion. According to the students this made the
discussions focused and efficient. They only had a
couple of minutes to try to discuss and try to find the
correct answer, whereby they must use the time
efficiently. One student group had the following
comments about this;
Emma: "The point of the discussions we had,
was to figure out an exact answer. Otherwise, when
we are discussing, I think the discussion very quickly
loses focus, or at least becomes a rather “free-
roaming” discussion."
Lise: "Yeah true, I think voting is very
important! I don’t think we would have bothered to
discuss with the person sitting next to us if it would
have been for nothing; if I didn’t cast a vote
afterwards."
Emma: "You motivation increases."
Ingrid: "You put more into the discussion, to find
the right answer."
Lise: "Yeah, that sums it up nicely!"
4 EXTERNAL TESTING OF SRS
AT HIGH SCHOOL IN
NORWAY
Four college teacher students tried out the SRS
system in the spring of 2010, as part of a research
project in their master studies. They had practice at a
high school in Trondheim, where they should teach
3
rd
-year students. SRS was used as a regular part of
their math classes for three weeks. Trials were
completed with an evaluation, in the form of
questionnaires and group interviews. Before the
trials started, the college teacher students came to
HiST where they received technological and
methodological training in the use of SRS. They also
CSEDU 2011 - 3rd International Conference on Computer Supported Education
388

borrowed a PC pre-installed with the SRS-Control
interface (SRS-Ci), a wireless router, 30 iPods and
chargers for the iPods. They obtained the following
experiences:
Methodology: A typical SRS session was conducted
as follows: the students came to classes and picked
up their iPod. The teacher started teaching as
normal, using traditional teaching methods. During
the course of the class, two multiple-choice sessions
with using the SRS were conducted. The teacher,
based on what students should have learned, or
sometimes to test whether students were following
the lecture, designed the assignments. The questions
were read out loud by the teacher, and at the same
time shown in writing (using, for example,
PowerPoint, overhead projector or a normal
whiteboard). The students were given a few minutes
to discuss the assignment among themselves in small
groups, before casting a vote. After the vote, the
results were discussed in class. Both students and
teachers participated in the discussion. All response
options were discussed, both those that were correct
and those that were incorrect.
The teacher students` experiences in relation to the
use of SRS: Based on our experiences, we believe
that the SRS can contribute as a positive tool in the
assessment of learning. An important point in this
context is that students become increasingly more
active both the learning process, as well as the
assessment of the teaching being given. Particularly
in large classes, we believe that the SRS can act as a
good tool to provide continuous feedback.
Communication with the students was an
important aspect of using SRS in our classes. By
communicating with the students to a greater extent
than by traditional teaching, we believe we made an
established a better report with them, which in turn
helped to improve the learning environment of the
class. Further on in our practise period we saw the
importance of activating and motivating students
during classes. By introducing the SRS in classes we
noticed that students became more active and
motivated, which also contributes to a more
including and forgiving classroom culture. As future
teachers, we therefore wish to stimulate the students
to be orally active and, and to encourage students to
participate in reflective discussions.
Our experience from the practice period is that
many teachers find it scary and difficult to try new
things in education. We experienced the same thing
when we started the research project for developing
an open SRS. It was actually a bit frightening to
introduce a brand new technological system in
classes, but after a short trial period we learnt that
we mastered it without major problems. The more
comfortable we were on the system itself, the better
our use of SRS in classes became. Our experience
suggests that when a teacher is willing to take a leap
of faith (technology-wise), it can improve the way
we teach. We hope that in the future, we don’t allow
ourselves to be locked into a fixed pattern of
teaching, but rather can challenge ourselves with
new and exciting teaching methods.
Student evaluation: results from interview and
questionnaire: Students experienced the SRS as a
positive element in mathematics teaching. The
training was more fun, it increased involvement and
the students expressed that they had a more active
role in classes. Student quotes:
"It was a positive experience to use the SRS. It
turns the math into something positive. "
"Using the SRS is fun, and when something is
fun, it is easier to learn."
"Exciting new way to learn!"
"It's anonymous, so no one is afraid to answer
incorrectly."
SRS was also seen as positive in relation to
students' reflections on their own learning and
learning process. In other words, SRS caused them
to reflect more of their own learning. By answering
questions during class, they got a feedback about
their own learning process, and they could use this
information to plan which parts of the curriculum
they had to work harder with. Students emphasized
the importance of getting a concrete response, which
allows them to prepare and plan further learning.
Student quotes:
Sturla: "SRS tells me where I stand in relation to
the curriculum."
Robert: "You got like a concrete response as to
what was right and what was wrong."
Sturla: "And then you find out what you need to
practice harder."
After each vote, the students were encouraged to
participate in a larger class discussion, where the
various options and responses were discussed. The
student perceived these discussions as very useful in
relation to their own learning. The discussions gave
them the opportunity to discuss with several fellow
students and teacher. They received clarification
about the different options while at the same time
processing the subject material in a more active way.
Student quotes:
"I learn more using the SRS is such a way that
the answer options are reviewed and discusses in
class after the vote.”
"When you reviewed and discussed the various
options - what was wrong, and why – we felt like we
EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES IN EDUCATION OF ENGINEERS
389

were absorbing knowledge in very much the same
way that a sponge is absorbing water.”
The SRS was perceived as an integral part of
their mathematics classes. They did not feel that the
use of SRS came at the expense of ordinary
teaching. On the contrary: several of the students
think that the system should be used as a regular part
of their teaching in all subjects. Furthermore, the
iPods were perceived as simple-to-use voting
devices; the quiz questions were rated as
satisfactory, and the teachers seemed comfortable
with using SRS. As with previous evaluations, the
teacher's explanation after the vote is once again
highlighted as essential. So when a teacher initiates a
larger class discussion after the vote, it’s important
that also the teacher contributes to the discussion.
From the student side, it is important that the teacher
involved and give his testimony about what is right
and wrong, and why.
5 EMERGING NEW MOBILE
ASSESSMENT SOLUTIONS
Today, students are unable to immediately verify
their learning during tests/exams as feedback is
published after several days or even weeks. This is
in particular the case for learning skills in vocational
education and training, and in higher education in
Europe. Usually educational institutions don’t have
access to a high number of computer science labs
where it is possible to run digital multiple-choice
tests for all campus students. On the other hand,
within a few years a lot of students will have access
to cheap Smartphone’s with high-resolution pressure
sensitive screens.
An innovative mobile learning project (Done-It
2011) is currently developing a new evaluation
model and mobile technology solution, where
assessment and test results for a class are turned into
an active, creative and collaborative learning process
by the use of immediate feedback:
• Verification feedback led by a teacher: why is
this particular answer correct and why are the
others incorrect.
• An elaborate feedback discussion run by
students: the answers are displayed but they
don’t know which are the (in)correct ones.
• An elaborative feedback discussion led by
one student: the deviation from the correct
answer without addressing why this is correct
and the other ones are incorrect.
The mobile evaluation system for Smartphone’s
is going to extend the open SRS. It will give the
teacher a new tool, allowing him/her to either give
verification or elaborative feedback to individual
students or groups of students immediately after a
test. This is a key factor helping students to improve
their skills by the use of active collaborative
supported learning. Students will, when they still
remember the questions in the test, learn why the
correct answer is correct and why the other ones are
incorrect. Thus, mobile technology provides new
evaluation and testing criteria for education
and training
.
The mobile student evaluation system has the
potential to become a gateway to active learning for
students that may be used for in-class laboratory
experiments, but also for distance training purposes.
Each student uses Smartphone’s to answer and mark
multiple-choice tests with a number of questions. An
embedded automatic marking system is included.
The training method includes using cases and/or
experiments demonstrating what to do, how
it works, and what the deviation will look like. The
new open mobile technology based evaluation
services are going to be designed such that they
improve industrial certification processes. The
system design has a strong pedagogical focus, such
that the usage of the assessment services is merged
into the instruction and training process. The system
will collect the KEYS (the correct alternatives) and
the DISTRACTIONS (the wrong answers). It is
expected that the first prototype is ready for testing
at the end of 2011.
6 CONCLUSIONS
The results obtained in undergraduate engineering
programs in Norway and Sweden during testing the
prototype solution for a new type of online Student
Response Services (SRS) for next generation mobile
handheld devices with pressure sensitive screens,
e.g. like iPod Touch, iPhone and Smartphone’s, is
reported. The teacher collects and visualizes the
responses from class at a the digital blackboard or
the PC screen, by utilizing state of the art SRS
decision process solutions consisiting of a controll
interface and mobile devices which the students may
use for polling. Our results point out that the
students appreciate attributes like feedback on
learning. Furthermore, the students commonly
appreciate increased involvement and more peer
learning through group discussions. SRS rather than
raising their hands let individual responses stay
confidential. The open SRS has been tested and used
in classes with from 7 to 208 students in Norway
and Sweden. The experiences and feedback we have
obtained is to a large extent independent of the size
CSEDU 2011 - 3rd International Conference on Computer Supported Education
390

of the group.
The students provide positive feedback with
respect to increased engagement and motivation,
which is accordance with results reported in the
literature. Many students feel it become fun to attend
the lectures. They also point out that the SRS has
become an integrated part of the teaching practises,
since it is intuitive, easy and fast to operate by the
teacher. A majority of them feel it leads to improved
understanding of the curriculum, though we haven’t
any indication if this is the case during the final
examination process. Further research is required in
order to detect if it is any differences in using SRS
with respect to gender.
The new services may extend and replace
existing response systems where universities and
Vocational Educational Training institutions must
buy dedicated and expensive hardware tools, so
called "clickers" or electronic voting systems, in
order to provide feedback from students during
training sessions. The prototype of the open SRS
services, which were finalized in December 2010,
use the existing wireless network and may run on
widely available mobile, wireless multi touch
pressure sensitive hand held devices. It is also
possible to use it in parallel on PC/MAC/laptop.
They are constructed for easy integration and use on
digital blackboards, as well as to the story telling
provided by the teacher. The students use mobile
devices like iPod Touch, iPhone or their mobile
phone, to interact anonymously with the teacher
through online questionnaires and voting sessions.
The use of SRS has significant benefits:
Instructors get immediate feedback on how well the
students are paying attention to a lecture, while
students get instant feedback on their understanding
of key concepts. The online SRS is designed to help
teachers to enhance learning effects by:
• Breaking the monotony of a lecture and allow
the students to actively take part in the lecture.
• Increasing teacher-student interaction.
• Give teacher and students “real-time”
anonymous feedback on learning effect.
• Use modern, cheap and widely available mobile
devices that start quickly in order to merge
their usage into the teachers storytelling
ACKNOWLEDGEMENTS
These results have been obtained with support from
the European Commission. This publication reflects
the views only of the authors, and the Commission
cannot be held responsible for any use, which may
be made of the information contained therein.
REFERENCES
Berg, B. L. (2007). Chapter 5: Focus group Interviewing.
I: Berg, Bruce L. (2007): Qualitative Research
Methods for the Social Sciences, Pearson Education
Inc. (s. 144-170).
Charmaz, K. (2001). Grounded Theory. In: Smith, J. A.,
Harre, R, Langenhove, L. (eds). Rethinking methods
in psychology. London: Sage Publications. (s.27-49).
Cozby, P. C. (2003). Methods in Behavioral Research, 8
th
ed. McGraw-Hill companies 2003.
Johannessen, A., Tufte, P. A., og Kristoffersen, L. (2004).
Introduksjon til samfunnsvitenskapelige metoder.
Abstrakt forlag as 2004.
Stav J. B, Nielsen K., Hansen-Nygård G. and Thorseth T.
M., 2010, Experiences obtained with integration of
Student Response Systems for iPod Touch and iPhone
into e-learning environments, Electronic Journal of e-
Learning, Volume 8, Issue 2, p 179 - 190, ISSN 1479-
4403
The Done-IT project, 2011, online at www.histproject.no.
This is a LLP KA3-ICT project, which is cofounded
by the European Commission during the period 2011-
12.
The EduMECCA project, 2010, online at
www.histproject.no. This was a LLP KA3-ICT
project, contract 143545-2008-LLP-NO-KA3-
KA3MP, which was cofounded by the European
Commission during the period 2009-10.
EXPERIENCES WITH ONLINE RESPONSE TECHNOLOGIES IN EDUCATION OF ENGINEERS
391