LEARNING AND GENDER DIFFERENCES IN A
NARRATIVE-CENTERED LEARNING ENVIRONMENT
John L. Nietfeld, Lucy R. Shores and Kristin F. Hoffmann
Department of Curriculum and Instruction, North Carolina State University, Raleigh, NC, U.S.A.
Keywords: Narrative-centered learning environments, Gender differences.
Abstract: This study examined the performance of 8
th
grade students within a narrative-centered learning environment
(NLE) focused on science content. Students showed significant increases in science content knowledge after
gameplay. Performance in the NLE was related to higher science self-efficacy and also led to increased
presence, interest, and transfer of science concepts. No differences were found between genders for content
knowledge, however different variables predicted success by gender. In addition, males entered the study
with higher levels of reported gaming experience and showed early advantages for game score that
dissipated by the end of play. Further findings and implications for learning are discussed.
1 INTRODUCTION
Situating game-based learning exercises within a
narrative-centered approach is one strategy for
enhancing student learning and engagement. Stories
draw audiences into plots and settings, thereby
opening perceptual, emotional, and motivational
opportunities for learning. Fantasy contexts in
educational games have been shown to provide
motivational benefits for learning (Parker & Lepper,
1992). Narrative features such as pacing and tension
can introduce additional challenge to learning tasks
and contribute to student motivation. Establishing
concrete connections between narrative context and
pedagogical subject matter has also been said to
support the assimilation of new ideas in young
learners (Wells, 1986).
Narrative learning environments (NLEs)
leverage a range of techniques for providing
effective, engaging learning experiences. Multi-user
virtual environments such as Quest Atlantis (Barab
et al., 2007) and River City (Ketelhut et al., 2007)
use rich narrative settings to contextualize inquiry-
based science learning scenarios with strong social
and ethical dimensions. Several empirical studies
from this work have yielded promising learning
results that support the promise of NLEs in the
classroom.
Gender is also a variable of interest within
NLE’s. Many factors influence gender’s effects on
student performance, engagement, and interest in
game-based learning environments. These include
but are not limited to ability, beliefs, virtual gaming
preferences, and characteristics of the environment
itself. Characteristics that make video games
appealing to boys and girls have been noted as
consistently different (Cassell & Jenkins, 1998).
Girls tend to prefer story development, relationships,
and collaboration, whereas boys tend to prefer
competition and aggression (Cassell & Jenkins,
1998). Also, recent research has found that girls
report higher self-efficacy for science and self-
efficacy for self-regulated learning than boys at the
middle school level (Britner & Pajares, 2006).
Moreover, the gender of characters that student
interact with within virtual environments have been
shown to affect student attitudes about math and
science (Kim & Baylor, 2006). Therefore, the extent
to which these elements generalize and impact
performance in NLEs is an important line of
research.
A number of interesting gender differences have
emerged in recent studies using NLEs. For example,
males tend to engage in more off-task behaviors
(Rowe et al., 2009), females report higher levels of
presence (McQuiggan et al., 2008a), and females,
when allowed, record greater amounts of notes
(McQuiggan et al., 2008b). Further investigation of
these effects as well as other factors that influence
male and female performance motivation within this
context is imperative for the production of NLEs
that are effective for both genders.
140
L. Nietfeld J., R. Shores L. and F. Hoffmann K..
LEARNING AND GENDER DIFFERENCES IN A NARRATIVE-CENTERED LEARNING ENVIRONMENT.
DOI: 10.5220/0003349301400144
In Proceedings of the 3rd International Conference on Computer Supported Education (CSEDU-2011), pages 140-144
ISBN: 978-989-8425-49-2
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
1.1 CRYSTAL ISLAND
In our laboratory we have developed an inquiry-
based NLE called CRYSTAL ISLAND - OUTBREAK (see
Figure 1)
in the domain of microbiology for middle
school students. CRYSTAL ISLAND - OUTBREAK
features a science mystery set on a recently
discovered volcanic island where a research station
has been established to study the unique flora and
fauna. The student plays the protagonist attempting
to ultimately discover that the milk on the island is
carrying an unidentified infectious disease by
utilizing resources at the research station. The story
opens by introducing the student to the island and
the members of the research team. As members of
the research team fall ill, it is the student’s task to
discover the cause of the specific source of the
outbreak. She is free to explore the world and
interact with other characters while forming
questions, generating hypotheses, collecting data,
and testing the hypotheses. Throughout the mystery,
the student can walk around the island and visit the
infirmary, the lab, the dining hall, and the living
quarters of each member of the team. The student
can pick up and manipulate objects, talk with
characters to gather clues about the source of the
disease. Facts and clues revealed during the
student’s interaction can be stored in a virtual
factsheet. Some characters within the game serve as
local experts on certain pathogenic diseases. During
the student’s interactions with such expert
characters, in-game quizzes are presented to test the
student’s knowledge of the character’s specific
expertise. In the course of the adventure, the student
must gather enough evidence to correctly identify
that is in fact the milk has been contaminated with
E-Coli. To win the game, the student must have
submit a correctly filled factsheet with information
about the source object, disease, and treatment.
Figure 1: The CRYSTAL ISLAND - OUTBREAK environment.
2 METHOD
An experiment was conducted involving the entire
eighth grade population of a North Carolina middle
school. The research questions included: 1. What
effect does completing the C
RYSTAL ISLAND -
OUTBREAK mystery have upon learning? 2. How
does gender affect gameplay?
2.1 Participants
A total of 130 eighth grade students ranging in age
from 12 to 15 (M = 13.28, SD = 0.48) participated in
our study. 73 were male and 57 were female.
Approximately 3% of the participants were
American Indian or Alaska Native, 2% were Asian,
32% were African American, 13% were Hispanic or
Latino, and 50% were European American. The
study was conducted prior to students’ exposure to
the microbiology curriculum unit of the North
Carolina state standard course of study in their
regular classes.
2.2 Materials
Gaming Demographics Survey. A three-question,
researcher-developed gaming demographics survey
was completed by the participants prior to
interacting with C
RYSTAL ISLAND - OUTBREAK. The
survey asked students to rate how frequently they
play games and their perceived game-playing skill
on a 5-point Likert scale. Also, the students were
asked to estimate how many hours they spend
playing games per week.
Science Learning Self-efficacy. A science
learning self-efficacy inventory was given prior to
the students’ interaction with C
RYSTAL ISLAND -
OUTBREAK. The inventory consisted of eight items
answered on a 5-point Likert scale and has been
shown to be internally reliable (Nietfeld, Cao, &
Osborne, 2006).
Perceived Interest Questionnaire (PIQ).
Situational interest was measured immediately
following the students’ interaction with C
RYSTAL
ISLAND - OUTBREAK. The perceived interest
questionnaire consists of ten items measured on a 5-
point Likert scale and has been shown to be
internally reliable (Schraw, 1997)
Presence Questionnaire (PQ). The Presence
Questionnaire is a 32-question validated measure
containing several subscales, including
involvement/control, naturalism of experience and
quality of interface. The Presence Questionnaire is
intended to measure a user’s perceptions of the
LEARNING AND GENDER DIFFERENCES IN A NARRATIVE-CENTERED LEARNING ENVIRONMENT
141
transparency of the actual technology. In other
words, this measure provides a quantitative
assessment of how immersed the user feels in the
CRYSTAL ISLAND - OUTBREAK environment.
Microbiology domain knowledge was measured
with a researcher-constructed, 16-item multiple-
choice test based upon North Carolina Standard
Course of Study curriculum and intended to measure
domain-related material integrated within the
learning environment. The measure was broken
down into 8 fact-level and 8 application-level
questions and administered before the students
interacted with C
RYSTAL ISLAND - OUTBREAK and
again immediately following gameplay.
Microbiology Transfer Tasks. Two researcher-
developed transfer tasks were created to test
students’ abilities to transfer skills and domain
knowledge utilized in the learning environment to a
similar microbiology-related task. Students
completed the two, short-answer transfer tasks
immediately following the interaction.
2.3 Procedure
Students completed the self-efficacy for science and
gaming demographic inventories two weeks prior to
the intervention. On the day of the experiment,
students received a brief orientation to the C
RYSTAL
ISLAND - OUTBREAK environment as well as
completed the microbiology content test. Students
interacted with the learning environment until they
completed the game or 60 minutes elapsed.
Immediately following the interaction, students
completed the microbiology posttest, PQ, and the
PIQ.
In-game actions were recorded by the system
during gameplay that included virtual book reading,
consulting a microbiology field manual,
conversations with non-player characters,
performance on in-game mini-quizzes, testing
objects for contaminants, diagnosis worksheet usage,
and completion of seven major sub-goals: talking to
the camp nurse, interviewing the two expert
characters, talking to the camp cook, working with
the laboratory scientist, testing the contaminated
object with the lab equipment, and submitting a
correct diagnosis worksheet. Diagnosis worksheet
usage was quantified based upon a summed total for
each entry.
Students received the most points for a correct
entry, half points for an incorrect entry, and no
points for not using the space at all. Also, a running,
progressive score was calculated based upon student
efficiency and performance during the interaction.
3 RESULTS
3.1 What Effect does Completing the
CRYSTAL ISLAND - OUTBREAK
Mystery have on Learning?
A hierarchical regression was performed to predict
performance on the microbiology post-test. As
microbiology prior knowledge was found to be
highly correlated with post-test performance (r =
.42), microbiology pre-test score was entered into
the first block and overall presence, self-efficacy for
science, diagnosis worksheet performance, final in-
game score, and goals completed were entered into
the second block. Both models were found to be
significant (F
(1, 126)
= 30.94, p < .001, R
2
= .20; F
(6,
121)
= 11.41, p < .001, R
2
= .36). For the second
model, only prior knowledge and number of goals
completed were found to be significant predictors.
Given these results, comparisons were made
between students who completed the C
RYSTAL
ISLAND - OUTBREAK mystery and those who did not
for variables related to self-regulated learning and
content learning gains. Overall, 58 students
completed the mystery. An ANCOVA controlling
for prior knowledge found students who completed
the C
RYSTAL ISLAND - OUTBREAK mystery scored
significantly better on the microbiology post-test
than those students who did not complete the
scenario (F
(2, 27)
= 26.67, p < .001). Moreover,
students who completed the mystery reported
significantly higher levels of presence (t
(126)
= 3.65,
p < .001), situational interest (t
(128)
= 2.45, p < .05),
diagnosis worksheet total (t
(128)
= 14.14, p < .001),
and performance on the transfer tasks (respectively,
t
(128)
= 2.49, p < .05; t
(128)
= 2.95, p < .01). Finally,
those who completed the mystery also had higher
initial levels of self-efficacy for science (t
(128)
= 4.41,
p < .001) than those who did not finish.
3.2 How does Gender Affect
Gameplay?
Prior to interacting with the environment, males
reported playing games more frequently (t
(128)
=
7.91, p < .01) and higher levels of perceived
gameplay skill (t
(128)
= 7.39, p < .01) than females.
During gameplay, males completed significantly
more goals (t
(128)
= 2.64, p < .01) and more
accurately and comprehensively completed the
diagnosis worksheet (t
(128)
= 2.14, p < .05) than
females. Significant gender differences in favour of
males were only found for the mandatory diagnosis
CSEDU 2011 - 3rd International Conference on Computer Supported Education
142
section (t
(128)
= 2.33, p < .05) not the hypothesis,
testing, or symptoms section. Interestingly, when
number of hours playing games was used as a
covariate there were no remaining gender
differences for goals completed or for the diagnosis
section of the diagnosis worksheet. Also, males
tended to receive significantly higher in-game scores
than females at the beginning but not at the end of
the interaction (see Table 1).
Table 1: Score by gender and minutes played.
Time
(minutes)
Males
(M, SD)
Females
(M, SD)
5 34.78 (24.42)** 23.26 (18.86)**
10 90.73 (98.32)* 56.12 (66.56)*
15 115.82 (110.89) 93.05 (100.38)
20 170.93 (180.81)* 115.19 (117.45)*
25 210.38 (239.39)* 135.07 (115.71)*
30 270.10 (320.63)* 155.95 (161.56)*
35 294.04 (395.15)* 181.11 (217.39)*
40 328.33 (385.07) 236.04 (287.46)
45 378.89 (413.54) 278.51 (329.26)
50 376.56 (419.75) 301.04 (354.21)
55 428.47 (434.08) 374.61 (393.05)
60 462.21 (442.19) 355.09 (399.79)
Note: * = p < .05, ** = p < .01
Of the 73 males that interacted with CRYSTAL
ISLAND - OUTBREAK 36 completed the mystery. For
the males, those who completed the mystery had
significantly higher levels of prior knowledge (t
(71)
=
3.05, p < .01), reported higher levels of overall
presence (t
(71)
= 2.26, p < .05), situational interest
(t
(71)
= 2.26, p < .05), self-efficacy for science (t
(71)
=
3.30, p < .01), diagnosis worksheet total (t
(71)
= 9.74,
p < .001), performance on the microbiology post-test
(t
(71)
= 4.41, p < .001) and performance on the
transfer tasks (t
(71)
= 2.45, p < .01; t
(71)
= 3.06, p <
.01). Of the 57 females, 22 of them completed the
C
RYSTAL ISLAND - OUTBREAK mystery. Considering
only the female population, those who completed the
mystery reported significantly higher levels of
overall presence (t
(54)
= 2.61, p < .05), self-efficacy
for science (t
(55)
= 2.84, p < .01), diagnosis
worksheet performance (t
(55)
= 10.25, p < .001), and
performance on the microbiology post-test (t
(55)
=
3.28, p < .01). No differences were found between
groups with microbiology prior knowledge.
Following the interaction with C
RYSTAL ISLAND -
OUTBREAK, males reported significantly higher
levels of overall presence than females (t (126) =
2.88, p < .01). Specifically, males reported
significantly higher levels of presence for the
involvement/control subscale (t (126) = 3.30, p <
.01) than females, but not for the naturalism or
interface subscales. Finally, comparisons of
microbiology prior knowledge (M = 6.26, SD =
1.99) and microbiology posttest knowledge (M =
8.66, SD = 2.98) revealed a significant increase in
scores overall (t
(129)
= 9.81, p < .001). However, no
differences were found between genders for content
learning gains, prior knowledge, or posttest
performance on microbiology. Both genders showed
significant increases in content knowledge (for boys,
t
(72)
= -7.76, p < .001; for girls, t
(56)
= -6.02, p <
.001).
4 DISCUSSION
Given the emerging field of on-line and computer-
based learning it is essential that appropriate
mechanisms are in place to encourage effective and
self-regulated learning for all students. The current
study reports on one particular attempt to examine
overall learning and differences associated with
gender within a game-based NLE.
In sum, playing C
RYSTAL ISLAND - OUTBREAK
led to significant content learning gains.
Performance was predicted both by prior knowledge
and goals completed within the game. The
pedagogical approach built into the character
interaction and student exploration was successful as
evidenced by the fact that students who completed
the mystery scored higher on the science posttest.
Moreover, these students reported higher levels of
engagement (presence, situational interest), strategy
use (diagnosis worksheet), and the ability to transfer
their knowledge than their peers who did not
complete the mystery.
Interesting gender differences also emerged
during interaction with C
RYSTAL ISLAND -
OUTBREAK. Boys who solved the mystery had
different predictors of success (e.g., prior
knowledge) than girls that finished. These findings
suggest girls and boys may be motivated to perform
well in these environments for different reasons.
Also, boys entered the study with higher perceived
skill and experience with computer games than girls.
LEARNING AND GENDER DIFFERENCES IN A NARRATIVE-CENTERED LEARNING ENVIRONMENT
143
Accordingly, they showed an advantage over girls in
the early stages of C
RYSTAL ISLAND - OUTBREAK
with respect to performance (goals completed) but
closed this gap to the point of statistically
nonsignificant differences by the end of play. Boys
still reported a higher level of presence after
gameplay but no differences were found between
genders on content knowledge. Boys also showed
advantages for overall game score and performance
on the diagnostic facet of the diagnosis worksheet
yet these advantages disappear when controlling for
prior experience playing games.
It is important to continue to identify such
differences as reported above in order to develop
gender-inclusive learning environments that promote
learning and motivation for all users. From a self-
regulated learning standpoint it will be important to
study means by which to scaffold strategy use,
metacognitive regulation, and motivational control
for all students in game-based environments such as
C
RYSTAL ISLAND OUTBREAK. Future research
should take up this challenge of integrating effective
learning scaffolds within such engaging
environments because they offer a unique
opportunity to customize learning for the individual
learner in ways traditional learning environments
cannot.
ACKNOWLEDGEMENTS
The authors wish to thank members of the
IntelliMedia lab for their assistance, Omer
Sturlovich and Pavel Turzo for use of their 3D
model libraries, and Valve Software for access to the
Source
TM
engine and SDK. This research was
supported by the National Science Foundation under
Grants REC-0632450, IIS-0757535, DRL-0822200,
IIS-0812291, and CNS-0540523. Any opinions,
findings, and conclusions or recommendations
expressed in this material are those of the authors
and do not necessarily reflect the views of the
National Science Foundation.
REFERENCES
Barab, S., Dodge, T., Tuzun, H., Job-Sluder, K., Jackson,
C., Arici, A., Job-Sluder, L., Carteaux, R., Gilbertson
Jr., J. & Heiselt, C. (2007). The Quest Atlantis project:
A socially-responsive play space for learning. In B. E.
Shelton & D. Wiley (Eds.) The Educational Design
and Use of Simulation Computer Games, (pp. 159-
186), Rotterdam, The Netherlands: Sense Publishers.
Britner, S. L., and Pajares, F. (2006). Sources of science
self-efficacy beliefs of middle school students. Journal
of Research in Science Teaching, 43, 485–499.
Cassell, J. & Jenkins, H. (1998). From Barbie to Mortal
Kombat: Gender and computer games. Cambirdge,
MA: MIT Press.
Egri, L. (1960). The Art of Dramatic Writing. New York,
NY: Simon and Schuster.
Harp, S. and Mayer, R. (1998). How seductive details do
their damage: A theory of cognitive interest in science
learning. Journal of Educational Psychology, 90(3),
414-434.
Ketelhut, D., Dede, C., Clarke, J., Nelson, B. & Bowman,
C. (2007). Studying situated learning in a multi-user
virtual environment. In E. Baker, J. Dickieson, W.
Wulfeck and H. O'Neil (Eds.) Assessment of Problem
Solving Using Simulations, (pp. 37-58), Lawrence
Erlbaum, Mahwah, NJ.
Kim, Y., Baylor, A. (2006). A social-cognitive framework
for pedagogical agents as learning companions.
Educational Technology Research & Development,
54, 569-590.
McQuiggan, S, Rowe, J., Lee, S., & Lester, J. (2008a).
Story-Based Learning: The Impact of Narrative on
Learning Experiences and Outcomes. In Proceedings
of the Ninth International Conference on Intelligent
Tutoring Systems, Montreal, Canada, pp. 530-539.
McQuiggan, S., Goth, J., Ha, E. Rowe, J. & Lester, J.
(2008b). Student Note-Taking in Narrative-Centered
Learning Environments: Individual Differences and
Learning Outcomes. In Proceedings of the Ninth
International Conference on Intelligent Tutoring
Systems (ITS-2008), Montreal, Canada, 510-519.
Nietfeld, J. L., Cao, L., & Osborne, J. W. (2006). The
effect of distributed monitoring exercises and feedback
on performance and monitoring accuracy.
Metacognition and Learning, 2, 159-179.
Parker, L. E., & Lepper, M. R. (1992). The effects of
fantasy contexts on children's learning and motivation:
Making learning more fun. Journal of Personality and
Social Psychology, 62, 625-633.
Schraw, G. (1997). Situational Interest in Literary Text.
Contemporary Educational Psychology. 22, 436-456.
Wells, C. (1986). The Meaning Makers: Children
Learning Language and Using Language to Learn,
Portsmouth, NH: Heineman
CSEDU 2011 - 3rd International Conference on Computer Supported Education
144