See discussions, stats, and author profiles for this publication at: https://www.researchgate.

net/publication/227277701

Integrating Learning and Engagement in Narrative-Centered Learning

Environments

Conference Paper · May 2010

DOI: 10.1007/978-3-642-13437-1_17 · Source: dx.doi.org

CITATIONS READS

95 381

4 authors, including:

Jonathan P. Rowe Lucy R. Shores

North Carolina State University North Carolina State University
100 PUBLICATIONS 2,620 CITATIONS 15 PUBLICATIONS 684 CITATIONS

SEE PROFILE SEE PROFILE

Bradford W. Mott
North Carolina State University
156 PUBLICATIONS 3,538 CITATIONS

SEE PROFILE

All content following this page was uploaded by Lucy R. Shores on 21 May 2014.

The user has requested enhancement of the downloaded file.

 Integrating Learning and Engagement in
Narrative-Centered Learning Environments

Jonathan P. Rowe1, Lucy R. Shores, Bradford W. Mott and James C. Lester

Department of Computer Science, North Carolina State University, Raleigh, NC 27695

{jprowe, lrshores, bwmott, lester}@ncsu.edu

Abstract. A key promise of narrative-centered learning environments is the

ability to make learning engaging. However, there is concern that learning and
engagement may be at odds in these game-based learning environments and
traditional learning systems. This view suggests that, on the one hand, students
interacting with a game-based learning environment may be engaged but
unlikely to learn, while on the other hand, traditional learning technologies may
promote deep learning but provide limited engagement. This paper presents
findings from a study with human participants that challenges the view that
engagement and learning need be opposed. A study was conducted with 153
middle school students interacting with a narrative-centered learning
environment. Rather than finding an oppositional relationship between learning
and engagement, the study found a strong positive relationship between
learning outcomes and increased engagement. Furthermore, the relationship
between learning outcomes and engagement held even when controlling for
students’ background knowledge and game-playing experience.

Keywords: Narrative-Centered Learning Environments, Game-Based

Learning, Engagement, Situational Interest, Presence.

1 Introduction

Narrative-centered learning environments show significant potential for providing

engaging learning experiences that are tailored to individual students. By leveraging
the motivational characteristics of narrative and games, along with the adaptive
pedagogy of intelligent tutoring systems, narrative-centered learning environments
offer a promising platform for students to acquire enhanced problem solving, strategic
and analytical thinking, decision making, and other twenty-first century skills [1,2].
As an active and growing area of research, narrative-centered learning environments
are under investigation in a range of domains, including language learning [3], anti-
bullying education [4], and middle school science [5].
Despite the ITS community’s growing interest in narrative-centered learning
environments, there is concern that the narrative and gameplay elements of these

1
Corresponding Author: Jonathan P. Rowe, Department of Computer Science, North Carolina
State University
systems may not contribute to improved learning outcomes. This belief stems in part
from a view that gains in engagement achieved by a narrative-centered learning
environment are primarily diversionary [6,7]. The view suggests that while students
may become engaged in the rich virtual environments or compelling characters
provided by many narrative-centered learning environments, the reasons for
engagement are tangential to learning [8]. In this view, there is a tradeoff between
learning and engagement, suggesting that on the one hand, students interacting with a
game-based learning environment may be engaged but unlikely to learn, and on the
other hand, traditional learning technologies may promote deep learning but provide
limited engagement [7,9,10].
This paper challenges the above view by presenting findings from an empirical
study investigating the relationship between learning and engagement in a narrative-
centered learning environment. This work assesses engagement by considering a
number of factors hypothesized to be associated with engagement, including
presence, situational interest, avoidance of “gaming the system,” and problem-solving
efficiency. Findings are presented from a study with 153 eighth-grade students
interacting with CRYSTAL ISLAND, a narrative-centered learning environment for
middle school microbiology. Results show that students who experienced higher
levels of engagement during interactions with the CRYSTAL ISLAND environment
achieved improved learning outcomes. Notably, this result is independent of students’
prior microbiology knowledge and gaming experience.

2 Background

Narrative-centered learning environments embed educational content and activities in

story-centric, problem-solving scenarios and interactive virtual worlds. Multi-user
virtual environments such as Quest Atlantis [11] and River City [5] use rich narrative
settings to contextualize inquiry-based science learning scenarios with strong social
and collaborative elements. Other work has utilized interactive narrative generation
and agent behavior planning to foster adaptive narrative experiences that are
pedagogically effective and tailored to individual students [4,12,13]. A key
motivation for this line of work is the development of systems that simultaneously
promote deep learning and high engagement.
For years, devising techniques for detecting and measuring student engagement has
been an important area of investigation within the ITS community [14,15,16]. A
number of techniques have been proposed to assess related factors such as student
motivation [14,17] and affective states such as flow [18]. Other work has sought to
devise automated models for detecting symptoms of disengagement, namely, off-task
behavior [16,19]. One of the most salient examples of off-task behavior is “gaming
the system,” where students exploit elements of a learning environment interface to
progress through a lesson without having mastered the associated content [19].
Engagement in narrative-centered learning environments can take several forms,
including engagement in the learning scenario and engagement in tangential or
aesthetic elements of the virtual environment [8]. Narrative-centered learning
environments often provide vast interactive environments, realistic physics, and
engaging characters, which may risk introducing seductive details into learning
experiences [20]. Seductive details have the potential to distract, disrupt, or divert
students’ attention from pedagogical objectives and to reduce students’ time-on-task.
To adequately investigate the complex nature of engagement in narrative-centered
learning environments, assessments of engagement should consider a variety of
factors. For example, students’ problem-solving efficiency within the virtual
environment is likely an indication of engagement, as well as resistance to seductive
details. Off-task behavior such as “gaming the system” can be viewed as evidence of
disengagement from a learning environment. In addition to these factors, we
hypothesize that students’ situational interest in a narrative-centered learning
experience, as well as their sense of presence in the narrative environment, are likely
contributors to engagement.
Situational interest is characterized by varying lengths of concentrated attention
coupled with affective reaction activated during a particular time period by certain
environmental stimuli [21,22]. Studies have shown that situational interest directed
towards an instructional task can influence cognitive performance [23] and facilitate
deeper learning [24]. Also, learning tasks and environments that yield significant
situational interest have been shown to benefit students who have previously been
disengaged in similar learning activities [25]. However, situational interest is not
exclusive to learning tasks; game design and adaptive scaffolding should encourage
interest in on-task actions, rather than interest in purely aesthetic or gameplay features
of narrative-centered learning environments [8].
Presence contributes to the goal of transparency in technology-mediated
interactions [26]. Although there has been substantial debate on formal definitions,
there is a general consensus that presence describes a user’s sense of “being there”
when interacting with a mediated environment [27,28]. Presence has been
alternatively defined as “the subjective experience of being in one place or
environment, even when one is physically situated in another” [29]. It is related to
students’ sense of transportation into a story, which is an important contributor to the
engaging quality of narratives. Presence is distinguished from related concepts such
as immersion and involvement. Immersion generally refers to the extent and nature of
technology-provided sensory stimuli; it is often associated with the pervasiveness and
fidelity of visual, auditory, olfactory, and tactile inputs [28]. Involvement refers to the
degree of attention and meaning devoted to some set of stimuli [29].

3 CRYSTAL ISLAND

Now in its third major iteration, CRYSTAL ISLAND (Figure 1) is a narrative-centered

learning environment built on Valve Software’s Source™ engine, the 3D game
platform for Half-Life 2. The curriculum underlying CRYSTAL ISLAND’s mystery
narrative is derived from the North Carolina state standard course of study for eighth-
grade microbiology. The environment is designed as a supplement to classroom
instruction. Students play the role of the protagonist, Alyx, who is attempting to
discover the identity and source of an infectious disease plaguing a newly established
research station. Several of the team’s members have fallen gravely ill, and it is the
 Figure 1. CRYSTAL ISLAND narrative-centered learning environment.

student’s task to discover the nature and cause of the outbreak.

CRYSTAL ISLAND’s narrative takes place in a small research camp situated on a
recently discovered tropical island. As students explore the camp, they investigate the
island’s spreading illness by forming questions, generating hypotheses, collecting
data, and testing hypotheses. Throughout their investigations, students interact with
virtual characters offering clues and relevant microbiology facts via multimodal
“dialogues” delivered through student menu choices and characters’ spoken language.
The dialogues’ content is supplemented by virtual books, posters, and other resources
encountered in several of the camp’s locations. As students gather useful information,
they have access to a personal digital assistant to take and review notes, consult a
microbiology field manual, communicate with characters, and report progress in
solving the mystery. To solve the mystery, students complete a diagnosis worksheet
to manage their working hypotheses and record findings about patients’ symptoms
and medical history, as well as any findings from tests conducted in the camp’s
laboratory. Once a student enters a hypothesized diagnosis, cause of illness, and
treatment plan into her diagnosis worksheet, the findings are submitted to the camp
nurse for review and possible revision.
To illustrate the behavior of CRYSTAL ISLAND, consider the following scenario.
Suppose a student has been interacting with non-player characters in the storyworld
and learning about infectious diseases. In the course of having members of the
research team become ill, she has learned that a pathogen is an illness that can be
transmitted from one organism to another. As she concludes her introduction to
infectious diseases, she learns from the camp nurse that the mystery illness seems to
be coming from food items the sick members recently ate. Some of the island’s
characters are able to help identify food items and symptoms that are relevant to the
scenario, while others provide helpful microbiology information. The student is
careful to take notes recording information about bacteria and viruses in her personal
digital assistant, and corroborates these notes with information contained in her
microbiology field manual. After forming several hypotheses about which food items
may be sickening the team members, the student discovers through a series of tests
that a container of unpasteurized milk in the dining hall is contaminated with bacteria.
By combining this information with her knowledge about the characters’ symptoms
and recent dining habits, the student infers that the disease is E. coli, for which ample
rest is the best immediate treatment plan. She records her findings in a diagnosis
worksheet, and submits them to the camp nurse for review and implementation.

4 Empirical Study

An experiment involving human participants was conducted with the entire eighth
grade population of a North Carolina middle school. The primary goal of the
experiment was to investigate the impact of different scaffolding techniques on
learning and engagement in the CRYSTAL ISLAND narrative-centered learning
environment. However, no condition effects were observed for either learning or
engagement. This paper’s findings come from a secondary analysis of the data, which
considers the experiment’s conditions as a whole.

4.1 Participants

A total of 153 eighth grade students ranging in age from 12 to 15 (M = 13.3, SD =

0.48) interacted with the CRYSTAL ISLAND environment during the study. Three of the
participants were eliminated due to incomplete data. Among the remaining students,
80 were male and 70 were female. Approximately 3% of the participants were
American Indian or Alaska Native, 2% were Asian, 32% were African American,
13% were Hispanic or Latino, and 50% were White. Although CRYSTAL ISLAND is
ultimately intended to be used concurrently with classroom coverage of an associated
microbiology unit, scheduling issues necessitated that the study be conducted prior to
students being exposed to the microbiology curriculum unit of the North Carolina
state standard course of study in their regular classes.

4.2 Materials and Apparatus

Students completed an online demographic survey and CRYSTAL ISLAND curriculum

test prior to the intervention. The curriculum test consisted of 16 multiple-choice
questions created by an interdisciplinary team of researchers. The test consisted of
eight factual and eight application questions assessing students’ knowledge of
pathogens, select diseases, and the scientific method.
Post-experiment materials were completed immediately following the CRYSTAL
ISLAND intervention. Included in these materials were the same curriculum test used
in the pre-experiment, a variation of the Perceived Interest Questionnaire [30], and the
Presence Questionnaire [29]. The interest scale was adapted from measures used by
Schraw to examine within-subject relationships with learning outcomes [30]. The
measure consists of ten Likert items measuring students’ situational interest related to
CRYSTAL ISLAND. To illustrate the scale, example items include the following: “I got
absorbed playing CRYSTAL ISLAND without trying to,” and “CRYSTAL ISLAND really
grabbed my attention.” The Presence Questionnaire (PQ) is a validated measure
containing several subscales, including involvement/control, naturalism of experience
and quality of interface [29]. The natural subscale is intended to assess the student’s
perception of the virtual environment’s consistency with reality, in terms of
locomotion and nature of the interaction. The interface quality subscale indicates how
seamlessly the control and display devices are integrated into the interactive
experience. Example items include the following: “How compelling was your sense
of moving around inside the virtual environment,” “How much did your experiences
in the virtual environment seem consistent with your real-world experiences,” and
“How much did the visual display quality interfere or distract you from performing
assigned tasks or required activities?”
In addition to pre- and post-experiment subjective measures, the CRYSTAL ISLAND
software calculated a numerical score to assess students’ progress and efficiency in
completing the science mystery. Students could view their scores in the upper left
corner of their screens throughout their interactions with the software. The score
consisted of a weighted sum of gameplay sub-scores, and incorporated time taken to
accomplish important goals, students’ ability to demonstrate microbiology content
knowledge, and evidence of careful hypothesis formulation. Students were penalized

Table 1. Point values for calculation of final game score.

for any attempt to “game the system” by repeatedly submitting incorrect diagnoses to
the camp nurse or guessing on content knowledge quizzes. Details of the score’s
calculation are shown in Table 1. As an objective measure assessing students’
understanding of the curricular content and performance at completing the CRYSTAL
ISLAND mystery, students’ final score is treated as a measure to investigate
engagement alongside subjective measures of presence and situational interest.

4.3 Participant Procedure

Participants entered the experiment room having completed the majority of pre-test
materials one week prior to the intervention. Students were initially provided general
details about the CRYSTAL ISLAND mystery and game controls during an introductory
presentation by a researcher. After the presentation, students completed the remaining
pre-test materials and received several CRYSTAL ISLAND supplementary documents.
These materials consisted of a CRYSTAL ISLAND backstory and task description, a
character handout, a map of the island, and an explanation of the game’s controls.
Participants were given 60 minutes to work on solving the mystery. Solving the
mystery consisted of several objectives including: learning about pathogens, viruses,
and bacteria; compiling the symptoms and recent history of the sick researchers;
recording details about diseases believed to be potentially afflicting the team
members; testing a variety of possible sources for the disease; and reporting the
solution—including cause, source, and treatment—to the camp nurse. Immediately
after solving CRYSTAL ISLAND’s science mystery, or 60 minutes of interaction,
participants completed the post-experiment questionnaires. Completion of post-
experiment materials took no longer than 30 minutes for participants. In total,
sessions lasted up-to 120 minutes.

5 Results

An investigation of learning found that on average, students answered 2.35 (SD =

2.75) more questions correctly on the post-test than they did on the pre-test. Matched
pairs t-tests (comparing post-test to pre-test scores) indicated that students’ learning
gains were significant, t(149) = 10.49, p < .001.

5.1 Learning and Engagement

Examining factors believed to reflect engagement and students’ understanding of the

curriculum, Pearson correlations indicated significant relationships between
microbiology background knowledge and presence, r = .17, p < .05, and final score,
r = .28, p < .01. Similar relationships were found between microbiology post-test
scores and presence, r = .295, p < .01, final score, r = .445, p < .01, and situational
interest, r = .239, p < .01. To more closely investigate the relationships between
learning and engagement, additional analyses controlling for background knowledge
were conducted.
 A partial correlation controlling for pre-test score found significant relationships
between microbiology post-test scores and two of our engagement measures,
presence, r = .25, p < .01, and final game score, r = .38, p < .01. The same type of
analysis also found a borderline significant relationship between situational interest
and post-test score, r = .15, p < 0.1. Offering further evidence for a connection
between learning and engagement in CRYSTAL ISLAND, a linear regression indicated
that microbiology background knowledge, presence, and final score were all
significant predictors of performance on the microbiology post-test, and the model as
a whole was significant, R2 = .33, F(3, 143) = 23.46, p < .001.
As a supplement to these findings, further analyses were conducted to determine
whether similar relationships held for the involved/control subscale of the Presence
Questionnaire, which provides a more specific measure of involvement in the
environment. A partial correlation controlling for microbiology background
knowledge revealed significant relationships between the involved/control subscale
and final score, r = .376, p < .01, situational interest, r = .181, p < .05, and
microbiology post-test performance, r = .334, p < .01.

5.2 Engagement and Individual Differences

Additional analyses were conducted to determine whether particular subpopulations

experienced different levels of engagement while interacting with the CRYSTAL
ISLAND environment. Pearson correlations indicated significant relationships between
game-playing frequency and presence, r = .269, p < .01, as well as between self-
perceived game-playing skill and presence, r = .178, p < .05. Game-playing frequency
was found to have a significant relationship with the PQ’s involved/control subscale, r
= .327, p < .01, as did game-playing skill, r = .211, p < .05. A significant relationship
between game-playing frequency and the PQ’s natural subscale was observed, r = .17,
p < .05. No significant relationships were found between game-playing frequency
and the PQ’s interface quality subscale, nor between game-playing skill and the
naturalism of experience or interface quality subscales. No significant correlation was
found between either of the game-playing demographics and situational interest, or
between either of the game-playing demographics and final game score.
A regression analysis was conducted to examine the simultaneous contributions of
game-playing frequency, microbiology background knowledge, presence, and final
score on microbiology post-test scores. The overall model was significant, R2 = .327,
F(4, 136) = 16.535, p < .01, but only microbiology background knowledge, presence,
and final score were significant predictors of post-test performance, not game-playing
frequency. A similar regression analysis was conducted to examine the contributions
of self-assessed game-playing skill, microbiology background knowledge, presence,
and final score on microbiology post-test scores. The overall model was significant,
R2 = .33, F(4, 136) = 16.750, p < .01, but again only microbiology background
knowledge, presence, and final score were significant predictors, not self-assessed
game-playing skill.
Examining gender, an independent samples t-test analyzing the relationship
between gender and presence found that males tended to feel more present in the
environment than females, t(139) = 3.01, p < .01. Similar results were found for the
Table 2. Raw scores by gender on content knowledge, situational interest, and presence
questionnaires.

involved/control subscale of the Presence Questionnaire: an independent samples t-

test analyzing the relationship between gender and the involved/control measure
found that males tended to feel significantly more involved/control when interacting
with CRYSTAL ISLAND than females, t(140) = 2.96, p < .01. Males also tended to rate
the interface quality more highly, t(140) = 1.97, p < .01, but no gender effect was
found on the PQ’s natural subscale. Table 2 displays raw scores, by gender, for each
of the content knowledge, situational interest, and presence measures.
Significant differences were observed between genders for gaming demographics.
Males reported significantly higher ratings for self-perceived game-playing skill, F(1,
143) = 57.49, p < .001, and reported playing games more frequently, F(1, 143) =
60.15, p < .001, than females. Although males tended to feel more present in
CRYSTAL ISLAND, an analysis of covariance controlling for game-playing frequency
found no significant effect of gender on presence, F(1, 138) = 2.01, p = .158.
Significant differences were not found between genders for situational interest or final
score.
A linear regression considering only the female population yielded a significant
model for predicting microbiology post-test performance, R2 = .25, F(2, 62) = 10.12,
p < .01, but only microbiology background knowledge and final score were
significant predictors, not presence.  

6 Discussion

The findings indicate that student engagement with the CRYSTAL ISLAND environment
was associated with improved learning outcomes. Results showed a significant
relationship between students’ pre-test scores and presence, as well as between pre-
test scores and final game scores. This suggests that students who demonstrated
greater prior content knowledge tended to become more engaged with the narrative
environment. However, all three measures for engagement—presence, situational
interest, and final game score—were found to be significantly associated with post-
test score, independent of pre-test score. These findings suggests that students who
were more engaged with the CRYSTAL ISLAND narrative environment tended to
experience greater learning gains, regardless of prior knowledge. The findings
contrast with perspectives that place engagement and learning at odds with one
another in narrative-centered learning environments. Further, analyses found no
relationships between game-playing experience and learning. This finding suggests
that both gamers and non-gamers who were engaged in the narrative-centered
learning experience achieved improved learning outcomes. Students can be
productively engaged in a narrative-centered learning environment, and this
relationship is independent of prior knowledge or game-playing experience.
The findings suggest that engagement and learning need not be at odds in
narrative-centered learning environments, and may in fact reinforce one another. We
hypothesize that well-designed story and gameplay elements may contribute to this
synergistic relationship. However, poorly designed story and gameplay elements may
detract from both engagement and learning by introducing seductive details and
promoting off-task behavior. Additional investigation is needed to determine which
elements of narrative-centered learning environments are most closely associated with
learning and engagement. These efforts will contribute to the development of models
to automatically detect student engagement and learning during narrative-centered
learning interactions.
Interesting findings were also observed concerning the effects of gender and game-
playing experience on presence. Males tended to be more present during CRYSTAL
ISLAND interactions than females. An initial interpretation might be that the game was
better designed for males than females. However, a significant correlation was also
observed between presence and game-playing experience. Furthermore, males tended
to have significantly greater game-playing experience than females. An ANCOVA
suggested that game-playing experience, not gender, may be the more predominant
factor associated with presence. These findings raise important questions about the
effective design of narrative-centered learning environments for males and females, as
well as gamers and non-gamers. However, additional investigation is necessary to
better understand these relationships.
Extending studies of narrative-centered learning interactions beyond individual
sessions is an essential next step for understanding the relationship between
engagement and learning in narrative-centered learning environments. Studies
spanning multiple sessions, along with in-class integration, are important to assess
how engagement can be sustained over time with narrative-centered learning
environments, how long-term engagement is related to deep learning and transfer, and
whether engagement can impact student attitudes and self-efficacy. To accommodate
these larger scale studies, devising additional subjective and objective measures for
engagement beyond those used in this work will also be important.

7 Conclusions

Narrative-centered learning environments offer a promising vehicle for delivering

experiences that are both effective and engaging. To investigate the hypothesis that
learning and engagement need not be in opposition in narrative-centered learning
environments, an empirical study was conducted with middle school students
interacting with the CRYSTAL ISLAND learning environment. It was found that
increased engagement was associated with improved learning outcomes, independent
of students’ prior content knowledge or game-playing experience. As narrative-
centered learning environments mature, it will become increasingly important to
understand how students can most effectively interact with them, and what role
narrative and game features can play in scaffolding learning and realizing sustained
engagement.

Acknowledgements

The authors wish to thank members of the IntelliMedia Group of North Carolina State
University for their assistance, Omer Sturlovich and Pavel Turzo for use of their 3D
model libraries, and Valve Software for access to the SourceTM engine and SDK. This
research was supported by the National Science Foundation under Grants REC-
0632450, IIS-0757535, DRL-0822200, IIS-0812291, and CNS-0540523. Any
opinions, findings, and conclusions or recommendations expressed in this material are
those of the authors and do not necessarily reflect the views of the National Science
Foundation.

References

1. Gee, J.P.: What Video Games Have to Teach Us about Learning and Literacy. Palgrave
Macmillan, New York, NY (2003)
2. Shaffer, D.W.: How Computer Games Help Children Learn. Palgrave Macmillan, New
York, NY (2006)
3. Johnson, W.L.: Serious use of a Serious Game for Language Learning. In: 13th
International Conference on Artificial Intelligence in Education, pp. 67–74. (2007)
4. Aylett, R., Louchart, S., Dias, J., Paiva, A., Vala, M.: FearNot! An Experiment in
Emergent Narrative. In: 5th International Conference on Intelligent Virtual Agents, pp.
305–316. (2005)
5. Ketelhut, D.: The Impact of Student Self-Efficacy on Scientific Inquiry Skills: An
Exploratory Investigation in River City, a Multi-User Virtual Environment. Journal of
Science Education and Technology. 16, 99–111 (2007)
6. Hallinen, N., Walker, E., Wylie, R., Ogan, A., and Jones, C. (2008) I was playing when I
learned: A Narrative Game for French Aspectual Distinctions. In: Workshop on
Intelligent Educational Games at the 14th International Conference on Artificial
Intelligence in Education, pp. 117–120. (2009)
7. Rai, D., Heffernan, N., Gobert, J., Beck, J.: Mily’s World: Math game involving
authentic activities in visual cover story. In: Workshop on Intelligent Educational Games
at the 14th International Conference on Artificial Intelligence in Education, pp. 125–128.
(2009)
8. Rowe, J., McQuiggan, S., Robison, J., Lester, J.: Off-Task Behavior in Narrative-
Centered Learning Environments. In: 14th International Conference on Artificial
Intelligence and Education, pp. 99–106. (2009)
9. Prensky, M.: Digital Game-based Learning. McGraw-Hill, New York (2001)
10. McNamara, D., Jackson, G., Graesser, A.: Intelligent tutoring and games (ITaG). In:
Workshop on Intelligent Educational Games at the 14th International Conference on
Artificial Intelligence in Education, pp. 1–10. (2009)
11. Barab, S., Dodge, T., Tuzun, H. et al.: The Quest Atlantis Project: A Socially-Responsive
Play Space for Learning. In: Shelton, B.E., and Wiley, D. (eds.) The Educational Design
 and Use of Simulation Computer Games, pp. 159–186. Sense Publishers, Rotterdam, The
Netherlands (2007)
12. Si, M., Marsella, S., Pynadath, D.: THESPIAN: An Architecture for Interactive
Pedagogical Drama. In: 12th International Conference on Artificial Intelligence in
Education, pp. 595–602. (2005)
13. Traum, D., Marsella, S., Gratch, J., Lee, J., Hartholt, A.: Multi-Party, Multi-Issue, Multi-
Strategy Negotiation for Multi-Modal Virtual Agents. In: 8th International Conference on
Intelligent Virtual Agents, pp. 117–130. (2008)
14. Vicente, A., Pain, H.: Informing the Detection of the Student's Motivational State: An
Empirical Study. In: 6th International Conference on Intelligent Tutoring Systems, pp.
933–943. (2002)
15. Beck, J.: Engagement Tracing: Using Response Times to Model Student Disengagement.
In: 12th International Conference on Artificial Intelligence in Education, pp. 88–95.
(2005)
16. Walonoski, J., Heffernan, N.: Detection and Analysis of Off-Task Gaming Behavior in
Intelligent Tutoring Systems. In: 8th International Conference on Intelligent Tutoring
Systems, pp. 382–391. (2006)
17. Johns, J., Woolf, B.: A Dynamic Mixture Model to Detect Student Motivation and
Proficiency. In: 21st National Conference on Artificial Intelligence, pp. 163–168. (2006)
18. D'Mello, S., Picard, R., Graesser, A.: Towards an Affect-Sensitive AutoTutor. Special
issue on Intelligent Educational Systems – IEEE Intelligent Systems. 22, 53–61 (2007)
19. Baker, R., Corbett, A., Koedinger, K., Wagner, A.: Off-Task Behavior in the Cognitive
Tutor Classroom: When Students "Game the System". In: ACM Conference on Human
Factors in Computing Systems, pp. 383–390. (2004)
20. Harp, S., Mayer, R.: How Seductive Details Do Their Damage: A Theory of Cognitive
Interest in Science Learning. Journal of Educational Psychology. 90(3), 414–434 (1998)
21. Hidi, S.: Interest and its Contribution as a Mental Resource for Learning. Review of
Educational Research. 60, 549–571 (1990)
22. Ainley, M., Hidi, S., Berndorff, D.: Interest, Learning, and the Psychological Processes
that Mediate their Relationship. Journal of Educational Psychology. 94, 545–561 (2002)
23. Schiefele, U.: Topic Interest, Text Representation, and Quality of Experience.
Contemporary Educational Psychology. 21, 3–18 (1996)
24. Wade, S., Buxton, W., Kelly, M.: Using Think-Alouds to Examine Reader-Text Interest.
Reading Research Quarterly. 34, 194–216 (1999)
25. Hidi, S., Harackiewicz, J.: Motivating the Academically Unmotivated: A Critical Issue
for the 21st Century. Review of Educational Research. 70, 151–179 (2000)
26. Norman, D.: The Invisible Computer. MIT Press, Cambridge, MA (1998)
27. Kim, Y., Baylor, A.: A Social-Cognitive Framework for Pedagogical Agents as Learning
Companions. Educational Technology Research & Development. 54, 569–590 (2006)
28. Schubert, T., Friedmann, F., Regenbrecht, H.: Embodied Presence in Virtual
Environments. In: R. Paton (ed.) Visual Representations and Interpretations, pp. 269–
278. Springer, London (1999)
29. Witmer, B., Singer, M.: Measuring Presence in Virtual Environments: A Presence
Questionnaire. Presence: Teleoperators and Virtual Environments. 7, 225–240 (1998)
30. Schraw, G.: Situational Interest in Literary Text. Contemporary Educational Psychology.
22, 436–456 (1997)

View publication stats