Exploring the Role of Sound Design in Serious Games: Impact on User

Experience and Learning Outcomes

Zijing Cao

, Ant

onio S

a Pinto

and Gilberto Bernardes

INESC TEC, University of Porto, Faculty of Engineering, Porto, Portugal

Keywords:

Serious Games, Sound Design, User Experience, Learning Outcomes.

Abstract:

Sound design plays an important role in serious games, inﬂuencing user experience and learning outcomes.

However, deriving general principles and best practices remains challenging, as most literature relies on case-

based studies in different application domains. Through a systematic review of the literature, 21 studies were

analyzed to address two key questions: 1) what types of serious games and application domains incorporate

sound design? and 2) what sound design strategies are implemented to enhance user experience and learning

outcomes? The ﬁndings show that serious games have mainly focused on education, healthcare, and training,

using sound to enhance motivation (50%), cognition (32%), and knowledge acquisition (18%). Furthermore,

sound design strategies fulﬁll distinct roles: sound effects enhance feedback and engagement, background

music inﬂuences motivation and cognitive processing, ambient sounds support navigation and emotional reg-

ulation, and dialogue facilitates knowledge acquisition. The ﬁndings highlight the need for further research to

establish standardized sound design principles to optimize user experience and learning outcomes in serious

games.

1 INTRODUCTION

Serious games are a category of video games devel-

oped for purposes beyond entertainment, including

training, advertising, simulation, and education (Susi

et al., 2007). They enable the creation of virtual envi-

ronments that would be impractical due to time, cost,

and safety constraints. These virtual spaces enable

innovative exploration and learning experiences that

transcend the physical world’s limitations (Kaczmar-

czyk et al., 2016). For example, a serious game de-

signed for ﬁreﬁghting training can effectively teach

procedural skills by simulating dangerous scenarios

that would be too risky or unethical to replicate in real

life (Mystakidis et al., 2022).

Sound design has become a signiﬁcant research

topic in game design. Research shows that sound de-

perience and learning outcomes. In particular, sound

design can improve learning engagement and mo-

tivation (Kao et al., 2021; Yang et al., 2019; Ku

et al., 2016; Barbosa and Nunes, 2015; Byun and Loh,

https://orcid.org/0009-0000-0214-4436

https://orcid.org/0000-0003-1629-8385

https://orcid.org/0000-0003-3884-2687

2015; Ke, 2008), provides critical support for visually

impaired users (Radecki et al., 2020; Connors et al.,

2014; Ferreira and Cavaco, 2014), improves learning

performance (Ku et al., 2016; Schuurink et al., 2008),

and reduces anxiety while improving auditory toler-

ance for people with autism spectrum disorder (ASD)

(Johnston et al., 2022, 2020).

Despite evidence supporting the importance of

sound design in serious games, guidelines for best

practices and effective design strategies remain

scarce. This study systematically reviews the litera-

ture on sound design impacts in serious games

, ad-

dressing two main research questions:

RQ1: What types of serious games and their appli-

cation domains have been proposed to explore the

impact of sound design?

A systematic literature review methodology was cho-

sen to provide a rigorous and transparent synthesis of exist-

ing studies, identifying patterns and gaps in sound design

strategies with strong empirical evidence. Although this

methodology offers a comprehensive view of academic re-

search, it cannot capture industry practices that often remain

unpublished due to commercial conﬁdentiality. Therefore,

while our review presents evidence-based ﬁndings, it may

not fully represent the state-of-the-art in industrial applica-

tions.

692

Cao, Z., Pinto, A. S. and Bernardes, G.

Exploring the Role of Sound Design in Serious Games: Impact on User Experience and Learning Outcomes.

DOI: 10.5220/0013504400003932

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 17th International Conference on Computer Supported Education (CSEDU 2025) - Volume 1, pages 692-700

ISBN: 978-989-758-746-7; ISSN: 2184-5026

RQ2: What sound design strategies have been imple-

mented in serious games targeting impact on user

experience and learning outcomes?

The remainder of the paper is structured as fol-

lows. Section 2 outlines the methods used to review

the literature on the impacts and outcomes of sound

design in serious games. Section 3 presents the re-

sults of the review. Finally, Section 4 summarizes the

ﬁndings of this work and provides directions for fu-

ture research.

2 METHODOLOGY

The methodology comprises data collection and tax-

onomic mapping. Data collection involved searching

scientiﬁc databases using predeﬁned terms and selec-

tion criteria aligned with our research questions. The

search was conducted on January 28, 2025. In the tax-

onomic mapping and categorization section, the se-

lected studies were analyzed and classiﬁed into seri-

ous games, sound design, and the impact of their in-

tersection. Figure 1 shows a graphical summary of

the literature review process.

2.1 Data Collection

2.1.1 Databases Searched

The primary databases selected for this study, Web of

Science and Scopus, cover our two key research do-

mains: serious games and the impact of sound de-

sign. The interdisciplinary nature of the topic re-

quired databases with comprehensive coverage in ed-

ucation, information technology, medicine, psychol-

ogy, and social sciences. Their indexing capabili-

ties thoroughly cover relevant literature while provid-

ing manageable search results. Additional databases,

Figure 1: Flowchart depicting the systematic literature re-

view procedure.

including the Association for Computing Machinery

(ACM), Ingentaconnect, the Institute of Electrical and

Electronics Engineers (IEEE), Springer Nature, Sci-

enceDirect, Wiley, and the Multidisciplinary Digital

Publishing Institute (MDPI), were accessed through

the Web of Science and Scopus indexing.

2.1.2 Search Terms

The search string included keywords from the two

key domains of the study: 1) sound, music, and

audio, and 2) serious games and game-based learn-

ing. The query ((serious game OR games-based

learning) AND (sound OR audio OR music))

was applied to titles, abstracts, and keywords in

both databases. Web of Science searched within

title, abstract, and keywords (including keyword

plus and author keywords), while Scopus searched

title, abstract, and keywords. This approach ensured

consistent coverage of literature at the intersection of

sound design and serious games.

2.1.3 Paper Selection Criteria

The initial search retrieved 912 papers in Scopus, cov-

ering the period from 1969 to 2025, and 547 papers in

Web of Science, from 1992 to 2025. The results were

reﬁned to include only articles presenting empirical

evidence through user studies on sound design’s im-

pact on user experience and learning outcomes. Fol-

lowing a manual selection process, 22 papers were

deemed relevant and included in the review, with 18

appearing in both Web of Science and Scopus. The

ﬁnal set comprised 11 conference papers (50%) and

11 journal articles (50%) published between 2008 and

2024.

2.2 Taxonomic Mapping and

Categorization

We used a structured data extraction scheme to cat-

egorize game mechanics, context, and sound design

strategies. Tables 1, 2, 3 present the resulting clas-

siﬁcations based on three taxonomies: serious games,

sound design, and their impact on user experience and

learning outcomes.

2.2.1 Categorization of Serious Games

We adopted the classiﬁcation framework proposed

by Laamarti et al. (2014) to characterize serious

games based on ﬁve main dimensions.

• Application Area: encompasses the different ap-

plication domains relevant to serious games, such

Exploring the Role of Sound Design in Serious Games: Impact on User Experience and Learning Outcomes

693

as education, well-being, training, advertisement,

interpersonal communication, healthcare, etc.

• Activity: refers to the function carried out by

the player in response to and/or as input to the

game—for example, physical exertion, physiolog-

ical, and mental.

• Modality: the channel by which information is

communicated from the computer to the human(s)

participating in the game-for example, visual, au-

ditory, haptic, or others.

• Interaction Style: Describes input methods rang-

ing from traditional interfaces (keyboard, mouse,

joystick) to advanced interaction systems (move-

ment tracking, tangible interfaces, brain-computer

interfaces, eye tracking).

• Environment: Deﬁnes the game’s digital

space characteristics, including dimensionality

(2D/3D), reality type (virtual/mixed), location

awareness, mobility features, and online capabili-

ties.

2.2.2 Categorization of Sound Design

Our sound design categorization follows game

sound classiﬁcations established by Brandon (2004)

and Collins (2008):

• Sound Effects. These are audio cues triggered by

players’ interactions with game elements, serving

multiple purposes such as providing feedback, is-

suing alerts, or reinforcing the narrative.

• Ambient Sound. Background audio that repre-

sents the surrounding environment, enhancing the

sense of place and immersion through environ-

mental cues such as rainfall, distant trafﬁc, or ur-

ban sounds.

• Dialogue. Voice-based interactions between char-

acters, including narrative exchanges and interac-

tive conversations.

• Music. Music refers to the structured arrange-

ment of sounds, combining elements like form,

harmony, melody, and rhythm—for example,

background music or a theme song that hints at

the game’s style.

2.2.3 Categorization of Impact of Games

The impact categories follow Connolly et al. (2012)

framework for serious games, encompassing both be-

havioral and learning outcomes.

• Knowledge Acquisition and Content Under-

standing Acquisition and comprehension of

domain-speciﬁc information and concepts within

serious games.

• Perceptual and Cognitive Skills. Develop-

ment of sensory processing and cognitive abilities

through gaming interactions, including attention,

memory, and problem-solving capabilities.

• Motor Skills: Fine and gross motor coordination

development through serious game interactions.

• Physiological Outcomes. Measurable changes in

physiological responses during gameplay, includ-

ing heart rate, stress levels, and neurological ac-

tivity.

• Affective and Motivational Outcomes. Changes

in emotional engagement and motivational factors

during serious game interactions, including par-

ticipation, enjoyment, and persistence.

• Behavior Change. Observable modiﬁcations in

user behaviors and habits resulting from serious

game engagement.

• Soft Skills. Enhance interpersonal and intraper-

sonal competencies through game-based learning,

including communication, leadership, and adapt-

ability.

• Social Outcomes. Development of collaborative

abilities and community engagement within and

beyond the serious game environment.

We also examined the evaluation methods used

in the reviewed studies, categorizing them as quan-

titative or qualitative based on data collection and

analysis techniques. User experience refers to par-

ticipants’ perceptions, emotions, and overall satisfac-

tion when interacting with the game, typically mea-

sured through questionnaires, engagement metrics, or

observational methods. Learning outcomes denote

measurable gains in knowledge, skills, or behavioral

changes resulting from gameplay, assessed through

tests, task performance metrics, or skill evaluations.

This distinction informed our analysis of how sound

design affects both dimensions in serious games.

3 RESULTS

This section presents the results of our review, cate-

gorizing 22 serious games based on the taxonomy of

Laamarti et al. (2014). Subsequently, we examine the

role of sound design in serious games, analyzing how

different sound elements impact user experience and

learning outcomes.

CSME 2025 - 6th International Special Session on Computer Supported Music Education

694

3.1 Serious Games

3.1.1 Game Application Area

Tables 1 and 2 present the classiﬁcation of 22 serious

games according to the taxonomy proposed by Laa-

marti et al. (2014).

These games span multiple disciplines, including

education (9), healthcare (6), training (5), and well-

being (2). Games in educational target diverse sub-

jects, such as mathematics (Ke, 2008; Radecki et al.,

2020; Ku et al., 2016; Yang et al., 2019), cyber-

security (Cao et al., 2023), language learning (Ver-

wimp et al., 2023), programming (Kao et al., 2021),

and software engineering (Alseid and Rigas, 2011).

Games in healthcare are designed for various appli-

cations, including tinnitus treatment (Schickler et al.,

2016), auditory attention diagnosis and training (Bar-

bosa and Nunes, 2015), anxiety reduction and toler-

ance improvement (Johnston et al., 2022), short-term

auditory and visual memory examination (Wers

enyi

and Csap

o, 2024), and reducing auditory hypersensi-

tivity in individuals with ASD (Zakari et al., 2017;

Johnston et al., 2020). Games in training are uti-

lized in contexts such as medical training (Rojas et al.,

2015), Levee Patroller training (Schuurink et al.,

2008), and environmental positioning (Byun and Loh,

2015; Boukhris and Menelas, 2017; Connors et al.,

2014). Games in well-being emphasize motor reha-

bilitation (Wongutai et al., 2021) and facilitating nav-

igation for visually impaired individuals (Marshall

et al., 2015).

Figure 2 illustrates the distribution of publications

across four disciplines: education, healthcare, train-

ing, and well-being. Most papers (77.3%) were pub-

lished in the past decade. The game with educational

goals has attracted the most attention except for 2015

to 2019 from researchers during different periods. Re-

search on serious games for training increased from

2005 to 2019 but saw no publications in the past ﬁve

years. People started to invest in healthcare in 2015,

and an increasing trend occurred. Only two publica-

tions investigated the game for well-being during the

last 20 years.

3.1.2 Environment

Tables 1 and 2 show that among the 22 serious games

reviewed, the majority (90.9%) were implemented for

PC or laptop, making it the most popular platform.

In comparison, only 2 games (9.1%) were designed

for mobile devices. Most games focused on mental

activity, with only 1 game (4.5%) requiring physical

exertion. Interaction styles varied across the games:

10 games (45.5%) used a keyboard and mouse, 4

Figure 2: Distribution of publications focusing on four ap-

plication area categories: Education, Training, Healthcare,

and Well-being.

(18.2%) employed touchscreen interaction, 3 (13.6%)

relied solely on a mouse, 2 (9.1%) combined move-

ment tracking with touchscreen interaction, and one

game each (4.5%) used movement tracking, simple

keyboard keystrokes, or a keyboard exclusively.

In terms of game environments, all the games

were virtual (100%), with 12 games (54.5%) de-

signed as 3D and 10 games (45.5%) as 2D. Six games

(27.3%) incorporated location awareness. Regarding

sensory modalities, 20 games (90.9%) involved both

auditory and visual elements, while 2 games (9.1%)

focused exclusively on auditory input. Additionally,

only 2 games (9.1%) were designed for online play.

3.2 Sound Design in Games

Table 3 presents serious games with sound design ad-

dressing different learning and behavioral outcomes

and impacts. The most frequently reported outcomes

were affective and motivational (11) and perceptual

and cognitive skills (7), followed by knowledge ac-

quisition and content understanding (4).

The following sound types inﬂuence emotion and

motivation: sound effects, music, dialogue, and am-

bient sounds. Sound effects enhance participant en-

gagement by providing auditory feedback (Ke, 2008;

Barbosa and Nunes, 2015; Ku et al., 2016), while the

dialogue, in the form of voiceover and self-similar

sounds, contributes to this engagement as well (Byun

and Loh, 2015; Kao et al., 2021). In addition, spa-

tialized sound has a positive effect on improving au-

ditory attention (Barbosa and Nunes, 2015; Johnston

et al., 2022). Ambient sounds have been shown to im-

prove navigation efﬁciency (Schuurink et al., 2008)

and reduce anxiety (Johnston et al., 2020, 2022). Mu-

sic has been shown to improve learning motivation,

engagement, immersion, and task performance in the

context of a VR shooting exergame (Schuurink et al.,

2008). Yang et al. (2019) and Ku et al. (2016) investi-

gated the inﬂuence of background music on gamiﬁed

Exploring the Role of Sound Design in Serious Games: Impact on User Experience and Learning Outcomes

695

Table 1: Categories of Serious Games - Application and Interaction.

Author and year Application

Area

Activity Modality Interaction Style

Wers

enyi and Csap

o (2024)

Healthcare Mental Auditory, Visual Mouse

Verwimp et al. (2023)

Education Mental Auditory, Visual Touchscreen

Cao et al. (2023)

Education Mental Auditory, Visual Keyboard, Mouse

Johnston et al. (2020, 2022)

Healthcare Mental Auditory, Visual

Motion tracking, Touchscreen

Kao et al. (2021)

Education Mental Auditory, Visual Keyboard and Mouse

Wongutai et al. (2021)

Well-being PE Auditory, Visual Motion tracking

Radecki et al. (2020)

Education Mental Auditory Touchscreen

Yang et al. (2019)

Education Mental Auditory, Visual Keyboard and Mouse

Boukhris and Menelas (2017)

Training Mental Auditory, Visual Mouse

Zakari et al. (2017) Healthcare Mental Auditory, Visual Touchscreen

Schickler et al. (2016) Healthcare Mental Auditory, Visual Touchscreen

Ku et al. (2016) Education Mental Auditory, Visual Keyboard and Mouse

Byun and Loh (2015)

Training Mental Auditory, Visual Keyboard and Mouse

Rojas et al. (2015)

Training Mental Auditory, Visual Keyboard and Mouse

Barbosa and Nunes (2015)

Healthcare Mental Auditory, Visual Keyboard and Mouse

Marshall et al. (2015)

Well-being Mental Auditory, Visual Keyboard

Ferreira and Cavaco (2014)

Education Mental Auditory, Visual Keyboard and Mouse

Connors et al. (2014)

Training Mental Auditory Keyboard

Alseid and Rigas (2011) Education Mental Auditory, Visual Keyboard and Mouse

Schuurink et al. (2008)

Training Mental Auditory, Visual Keyboard and Mouse

Ke (2008)

Education Mental Auditory, Visual Mouse

Activity: PE = Physical exertion.

Table 2: Categories of Serious Games - Environment.

Author and year

Environment

Mobile Participant 2D/3D LA Online R/V/M

Wers

enyi and Csap

o (2024)

No University Students 2D No No Virtual

Verwimp et al. (2023)

No Kindergarten Children 2D No No Virtual

Cao et al. (2023)

No Adults 2D No No Virtual

Johnston et al. (2020, 2022)

No ASD adolescents 3D Yes No Virtual

Kao et al. (2021)

No Adults 2D No Yes Virtual

Wongutai et al. (2021)

No Adults 3D No No Virtual

Radecki et al. (2020)

No Blind Children 2D Yes No Virtual

Yang et al. (2019)

No Uni/highschool students 2D No No Virtual

Boukhris and Menelas (2017)

No University Students 3D Yes No Virtual

Zakari et al. (2017)

Yes ASD children 2D No No Virtual

Schickler et al. (2016)

Yes Tinnitus/healthy 3D Yes No Virtual

Ku et al. (2016)

No University Students 2D No No Virtual

Byun and Loh (2015)

No University Students 3D No No Virtual

Rojas et al. (2015)

No University Students 3D No No Virtual

Barbosa and Nunes (2015)

No Adults 3D No No Virtual

Marshall et al. (2015)

No University Students 3D No No Virtual

Ferreira and Cavaco (2014)

No Visually impaired students 3D Yes No Virtual

Connors et al. (2014)

No Blind Adolescents 3D Yes No Virtual

Alseid and Rigas (2011)

No University Students 2D No No Virtual

Schuurink et al. (2008)

No Adults 3D No No Virtual

Ke (2008)

No Elementary Students 2D No Yes Virtual

Participant: ASD = Autism spectrum disorder.

CSME 2025 - 6th International Special Session on Computer Supported Music Education

696

learning, indicating that Serialists

are generally un-

affected by background music, whereas Holists

may

beneﬁt from mismatched background music in cer-

tain contexts. Ku et al. (2016) further suggested that

Holists might not consistently prefer listening to mu-

sic, as they frequently toggled it on and off during

tasks.

Sound categories that affect perceptual and cogni-

tive skills include sound effects, music, and ambient

sounds. Sound effects have a positive effect on par-

ticipants‘ cognitive performance. For example, famil-

iar, recognizable sounds can enhance memory reten-

tion and accuracy (Wers

enyi and Csap

o, 2024), and

sound notiﬁcation can improve attention (Cao et al.,

2023). Spatialized sound effects have a positive effect

on perceived location, for example, training tinnitus

patients to focus on target sounds and thereby sup-

press irrelevant background sounds (Schickler et al.,

2016), participants identifying the correct direction of

sounds (Boukhris and Menelas, 2017), and training

blind teenagers in navigation and spatial perception

skills (Connors et al., 2014). Ambient sounds such as

white noise have had some negative effects, reducing

the perception of visual reality by participants (Rojas

et al., 2015) and increasing the time taken to com-

plete tasks and reducing the precision of positioning

(Schickler et al., 2016).

Sound categories that affect knowledge acquisi-

tion and understanding include sound effects, dia-

logue, and ambient sounds. The dialogue sounds

in KlankKr8 have been shown to positively impact

learning by reinforcing the association between letters

and speech sounds, thereby improving engagement

and enhancing performance in letter-speech sound

tasks (Verwimp et al., 2023). In the study by Fer-

reira and Cavaco (2014), conversations were utilized

as content in the voice recording game function, en-

compassing math problems, answers, and feedback.

These recordings help students comprehend the con-

tent without relying on visual elements. Addition-

ally, in the study by Radecki et al. (2020), sound ef-

fects include synthetic sounds across various dimen-

sions, such as frequency (pitch), tone (timbre), and

volume dynamics. These sound effects mimic fea-

tures like color, distance, and geometric edges, en-

abling visually impaired children to understand im-

ages and shapes through auditory perception.

Figure 3 shows the distribution of the number of

Serialists are learners who process information in a

structured, step-by-step manner, focusing on details ﬁrst

and building knowledge sequentially.

Holists prefer to process information by understanding

the big picture ﬁrst, making connections between concepts

before focusing on individual details.

Figure 3: Distribution of publications focusing on four

sound categories: Sound effect, Ambient sound, Dialogue,

and Music.

publications investigated in the four sound design cat-

egories mentioned above. It appears that most papers

(80%) have been published during the past ten years.

The sound effect has always attracted the most atten-

tion from researchers during different periods. Re-

search on ambient sound in games has shown an in-

creasing trend from 2005 to 2025. Fewer studies in-

vestigate the role of dialogue and music in serious

games, likely because their implementation and eval-

uation are more complex than other sound elements.

3.3 Evaluation Method

Table 3 presents the methods used in the reviewed

studies. Of the 22 papers, 10 focused on learning

outcomes, 11 focused on user experience, and 1 ad-

dressed both. The majority, 17 studies (77%), re-

ported quantitative data, while 5 studies (23%) incor-

porated both quantitative and qualitative data.

In assessing user experience, most studies em-

ployed quantitative methods: 7 (32%) used question-

naires (including 1 qualitative questionnaire), 3 (14%)

utilized task performance metrics, 2 (9%) included

subjective ratings or rankings, 2 (9%) measured in-

game interaction times, and 1 (5%) employed elec-

tromyography (EMG).

For learning outcomes, 5 (23%) studies conducted

pre-/post-tests, 5 (23%) used task performance met-

rics, 3 (14%) employed questionnaires, and 2 (9%)

measured in-game interaction times. Regarding qual-

itative research, 2 (9%) studies collected user feed-

back, while 1 (5%) used interviews and behavioral

observations.

The one study that assessed both user experience

and learning outcomes incorporated task performance

metrics, in-game interaction times, and qualitative

user feedback.

Exploring the Role of Sound Design in Serious Games: Impact on User Experience and Learning Outcomes

697

Table 3: Classiﬁcation of Sound Design and Evaluation Method.

Author and year Sound Type Behavioral and Learning Outcomes Method Impact

Wers

enyi and Csap

o (2024)

SE Perceptual, cognitive skills Quantitative Learning outcomes

Cao et al. (2023) SE, Am

Perceptual, cognitive skills Quantitative User experience

Boukhris and Menelas (2017)

SE Perceptual, cognitive skills Quantitative Learning outcomes

Schickler et al. (2016) SE, Am

Perceptual, cognitive skills Quantitative User experience

Marshall et al. (2015) SE, Am

Perceptual, cognitive skills Quantitative User experience

Rojas et al. (2015) SE, Am, Mu

Perceptual, cognitive skills Quantitative User experience

Connors et al. (2014)

SE Perceptual, cognitive skills Quantitative Learning outcomes

Verwimp et al. (2023) SE, Di

Knowledge acquisition, understanding Quantitative Learning outcomes

Radecki et al. (2020) SE, Am

Knowledge acquisition, understanding

Both

Learning outcomes

Ferreira and Cavaco (2014)

SE Knowledge acquisition, understanding

Both

Learning outcomes

Alseid and Rigas (2011)

SE Knowledge acquisition, understanding Quantitative User experience

Johnston et al. (2020, 2022) SE, Am

Affective, motivational outcomes Quantitative User experience

Wongutai et al. (2021) SE, Mu

Affective, motivational outcomes Quantitative User experience

Kao et al. (2021)

Di Affective, motivational outcomes Quantitative Learning outcomes

Yang et al. (2019)

Mu Affective, motivational outcomes Quantitative Learning outcomes

Zakari et al. (2017)

SE Affective, motivational outcomes Quantitative User experience

Ku et al. (2016)

Mu Affective, motivational outcomes Quantitative Learning outcomes

Barbosa and Nunes (2015) SE, Am

Affective, motivational outcomes

Both Both

Byun and Loh (2015) SE, Di, Mu

Affective, motivational outcomes Quantitative User experience

Schuurink et al. (2008)

Am Affective, motivational outcomes

Both

User Experience

Ke (2008)

SE Affective, motivational outcomes

Both

Learning outcomes

Sound Type: SE = Sound Effect, Am = Ambient Sound, Mu = Music, Di = Dialogue. Method: Both = Qualitative and

Quantitative. Impact: Both = Learning Outcomes and User Experience.

4 DISCUSSION, LIMITATIONS,

AND FUTURE RESEARCH

Our ﬁndings indicate a growing interest in sound de-

sign within the context of serious games, particularly

concerning its usability and impact, with a notable

increase in research since 2015. However, research

on sound design has lagged behind the signiﬁcant ad-

vancements in the ﬁeld of serious games. Given the

limited evidence, we discuss our results and highlight

key research gaps in the following subsections to pro-

vide guidance for future research in this domain.

4.1 Discussion

In addressing RQ1 (What types of serious games and

their application domains have been proposed to ex-

plore the impact of sound design?), the study conﬁrms

that sound design is primarily adopted in education,

healthcare, and training. However, other potential do-

mains, such as advertising and interpersonal commu-

nication, remain underexplored. Despite the poten-

tial beneﬁts of mobile gaming, PC-based implemen-

tations dominate serious game development (90.9%),

suggesting that accessibility and portability remain

underutilized. With their touch controls and sensor-

based feedback, mobile platforms offer opportunities

for adaptive sound design. Although some applica-

tions exist for rehabilitation and language learning, a

broader exploration of sound-driven mobile games is

needed. Most serious games emphasize cognitive en-

gagement, with minimal focus on physical exertion.

However, movement-based learning could beneﬁt sig-

niﬁcantly from rhythmic cues, spatialized audio, and

real-time soniﬁcation to enhance motor coordination

and immersion.

Addressing RQ2 (What sound design strategies

have been implemented in serious games targeting

impact on user experience and learning outcomes?),

our ﬁndings indicate that sound effects, dialogue, mu-

sic, and ambient sounds contribute to engagement,

cognitive performance, and emotional responses, but

their implementation varies signiﬁcantly. While

studies highlight the beneﬁts of feedback sounds,

voiceovers, and spatialized audio, further research is

needed to understand their interaction with learning

processes and user characteristics.

While background music can inﬂuence user en-

gagement, ﬁndings regarding its effects on different

cognitive styles remain inconsistent. Few studies ex-

amine how auditory elements contribute to cognitive

load or how personalized sound environments could

CSME 2025 - 6th International Special Session on Computer Supported Music Education

698

optimize learning. Rather than serving as passive

background accompaniment, music can enhance mo-

tivation, emotional regulation, and cognitive process-

ing by reinforcing narrative elements and guiding user

interactions. Future research should explore context-

aware adaptive music that dynamically adjusts based

on player actions or learning progress. Additionally,

synchronized sound cues and rhythm-based feedback

may improve attention, retention, and immersion.

Most research focuses on short-term effects, such

as engagement and immediate cognitive improve-

ments, with little investigation into long-term im-

pacts, such as learning retention, behavioral change,

and sustained motivation. One barrier to longitu-

dinal studies is maintaining participant engagement

over extended periods due to resource constraints and

scheduling conﬂicts. Standardized evaluation frame-

works could mitigate inconsistencies and facilitate ro-

bust comparisons across studies.

4.2 Limitations and Future Research

The scope of our review was restricted to litera-

ture available in scientiﬁc repositories, which may

not fully capture the ongoing development of serious

games within the industry. Many serious game plat-

forms have been created and explored without corre-

sponding scientiﬁc publications, such as Scratch

Alice

, and Kahoot!.

Additionally, commercial

companies have made signiﬁcant contributions to se-

rious gaming, with tools like Minecraft

, Kodu Game

Lab

, and platforms like Celestory

and Serious Fac-

tory

. Given the importance of sound design in

these developments, unpublished industry advance-

ments may offer valuable insights.

Future research should investigate a wider range

of serious games, covering academic and commercial

implementations, to improve understanding of sound

design’s role in learning. Longitudinal studies are

particularly needed to investigate the sustained im-

pact of auditory elements on motivation, knowledge

Scratch: https://scratch.mit.edu/, developed by the

Massachusetts Institute of Technology (MIT).

Alice: https://https://www.alice.org/, Carnegie Mellon

University (CMU).

Kahoot!: https://www.kahoot.com, developed by

Morten Versvik with the Norwegian University of Science

and Technology (NTNU).

Minecraft: https://education.minecraft.net/, developed

by Microsoft.

Kodu: https://www.kodugamelab.com/, developed by

Microsoft.

Celestory: https://www.celestory.io/learning.

Serious Factory: https://seriousfactory.com/en/elearning-

solutions/serious-games

retention, and cognitive skill development over ex-

tended periods. Additionally, standardized evalua-

tion frameworks could help mitigate inconsistencies

across studies, facilitating more robust comparisons

and deeper insights into how different sound design

strategies inﬂuence user engagement and learning ef-

fectiveness.

ACKNOWLEDGEMENT

The ﬁrst author would like to thank China Schol-

arship Council (CSC)

for ﬁnancial support (Grant

No.202307920001).

REFERENCES

Alseid, M. and Rigas, D. (2011). The role of earcons

and auditory icons in the usability of avatar-based

e-learning interfaces. In 2011 Developments in E-

systems Engineering, pages 276–281. IEEE.

Barbosa, L. F. and Nunes, F. L. (2015). Whisperluck: A

game using virtual reality to evaluate and train the au-

ditory attention in adults. In 2015 XVII Symposium on

Virtual and Augmented Reality, pages 112–120. IEEE.

Boukhris, S. and Menelas, B.-A. J. (2017). Towards the

use of a serious game to learn to identify the location

of a 3d sound in the virtual environment. In Human-

Computer Interaction. Interaction Contexts: 19th In-

ternational Conference, HCI International 2017, Van-

couver, BC, Canada, July 9-14, 2017, Proceedings,

Part II 19, pages 35–44. Springer.

Brandon, A. (2004). Audio for Games: Planning, Process,

and Production (New Riders Games). New Riders

Games.

Byun, J. and Loh, C. S. (2015). Audial engagement: Effects

of game sound on learner engagement in digital game-

based learning environments. Computers in Human

Behavior, 46:129–138.

Cao, Z., Magalh

aes, E., and Bernardes, G. (2023). Sound

design impacts user experience and attention in seri-

ous game. In Joint International Conference on Seri-

ous Games, pages 95–110. Springer.

Collins, K. (2008). Game sound: an introduction to the

history, theory, and practice of video game music and

sound design. Mit Press.

Connolly, T. M., Boyle, E. A., MacArthur, E., Hainey, T.,

and Boyle, J. M. (2012). A systematic literature re-

view of empirical evidence on computer games and

serious games. Computers & education, 59(2):661–

686.

Connors, E. C., Chrastil, E. R., S

anchez, J., and Merabet,

L. B. (2014). Action video game play and transfer

https: //www.csc.edu.cn.

Exploring the Role of Sound Design in Serious Games: Impact on User Experience and Learning Outcomes

699

of navigation and spatial cognition skills in adoles-

cents who are blind. Frontiers in human neuroscience,

8:133.

Ferreira, F. and Cavaco, S. (2014). Mathematics for all:

a game-based learning environment for visually im-

paired students. In 2014 IEEE Frontiers in Education

Conference (FIE) Proceedings, pages 1–8. IEEE.

Johnston, D., Egermann, H., and Kearney, G. (2020).

Soundﬁelds: A virtual reality game designed

to address auditory hypersensitivity in individuals

with autism spectrum disorder. Applied Sciences,

10(9):2996.

Johnston, D., Egermann, H., and Kearney, G. (2022). The

use of binaural based spatial audio in the reduction

of auditory hypersensitivity in autistic young people.

International journal of environmental research and

public health, 19(19):12474.

Kaczmarczyk, J., Davidson, R., Bryden, D., Haselden, S.,

and Vivekananda-Schmidt, P. (2016). Learning de-

cision making through serious games. The clinical

teacher, 13(4):277–282.

Kao, D., Ratan, R., Mousas, C., and Magana, A. J. (2021).

The effects of a self-similar avatar voice in educational

games. Proceedings of the ACM on Human-Computer

Interaction, 5(CHI PLAY):1–28.

Ke, F. (2008). A case study of computer gaming for math:

Engaged learning from gameplay? Computers & ed-

ucation, 51(4):1609–1620.

Ku, O., Hou, C.-C., and Chen, S. Y. (2016). Incorpo-

rating customization and personalization into game-

based learning: A cognitive style perspective. Com-

puters in Human Behavior, 65:359–368.

Laamarti, F., Eid, M., and El Saddik, A. (2014). An

overview of serious games. International Journal of

Computer Games Technology, 2014(1):358152.

Marshall, J. B., Tyson, G., Llanos, J., Sanchez, R. M., and

Marshall, F. B. (2015). Serious 3d gaming research for

the vision impaired. In 2015 17th International Con-

ference on E-health Networking, Application & Ser-

vices (HealthCom), pages 468–471. IEEE.

Mystakidis, S., Besharat, J., Papantzikos, G., Christopou-

los, A., Stylios, C., Agorgianitis, S., and Tselentis, D.

(2022). Design, development, and evaluation of a vir-

tual reality serious game for school ﬁre preparedness

training. Education Sciences, 12(4):281.

Radecki, A., Bujacz, M., Skulimowski, P., and Strumiłło,

P. (2020). Interactive soniﬁcation of images in seri-

ous games as an education aid for visually impaired

children. British Journal of Educational Technology,

51(2):473–497.

Rojas, D., Cowan, B., Kapralos, B., Colllins, K., and

Dubrowski, A. (2015). The effect of sound on visual

realism perception and task completion time in a cel-

shaded serious gaming virtual environment. In 2015

Seventh International Workshop on Quality of Multi-

media Experience (QoMEX), pages 1–6. IEEE.

Schickler, M., Pryss, R., Reichert, M., Schobel, J.,

Langguth, B., and Schlee, W. (2016). Using mo-

bile serious games in the context of chronic disor-

ders: a mobile game concept for the treatment of tin-

nitus. In 2016 IEEE 29th international symposium on

computer-based medical systems (CBMS), pages 343–

348. IEEE.

Schuurink, E. L., Houtkamp, J., and Toet, A. (2008). En-

gagement and emg in serious gaming: Experimenting

with sound and dynamics in the levee patroller train-

ing game. In Fun and Games: Second International

Conference, Eindhoven, The Netherlands, October

20-21, 2008. Proceedings, pages 139–149. Springer.

Susi, T., Johannesson, M., and Backlund, P. (2007). Serious

games: An overview.

Verwimp, C., Snellings, P., Wiers, R. W., and Tijms, J.

(2023). A randomised proof-of-concept trial on the

effectiveness of a game-based training of phoneme-

grapheme correspondences in pre-readers. Journal of

Computer Assisted Learning, 39(5):1607–1619.

Wers

enyi, G. and Csap

A. B. (2024). Comparison of audi-

tory and visual short-term memory capabilities using

a serious game application. Infocommunications Jour-

nal, 16(2):51–60.

Wongutai, K., Palee, P., and Choosri, N. (2021). The effect

of sound in vr exergame to adult player: a primary in-

vestigation. In 2021 Joint International Conference

on Digital Arts, Media and Technology with ECTI

Northern Section Conference on Electrical, Electron-

ics, Computer and Telecommunication Engineering,

pages 1–4. IEEE.

Yang, T.-C., Chen, M. C., and Chen, S. Y. (2019). The ef-

fects of background music on game-based learning: A

cognitive style approach. The Asia-Paciﬁc Education

Researcher, 28:495–508.

Zakari, H. M., Poyade, M., and Simmons, D. (2017). Sin-

bad and the magic cure: A serious game for chil-

dren with asd and auditory hypersensitivity. In Games

and Learning Alliance: 6th International Conference,

GALA 2017, Lisbon, Portugal, December 5–7, 2017,

Proceedings 6, pages 54–63. Springer.

CSME 2025 - 6th International Special Session on Computer Supported Music Education

700