Enhancing Cognitive Conversion: The Impact of Sensory Stimuli on

Visual Working Memory and Future Applications

Qiuhan Chen

School of Psychology, The University of Sydney, Camperdown/Darlington, 2006, Sydney, Australia

Keywords: Visual Working Memory (VWM), Sensory Stimuli (SS), Cognitive Development, Artificial Intelligence (AI)

and Reinforcement Learning (RL).

Abstract: Visual working memory (VWM), a crucial short-term and long-term memory component, underpins cognitive

performance and daily life. This study proposes optimizing VWM formation by strategically designing

sensory stimuli (SS) to enhance the cognitive conversion from sensory memory to VWM. The present study

hypothesizes integrating various SS parameters based on a systematic review of previous research to improve

the VWM's overall performance, including attentional selection and VWM consolidation. It also highlights

the conducive role of multi-ss interactions in expanding the potential of improving cognitive conversion.

Targeting the optimal period for cognitive development, this research focuses on children to maximize the

potential for enhanced thinking abilities. The improved cognitive conversion in VWM formation may serve

as an effective intervention for cognitive impairments. To personalize training and adapt to individual

cognitive growth, this study explores the incorporation of AI markers and Reinforcement Learning (RL) into

smart toys, providing dynamic feedback and optimizing VWM development during interactive play. However,

there are still some challenges at play considering the speciality of the targeted children population and the

VWM mechanism's complexity. This research suggested some suggestions, underscoring the value of flexible

interdisciplinary method transformation.

1 INTRODUCTION

Visual working memory (VWM) is a cognitive

powerhouse that temporarily stores and manipulates

visual information (Ware, 2013). It is a last stage in

Visual Short-Term Memory (VSTM), where the

order is: iconic memory, fragile VSTM, and VWM.

Each characterized by different capacities and

durations: iconic memory serves as a high-capacity

buffer for transient sensory information immediately

after stimulus presentation, while VWM has more

stringent capacity limits, typically accommodating

approximately 2-4 objects for longer retention (Sligte,

2010). The active VWM process may be primarily

responsible for the retention of knowledge about the

10-item array over 1000 ms (Bradley and Pearson,

2012). This progression highlights the crucial role of

understanding the transition from sensory memory to

VWM. VWM's capacity is limited but can be

modulated by stimulus characteristics, encoding

strategies, and individual differences. Understanding

the impact of SS on VWM increases efficiency by

allowing us to apply the appropriate sensory stimuli

to the appropriate tasks.

The way SS is encoded is highly dependent on the

sensory features present in a stimulus. So, by

manipulating SS, we can directly influence the

encoding process, leading to improvements in storage

of information in VWM. Therefore, it is meangful to

explore the impact of SS on VWM.

This study first summarizes and integrates

relevant current studies on conversion efficiency to

VWM. Next, this study will propose potential future

applications in three dimensions based on the stimuli

and the process. Incorporating the cognitive

differences across different age groups, this study will

explore the design of an improved user interface and

experience, with a particular focus on promoting

early cognitive growth and preventing mental

diseases associated with cognitive decline.

Furthermore, this study innovatively proposes the

potential of incorporating AI markers into these

designs to predict cognitive boosts and decreases.

This approach has the potential to not only offer

valuable guidance for improved prevention,

providing reassurance about the study's practical

implications, but also to create an adaptive learning

system that caters to individual needs. RL allows AI

Chen, Q.

Enhancing Cognitive Conversion: The Impact of Sensory Stimuli on Visual Working Memory and Future Applications.

DOI: 10.5220/0014111300004942

Paper published under CC license (CC BY-NC-ND 4.0)

In Proceedings of the 2nd International Conference on Applied Psychology and Marketing Management (APMM 2025), pages 217-222

ISBN: 978-989-758-791-7

217

systems to iterate, categorize, and analyze real-world

data more effectively and accurately during human-

AI interactions.

2 THE IMPACT OF SS ON VWM

2.1 Theoretical Foundations of VWM

Formation

VWM formation begins with the sensory processing

of visual input (VI) by the eyes (Atkinson and

Shiffrin, 2024). This initial input creates IC, a brief,

high-capacity sensory buffer where salient features

are prioritized even pre-attentively. Because VWM

has limited capacity, attention acts as a critical filter,

selectively transferring information from IC to create

visual working memory representations (VWMR).

This transfer is goal-directed and shapes VWMR,

determining the format and structure of the visual

information retained. These representations, which

can be static or dynamic, are actively maintained and

manipulated through the interaction of a distributed

brain network. Sensory areas initially encode the

visual information Therefore, the SS of the initial VI

are crucial in shaping the content and fidelity of the

resulting VWMR. The limitations of VWM arise in

part from the capacity to process this sensory

information, highlighting the importance of efficient

selection for successful memory formation.

2.2 Analysis of Specific Visual

Properties’ Impact on VWM

Research suggests that several key visual attributes,

including spatial frequency, orientation, motion,

contrast, and color, significantly influence VWM

(Magnussen, 2000).

2.2.1 Speed of Dynamic VI

Recent research indicates that the precision of VWM

responses is significantly higher for moving stimuli

compared to stationary stimuli (Chung et al., 2023).

This enhanced precision, though with a slight

temporal lag, supports the hypothesis that object

motion facilitates the integration of object

information over extended periods. This notion aligns

with findings from memory masking experiments,

which have shown that VSTM for motion primarily

encodes the speed of the stimulus, rather than its

spatial or temporal frequency content (McKeefry et

al., 2007). Given that VWM is considered to be

embedded within VSTM, the speed of a moving

stimulus should be regarded as a critical motion-

related SS feature for optimizing VWM research and

application.

2.2.2 Color

When it comes to VWM and object recognition,

colour is crucial. Understanding the mechanism of the

visual cognitive process and assessing memory

capacity are made easier with the help of colour

VWM decoding in the human brain. Target letters in

an n-back task were changed into Baker-Miller pink,

blue, red, or black in one behavioural experiment that

assessed working memory performance (Galvez,

2015). It was found that the color had no effect on

accuracy, but red is the most easily caught by

attention. Recent research has identified different

representations of specific sensory features (like color)

in VWM can be measured using scalp

electroencephalogram (EEG). The accuracy of colour

decoding during the encoding, early, and late

maintaining stages was 81. 58%, 79. 36%, and 77.

06%, respectively. These decoding accuracy levels

were higher than those during the pre-stimuli stage

(67. 34%), and during the maintaining stage, they

may predict the memory performance of the

participants. The EEG colour graph convolutional

network (ECo-GCN) model offers a useful method

for investigating human cognitive function, and EEG

data during the maintenance stage may be more

sensitive than behavioural testing to predict human

VWM performance (Galvez, 2015). In the future

study, EEG may further explore the performance of

various color decoded from encoding process, thus

finding the best color for SS design.

So far, there is little research indicating what kind

of orientation, and geometric features may be

beneficial to VWM. Future study may explore further

regarding this field.

2.3 SS Through IC

In both retinotopic and non-retinotopic frames of

reference, researchers have found that the contents of

IC can be altered by later VI before being translated

into conscious awareness in a time-locked manner.

This means that new VI can affect objects and spatial

locations. Additionally, the IC can revise its contents

based on the configuration of the new representation,

causing the original sensory representation to be

altered or updated before being selected for visual

processing machine, namely VWM. This can lead to

an incongruity between the initial sensory experience

and the encoded information in the VWM. Therefore,

APMM 2025 - International Conference on Applied Psychology and Marketing Management

218

it should be noticed that there is a close interplay

between early sensory processing and later cognitive

processes , which is the conversion from IC to VWM.

The quality of information in IC directly affects the

quality of the VWMR. While IC appears to encode

motion information less effectively than color or

orientation, the inclusion of moving information has

a beneficial impact on long-term memory (LTM)

(Bradley and Pearson, 2012 & Che et al., 2022).

Given VWM’s intermediate position between iconic

and LTM, careful consideration must be given to the

specific effects of different sensory features during

the stimulus design to optimise memory at all stages.

2.4 Multi-SS Interaction

Researchers have underscored the significance of

multi-ss interaction in positively improve VWM

process.

To begin with, researchers have found the effect of

sensory stimulation, where both visual and auditory

stimulation at frequencies of 4 Hz and 7 Hz

significantly improved VWM performance compared

to baseline conditions. A combination of the 4 Hz and

7 Hz conditions into a single stimulation revealed a

significant increase in VWM capacity (Matthews et

al., 2007). The calibration of SS targeted to address

VWM deficits of limited capcity, thus highlighting the

promise of using sensory stimulation as a tool for

cognitive enhancement

Moreover, according to the current study,

performing a visual orientation sequence task

significantly improved after practicing a tactile

sequence task. We showed that switching from a

tactile to a visual task could have a beneficial training

effect. It was also possible to apply this training effect

to other behavioural tasks that included the same

cognitive processes (Pileckyte and Soto-Faraco,

2024). This enlightens us to combine various SS like

tactile sense to achieve an elaborated design of smart

toys for different purposes.

2.5 Other Conditions of SS that

Influence the Performance of

VWM

Beyond core stimulus attributes, emerging evidence

reveals other SS conditions significantly modulate

VWM. For instance, VWM capacity improvements

are linked to modulated temporal expectations, not

just periodic entrainment (Matthews et al., 2007),

highlighting the influence of temporal context.

Distractors also introduce biases in VWM

representations based on similarity, interacting with

maintained information (Guo et al., 2021).

Furthermore, more meaningful stimuli recruit

additional VWM resources, facilitating retention

(Teng and Kravitz, 2019). Even categorical reporting

biases are heightened following unattended storage,

with biased gaze patterns emerging during

maintenance (Asp et al., 2021). Importantly, the

degree of congruency between sensory features, both

within and across modalities, influences VWM

encoding; congruent multi-SS may be encoded more

readily, while incongruent stimuli may require more

processing resources, potentially negatively impacting

VWM performance (Linde-Domingo and Spitzer,

2023). These findings collectively demonstrate that

various SS conditions beyond basic attributes

powerfully influence VWM, by impacting encoding,

maintenance, and retrieval.

Furthermore, attentional allocation across sensory

modalities profoundly shapes visual information

processing and storage in VWM. Distributing

attention across multiple modalities significantly

alters audiovisual processing (Linde-Domingo and

Spitzer, 2023), and recent findings indicate that

occipital, parietal, and frontal cortices selectively

maintain task-relevant features (Mishra and Gazzaley,

2012). These findings demonstrate that attentional

allocation must be controlled when manipulating SS.

Last but not least, a study found that highly

discriminable sensory features were encoded into

VWM via an object-based effect(OBE), leading to

prolonged search time. The study also revealed that

the initial encoding is not "smart" as VWM takes in a

holistic representation of an object with all its sensory

features. Task-irrelevant information is still processed

and affects performance, even if it is not task-relevant

(Yu and Shim, 2017). In addition, the capacity of

VWM must be understood in terms of integrated

objects rather than individual features. At least four

features can be joined in this manner with no cost in

terms of storage capacity (Luck and Ford, 1998).

This has implications for stimulus design in studies

relying on VWM, as it is impossible to only process

the intended task-relevant information. It needs to

design SS with a low amount of irrelevant sensory

information to improve performance.

3 FUTURE APPLICATIONS

SS play a pivotal role in the transition from IC to

VWM, significantly influencing cognitive processes.

The potential future applications of this

understanding are vast and promising.

Enhancing Cognitive Conversion: The Impact of Sensory Stimuli on Visual Working Memory and Future Applications

219

3.1 Strength in Children—Better Brain

Activation

When researchers compared performance between

update and non-update trials in a 2-back task, they

discovered that VWM deficits increased with age.

Older people exhibited reduced neuromodulations to

the working memory updating process in the task-

sensitive areas in addition to performance losses (Qin

and Basak, 2020). Therefore, children show potential

for a boost in the formation of VWM.

3.2 Practical Implications

3.2.1 User Interface and Experience Design

Integrating visual colours and geometric shapes into

user interface designs can improve usability and

accessibility by making information more engaging

and easier to process. As educational paradigms

evolve, integrating multi-sensory theory into the

design of children’s educational toys presents a

promising avenue for enhancing learning outcomes

(Fan et al., 2024). Instructional visualizations should

be designed to manage this balancing act and to

support information processing optimally (Brucker et

al., 2014). That, in particular, prospects promising

development in (1) Cognitive Development in Early

Childhood and (2) Cognitive Health and Well-being

of children.

(1) Cognitive development in early childhood

Developing multi-sensory learning strategies

through the use of tactile materials (e. g. three-

dimensional geometric shapes) or learning tools with

sound effects can promote children's all-round, multi-

sensory cognitive development. Optimising the visual

design of classroom environments The overall design

of the classroom can also take into account the SS to

create an environment that is stimulating to the senses

and promotes learning.

(2) Cognitive health and well-being.

Tactile stimulation training could partially reverse

age-related cognitive decline among older adults and

increase processing speed in younger participants

(Reuter et al., 2014). Developing cognitive training

tools and interactive games that leverage visual

colours and geometric shapes can stimulate cognitive

functions and help prevent conditions like dementia

in older adults. Integrating visual and geometric

stimuli in therapeutic programs can aid in

rehabilitating cognitive functions in children with

cognitive impairments (Wang et al., 2021).

3.2.2 Artificial Intelligence and Machine

Learning

As robust and interpretable AI-guided marker has

been built for early dementia prediction in real-world

clinical settings and the ability of both memory and

thinking change with age, utilizing visual colours and

geometric shapes as markers in AI appear to be

possible in better understanding the process of

cognitive development and establishing adaptive

learning systems (Kale et al., 2024 & Harvard Health

Publishing, 2017). For example, AI markers

integrated into children’s educational toys can track

and analyze children's cognitive development,

identifying key developmental milestones and

intelligence boosts. At the same time, adaptive

learning systems that modify visual and geometric

content based on a learner's cognitive level can

provide personalised challenges, enhancing cognitive

development and retention of complex concepts.

Long-term observation of the training effect can be

shown by rising trends of grades of difficulty in the

design of SS and children's performance, as the

improved accuracy of VWM through training may

partially transfer to more VWM tasks with other

stimuli (Bi et al., 2020).

Further, it is beneficial to AI systems as well.

They can categorize and analyze data more

effectively and accurately by RL in recording long-

term pattern recognition trials, enhancing the overall

performance of AI-driven data analysis.

Recently, researchers discovered that, compared

to a conventional motor-only paradigm, the multi-

modal technique maintained consistent engagement

by EEG biomarkers. That highlights the benefit of a

comprehensive robotic intervention combining motor,

cognitive, and auditory functions (Oliver et al., 2024).

Incorporating the fundamental principles of VWM

and the various characteristics of cognitive

development stages into "Smart Toys" design for

human-computer interaction can significantly

improve cognitive development efficiency. By going

beyond simple, functional interactions, these toys can

be enhanced by integrating memory formation

principles, ultimately fostering greater cognitive

growth.

3.3 Influential Factors for the Capacity

of Forming VWM in Children

3.3.1 Cognitive Load and Interference

VWM load has a late neuronal response on auditory

stimuli, which reduces the likelihood that these SS

APMM 2025 - International Conference on Applied Psychology and Marketing Management

220

will be reported (Brockhoff, 2023). VWM

performance may be hampered by auditory stimuli

when there is a significant cognitive load. The

efficacy of memory tasks may be hampered when

visual and aural cues vie for processing resources (He

et al., 2022). In children, managing cognitive load

effectively is vital for optimizing learning outcomes

and ensuring that sensory information is

appropriately encoded into VWM.

3.3.2 Complexity of VI

The complexity of VI plays a significant role in how

well children can encode information into VWM.

Studies manipulating figural complexity have shown

that as complexity increases, distinguishing between

target and probe stimuli becomes more challenging

for young children (Suda et al., 2024). This indicates

that simpler, well-defined shapes and colors may

facilitate better encoding and retention in VWM.

3.3.3 Dynamic Coding Mechanisms

Research suggests that dynamic coding may help

separate SS from VWM contents during tasks

requiring memory maintenance. This mechanism

allows for effective encoding and retrieval of

information while minimizing interference from

ongoing sensory stimuli (Degutis et al., 2024).

Understanding how these dynamic processes operate

in children could lead to better educational strategies

that leverage multi-sensory approaches.

4 CONCLUSION

This study seeks to elucidate the impact of SS on

VWM and its potential applications for both human

enhancement and artificial intelligence. Synthesizing

existing research on optimizing SS for VWM

formation, this research proposes that dynamic,

congruent multisensory stimuli incorporating

appropriate visual speed represent a promising

avenue for future exploration. The identified features

that enhance VWM formation have direct practical

implications, particularly for the design of smart toys

for children and the development of more robust AI

systems.

The novelty of this research lies in its systematic

approach to summarizing key SS features that

influence VWM. This study addresses a significant

gap in the current design of many smart toys, which

often prioritize aesthetic user interfaces and diverse

content while overlooking the scientific principles

underlying VWM formation. This finding highlights

the potential of multi-SS design and the utilization of

AI-markers in smart toy programming to enhance

learning and cognitive development.

To mitigate bias related to individual sensory

preferences, including color, investigations into

personal preferences should precede any behavioral

experiments. Furthermore, this research strongly

advocates for the use of other advanced brain imaging

research methods represented by EEG in future

studies to achieve a more precise and nuanced

understanding of the neural mechanisms underlying

VWM responses to different SS features. This

approach could lead to more personalized and

effective designs. Future research should aim to

establish a more precise and generalizable multi-SS

model, incorporating the various identified sensory

features to maximize VWM performance.

REFERENCES

C. Ware. Foundations for an Applied Science of Data

Visualization. In Information Visualization (pp. 1–

30). Elsevier, (2013)

I. G. Sligte. Detailed sensory memory, sloppy working

memory. Front. Psychol., 1, (2010)

C. Bradley & J. Pearson. The sensory components of high-

capacity iconic memory and visual working memory.

Front. Psychol., 3, (2012)

R. C. Atkinson & R. M. Shiffrin. Reprint of: Human

memory: A proposed system and its control processes.

J. Mem. Lang., 136, 104479, (2024)

S. Magnussen. Low-level memory processes in vision.

Trends Neurosci., 23(6), 247–251, (2000)

Y. H. Chung, M. W. Schurgin & T. F. Brady. The role of

motion in visual working memory for dynamic stimuli:

More lagged but more precise representations of

moving objects. Atten. Percept. Psychophys., 85(5),

1387–1397, (2023)

D. J. McKeefry, M. P. Burton & C. Vakrou. Speed

selectivity in visual short-term memory for motion. Vis.

Res., 47(18), 2418–2425, (2007)

M. Galvez. The effect of color on working memory

performance. HIM (1990-2015), 620, (2015)

M. Galvez. The effect of color on working memory

performance. (2015). Available online at:

https://stars.library.ucf.edu/honorstheses1990-

2015/620/

X. Che, Y. Zheng, X. Chen, S. Song & S. Li. Decoding

color visual working memory from EEG signals using

graph convolutional neural networks. Int. J. Neural

Syst., 32(02), 2250003, (2022)

W. J. Matthews, C. Benjamin & C. Osborne. Memory for

moving and static images. Psychon. Bull. Rev., 14(5),

989–993, (2007)

Enhancing Cognitive Conversion: The Impact of Sensory Stimuli on Visual Working Memory and Future Applications

221

I. Pileckyte & S. Soto-Faraco. Sensory stimulation

enhances visual working memory capacity. Commun.

Psychol., 2(1), 109, (2024)

T. Guo, Y. Ren, Y. Yu, Y. Yu, Y. Hasegawa, Q. Wu, J.

Yang, S. Takahashi, Y. Ejima & J. Wu. Improving

visual working memory with training on a tactile

orientation sequence task in humans. Sage Open, 11(3),

21582440211031549, (2021)

C. Teng & D. J. Kravitz. Visual working memory directly

alters perception. Nat. Hum. Behav., 3(8), 827–836,

(2019)

I. E. Asp, V. S. Störmer & T. F. Brady. Greater visual

working memory capacity for visually matched stimuli

when they are perceived as meaningful. J. Cogn.

Neurosci., 33(5), 902–918, (2021)

J. Linde-Domingo & B. Spitzer. Geometry of visuospatial

working memory information in miniature gaze

patterns. Nat. Hum. Behav., 8(2), 336–348, (2023)

J. Mishra & A. Gazzaley. Attention distributed across

sensory modalities enhances perceptual performance. J.

Neurosci., 32(35), 12294–12302, (2012)

Q. Yu & W. M. Shim. Occipital, parietal, and frontal

cortices selectively maintain task-relevant features of

multi-feature objects in visual working memory.

NeuroImage, 157, 97–107, (2017)

S. J. Luck & M. A. Ford. On the role of selective attention

in visual perception. Proc. Natl. Acad. Sci., 95(3), 825–

830, (1998)

S. Qin & C. Basak. Age-related differences in brain

activation during working memory updating: An fMRI

study. Neuropsychologia, 138, 107335, (2020)

Y. Fan, D. K. Chong & Y. Li. Beyond play: A comparative

study of multi-sensory and traditional toys in child

education. Front. Educ., 9, 1182660, (2024)

B. Brucker, K. Scheiter & P. Gerjets. Learning with

dynamic and static visualizations: Realistic details only

benefit learners with high visuospatial abilities. Comput.

Hum. Behav., 36, 330–339, (2014)

E. Reuter, C. Voelcker‐Rehage, S. Vieluf & B. Godde.

Effects of age and expertise on tactile learning in

humans. Eur. J. Neurosci., 40(3), 2589–2599, (2014)

G. Wang, M. Zhao, F. Yang, L. J. Cheng & Y. Lau. Game-

based brain training for improving cognitive function in

community-dwelling older adults: A systematic review

and meta-regression. Arch. Gerontol. Geriatr., 92,

104260, (2021)

M. Kale, N. Wankhede, R. Pawar, S. Ballal, R. Kumawat,

M. Goswami, M. Khalid, B. Taksande, A. Upaganlawar,

M. Umekar, S. R. Kopalli & S. Koppula. AI-driven

innovations in Alzheimer’s disease: Integrating early

diagnosis, personalized treatment, and prognostic

modelling. Ageing Res. Rev., 101, 102497, (2024)

Harvard Health Publishing. How memory and thinking

ability change with age. (2017). Available online at:

https://www.health.harvard.edu/mind-and-mood/how-

memory-and-thinking-ability-change-with-age

T. Bi, X. Wang, H. Kou & J. Qiu. The effect of visual

working memory training could transfer across stimuli.

Psychol. Res. Behav. Manag., 13, 55–66, (2020)

S. Oliver, T. Kitago, A. Buchwald & S. F. Atashzar. EEG-

based analysis of brain responses in multi-modal

human-robot interaction: Modulating engagement

(Version 1). arXiv, (2024)

L. Brockhoff, L. Vetter, M. Bruchmann, S. Schindler, R.

Moeck & T. Straube. The effects of visual working

memory load on detection and neural processing of

task-unrelated auditory stimuli. Sci. Rep., 13(1), 4342,

(2023)

Y. He, Z. Guo, X. Wang, K. Sun, X. Lin, X. Wang, F. Li,

Y. Guo, T. Feng, J. Zhang, C. Li, W. Tian, X. Liu & S.

Wu. Effects of audiovisual interactions on working

memory task performance—Interference or facilitation.

Brain Sci., 12(7), 886, (2022)

M. Suda, T. Ikeda & M. Kikuchi. Visual working memory

resolution defined by figural complexity in

kindergarten children. J. Vis., 24(1), 4, (2024)

J. K. Degutis, S. Weber, J. Soch & J.-D. Haynes. Neural

dynamics of visual working memory representation

during sensory distraction. bioRxiv, (2024)

APMM 2025 - International Conference on Applied Psychology and Marketing Management

222