Impact of Dopamine on Reward Related Behavior of Animals and

Therapeutic Role

Chenyiwei Fu

Beijing International Bilingual Academy, Hebei, 065201, China

Keywords: Dopamine, Experimental Conditioning, Reinforcement Learning, Parkinson’s Disease.

Abstract: Nowadays, some researchers proposed that human behavior are closely related to dopamine development.

The effect of dopamine on esthesia of reward of humans and animals is very significant. This paper will use

a literature review method to elaborate on the role of dopamine in experimental conditioning, animal reward

circuitry, and what role dopamine plays in human behavior, such as reinforcement learning. This paper will

also specifically illustrate the circuitry of dopamine secretion and the reinforcement process of the secretion

process. As it is difficult to achieve human experiments, a large number of animal experiments will be used

to demonstrate its effect on animals, then further apply some results on human. The paper found that the

secretion of dopamine played a significant role in reward related behavior under experimental conditioning.

This conclusion can also generalize to human behavior to a certain extent because of the genetic similarity

between mice and other animals and humans. Therefore, this paper, via literature review, will show the role

of dopamine in Parkinson's syndrome and the treatment and therapeutic effects of effective drugs on the

syndrome.

1 INTRODUCTION

Neurotransmitter dopamine plays a significant role in

human learning, motivation, and many other aspects.

This paper will further strengthen the irreplaceable

role of dopamine in reward circuitry by combining

past literature, presenting the trajectory of dopamine

in human brains and animal brains, and combine with

experiments under Experimental conditioning,

showing the degree that dopamine is practically

involved in reward related behaviors in animals. In

addition, dopamine is a neurotransmitter, and its

system regulation disorder is the main cause of

Parkinson's syndrome. Insufficient dopamine

secretion leads to Parkinson's syndrome. Thus, it can

be inferred that the secretion of dopamine is closely

related to Parkinson's disease. Therefore, this paper,

with a method of literature review, will also present

the role of dopamine in Parkinson's syndrome and the

treatment and therapeutic effects of effective drugs on

the syndrome.

2 DEFINITIONS

2.1 Midbrain Dopamine

The Midbrain dopamine, also known as

dopaminergic neurons in ventral mesodiencephalon

(mdDA), is responsible for several functions as

voluntary movements control, motivation behavior,

maintaining working memories, adjusting emotions

and more importantly, associations with rewarding

stimuli (Bissonette & Roesch 2016).

Midbrain dopamine(mdDA) has a great influence

on Reinforcement learning by adjusting strength of

synaptic connection between neurons (Bromberg-

Martin, Matsumoto & Hikosaka 2010). More

specifically, this would permit a individual to learn

the ideal choice of activities to pick up rewards, given

adequate trial-and-error involvement. According to

Montague et al (1996), such process could be

described as a modified Hebbian rule. When a cell

affect it’s neibors which eventually result in a reward

or punishment, brain would release dopamine in

order to reinforce the connection between two cells,

and eventually result in repeated behavior. Referring

to(Figure 1.)

134

Fu, C.

Impact of Dopamine on Reward Related Behavior of Animals and Therapeutic Role.

DOI: 10.5220/0011203200003444

In Proceedings of the 2nd Conference on Artiﬁcial Intelligence and Healthcare (CAIH 2021), pages 134-140

ISBN: 978-989-758-594-4

Figure 1: Nigrostriatal pathway and Mesocortical pathway

displaying release of dopamine.

Furthermore, Mesolimbic, Mesocortical, as well

as Nigrostriatal pathways are indispensable in the

connective process of midbrain dopamine cells. The

mesolimbic and nigrostriatal pathways are an integral

part of the basal ganglia through its reciprocal

connections to the ventral and dorsal striatum

respectively (Ikemoto 2007). The nigrostriatal

pathway is shown emitting mainly from SNc

(Substantial Nigra pars compacta) neurons, which

provide dopaminergic tone necessary for voluntary

movements and also carrying salience and PE

(predicted error) signal (Bissonette & Roesch 2016).

Simultaneously, VTA (Ventral tegmental area) is also

emanating from the Basal Ganglia through

Mesocortical pathway to both the ventral striatum as

well as Cortex. Corticostriatal input simultaneously

travels to Dorsal Striatum and ultimately result in

bodily movement, or behavior (Bissonette & Roesch

2016).

2.2 Reinforcement Learning

Reinforcement learning could be simply describe as

learning from past experiences, or Law of Effect

proposed by Edward Thorndike. Thorndike took

observation of cats in a puzzle box possessing certain

mechanism for cats to learn and ultimately, escape.

This is a traditional Instrumental conditioning, or

Operant conditioning experiment, a term used for

experiments which reinforcement is contingent upon

behavior (Sutton&Barto 2018). Furthermore,

researchers placed food, as primary Reinforcement

next to the confinement.

Qualitative data are taken during the observation.

“The cat that is clawing all over the box in her

impulsive struggle will probably claw the string or

loop or button so as to open the door. And gradually

all the other non-successful impulses will be stamped

out and the particular impulse leading to the

successful act will be stamped in by the resulting

pleasure, until, after many trials, the cat will, when

put in the box, immediately claw the button or loop

in a definite way” (Burnham 1972).

3 CONNECTION BETWEEN

REFINFORCEMENT

LEARNING BEHAVIOR AND

FUNCTION OF DOPAMINE

Instead of conduct in-depth research directly from

human behavior, many experiments on reinforcement

learning are initiated with animals, especially mice,

due to similar brain structures and past knowledge on

circuit when dopamine neurons are triggered and

released. Two models of dopamine projection

systems released from ventral midbrain to ventral

stratum are responsible for reward related behavior

(Haber 2014).

3.1 Dopamine Projection System from

Ventral Midbrain

Recently, researchers have been utilizing Rats in

order to refine our understanding localization within

the ventral striatum and VTA that are responsible for

the rewarding effects of drugs of abuse (Ikemoto &

Wise 2004). Medial olfactory tubercle not only plays

an important role in relevant to drug reward, but also

shares a common function with the medial shell; they

further suggest that the accumbens shell is

functionally heterogeneous, as is the olfactory

tubercle (Ikemoto 2007).

Rats could learn to lever-press for cocaine or

amphetamine into the olfactory tubercle, although the

medial portion of the tubercle is more responsive to

the rewarding effects of these drugs than the lateral

portion (Ikemoto 2003). A representation of

dopamine signal circuitry has once been illustrated in

“precisely timed dopamine signals establish distinct

kinematic representations of skilled movements”

(Leventhal & Bova 2020). The aim of this study was

to determine the effects of precisely timed

dopaminergic manipulations on a relatively

unconstrained motor skill. This process effectively

illustrated the circuitry movement in relevant to

dopamine, with the use of experimental conditioning.

Rats were utilized considering ethical guidelines in

human is scarcely attainable. The procedure involves

stimulating or inhibiting midbrain dopamine neurons

in different time period of several groups while rats

Impact of Dopamine on Reward Related Behavior of Animals and Therapeutic Role

135

are performing a skilled reaching task, in which the

coordinated forelimb and digit movements to reach

for, grasp, and consume sugar pellets were involved.

In detail, researchers had stimulated or inhibited

substantia nigra pars compacta (SNc) dopamine

neurons at specific moments during rat skilled

reaching. Different Viruses as Tyrosine hydroxylase

(TH)-Cre+ rats were injected bilaterally with a

double-floxed channelrhodopsin (ChR2),

archaerhodopsin (Arch), or control EYFP construct

into SNc.

Explain further into the skilled reaching task,

training and testing were carried out in custom-built

skilled reaching chambers housed within soundproof,

ventilated cabinets (Leventhal & Bova 2020). Trials

were initiated with rats breaking a photobeam at the

back of the chamber, which caused a pellet to be

delivered in front of the reaching slot. Rats could

make multiple reaches until the pellet delivery arm

descended 2 s after the video trigger event. Following

training, optical fibers were implanted over SNc

contralateral to the rat ’ s preferred reaching paw.

Immunohistochemistry confirmed that opsin

expression was restricted to TH-expressing neurons

in SNc projecting to striatum (Leventhal & Bova

2020).

As a result, activity of SNc dopamine neuron

which was altered gradually changes skilled reaching

outcomes. Furthermore, dopamine neuron

stimulation caused a trend of decline in performance.

This shows that dopamine plays an indispensable role

in reinforcement learning and similar skill

acquisition.

Figure 2: Examples of immunohistochemistry from rats for each group.

3.2 Dopamine Projection from Ventral

Tegmental Area

Lichtenburg et al (2018) illustrated inner,

dopaminergic circuitry, or movement while rats are

encountering rewarding tasks or decision making

(Lichtenberg, etc. Reward related behaviors are often

signaled by DA system. Studies have shown that

Dopamine neurons in the ventral tegmental area

(VTA) and subsequent DA release into the NAc

increases and decreases in response to events that are

better or worse than expected. In the appetitive

context, release of dopamine are usually determined

by prediction of potential reward. In contrast, in

aversive contexts, unavoidable aversive events as

shocks or air puff, significantly reduce DA release,

whereas unexpected omission of aversive events or

the cues that predict avoidable shock reliably elicit

phasic DA release (McCutcheon 2012).

CAIH 2021 - Conference on Artiﬁcial Intelligence and Healthcare

136

In order to detect and obtain information on how

Dopamine signals are modulated by appetitive and

aversive events from conspecific (opponent), the

research further recorded accumbal, dopamine

release utilizing Fast-Scan Cyclic Voltammetry

(FSCV), which is useful in detection of release of DA

in response to different contexts.

Eight rats were observed during performance of

Pavlovian Social Distress Paradigm. Each group

involved two rats, with the recording rat and

Conspecific, the opponent. They were separated by a

guillotine door which is transparent, allowing visual,

smelly, and vocal action permeate through. One of

each directional light, food cup, and shock grid was

placed in each room with a house light in the middle,

placed right above the delineated line. To initiate each

trial, the researcher first turn on the houselight; after

5 seconds, three different stimuli was randomly

applied to each room correlated to reward, neutral or

shock(punishment) outcomes., with no precursors to

rats but display as which light would be on. Outcomes

was eventually displayed while rats would then

experience the FSCV session mentioned above.

False color plot from FSCV indicates that DA

(dopamine) from rats merely released when outcome

cue was displayed in contrast with little, or declined

reaction after the directional cue. This indicates a

correlation between DA release and reaction in

response to a reward circuitry, or experimental

conditioning. Some researchers also argue a depletion

of DA neuron while individual encounter a reward-

punishment process (Willard, etc. 2019, Bouchard

2015, Morita 2018, Lindahl & Hellgren 2017).

Figure 3: Fast-Scan Cyclic Voltammetry (FSCV) image presenting release of dopamine within rats.

4 FUNCTIONS OF DOPAMINE

4.1 Role in the Prefrontal Cortex of the

Brain

Researchers believe that the role of dopamine is not

only to use rewards to learn the value of past

behaviors, but also that dopamine plays an

indispensable role in the prefrontal cortex of the

brain, enabling us to learn new tasks efficiently,

quickly and flexibly (Wang, Kurth-Nelson, Kumaran

2018).

These researchers from London (deepmind

organization) tested their theories by simulating six

meta-learning experiments in the field of

reconstructed neuroscience-each experiment requires

an agent to perform tasks that use the same basic

principles (or the same set of Skills), but different in

some ways.

An experiment they replicated is called the

Harlow experiment, which was a psychology

experiment in the 1940s to explore the concept of

meta-learning. In the original test, a group of

monkeys were shown two unfamiliar objects, and

only one of them would give them food rewards. The

two objects were displayed 6 times in total, each time

they were placed randomly, so the monkey must

know which one will give them food rewards. Then,

they were shown two other new objects again, and

again, only one of them would give them food.

During this training process, the monkey develops

a strategy to select objects that can be rewarded: it

learns to choose randomly the first time, and then, the

next time it chooses specific objects based on reward

feedback, instead of from left to right. Right

selection. This experiment shows that monkeys can

internalize the basic principles of tasks and learn an

abstract structure of rules—in fact, they learn how to

learn.

Impact of Dopamine on Reward Related Behavior of Animals and Therapeutic Role

137

In fact, researchers found that the meta-RL

(reinforcement learning) agent can learn how to

quickly adapt to various tasks with different rules and

structures. Moreover, since the network has learned

how to adapt to various tasks, it has also learned

general principles on how to learn effectively (Wang,

Kurth-Nelson, Kumaran 2018, Wang, Smith &

Delgado 2016). They also found that most of the

learning takes place in the recurrent network, which

supports the view that the role of dopamine in the

learning process is more important than previously

thought. While traditionally, dopamine is thought to

strengthen the synaptic connections of the prefrontal

system, thereby strengthening specific behaviors.

4.2 Role in the Parkinson’s Disease

Another function of dopamine is reflected in the

concept of Parkinson’s disease (Opara, Małecki &

Socha 2017, Radhakrishnan & Goyal 2018, Seppi &

Ray Chaudhuri 2019). With modern development,

brain detection instruments like CT scan, MRI, and

FMRI have been widely used in psychology,

neuroscience and even medical fields (Tocchio,

Kline-Fath, Kanal, Schmithorst & Panigrahy 2015,

Villanueva-Meyer, Mabray & Cha 2017). Among

them, FMRI has been heavily invested in the research

on the reinforcement learning process and reward

circuitry described in this article (Wang, Smith &

Delgado 2016, Glover 2011). Parkinson's disease

causes a characteristic combination of motor

symptoms due to progressive neurodegeneration of

dopaminergicneurons in the substantia nigra pars

compacta (Glover 2011).

Currently, although there is no cure for

Parkinson's disease with good results, we still cannot

deny the existence of effective drugs (Seppi & Ray

Chaudhuri 2019). In addition to being used for

patients, these drugs can also test the change in

dopamine and it's influence on parkinson's disease

(Wang, Kurth-Nelson, Kumaran, et al 2018).

Approaches in relevance to capturing dopamine

variation in cerebral function, is to test patients in two

conditions under FMRI scan; one is after dopamine

withdrawal with relatively low levels of dopamine in

a pragmatic OFF-medication state and once after

dopamine intake with relatively high levels of

dopamine in an ON-medication state. The differences

in the patient's behaviour and neural activation

between the ON- and OFF-medication state,

considered together with the behaviour and activation

patterns of healthy control participants, is then used

to infer the functional effects of dopamine in the

human brain. As shown in the figure, with the

application of medicines, the overall performance of

the patient has risen to a plateau, which is the top of

the performance, and then decline. (figure 4)

Figure 4: Performance mediated by nigro-dorsal stratal

circuit.

5 CONCLUSIONS

From the analysis of theory and actual cases, when

animals encounter certain tasks, the secretion of

dopamine plays a significant role. When the hub

tissues in animal brains encounter the same results or

similar stimuli again and again, the secretion

channels of dopamine are narrowed again and again

until a fixed circuitry is formed. Furthermore, such

discovery also further contributes to modern

education or animal domestication. Human education

is more from the perspective of students, similar to

analogy, to promote students' learning. Unlike

humans, although animals are far inferior to humans

in their cognitive and observational abilities, animal

trainers can also ensure that animals learn and

understand certain tasks through such methods that

have been used extensively, assimilated and similar

to reinforcement learning. Simutaneously, one still

need to be skeptical of dopamine effect on human.

Although many experiments, including diseases as

Parkinson's disease, have demonstrated the role of

dopamine in human brain control and learning, too

many experiments, especially those related to

injection, violate the ethical guidelines of human

experiments, and it is difficult for researchers to reach

one solid conclusion stating a directly, causal effect

between the two.

REFERENCES

Bissonette, G. B., & Roesch, M. R. (2016). Development

and function of the midbrain dopamine system: what

we know and what we need to. Genes, brain, and

CAIH 2021 - Conference on Artiﬁcial Intelligence and Healthcare

138

behavior, 15(1), 62–73.

https://doi.org/10.1111/gbb.12257

Bova A, Gaidica M, Hurst A, Iwai Y, Hunter J, Leventhal

DK. (2020). Precisely timed dopamine signals establish

distinct kinematic representations of skilled

movements. Elife. Nov 27; 9: e61591. doi:

10.7554/eLife.61591. PMID: 33245045; PMCID:

PMC7861618.

Bromberg-Martin, E. S., Matsumoto, M., & Hikosaka, O.

(2010). Dopamine in motivational control: rewarding,

aversive, and alerting. Neuron, 68(5), 815–834.

https://doi.org/10.1016/j.neuron.2010.11.022

Burnham J. C. (1972). Thorndike's puzzle boxes. Journal of

the history of the behavioral sciences, 8, 159–167.

https://doi.org/10.1002/1520-

6696(197204)8:2<159::aid-jhbs2300080202>3.0.co;2

Glover G. H. (2011). Overview of functional magnetic

resonance imaging. Neurosurgery clinics of North

America, 22(2), 133–vii.

https://doi.org/10.1016/j.nec.2010.11.001

Haber S. N. (2014). The place of dopamine in the cortico-

basal ganglia circuit. Neuroscience, 282, 248–257.

https://doi.org/10.1016/j.neuroscience.2014.10.008

Ikemoto S, Wise RA. (2004). Mapping of chemical trigger

zones for reward. Neuropharmacology. 47 Suppl

1:190-201. doi: 10.1016/j.neuropharm.2004.07.012.

PMID: 15464137.

Ikemoto S. (2003). Involvement of the olfactory tubercle in

cocaine reward: intracranial self-administration

studies. The Journal of neuroscience: the official

journal of the Society for Neuroscience, 23(28), 9305-

9311. https://doi.org/10.1523/JNEUROSCI.23-28-

09305.2003

Ikemoto S. (2007). Dopamine reward circuitry: two

projection systems from the ventral midbrain to the

nucleus accumbens-olfactory tubercle complex. Brain

research reviews, 56(1), 27–78.

https://doi.org/10.1016/j.brainresrev.2007.05.004

Kashtelyan, V., Lichtenberg, N. T., Chen, M. L., Cheer, J.

F., & Roesch, M. R. (2014). Observation of reward

delivery to a conspecific modulates dopamine release

in ventral striatum. Current biology : CB, 24(21), 2564–

2568. https://doi.org/10.1016/j.cub.2014.09.016

Leventhal D, Bova A. (2020). Precisely-timed dopamine

signals establish distinct kinematic representations of

skilled movements. figshare.

Lichtenberg, N. T., Lee, B., Kashtelyan, V., Chappa, B. S.,

Girma, H. T., Green, E. A., Kantor, S., Lagowala, D.

A., Myers, M. A., Potemri, D., Pecukonis, M. G.,

Tesfay, R. T., Walters, M. S., Zhao, A. C., Blair, R.,

Cheer, J. F., & Roesch, M. R. (2018). Rat behavior and

dopamine release are modulated by conspecific

distress. eLife, 7, e38090.

https://doi.org/10.7554/eLife.38090

Lindahl, M., & Hellgren Kotaleski, J. (2017). Untangling

Basal Ganglia Network Dynamics and Function: Role

of Dopamine Depletion and Inhibition Investigated in a

Spiking Network Model. eNeuro, 3(6),

ENEURO.0156-16.2016.

https://doi.org/10.1523/ENEURO.0156-16.2016

McCutcheon, J. E. etc. (2012). Encoding of aversion by

dopamine and the nucleus accumbens. Frontiers in

neuroscience, 6, 137.

https://doi.org/10.3389/fnins.2012.00137

Meder, D., Herz, D. M., Rowe, J. B., Lehéricy, S., &

Siebner, H. R. (2019). The role of dopamine in the brain

- lessons learned from Parkinson's disease.

NeuroImage, 190, 79–93.

Morita, K., & Kato, A. (2018). A Neural Circuit

Mechanism for the Involvements of Dopamine in

Effort-Related Choices: Decay of Learned Values,

Secondary Effects of Depletion, and Calculation of

Temporal Difference Error. eNeuro, 5(1),

ENEURO.0021-18.2018.

https://doi.org/10.1523/ENEURO.0021-18.2018

Oleson, E. B., etc. (2012). Subsecond dopamine release in

the nucleus accumbens predicts conditioned

punishment and its successful avoidance. The Journal

of neuroscience: the official journal of the Society for

Neuroscience, 32(42), 14804–14808.

Opara, J., Małecki, A., Małecka, E., & Socha, T. (2017).

Motor assessment in Parkinson`s disease. Annals of

agricultural and environmental medicine: AAEM,

24(3), 411–415.

https://doi.org/10.5604/12321966.1232774

Radhakrishnan, D. M., & Goyal, V. (2018). Parkinson's

disease: A review. Neurology India, 66(Supplement),

S26–S35. https://doi.org/10.4103/0028-3886.226451

Seppi, K., Ray Chaudhuri, K., & the collaborators of the

Parkinson's Disease Update on Non-Motor Symptoms

Study Group on behalf of the Movement Disorders

Society Evidence-Based Medicine Committee (2019)

Movement disorders : official journal of the Movement

Disorder Society, 34(2), 180-198.

https://doi.org/10.1002/mds.27602

Sveinbjornsdottir S. (2016). The clinical symptoms of

Parkinson's disease. Journal of neurochemistry, 139

Suppl 1, 318–324. https://doi.org/10.1111/jnc.13691

Tocchio, S., Kline-Fath, B., Kanal, E., Schmithorst, V. J.,

& Panigrahy, A. (2015). MRI evaluation and safety in

the developing brain. Seminars in perinatology, 39(2),

73–104. https://doi.org/10.1053/j.semperi.2015.01.002

Villanueva-Meyer, J. E., Mabray, M. C., & Cha, S. (2017).

Current Clinical Brain Tumor Imaging. Neurosurgery,

81(3), 397–415. https://doi.org/10.1093/neuros/nyx103

Wang, J.X., Kurth-Nelson, Z., Kumaran, D. et al. (2018).

Prefrontal cortex as a meta-reinforcement learning

system. Nat Neurosci 21, 860–868.

https://doi.org/10.1038/s41593-018-0147-8

Wang, K. S., Smith, D. V., & Delgado, M. R. (2016). Using

fMRI to study reward processing in humans: past,

present, and future. Journal of neurophysiology,

115(3), 1664–1678.

https://doi.org/10.1152/jn.00333.2015

Willard, A. M., Bouchard, R. S., & Gittis, A. H. (2015).

Differential degradation of motor deficits during

gradual dopamine depletion with 6-hydroxydopamine

in mice. Neuroscience, 301, 254–267.

https://doi.org/10.1016/j.neuroscience.2015.05.068

Impact of Dopamine on Reward Related Behavior of Animals and Therapeutic Role

139

Willard, A. M., Isett, B. R., Whalen, T. C., Mastro, K. J.,

Ki, C. S., Mao, X., & Gittis, A. H. (2019). State

transitions in the substantia nigra reticulata predict the

onset of motor deficits in models of progressive

dopamine depletion in mice. eLife, 8, e42746.

https://doi.org/10.7554/eLife.42746

CAIH 2021 - Conference on Artiﬁcial Intelligence and Healthcare

140