The second one is the antisymmetric term from the
synaptic matrix. The terms following that correspond
to the pattern under retrieval and the interference term
due to the other condensed pattern.
2.3 Assessing the Retrieval Properties
In this paper, we aim to assess the retrieval proper-
ties of a network of associative memory with more
than one condensed memories. We shall work in the
framework of an initial value problem, in which the
initial (t = 0) neural network conﬁguration, {σ
i
(0)},
has an overlap m
k
with the stored pattern ξ
k
i
. The evo-
lution of the system to a state that has overlap m
l
with
a stored pattern ξ
l
i
that is sufﬁciently close to unity is
referred to as the successful of the latter pattern. In
the presence of interference from an additional con-
densed pattern, the following quantities are of partic-
ular concern:
1. Retrieval quality, i.e., the degree of closeness of
the ﬁnal state to the relevant pattern under re-
trieval.
2. Convergence time, i.e., time taken by the neural
network to converge to a ﬁnal state close to the
corresponding stored pattern.
The simulation is done on self-consistent single-spin
dynamics that is obtained using the Dynamical Mean
Field Theory described in the following section.
3 DYNAMICAL MEAN FIELD
THEORY
Under the assumption that each spin is coupled to
all the other spins we shall show that the internal
ﬁeld, h
i
(t), which depends explicitly on the states of
all the spins, can be replaced by an effective ‘mean
ﬁeld’ which depends only on some macroscopic or-
der parameters. This effective internal ﬁeld is a time-
dependent random process and is different from the
averaged internal ﬁeld [h
i
(t)]. The random processes
for effective ﬁeld h
i
(t) can be constructed conve-
niently using the Dynamical Generating Functional
technique. This effective ﬁeld can be used to generate
stochastic spin trajectories in a monte-carlo simula-
tion.
We are interested in the statistical properties of a
large, but ﬁnite number N
T
of spin trajectories over
t
f
time steps, at the sites i = 1,. . . , N
T
, out of a sys-
tem in which the total number of spins, N, may be
very large. To derive these properties, we consider
the dynamical generating function
h
Z(l)
i
J
for the lo-
cal ﬁelds h
i
(t), i = 1,...,N
T
; t = 1,. . . ,t
f
in a system
with just 2 patterns:
hZ(l)i
J
= hTr
σ(t)
R
∏
N
i=1
∏
t
f
t=1
{h
i
(t)Θ(σ
i
(t +1)
h
i
(t)) × δ(h
i
(t) − [
1+η
2
]
1/2
∑
j6=i
J
s
i j
σ
j
(t) +
h
1−η
2
i
1/2
∑
j6=i
J
as
i j
σ
j
(t) +
J
1
N
ξ
1
i
ξ
1
j
+
J
2
N
ξ
2
i
ξ
2
j
)}
×exp(
∑
t
f
t=0
∑
N
T
i=1
l
i
(t)h
i
(t))i
J
. (7)
Here, h···i
J
represents the average over the all possi-
ble random couplings and Tr
σ
denotes the sum over
all 2
N t
f
possible states of the spin-system, σ
i
(t) = ±1.
θ(x) and δ(x) are the unit step function and the dirac
delta function respectively. These functions ensure
that only those ‘spin paths’ σ
i
(t) which are consistent
with the equations of motion (5) and (6)contribute to
h
Z(l)
i
J
.
The calculation of
h
Z(l)
i
J
follows, closely, the
derivation given in (Eissfeller and Opper, 1994) (for
the asynchronous case with J
0
= 0), which in turn is a
generalization of the derivation given by (Henkel and
Opper, 1991) for the synchronous dynamics of a neu-
ral network. Analysis showed us that in the large-N
limit, the generating function can be completely fac-
torised into independent components for the N
T
spins:
h
Z(l)
i
J
∝
∏
N
T
i=1
D
Tr
σ
i
(t)
R
∏
t
{
dh
i
(t)Θ (σ
i
(t + 1)
h
i
(t))
}
exp
{
i
∑
t
l
i
(t)h
i
(t)
}
∏
t
δ
h
i
(t)− J
1
m
1
(t)ξ
2
i
−J
2
m
2
(t)ξ
2
i
− φ
i
(t)− η
∑
s
K(t,s)σ
i
(s)
φ
. (8)
In this form of the generating function, we see that
the dynamics of the spin system is described by the
uncorrelated system of dynamical equations:
σ
i
(t +1) = sign (h
i
(t)) , (9)
where
h
i
(t) = J
1
m
1
(t)ξ
1
i
+ J
2
m
2
(t)ξ
2
i
+ φ
i
(t) +
η
∑
s<t
K(t,s)σ
i
(s). (10)
We have effectively replaced the time-independent
random couplings to other spins by a Gaussian ran-
dom variables φ
i
(t), with zero mean and covariance
h
φ
i
(t)φ
i
(s)
i
φ
= C(t, s), introduced independently for
each site i. In the above equation (Eq. 10),the ﬁrst two
terms in the above ‘effective’ local ﬁeld come from
the mean ﬁeld theory and are responsible pattern re-
trieval, the third term is a Gaussian noise, while the
fourth term represents a retarded self-interaction.
The order parameters can be rewritten in terms of
the Gaussian averages:
C(t,s) =
h
φ(t)φ(s)
i
φ
=
h
σ(t)σ(s)
i
φ
, (11)
K(t,s) = −i
ˆ
h(s)σ(t)
φ
=
∂
∂φ(s)
σ(t)
φ
.(12)
MEAN FIELD MONTE CARLO STUDIES OF ASSOCIATIVE MEMORY - Understanding the Dynamics of a
Many-pattern Model
397