NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells
in a Nonlinear Model for HIV Dynamics:
Protease Inhibitors-based Approach
C. A. Pe˜na Fern´andez
1
a
, A. B. B. F. Cunha
2
and M. A. Alves
2
1
Electrotechnical Department at Federal Institute of Bahia, Rod. BR 324, KM 102, Feira de Santana, Brazil
2
Computer Science Department at Federal Institute of Bahia, Rod. BR 324, KM 102, Feira de Santana, Brazil
Keywords:
HIV, Neural Networks, Dynamic Backpropagation, Protease Inhibitors.
Abstract:
The present paper uses the learning ability of neural networks (NN) to design a nonlinear model and a nonlinear
controller that reduces the number of infected/uninfected CD4+ T cells into the HIV dynamic when an antiviral
therapy based on protease inhibitors is applied. The dynamic of the closed-loop system based on such therapy
is analyzed to understand the stability of infected/uninfected CD4+ T cells according to a global feedback law
that regards un-modeled dynamic terms. To this end, a robust control scheme based on NARMA-L2 approach
and a modified version of an already existing dynamic backpropagation algorithm is used to improve the
antiviral therapy performance (strongly related to the tracking error). The robustness of the proposed model
shows that antiviral therapy performance guarantees less infected CD4+ T cells.
1 INTRODUCTION
The study on the behavior of sensitive and resis-
tant cells infected by AIDS, a communicable disease
caused by the human immunodeficiency virus (HIV),
by using retroviral drugs and the study of responses
of these cells to certain treatments have been a high-
light in last years (Althaus and Boer, 2011; Hattaf
et al., 2009; Luo et al., 2012; Magnus and Regoes,
2011; Roy and Wodarz, 2012; Wang et al., 2014; Wil-
son, 2012; Wodarz and Hamer, 2007; Wodarz and
Hamer, 2007). The treatment for HIV/AIDS relies
on anti-retroviral drugs that suppress HIV viral load
below the limit of detection. Some drug classes are
reverse transcriptase inhibitors (RTI) and protease in-
hibitors (PIs). HIV is a deadly disease in a lack of
treatment. Time since the initial infection till death
in an untreated patient is 9-10 years. HIV infec-
tion dilapidates the patients’ health since it attacks
three extremely important cell populations, the CD4+
T cells, macrophages, and dendritic cells. In recent
works, the main goal has been to understand the be-
havior of an infected CD4+ T cells population or
the rate of infected CD4+ T cells when therapy is
applied (Pinto and Carvalho, 2015b; Pinto and Car-
valho, 2015a; Gumel et al., 2001; Karrakchou et al.,
a
https://orcid.org/0000-0003-0934-5761
2006; Min et al., 2008; Allali et al., 2018; Loudon and
Pankavich, 2016; Korpusik, 2017; Wang et al., 2019).
However, the correctness of any HIV model depends
on suitable approaches whereas by using suitable pa-
rameters for deterministic models or by finding a suit-
able performance for learning algorithms in compu-
tational methods (Wang et al., 2019). Nevertheless,
the adjusting process of the parameters and the per-
formance of the computational model yields a big
complexity on the better way to find the optimal solu-
tion in nonlinear programming problems. Such com-
plexity can be defined in terms of the constraints for
stochastic models as well as choosing the right parti-
cles to model (Liu et al., 2019; Pandit and Boer, 2019;
Cheng et al., 2019; Shi and Dong, 2019). Artificial in-
telligence (AI) techniques have been used extensively
in numerous approaches to model HIV around sev-
eral CD4+ T cells (Xiang et al., 2019; Zhang et al.,
2019; Wang et al., 2019; Gupta et al., 2019). Amongst
AI numerous techniques, the Artificial Neural Net-
works (ANN) has dominated in its capability and ap-
plicability in different fields (Cybenko, 1989). In
the work reported here, it will be presented an ap-
proach to model the behavior of uninfected/infected
CD4+ T cells based on NARMA approach, which
will be implemented with neural networks and known
as NARMA-L2. In this approach, the performance
Fernández, C., Cunha, A. and Alves, M.
NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells in a Nonlinear Model for HIV Dynamics: Protease Inhibitors-based Approach.
DOI: 10.5220/0008980606750683
In Proceedings of the 12th International Conference on Agents and Artificial Intelligence (ICAART 2020) - Volume 2, pages 675-683
ISBN: 978-989-758-395-7; ISSN: 2184-433X
Copyright
c
2022 by SCITEPRESS Science and Technology Publications, Lda. All rights reserved
675
of the dynamic model is mainly composed by using
an improved learning rule in neural networks based
on sigmoid-functions, as activation function, and the
dynamic backpropagation of the training error. The
controller developed on this approach uses the in-
verse model principle to synthesize an antiviral ther-
apy based on PI whose adjusting method updates the
value of weights of the neurons that model the HIV
dynamics. Next, it is hoped that a good adjusting
of those values implies a better efficiency of antiviral
therapy according to the response of resistant CD4+
T cells.
This paper presents the proposal as follows: in
Section 2 will be reviewed the stochastic model based
on the approach in (Wang et al., 2014). It will be
assumed that all antiviral therapy based on PI can
be decomposed in decoupled inputs such that the in-
fected/uninfected CD4+ T cells have independent re-
sponses. In order to impose a performance on the
antiviral therapy and the decreasing/increasing of in-
fected/uninfected CD4+ T cells, an optimal control
law based on the NARMA-L2 approach will be pro-
posed. To this end, in Section 3 will be synthesized
a control scheme with the aid of NNs and suitable
learning algorithms. In Section 4 are shown differ-
ent parametric structures for the NARMA-L2 model
by means different learning rules in order to adjust the
synaptic weights of NNs into NARMA-L2 scheme as
well as the performance of the antiviral therapy repre-
sented by the controller. Finally, closing remarks are
made in Section 5.
2 THEORETICAL BACKGROUND
ON HIV
As well known, a deterministic model can be based
on a system of ordinary differential equations that
represent the dynamic of uninfected CD4+ T cells,
drug-sensitive infectious virus, drug-resistant infec-
tious virus, drug-sensitive infected CD4+ T cells,
and drug-resistant infected CD4+ T cells. In (Wang
et al., 2014) such dynamics is defined by ODE system
(1a)-(1e), where T, V
s
, V
r
, T
s
and T
r
represent unin-
fected CD4+ T cells, drug-sensitive infectious virus,
drug-resistant infectious virus, drug-sensitiveinfected
CD4+ T cells, and drug-resistant infected CD4+ T
cells, respectively. According to the epidemiology of
the disease, the uninfected CD4+ T cells (T) are pro-
duced at a rate of λ and die at a rate of d. These cells,
when in contact with HIV, are infected at a rate k
s
, by
drug-sensitive viruses and move τ time units later, to
the T
s
class. Furthermore, T cells may be infected at
a rate of k
r
by drug-resistant viruses and move to the
T
r
class after τ time units later. The term e
mτ
is the
probability of T
s
and T
r
cells surviving in the inter-
val τ, where
1
m
represents the average life of infected
CD4+ T cells before they become actively productive.
The terms 1 n
s
rt
and 1 n
r
rt
represent the proportion
of T
s
and T
r
cells eliminated by reverse transcriptase
inhibitors (RTIs). The infected CD4+ T cells and the
virus particles die at rates δ and c, respectively. TheV
s
and V
r
particles are produced by the infected CD4+ T
cell populations with bursting sizes of drug-sensitive
strain (N
s
) and of drug-resistant strain (N
r
). Through-
out the infection, a proportion s (0 < s < 1) of T
s
cells
can become T
r
cells. The proportion of virus particles
that are not eliminated by PIs is represented by 1 n
s
p
and 1 n
r
p
, where n
s
p
is the efficacy of PIs for wild
type strain and n
r
p
is the efficacy of PIs for mutants.
According to (Pinto and Carvalho, 2015b), on
stochastic approach for model (1a)-(1e), it is believed
that the death rates d, δ and c can be substituted
by random parameters, more specifically, d + σ
1
˙
B
1
,
δ+ σ
1
˙
B
1
and c+ σ
2
˙
B
2
, respectively, where B
1
(t) and
B
2
(t) are independent standard wiener processes. In
the next sections, such wiener processes will be used
as exogenous variables into the learning algorithms.
2.1 HIV Model by Including PI
By assuming that the constant recruitment number of
new uninfected cells T and that the rate of infection
of CD4+ T cells k
s
by a free virus has been saturated
probably because of overcrowding of free virus the
model (1a)-(1e) can be rewritten as
˙
h = G(h) (2)
where h = [T T
s
V
s
T
r
V
r
]
T
IR
5
is the state vector
and G(h) = [G
1
(h) G
2
(h) G
3
(h) G
4
(h) G
5
(h)]
T
IR
5
is assumed continuouslydifferentiable on the parame-
ters (T,T
s
,V
s
,T
r
,V
r
,t) D
T
×D
T
s
×D
V
s
×D
T
r
×D
V
r
×
[0,t], being D
T
,D
T
s
,D
V
s
,D
T
r
,D
V
r
IR
+
open and
convex sets, and each G
i
(h), for i = 1,.. .,5, repre-
sents the right-hand side of (1a)-(1e), respectively.
Now, let p = (p
1
, p
2
, p
3
, p
4
, p
5
) be an equilibrium
point of the nonlinear system (1a)-(1e) and suppose
that the functions G
i
, for i = 1, ...,5, are continuously
differentiable. Expanding the right-hand side of (2)
into its Taylor series on the point p yields
˙
h = M(p)h+ (p)u + N(p) (3)
where M(p) is the Jacobian matrix at equilibrium
point p, defined by M(p) = [G(h)/h
i
] (for i =
1,.. . ,5), u IR
2
is the antiviral therapy defined by
u = [n
s
p
n
r
p
]
T
, (p) is the input matrix with full
rank, defined by (p) = [G(h)/u
i
] (for i = 1,2)
and N(p) = N(h)|
h=p
IR
5
represents the higher-
order terms of expansion. In the antiviral therapy,
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
676
dT
dt
= λ dT(t) K
s
(1 n
s
rt
)V
s
(t)T(t) k
r
(1 n
r
rt
)V
r
(t)T(t) (1a)
dT
s
dt
= (1 s)k
s
e
mτ
(1 n
s
rt
)V
s
(t τ)T(t τ) δT
s
(t) (1b)
dV
s
dt
= N
s
δ(1 n
s
p
)T
s
(t) cV
s
(t) (1c)
dT
r
dt
= sk
s
e
mτ
(1 n
s
rt
)V
s
(t τ)T(t τ) + k
r
e
mτ
(1 n
r
rt
)V
r
(t τ)T(t τ) δT
r
(t) (1d)
dV
r
dt
= N
r
δ(1 n
r
p
)T
r
(t) cV
r
(t) (1e)
Figure 1.
the efficacy of PI, n
s
p
and n
r
p
, are considered bounded
Lebesgue integrable functions, i.e.,
j=1
Z
n
s
p, j
< and
j=1
Z
n
r
p, j
< .
If, for instance, n
s
p
= 1, the blockage is 100% effec-
tive. On the other hand, if n
s
p
= 0, there is no block-
age.
The term N(h)|
h=p
is associated with an un-
modeled dynamic and it has an important role to mea-
sure the performance of the alternative model based
on neural networks, which will be shown in the next
sections.
2.2 Analysis on Inner Close-loop
In the work reported here, it will be assumed that
there exists an initial quantity of uninfected CD4+ T
cells, T
> 0, such that the antiviral therapy guaran-
tees a number of uninfected CD4+ T cells close to T
.
From (3), if T
> δ
v
(t) > 0, being δ
v
(t) a known 1-
diffeomorphism, and h is assumed to be the control
input of the inner close-loop system associated to (3)
then the error converges exponentially to small ball
in W containing the origin whose radius will be ad-
justed by any known constant. The following lemma
explains this idea:
Lemma 1. Let e be the error associated with in-
ner close-loop system defined by e = Π
1
(h), where
Π
1
(h) IR
5
is a known matrix. If T
> δ
v
(t) > 0 and
h = ρ, then e converges exponentially to a small ball
containing the origin whose radius can be adjusted
by a k-th known constant M
k
< 0, for k = 1,... , 5,
t 0, being ρ = [T
···]
T
= Π
2
(e) IR
5
such that
˙
Π
2
(e) < M and
M = [M
1
,M
2
,... ,M
5
]
T
being M
k
< 0 a known constant.
Proof: Let the Lyapunov function
V =
1
2
5
i=2
e
2
i
!
=
1
2
5
i=2
Π
i
1
(h)
2
!
,
differentiating it along the close-loop defined by e =
Π
1
(h), h = ρ and ρ = Π
2
(e) one obtains
˙
V =
5
i=2
5
k=1
∂Π
i
1
h
k
˙
h
k
!
Π
i
1
(h)
=
5
i=2
5
k=1
∂Π
i
1
h
k
˙
Π
k
2
(e)
!
Π
i
1
(h)
=
5
i=2
5
k=2
∂Π
i
1
h
k
˙
Π
k
2
(e)Π
i
1
(h) +
∂Π
i
1
h
k
˙
δ
v
Π
i
1
(h)
!
<
5
i=2
5
k=2
∂Π
i
1
h
k
˙
Π
k
2
(e)Π
i
1
(h)
!
=
5
i=2
5
k=2
∂Π
i
1
h
k
M
k
Π
i
1
(h)
!
< 0.
where M
k
is the k-th element of M, Π
i
1
(h) and Π
k
2
(e)
are the i-th and k-th elements of Π
1
(h) and Π
2
(e),
respectively.
Noting T
δ
v
(t) > 0 (i.e.,
˙
δ
v
(t) < 0 ) it can be
seen that V exponentially converges to a small ball
containing the origin. The convergence rate is at least
defined by Π
1
(h). The radius of the ball can be ad-
justed by a constant M
k
, for k = 1, ...,5. Therefore,
e converges exponentially to the small ball contain-
ing the origin, and the radius of the ball can also be
adjusted by any constant M
k
. This ends the proof .
So, the control problem considered in this paper is
to find an improved version of a global feedback con-
troller that represents the antiviral therapy such that
the inner close-loop of system (3) guarantees that the
number of uninfected CD4+ T cells remains close to
T
. To this end, in the next sections, it will be used
neural networks (NNs) to adjust this optimal global
feedback law.
NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells in a Nonlinear Model for HIV Dynamics: Protease Inhibitors-based
Approach
677
3 NARMA-L2-BASED THERAPY
The optimal control problem will an objective func-
tional, a set of state variables h and a set of control
variables u in time t, 0 t t
f
such that there is a
generic global feedback control u = λ
1
(h,h
r
) where
h
r
is the target response, λ
1
: IR
5
× IR
5
IR
2
is a
known nonlinear function such that there is an opti-
mal control law u , λ
1
that satisfies J(λ
(h,h
r
)) =
max
λ
1
(h,h
r
)U
J(u) being U IR
2
and
J(u) =
Z
t
f
0
T(t) u
T
Ru
dt
for t
f
> 0 and any symmetric definite positive ma-
trix R. Our goal is to seek to maximize the ob-
jective functional J(u) by increasing the population
of the uninfected CD4+ T cells, reducing viral load,
and minimizing the cost treatment. To this end, the
nonlinear model (1a)-(1e) will be designed by using
a robust adaptive neural network (NN) with the aid
of NARMA-L2 approach and its learning ability. In
this way, the maximization problem will be associated
with the nonlinear programming problem restricted to
the learning rule of NNs.
Let
˜
h = h ρ be the error between the state vec-
tor h and the state vector defined by Lemma 1 on the
close-loop system. Thus, for
˜
h one obtains
˙
˜
h+ M(p)
˜
h = (p)u Γ(p,ρ) (4)
where
Γ(p,ρ) ,
˙
ρ+ M(p)ρ + N(p).
Noting the learning ability of NN, the term Γ(p, h)
can be approximated on-line by using NNs whose ac-
tivation function is a sigmoid. Let σ(p,h) IR
m
be
a vector of continuous sigmoid functions, according
to the approximation property of neural networks in
(Cybenko, 1989), if Γ(p,ρ) is a continuous function
of p and ρ then
Γ(p,ρ) = ψ
T
σ(p,ρ) + ε
n
(p,ρ), (5)
where ψ , {ψ
ij
} IR
m
×3
is an unknown optimal
constant weight vector, previously calculated, and de-
fined by
ψ = arg min
ςIR
m
×3
max
(p,ρ)
kΓ(p,ρ) ς
T
σ(p,ρ)k
where is a compact set and ε
n
is the error associated
with reconstruction of the optimal weight vector. The
approximations results for NN indicate that if m
is
sufficiently large then the reconstruction error can be
arbitrarily small on . In this way, it will assumed
that kε
n
(p,ρ)k ϕ
, for (p,ρ) , where ϕ
is an
unknown constant.
TDL
TDL
...
...
...
...
...
TDL
-
NN I
NN II
...
...
...
...
...
...
...
...
...
...
(x5)
E(ψ)
T
h
˙
h = M(p)h
+(p)u+ N(p)
Figure 2: Closed-loop system by using NARMA-L2 ap-
proach in antiviral therapy.
Without loss of generality, by using (5), the equa-
tion (4) can be rewritten as
˙
˜
h+ M(p)
˜
h = (p)u ψ
T
σ(p,ρ) ε
n
(p,ρ)
s.t. kε
n
(p,ρ)k ϕ
,
(6)
where ψ and ϕ
must be estimated in order to guaran-
tee the robustness of the model (4). Thus, let
ˆ
ψ and
ˆ
ϕ
be the estimates of ψ and ϕ
, respectively. With
aid of Lemma 1 the following theorem is obtained:
Theorem 1. Assuming T
> δ
v
(t) > 0, the global
feedback law
u =
1
(p)
K
p
˜
h+
ˆ
ψ
T
σ
ˆ
ϕ
T(
˜
h)
, (7)
guarantee that e,
˜
h, (
ˆ
ψψ) and (
ˆ
ϕ
ϕ
) converge to
a ball containing the origin whose radius can be made
as small as possible by selecting a suitable value for
γ
1
, γ
2
, δ
1
, δ
2
, ψ
0
, and ϕ
0
, where
˙
ˆ
ψ = γ
1
σ
˜
h
T
δ
1
(
ˆ
ψ ψ
0
) (8)
˙
ˆ
ϕ
= γ
2
˜
h
T
T(
˜
h) δ
2
(
ˆ
ϕ
ϕ
0
) (9)
are update laws, K
p
is a positive definite constant ma-
trix, T(
˜
h) is a known sigmoid function and γ
i
, δ
i
(for
i = 1,2) are positive constants.
Proof: Let the Lyapunov function
V =
1
2
5
i=2
e
2
i
!
+
1
2
˜
h
T
˜
h+
γ
1
1
2
(
ˆ
ψ ψ)
T
(
ˆ
ψ ψ)
+
γ
1
2
2
(
ˆ
ϕ
ϕ
)
2
,
and differentiating it along of the closed-loop repre-
sented by Lemma 1 and (7), one obtains
˙
V =
5
i=2
5
k=1
∂Π
i
1
h
k
˙
h
k
+
˜
h
˙
˜
h
+ γ
1
1
(
ˆ
ψ ψ)
T
˙
ˆ
ψ+ γ
1
2
(
ˆ
ϕ
ϕ
)
T
˙
ˆ
ϕ
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
678
0 10 20 30 40
-40
-20
0
20
40
60
LM
GD
BFGS
0 10 20 30 40
-500
0
500
1000
1500
2000
0 10 20 30 40
-1500
-1000
-500
0
500
1000
1500
2000
0 10 20 30 40
-2000
-1000
0
1000
2000
0 10 20 30 40
-2000
-1500
-1000
-500
0
500
E(ψ) for T
E(ψ) for T
s
E(ψ) for T
r
E(ψ) for V
r
E(ψ) for V
s
t (s)
t (s)t (s) t (s)
t (s)
Figure 3: Training error for B
1
(t), E(ψ), of NN II by using three algorithms: Levenberg-Marquardt (LM), gradient descent
(GD), Broyden-Fletcher-Goldfarb-Shanno (BFGS).
0 10 20 30 40
-2
-1
0
1
2
3
4
0 10 20 30 40
-0.8
-0.6
-0.4
-0.2
0
0.2
0.4
0 10 20 30 40
-0.6
-0.4
-0.2
0
0.2
0.4
0.6
0 10 20 30 40
-1
-0.5
0
0.5
1
0 10 20 30 40
-40
-20
0
20
40
60
80
100
E(ψ) for T
E(ψ) for T
s
E(ψ) for T
r
E(ψ) for V
r
E(ψ) for V
s
t (s)
t (s)
t (s)
t (s)
t (s)
Figure 4: Training error for B
2
(t), E(ψ), of NN II by using three algorithms: Levenberg-Marquardt (LM), gradient descent
(GD), Broyden-Fletcher-Goldfarb-Shanno (BFGS).
or, by using (7) into (4),
˙
V =
5
i=2
5
k=1
∂Π
i
1
h
k
˙
h
k
+
˜
h
K
p
˜
h+
ˆ
ψ
T
σ
ˆ
ϕ
T(
˜
h)
Γ(p,ρ) + M(p)
˜
h
+ γ
1
1
(
ˆ
ψ ψ)
T
˙
ˆ
ψ
+ γ
1
2
(
ˆ
ϕ
ϕ
)
T
˙
ˆ
ϕ
< c
1
V + c
2
,
where c
1
is a positive constant which depends on K
p
,
γ
1
, γ
2
, δ
1
, δ
2
, and
c
2
=
γ
1
1
δ
1
2
(
ˆ
ψ ψ
0
)
T
(
ˆ
ψ ψ
0
)
+
γ
1
2
δ
2
2
(
ˆ
ϕ
ϕ
0
)
2
+ 2b
,
being b
a constant which satisfies b
= e
(b
+1)
[i.e.,
b
= 0.2785]. Without loss of generality, it can be
noted that V converges exponentially to a small ball
containing the origin and the radius of the ball de-
pends on γ
1
, γ
2
, r, δ
1
, δ
2
, ψ
0
and ϕ
0
. Therefore, e,
˜
h,
(
ˆ
ψ ψ) and (
ˆ
ϕ
ϕ
) will converge exponentially to
the small ball of the origin. This ends the proof .
The NN-based controller (7) uses the update laws
(8)-(9) to approximate un-modeled dynamics. In this
way, the laws (8)-(9) represent the learning ability of
the NN responsible to optimize the synaptic weights
(
ˆ
ψ
T
σ) that indirectly will ensure that the reconstruc-
tion and tracking errors converge toward a small ball
containing the origin in the sets and W . Therefore,
in the work reported here, the global feedback law in
0 10 20 30 40
× 10
-4
-4
-2
0
2
4
6
0 5 10 15 20 25 30 35 40 45
0
1
2
3
4
T
r
V
s
V
r
35 40 45
0
0.1
0.2
t (s)
t (s)
h for T
s
,V
s
,V
r
y
k+2
for T
s
,V
s
,V
r
Figure 5: Validation of NARMA-L2 model by comparing
(y) of NN II with output (h) of model defined by (3), with
exogenous brownian noise B
1
(t).
Theorem 1 will be based on an alternative learning
rule that combines the laws (8)-(9) with an already
existing algorithm for dynamic backpropagation.
3.1 NARMA-L2 Training Algorithm
The NARMA-L2 scheme to design the antiviral ther-
apy u is based on two NNs, the NN II is composed
of ve NNs and it will be used to model the system
(3). The NN I will be used as antiviral therapy by
NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells in a Nonlinear Model for HIV Dynamics: Protease Inhibitors-based
Approach
679
5 10 15 20 25 30 35 40 45
0
1
2
3
0 5 10 15 20 25 30 35 40 45
-0.5
0
0.5
1
1.5
2
t (s)
t (s)
h for T
s
,V
s
,V
r
y
k+2
for T
s
,V
s
,V
r
Figure 6: Validation of NARMA-L2 model by comparing
(y) of NN II with output (h) of model defined by (3), with
exogenous brownian noise B
2
(t).
means of the global feedback law in Theorem 1 (see
Fig. 2). The input T
represents a reference for T(t),
and with this last one the main objective of the pro-
posal reported here is to find an antiviral therapy that
guarantees a better decreasing of drug-sensitive infec-
tious virus V
s
, drug-resistant infectious virus V
r
and
drug-resistant infected CD4+ T cells T
r
.
In order to verify the effectiveness of the
NARMA-L2, the NN II is trained by using three algo-
rithms: Levenberg-Marquardt, Gradient Descent, and
BFGS
1
. All these, based on the error backpropagation
principle for off-line mode. It will be chosen the al-
gorithm more suitable to model each state variable of
the h vectorin order to guarantee a better performance
between the system and the controller represented by
NN I. On the other hand, the NN I is trained by us-
ing only dynamic backpropagation and to this end,
the BFGS approach is modified according to feedback
law in Theorem 1.
Since the inputs associated with PIs were assumed
to be mutually decoupled, the NARMA-L2 model
was designed individually for each state variable. A
standard NARMA-L2 model can be defined by
y
k+2
=
¯
f
0
[TDL(y
k
,n),TDL(u
k
,n)]
+ ¯g
0
[TDL(y
k
,n),TDL(u
k
,n)]u
k
,
where TDL(p
k
,n) , p
k
,... , p
kn+1
represents a
tapped delay line with n regressors (see TDL blocks in
Fig. 2);
¯
f
0
,
¯
F and ¯g
0
,
¯
F/u
k
are evaluated around
TDL(y
k
,n), TDL(u
k
,n) (with u
k
= 0) for any nonlin-
ear function
¯
F(·) (Narendra and Parthasarathy, 1990).
1
Broyden-Fletcher-Goldfarb-Shanno.
To guarantee the laws
ˆ
ψ and
ˆ
ϕ
, a segmented
training based on BFGS algorithm will be used to
compute the synaptic weights of NN I from a mod-
eling error between the system (3) and NN II. Now,
the learning rule proposed here can be described by
using kε
n
(p,ρ)k , kε
n
(ψ)k as goal function accord-
ing to next steps:
s1. Init B
0
= I, kε
n
(ψ
0
)k = n
e
and
ˆ
ϕ
= n
v
,
where B
0
is the initial condition for a Hessian ma-
trix of ε
n
(ψ), n
e
100 and n
v
< 0.001. From (9),
if
ˆ
ϕ
= n
v
then
ˆ
ϕ
=
γ
2
δ
2
˜
h
T
k
T(
˜
h) +
ϕ
0
δ
2
,
thus the BFGS algorithm becomes as
s2. While kε
n
(ψ
k
)k >
γ
2
δ
2
˜
h
T
k
T(
˜
h) +
ϕ
0
δ
2
do
s3. ∆ψ
k
= α
k
B
1
k
∇ε
n
(ψ
k
);
s4. Compute ∇ε
n
(ψ
k+1
);
where ∇ε
n
(ψ
k
) represents the gradient descent as
first-order iterative optimization tool. So,
s5. E
k
(ψ) = ∇ε
n
(ψ
k+1
) ∇ε
n
(ψ
k
);
s6. β
k
= I E
k
∆ψ
T
k
/E
T
k
∆ψ
k
;
Next, from step (s3.) and (8), by using the pseu-
doinverse of σ(p,h), σ
+
(p,h) yields
s7.
˜
h
T
k
=
σ
+
γ
1
δ
1
(ψ
k
ψ
0
) α
k
B
1
k
∇ε
n
(ψ
k
)
;
s8. B
1
k+1
=β
k
B
1
k
β
k
+ ∆ψ
T
k
∆ψ
k
/E
T
k
∆ψ
k
;
s9. ψ
k+1
= ψ
k
+ ∆ψ
k
;
s10. Compute kε
n
(ψ
k
)k and go to step (s2.);
It can be seen that the term K
p
˜
h is used to make
h converge to ρ while the term
ˆ
ϕ
T(
˜
h) is used to
compensate the reconstruction error kε
n
(ψ
k
)k of NN
I. The term
ˆ
ψ
T
σ in (7) uses a dynamic learning rule
based on laws (8)-(9) and BFGS algorithm to adjust
the synaptic weights of NN I and to enhance the ro-
bustness of the un-modeled dynamics N(p).
4 SIMULATION
The simulation tests were done by using the nonlinear
NARMA-L2-based controller with a two-layer NN in
order to implement (7) (in Theorem 1). All this, when
the antiviral therapy is applied with initial uninfected
CD4+ T cells.
The NARMA-L2 model was set with n = 3 re-
gressors (in TDL blocks), 20 iterations at BFGS al-
gorithm and 30 hidden neurons at the middle layer
of NNI. In previous simulation tests, it was noted
that when the number of hidden neurons is less than
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
680
t (s)
V
s
(t)
Figure 7: Validation of the drug-sensitive infectious virus
V
s
based on the global feedback law (7), in Theorem 1,
by comparing the behavior of each variable with control u
(solid line) and without control u (dash line).
t (s)
V
r
(t)
Figure 8: Validation of the drug-resistant infectious virus
V
r
based on the global feedback law (7), in Theorem 1,
by comparing the behavior of each variable with control u
(solid line) and without control u (dash line).
30 the weights were adjusted to values that were not
enough robust. In order to train the NNI, it was con-
sidered that the antiviral therapy u represents 10% at
PIs efficacy, i.e., n
s
p
= 0.1 and n
r
p
= 0.1 for t > 0.
Complementary, since the signals B
i
(t), for i = 1, 2,
have independent increments for t > 0, the future
increments of the death rates are independent of the
past values. So, such signals were set as pseudo-
random binary signals (PRBS) with maxim ampli-
tude 1 and samples between 10 and 100 ms. The
parameters of model (1a)-(1e) were set as in (Wang
et al., 2014; Pinto and Carvalho, 2015b): λ = 100;
d = 0.1; m = 0.01; k
s
= 2.4× 10
6
; k
r
= 2× 10
6
;
s = 3× 10
5
; δ = 1; N
s
= 4800; N
r
= 4000; c = 23;
τ = 1; n
s
rt
= 0.4; n
r
rt
= 0.2; σ
1
= σ
2
= 0.1; with
initial conditions T(0) = T
= 1000, T
s
(0) = 1 and
t (s)
T
r
(t)
Figure 9: Validation of the drug-resistant infected CD4+ T
cells T
r
based on the global feedback law (7), in Theorem
1, by comparing the behavior of each variable with control
u (solid line) and without control u (dash line).
t (s)
T(t)
Figure 10: Validation of the uninfected CD4+ T cells T
based on the global feedback law (7), in Theorem 1, by
comparing the behavior of each variable with control u
(solid line) and without control u (dash line).
T
r
(0) = V
s
(0) = V
r
(0) = 0.01. In order to set the con-
troller, the parameters of nonlinear law (7) were set
as K
p
= γ
1
= γ
2
= k
2
= k
3
= 1, δ
1
= δ
2
= 0.01 and
T(
˜
h) = tanh(
˜
h).
In Fig. 5 and Fig. 6 can be seen the behavior of
variablesV
s
,V
r
and T
r
modeled by NN II on NARMA-
L2 approach by considering B
i
(t), for i = 1,2, as
Brownian signals. Complementarily, in Fig. 3 and
Fig. 4 can be seen the adjustment error of five sig-
nals associated with the NN II. For each NN was used
three algorithms based on the error backpropagation
principle. It can be noted that the BFGS algorithm
has a better approximation than Levenberg-Marquardt
(LM) and Gradient Descent (GD) approaches, except
for the behavior of T. For this reason, it will be
used the BFGS approach to model the responses of
NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells in a Nonlinear Model for HIV Dynamics: Protease Inhibitors-based
Approach
681
the variables T
s
, T
r
, V
r
and V
s
, while the Levenberg-
Marquardt algorithm will be used to model the re-
sponse of T.
4.1 Validation of the Antiviral Therapy
Since the main objective is to reduce drug-resistant in-
fected CD4+ T cells in order to guarantee T close to
T
, then the global feedback law (7) will use the abil-
ity of NNs to approximate nonlinear and un-modeled
behaviors (i.e., N(p)) in order to improve the perfor-
mance of the antiviral therapy. To this end, the signals
B
i
, for i = 1,2, will be set as exogenous inputs to the
closed-loop of (2) and the input u will be represented
by the outputs of the NARMA-L2-based controller.
By comparing the Fig. 7 and Fig. 8 can be seen
that the drug-sensitive infectious virus (V
s
) decreases
faster than the drug-resistant infectious virus (V
r
)
when the antiviral therapy u is applied during 100
days. It can be noted that the antiviral therapy guaran-
tees a decreasing of the contagion effect of the HIV,
see that T
r
decreases significantly on time domain in-
stead of the case when the antiviral therapy is not used
(see dash lines in Fig. 8).
Although the behavior of T was model by using
the LM approach in NN II, Fig. 10 shows that the
control law (7) guarantees that the concentration of
uninfected CD4+T cells decreasing close to a neigh-
borhood of T
(= T(0) = 1000). Without treatments
(without control u), the number of uninfected cells de-
creases drastically.
5 FINAL REMARKS
The controller in Theorem 1 differs from other con-
tributions mainly because the optimization approach
used to maximize any objective functional J(u) is
based on a nonlinear programming rule. Never-
theless, here was chosen the use of NARMA-L2
approach together with nonlinear programming for
training NN because it improves the performance of
the model and the controller used as antiviral ther-
apy, both reducing viral load, and minimizing the cost
treatment. Here, it was assumed that un-modeled in-
formation could be related to the learning ability of
the NNs as well as it was proposed an alternative
learning technique based on the dynamic of the back-
propagation approach. In this way, the robustness of
the NN-based controller (7) was improved by includ-
ing the nonlinear term
ˆ
ϕ
T(
˜
h) into the nonlinear
control law (see (7) in Theorem 1 and step (s2.)) such
that the convergence radius of the tracking error can
be regulated by known parameters. It is hoped that in
future works the NN-based controller can be designed
by using three or more layers to improve the perfor-
mance of the term
ˆ
ψ
T
σ(q
,h) and the compensation
of the un-modeled dynamics N(p).
REFERENCES
Allali, K., Meskaf, A., and Tridane, A. (2018). Mathemati-
cal modeling of the adaptive immune responses in the
early stage of the HBV infection. International Jour-
nal of Differential Equations, 2018:1–13.
Althaus, C. L. and Boer, R. J. D. (2011). Implications
of CTL-mediated killing of HIV-infected cells dur-
ing the non-productive stage of infection. PLoS ONE,
6(2):e16468.
Cheng, Y., Li, M., and Zhang, F. (2019). A dynamics
stochastic model with HIV infection of CD4+ t-cells
driven by l´evy noise. Chaos, Solitons & Fractals,
129:62–70.
Cybenko, G. (1989). Approximation by superposition of
sigmoidal functions. Mathematics of Control, Signals,
and Systems, 2:303–314.
Gumel, A., Shivakumar, P., and Sahai, B. (2001). A math-
ematical model for the dynamics of HIV-1 during the
typical course of infection. Nonlinear Analysis: The-
ory, Methods & Applications, 47(3):1773–1783.
Gupta, A., Salau, A. O., Chaturvedi, P., Akinola, S. A.,
and Nwulu, N. I. (2019). Artificial neural networks:
Its techniques and applications to forecasting. In
2019 International Conference on Automation, Com-
putational and Technology Management (ICACTM).
IEEE.
Hattaf, K., Rachik, M., Saadi, S., and Yousfi, N. (2009).
Optimal control of treatment in a basic virus infection
model. Applied Mathematical Sciences, 3(20):949–
958.
Karrakchou, J., Rachik, M., and Gourari, S. (2006). Op-
timal control and infectiology: Application to an
HIV/AIDS model. Applied Mathematics and Com-
putation, 177(2):807–818.
Korpusik, A. (2017). A nonstandard nite difference
scheme for a basic model of cellular immune response
to viral infection. Communications in Nonlinear Sci-
ence and Numerical Simulation, 43:369–384.
Liu, Q., Jiang, D., Hayat, T., and Alsaedi, A. (2019).
Stationary distribution and extinction of a stochastic
HIV-1 infection model with distributed delay and lo-
gistic growth. Journal of Nonlinear Science.
Loudon, T. and Pankavich, S. (2016). Mathematical anal-
ysis and dynamic active subspaces for a long term
model of HIV. Mathematical Biosciences and Engi-
neering, 14(3):709–733.
Luo, R., Piovoso, M. J., Martinez-Picado, J., and Zu-
rakowski, R. (2012). HIV model parameter estimates
from interruption trial data including drug efficacy and
reservoir dynamics. PLoS ONE, 7(7):e40198.
Magnus, C. and Regoes, R. R. (2011). Restricted occupancy
models for neutralization of HIV virions and popu-
ICAART 2020 - 12th International Conference on Agents and Artificial Intelligence
682
lations. Journal of Theoretical Biology, 283(1):192–
202.
Min, L., Su, Y., and Kuang, Y. (2008). Mathematical anal-
ysis of a basic virus infection model with application
to HBV infection. Rocky Mountain Journal of Mathe-
matics, 38(5):1573–1585.
Narendra, K. S. and Parthasarathy, K. (1990). Identifica-
tion and control of dynamical systems using neural
networks. IEEE Transactions of Neural Networks,
1(1):4–27.
Pandit, A. and Boer, R. J. D. (2019). Stochastic inheritance
of division and death times determines the size and
phenotype of CD8+ t cell families. Frontiers in Im-
munology, 10.
Pinto, C. M. and Carvalho, A. R. (2015a). Effect of drug-
resistance in a fractional complex-order model for
HIV infection. IFAC-PapersOnLine, 48(1):188–189.
Pinto, C. M. and Carvalho, A. R. (2015b). Stochastic model
for HIV dynamics in HIV specific helper cells. IFAC-
PapersOnLine, 48(1):184–185.
Roy, S. M. and Wodarz, D. (2012). Infection of HIV-
specific CD4 t helper cells and the clonal composi-
tion of the response. Journal of Theoretical Biology,
304:143–151.
Shi, P. and Dong, L. (2019). Dynamical behaviors of
stochastic virus dynamic models with saturation re-
sponses. Mathematical Biosciences, 308:20–26.
Wang, G., Wei, W., Jiang, J., Ning, C., Chen, H., Huang, J.,
Liang, B., Zang, N., Liao, Y., Chen, R., Lai, J., Zhou,
O., Han, J., Liang, H., and Ye, L. (2019). Application
of a long short-term memory neural network: a bur-
geoning method of deep learning in forecasting HIV
incidence in guangxi, china. Epidemiology and Infec-
tion, 147.
Wang, Y., Brauer, F., Wu, J., and Heffernan, J. M. (2014).
A delay-dependent model with HIV drug resistance
during therapy. Journal of Mathematical Analysis and
Applications, 414(2):514–531.
Wilson, D. P. (2012). HIV treatment as prevention:
Natural experiments highlight limits of antiretrovi-
ral treatment as HIV prevention. PLoS Medicine,
9(7):e1001231.
Wodarz, D. and Hamer, D. H. (2007). Infection dynamics in
HIV-specific CD4 tcells: Does a CD4 t cell boost ben-
efit the host or the virus? Mathematical Biosciences,
209(1):14–29.
Xiang, Y., Fujimoto, K., Schneider, J., Jia, Y., Zhi, D.,
and Tao, C. (2019). Network context matters: graph
convolutional network model over social networks
improves the detection of unknown HIV infections
among young men who have sex with men. Jour-
nal of the American Medical Informatics Association,
26(11):1263–1271.
Zhang, Z., Zhang, J., Cheng, F., and Liu, F. (2019). A novel
stability criteria of a class nonlinear fractional-order
HIV-1 system with multiple delay. International Jour-
nal of Control, Automation and Systems, 17(9):2274–
2283.
NARMA-L2-based Antiviral Therapy for Infected CD4+ T Cells in a Nonlinear Model for HIV Dynamics: Protease Inhibitors-based
Approach
683