LQG CONTROL UNDER AMPLITUDE

AND VARIANCE CONSTRAINTS

A. Kr

´

olikowski†, D. Horla, T. Kubiak

Pozna

´

n University of Technology,

Institute of Control and Information Engineering,

ul. Piotrowo 3A, 60-965 Pozna

´

n, POLAND.

Keywords:

LQG, Amplitude constraint, Variance constraint.

Abstract:

In this paper, the amplitude and variance-constrained LQG control is considered for a plant given by discrete-

time ARMAX model. The minimization of constrained quadratic cost is approached by Kalman ﬁlter, ap-

proximation of the probability density function (pdf) of the state by the Gaussian one and by by tuning of the

Lagrange multiplier. The obtained optimization algorithm is simulated for second-order stable plant model

and different constraints.

1 INTRODUCTION

Control input constraints are ubiquitous in many con-

trol applications, therefore including them in a con-

trol system design is of practical importance. Hard-

limit input constraint and variance or mean-square in-

put constraint are of the most frequent occurrence in

industrial control processes. Neglecting these con-

straints in the controller design may lead to perfor-

mance deterioration or even instability of the control

system. Speciﬁcally, the unstable open-loop systems

in the presence of constrained control signal can not

be globally stabilizable.

The problem addressed in this paper is the LQG con-

trol of ARMAX plant in the presence of simultane-

ous amplitude and variance constrained input. The

constrained control problem is approached using the

Kalman ﬁlter and approximation of the pdf of the

Kalman ﬁlter output by the Gaussian pdf. Analy-

sis and computer simulations of second-order systems

are given.

It should be noted that in the literature the considered

LQG control problem is treated mostly for separate

control constraints, see for example in (Kr

´

olikowski,

1997, M

¨

akil

¨

a, 1982, M

¨

akil

¨

a et at, 1984, Toivonen,

1983).

2 PROBLEM FORMULATION

The plant is given by a discrete-time ARMAX model

A(q

−1

)y

t

= B(q

−1

)u

t

+ C(q

−1

)e

t

, (1)

where A, B, C are polynomials in the backward shift

operator q

−1

, i.e.,A = 1 + a

1

q

−1

+ · · · + a

na

q

−na

,

B = b

1

q

−1

+ · · · + b

nb

q

−nb

, C = 1 + c

1

q

−1

+

· · · +c

nc

q

−nc

, y

t

is the output, u

t

is the control input,

and {e

t

} is assumed to be a sequence of independent

random variables with zero mean and variance σ

2

e

.

Consider the stationary cost function

J

1

= E[y

2

t

+ q

u

u

2

t

] = σ

2

y

+ q

u

σ

2

u

, (2)

where the output and input variances E[y

2

t

], E[u

2

t

] are

denoted as σ

2

y

and σ

2

u

, respectively, and q

u

≥ 0.

The amplitude and variance constraints imposed on

the control input are given as follows

|u

t

| ≤ α, (3)

σ

2

u

≤ c

2

. (4)

It is known that ARMAX model (1) has an equivalent

innovation state space representation

x

t+1

= F x

t

+ gu

t

+ k

e

e

t

, (5)

y

t

= h

T

x

t

+ e

t

, (6)

for na = nb = nc = n, where the corresponding vec-

tors are g

= (b

1

, . . . , b

n

)

T

, k

e

= (c

1

− a

1

, . . . , c

n

−

292

Królikowski A., Horla D. and Kubiak T. (2005).

LQG CONTROL UNDER AMPLITUDE AND VARIANCE CONSTRAINTS.

In Proceedings of the Second International Conference on Informatics in Control, Automation and Robotics, pages 292-296

Copyright

c

SciTePress

a

n

)

T

, h = (1, 0, . . . , 0)

T

, and

F =

−a

1

1 . . . 0

. . . . . 0

−a

n−1

. . . . 1

−a

n

. . . . 0

.

The associated Kalman ﬁlter is

ˆx

t+1

= F ˆx

t

+ g

u(t) + k˜y

t

, (7)

where k

is the stationary gain vector, and ˜y

t

= y

t

−

h

T

ˆx

t

with variance σ

2

˜y

= h

T

P

k

h

+ σ

2

e

. The matrix

P

k

is the solution to the following Riccati equation

P

k

= F P

k

F

T

−

−(F P

k

h

+ σ

2

ξ

k

ξ

)(F P

k

h

+ σ

2

e

k

e

)

T

× (8)

×(h

T

P

k

h

+ σ

2

e

)

−1

+ k

e

k

T

e

σ

2

e

.

The goal of the control is to minimize the loss func-

tion J

1

under the given structure of the controller

speciﬁed by the feedback gain vector f

in the case of

the Kalman ﬁlter-based controller subject to the am-

plitude and variance constrints (3), (4). Thus, the con-

strained control law has a form

u

t

= sat(f

T

ˆx

t

; α), (9)

where sat denotes a saturation function and ˆx

t

is the

output of the Kalman ﬁlter (7).

3 CONTROL UNDER

AMPLITUDE CONSTRAINT

Consider now the cost function

J = E[x

T

t

Q

x

x

t

+ q

u

u

2

t

] = trQ

x

R

x

+ q

u

σ

2

u

, (10)

where R

x

= Ex

t

x

T

t

, R

x

= R

ˆx

+ P

k

and R

ˆx

=

E ˆx

t

ˆx

T

t

. If the weight matrix Q

x

is such that Q

x

=

h

h

T

then it is easy to see that the cost function J (10)

can be considered as an alternative formulation for J

1

(2) w.r.t. minimization.

Using any stabilizing feedback control law, the fol-

lowing stationary equation for R

ˆx

resulting from (7)

can be derived

R

ˆx

= F R

ˆx

F

T

+ F R

ˆxu

g

T

+ g

R

T

ˆxu

F

T

+

+σ

2

u

g

g

T

+ σ

2

˜y

k

k

T

, (11)

where R

ˆxu

= E ˆx

t

u

t

. The approximate expressions

for σ

2

u

and R

ˆxu

under the constrained control law (9)

are (Toivonen, 1983):

σ

2

u

= σ

2

g

1

(σ), R

ˆxu

= R

ˆx

f

g

2

(σ), (12)

where

σ

2

= f

T

R

ˆx

f

(13)

and g

1

(σ) = erf(ασ

−1

2

−

1

2

) − ασ

−1

2

1

2

ierfc(α×

×σ

−1

2

−

1

2

), g

2

(σ) = erf (ασ

−1

2

−

1

2

). Introducing

(12), (13) into (11) one obtains an equation that en-

ables iterative calculation of R

ˆx

. The corresponding

cost function (10) takes then the form

J(f

) = tr(Q

x

+ q

u

g

1

(σ)ff

T

)R

ˆx

+

+trQ

x

P

k

= (14)

= J

f

(f

) + trQ

x

P

k

.

Using the gradient of J

f

(f) the following iterative

algorithm for calculating the feedback gain f

in the

control law (9) can be proposed (Tovoinen, 1983)

f

(k+1)

= f

(k)

+ α

k

s

(k)

, (15)

where α

k

is the step length and

s

(k)T

= d

(k)

∂J

f

∂f

(k)T

R

(k)

ˆx

−1

(16)

for the gradient given as

∂J

f

∂f

(k)T

= e

(k)T

R

(k)

ˆx

. (17)

Calculations for k-th iteration are performed for

f

(k)

. Expressions for d

(k)

, e

(k)

are given as follows

(Tovoinen, 1983):

d

(k)

= −

1

2

h

g

1

(σ

(k)

) + h

1

(σ

(k)

)σ

2(k)

×

×

g

T

S

(k)

g

+ q

u

i

−1

,

e

(k)T

= 2

h

g

1

(σ

(k)

) + h

1

(σ

(k)

)σ

2(k)

×

×

g

T

S

(k)

g

+ q

u

f

(k)T

+

+ g

2

(σ

(k)

)g

T

S

(k)

F +

+ 2h

2

(σ

(k)

)g

T

S

(k)

F R

(k)

ˆx

f

(k)

f

(k)T

i

,

where S

(k)

is a positive deﬁnite solution of the equa-

tion

S

(k)

= F

T

S

(k)

F + Q

x

+

+f

(k)

h

g

1

(σ

(k)

) + h

1

(σ

(k)

)σ

2(k)

×

×

g

T

S

(k)

g

+ q

u

+

+ 2h

2

(σ

(k)

)g

T

S

(k)

F R

(k)

ˆx

f

(k)

i

f

(k)T

+

+g

2

(σ

(k)

)(F

T

S

(k)

g

f

(k)T

+ f

(k)

g

T

S

(k)

F ).

As an initial iteration for calculation of R

(k)

ˆx

one can

take for example R

(0)

ˆx

= Σ

e

= k

k

T

σ

2

e

, and f

(0)

where f

(0)

results from the standard unconstrained

solution of LQG problem. It is convenient to take

the same value of f

(0)

as an initial iteration in (15).

It can be shown (Tovoinen, 1983) that there is a con-

stant a > 0 such that for every α

k

∈ (0, a) it holds

J

f

(f

(k+1)

) < J

f

(f

(k)

),

if (

∂J

f

∂f

)

(k)

) 6= 0. Thus, the proper choice of step α

k

assures the convergence of the algorithm.

4 CONTROL UNDER VARIANCE

CONSTRAINT

In the case of variance constraint given by the inequal-

ity (4) the associated Lagrangian is

L = J + λ(σ

2

u

− c

2

) (18)

or alternatively, the Lagrangian L can be rewritten

L = trQ

x

R

x

+ (q

u

+ λ)σ

2

u

, (19)

where λ ≥ 0 is the Lagrange multiplier. The Kuhn-

Tucker necessary conditions for the constrained min-

imum of L are

∂L

∂λ

≤ 0,

∂L

∂f

= 0. (20)

The optimal variance constrained control strategy can

be computed by solving the conditions (20). In prac-

tice, this is done iteratively, as it will be shown in Sec-

tion 5.

The controller to be designed is of the form

u

t

= f

T

ˆx

t

, (21)

where f

follows from appropriate Riccati equation

and ˆx

t

is the Kalman ﬁlter output. The minimiza-

tion of the Lagrangian (19) w.r.t. all admissible u

t

is

closely related to the minimization of the loss func-

tion J subject to the constraint (4). If u

∗

t

= f

∗T

ˆx

t

minimizes the Lagrangian (19), and the inequality

constraint (4) and complementary condition

λ(σ

2

u

− c

2

) = 0 (22)

are fulﬁlled at u

∗

t

, then u

∗

t

is also an optimal control

signal for variance-constrained control problem.

A major problem is the determination of appropriate

estimates for the Lagrange multiplier λ such that the

conditions (4) and (22) are satisﬁed for u

∗

t

. In prac-

tice this is done iteratively where each iteration step

k consists of solving a standard LQG problem, i.e.

of minimizing the Lagrangian (19) with λ = λ

(k)

and of updating the Lagrange multiplier according to

a suitable algorithm. A realization of this algorithm

needs the appropriate equations for R

ˆx

and σ

2

u

, (see

eqns.(25), (26)).

An iterative algorithm for updating the Lagrange mul-

tiplier λ

(k)

proposed in (M

¨

akil

¨

a, 1982, M

¨

akil

¨

a et at,

1984) can be combined with an algorithm described

in Section 3 to yield the algorithm given below.

5 SIMULTANEOUS AMPLITUDE

AND VARIANCE

CONSTRAINTS

First, it can be observed that the amplitude constraint

α (3) restricts itself the input variance because σ

2

u

≤

α

2

. Taking into account (4) and assuming c

2

= γσ

2

e

one obtains

γ ≤

α

2

σ

2

e

. (23)

This means that if for a given amplitude constraint α,

a given variance constraint has a form γ ≥

α

2

σ

2

e

then it

is automatically fulﬁlled and optimization of the feed-

back gain can only be performed wrt amplitude con-

straint as shown in Section 3. On the other hand, if

for a given α, a given variance constraint is such that

γ <

α

2

σ

2

e

then a problem may have an optimization

sense according to the problem formulated in Section

2. The proposed algorithm consists of the following

steps:

step 1: Take λ

(0)

> 0, h

0

= 1, 0 < α

0

< 1.

step 2: Calculate f

(k)

according to the method given

in Section 3 for

q

(k)

u

= q

u

+ λ

(k)

. (24)

step 3: Calculate R

(k)

ˆx

according to eqn. (11) taking

into account (12), (13), i.e.

R

(k)

ˆx

= F R

(k)

ˆx

F

T

+

+(F R

(k)

ˆx

f

(k)

g

T

+ g

f

T (k)

R

T (k)

ˆx

F

T

) ×

×g

2

(σ

(k)

) + g

g

T

f

T (k)

R

(k)

ˆx

f

(k)

g

1

(σ

(k)

) +

+k

k

T

σ

2

˜y

(25)

and

σ

2(k)

u

= f

T (k)

R

(k)

ˆx

f

(k)

g

1

(σ

(k)

), (26)

σ

2(k)

= f

T (k)

R

(k)

ˆx

f

(k)

. (27)

step 4: Check out the value (22), i.e.

ψ

(k)

= λ

(k)

(σ

2(k)

u

− c

2

). (28)

If ψ

(k)

is sufﬁciently close to zero, according to

some prescribed criterion then STOP, otherwise go

to step 5.

step 5: If k = 0, then go to step 6, otherwise update

h

k

(if positive) according to

h

k

= h

k−1

+

∆λ

(k)

+ h

k−1

∆ψ

(k)

∆ψ

(k)

, (29)

where ∆λ

(k)

= λ

(k)

− λ

(k−1)

, ∆ψ

(k)

= ψ

(k)

−

ψ

(k−1)

and ψ

(k)

is given by (27).

step 6: Update the multiplier λ

(k)

according to

λ

(k+1)

= λ

(k)

+ sat(β

k

h

k

ψ

(k)

; aλ

(k)

), (30)

where 0 < a < 1.

step 7: Calculate β

k+1

according to

β

k+1

= β

k

(γ

0

− β

k

)(γ

0

− 1)

−1

, (31)

where γ

0

> 1. Take k → k + 1 and go to step 2.

It should be noted that in the case of tight constraints

the problem may not have a solution, i.e. the set of

feedback gains for which the cost function has ﬁnite

values can be empty.

6 SIMULATION RESULTS

Consider the ARMAX plant described by the fol-

lowing stable model A = 1 + 1.8q

−1

− 0.9q

−2

,

B(q

−1

) = q

−1

, C(q

−1

) = 1 where the noise vari-

ance is set at σ

2

e

= 1.0.

The performance of the iterative algorithm given in

Section 5 is illustrated in Figs.1,2 for constraints

α = 3.0 and c

2

= 2.0, initial value q

u

= 0.01 and

Q

x

= (1, 0)

T

(1, 0), λ

(0)

= 1.0, α

0

= 0.5, γ

0

= 5.0,

a = 0.06. The corresponding plots for α = 3.0 and

c

2

= 3.0 are shown in Figs.3,4. It can be seen that

the input variances attain their constraint values. It is

worthy to notice that the condition (23) is fulﬁlled for

both values of constraint c

2

. The plots of signals for

α = 3.0 and c

2

= 3.0 are shown in Fig.5, where one

can see that the control signal attains sometimes its

constraint.

7 CONCLUSIONS

The algorithm solving the amplitude and variance-

constrained LQG control problem is given for plant

described by ARMAX model. For unstable open-

loop systems there is a lower bound of variance con-

straint which can be imposed on the control signal to

preserve closed-loop stability, however imposing hard

amplitude constraint is not allowable.

For the self-tuning control implementation the esti-

mates

ˆ

F

t

, ˆg

t

,

ˆ

k

e,t

can be easily obtained from on-

line estimation of the ARMAX model parameters

a

1

, . . . , a

na

, b

1

, . . . , b

nb

, c

1

, . . . , c

nc

.

REFERENCES

Kr

´

olikowski, A. (1997). Self-tuning control under

variance and amplitude constraints, Prepr. 11th

IFAC Symp.on Syst.Ident., SYSID’97, Kitakyushu,

8 - 11 July 1997, Vol.1, pp.345-350.

M

¨

akil

¨

a, P.M. (1982). Constrained liner quadratic

gaussian control for process application,

Academic dissertation, Report 82-6, Process

Control Laboratory, Abo, Finland.

M

¨

akil

¨

a, P.M., T. Westerlund and H.T. Toivonen

(1984). Constrained linear quadratic gaussian con-

trol with process applications. Automatica, 20(1),

pp.15-29.

Toivonen H.T. (1983). Suboptimal control of

discrete stochastic amplitude constrained systems,

Int.J.Control, Vol.37, No.3, pp.493-502.

Toivonen, H.T. (1983). Variance constrained self tun-

ing control, Automatica, 19(4), pp.415-418.

0 50 100 150 200 250 300

−1.4

−1.35

−1.3

−1.25

−1.2

−1.15

f

1

0 50 100 150 200 250 300

−0.9

−0.88

−0.86

−0.84

−0.82

−0.8

−0.78

f

2

Figure 1: Plots of feedback gains f

1

, f

2

; c

2

= 2.0

0 50 100 150 200 250 300

0.2

0.4

0.6

0.8

q

u

0 50 100 150 200 250 300

1.8

2

2.2

2.4

2.6

2.8

σ

2

u

Figure 2: Plots of the weight q

(k)

u

and variance σ

2

u

; c

2

= 2.0

0 50 100 150 200 250 300

−1.8

−1.7

−1.6

−1.5

−1.4

f

1

0 50 100 150 200 250 300

−1

−0.98

−0.96

−0.94

−0.92

f

2

Figure 3: Plots of feedback gains f

1

, f

2

; c

2

= 3.0

0 50 100 150 200 250 300

0

0.05

0.1

0.15

0.2

q

u

0 50 100 150 200 250 300

2.9

3

3.1

3.2

3.3

3.4

σ

2

u

Figure 4: Plots of the weight q

(k)

u

and variance σ

2

u

; c

2

= 3.0

0 20 40 60 80 100 120 140 160 180 200

−4

−2

0

2

4

u

0 20 40 60 80 100 120 140 160 180 200

−10

−5

0

5

x

1

0 20 40 60 80 100 120 140 160 180 200

−5

0

5

10

x

2

Figure 5: Plots of signals for c

2

= 3.0 and α = 3.0