# Discrete-Time Drift Counteracting Stochastic Optimal Control and Intelligent Vehicle Applications

### Ilya Kolmanovsky, John Michelini

#### 2008

#### Abstract

In this paper we present a characterization of a stochastic optimal control in the problem of maximizing expected time to violate constraints for a nonlinear discrete-time system with a measured (but unknown in advance) disturbance modeled by a Markov chain. Such an optimal control may be viewed as providing drift counteraction and is, therefore, referred as the drift counteracting stochastic optimal control. The developments are motivated by an application to an intelligent vehicle which uses an adaptive cruise control to follow a randomly accelerating and decelerating vehicle. In this application, the control objective is to maintain the distance to the lead vehicle within specified limits for as long as possible with only gradual (small) accelerations and decelerations of the follower vehicle so that driver comfort can be increased and fuel economy can be improved.

#### References

- Afanas'ev, V.N., Kolmanovskii, V.B., and Nosov, V.R. (1996). Mathematical Theory of Control Systems Design. Kluwer Academic Publishers.
- Altman, E. (1999). Constrained Markov Decision Processes. Chapman and Hall/CRC.
- Dynkin, E.B., and Yushkevich, A.A. (1967). Markov Processes: Theorems and Problems. Nauka, Moscow, in Russian. English translation published by Plenum, New York, 1969.
- Kolmanovsky, I., Sivergina, I., and Lygoe, B. (2002). Optimization of powertrain operating policies for feasibility assessment and calibration: Stochastic dynamic programming approach. Proceedings of 2002 American Control Conference. Anchorage, AK, pp. pp. 1425- 1430.
- Kolmanovsky, I., and Maizenberg, T.L. (2002). Optimal containment control for a class of stochastic systems perturbed by Poisson and Wiener processes. IEEE Transactions on Automatic Control. Vol. 47, No. 12, pp. 2041-2046.
- Kolmanovsky, I., Lezhnev, L., and Maizenberg, T.L. (2008). Discrete-time drift counteraction stochastic optimal control: Theory and application-motivated examples. Automatica, Vol. 44 , No. 1, pp. 177-184.

#### Paper Citation

#### in Harvard Style

Kolmanovsky I. and Michelini J. (2008). **Discrete-Time Drift Counteracting Stochastic Optimal Control and Intelligent Vehicle Applications** . In *Proceedings of the 2nd International Workshop on Intelligent Vehicle Control Systems - Volume 1: IVCS, (ICINCO 2008)* ISBN 978-989-8111-34-0, pages 7-16. DOI: 10.5220/0001509700070016

#### in Bibtex Style

@conference{ivcs08,

author={Ilya Kolmanovsky and John Michelini},

title={Discrete-Time Drift Counteracting Stochastic Optimal Control and Intelligent Vehicle Applications},

booktitle={Proceedings of the 2nd International Workshop on Intelligent Vehicle Control Systems - Volume 1: IVCS, (ICINCO 2008)},

year={2008},

pages={7-16},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0001509700070016},

isbn={978-989-8111-34-0},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 2nd International Workshop on Intelligent Vehicle Control Systems - Volume 1: IVCS, (ICINCO 2008)

TI - Discrete-Time Drift Counteracting Stochastic Optimal Control and Intelligent Vehicle Applications

SN - 978-989-8111-34-0

AU - Kolmanovsky I.

AU - Michelini J.

PY - 2008

SP - 7

EP - 16

DO - 10.5220/0001509700070016