Author:
Enso Ikonen
Affiliation:
University of Oulu, Finland
Keyword(s):
Markov decision process, generalized cell-to-cell mapping, qualitative modelling.
Related
Ontology
Subjects/Areas/Topics:
Adaptive Signal Processing and Control
;
Discrete Event Systems
;
Hybrid Dynamical Systems
;
Informatics in Control, Automation and Robotics
;
Intelligent Control Systems and Optimization
;
Machine Learning in Control Applications
;
Nonlinear Signals and Systems
;
Signal Processing, Sensors, Systems Modeling and Control
Abstract:
Predictive and optimal process control using finite Markov chains is considered. A basic procedure is outlined, consisting of discretization of plant input and state spaces; conversion of a (a priori) plant model into a set of finite state probability transition maps; specification of immediate costs for state-action pairs; computation of an optimal or a predictive control policy; and, analysis of the closed-loop system behavior. An application, using a MATLAB toolbox developed for MDP-based process control design, illustrates the approach in the control of a multivariable plant with both discrete and continuous action variables. For problems of size of practical significance (thousands of states), computations can be performed on a standard office PC. The aim of the work is to provide a basic framework for examination of nonlinear control, emphasizing in on-line learning from uncertain data.