Christoph Walter, Tino Krueger and Norbert Elkmann
Fraunhofer Institute for Factory Operation and Automation
Sandtorstrasse 22, 39106 Magdeburg, Germany
Keywords: Distributed control system, fault tolerance, self-tuning controller.
Abstract: Here we present the position control system of a swimming inspection robot for large under-ground con-
crete pipes that are partially filled with wastewater. The system consists of a laser-based measurement sub-
system for position determination and a mechanical rudder to move the robot laterally within the pipe. The
required software components are implemented as services following a CORBA-based architecture. To
automatically adapt the position control system to different environment conditions, a self-tuning controller
is used. The controller has hybrid requirements regarding latency and interarrival times of computed posi-
tion values. In this contribution, we describe the architectural support for this application as well as how the
system deals with excessive latencies due to transient overload.
The Emschergenossenschaft based in Germany is
currently planning the Emscher sewer system, ar-
guably the largest residential water management
project in Europe in years to come. The new em-
scher sewer will show a total length of 51 km and
diameters between 1.4 and 2.8 m in depths up to 40
m under surface. The Emschergenossenschaft en-
gaged the Fraunhofer Institute for Factory Operation
and Automation (IFF) in Magdeburg, Germany, as
the general contractor to develop automatic inspec-
tion and cleaning systems to meet the requirements
imposed by legal guidelines.
Large under-ground concrete pipes that are par-
tially filled with wastewater are a hazardous envi-
ronment for man. Nevertheless, inspection of such
pipes must be performed on a regular basis. Many of
today’s remote controlled inspection systems for
underground sewer pipes consist of a single TV-
camera and are designed for pipe diameters below
one meter. The recent development of automatic
inspection systems (Elkmann et al., 2005) equipped
with advanced sensors makes it possible for an op-
erator to perform this task from an outside position
even for pipes with a diameter between 1.4 to 2.8
meters. The cable-guided damage surveying system
(SEK) (see Fig. 1) was developed to be a versatile
and easy-to-use tool to detect various kinds of dam-
ages above and underneath the water-line with high
The inspection process is largely automated
(Elkmann et al., 2006) and supervised by an operator
from within a service-vehicle outside of the sewer.
To ensure an optimal inspection result the system
must acquire data from a centered position.
Figure 1: The cable-guided damage surveying system
within a sewer pipe.
For this reason, it is our goal to keep the robot near
the center of the pipe during the data acquisition
phase. We must also deal with different environment
conditions. These include water level, flow velocity
and pipe diameter. A traditional controller with fixed
parameter set was unable to maintain a stable posi-
Walter C., Krueger T. and Elkmann N. (2008).
In Proceedings of the Fifth International Conference on Informatics in Control, Automation and Robotics - ICSO, pages 28-34
DOI: 10.5220/0001489100280034
tion depending on the current conditions. To tackle
this problem the position control system of the robot
was fitted with a self-tuning controller that is able to
identify model based control parameters prior to data
The main part of the controller was implemented
using a software programmable logic controller
(PLC). The input to the controller is the current lat-
eral position relative to the pipe axis. An iterative
algorithm using data from various sensors, mainly
three banks of laser distance sensors, determines the
position. The PC-based implementation of this algo-
rithm has an unknown worst-case execution time
(WCET). The actual execution time is depending on
the internal state of the software as well as on the
current input data. Further delay stems from the
transmission of data over the Ethernet network be-
tween PLC and PC as well as from the scheduling
scheme of the calculation task. While the actual con-
troller is robust against certain latency and jitter of
the arrival of position data, we must consider cases
when position data does not arrive in time. This may
happen due to unusual long execution time of the
position determination algorithm or due to transient
overload of the communication channel.
Another challenge is the second part of the self-
tuning controller, which performs the identification
of the system’s behavior and which derives an ade-
quate control parameter set. The algorithm used here
is very sensitive to jitter of the interarrival times of
position measurements and to missing samples.
An architectural approach for solving both prob-
lems is described in section 3. In section 4, the ac-
tual controller is presented in more detail. Section 5
briefly discusses the dynamic quality of service ap-
proach used to avoid failure in case of transient
Realizing time critical applications using complex
distributed systems is a challenging task. On one
hand, considerable work has been done to introduce
time constraints or timely predictable behavior. On
the other hand, a number of approaches have been
published to relax time constraints of certain appli-
cations by introducing fault tolerance mechanisms.
In the field of communication middleware, the
Object Management Group (OMG) has released
extensions to the known CORBA-specification to
extend it with real-time features (Wolfe et al., 1997).
The primary focus here is on the introduction of
priorities when executing method calls, which re-
quires appropriate system software support. Soft-
ware libraries implementing this standard were also
made available (Schmidt et al., 1998).
To relax time constraints, approaches that try to
balance quality of service and required computation
time were considered. This includes the classical
“anytime” (Dean, Boddy, 1988) or “flexible”
(Hendler, 1989) algorithms, where quality of service
can be dynamically traded against shorter computa-
tion times. This scheme can be used to ensure the
timeliness of computations. The authors of (Nett et
al., 1998) have also dealt with meeting task dead-
lines. They use a dynamic task scheduling scheme
with detection and handling of timing faults. Faults
are handled via a task-pair strategy with invocation
of an alternative exception task in case of a timing
3.1 Modeling Data Flow using Services
We developed a service oriented CORBA-based
software architecture focusing on data flow and dis-
tributed processing. In our system, an application is
constructed by combining software modules that
provide functionality to other modules via a generic
interface. The modules may be arbitrarily distributed
over a number of computers in a network. The inter-
face of the services is data centric. It allows services
to offer data via named buffers to other services or
clients. Each buffer represents a time series of spe-
cific data objects (see Fig. 2). A service can repre-
sent a data source (e.g. a sensor), a processing mod-
ule or a data consumer (e.g. an actuator, a database,
or a GUI). Services that represent sensors or sensor
systems may provide data that represents a single
measurement, a time series of measurements, a vec-
tor, an image, or any other types of data that are
generated by the sensor system. Other services may
not provide sensor data but processing results.
By connecting the services via the pub-
lish/subscribe paradigm the data flow of the applica-
tion is modeled. In this communication model, the
services play different roles. The module providing
data is called the service provider. The data con-
sumer is called the service client. Please note that in
our system a client does not subscribe to specific
data elements but to notifications on when such ele-
ments become available.
Figure 2: A service provider offers data buffers to clients
via a generic interface.
The data objects that are being exchanged between
services satisfy a given scheme. They consist of bi-
nary data and metadata in XML syntax. The most
important element of the metadata is the timestamp,
which indicates the recording time of the original
data. The timestamp acts as a key for the data object
within the given data buffer. The binary data can
contain information in any format. A client can re-
quest individual data elements or arbitrary blocks of
data elements from a buffer. This is achieved by
sending structured queries with a simple syntax to
the accordant service. The client can then process
these data items and/or eventually offer the results to
further clients. The foundation of the used time-
stamps is the Newtonian time model. Timestamps
are generated based on synchronized local clocks. A
simple clock synchronization is carried out via the
network by a dedicated service. The accuracy of
synchronization between time-server and any client
is better than 100 µs. This is well below the shortest
interarrival times of data elements in our application.
It was achieved using common PC-hardware, Giga-
bit-Ethernet, and a Windows NT-based operating
system. Accuracy may be improved by using more
sophisticated clock synchronization algorithms.
3.2 Implementation of Position
Determining the position of the robot is of great im-
portance. Knowledge of the position is not only re-
quired for position control, but also to process and
interpret other sensors’ data. This includes the regis-
tration of images from a high-resolution photo-
graphic camera system, the analysis of data from an
ultrasonic underwater scanner, the determination of
the magnitude of corrosion and incrustations, or the
precise measurement of possible deviation of the
position of individual pipe segments. These tasks
cannot be performed without knowledge of the pose
of each sensor during data acquisition.
The heart of the position determination subsys-
tem is an algorithm that was specifically developed
for this purpose. It is implemented as a service (the
position determination service; PDS). It is based on
the measurements of a number of laser ranging sen-
sors plus an inclination sensor. A discussion of the
exact procedure can be found in (Elkmann et al.,
Measurements from the laser sensors are ac-
quired by the onboard PLC. The PLC is imple-
mented in software on an embedded PC and tightly
coupled with an interface service (IS). The IS has
access to the PLC’s memory and is configured to
insert time-stamped measurements into a buffer
queue at a rate of 100 Hz. This rate is consistent
with sensor capabilities and sufficient for our appli-
The PDS is notified of the arrival of new data
elements in the IS’s buffer. Its goal is to process all
data elements that are produced by the IS. To do this
the PDS uses the following strategy of data access:
Upon notification of a new data element, it asks the
IS for all elements that succeed the last element the
PDS has already processed. This is done via a single
“get”-operation. By using this technique, it is possi-
ble for the PDS to catch up in case of a previous
transient overload. For each received data element,
the system’s position is calculated and eventually
made available for further processing by other cli-
4.1 Controller Design
One challenge in the context of stable data acquisi-
tion between real time and non-real time systems
(hybrid systems) is the implementation of non-linear
controllers. This chapter describes the motivation of
using a non-linear controller in this area. In addition,
it gives a short introduction of calculating them.
When travelling through a partially filled sewer,
the position of the SEK is controlled by a rudder arm
mounted on top of the robot. The rudder can be
<< service >>
<< service client >>
buffer A
time data
buffer B
time data
… …
ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics
turned relative to the longitudinal axis of the pipe. In
turn, buoyancy forces the SEK to change its posi-
tion. Due to disturbances, a controller has to be inte-
grated for closed-loop navigation. Self-tuning con-
trollers belong to the denomination of adaptive con-
trollers. They are used to fit the plant on a variable
environment. In fact, the flow velocity of the waste-
water, the water level, and the pipe diameter may
change depending on the actual sewer and on current
conditions like the time of day or the weather situa-
tion. This motivates to use adaptive controllers for
the robot. The benefit of adaptive controllers com-
pared to fixed structure designs is the broad field of
applications. Here we will focus on the self-tuning
controller as a substitute of indirect adaptive control-
Intermediate instances are required for the calcu-
lation of the controller parameters (see Fig. 3).
Figure 3: Schematic diagram of a self-tuning controller.
The calculation of a self-tuning controller contains
the following steps:
Recursive estimation of process parameters
(system identification)
Suitable determination of a control law with ad-
justable parameters
Calculation of controller parameters based on
the desired closed-loop system
Update the parameters using the IS
The advantage of this method comprises the possi-
bility to secure each step of calculation before the
controller parameters are used. In addition, the
closed loop can be pushed to a desired behavior
while the parameters of the plant are changing.
The plant is depicted by a discrete autoregressive
moving average with an extra input model
(ARMAX-Model) of fourth-order. The advantage of
ARMAX-Models compared to simpler models is the
adequate freedom in describing the properties of the
disturbance term.
That gives the model:
where y(t) is the plant output, u(t) is the control in-
put and e(t) the prediction error. In this case, the
least–squares criterion:
is being used to identify the transmission behavior of
the process. The parameters are time varying in a
small range during operation. Hence, it is desirable
to base the identification on the most recent data
rather than on the older data. This can be achieved
by exponential discounting (λ) of old data. Then, it
can be shown (Lennard Ljung, 1999) that the ex-
tended recursive least square scheme (ERLS) be-
(t))1)-(t)P(t (
])1( (t) - L(t)[y(t) 1)-(t )(t
Using ERLS it is advisable to consider the persistent
excitation in view of convergence of the parameter
estimation. In this step, the indirect method permits
to prove the estimated process parameters on sepa-
rate recorded test series. If the verification is suc-
cessful, the calculation of the controller parameter is
Fig. 4 depicts exemplary the change of positions
predicted by a calculated model towards measured
data. The rudder angle as input to the ERLS was
oscillating between minus five and five degrees. The
calculated model is afterwards the starting point for
the controller design.
The idea regarding the controller design is to
compensate the plant with the inverse system and a
desired transmission behavior.
Figure 4: Measured and predicted positions of the SEK.
Assuming that the desired close loop behavior as
shown in equation 5 where z-1 is backward shift
The actual closed loop is:
with the controller:
and the plant behavior:
The suitable controller law is then given by equation
9 without solving a Diophantine equation.
)(zB)A(z )D(z
))(zB-)(zA)(B(z )C(z
In terms of the causality of the controller law, we get
the following conditions for the implementation:
The transmission behavior of the plant has to be
stable, respect to the inverse implementation
in the controller law.
The polynomial of the decoupled model
should only have zeros within
the unit circle.
The estimated plant model should not include a
minimum phase system or must be corrected
because of the stability of the open loop.
The controller law has to be causal.
The calculated controller is therefore less sensitive
to noise compared to other designs like the pole
placement variant, which is described by (Aström,
Wittenmark, 1992).
4.2 Implementation
The self-tuning controller was implemented in the
form of two main modules. The first module is the
actual controller. It runs as a real-time task within
the onboard PLC. Position-data is fed into it by the
IS as described in section 3.2. The IS acts also as a
client similar to the PDS but with a different strategy
of data access: The controller doesn’t want to proc-
ess all data elements that are produced by the PDS,
but needs the most recent one. For this reason the IS
upon notification of new data requests the most re-
cent data element from the PDS and feeds it into the
Figure 5: Structure and data flow of controller implemen-
The second module is a service, which computes the
control parameter set (system identification service;
SIS). It needs two kinds of data elements as input:
(1) all sampled rudder arm angles during the system
identification process and (2) the corresponding sys-
tem position. The arrival of these data is not time
critical but it is necessary that all samples are being
processed. It therefore uses a similar data access
strategy as the PDS to gather all data elements that
describe the rudder movement from the IS. Further-
more, it requests for each such element the two clos-
est system positions from the PDS and interpolates
between them to get the corresponding system posi-
tion. Requesting closest elements given a certain
timestamp is another feature of the data-access
mechanism of our architecture. After the SIS has
collected enough data, it can compute the control
parameters and send them to the main control mod-
ule via the IS. The described data flow is shown in
Fig. 5.
ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics
Even though we have successfully implemented the
system identification part of the controller as a non-
real-time service while guarantying the required data
quality, the actual controller may still suffer from
excessive latencies in the arrival of position data
from the PDS. This is because of (1) the iterative
algorithm used in the PDS and (2) because of large
latencies in the communication between the distrib-
uted services due to transient overload of the com-
munication channel.
To avoid system failure in such cases we inte-
grated a dynamic quality of service approach that
uses the timestamps of the position data available to
the controller implemented in the PLC to determine
the age of that data. If a certain age is exceeded, the
controller uses an alternative position that it com-
putes by a simple prediction mechanism based on
the last known positions. This fall back mechanism
is triggered if the position data available to the PLC
becomes older then 90 ms. While this is a crude
method and could easily be improved by using a
more accurate model based approach, it is sufficient
to deal with short disruptions of communication. If
however the position data becomes older then 900
ms, the controller is put into a safe state. In this
state, the rudder is moved into centered position,
which is the state that most unlikely causes a colli-
sion with the wall. The controller resumes as soon as
position data not older then 90 ms becomes available
We performed experiments with a prototypic im-
plementation of the system under operating condi-
tions. To determine the actual interarrival times of
data produced by the PDS, we used a test-client that
was consuming position data in the same way as the
IS. In Fig. 6, the interarrival times recorded by the
test-service are shown. Well under 5 percent of the
position values arrived slightly later then usual. Less
then 1 percent of that were delayed more than 90 ms.
When that happened, approximately 50 percent were
delayed more than 900 ms.
Figure 6: Interarrival times of position data at Test-
When observing the behavior of the robot during the
experiment, no significant deviation from the ideal
position could be noticed. The system maintained
position near the center of the pipe when missing the
90 ms deadline. This was also the case when briefly
missing the 900 ms deadline. This can be attributed
to the fact that we only had small turbulences that
required only slight corrections by the position con-
trol system. However, we expect that missing the
900 ms deadline may have a significant effect in
case of more severe turbulences.
Here we presented an application of a distributed
control system for position control of a boat-like
inspection robot for partially filled sewer pipes. We
discussed the hybrid requirements of an advanced
self-tuning controller regarding timeliness and inter-
arrival times of its input data. We presented a
scheme for detecting and handling excessive time
delays to prevent system failure by using dynami-
cally reduced quality of the position controller. It
was also discussed, how the controller was inte-
grated into a non-real-time service oriented software
system. We conclude that in case of our application
it is feasible to use soft- or non-real-time compo-
nents in the control loop. Prerequisites are (1) a de-
terministic data acquisition with time stamping and
(2) a hard-real-time controller core with fault toler-
ance mechanisms.
Elkmann N., Reimann B., Schulenburg E., Althoff H.,
2005. Automated inspection system for large under-
ground concrete pipes under operating conditions.
In Proc. International Conference on Field and Ser-
vice Robotics.
Elkmann N., Kutzner S., Saenz J., Reimann B., Schultke
F., Althoff H., 2006. Fully automatic inspection sys-
tems for large underground concrete pipes partially
filled with wastewater. In Proc. of International Con-
ference Intelligent Robots and Systems (IROS06).
Wolfe V. F., DiPippo L. C., Cooper G., Johnston R.,
Kortmann P., Thuraisingham B. M., 1997. Real-Time
CORBA. In Proc. of the Third IEEE Real-Time Tech-
nology and Applications Symposium.
Schmidt D. C., Levine D. L., Mungee S., 1998. The De-
sign of the TAO Real-Time Object Request Broker. In
Computer Communications 21(4), p. 294-324
Dean T. L., Boddy M., 1988. An analysis of time-
dependent planning. In Proceedings of the Seventh Na-
tional Conference on Artificial Intelligence, Minnea-
polis, Minnesota, p. 49-54.
Hendler J. A., 1989. Real-time planning. In Working Notes
of the AAAI Spring Symposium on Planning and
Search, Stanford.
Nett E., Gergeleit M., Mock M., 1998. An adaptive ap-
proach to object-oriented real-time computing. In Pro-
ceedings of ISORC'98, p. 342-349.
Ljung L., 1999. System Identification, PTR Prentice Hall
Astrom K., Wittenmark B., 1992. Adaptive Control, Pear-
son Education
ICINCO 2008 - International Conference on Informatics in Control, Automation and Robotics