USING CO-EXISTING ATTRACTORS OF A SENSORIMOTOR LOOP
FOR THE MOTION CONTROL OF A HUMANOID ROBOT
Matthias Kubisch, Benjamin Werner and Manfred Hild
Neurorobotics Research Laboratory, Department of Computer Science, Humboldt-Universität zu Berlin
Berlin, Germany
Keywords:
Humanoid robots, Limit cycle walking, Co-existing attractors, Neural controller, Sensorimotor loop.
Abstract:
The implementation of a biped robot gait is a challenging task within the field of mobile robotics. Particularly,
when the robot is subject to unknown disturbance in constantly changing terrain, a stable and robust gait is
crucial. Regarding the machine together with the controller as an integrated system, the Dynamical Systems
Approach yields a new perspective on legged robots. So called Limit Cycle Walkers have shown their inherent
stability against moderate disturbances of different kinds because gaits can be constructed as attractors of the
dynamical system. Here, we will show how co-existing attractors in neural sensorimotor loops can be used for
the construction of robot gaits and for easy switching among behaviours. The results are demonstrated using
a humanoid robot with neural control and it is shown that walking and standing upright can be implemented
as co-existing attractors of the same pure sensorimotor loop.
1 INTRODUCTION
Legged robots should be able to walk around robustly
in different types of terrain. Also, it is desirable that
switching between various behaviours, like standing
upright, walking forward or walking in different di-
rections, takes place smoothly and reliably. Oscil-
lations have been discovered in countless biological
systems and almost all gaits comprise repetitive pat-
terns.
As (Hein, 2007; Hein et al., 2007) have shown,
different gaits for bipedal robots can be found by
artificial evolution using a central neural oscillator.
The robot’s actuators were driven using weighted and
phase-shifted variants of a single fundamental sine
wave but the resulting gaits were not inherently sta-
ble because they lacked sensory feedback.
Limit cycle walkers (Solomon et al., 2010;
Collins, 2005) have shown their benefits in terms
of gait stability and simplifying the control strategy
when used in a tightly closed sensorimotor loop. So,
a gait has been successfully expressed as an attractor
of the overall system which underlines the potential
of the dynamical systems approach.
In the paper at hand we describe a sensorimotor
loop as the basis for humanoid robot walking and
show how to use the inherent properties of its attrac-
tors to switch the robot’s behaviours from standing to
walking and vice versa. The rest of the paper is or-
ganised as follows: In the next section, we briefly re-
cap the terminology of discrete dynamical systems,
especially discrete time recurrent neural networks and
discuss switching between co-existing attractors. Sec-
tion 3 describes the implementation of a neural con-
troller for humanoid robot walking with the use of
the proposed co-existing attractors of the sensorimo-
tor loop. We demonstrate that behaviours like stand-
ing upright and walking can co-exist within the same
pure sensorimotor loop and that it is easy to switch be-
tween them. Finally, we give an outlook on our future
research.
2 NEURAL DYNAMICS
Dynamical systems theory (Guckenheimer and
Holmes, 1983; Thompson and Stewart, 1986) has be-
come an important tool for roboticists and turned out
to be beneficial for the construction and understand-
ing of recurrent neural networks (Hild et al., 2007),
sensorimotor control loops (Martius et al., 2008) and
the physical system—the robot itself. To briefly recap
the terminology, consider the neural network with the
update rule
x(t + 1) = tanh (Wx(t) + b + u(t)), (1)
385
Kubisch M., Werner B. and Hild M..
USING CO-EXISTING ATTRACTORS OF A SENSORIMOTOR LOOP FOR THE MOTION CONTROL OF A HUMANOID ROBOT.
DOI: 10.5220/0003682203850390
In Proceedings of the International Conference on Neural Computation Theory and Applications (NCTA-2011), pages 385-390
ISBN: 978-989-8425-84-3
Copyright
c
2011 SCITEPRESS (Science and Technology Publications, Lda.)
where x(t) M is the state of the dynamical system
at discrete time t N. The manifold M of all possi-
ble states is called the state space or phase space and
is M = (1, +1)
N
R
N
for N neurons due to the hy-
perbolic tangent transfer function. A trajectory is an
ordered set of successive states, and is called a p-orbit
if it has a definite periodicity p. The weight matrix W
and bias vector b constitute the configuration of the
system. Optionally, a control input u(t) is used to ex-
ternally manipulate the system.
A set towards which the system evolves over time
is called an attractor, which can be a point, curve
or manifold in phase space. Attractors can co-exist
within the same system and, depending on initial con-
ditions, the system evolves in either way. The set of
initial conditions leading to the same attractor is the
basin of that attractor, whereas borders between adja-
cent basins are called separatrices.
2.1 Co-existing Attractors in Discrete
Time Recurrent Neural Networks
Discrete time recurrent neural networks can exhibit a
variety of functionality—even if the number of neu-
rons is small. Consider the network’s configuration
W =
1.282 0.4
0.1 0.865
b =
0.04
0.02
(2)
with only two neurons and, for now, u(t) = 0. The
corresponding phase space is depicted in Figure 1.
This dynamical system has two co-existing attractors,
namely a stable fixed point and a quasiperiodic orbit,
and in between the separatrix. The system’s outputs
are bounded by [1, +1].
Usually we would expect an unstable fixed point
inside the quasiperiodic orbit, a so-called repellor.
But in this configuration we can identify another com-
plete basin inside and therefore we find another, so-
Figure 1: Phase space of the neural network with co-
existing attractors. A basin with a stable fixed point is being
enveloped by another basin with a quasiperiodic orbit.
Figure 2: Switching between co-existing attractors with
short and small control inputs, cf. also Figure 3.
called co-existing attractor. This attractor is a stable
fixed point with the vector field having a curl in that
point. The expected unstable regime has become a
separatrix, here in form of a closed unstable orbit.
Please note, if we would invert the system by in-
terchanging inputs and outputs, the time is flowing
backwards and attractors become unstable. In turn,
formerly unstable areas become attractors. So, the
separatrix becomes a stable orbit and the area out-
side the former quasiperiodic orbit diverges beyond
all bounds. This is due to the poles of the inverse hy-
perbolic tangent.
2.2 Switching Co-existing Attractors
Switching between co-existing attractors can be re-
alised by introducing a controller u(t) = K(x(t))
which usually depends on the current state and pro-
vides the control input u(t). Switching from the stable
fixed point to the quasiperiodic orbit takes place when
the control input is powerful enough to hop over the
separatrix as depicted in Figure 2.
Here, the direction of the control input is of lower
importance since the orbit envelops the fixed point.
Switching to the basin of the fixed point mainly de-
pends on the phase of the oscillation. So the controller
needs an exact timing and pulse shape for a suitable
jump.
Actually, the system together with the controller
form a new dynamical system of higher dimension
than the initial one. But, since we are considering
short and small control inputs, we omit additional di-
mensions for convenience.
Switching between attractors is of major interest
since different behaviours can be expressed as co-
existing attractors (Hild and Kubisch, 2011). In other
NCTA 2011 - International Conference on Neural Computation Theory and Applications
386
Figure 3: Switching between co-existing attractors: output
of x
2
. Small control inputs u(t
1
) = (0, 0.05) and u(t
2
) =
(0, 0.04) are exerted to the system, each with a length of
only one time step.
words, to blend over from one behaviour to another,
we simply have to switch attractors. The next sec-
tion describes how behaviours like walking and stand-
ing upright can be formulated as attractors of the
robot’s sensorimotor system and how switching be-
tween them both can take place.
3 A PURE SENSORIMOTOR GAIT
The type of gait that is to be shaped, highly depends
on the robot’s morphology, i. e. actuators, elasticities,
sensors and so on. For a better understanding of the
complex interplay, one has to bring to mind which
parts are mandatory to shape an attractor. A generic
walking pattern can therefore be separated into parts
resulting in the following sequence:
First, the weight has to be shifted within the
frontal plane to one side using the hip and ankle roll
motors. This lifts the hip and enables sagittal leg mo-
tion. The ankle and hip pitch motors of the stance
leg can now push the body forward so that the swing
leg has enough distance to the ground and can freely
move forward. After the swing leg has touched the
ground steadily, the weight is shifted back to the other
side. The necessary weight shift is achieved by an os-
cillatory lateral movement of the full body. Walking
forward can then be realised by superposing sagittal
leg movements with the same frequency using left and
right, hip and ankle roll motors.
3.1 Structure of the Sensorimotor Loop
An upright standing humanoid robot can approxi-
mately be regarded as an inverted pendulum with a
Figure 4: The A-Series humanoid robot used for walking
experiments. Among others, two acceleration sensors are
mounted on top of the robot’s shoulders.
single contact point to the ground. To stabilise the up-
right posture the controller has to exert enough torque
to swing back the robot in case of disturbance. To
initiate the walking motion, the controller has to shift
the weight smoothly. This is realised by direct sen-
sory feedback forming a closed sensorimotor loop.
The A-Series humanoid robot (Hild, 2007;
Werner, 2008) as depicted in Figure 4 possesses six-
teen 2D acceleration sensors which are distributed all
over the body. Two of them are mounted on top of the
robot’s shoulders and their sensory values are added
together to measure the lateral and sagittal accelera-
tion of the robot’s torso. The lateral sensory informa-
tion is weighted and fed back to the left and right, hip
and ankle roll motors (cf. Figure 5).
Robot and neural control together constitute a dy-
namical system holding co-existing attractors such as
lying on back or belly. Even when the motors are
not actively driven, due to stiction and activated mo-
tor brakes, standing upright is also an attractor, how-
ever with a very small basin. Imprecisely speaking,
this is a blurred fixed point, i. e. a densely packed
Figure 5: First attempt of a sensorimotor loop for robot
walking. Lateral acceleration data are directly fed back to
the actuators.
USING CO-EXISTING ATTRACTORS OF A SENSORIMOTOR LOOP FOR THE MOTION CONTROL OF A
HUMANOID ROBOT
387
Figure 6: Lateral acceleration data of an about 40 second trial. At the beginning, the internal oscillator initiates the lateral
body movement. After approximately 22 seconds the motion is manually stopped and again reinitialised by the experimenter.
Finally, the controller internally disables the sensorimotor loop causing the robot to stop.
set of fixed points. So, the stated postures are all sta-
ble fixed points of the system. When driving a lat-
eral oscillation the dynamical system now possesses
a quasiperiodic orbit or limit cycle. A small impulse
within the frontal plane, e. g. a gentle push, makes the
robot leave the fixed point standing and start swing-
ing. In turn, if the robot is being externally held by
the experimenter, the system falls back into the basin
of the standing posture’s fixed point. Figure 6 depicts
the data from lateral acceleration sensors during a 40
second trial with several starts and stops of the os-
cillation. Figure 9 on the next page shows successive
snapshots of an A-Series humanoid robot, performing
lateral oscillation.
3.2 Widening the Basin of Attraction
For now, the simplicity of our first attempt to build
this sensorimotor loop has a drawback: The outputs
of the acceleration sensors usually contain a variety
of different high frequencies due to non-linearities in
the motion of the physical system and sensor noise.
Most essentially, when the feet are touching or are
lifted from the ground, these events produce percep-
tible impulses yielding salient peaks in the acceler-
ation data. If such high frequency components are
fed back to the motors, this would lead to unintended
power consumption and heat production as well as ad-
ditional non-linear distortion of the system’s lateral
movement. Also, the correct phase is needed for a
quick buildup of the oscillation. Therefore, the loop
is extended by a neural implementation of two first
order recursive filters (cf. Figure 7).
The acceleration signal will be low-pass filtered
and phase shifted by
π
4
per neuron. Approximately,
the filtering neurons’ outputs can be regarded sinu-
soids with almost the same amplitude and fundamen-
tal. Thus, mixing those signals yields another one
Figure 7: Sensorimotor loop with low-pass filtering and
phase correction. Filtering is implemented as neural ana-
logues of recursive filters.
with the same frequency but with different ampli-
tude and phase. The mix is then fed to the motors.
Figure 8 shows the acceleration signal and its low-
pass filtered variants, together with the mixed and
again phase-corrected motor signal. As can be seen,
filtering performed effectively, resulting in a strong
reduction of high frequency components and a cor-
rected phase. Moreover, it turned out that damping
the high frequencies widens the basin of attraction for
the quasiperiodic orbit and therefore increases the ro-
bustness of the robot’s gait.
Figure 8: Acceleration data and its low-pass filtered vari-
ants; mixed and phase-corrected motor signal.
NCTA 2011 - International Conference on Neural Computation Theory and Applications
388
Figure 9: An A-Series humanoid robot performing lateral oscillation.
3.3 Stability of the Walking Pattern
Amplitude and frequency of the gait usually change
on different terrain; especially on soft surfaces, e. g.
on carpet, the amplitude of the oscillation is strongly
decreased. Keeping synaptic weights constant, it can
be mentioned that robots of the same series slightly
differ in walking amplitude and frequency. This is
due to minor variations in construction, material and
battery charge level as well as mechanical wearout of
gears and soles.
Due to its attractor properties, the gait has an in-
herent stability against small disturbances, e. g. mi-
nor variations in surface structure. For larger distur-
bances or major environmental variations like chang-
ing ground slopes or obstacles, further balancing
methods are required.
For now, knee joints have not yet been used, yield-
ing a penguin-like gait. Bending knees, the ampli-
tude of the lateral oscillation can be decreased since
the legs need less distance to the ground. This will
very likely improve the gait stability once again and
reduces the risk to fall over sidewards.
3.4 Starting and Stopping the Motion
At the beginning, the robot is initialised at the stable
fixed point, i. e. the upright posture. So far, the robot
needs an external impulse, e. g. a gentle push from
the experimenter, to get out of the basin of the fixed
point, into the basin of the quasiperiodic attractor, and
hence, starts the lateral oscillation.
When the robot is to act autonomously the neural
controller has to initialise the swinging motion on its
own to leave the stable fixed point on a transition to
the quasiperiodic orbit. For this, an internal oscillator
with almost the same eigenfrequency as the robot’s
sensorimotor oscillation is temporarily connected to
the actuators to initiate the motion smoothly and ro-
bustly.
The oscillator is likewise implemented as a neu-
ral network (Pasemann et al., 2003) and well dosed
through neural gates. When the amplitude has
reached a certain threshold, the oscillator is cut off
from the sensorimotor loop. On the other hand, if the
controller has to stop the motion from inside, the sen-
sory feedback is simply cut off with the use of a neu-
ral gate. After this, the oscillation fades out, cf. once
again Figure 6.
This means that the lateral oscillation and thus the
walking pattern can be started and stopped either from
outside the body by an external disturbance or from
inside by the controller itself. This equivalence may
shed new light on how behaviours of living beings are
triggered and merged.
4 CONCLUSIONS AND
OUTLOOK
Using the dynamical systems approach, we have
shown that behaviours like walking and standing can
co-exist as attractors of the same pure sensorimotor
loop. For this, we used a neural implementation of a
minimalist control loop where acceleration data are
directly fed back to the actuators. We further ex-
tended the controller with a filtering mechanism that
widens the basin of attraction for the walking pattern
and thus enhance the stability of the gait. We state
that attractor-based behaviours are inherently robust
and can easily be switched while producing smooth
transitions among behaviours.
Since no explicit body model is required, the pro-
posed technique is independent of the humanoid robot
platform used for walking experiments. We have suc-
cessfully tested our approach with the humanoid robot
Myon (Hild et al., 2011). Although the Myon robot
significantly differs in size, weight and the type of
actuation, we were able to gain comparable results
which will be reported in an upcoming paper. If accel-
eration sensors are not available, the position sensors
of the hip roll joints can be used alternatively.
The proposed sensorimotor loop can be enhanced
in terms of reactivity and stability by replacing the
aforementioned filtering neurons with a Slow Feature
Analysis (Höfer and Hild, 2010). This also extends
the robot’s capabilities with a fall-over prediction by
detecting slow varying changes in sensory data.
Also the shape and elastic properties of legs and
feet can increase the stability and efficiency of the
gait (Collins, 2005; Schneider, 2006). Arched foot
shapes apparently outperform pure flat and rectan-
USING CO-EXISTING ATTRACTORS OF A SENSORIMOTOR LOOP FOR THE MOTION CONTROL OF A
HUMANOID ROBOT
389
gular shapes in terms of dynamics and controllabil-
ity. Current work incorporates the redesign of a hu-
manoid robot’s foot under these pure mechanical as-
pects (Richter, 2011). Furthermore, we currently en-
hance the sensorimotor gait with recent balance re-
covery techniques using the humanoid robot Myon
(Kubisch et al., 2011).
ACKNOWLEDGEMENTS
The authors would like to thank all the former mem-
bers of the Humanoid Team Humboldt for their great
work to bring the ve A-Series robots to life. This
work has been partially supported by the European
research project ALEAR (FP7, ICT-214856).
REFERENCES
Collins, S. H. (2005). A Bipedal Walking Robot with Effi-
cient and Human-Like Gait. In Proceedings of IEEE
International Conference on Robotics and Automa-
tion, pages 1995–2000.
Guckenheimer, J. and Holmes, P. (1983). Nonlinear Oscil-
lations, Dynamical Systems and Bifurcations of Vector
Fields. Springer-Verlag, New York.
Hein, D. (2007). Simloid: Evolution of Biped Walking
Using Physical Simulation. Master’s thesis, Depart-
ment of Computer Science, Humboldt-Universität zu
Berlin.
Hein, D., Hild, M., and Berger, R. (2007). Evolution of
biped walking using neural oscillators and physical
simulation. In RoboCup 2007: Proceedings of the In-
ternational Symposium LNAI, Springer.
Höfer, S. and Hild, M. (2010). Using Slow Feature Analysis
to Improve the Reactivity of a Humanoid Robot’s Sen-
sorimotor Gait Pattern. In Proceedings of the Interna-
tional Conference on Neural Computation (ICNC).
Hild, M. (2007). Neurodynamische Module zur Bewegungs-
teuerung autonomer mobiler Roboter. PhD thesis, De-
partment of Computer Science, Humboldt-Universität
zu Berlin.
Hild, M. and Kubisch, M. (2011). Self-Exploration of
Autonomous Robots Using Attractor-Based Behav-
ior Control and ABC-Learning. In Proceedings of
the 11th Scandinavian Conference on Artificial Intel-
ligence, Trondheim, Norway.
Hild, M., Kubisch, M., and Göhring, D. (2007). How to
Get from Interpolated Keyframes to Neural Attractor
Landscapes – and Why. In Proceedings of the 3rd Eu-
ropean Conference on Mobile Robots, Freiburg, Ger-
many.
Hild, M., Siedel, T., Benckendorff, C., Kubisch, M., and
Thiele, C. (2011). Myon: Concepts and Design of a
Modular Humanoid Robot Which Can Be Reassem-
bled During Runtime. In Proceedings of the 14th
International Conference on Climbing and Walking
Robots, Paris, France.
Kubisch, M., Benckendorff, C., and Hild, M. (2011). Bal-
ance recovery of a humanoid robot using cognitive
sensorimotor loops (CSLs). In Proceedings of the
14th International Conference on Climbing and Walk-
ing Robots and the Support Technologies for Mobile
Machines.
Martius, G., Fiedler, K., and Herrmann, J. M. (2008). Struc-
ture from Behavior in Autonomous Agents. In Pro-
ceedings of the IEEE/RSJ International Conference on
Intelligent Robots and Systems, pages 858–862.
Pasemann, F., Hild, M., and Zahedi, K. (2003). SO(2)-
Networks as Neural Oscillators. In Proceedings of the
International Work Conference on Artificial and Nat-
ural Neural Networks (IWANN), pages 144–151.
Richter, E. (2011). Redesign und Fertigung eines Fußes
für humanoide Roboter. Master’s thesis, Leibniz In-
stitute of Polymer Research, Technische-Universität
Dresden.
Schneider, A. (2006). Local Positive Velocity Feedback
for the Movement Control of Elastic Joints in Closed
Kinematic Chains: A Modelling and Simulation Study
of a 2DoF Arm and a 3DoF Insect Leg. PhD thesis,
Department of Biology, Universität Bielefeld.
Solomon, J. H., Wisse, M., and Hartmann, M. J. (2010).
Fully Interconnected, Linear Control for Limit Cycle
Walking. Adaptive Behavior, 18.
Thompson, J. M. T. and Stewart, H. B. (1986). Nonlinear
Dynamics and Chaos. Wiley, Chichester U.K.
Werner, B. (2008). Sensomotorische Erzeugung eines
Gangmusters für humanoide Roboter. Studienarbeit
am Lehrstuhl für Künstliche Intelligenz, Humboldt-
Universität zu Berlin.
NCTA 2011 - International Conference on Neural Computation Theory and Applications
390