METHODOLOGY FOR LEARNING VISUAL REACTIVE BEHAVIOURS IN ROBOTICS THROUGH REINFORCEMENT AND IMAGE-BASED STATES

Pablo Quintía, José E. Domenech, Cristina Gamallo, Carlos V. Regueiro

2007

Abstract

This article describes the development of a methodology for the learning of visual and reactive behaviours using reinforcement learning. With the use of artificial vision the environment is perceived in 3D, and it is possible to avoid obstacles that are invisible to other sensors that are more common in mobile robotics. Reinforcement learning reduces the need for intervention in behaviour design, and simplifies its adjustment to the environment, the robot and the task. The designed methodology is intended to be general; thus, in order to change the desired behaviour, only the reinforcement and the filtering of the image need to be changed. For the definition of the reinforcement a laser sensor is used, and for the definition of the states a fixed 3x3 grid is used. The behaviours learned were wall following, object following, corridor following and platform following. Results are presented with a Pioneer 2 AT. A Gazebo 3D simulator was used for the Learning and testing phase, and a test of the wall following behaviour was carried out in a real environment.

References

  1. Boada, M., Barber, R., and Salichs, M. (2002). Visual approach skill for a mobile robot using learning and fusion of simple skills. Robotics and Autonomous Systems, 38:157-170.
  2. Gaskett, C., Fletcher, L., and Zelinsky, A. (2000). Reinforcement learning for a vision based mobile robot. In Proc. IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS), volume 1, pages 403-409.
  3. Martínez-Mar ín, T. and Duckett, T. (2005). Fast reinforcement learning for vision-guided mobile robots. In IEEE Int. Conf. on Robotics and Automation (ICRA), pages 4170-4175.
  4. Michels, J., Saxena, A., and Ng, A. (2005). High speed obstacle avoidance using monocular vision and reinforcement learning. In Proc. Int. Conf. on Machine Learning.
  5. Millán, J. R., Posenato, D., and Dedieu, E. (2002). Continuous-action Q-Learning. Machine Learning, 49:247, 265.
  6. Moreno, D., Regueiro, C., Iglesias, R., and Barro, S. (2004). Using prior knowledge to improve reinforcement learning in mobile robotics. In Towards Autonomous Robotic Systems (TAROS).
  7. Nakamura, T. and Asada, M. (1995). Motion sketch: Acquisition of visual motion guided behaviors. In IJCAI, pages 126-132.
  8. Nehmzow, U. (1999). Vision processing for robot learning. Industrial Robot, 26(2):121-130.
  9. Regueiro, C., Domenech, J., Iglesias, R., and Correa, J. (2006). Acquiring contour following behaviour in robotics through q-learning and image-based states. In Proc. XV Int. Conf. on Computer and Information Sciences Engineering (CISE), pages 403-409.
  10. Regueiro, C., Rodríguez, M., Correa, J., Moreno, D., Iglesias, R., and Barro, S. (2002). A control architecture for mobile robotics based on specialists. In Leondes, C., editor, Intelligent Systems: Technology and Applications, volume 6, pages 337-360. CRC Press.
  11. Ruiz, J., Montero, P., Martín, F., and Matellán, V. (2005). Vision based behaviors for a legged robot. In Proc. Workshop en Agentes Físicos (WAF), pages 59-66.
  12. Wyatt, J. (1995). Issues in putting reinforcement learning onto robots. In 10th Biennal Conference of the AISB, Sheffield, UK.
Download


Paper Citation


in Harvard Style

Quintía P., E. Domenech J., Gamallo C. and V. Regueiro C. (2007). METHODOLOGY FOR LEARNING VISUAL REACTIVE BEHAVIOURS IN ROBOTICS THROUGH REINFORCEMENT AND IMAGE-BASED STATES . In Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO, ISBN 978-972-8865-83-2, pages 149-156. DOI: 10.5220/0001628901490156


in Bibtex Style

@conference{icinco07,
author={Pablo Quintía and José E. Domenech and Cristina Gamallo and Carlos V. Regueiro},
title={METHODOLOGY FOR LEARNING VISUAL REACTIVE BEHAVIOURS IN ROBOTICS THROUGH REINFORCEMENT AND IMAGE-BASED STATES},
booktitle={Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,},
year={2007},
pages={149-156},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0001628901490156},
isbn={978-972-8865-83-2},
}


in EndNote Style

TY - CONF
JO - Proceedings of the Fourth International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO,
TI - METHODOLOGY FOR LEARNING VISUAL REACTIVE BEHAVIOURS IN ROBOTICS THROUGH REINFORCEMENT AND IMAGE-BASED STATES
SN - 978-972-8865-83-2
AU - Quintía P.
AU - E. Domenech J.
AU - Gamallo C.
AU - V. Regueiro C.
PY - 2007
SP - 149
EP - 156
DO - 10.5220/0001628901490156