Real-Time Hand Gesture Control of a Robotic Arm with Programmable Motion Memory

Daniel Giraldi Michels, Davi Giraldi Michels, Lucas Alexandre Zick, Dieisson Martinelli, André Schneider de Oliveira, Vivian Kalempa

2025

Abstract

The programming of industrial robots is traditionally a complex task that requires specialized knowledge, limiting the flexibility and adoption of automation in various sectors. This paper presents the development and validation of a programming by demonstration system to simplify this process, allowing an operator to intuitively teach a task to a robotic arm. The methodology employs the MediaPipe library for real-time hand gesture tracking, using a conventional camera to translate human movements into a robot-executable trajectory. The system is designed to learn a manipulation task, such as ’pick and place’, and store it for autonomous reproduction. The experimental validation, conducted through 50 consecutive cycles of the task, demonstrated the high robustness and effectiveness of the approach, achieving a 98% success rate. Additionally, the results confirmed the excellent precision and repeatability of the method, evidenced by a standard deviation of only 0.0126 seconds in the cycle time. A video demonstrating the system’s functionality is available for illustrative purposes, separate from the quantitative validation data. It is concluded that the proposed approach is a viable and effective solution for bridging the gap between human intention and robotic execution, contributing to the democratization of automation by offering a more intuitive, accessible, and flexible programming method.

Paper Citation

in Harvard Style

Michels D., Michels D., Zick L., Martinelli D., Schneider de Oliveira A. and Kalempa V. (2025). Real-Time Hand Gesture Control of a Robotic Arm with Programmable Motion Memory. In Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO; ISBN 978-989-758-770-2, SciTePress, pages 292-300. DOI: 10.5220/0013717800003982

in Bibtex Style

@conference{icinco25,
author={Daniel Michels and Davi Michels and Lucas Zick and Dieisson Martinelli and André Schneider de Oliveira and Vivian Kalempa},
title={Real-Time Hand Gesture Control of a Robotic Arm with Programmable Motion Memory},
booktitle={Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO},
year={2025},
pages={292-300},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013717800003982},
isbn={978-989-758-770-2},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 22nd International Conference on Informatics in Control, Automation and Robotics - Volume 2: ICINCO
TI - Real-Time Hand Gesture Control of a Robotic Arm with Programmable Motion Memory
SN - 978-989-758-770-2
AU - Michels D.
AU - Michels D.
AU - Zick L.
AU - Martinelli D.
AU - Schneider de Oliveira A.
AU - Kalempa V.
PY - 2025
SP - 292
EP - 300
DO - 10.5220/0013717800003982
PB - SciTePress