Semantic State Estimation in Robot Cloth Manipulations Using Domain Adaptation from Human Demonstrations

Georgies Tzelepis, Eren Aksoy, Júlia Borràs, Guillem Alenyà

2024

Abstract

Deformable object manipulations, such as those involving textiles, present a significant challenge due to their high dimensionality and complexity. In this paper, we propose a solution for estimating semantic states in cloth manipulation tasks. To this end, we introduce a new, large-scale, fully-annotated RGB image dataset of semantic states featuring a diverse range of human demonstrations of various complex cloth manipulations. This effectively transforms the problem of action recognition into a classification task. We then evaluate the generalizability of our approach by employing domain adaptation techniques to transfer knowledge from human demonstrations to two distinct robotic platforms: Kinova and UR robots. Additionally, we further improve performance by utilizing a semantic state graph learned from human manipulation data.

Paper Citation

in Harvard Style

Tzelepis G., Aksoy E., Borràs J. and Alenyà G. (2024). Semantic State Estimation in Robot Cloth Manipulations Using Domain Adaptation from Human Demonstrations. In Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP; ISBN 978-989-758-679-8, SciTePress, pages 172-182. DOI: 10.5220/0012368200003660

in Bibtex Style

@conference{visapp24,
author={Georgies Tzelepis and Eren Aksoy and Júlia Borràs and Guillem Alenyà},
title={Semantic State Estimation in Robot Cloth Manipulations Using Domain Adaptation from Human Demonstrations},
booktitle={Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP},
year={2024},
pages={172-182},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0012368200003660},
isbn={978-989-758-679-8},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 19th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP
TI - Semantic State Estimation in Robot Cloth Manipulations Using Domain Adaptation from Human Demonstrations
SN - 978-989-758-679-8
AU - Tzelepis G.
AU - Aksoy E.
AU - Borràs J.
AU - Alenyà G.
PY - 2024
SP - 172
EP - 182
DO - 10.5220/0012368200003660
PB - SciTePress