A Summarized Semantic Structure to Represent Manipulation Actions

Tobias Strübing, Fatemeh Ziaeetabar, Florentin Wörgötter

Abstract

To represent human manipulation actions in a simple and understandable way, we had proposed a framework called enriched semantic event chains (eSEC) which creates a temporal sequence of static and dynamic spatial relations between objects in a manipulation. The eSEC framework has so far only been used in manipulation actions consisting of one hand. As the eSECs descriptors are in the form of huge matrices, we need to have a concise version of them. Here, we want to extend this framework to interactions which involve more hands. Therefore, we applied statistical and semantic analyses to summarize the current eSEC while preserving its important features and introducing an enhanced eSEC (e2SEC). This summarization is done by reducing the number of rows in an eSEC matrix and merging semantic spatial relations between manipulated objects. Eventually, we presented the new e2SEC framework which has 20% fewer rows, 16.7% less static spatial and 11.1% less dynamic spatial relations while still maintaining the eSEC efficiency in recognition and differentiation of manipulation actions. This simplification paves the way for a simpler recognition and predicting complex actions and interactions in a shorter time and is beneficial in real time applications such as human-robot interactions.

Download


Paper Citation


in Harvard Style

Strübing T., Ziaeetabar F. and Wörgötter F. (2021). A Summarized Semantic Structure to Represent Manipulation Actions.In Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP, ISBN 978-989-758-488-6, pages 370-379. DOI: 10.5220/0010258803700379


in Bibtex Style

@conference{visapp21,
author={Tobias Strübing and Fatemeh Ziaeetabar and Florentin Wörgötter},
title={A Summarized Semantic Structure to Represent Manipulation Actions},
booktitle={Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,},
year={2021},
pages={370-379},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010258803700379},
isbn={978-989-758-488-6},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 16th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 4: VISAPP,
TI - A Summarized Semantic Structure to Represent Manipulation Actions
SN - 978-989-758-488-6
AU - Strübing T.
AU - Ziaeetabar F.
AU - Wörgötter F.
PY - 2021
SP - 370
EP - 379
DO - 10.5220/0010258803700379