Segregational Soft Dynamic Time Warping and Its Application to Action Prediction

Victoria Manousaki, Victoria Manousaki, Antonis Argyros, Antonis Argyros

2022

Abstract

Aligning the execution of complete actions captured in segmented videos has been a problem explored by Dynamic Time Warping (DTW) and Soft Dynamic Time Warping (S-DTW) algorithms. The limitation of these algorithms is that they cannot align unsegmented actions, i.e., actions that appear between other actions. This limitation is mitigated by the use of two existing DTW variants, namely the Open-End DTW (OE-DTW) and the Open-Begin-End DTW (OBE-DTW). OE-DTW is designed for aligning actions of known begin point but unknown end point, while OBE-DTW handles continuous, completely unsegmented actions with unknown begin and end points. In this paper, we combine the merits of S-DTW with those of OE-DTW and OBE-DTW. In that direction, we propose two new DTW variants, the Open-End Soft DTW (OE-S-DTW) and the Open-Begin-End Soft DTW (OBE-S-DTW). The superiority of the proposed algorithms lies in the combination of the soft-minimum operator and the relaxation of the boundary constraints of S-DTW, with the segregational capabilities of OE-DTW and OBE-DTW, resulting in better and differentiable action alignment in the case of continuous, unsegmented videos. We evaluate the proposed algorithms on the task of action prediction on standard datasets such as MHAD, MHAD101-v/-s, MSR Daily Activities and CAD-120. Our experimental results show the superiority of the proposed algorithms to existing video alignment methods.

Paper Citation

in Harvard Style

Manousaki V. and Argyros A. (2022). Segregational Soft Dynamic Time Warping and Its Application to Action Prediction. In Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP; ISBN 978-989-758-555-5, SciTePress, pages 226-235. DOI: 10.5220/0010882300003124

in Bibtex Style

@conference{visapp22,
author={Victoria Manousaki and Antonis Argyros},
title={Segregational Soft Dynamic Time Warping and Its Application to Action Prediction},
booktitle={Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP},
year={2022},
pages={226-235},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0010882300003124},
isbn={978-989-758-555-5},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 17th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2022) - Volume 5: VISAPP
TI - Segregational Soft Dynamic Time Warping and Its Application to Action Prediction
SN - 978-989-758-555-5
AU - Manousaki V.
AU - Argyros A.
PY - 2022
SP - 226
EP - 235
DO - 10.5220/0010882300003124
PB - SciTePress