Action Tube Generation by Person Query Matching for Spatio-Temporal Action Detection

Kazuki Omi, Jion Oshima, Toru Tamaki

2025

Abstract

This paper proposes a method for spatio-temporal action detection (STAD) that directly generates action tubes from the original video without relying on post-processing steps such as IoU-based linking and clip splitting. Our approach applies query-based detection (DETR) to each frame and matches DETR queries to link the same person across frames. We introduce the Query Matching Module (QMM), which uses metric learning to bring queries for the same person closer together across frames compared to queries for different people. Action classes are predicted using the sequence of queries obtained from QMM matching, allowing for variable-length inputs from videos longer than a single clip. Experimental results on JHMDB, UCF101-24 and AVA datasets demonstrate that our method performs well for large position changes of people while offering superior computational efficiency and lower resource requirements.

Paper Citation

in Harvard Style

Omi K., Oshima J. and Tamaki T. (2025). Action Tube Generation by Person Query Matching for Spatio-Temporal Action Detection. In Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP; ISBN 978-989-758-728-3, SciTePress, pages 261-268. DOI: 10.5220/0013089500003912

in Bibtex Style

@conference{visapp25,
author={Kazuki Omi and Jion Oshima and Toru Tamaki},
title={Action Tube Generation by Person Query Matching for Spatio-Temporal Action Detection},
booktitle={Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP},
year={2025},
pages={261-268},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0013089500003912},
isbn={978-989-758-728-3},
}

in EndNote Style

TY - CONF

JO - Proceedings of the 20th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 2: VISAPP
TI - Action Tube Generation by Person Query Matching for Spatio-Temporal Action Detection
SN - 978-989-758-728-3
AU - Omi K.
AU - Oshima J.
AU - Tamaki T.
PY - 2025
SP - 261
EP - 268
DO - 10.5220/0013089500003912
PB - SciTePress