Human Object Interaction Detection Primed with Context

Maya Antoun, Daniel Asmar

2023

Abstract

Recognizing Human-Object Interaction (HOI) in images is a difficult yet fundamental requirement for scene understanding. Despite the significant advances deep learning has achieved so far in this field, the performance of state of the art HOI detection systems is still very low. Contextual information about the scene has shown improvement in the prediction. However, most works that use semantic features rely on general word embedding models to represent the objects or the actions rather than contextual embedding. Motivated by evidence from the field of human psychology, this paper suggests contextualizing actions by pairing their verbs with their relative objects at an early stage. The proposed system consists of two streams: a semantic memory stream on one hand, where verb-object pairs are represented via a graph network by their corresponding feature vector; and an episodic memory stream on the other hand in which human-objects interactions are represented by their corresponding visual features. Experimental results indicate that our proposed model achieves comparable results on the HICO-DET dataset with a pretrained object detector and superior results on HICO-DET with finetuned detector.

Download


Paper Citation


in Harvard Style

Antoun M. and Asmar D. (2023). Human Object Interaction Detection Primed with Context. In Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP; ISBN 978-989-758-634-7, SciTePress, pages 59-68. DOI: 10.5220/0011612200003417


in Bibtex Style

@conference{visapp23,
author={Maya Antoun and Daniel Asmar},
title={Human Object Interaction Detection Primed with Context},
booktitle={Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP},
year={2023},
pages={59-68},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0011612200003417},
isbn={978-989-758-634-7},
}


in EndNote Style

TY - CONF

JO - Proceedings of the 18th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications (VISIGRAPP 2023) - Volume 5: VISAPP
TI - Human Object Interaction Detection Primed with Context
SN - 978-989-758-634-7
AU - Antoun M.
AU - Asmar D.
PY - 2023
SP - 59
EP - 68
DO - 10.5220/0011612200003417
PB - SciTePress