Authors:
G. J. Burghouts
;
L. de Penning
;
M. Kruithof
;
P. Hanckmann
;
J-M Ten Hove
;
S. Landsmeer
;
S. P. van den Broek
;
R. den Hollander
;
C. van Leeuwen
;
S. Korzec
;
H. Bouma
and
K. Schutte
Affiliation:
TNO, Netherlands
Keyword(s):
Human Behavior Understanding, Search Engine, Video Retrieval, Action Recognition, Textual Description, Meta Data, Indexing, 48 Human Actions.
Related
Ontology
Subjects/Areas/Topics:
Applications
;
Computer Vision, Visualization and Computer Graphics
;
Image and Video Analysis
;
Image Understanding
;
Learning of Action Patterns
;
Pattern Recognition
;
Software Engineering
;
Video Analysis
Abstract:
The contribution of this paper is a search engine that recognizes and describes 48 human actions in realistic videos. The core algorithms have been published recently, from the early visual processing (Bouma, 2012), discriminative recognition (Burghouts, 2012) and textual description (Hanckmann, 2012) of 48 human actions. We summarize the key algorithms and specify their performance. The novelty of this paper is that we integrate these algorithms into a search engine. In this paper, we add an algorithm that finds the relevant spatio-temporal regions in the video, which is the input for the early visual processing. As a result, meta-data is produced by the recognition and description algorithms. The meta-data is filtered by a novel algorithm that selects only the most informative parts of the video. We demonstrate the power of our search engine by retrieving relevant parts of the video based on three different queries. The search results indicate where specific events occurred, and wh
ich actors and objects were involved. We show that events can be successfully retrieved and inspected by usage of the proposed search engine.
(More)