Authors:
Reham Abobeah
1
;
Marwan Torki
2
;
Amin Shoukry
3
and
Jiro Katto
4
Affiliations:
1
CSE Department, Egypt-Japan University of Science and Technology, Alexandria, Egypt, CSE Department, Al-Azhar University, Cairo and Egypt
;
2
CSE Department, Alexandria University, Alexandria and Egypt
;
3
CSE Department, Egypt-Japan University of Science and Technology, Alexandria, Egypt, CSE Department, Alexandria University, Alexandria and Egypt
;
4
Computer Science and Communication Engineering Department, Waseda University, Tokyo 169-8555 and Japan
Keyword(s):
Temporal Alignment, Synchronization, Attention Mechanisms, Bi-directional Attention.
Abstract:
In this paper, a novel technique is introduced to address the video alignment task which is one of the hot topics in computer vision. Specifically, we aim at finding the best possible correspondences between two overlapping videos without the restrictions imposed by previous techniques. The novelty of this work is that the video alignment problem is solved by drawing an analogy between it and the machine comprehension (MC) task in natural language processing (NLP). Simply, MC seeks to give the best answer to a question about a given paragraph. In our work, one of the two videos is considered as a query, while the other as a context. First, a pre-trained CNN is used to obtain high-level features from the frames of both the query and context videos. Then, the bidirectional attention flow mechanism; that has achieved considerable success in MC; is used to compute the query-context interactions in order to find the best mapping between the two input videos. The proposed model has been tr
ained using 10k of collected video pairs from ”YouTube”. The initial experimental results show that it is a promising solution for the video alignment task when compared to the state of the art techniques.
(More)