Simultaneous Camera Calibration and Temporal Alignment of 2D and 3D Trajectories

Joni Herttuainen, Tuomas Eerola, Lasse Lensu, Heikki Kälviäinen

Abstract

In this paper, we present an automatic method that given the 2D and 3D motion trajectories recorded with a camera and 3D sensor, automatically calibrates the camera with respect to the 3D sensor coordinates and aligns the trajectories with respect to time. The method utilizes a modified Random Sample Consensus (RANSAC) procedure that iteratively selects two points from both trajectories, uses them to calculate the scale and translation parameters for the temporal alignment, computes point correspondences, and estimates the camera matrix. We demonstrate the approach with a setup consisting of a standard web camera and Leap Motion sensor. We further propose necessary object tracking and trajectory filtering procedures to produce proper trajectories with the setup. The result showed that the proposed method achieves over 96% success rate with a test set of complex trajectories.

References

  1. Abdel-Aziz, Y. (1971). Direct linear transformation from comparator coordinates in close-range photogrammetry. In ASP Symposium on Close-Range Photogrammetry.
  2. Caspi, Y. and Irani, M. (2000). A step towards sequenceto-sequence alignment. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, volume 2, pages 682-689.
  3. Cleveland, W. S. (1979). Robust locally weighted regression and smoothing scatterplots. Journal of the American Statistical Association, 74(368):829-836.
  4. El Akkad, N., Merras, M., Saaidi, A., and Satori, K. (2014). Camera self-calibration with varying intrinsic parameters by an unknown three-dimensional scene. The Visual Computer, 30(5):519-530.
  5. Fischler, M. A. and Bolles, R. C. (1981). Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395.
  6. Heikkila, J. and Silvén, O. (1997). A four-step camera calibration procedure with implicit image correction. In IEEE Conference on Computer Vision and Pattern Recognition, pages 1106-1112.
  7. Henriques, J. F., Caseiro, R., Martins, P., and Batista, J. (2015). High-speed tracking with kernelized correlation filters. IEEE Transactions on Pattern Analysis and Machine Intelligence, 37(3):583-596.
  8. Hiltunen, V., Eerola, T., Lensu, L., and Kalviainen, H. (2014). Comparison of general object trackers for hand tracking in high-speed videos. In 2014 22nd International Conference on Pattern Recognition (ICPR), pages 2215-2220. IEEE.
  9. Knoop, S., Vacek, S., and Dillmann, R. (2009). Fusion of 2d and 3d sensor data for articulated body tracking. Robotics and Autonomous Systems, 57(3):321-329.
  10. Kuronen, T., Eerola, T., Lensu, L., Takatalo, J., Häkkinen, J., and Kälviäinen, H. (2015). High-speed hand tracking for studying human-computer interaction. In Scandinavian Conference on Image Analysis, pages 130-141.
  11. Liu, P., Shi, J., Zhou, J., and Jiang, L. (2003). Camera self-calibration using the geometric structure in real scenes. In Computer Graphics International, 2003. Proceedings, pages 262-265.
  12. Noguchi, M. and Kato, T. (2007). Geometric and timing calibration for unsynchronized cameras using trajectories of a moving marker. In IEEE Workshop on Applications of Computer Vision (WACV), pages 20-20. IEEE.
  13. Rahimi, A., Dunagan, B., and Darrell, T. (2004). Simultaneous calibration and tracking with a network of nonoverlapping sensors. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR), volume 1, pages I-187.
  14. Rangarajan, K., Allen, W., and Shah, M. (1993). Matching motion trajectories using scale-space. Pattern recognition, 26(4):595-610.
  15. Sturm, P. (2000). A case against kruppa's equations for camera self-calibration. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 22(10):1199- 1204.
  16. Triggs, B. (1998). Autocalibration from planar scenes. In European Conference on Computer Vision (ECCV), pages 89-105.
  17. Wu, Q., Shao, T.-C., and Chen, T. (2007). Robust selfcalibration from single image using ransac. In Third International Symposium on Visual Computing, pages 230-237.
  18. Zhao, Y. and Lv, X. (2012). An approach for camera selfcalibration using vanishing-line. Information Technology Journal, 11(2):276-282.
Download


Paper Citation


in Harvard Style

Herttuainen J., Eerola T., Lensu L. and Kälviäinen H. (2017). Simultaneous Camera Calibration and Temporal Alignment of 2D and 3D Trajectories . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 445-450. DOI: 10.5220/0006126304450450


in Bibtex Style

@conference{visapp17,
author={Joni Herttuainen and Tuomas Eerola and Lasse Lensu and Heikki Kälviäinen},
title={Simultaneous Camera Calibration and Temporal Alignment of 2D and 3D Trajectories},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={445-450},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006126304450450},
isbn={978-989-758-227-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - Simultaneous Camera Calibration and Temporal Alignment of 2D and 3D Trajectories
SN - 978-989-758-227-1
AU - Herttuainen J.
AU - Eerola T.
AU - Lensu L.
AU - Kälviäinen H.
PY - 2017
SP - 445
EP - 450
DO - 10.5220/0006126304450450