Nonlocal Regularizing Constraints in Variational Optical Flow

Joan Duran, Antoni Buades

Abstract

Optical flow methods try to estimate a dense correspondence field describing the motion of the objects in an image sequence. We introduce novel nonlocal regularizing constraints for variational optical flow computation. While the use of similarity weights has been restricted to the regularization term so far, the proposed data terms permit to implicitly use the image geometry in order to regularize the flow and better locate motion discontinuities. The experimental results illustrate the superiority of the new constraints with respect to the classical brightness constancy assumption as well as to nonlocal regularization strategies.

References

  1. Alvarez, L., Deriche, R., Papadopoulo, T., and Sánchez, J. (2002). Symmetrical dense optical flow estimation with occlusions detection. In Proc. European Conf. Computer Vision (ECCV), volume 2350 of Lecture Notes in Comp. Sci., pages 721-735, Copenhagen, Denmark.
  2. Alvarez, L., Weickert, J., and Sánchez, J. (2000). Reliable estimation of dense optical flow fields with large displacements. Int. J. Comput. Vis., 39(1):41-56.
  3. Barker, S., Scharstein, D., Lewis, J., Roth, S., Black, M., and Szeliski, R. (2011). A database and evaluaton methodology for optical flow. Int. J. Comput. Vis., 92(1):1-31.
  4. Black, M. and Anandan, P. (1996). The robust estimation of multiple motions: Parametric and piecewise smooth flow fields. Comput. Vis. Image Underst., 63(1):75- 104.
  5. Brox, T., Bruhn, A., Papenberg, N., and Weickert, J. (2004). High accuracy optical flow estimation based on a theory for warping. In Proc. European Conf. Computer Vision (ECCV), volume 3024 of Lecture Notes in Comp. Sci., pages 25-36, Prague, Czech Republic.
  6. Brox, T. and Malik, J. (2011). Large displacement optical flow: Descriptor matching in variational motion estimation. IEEE Trans. Pattern Anal. Mach. Intell., 33(3):500-513.
  7. Bruhn, A., Weickert, J., and Schnörr, C. (2005). Lucas/Kanade meets Horn/Schunck: Combining local and global optic flow methods. Int. J. Comput. Vis., 61(3):211-231.
  8. Buades, A., Coll, B., and Morel, J.-M. (2005). A review of image denoising algorithms, with a new one. SIAM Multiscale Model. Simul., 4(2):490-530.
  9. Chambolle, A. (2004). An algorithm for total variation minimization and applications. J Math. Imaging Vis., 20(1-2):89-97.
  10. Horn, B. and Schunck, B. (1981). Determining optical flow. In Technical Symposium East, pages 319-331. International Society for Optics and Photonics.
  11. Kennedy, R. and Taylor, C. (2015). Optical flow with geometric occlusion estimation and fusion of multiple frames. In Proc. Int. Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), volume 8932 of Lecture Notes in Comp. Sci., pages 364-377, Hong Kong.
  12. Lowe, D. (2004). Distinctive image features from scaleinvariant keypoints. Int. J. Comput. Vis., 60(2):91- 110.
  13. Lucas, B. and Kanade, T. (1981). An iterative image registration technique with an application to stereo vision. In Proc. Int. Joint Conf. Artificial Intelligence , pages 674-679, Vancouver, BC, Canada.
  14. Mémin, E. and Pérez, P. (1998). Dense estimation and object-based segmentation of the optical flow with robust techniques. IEEE Trans. Image Process., 7(5):703-719.
  15. Mileva, Y., Bruhn, A., and Weickert, J. (2007). Illumination-robust variational optical flow with photometric invariants. In Proc. DAGM Symposium on Pattern Recognition, volume 4713 of Lecture Notes in Comp. Sci., pages 152-162, Heidelberg, Germany.
  16. Nagel, H.-H. and Enkelmann, W. (1986). An investigation of smoothness constraints for the estimation of displacement vector fields from image sequences. IEEE Trans. Pattern Anal. Mach. Intell., 8(5):565-593.
  17. Papenberg, N., Bruhn, A., Brox, T., Didas, S., and Weickert, J. (2006). Highly accurate optic flow computation with theoretically justified warping. Int. J. Comput. Vis., 67(2):141-158.
  18. Ranftl, R., Bredies, K., and Pock, T. (2014). Non-local total generalized variation for optical flow estimation. In Proc. European Conf. Computer Vision (ECCV), volume 8698 of Lecture Notes in Comp. Sci., pages 439- 454, Zurich, Switzerland. Springer Berlin Heidelberg.
  19. Rudin, L., Osher, S., and Fatemi, E. (1992). Nonlinear total variation based noise removal algorithms. Physica D, 60:259-268.
  20. Steinbrücker, F., Pock, T., and Cremers, D. (2009). Large displacement optical flow computation without warping. In Proc. IEEE Int. Conf. Computer Vision (ICCV), pages 1609-1614, Kyoto, Japan.
  21. Trobin, W., Pock, T., Cremers, D., and Bischof, H. (2008). An unbiased second-order prior for high-accuracy motion estimation. In Proc. DAGM Symposium on Pattern Recognition, volume 5096 of Lecture Notes in Comp. Sci., pages 396-405, Munich, Germany.
  22. Vogel, C., Roth, S., and Schindler, K. (2013). An evaluation of data costs for optical flow. In Proc. German Conference Pattern Recognition (GCPR), volume 8142 of Lecture Notes in Comp. Sci., pages 343- 353, Saarbrücken, Germany.
  23. Wedel, A., Pock, T., Zach, C., Cremers, D., and Bischof, H. (2009). An improved algorithm for TV-L1 optical flow. In Proc. Statistical and Geometrical Approached to Visual Motion Analysis, volume 5604 of Lecture Notes in Comp. Sci., pages 23-45, Dagstuhl Castle, Germany.
  24. Werlberger, M., Pock, T., and Bischof, H. (2010). Motion estimation with non-local total variation regularization. In Proc. IEEE Conf. Computer Vision and Pattern Recognition (CVPR), pages 2464-2471, San Francisco, CA, USA.
  25. Werlberger, M., Trobin, W., Pock, T., Bischof, H., Wedel, A., and Cremers, D. (2009). Anisotropic Huber-L1 optical flow. In Proc. British Machine Vision Conference (BMVC), pages 108.1-108.11, London, UK.
  26. Xiao, J., Cheng, H., Sawhney, H., Rao, C., and Isnardi, M. (2006). Bilateral filtering-based optical flow estimation with occlusion detection. In Proc. European Conf. Computer Vision (ECCV), volume 3951 of Lecture Notes in Comp. Sci., pages 211-224, Graz, Austria.
  27. Yoon, K. and Kweon, I. (2006). Adaptive support-weight approach for correspondece search. IEEE Trans. Pattern Anal. Mach. Intell., 28(4):650-656.
  28. Zach, C., Pock, T., and Bischof, H. (2007). A duality based approach for realtime TV-L1 optical flow. In Proc. DAGM Symposium on Pattern Recognition, volume 4713 of Lecture Notes in Comp. Sci., pages 214-223, Heidelberg, Germany.
  29. Zimmer, H., Bruhn, A., and Weickert, J. (2011). Optic flow in harmony. Int. J. Comput. Vis., 93(3):368-388.
  30. Zimmer, H., Bruhn, A., Weickert, J., Valgaerts, L., Salgado, A., Rosenhahn, B., and Seidel, H.-P. (2009). Complementary optic flow. In Proc. Int. Conf. Energy Minimization Methods in Computer Vision and Pattern Recognition (EMMCVPR), volume 5681 of Lecture Notes in Comp. Sci., pages 207-220, Bonn, Germany.
Download


Paper Citation


in Harvard Style

Duran J. and Buades A. (2017). Nonlocal Regularizing Constraints in Variational Optical Flow . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 151-161. DOI: 10.5220/0006098501510161


in Bibtex Style

@conference{visapp17,
author={Joan Duran and Antoni Buades},
title={Nonlocal Regularizing Constraints in Variational Optical Flow},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={151-161},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006098501510161},
isbn={978-989-758-227-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - Nonlocal Regularizing Constraints in Variational Optical Flow
SN - 978-989-758-227-1
AU - Duran J.
AU - Buades A.
PY - 2017
SP - 151
EP - 161
DO - 10.5220/0006098501510161