A Convex Approach for Non-rigid Structure from Motion Via Sparse Representation

Junjie Hu, Terumasa Aoki

Abstract

This paper presents a convex solution for simultaneously recovering 3D non-rigid structures and camera motions from 2D image sequences based on sparse representation. Most existing methods rely on low rank assumption. However, it will lead to poor reconstruction for objects with strong local deformation. Also, when camera motion is unknown, there is no convex solution for non-rigid structure from motion (NRSfM). In order to solve this problem, we estimate non-rigid structures by sparse representation. In this paper, we estimate camera motions through a sparse spectral-norm minimization approach, and then a fast l1-norm minimization algorithm is introduced to reconstruct 3D structures. Both of them are convex, therefore, our method gives a global optimum. Our method can handle objects with strong local deformation and also doesn’t need low rank prior. Experimental results show that our method achieves state-of-the-art reconstruction performance on CMU benchmark dataset.

References

  1. Agudo, A., Agapito, L., Calvo, B., and Montiel, J. (2014). Good vibrations: A modal analysis approach for sequential non-rigid structure from motion. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1558-1565.
  2. Agudo, A. and Moreno-Noguer, F. (2015). Simultaneous pose and non-rigid shape with particle dynamics. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 2179-2187.
  3. Akhter, I., Sheikh, Y., Khan, S., and Kanade, T. (2008). Nonrigid structure from motion in trajectory space. In Advances in neural information processing systems, pages 41-48.
  4. Akhter, I., Sheikh, Y., Khan, S., and Kanade, T. (2011). Trajectory space: A dual representation for nonrigid structure from motion. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 33(7):1442- 1456.
  5. Bregler, C., Hertzmann, A., and Biermann, H. (2000). Recovering non-rigid 3d shape from image streams. In Computer Vision and Pattern Recognition, 2000. Proceedings. IEEE Conference on, volume 2, pages 690- 696. IEEE.
  6. Carlo, T. and Kanade, T. (1992). Shape and motion from image streams under orthography: a factorization method. International Journal of Computer Vision, 9(2):137-154.
  7. Dai, Y., Li, H., and He, M. (2014). A simple priorfree method for non-rigid structure-from-motion factorization. International Journal of Computer Vision, 107(2):101-122.
  8. Del Bue, A., Xavier, J., Agapito, L., and Paladini, M. (2012). Bilinear modeling via augmented lagrange multipliers (balm). Pattern Analysis and Machine Intelligence, IEEE Transactions on, 34(8):1496-1508.
  9. Fragkiadaki, K., Salas, M., Arbelaez, P., and Malik, J. (2014). Grouping-based low-rank trajectory completion and 3d reconstruction. In Advances in Neural Information Processing Systems, pages 55-63.
  10. Garg, R., Roussos, A., and Agapito, L. (2013). Dense variational reconstruction of non-rigid surfaces from monocular video. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1272-1279.
  11. Gotardo, P. F. and Martinez, A. M. (2011). Non-rigid structure from motion with complementary rank-3 spaces. In Computer Vision and Pattern Recognition (CVPR), 2011 IEEE Conference on, pages 3065-3072.
  12. Lee, H., Battle, A., Raina, R., and Ng, A. Y. (2006). Efficient sparse coding algorithms. In Advances in neural information processing systems, pages 801-808.
  13. Paladini, M., Bartoli, A., and Agapito, L. (2010). Sequential non-rigid structure-from-motion with the 3d-implicit low-rank shape model. In Computer Vision-ECCV 2010, pages 15-28. Springer.
  14. Park, H. S., Shiratori, T., Matthews, I., and Sheikh, Y. (2010). 3d reconstruction of a moving point from a series of 2d projections. In ECCV, pages 158-171.
  15. Tao, L. and Matuszewski, B. (2013). Non-rigid structure from motion with diffusion maps prior. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 1530-1537.
  16. Torresani, L., Hertzmann, A., and Bregler, C. (2008). Nonrigid structure-from-motion: Estimating shape and motion with hierarchical priors. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 30(5):878-892.
  17. Xiao, J., Chai, J., and Kanade, T. (2004). A closed-form solution to non-rigid shape and motion recovery. In ECCV, pages 573-587.
  18. Yezzi, A. J. and Soatto, S. (2003). Deformotion: Deforming motion, shape average and the joint registration and approximation of structures in images. International Journal of Computer Vision, 53(2):153-167.
  19. Zhang, P. B. and Hung, Y. S. (2015). Non-rigid structure from motion through estimation of blend shapes. In Digital Image Computing: Techniques and Applications (DICTA), 2015 International Conference on, pages 1-7. IEEE.
  20. Zhou, X., Leonardos, S., Hu, X., and Daniilidis, K. (2015). 3d shape estimation from 2d landmarks: A convex relaxation approach. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pages 4447-4455.
  21. Zhu, Y. and Lucey, S. (2015). Convolutional sparse coding for trajectory reconstruction. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 37(3):529-540.
Download


Paper Citation


in Harvard Style

Hu J. and Aoki T. (2017). A Convex Approach for Non-rigid Structure from Motion Via Sparse Representation . In Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017) ISBN 978-989-758-227-1, pages 333-339. DOI: 10.5220/0006078603330339


in Bibtex Style

@conference{visapp17,
author={Junjie Hu and Terumasa Aoki},
title={A Convex Approach for Non-rigid Structure from Motion Via Sparse Representation},
booktitle={Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)},
year={2017},
pages={333-339},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006078603330339},
isbn={978-989-758-227-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 12th International Joint Conference on Computer Vision, Imaging and Computer Graphics Theory and Applications - Volume 6: VISAPP, (VISIGRAPP 2017)
TI - A Convex Approach for Non-rigid Structure from Motion Via Sparse Representation
SN - 978-989-758-227-1
AU - Hu J.
AU - Aoki T.
PY - 2017
SP - 333
EP - 339
DO - 10.5220/0006078603330339