A Tensor-based Technique for Structure-aware Image Inpainting

Adib Akl, Charles Yaacoub

Abstract

Image inpainting is an active area of study in computer graphics, computer vision and image processing. Different image inpainting algorithms have been recently proposed. Most of them have shown their efficiency with different image types. However, failure cases still exist, especially when dealing with local image variations. This paper presents an image inpainting approach based on structure layer modeling, where this latter is represented by the second-moment matrix, also known as the structure tensor. The structure layer of the image is first inpainted using the non-parametric synthesis algorithm of Wei and Levoy, then the inpainted field of second-moment matrices is used to constrain the inpainting of the image itself. Results show that using the structural information, relevant local patterns can be better inpainted comparing to the standard intensity-based approach.

References

  1. Kwatra, V., Schödl, A., Essa, I., Turk, G., Bobick, A., 2003. "Graphcut Textures: Image and Video Synthesis Using Graph Cuts," Proc. of ACM SIGGRAPH, pp. 277-286.
  2. Bargteil, A. W., Sin, F., Michaels, J. E., Goktekin, T. G., O'Brien, J. F., 2006. "A Texture Synthesis Method for Liquid Animations," Proc. ACM SIGGRAPH/Eurographics Symposium on Computer Animation, Vienna, Austria, September 2-4.
  3. Yamauchi, H., Haber, J., Seidel, H-P., 2003. "Image restoration using multiresolution texture synthesis and image inpainting,” Proc. Int. Conf. Comput. Graph.
  4. Winkenbach, G., Salesin, D. H., 1994. “Computergenerated pen-and-ink illustration,” Proc. of SIGGRAPH 94, pp. 91-100, Orlando, Florida.
  5. Bertalmio, M., Sapiro, G., Caselles, V., Ballester, C., 2000. “Image inpainting,” Proc. of the 27th annual conference on Computer graphics and interactive techniques, pp. 417-424.
  6. Akl, A., Yaacoub, C., Donias, M., Da Costa, J.-P., Germain, C., 2015. Texture Synthesis Using the Structure Tensor. IEEE Transactions on Image Processing, 24 (11), art. no. 7163318, pp. 4082-4095.
  7. Paget, R., Longstaff, I.D., 1998. "Texture synthesis via a non causal nonparametric multiscale markov random field," IEEE Trans. on Image Processing, vol. 7(6), pp. 925-931.
  8. Efros, A., Leung, T., 1999. “Texture synthesis by nonparametric sampling,” International Conference on Computer Vision, vol. 2, pp. 1033-1038.
  9. Chan, T., Shen, J., 2001. " Non-texture inpainting by curvature-driven diffusions," J. Visual Comm. Image Rep.
  10. Wei, L.-Y., Levoy, M., 2000. "Fast texture synthesis using tree-structured vector quantization," Proc. of ACM SIGGRAPH 2000, pp. 479-488.
  11. Portilla, J., Simoncelli, E.P., 2000. "A Parametric Texture Model based on Joint Statistics of Complex Wavelet Coefficients," Int'l Journal of Computer Vision, vol.40(1), pp. 49-71.
  12. Akl, A., Yaacoub, C., Donias, M., Da Costa, J.-P., Germain, C., 2015. Two-stage color texture synthesis using the structure tensor field. GRAPP 2015 - 10th International Conference on Computer Graphics Theory and Applications; VISIGRAPP, Proceedings, pp. 182-188.
  13. Bigun, J., Granlund, G., 1987. “Optimal Orientation Detection of Linear Symmetry,” International Conference on Computer Vision, ICCV, (London). Piscataway: IEEE Computer Society Press, Piscataway. pp. 433-438.
  14. Aujol, J., Ladjal, S., Masnou, S., 2009. "Exemplar-based inpainting from a variational point of view," SIAM Journal on Mathematical Analysis.
  15. Akl, A., Yaacoub, C., Donias, M., Da Costa, J.-P., Germain, C., 2014. Structure tensor based synthesis of directional textures for virtual material design. 2014 IEEE International Conference on Image Processing, ICIP 2014, art. no. 7025986, pp. 4867-4871.
  16. ITU-R Recommendation BT.601-7, “Studio encoding parameters of digital television for standard 4:3 and wide screen 16:9 aspect ratios,” ITU-R, Mar. 2011.
  17. Xu, Y.Q., Guo, B., Shum, H., 2000. “Chaos mosaic: Fast and memory efficient texture synthesis,” In Tech. Rep. MSRTR-2000-32, Microsoft Research.
  18. Akl, A., Iskandar, J., 2015. Structure tensor regularization for texture analysis. 5th International Conference on Image Processing, Theory, Tools and Applications 2015, IPTA 2015, art. no. 7367217, pp. 592-596.
  19. Criminisi, A., Pérez, P., Toyama, K., 2004. "Region filling and object removal by exemplar-based image inpainting," Microsoft Research, Cambridge (UK) and Redmond (US).
  20. Kullback, S., Leibler, R.A.., 1951. “On information and sufficiency,” Ann. Math. Statist., vol. 22(1), pp. 79- 86.
  21. Akl, A., Iskandar, J., 2016. Second-moment matrix adaptation for local orientation estimation. International Conference on Systems, Signals, and Image Processing, 2016-June.
Download


Paper Citation


in Harvard Style

Akl A. and Yaacoub C. (2017). A Tensor-based Technique for Structure-aware Image Inpainting . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 599-605. DOI: 10.5220/0006214605990605


in Bibtex Style

@conference{icpram17,
author={Adib Akl and Charles Yaacoub},
title={A Tensor-based Technique for Structure-aware Image Inpainting},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={599-605},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006214605990605},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - A Tensor-based Technique for Structure-aware Image Inpainting
SN - 978-989-758-222-6
AU - Akl A.
AU - Yaacoub C.
PY - 2017
SP - 599
EP - 605
DO - 10.5220/0006214605990605