FAST REAL-TIME SEGMENTATION AND TRACKING OF MULTIPLE SUBJECTS BY TIME-OF-FLIGHT CAMERA - A New Approach for Real-time Multimedia Applications with 3D Camera Sensor

Piercarlo Dondi, Luca Lombardi

Abstract

Time-of-Flight cameras are a new kind of sensors that use near-infrared light to provide distance measures of an environment. In this paper we present a very fast method for real-time segmentation and tracking, that exploits the peculiar characteristics of these devices. The foreground segmentation is achieved by a dynamic thresholding and region growing: an appropriate correction based on flexible intensity thresholding and mathematical morphology is used to partially compensate one of the most common problem of the TOF cameras, the noise generated by sun light. By the use of a Kalman filter for tracking the retrieved objects the system is able to correctly handle the occlusions and to follow multiple objects placed at different distances. The proposed system is our basic step for complex multimedia applications, such as augmented reality. An example of mixed reality that includes the integration of color information, supplied by a webcam is shown in the experimental results.

References

  1. Bartczak, B., Schiller, I., Beder, C., and Koch, R. (2008). Integration of a time-of-flight camera into a mixed reality system for handling dynamic scenes, moving viewpoints and occlusions in real-time. In 3DPVT08, Fourth International Symposium on 3D Data Processing, Visualization and Transmission.
  2. Bevilacqua, A., Stefano, L. D., and Azzari, P. (2006). People tracking using a time-of-flight depth sensor. In AVSS 06, Video and Signal Based Surveillance. IEEE Computer Society.
  3. Bianchi, L., Dondi, P., Gatti, R., L.Lombardi, and Lombardi, P. (2009). Evaluation of a foreground segmentation algorithm for 3d camera sensor. In ICIAP 2009, 15th International Conference of Image Analysis and Processing. Springer.
  4. Bleiweiss, A. and Werman, M. (2009). Real-time foreground segmentation via range and color imaging. In Dyn3D09, Proceedings of DAGM2009 Workshop on Dynamic 3D Imaging. Springer.
  5. Crabb, R., Tracey, C., Puranik, A., and Davis, J. (2008). Real-time foreground segmentation via range and color imaging. In CVPRW 2008, Computer Vision and Pattern Recognition Workshops. IEEE Computer Society.
  6. Guomundsson, S., Larsen, R., Aanaes, H., Pardas, M., and Casas, J. R. (2008). Tof imaging in smart room environments towards improved people tracking. In CVPRW 2008, Computer Vision and Pattern Recognition Workshops. IEEE Computer Society.
  7. Hansen, D. W., Hansen, M. S., Kirschmeyer, M., Larsen, R., Silvestre, D., and Silvestre, D. (2008). Cluster tracking with time-of-flight cameras. In CVPRW 2008, Computer Vision and Pattern Recognition Workshops. IEEE Computer Society.
  8. Kolb, A., Barth, E., Koch, R., and Larsen, R. (2010). Timeof-flight cameras in computer graphics. In Computer Graphics Forum volume 29, issue 1. Wiley.
  9. Lindner, M. and Kolb, A. (2007). Data-fusion of pmd-based distance-information and high-resolution rgb-images. In ISSCS 2007, International Symposium on Signals, Circuits and Systems.
  10. Oggier, T., Lehmann, M., Kaufmann, R., Schweizer, M., Richter, M., Metzler, P., Lang, G., Lustenberger, F., and Blanc, N. (2004). An all-solid-state optical range camera for 3d real-time imaging with sub-centimeter depth resolution (swissranger). In SPIE 2004, Society of Photo-Optical Instrumentation Engineers Conference Series.
  11. Oprisescu, S., Falie, D., Ciuc, M., and Buzuloiu, V. (2007). Measurements with tof cameras and their necessary corrections. In ISSCS 2007, International Symposium on Signals, Circuits and Systems.
  12. Parvizi, E. and Wu, Q. J. (2008). Multiple object tracking based on adaptive depth segmentation. In Canadian Conference of Computer and Robot Vision, pages 273-277. IEEE Computer Society.
  13. Reulke, R. (2006). Combination of distance data with high resolution images. In IEVM06, Image Engeeniring and Vision Metrology.
  14. Santrac, N., Friedland, G., and Rojas, R. (2006). High resolution segmentation with a time-of-flight 3dcamera using the example of a lecture scene. Technical report, http://www.inf.fu-berlin.de/inst/agki/eng/index.html.
Download


Paper Citation


in Harvard Style

Dondi P. and Lombardi L. (2011). FAST REAL-TIME SEGMENTATION AND TRACKING OF MULTIPLE SUBJECTS BY TIME-OF-FLIGHT CAMERA - A New Approach for Real-time Multimedia Applications with 3D Camera Sensor . In Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011) ISBN 978-989-8425-47-8, pages 582-587. DOI: 10.5220/0003322505820587


in Bibtex Style

@conference{visapp11,
author={Piercarlo Dondi and Luca Lombardi},
title={FAST REAL-TIME SEGMENTATION AND TRACKING OF MULTIPLE SUBJECTS BY TIME-OF-FLIGHT CAMERA - A New Approach for Real-time Multimedia Applications with 3D Camera Sensor},
booktitle={Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011)},
year={2011},
pages={582-587},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003322505820587},
isbn={978-989-8425-47-8},
}


in EndNote Style

TY - CONF
JO - Proceedings of the International Conference on Computer Vision Theory and Applications - Volume 1: VISAPP, (VISIGRAPP 2011)
TI - FAST REAL-TIME SEGMENTATION AND TRACKING OF MULTIPLE SUBJECTS BY TIME-OF-FLIGHT CAMERA - A New Approach for Real-time Multimedia Applications with 3D Camera Sensor
SN - 978-989-8425-47-8
AU - Dondi P.
AU - Lombardi L.
PY - 2011
SP - 582
EP - 587
DO - 10.5220/0003322505820587