Towards a Multi-camera Mouse-replacement Interface

John Magee, Zheng Wu, Harshith Chennamaneni, Samuel Epstein, Diana H. Theriault, Margrit Betke

2010

Abstract

We present our efforts towards a multi-camera mouse-replacement system for computer users with severe motion impairments. We have worked with individuals with cerebral palsy or multiple sclerosis who use a publiclyavailable interface that tracks the user’s head movements with a single video camera and translates them into mouse pointer coordinates on the screen. To address the problem that the interface can lose track of the user’s facial feature due to occlusion or spastic movements, we started to develop a multi-camera interface. Our multi-camera capture system can record synchronized images from multiple cameras and automatically analyze the camera arrangement.We recorded 15 subjects while they were conducting a hands-free interaction experiment. We reconstructed via stereoscopy the three-dimensional movement trajectories of various facial features. Our analysis shows that single-camera interfaces based on twodimensional feature tracking neglect to take into account the substantial feature movement in the third dimension.

References

  1. A. H. Gee and R. Cipolla. Tracking faces. In R. Cipolla and A. Pentland, editors, Computer Vision for Human-Machine Interaction, pages 113-122. Cambridge University Press, 1998.
  2. W. T. Freeman, D. Anderson, P. Beardsley, C. Dodge, H. Kage, K. Kyuma, Y. Miyake, M. Roth, K. Tanaka, C. Weissman, and W. Yerazunis. Computer vision for interactive computer graphics. IEEE Computer Graphics and Applications, 18(3):42-53, May 1998.
  3. M. Turk and R. George. Perceptual user interfaces. Comm. of the ACM, 43(3), 2000.
  4. M. Betke, J. Gips, and P. Fleming. The camera mouse: Visual tracking of body features to provide computer access for people with severe disabilities. IEEE Transactions on Neural Systems and Rehabilitation Engineering, 10(1):1-10, 2002.
  5. J. Varona, C. Manresa-Yee, and F. J. Perales. Hands-free vision-based interface for computer accessibility. Journal of Network and Computer Applications, 31(4):357-374, 2008.
  6. R. Kjeldsen. Improvements in vision-based pointer control. In 8th International ACM SIGACCESS Conference on Computers and accessibility (Assets 7806), pages 189-196, 2006.
  7. D. O. Gorodnichy and G. Roth. Nouse 'use your nose as a mouse' perceptual vision technology for hands-free games and interfaces. Image and Vision Computing, 22(12):931-942, 2004.
  8. E. A. Cansizoglu and M. Betke. An information fusion approach for multiview feature tracking. In 20th International Conference on Pattern Recognition (ICPR 2010), Istanbul, Turkey, 2010. 4 pp.
  9. T. K. Marks, J. R. Hershey, and J. R. Movellan. Tracking motion, deformation, and texture using conditionally Gaussian processes. IEEE Transactions on Pattern Analysis and Machine Intelligence, 32(2):348-363, 2010.
  10. C. John, U. Schwanecke, and H. Regenbrecht. Real-time volumetric reconstruction and tracking of hands and face as a user interface for virtual environments. In VR 2009: IEEE Virtual Reality Conference, pages 241-242, 2009.
  11. M. Rätsch, C. Blumer, G. Teschke, and T. Vetter. Coarse-to-fine particle filters for multiobject human computer interaction. In IEEE International Workshop on Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS 2009), pages 440-445, September 2009.
  12. K. Sidorov, Y. Hicks, D. Marshall, S. Sanei, and J. Chambers. Real time multi camera 3D tracking system. In 3rd European Conference on Visual Media Production (CVMP 2006), page 191, London, UK, 2006.
  13. F. Dornaika and B. Raducanu. Three-dimensional face pose detection and tracking using monocular videos: Tool and application. IEEE Transactions on Systems, Man, and Cybernetics, Part B: Cybernetics, 39(4):935-944, 2009.
  14. G. Bradski and A. Kaehler. Learning OpenCV, chapter 11-12. O'Reilly, 2008.
  15. R. I. Hartley and A. Zisserman. Multiple View Geometry in Computer Vision. Cambridge University Press, 2003.
  16. M. A. Fischler and R. C. Bolles. Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Communications of the ACM, 24(6):381-395, 1981.
  17. Camera Mouse - Innovative software for people with disabilities, http://www.cameramouse.org, accessed April 2010.
Download


Paper Citation


in Harvard Style

Magee J., Wu Z., Chennamaneni H., Epstein S., H. Theriault D. and Betke M. (2010). Towards a Multi-camera Mouse-replacement Interface . In Proceedings of the 10th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2010) ISBN 978-989-8425-14-0, pages 33-42. DOI: 10.5220/0003001700330042


in Bibtex Style

@conference{pris10,
author={John Magee and Zheng Wu and Harshith Chennamaneni and Samuel Epstein and Diana H. Theriault and Margrit Betke},
title={Towards a Multi-camera Mouse-replacement Interface},
booktitle={Proceedings of the 10th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2010)},
year={2010},
pages={33-42},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003001700330042},
isbn={978-989-8425-14-0},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Workshop on Pattern Recognition in Information Systems - Volume 1: PRIS, (ICEIS 2010)
TI - Towards a Multi-camera Mouse-replacement Interface
SN - 978-989-8425-14-0
AU - Magee J.
AU - Wu Z.
AU - Chennamaneni H.
AU - Epstein S.
AU - H. Theriault D.
AU - Betke M.
PY - 2010
SP - 33
EP - 42
DO - 10.5220/0003001700330042