Static and Dynamic Approaches for Pain Intensity Estimation using Facial Expressions

Niloufar Zebarjadi, Iman Alikhani

Abstract

Self-report is the most conventional means of pain intensity assessment in clinical environments. But, it is not an accurate metric or not even possible to measure in many circumstances, e.g. intensive care units. Continuous and automatic pain level evaluation is an advantageous solution to overcome this issue. In this paper, we aim to map facial expressions to pain intensity levels. We extract well-known static (local binary pattern(LBP) and dense scale-invariant feature transform (DSIFT)) and dynamic (local binary patterns on three orthogonal planes (LBP-TOP) and three dimensional scale-invariant feature transform (3D-SIFT)) facial feature descriptors and employ the linear regression method to label a number between zero (no pain) to five (strong pain) to each testing sequence. We have evaluated our methods on the publicly available UNBC-McMaster shoulder pain expression archive database and achieved average mean square error (MSE) of 1.53 and Pearson correlation coefficient (PCC) of 0.79 using leave-one-subject-out cross validation. Acquired results prove the superiority of dynamic facial features compared to the static ones in pain intensity determination applications.

References

  1. Ahonen, T., Hadid, A., and Pietikäinen, M. (2004). Face recognition with local binary patterns. In Computer vision-eccv 2004, pages 469-481. Springer.
  2. Ahonen, T., Hadid, A., and Pietikainen, M. (2006). Face description with local binary patterns: Application to face recognition. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 28(12):2037-2041.
  3. Ashraf, A. B., Lucey, S., Cohn, J. F., Chen, T., Ambadar, Z., Prkachin, K. M., and Solomon, P. E. (2009). The painful face-pain expression recognition using active appearance models. Image and vision computing, 27(12):1788-1796.
  4. Kaltwang, S., Rudovic, O., and Pantic, M. (2012). Continuous pain intensity estimation from facial expressions. In Advances in Visual Computing, pages 368- 377. Springer.
  5. Khan, R. A., Meyer, A., Konik, H., and Bouakaz, S. (2013). Pain detection through shape and appearance features. In Multimedia and Expo (ICME), 2013 IEEE International Conference on, pages 1-6. IEEE.
  6. Krig, S. (2014). Computer Vision Metrics: Survey, Taxonomy, and Analysis. Apress.
  7. Lowe, D. G. (2004). Distinctive image features from scaleinvariant keypoints. International Journal of Computer Vision, 60(2):91-110.
  8. Lucey, P., Cohn, J. F., Matthews, I., Lucey, S., Sridharan, S., Howlett, J., and Prkachin, K. M. (2011a). Automatically detecting pain in video through facial action units. IEEE Transactions on Systems, Man, and Cybernetics, Part B (Cybernetics), 41(3):664-674.
  9. Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., Chew, S., and Matthews, I. (2012). Painful monitoring: Automatic pain monitoring using the unbcmcmaster shoulder pain expression archive database. Image and Vision Computing, 30(3):197-205.
  10. Lucey, P., Cohn, J. F., Prkachin, K. M., Solomon, P. E., and Matthews, I. (2011b). Painful data: The unbcmcmaster shoulder pain expression archive database. In Automatic Face & Gesture Recognition and Workshops (FG 2011), 2011 IEEE International Conference on, pages 57-64. IEEE.
  11. Neshov, N. and Manolova, A. (2015). Pain detection from facial characteristics using supervised descent method. In Intelligent Data Acquisition and Advanced Computing Systems: Technology and Applications (IDAACS), 2015 IEEE 8th International Conference on, volume 1, pages 251-256. IEEE.
  12. Ojala, T., Pietikäinen, M., and Harwood, D. (1996). A comparative study of texture measures with classification based on featured distributions. Pattern Recognition, 29(1):51-59.
  13. Ojala, T., Pietikäinen, M., and Mäenpää, T. (2002). Multiresolution gray-scale and rotation invariant texture classification with local binary patterns. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 24(7):971-987.
  14. Rathee, N. and Ganotra, D. (2015). A novel approach for pain intensity detection based on facial feature deformations. Journal of Visual Communication and Image Representation, 33:247-254.
  15. Roy, S. D., Bhowmik, M. K., Saha, P., and Ghosh, A. K. (2016). An approach for automatic pain detection through facial expression. Procedia Computer Science, 84:99-106.
  16. Scovanner, P., Ali, S., and Shah, M. (2007). A 3- dimensional sift descriptor and its application to action recognition. In Proceedings of the 15th International Conference on Multimedia, pages 357-360. ACM.
  17. Vedaldi, A. and Fulkerson, B. (2010). Vlfeat: An open and portable library of computer vision algorithms.
  18. In Proceedings of the 18th ACM international conference on Multimedia, pages 1469-1472. ACM.
  19. Zhao, G. and Pietikainen, M. (2007). Dynamic texture recognition using local binary patterns with an application to facial expressions. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 29(6):915- 928.
Download


Paper Citation


in Harvard Style

Zebarjadi N. and Alikhani I. (2017). Static and Dynamic Approaches for Pain Intensity Estimation using Facial Expressions . In Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, (BIOSTEC 2017) ISBN 978-989-758-213-4, pages 291-296. DOI: 10.5220/0006141502910296


in Bibtex Style

@conference{healthinf17,
author={Niloufar Zebarjadi and Iman Alikhani},
title={Static and Dynamic Approaches for Pain Intensity Estimation using Facial Expressions},
booktitle={Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, (BIOSTEC 2017)},
year={2017},
pages={291-296},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006141502910296},
isbn={978-989-758-213-4},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 10th International Joint Conference on Biomedical Engineering Systems and Technologies - Volume 5: HEALTHINF, (BIOSTEC 2017)
TI - Static and Dynamic Approaches for Pain Intensity Estimation using Facial Expressions
SN - 978-989-758-213-4
AU - Zebarjadi N.
AU - Alikhani I.
PY - 2017
SP - 291
EP - 296
DO - 10.5220/0006141502910296