Domain Adaptation Transfer Learning by SVM Subject to a Maximum-Mean-Discrepancy-like Constraint

Xiaoyi Chen, Régis Lengellé

Abstract

This paper is a contribution to solving the domain adaptation problem where no labeled target data is available.A new SVM approach is proposed by imposing a zero-valued Maximum Mean Discrepancy-like constraint.This heuristic allows us to expect a good similarity between source and target data, after projection onto an efficient subspace of a Reproducing Kernel Hilbert Space. Accordingly, the classifier will perform well on source and target data. We show that this constraint does not modify the quadratic nature of the optimization problem encountered in classic SVM, so standard quadratic optimization tools can be used. Experimental results demonstrate the competitiveness and efficiency of our method.

References

  1. Blitzer, J., Dredze, M., Pereira, F., et al. (2007). Biographies, bollywood, boom-boxes and blenders: Domain adaptation for sentiment classification. In ACL, volume 7, pages 440-447.
  2. Bruzzone, L. and Marconcini, M. (2010). Domain adaptation problems: A dasvm classification technique and a circular validation strategy. Pattern Analysis and Machine Intelligence, IEEE Transactions on, 32(5):770- 787.
  3. Dudley, R. M. (1984). A course on empirical processes. In Ecole d'été de Probabilités de Saint-Flour XII-1982, pages 1-142. Springer.
  4. Dudley, R. M. (2002). Real analysis and probability, volume 74. Cambridge University Press.
  5. Fortet, R. and Mourier, E. (1953). Convergence de la répartition empirique vers la réparation théorique. Ann. Scient. Ócole Norm. Sup., pages 266-285.
  6. Gretton, A., Borgwardt, K. M., Rasch, M. J., Schölkopf, B., and Smola, A. (2012). A kernel two-sample test. J. Mach. Learn. Res., 13:723-773.
  7. Huang, C.-H., Yeh, Y.-R., and Wang, Y.-C. F. (2012). Recognizing actions across cameras by exploring the correlated subspace. In Computer Vision-ECCV 2012. Workshops and Demonstrations, pages 342- 351. Springer.
  8. Huang, J., Gretton, A., Borgwardt, K. M., Schölkopf, B., and Smola, A. J. (2006). Correcting sample selection bias by unlabeled data. In Advances in neural information processing systems, pages 601-608.
  9. Jiang, J. (2008). A literature survey on domain adaptation of statistical classifiers. URL: http://sifaka. cs. uiuc. edu/jiang4/domainadaptation/survey.
  10. Li, L., Zhou, K., Xue, G.-R., Zha, H., and Yu, Y. (2011). Video summarization via transferrable structured learning. In Proceedings of the 20th international conference on World wide web, pages 287-296. ACM.
  11. Liang, F., Tang, S., Zhang, Y., Xu, Z., and Li, J. (2014). Pedestrian detection based on sparse coding and transfer learning. Machine Vision and Applications, 25(7):1697-1709.
  12. Pan, S. J., Tsang, I. W., Kwok, J. T., and Yang, Q. (2011). Domain adaptation via transfer component analysis. Neural Networks, IEEE Transactions on, 22(2):199- 210.
  13. Pan, S. J. and Yang, Q. (2010). A survey on transfer learning. Knowledge and Data Engineering, IEEE Transactions on, 22(10):1345-1359.
  14. Patel, V. M., Gopalan, R., Li, R., and Chellappa, R. (2015). Visual domain adaptation: A survey of recent advances. IEEE signal processing magazine, 32(3):53- 69.
  15. Paulsen, V. I. (2009). An introduction to the theory of reproducing kernel hilbert spaces. Lecture Notes.
  16. Quanz, B. and Huan, J. (2009). Large margin transductive transfer learning. In Proceedings of the 18th ACM conference on Information and knowledge management, pages 1327-1336. ACM.
  17. Ren, J., Liang, Z., and Hu, S. (2010). Multiple kernel learning improved by mmd. In Advanced Data Mining and Applications, pages 63-74. Springer.
  18. Schölkopf, B., Herbrich, R., and Smola, A. J. (2001). A generalized representer theorem. In Computational learning theory, pages 416-426. Springer.
  19. Serfling, R. J. (2009). Approximation theorems of mathematical statistics, volume 162. John Wiley & Sons.
  20. Smola, A. (2006). Maximum mean discrepancy. In 13th International Conference, ICONIP 2006, Hong Kong, China, October 3-6, 2006: Proceedings.
  21. Smola, A., Gretton, A., Song, L., and Schölkopf, B. (2007). A hilbert space embedding for distributions. In Algorithmic Learning Theory, pages 13-31. Springer.
  22. Steinwart, I. (2002). On the influence of the kernel on the consistency of support vector machines. The Journal of Machine Learning Research, 2:67-93.
  23. Tan, Q., Deng, H., and Yang, P. (2012). Kernel mean matching with a large margin. In Advanced Data Mining and Applications, pages 223-234. Springer.
  24. Tohmé, M. and Lengellé, R. (2008). F-svc: A simple and fast training algorithm soft margin support vector classification. In Machine Learning for Signal Processing, 2008. MLSP 2008. IEEE Workshop on, pages 339- 344. IEEE.
  25. Uguroglu, S. and Carbonell, J. (2011). Feature selection for transfer learning. In Machine Learning and Knowledge Discovery in Databases, pages 430-442. Springer.
  26. Yang, S., Lin, M., Hou, C., Zhang, C., and Wu, Y. (2012). A general framework for transfer sparse subspace learning. Neural Computing and Applications, 21(7):1801- 1817.
  27. Zhang, P., Zhu, X., and Guo, L. (2009). Mining data streams with labeled and unlabeled training examples. In Data Mining, 2009. ICDM'09. Ninth IEEE International Conference on, pages 627-636. IEEE.
  28. Zhang, Z. and Zhou, J. (2012). Multi-task clustering via domain adaptation. Pattern Recognition, 45(1):465- 473.
Download


Paper Citation


in Harvard Style

Chen X. and Lengellé R. (2017). Domain Adaptation Transfer Learning by SVM Subject to a Maximum-Mean-Discrepancy-like Constraint . In Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-222-6, pages 89-95. DOI: 10.5220/0006119900890095


in Bibtex Style

@conference{icpram17,
author={Xiaoyi Chen and Régis Lengellé},
title={Domain Adaptation Transfer Learning by SVM Subject to a Maximum-Mean-Discrepancy-like Constraint},
booktitle={Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2017},
pages={89-95},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0006119900890095},
isbn={978-989-758-222-6},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 6th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Domain Adaptation Transfer Learning by SVM Subject to a Maximum-Mean-Discrepancy-like Constraint
SN - 978-989-758-222-6
AU - Chen X.
AU - Lengellé R.
PY - 2017
SP - 89
EP - 95
DO - 10.5220/0006119900890095