Sparse Physics-based Gaussian Process for Multi-output Regression using Variational Inference

Ankit Chiplunkar, Emmanuel Rachelson, Michele Colombo, Joseph Morlier

Abstract

In this paper a sparse approximation of inference for multi-output Gaussian Process models based on a Variational Inference approach is presented. In Gaussian Processes a multi-output kernel is a covariance function over correlated outputs. Using a general framework for constructing auto- and cross-covariance functions that are consistent with the physical laws, physical relationships among several outputs can be imposed. One major issue with Gaussian Processes is efficient inference, when scaling up-to large datasets. The issue of scaling becomes even more important when dealing with multiple outputs, since the cost of inference increases rapidly with the number of outputs. In this paper we combine the use of variational inference for efficient inference with multi-output kernels enforcing relationships between outputs. Results of the proposed methodology for synthetic data and real world applications are presented. The main contribution of this paper is the application and validation of our methodology on a dataset of real aircraft flight tests, while imposing knowledge of aircraft physics into the model.

References

  1. Alvarez, M. and Lawrence, N. D. (2009). Sparse convolved gaussian processes for multi-output regression. In Koller, D., Schuurmans, D., Bengio, Y., and Bottou, L., editors, Advances in Neural Information Processing Systems 21, pages 57-64. Curran Associates, Inc.
  2. Alvarez, M. A., Luengo, D., and Lawrence, N. D. (2009). Latent force models. In Dyk, D. A. V. and Welling, M., editors, AISTATS, volume 5 of JMLR Proceedings, pages 9-16. JMLR.org.
  3. Ílvarez, M. A., Luengo, D., Titsias, M. K., and Lawrence, N. D. (2010). Efficient multioutput gaussian processes through variational inducing kernels. In Teh, Y. W. and Titterington, D. M., editors, AISTATS, volume 9 of JMLR Proceedings, pages 25-32. JMLR.org.
  4. Bonilla, E., Chai, K. M., and Williams, C. (2008). Multitask gaussian process prediction. In Platt, J., Koller, D., Singer, Y., and Roweis, S., editors, Advances in Neural Information Processing Systems 20, pages 153-160. MIT Press, Cambridge, MA.
  5. Boyle, P. and Frean, M. (2005). Dependent gaussian processes. In In Advances in Neural Information Processing Systems 17, pages 217-224. MIT Press.
  6. Constantinescu, E. M. and Anitescu, M. (2013). Physicsbased covariance models for gaussian processes with multiple outputs. International Journal for Uncertainty Quantification, 3.
  7. Deisenroth, M. P. and Ng, J. W. (2015). Distributed gaussian processes. In Proceedings of the 32nd International Conference on Machine Learning, ICML 2015, Lille, France, 6-11 July 2015, pages 1481-1490.
  8. Quionero-candela, J., Rasmussen, C. E., and Herbrich, R. (2005). A unifying view of sparse approximate gaussian process regression. Journal of Machine Learning Research, 6:2005.
  9. Rasmussen, C. E. and Williams, C. K. I. (2005). Gaussian Processes for Machine Learning (Adaptive Computation and Machine Learning). The MIT Press.
  10. Snelson, E. and Ghahramani, Z. (2006). Sparse gaussian processes using pseudo-inputs. In ADVANCES IN NEURAL INFORMATION PROCESSING SYSTEMS, pages 1257-1264. MIT press.
  11. Solak, E., Murray-smith, R., Leithead, W. E., Leith, D. J., and Rasmussen, C. E. (2003). Derivative observations in gaussian process models of dynamic systems. In Becker, S., Thrun, S., and Obermayer, K., editors, Advances in Neural Information Processing Systems 15, pages 1057-1064. MIT Press.
  12. Stein, M. L. (1999). Interpolation of Spatial Data: Some Theory for Kriging. Springer, New York.
  13. Titsias, M. K. (2009). Variational learning of inducing variables in sparse gaussian processes. In In Artificial Intelligence and Statistics 12, pages 567-574.
  14. Vanhatalo, J., Riihimäki, J., Hartikainen, J., Jylänki, P., Tolvanen, V., and Vehtari, A. (2013). Gpstuff: Bayesian modeling with gaussian processes. J. Mach. Learn. Res., 14(1):1175-1179.
Download


Paper Citation


in Harvard Style

Chiplunkar A., Rachelson E., Colombo M. and Morlier J. (2016). Sparse Physics-based Gaussian Process for Multi-output Regression using Variational Inference . In Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM, ISBN 978-989-758-173-1, pages 437-445. DOI: 10.5220/0005700504370445


in Bibtex Style

@conference{icpram16,
author={Ankit Chiplunkar and Emmanuel Rachelson and Michele Colombo and Joseph Morlier},
title={Sparse Physics-based Gaussian Process for Multi-output Regression using Variational Inference},
booktitle={Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,},
year={2016},
pages={437-445},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005700504370445},
isbn={978-989-758-173-1},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 5th International Conference on Pattern Recognition Applications and Methods - Volume 1: ICPRAM,
TI - Sparse Physics-based Gaussian Process for Multi-output Regression using Variational Inference
SN - 978-989-758-173-1
AU - Chiplunkar A.
AU - Rachelson E.
AU - Colombo M.
AU - Morlier J.
PY - 2016
SP - 437
EP - 445
DO - 10.5220/0005700504370445