EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET

Fabio Roda, Leo Liberti, Franco Raimondi

Abstract

In this paper we report our experience in the implementation of three collaborative filtering algorithms (user-based k-nearest neighbour, Slope One and TMW, our original algorithm) to provide a recommendation service on an existing website. We carry out the comparison by means of a typical metric, namely the accuracy (RMSE). Usually, evaluations for these kinds of algorithms are carried out using off-line analysis, withholding values from a dataset, and trying to predict them again using the remaining portion of the dataset (the so-called “leave-n-out approach”). We adopt a “live” method on an existing website: when a user rates an item, we also store in parallel the predictions of the algorithms on the same item. We got some unexpected results. In the next sections we describe the algorithms, the benchmark, the testing method, and discuss the outcome of this exercise. Our contribution is a report of the initial phase of a Recommender Systems project with a focus on some possible difficulties on the interpretation of the initial results.

References

  1. Aggarwal, C., Wolf, J., Wu, K.-L., and Yu, P. (1999). Horting hatches an egg: a new graph-theoretic approach to collaborative filtering. In Fayyad, U., Chaudhuri, S., and Madigan, D., editors, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 201-212, New York. ACM.
  2. Breese, J., Heckerman, D., and Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. Technical Report MSR-TR-98-12, Microsoft Research.
  3. Huang, Z., Chung, W., and Chen, H. (2004). A graph model for e-commerce recommender systems. Journal of the American Society for Information Science and Technology, 55(3):259-274.
  4. Lemire, D. and Maclachlan, A. (2005). Slope one predictors for online rating-based collaborative filtering. In Proceedings of SIAM Data Mining (SDM'05).
  5. Punnen, A. (1991). A linear time algorithm for the maximum capacity path problem. European Journal of Operational Research, 53:402-404.
  6. Roda, F., Liberti, L., and Raimondi, F. (2009). Combinatorial optimization based recommender systems. In Proceedings of the 8th Cologne-Twente workshop (CTW09) on Graphs and Combinatorial Optimization, Paris.
  7. Vozalis, E. and Margaritis, K. (2003). Analysis of recommender systems algorithms. In Lipitakis, E., editor, The 6th Hellenic European Conference on Computer Mathematics & its Applications, pages 732-745, Athens. Athens University of Economics and Business.
Download


Paper Citation


in Harvard Style

Roda F., Liberti L. and Raimondi F. (2011). EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET . In Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST, ISBN 978-989-8425-51-5, pages 603-606. DOI: 10.5220/0003336506030606


in Bibtex Style

@conference{webist11,
author={Fabio Roda and Leo Liberti and Franco Raimondi},
title={EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET},
booktitle={Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},
year={2011},
pages={603-606},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003336506030606},
isbn={978-989-8425-51-5},
}


in EndNote Style

TY - CONF
JO - Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,
TI - EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET
SN - 978-989-8425-51-5
AU - Roda F.
AU - Liberti L.
AU - Raimondi F.
PY - 2011
SP - 603
EP - 606
DO - 10.5220/0003336506030606