# EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET

### Fabio Roda, Leo Liberti, Franco Raimondi

#### Abstract

In this paper we report our experience in the implementation of three collaborative filtering algorithms (user-based k-nearest neighbour, Slope One and TMW, our original algorithm) to provide a recommendation service on an existing website. We carry out the comparison by means of a typical metric, namely the accuracy (RMSE). Usually, evaluations for these kinds of algorithms are carried out using off-line analysis, withholding values from a dataset, and trying to predict them again using the remaining portion of the dataset (the so-called “leave-n-out approach”). We adopt a “live” method on an existing website: when a user rates an item, we also store in parallel the predictions of the algorithms on the same item. We got some unexpected results. In the next sections we describe the algorithms, the benchmark, the testing method, and discuss the outcome of this exercise. Our contribution is a report of the initial phase of a Recommender Systems project with a focus on some possible difficulties on the interpretation of the initial results.

#### References

- Aggarwal, C., Wolf, J., Wu, K.-L., and Yu, P. (1999). Horting hatches an egg: a new graph-theoretic approach to collaborative filtering. In Fayyad, U., Chaudhuri, S., and Madigan, D., editors, Proceedings of the fifth ACM SIGKDD international conference on Knowledge discovery and data mining, pages 201-212, New York. ACM.
- Breese, J., Heckerman, D., and Kadie, C. (1998). Empirical analysis of predictive algorithms for collaborative filtering. Technical Report MSR-TR-98-12, Microsoft Research.
- Huang, Z., Chung, W., and Chen, H. (2004). A graph model for e-commerce recommender systems. Journal of the American Society for Information Science and Technology, 55(3):259-274.
- Lemire, D. and Maclachlan, A. (2005). Slope one predictors for online rating-based collaborative filtering. In Proceedings of SIAM Data Mining (SDM'05).
- Punnen, A. (1991). A linear time algorithm for the maximum capacity path problem. European Journal of Operational Research, 53:402-404.
- Roda, F., Liberti, L., and Raimondi, F. (2009). Combinatorial optimization based recommender systems. In Proceedings of the 8th Cologne-Twente workshop (CTW09) on Graphs and Combinatorial Optimization, Paris.
- Vozalis, E. and Margaritis, K. (2003). Analysis of recommender systems algorithms. In Lipitakis, E., editor, The 6th Hellenic European Conference on Computer Mathematics & its Applications, pages 732-745, Athens. Athens University of Economics and Business.

#### Paper Citation

#### in Harvard Style

Roda F., Liberti L. and Raimondi F. (2011). **EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET** . In *Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,* ISBN 978-989-8425-51-5, pages 603-606. DOI: 10.5220/0003336506030606

#### in Bibtex Style

@conference{webist11,

author={Fabio Roda and Leo Liberti and Franco Raimondi},

title={EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET},

booktitle={Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,},

year={2011},

pages={603-606},

publisher={SciTePress},

organization={INSTICC},

doi={10.5220/0003336506030606},

isbn={978-989-8425-51-5},

}

#### in EndNote Style

TY - CONF

JO - Proceedings of the 7th International Conference on Web Information Systems and Technologies - Volume 1: WEBIST,

TI - EVALUATION OF COLLABORATIVE FILTERING ALGORITHMS USING A SMALL DATASET

SN - 978-989-8425-51-5

AU - Roda F.

AU - Liberti L.

AU - Raimondi F.

PY - 2011

SP - 603

EP - 606

DO - 10.5220/0003336506030606