loading
Papers

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Kamel Aouiche and Daniel Lemire

Affiliation: LICEF, University of Quebec at Montreal, Canada

ISBN: 978-972-8865-88-7

Keyword(s): Probabilistic estimation, skewed distributions, sampling, hashing.

Related Ontology Subjects/Areas/Topics: Data Warehouses and OLAP ; Databases and Information Systems Integration ; Enterprise Information Systems

Abstract: Even if storage was infinite, a data warehouse could not materialize all possible views due to the running time and update requirements. Therefore, it is necessary to estimate quickly, accurately, and reliably the size of views. Many available techniques make particular statistical assumptions and their error can be quite large. Unassuming techniques exist, but typically assume we have independent hashing for which there is no known practical implementation. We adapt an unassuming estimator due to Gibbons and Tirthapura: its theoretical bounds do not make unpractical assumptions. We compare this technique experimentally with stochastic probabilistic counting, LOGLOG probabilistic counting, and multifractal statistical models. Our experiments show that we can reliably and accurately (within 10%, 19 times out 20) estimate view sizes over large data sets (1.5 GB) within minutes, using almost no memory. However, only GIBBONS-TIRTHAPURA provides universally tight estimates irrespective of the size of the view. For large views, probabilistic counting has a small edge in accuracy, whereas the competitive sampling-based method (multifractal) we tested is an order of magnitude faster but can sometimes provide poor estimates (relative error of 100%). In our tests, LOGLOG probabilistic counting is not competitive. Experimental validation on the US Census 1990 data set and on the Transaction Processing Performance (TPC H) data set is provided. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 3.214.184.124

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Aouiche K.; Lemire D. and (2007). UNASSSUMING VIEW-SIZE ESTIMATION TECHNIQUES IN OLAP - An Experimental Comparison.In Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS, ISBN 978-972-8865-88-7, pages 145-150. DOI: 10.5220/0002354601450150

@conference{iceis07,
author={Kamel Aouiche and Daniel Lemire},
title={UNASSSUMING VIEW-SIZE ESTIMATION TECHNIQUES IN OLAP - An Experimental Comparison},
booktitle={Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS,},
year={2007},
pages={145-150},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0002354601450150},
isbn={978-972-8865-88-7},
}

TY - CONF

JO - Proceedings of the Ninth International Conference on Enterprise Information Systems - Volume 3: ICEIS,
TI - UNASSSUMING VIEW-SIZE ESTIMATION TECHNIQUES IN OLAP - An Experimental Comparison
SN - 978-972-8865-88-7
AU - Aouiche, K.
AU - Lemire, D.
PY - 2007
SP - 145
EP - 150
DO - 10.5220/0002354601450150

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.