loading
Documents

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Tomonari Masada ; Yuichiro Shibata and Kiyoshi Oguri

Affiliation: Nagasaki University, Japan

ISBN: 978-989-8425-53-9

Keyword(s): Maximal substrings, Document clustering, Suffix array, Bayesian modeling.

Related Ontology Subjects/Areas/Topics: Artificial Intelligence ; Biomedical Engineering ; Data Engineering ; Data Mining ; Databases and Information Systems Integration ; Enterprise Information Systems ; Health Information Systems ; Information Systems Analysis and Specification ; Knowledge Management ; Ontologies and the Semantic Web ; Sensor Networks ; Signal Processing ; Society, e-Business and e-Government ; Soft Computing ; Software Agents and Internet Computing ; Web 2.0 and Social Networking Controls ; Web Information Systems and Technologies

Abstract: This paper provides experimental results showing how we can use maximal substrings as elementary features in document clustering. We extract maximal substrings, i.e., the substrings each giving a smaller number of occurrences even after adding only one character at its head or tail, from the given document set and represent each document as a bag of maximal substrings after reducing the variety of maximal substrings by a simple frequency-based selection. This extraction can be done in an unsupervised manner. Our experiment aims to compare bag of maximal substrings representation with bag of words representation in document clustering. For clustering documents, we utilize Dirichlet compound multinomials, a Bayesian version of multinomial mixtures, and measure the results by F-score. Our experiment showed that maximal substrings were as effective as words extracted by a dictionary-based morphological analysis for Korean documents. For Chinese documents, maximal substrings were not so ef fective as words extracted by a supervised segmentation based on conditional random fields. However, one fourth of the clustering results given by bag of maximal substrings representation achieved F-scores better than the mean F-score given by bag of words representation. It can be said that the use of maximal substrings achieved an acceptable performance in document clustering. (More)

PDF ImageFull Text

Download
Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 54.221.75.115

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Masada T., Shibata Y. and Oguri K. (2011). DOCUMENTS AS A BAG OF MAXIMAL SUBSTRINGS - An Unsupervised Feature Extraction for Document Clustering.In Proceedings of the 13th International Conference on Enterprise Information Systems - Volume 1: ICEIS, ISBN 978-989-8425-53-9, pages 5-13. DOI: 10.5220/0003403300050013

@conference{iceis11,
author={Tomonari Masada and Yuichiro Shibata and Kiyoshi Oguri},
title={DOCUMENTS AS A BAG OF MAXIMAL SUBSTRINGS - An Unsupervised Feature Extraction for Document Clustering},
booktitle={Proceedings of the 13th International Conference on Enterprise Information Systems - Volume 1: ICEIS,},
year={2011},
pages={5-13},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0003403300050013},
isbn={978-989-8425-53-9},
}

TY - CONF

JO - Proceedings of the 13th International Conference on Enterprise Information Systems - Volume 1: ICEIS,
TI - DOCUMENTS AS A BAG OF MAXIMAL SUBSTRINGS - An Unsupervised Feature Extraction for Document Clustering
SN - 978-989-8425-53-9
AU - Masada T.
AU - Shibata Y.
AU - Oguri K.
PY - 2011
SP - 5
EP - 13
DO - 10.5220/0003403300050013

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.