loading
Documents

Research.Publish.Connect.

Paper

Paper Unlock

Authors: Jingwen Wang and Jie Wang

Affiliation: University of Massachusetts, United States

ISBN: 978-989-758-158-8

ISSN: 2184-3228

Keyword(s): Article Extraction, Text Automation, Density, Similarity.

Related Ontology Subjects/Areas/Topics: Artificial Intelligence ; Business Analytics ; Data Analytics ; Data Engineering ; Information Extraction ; Interactive and Online Data Mining ; Knowledge Discovery and Information Retrieval ; Knowledge-Based Systems ; Soft Computing ; Symbolic Systems ; Web Mining

Abstract: We present a new method called qRead to achieve real-time content extractions from web pages with high accuracy. Early approaches to content extractions include empirical filtering rules, Document Object Model (DOM) trees, and machine learning models. These methods, while having met with certain success, may not meet the requirements of real-time extraction with high accuracy. For example, constructing a DOM-tree on a complex web page is time-consuming, and using machine learning models could make things unnecessarily more complicated. Different from previous approaches, qRead uses segment densities and similarities to identify main contents. In particular, qRead first filters obvious junk contents, eliminates HTML tags, and partitions the remaining text into natural segments. It then uses the highest ratio of words over the number of lines in a segment combined with similarity between the segment and the title to identify main contents. We show that, through extensive experiments, qR ead achieves a 96.8% accuracy on Chinese web pages with an average extraction time of 13.20 milliseconds, and a 93.6% accuracy on English web pages with an average extraction time of 11.37 milliseconds, providing substantial improvements on accuracy over previous approaches and meeting the real-time extraction requirement. (More)

PDF ImageFull Text

Download
CC BY-NC-ND 4.0

Sign In Guest: Register as new SciTePress user now for free.

Sign In SciTePress user: please login.

PDF ImageMy Papers

You are not signed in, therefore limits apply to your IP address 100.24.125.162

In the current month:
Recent papers: 100 available of 100 total
2+ years older papers: 200 available of 200 total

Paper citation in several formats:
Wang, J. and Wang, J. (2015). qRead: A Fast and Accurate Article Extraction Method from Web Pages using Partition Features Optimizations.In Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1 KDIR: KDIR, (IC3K 2015) ISBN 978-989-758-158-8, ISSN 2184-3228, pages 364-371. DOI: 10.5220/0005613603640371

@conference{kdir15,
author={Jingwen Wang. and Jie Wang.},
title={qRead: A Fast and Accurate Article Extraction Method from Web Pages using Partition Features Optimizations},
booktitle={Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1 KDIR: KDIR, (IC3K 2015)},
year={2015},
pages={364-371},
publisher={SciTePress},
organization={INSTICC},
doi={10.5220/0005613603640371},
isbn={978-989-758-158-8},
}

TY - CONF

JO - Proceedings of the 7th International Joint Conference on Knowledge Discovery, Knowledge Engineering and Knowledge Management - Volume 1 KDIR: KDIR, (IC3K 2015)
TI - qRead: A Fast and Accurate Article Extraction Method from Web Pages using Partition Features Optimizations
SN - 978-989-758-158-8
AU - Wang, J.
AU - Wang, J.
PY - 2015
SP - 364
EP - 371
DO - 10.5220/0005613603640371

Login or register to post comments.

Comments on this Paper: Be the first to review this paper.