Authors:
Kengo Ooi
and
Takashi Ninomiya
Affiliation:
Ehime University, Japan
Keyword(s):
Machine Learning, Feature Selection, Grafting, ℓ1-Regularized Logistic Regression.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Computational Intelligence
;
Data Mining
;
Databases and Information Systems Integration
;
Enterprise Information Systems
;
Evolutionary Computing
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Machine Learning
;
Sensor Networks
;
Signal Processing
;
Soft Computing
;
Symbolic Systems
Abstract:
Finding features for classifiers is one of the most important concerns in various fields, such as information
retrieval, speech recognition, bio-informatics and natural language processing, for improving classifier prediction
performance. Online grafting is one solution for finding useful features from an extremely large feature
set. Given a sequence of features, online grafting selects or discards each feature in the sequence of features
one at a time. Online grafting is preferable in that it incrementally selects features, and it is defined as an
optimization problem based on ℓ1-regularized logistic regression. However, its learning is inefficient due to
frequent parameter optimization. We propose two improved methods, in terms of efficiency, for online grafting
that approximate original online grafting by testing multiple features simultaneously. The experiments
have shown that our methods significantly improved efficiency of online grafting. Though our methods are
approximation t
echniques, deterioration of prediction performance was negligibly small.
(More)