Author:
Shuichi Shinmura
Affiliation:
Dept. of Economics, Japan
Keyword(s):
Linear Discriminant Function (LDF), Logistic Regression, SVM, Minimum Number of Misclassifications (MNM), Revised IP-OLDF based on MNM Criterion, Linear Separable Data, K-Fold Cross-validation.
Related
Ontology
Subjects/Areas/Topics:
Artificial Intelligence
;
Knowledge Discovery and Information Retrieval
;
Knowledge-Based Systems
;
Mathematical Modeling
;
Mathematical Programming
;
Methodologies and Technologies
;
Operational Research
;
Optimization
;
Symbolic Systems
Abstract:
Fisher proposed a linear discriminant function (LDF) based on the maximization of the variance ratio. If data satisfies Fisher’s assumption, the same LDF is easily derived from a variance-covariance matrix. When the variance-covariance matrices of two classes are not the same, a quadratic discriminant function (QDF) can be used. These discriminant functions have three problems. First, none of them can correctly discriminate between xi on the discriminant hyperplane (the unresolved problem of discriminant analysis). Second, LDF and QDF cannot always recognize linear separable data, and the number of misclassifications (NM) made by these functions is usually higher than that of logistic regression. Third, these functions are not obtained if the value of some independent variable is constant, because the inverse matrix cannot be calculated. These facts mean that LDF and QDF should not be used for important discriminations. On the contrary, a revised Optimal Linear Discriminant Function
by Integer Programming (Revised IP-OLDF) based on the Minimum NM (MNM) criterion resolves these problems completely. In addition, the mean error rate of Revised IP-OLDF is often less than those of LDF, logistic regression, and Support Vector Machines (SVM) under 100-fold cross-validation.
(More)