On Mean-Optimal Robust Linear Discriminant Analysis
Xiangyu Li, Hua Wang
TKDD - 2024
Linear discriminant analysis (LDA) is widely used for dimensionality reduction under supervised learning settings. Traditional LDA objective aims to minimize the ratio of the squared Euclidean distances that may not perform optimally on noisy datasets. Multiple robust LDA objectives have been proposed to address this problem, but their implementations have two major limitations. One is that their mean calculations use the squared ℓ-2 norm distance to center the data, which is not valid when the objective depends on other distance functions. The second problem is that there is no generalized optimization algorithm to solve different robust LDA objectives. In addition, most existing algorithms can only guarantee the solution to be locally optimal, rather than globally optimal. In this paper, we review multiple robust loss functions and propose a new and generalized robust objective for LDA. Besides, to better remove the mean value within data, our objective uses an optimal way to center the data through learning. As one important algorithmic contribution, we derive an efficient iterative algorithm to optimize the resulting non-smooth and non-convex objective function. We theoretically prove that our solution algorithm guarantees that both the objective and the solution sequences converge to globally optimal solutions at a sub-linear convergence rate. The results of comprehensive experimental evaluations demonstrate the effectiveness of our new method, achieving significant improvements compared to the other competing methods.
Links
- View publications from Xiangyu Li
- View publications from Hua Wang
- View publications in the project, An Intelligence-Driven Patient Care Approach to Reduce Medical Errors
- View publications in the project, Intelligent Prediction of Traffic Conditions via Integrated Data-Driven Crowdsourcing and Learning
- View publications in the project, Mining Brain Imaging Genomics Data for Improved Cognitive Health
- View publications in the project, Prediction of coronavirus infections and complications at the individual and the population levels from genomic, proteomic, clinical and behavioral data sources
- View publications researching Embeddings
- View publications researching Optimization
- View publications researching Robust Learning Models
- View publications applied to Computer Vision