Orthogonal least squares regression for feature extraction

详细信息    查看全文

• 作者：Haifeng Zhao^aAuthor Vitae ; Zheng Wang^aAuthor Vitae ; Feiping Nie^b ; ^{feipingnie@gmail.com" class="auth_mail" title="E-mail the corresponding author}Author Vitae
• 关键词：Feature extraction ; Least squares regression ; Orthogonal constraint ; Unbalanced orthogonal procrustes problem
• 刊名：Neurocomputing
• 出版年：2016
• 出版时间：5 December 2016
• 年：2016
• 卷：216
• 期：Complete
• 页码：200-207
• 全文大小：1252 K

文摘

In many data mining applications, dimensionality reduction is a primary technique to map high-dimensional data to a lower dimensional space. In order to preserve more local structure information, we propose a novel orthogonal least squares regression model for feature extraction in this paper. The main contributions of this paper are shown as follows: first, the new least squares regression method is constructed under the orthogonal constraint which can preserve more discriminant information in the subspace. Second, the optimization problem of classical least squares regression can be solved easily. However the proposed objective function is an unbalanced orthogonal procrustes problem, it is so difficult to obtain the solution that we present a novel iterative optimization algorithm to obtain the optimal solution. The last one, we also provide a proof of the convergence for our iterative algorithm. Additionally, experimental results show that we obtain a global optimal solution through our iterative algorithm even though the optimization problem is a non-convex problem. Both theoretical analysis and empirical studies demonstrate that our method can more effectively reduce the data dimensionality than conventional methods.