Fast rates by transferring from auxiliary hypotheses
详细信息 查看全文
- 作者：Ilja Kuzborskij ; Francesco Orabona
- 关键词：Fast ; rate generalization bounds ; Transfer learning ; Domain adaptation ; Rademacher complexity ; Smooth loss functions ; Strongly ; convex regularizers
- 刊名：Machine Learning
- 出版年：2017
- 出版时间：February 2017
- 年：2017
- 卷：106
- 期：2
- 页码：171-195
- 全文大小：
- 刊物类别：Computer Science
- 刊物主题：Artificial Intelligence (incl. Robotics); Control, Robotics, Mechatronics; Computing Methodologies; Simulation and Modeling; Language Translation and Linguistics;
- 出版者：Springer US
- ISSN：1573-0565
- 卷排序：106
文摘
In this work we consider the learning setting where, in addition to the training set, the learner receives a collection of auxiliary hypotheses originating from other tasks. We focus on a broad class of ERM-based linear algorithms that can be instantiated with any non-negative smooth loss function and any strongly convex regularizer. We establish generalization and excess risk bounds, showing that, if the algorithm is fed with a good combination of source hypotheses, generalization happens at the fast rate \(\mathcal {O}(1/m)\) instead of the usual \(\mathcal {O}(1/\sqrt{m})\). On the other hand, if the source hypotheses combination is a misfit for the target task, we recover the usual learning rate. As a byproduct of our study, we also prove a new bound on the Rademacher complexity of the smooth loss class under weaker assumptions compared to previous works.