基于相似度的神经网络多源迁移学习算法
|
摘要
为了解决迁移学习中的"负迁移"问题,提出了基于相似度的神经网络多源迁移学习算法。该算法是以经典的BP神经网络模型为基分类器,利用梯度下降法对各个源领域与目标域之间的相似度进行学习和优化,把各个源领域的网络权重参数信息按照与目标域之间的相似程度迁移到目标域中,提高机器学习算法在目标域的分类性能。在UCI数据的Letter-recognition数据集以及20Newsgroups文本数据集上进行实验。实验结果表明了MTL-SNN算法比传统的多源迁移学习算法以及BP神经网络算法在分类准确率上有所提升,因此MTL-SNN算法有效地解决了"负迁移"问题。
In order to solve the problem of "negative transfer"in transfer learning,a similarity-based neural network multi-source transfer learning algorithm is proposed. The algorithm uses the gradient descent method to learn and optimize the similarity between each source and target domain,which is based on the classical BP neural network model. According to the degree of similarity with the target domain,the network weight parameter information of each source domain is migrated to the target domain,and the classification performance of the machine learning algorithm in the target domain is improved. The test was carried out on the Letter-recognition data set of UCI data and the 20 Newsgroups text data set. The experimental results show that the MTL-SNN algorithm has improved classification accuracy compared with the traditional multi-source migration learning algorithm and BP neural network algorithm. Therefore,the MTL-SNN algorithm effectively solves the "negative transfer"problem.
In order to solve the problem of "negative transfer"in transfer learning,a similarity-based neural network multi-source transfer learning algorithm is proposed. The algorithm uses the gradient descent method to learn and optimize the similarity between each source and target domain,which is based on the classical BP neural network model. According to the degree of similarity with the target domain,the network weight parameter information of each source domain is migrated to the target domain,and the classification performance of the machine learning algorithm in the target domain is improved. The test was carried out on the Letter-recognition data set of UCI data and the 20 Newsgroups text data set. The experimental results show that the MTL-SNN algorithm has improved classification accuracy compared with the traditional multi-source migration learning algorithm and BP neural network algorithm. Therefore,the MTL-SNN algorithm effectively solves the "negative transfer"problem.
引文
1 Dai W,Xue G R,Yang Q,et al.Co-clustering based classification for out-of-domain documents[C]//ACM Sigkdd International Conference on Knowledge Discovery&Data Mining.New York:ACMPress,2007:210-219
2 Dai W,Xue G R,Yang Q,et al.Transferring naive bayes classifiers for text classification[C]//Proceedings of the Twenty-Second AAAIConference on Artificial Intelligence,July 22-26,2007,Vancouver,British Columbia,Canada.Palo Alto,California:AAAI Press,2007:540-545
3 Xing D,Dai W,Xue G,et al.Bridged refinement for transfer learning[C]//European Conference on Principles&Practice of Knowledge Discovery in Databases.Berlin:Springer-Verlag,2007:324-335
4庄福振,罗平,何清,等.迁移学习进展[J].软件学报,2015,26(1):26-39Zhuang Fuzhen,Luo Ping,He Qing,et al.Survey on transfer learning research[J].Journal of Software,2015,26(1):26-39
5孟佳娜.迁移学习在文本分类中的应用研究[D].大连:大连理工大学,2011Meng Jiana.Application research of migration learning in text classification[D].Da'lian:Dalian University of Technology,2011
6 Ceci M,Appice A,Barile N,et al.Transductive learning from relational data[C]//International Conference on Machine Learning&Data Mining in Pattern Recognition.Berlin:Springer-Verlag,2007:324-338
7 Yao Y,Doretto G.Boosting for transfer learning with multiple sources[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE,2010:1855-1862
8 Dai W,Yang Q,Xue G R,et al.Boosting for transfer learning[C]//International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers,2007:193-200
9 Eaton E,Desjardins M.Selective transfer between learning tasks using task-based boosting[C]//AAAI Conference on Artificial Intelligence.Palo Alto,California:AAAI Press,2011:1-8
10 Pan S J,Ni X,Sun J T,et al.Cross-domain sentiment classification via spectral feature alignment[C]//International Conference on World Wide Web.Raleigh,North Carolina:ACM Press,2010:166-168
11 Xu Z,Sun S.Multi-view Transfer Learning with Adaboost[C]//IEEE International Conference on Tools with Artificial Intelligence.Boca Raton,FL:IEEE,2011:309-402
12卞则康,王士同.基于相似度学习的多源迁移算法[J].控制与决策,2017,32(11):1941-1948Bian Zekang,Wang Shitong.Multi-source migration algorithm based on similarity learning[J].Control and Decision,2017,32(11):1941-1948
13张倩,李明,王雪松,等.一种面向多源领域的实例迁移学习[J].自动化学报,2014,40(6):1176-1183Zhang Qian,Li Ming,Wang Xuesong,et al.An example migration learning for multi-source domain[J].Acta Automatica Sinica,2014,40(6):1176-1183
14许夙晖,慕晓冬,柴栋,等.基于极限学习机参数迁移的域适应算法[J].自动化学报,2018,44(2):311-317Xu Yuhui,Mu Xiaodong,Chai Dong,et al.Domain adaptation algorithm based on parameter migration of extreme learning machine[J].Acta Automatica Sinica,2018,44(2):311-317
15唐诗淇,文益民,秦一休.一种基于局部分类精度的多源在线迁移学习算法[J].软件学报,2017,28(11):2940-2960Tang Shiqi,Wen Yimin,Qin Yixiu.A multi-source online migration learning algorithm based on local classification accuracy[J].Journal of Software,2017,28(11):2940-2960
16洪佳明,印鉴,黄云,等.Tr SVM:一种基于领域相似性的迁移学习算法[J].计算机研究与发展,2011,48(10):1823-1830Hong Jiaming,Yin Jian,Huang Yun,et al.Tr SVM:a migration learning algorithm based on domain similarity[J].Journal of Computer Research and Development,2011,48(10):1823-1830
2 Dai W,Xue G R,Yang Q,et al.Transferring naive bayes classifiers for text classification[C]//Proceedings of the Twenty-Second AAAIConference on Artificial Intelligence,July 22-26,2007,Vancouver,British Columbia,Canada.Palo Alto,California:AAAI Press,2007:540-545
3 Xing D,Dai W,Xue G,et al.Bridged refinement for transfer learning[C]//European Conference on Principles&Practice of Knowledge Discovery in Databases.Berlin:Springer-Verlag,2007:324-335
4庄福振,罗平,何清,等.迁移学习进展[J].软件学报,2015,26(1):26-39Zhuang Fuzhen,Luo Ping,He Qing,et al.Survey on transfer learning research[J].Journal of Software,2015,26(1):26-39
5孟佳娜.迁移学习在文本分类中的应用研究[D].大连:大连理工大学,2011Meng Jiana.Application research of migration learning in text classification[D].Da'lian:Dalian University of Technology,2011
6 Ceci M,Appice A,Barile N,et al.Transductive learning from relational data[C]//International Conference on Machine Learning&Data Mining in Pattern Recognition.Berlin:Springer-Verlag,2007:324-338
7 Yao Y,Doretto G.Boosting for transfer learning with multiple sources[C]//Proceedings of the 2010 IEEE Computer Society Conference on Computer Vision and Pattern Recognition.San Francisco:IEEE,2010:1855-1862
8 Dai W,Yang Q,Xue G R,et al.Boosting for transfer learning[C]//International Conference on Machine Learning.San Francisco:Morgan Kaufmann Publishers,2007:193-200
9 Eaton E,Desjardins M.Selective transfer between learning tasks using task-based boosting[C]//AAAI Conference on Artificial Intelligence.Palo Alto,California:AAAI Press,2011:1-8
10 Pan S J,Ni X,Sun J T,et al.Cross-domain sentiment classification via spectral feature alignment[C]//International Conference on World Wide Web.Raleigh,North Carolina:ACM Press,2010:166-168
11 Xu Z,Sun S.Multi-view Transfer Learning with Adaboost[C]//IEEE International Conference on Tools with Artificial Intelligence.Boca Raton,FL:IEEE,2011:309-402
12卞则康,王士同.基于相似度学习的多源迁移算法[J].控制与决策,2017,32(11):1941-1948Bian Zekang,Wang Shitong.Multi-source migration algorithm based on similarity learning[J].Control and Decision,2017,32(11):1941-1948
13张倩,李明,王雪松,等.一种面向多源领域的实例迁移学习[J].自动化学报,2014,40(6):1176-1183Zhang Qian,Li Ming,Wang Xuesong,et al.An example migration learning for multi-source domain[J].Acta Automatica Sinica,2014,40(6):1176-1183
14许夙晖,慕晓冬,柴栋,等.基于极限学习机参数迁移的域适应算法[J].自动化学报,2018,44(2):311-317Xu Yuhui,Mu Xiaodong,Chai Dong,et al.Domain adaptation algorithm based on parameter migration of extreme learning machine[J].Acta Automatica Sinica,2018,44(2):311-317
15唐诗淇,文益民,秦一休.一种基于局部分类精度的多源在线迁移学习算法[J].软件学报,2017,28(11):2940-2960Tang Shiqi,Wen Yimin,Qin Yixiu.A multi-source online migration learning algorithm based on local classification accuracy[J].Journal of Software,2017,28(11):2940-2960
16洪佳明,印鉴,黄云,等.Tr SVM:一种基于领域相似性的迁移学习算法[J].计算机研究与发展,2011,48(10):1823-1830Hong Jiaming,Yin Jian,Huang Yun,et al.Tr SVM:a migration learning algorithm based on domain similarity[J].Journal of Computer Research and Development,2011,48(10):1823-1830