An experimental study on stability and generalization of extreme learning machines

详细信息    查看全文

• 作者：Aimin Fu ; Chunru Dong ; Laisheng Wang
• 关键词：Extreme learning machine ; Generalization capability ; Uncertainty ; Fuzziness
• 刊名：International Journal of Machine Learning and Cybernetics
• 出版年：2015
• 出版时间：February 2015
• 年：2015
• 卷：6
• 期：1
• 页码：129-135
• 全文大小：235 KB
• 参考文献：1. Schmidt WF, Kraaijveld MA, Duin PW (1992) Feed-forward neural networks with random weights. Proceedings of 11th IAPR International Conference on Pattern Recognition Methodology and Systems, 2: 1-
  2. Igelnik B, Pao YH (1995) Stochastic choice of basis functions in adaptive function approximation and the functional-link net. IEEE Trans Neural Netw 6(6):1320-329 CrossRef
  3. Li JY, Chow WS, Igelnik B, Pao YH (1997) Comments on “Stochastic choice of basis functions in adaptive function approximation and the functional-link net- IEEE Trans Neural Netw 8(2):452-54
  4. Broomhead DS, Lowe D (1988) Multivariable functional interpolation and adaptive networks. Complex Syst 2:321-55
  5. Lowe D (1989) Adaptive radial basis function nonlinearities, and the problem of generalization. In: Proceedings of the 1st IEEE Conference on Artificial Neural Networks, London, UK, pp 171-75
  6. Huang GB, Zhu QY, Siew CK (2004) Extreme learning machine: a new learning scheme of feed-forward neural networks. In: Proceedings of 2004 IEEE International Joint Conference on Neural Network, 2: 985-90
  7. Huang GB, Zhu QY, Siew CK (2006) Extreme learning machine: theory and applications. Neurocomputing 70(1):489-01 CrossRef
  8. Wang Y, Cao F, Yuan Y (2001) A study on effectiveness of extreme learning machine. Neurocomputing 74(16):2483-490 CrossRef
  9. Wang X, Chen A, Feng H (2001) Upper integral network with extreme learning mechanism. Neurocomputing 74(16):2520-525 CrossRef
  10. Wang XZ, Shao QY, Qing M, Zhai JH (2013) Architecture selection for networks trained with extreme learning machine using localized generalization error model. Neurocomputing 102:3- CrossRef
  11. Wang R, Kwong S, Wang XZ (2012) A study on random weights between input and hidden layers in extreme learning machine. Soft Comput 16(9):1465-475 CrossRef
  12. Wu J, Wang ST, Chung FL (2011) Positive and negative fuzzy rule system, extreme learning machine and image classification. Int J Mach Learn Cybernet 2(4):261-71 CrossRef
  13. Chacko BP, Krishnan VRV, Raju G, Anto PB (2012) Handwritten character recognition using wavelet energy and extreme learning machine. Int J Mach Learn Cybernet 3(2):149-61 CrossRef
  14. Huang GB, Wang DH, Lan Y (2011) Extreme learning machines: a survey. Int J Mach Learn Cybernet 2(2):107-22 CrossRef
  15. Zhai JH, Xu HY, Wang XZ (2012) Dynamic ensemble extreme learning machine based on sample entropy. Soft Comput 16(9):1493-502 CrossRef
  16. Xue XW, Yao M, Wu ZH, Yang JH (2013) Genetic ensemble of extreme learning machine. Neurocomputing, ISSN 0925-2312. http://dx.doi.org/10.1016/j.neucom.2013.09.042
  17. Zhai J, Xu H, Li Y (2013) Fusion of extreme learning machine with fuzzy integral. Int J Uncertain Fuzziness Knowl Based Systems 21(2):23-4 CrossRef
  18. Yu Q, Heeswijk MV, Miche Y, Nian R, He B, Séverin E, Lendasse A (2013) Ensemble delta test-extreme learning machine (DT-ELM) for regression. Neurocomputing, ISSN 0925-2312. http://dx.doi.org/10.1016/j.neucom.2013.08.041
  19. Courrieu P (2005) Fast computation of moore-penrose inverse matrices. Neural Inf Process Lett Rev 8(2):25-9
  20. Deluca A, Termini S (1972) A definition of non-probabilistic entropy in the setting of fuzzy sets theory. Inf Control 20:301-12 CrossRef
  21. Frank A, Asuncion
• 作者单位：Aimin Fu (1)
  Chunru Dong (2)
  Laisheng Wang (1)
  
  1. College of Science, China Agricultural University, Beijing, 100083, China
  2. College of Mathematics and Computer Science, Hebei University, Baoding, 071002, China
  
• 刊物类别：Engineering
• 刊物主题：Artificial Intelligence and Robotics
  Statistical Physics, Dynamical Systems and Complexity
  Computational Intelligence
  Control , Robotics, Mechatronics
  
• 出版者：Springer Berlin / Heidelberg
• ISSN：1868-808X

文摘

This paper gives an experimental study on the stability of an extreme learning machine (ELM) and its generalization capability. Focusing on the relationship between uncertainty of an ELM’s output on the training set and the ELM’s generalization capability, the experiments show some new results in the viewpoint of classical pattern recognition. The study provides some useful guidelines to choose a fraction of ELMs with better generalization from an ensemble for classification problems.