手语视频中人脸检测系统的研究与实现
|
摘要
手语不仅包含手和手臂的动作,而且也包含表情、头部姿态和体势等动作。有实验表明,手语中只有手势而没有表情时,人所能理解的内容估计不超过60%。作为表情识别和头部姿态估计的前提,人脸检测具有重要的研究意义。本文针对手语视频中的人脸检测问题,结合Adaboost和肤色检测方法,改进手语视频中人脸检测的效果,并最终实现了一个高召回率和高准确率的人脸检测系统。具体的研究工作如下：
1、本文首先研究了Adaboost人脸检测算法。针对手语视频中存在的手部干扰问题,本文在训练分类器时将大量人手样本加入到负样本集中,训练了一个正面人脸分类器。针对手语视频中头部呈现多姿态情况,本文又训练了侧面人脸分类器。实验结果表明Adaboost人脸检测算法具有较高准确率但召回率较低。
2、本文研究了基于肤色的人脸检测算法。实验结果表明肤色人脸检测算法具有较高的召回率,但是其准确率和精度较低。
3、本文将肤色人脸检测算法和Adaboost人脸检测算法相结合。利用Adaboost人脸分类器扫描肤色分割后的图像,精确定位人脸。实验表明,本方法不但有较高召回率,而且还有较高的准确率和精度。提高了手语视频的人脸检测效果。
4、最后,本文设计与实现了结合Adaboost和肤色检测方法的人脸检测原型系统。
Sign language contains not only hand and arm movements, but also other actions such as expression, head and body. The experiment shows that the contents of people understand are not more than 60% if the sign language has only gestures not expressions during the communication. As the premise of expression recognition and head pose estimation, face detection has the important research value. The solutions on the face detection problems are put forward in the sign language video in the paper, which combines the color of skin face detection algorithm with Adaboost face detection algorithm, and finally a face detection system is realized with a high recall rate and a high accuracy rate. Specific researches are as follows:
1. Adaboost face detection algorithm is studied in this paper firstly. With the problems of the hand interference in sign language video, lots of hand samples have been put into the negative samples in training classifier, and a positive face classifier has been trained. Owing to the pose varied of head in sign language video, a profile face classifier is trained in this paper. Experimental results show that it has higher accuracy but lower recall rate.
2. The color of skin face detection algorithm is studied in this paper. Experimental results show that color of skin face detection algorithm has higher recall rate, but lower precision and accuracy.
3. To improve the face detection effect in the sign language video, the color of skin face detection algorithm and Adaboost face detection algorithm are combined. The skin-color segmentation image is scanned by Adaboost face classifier to locate face area accurately. Experimental results show that this method not only has high, and there's a recall rate high precision and accuracy.
4. At last, a face detection system, which combines the color of skin face detection algorithm with Adaboost face detection algorithm, has been designed.
1、本文首先研究了Adaboost人脸检测算法。针对手语视频中存在的手部干扰问题,本文在训练分类器时将大量人手样本加入到负样本集中,训练了一个正面人脸分类器。针对手语视频中头部呈现多姿态情况,本文又训练了侧面人脸分类器。实验结果表明Adaboost人脸检测算法具有较高准确率但召回率较低。
2、本文研究了基于肤色的人脸检测算法。实验结果表明肤色人脸检测算法具有较高的召回率,但是其准确率和精度较低。
3、本文将肤色人脸检测算法和Adaboost人脸检测算法相结合。利用Adaboost人脸分类器扫描肤色分割后的图像,精确定位人脸。实验表明,本方法不但有较高召回率,而且还有较高的准确率和精度。提高了手语视频的人脸检测效果。
4、最后,本文设计与实现了结合Adaboost和肤色检测方法的人脸检测原型系统。
Sign language contains not only hand and arm movements, but also other actions such as expression, head and body. The experiment shows that the contents of people understand are not more than 60% if the sign language has only gestures not expressions during the communication. As the premise of expression recognition and head pose estimation, face detection has the important research value. The solutions on the face detection problems are put forward in the sign language video in the paper, which combines the color of skin face detection algorithm with Adaboost face detection algorithm, and finally a face detection system is realized with a high recall rate and a high accuracy rate. Specific researches are as follows:
1. Adaboost face detection algorithm is studied in this paper firstly. With the problems of the hand interference in sign language video, lots of hand samples have been put into the negative samples in training classifier, and a positive face classifier has been trained. Owing to the pose varied of head in sign language video, a profile face classifier is trained in this paper. Experimental results show that it has higher accuracy but lower recall rate.
2. The color of skin face detection algorithm is studied in this paper. Experimental results show that color of skin face detection algorithm has higher recall rate, but lower precision and accuracy.
3. To improve the face detection effect in the sign language video, the color of skin face detection algorithm and Adaboost face detection algorithm are combined. The skin-color segmentation image is scanned by Adaboost face classifier to locate face area accurately. Experimental results show that this method not only has high, and there's a recall rate high precision and accuracy.
4. At last, a face detection system, which combines the color of skin face detection algorithm with Adaboost face detection algorithm, has been designed.
引文
[1]Mehrabian A. Communication without words. Psychology Today,1968,2(4):53-56.
[2]Yang M H, Kriegman D J and Ahuja N. Detecting faces in images:A survey. Pattern Analysis and Machine Intelligence,2002,24(1):34-58.
[3]姜峰,高文,王春立,姚鸿勋,赵德斌.非特定人手语识别进展及关键问题研究思路.软件学报,2007,18(3)：477-489.
[4]Bridges B, Metzger M. Deaf Tend Your:Non-manual Signals in American Sign Language,1996: 10-20.
[5]Bridges B, Metzger M. Phonological and Prosodic Layering of Nonmanuals in American Sign Language,2000:213-241.
[6]Ong S C W, Ranganath S. Automatic sign language analysis:a survey and the future beyond lexical meaning. IEEE TPAMI,2005,27(6):873-891.
[7]Bridges B, Metzger M. The Acquisition of Non-manual Morphology in Asl. G. Morgan & B. Woll(eds.)Current Developments in the Study of Signed Language Acquisition,2002:159-181.
[8]Hjelmas E, Low B K. Face detection:A survey. Computer Vision and Image Understanding, 2001:236-274.
[9]Zhang C, Zhang Z Y. A Survey of Recent Advances in Face Detection. Technical Report,2010.
[10]Kamruzzaman S M, Siddiqi F A, Shen C H, et al. Face Detection with Effective Feature Extraction. Computing Research Repository,2010:1-6.
[11]田捷,杨鑫.生物特征识别技术理论与应用.北京：电子工业出版社,2005.
[12]Peer P, Kovac J, Solina F. Human Skin Color Clustering for Face Detection. Eurocon 2003 International Conference on Computer as a Tool,2003:144-148.
[13]Yang G, Huang T S. Human Face Detection in Complex Background. Pattern Recognition, 199427(1):53-63.
[14]Lam K M.et al. Locating and extracting the eye in human face images. Pattern Recognition, 1996,29(5):771-779.
[15]Phung S L, Chai D and Bouzerdoum A. A Novel Skin Color Model in YCrCb Space and its Application to Human Face Detection. ICIP,2002,1:I-289-I-292.
[16]Kim H, Kang W, Shin J, Park S. Face Detection Using Template Matching and Ellipse Fitting. IEICE Trans.Inf.&Syst,2000:2008-2011.
[17]Yang M, Ahuja N. Face Detection and Gesture Recognition for Human Computer Interaction. Kluwer Academic Publishers Group,2001.
[18]Pakazad S K, Hajati F and Farahani S D. A Face Detection Framework based on Selected Face Components. Journal of Applied Science,2011,11(2):247-256.
[19]Craw I, Ellis H, Lishman J. Finding face features. Proc. Second European Conf. Computer Vision,1992:92-96.
[20]Tsukamoto A, Lee C W and Tsuji S. Detection and Tracking of Human Face with Synthesized Templates. Proc. First Asian Conf. Computer Vision,1993:183-186.
[21]Yullie A, Hallinan P. Feature exaction from faces using deformable templates. Intl. Journal of Computer Vision.1992,8(2):99-111.
[22]Kwon Y H, Lobo N V. Patten Matching as a Correlation on the Discrete Motion Group,1999: 22-35.
[23]Cootes T F et al. Active Shape Models-Their Training and Application. Computer Vision and Image Understanding,1995,61(1):38-59.
[24]Cootes T F, Edwards G J and Taylor C J. Active appearance models. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2001,23(6):681-685.
[25]Cootes T F, Twininga C J, Babalolaa K O et al. Diffeomorphic statistical shape models. Image and Vision Computing,2008.26(3):326-332.
[26]Sirohey S A. Human Face Segmentation and Identification. Technical Report CS-TR-3176, Univ.of Maryland,1993.
[27]Leung T K, Burl M C and Perona P. Finding Faces in Cluttered Scenes Using Random Labeled Graph Matching, Proc. Fifth IEEE Int'l Conf. Computer Vision,1995:637-644.
[28]Dai Y, Nakano Y. Face-Texture Model Based on SGLD and Its Application in Face Detection in a Color Scene. Pattern Recognition,1996,29(6):1007-1017.
[29]Chen D.S, Liu Z K. A Survey of Skin Color Detection. Chinese Journal of Computers,2006, 29(2):194-207.
[30]Phung S L, Bouzerdoum A, Chai D. Skin segmentation using color pixel classification: Analysis and comparison. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005, 27(1):149-154.
[31]Burgin W, Pantofaru C, William D S. Using Depth Information to Improve Face Detection. Proc.of Human-Robot Interaction (HRI),2011.
[32]Chai D, Ngan K N. Face segmentation using skin-color map in videophone applications. IEEE Transactions on Circuits and Systems for Video Technology,1999,9(4):551-564.
[33]Wang Y, Yuan B. A novel approach for human face detection from color images under complex background [J]. Pattern Recognition,2001,34(10):1983-1192.
[34]黄金海.彩色图像中的正面人脸检测：(硕士论文).西安：西安电子科技大学,2005,1.
[35]Terrillon J C, Shirazi M N, Fukamachi H and Akamatsu S. Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images.Proc.IEEE Int'l Conf.on Face and Gesture Recognition,2000:54-61.
[36]Wen H L, Tsun L C. Skin Color-Based Face Detection in Color Images,2006:758-761.
[37]Li Q, H Ji. Face Detection in complex background based on Gaussian models and neural networks. Signal Processing,2006 8th International conference on,2006:0-7803-9736-3.
[38]Phuong P N, Hyun J K. Color-based Face Detection using Combination of Modified Local Binary Patterns and embedded Hidden Markov Models. SICE-ICASE,2006. International Joint Conference,2006:5598-5603.
[39]Turk M. A Pentland Eigenfaces for Recognition [J]. Cognitive euroscience,1991,3(1):71-86.
[40]EI-Bakry H M, Qiang F Z. Fast Neural Implementation of PCA for Face Detection. Neural Networks,2006. UCNN 06. International Joint Conference on,2006:806-811.
[41]Hamdy A, Elmabdy M and Elsabrouty M. Face detection using PCA and skin-tone extraction for drowsy driver application. Information and Communications Technology,2007:135-137.
[42]Kamruzzaman S M, Siddiqi F A, Islam S M et al. Rotation Invariant Face Detection Using Wavelet, PCA and Radial Basis Function Networks. Computing Research Repository,2010:256-258.
[43]Yujie Z et al. Fuzzy Kernel Fisher Discriminant Algorithm with Application to Face Recognition. In Intelligent Control and Automation,2006.WCICA 2006. The 6th World Congress on,2006:9669-9672.
[44]Rowley H A., S.Baluja, and T.Kanade. Neural network-based face detection.IEEE Trans. Pattern Anal. Mach. Intell.20,1998:23-38.
[45]EI-Bakry H.M, H.Stoyan. Fast neural networks for sub-matrix detection. Circuits and Systems, 2004,5:764-767.
[46]Kumar T, Singh K V, Malik S. Artificial Neural Network in face Detection. International Journal of Computer Aplications,2011,14(3):5-7.
[47]Kaushal A, Raina J P S. Face Detection using Neural Network&Garbor Wavelet Transform. International Journal of Computer Science and Technology,2011,1(1):58-36.
[48]Osuna E, Preund R, Girosi F. Training support vector machines:an application to face detection. In:Proc. of Computer Vision and Pattern Recognition, Pucrto Rico,1997:130-136.
[49]Hotta K. View independent face detection based on combination of local and global kernels. International Conference on Computer Vision Systems,2007:598-601.
[50]陈茂林,戚飞虎.自组织隐马尔可夫模型的人脸检测研究[J].计算机学报,2002,25(11)：1165-1169.
[51]Viola Paul, Jones Michael. Rapid Object Detection using a Boosted Cascade of Simple Feature. In:Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA,2001.
[52]Li S Z, Zheng Q. FloatBoost learning and statistical face detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2004,26(9):1112-1123.
[53]Shirazi H, Vasconcelos N. Asymmetric boosting. In Proc. of ICML,2007.
[54]Yang M,Crenshaw J,Augustine B,Mareachen R, Wu Y. AdaBoost-based face detection for embedded systems,2010:1116-1125.
[55]Jang J S and Kim J H. Fast and robust face detection using evolutionary pruning. IEEE Trans. on Evolutionary Computation,2008,12(5):562-571.
[56]Schapire R E. The strength of weak learns ability. Machine Learning,1990,5(2):197-227.
[57]Valiant L G. A theory of the learnable. Communications of the ACM,1984,27(11):1134-1142.
[58]Kearns M, Valiant L G. Learning Boolean Formulae or finite automata is as hard as factoring. Harvard University Alien Computation Laboratory, Aug,1988.
[59]Kearns M, Valiant L G. Cryptongraphic Limitations on Learning Boolean Formulate and Finite Automata. Journal of the ACM,1994,41(1):67-95.
[60]Freund Y. Boosting a Weak Learning Algorithm by Majority. Information and Computation, 1995,141(2):256-285.
[61]Freund Y, Schapire R E. A Decision-theoretic Generalization of Online Learning and Application to Boosting. Journal of Computer and System Sciences,1997,55(1):119-139.
[62]Rainer Lienhart and Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. ICIP,2002:9000-9003.
[2]Yang M H, Kriegman D J and Ahuja N. Detecting faces in images:A survey. Pattern Analysis and Machine Intelligence,2002,24(1):34-58.
[3]姜峰,高文,王春立,姚鸿勋,赵德斌.非特定人手语识别进展及关键问题研究思路.软件学报,2007,18(3)：477-489.
[4]Bridges B, Metzger M. Deaf Tend Your:Non-manual Signals in American Sign Language,1996: 10-20.
[5]Bridges B, Metzger M. Phonological and Prosodic Layering of Nonmanuals in American Sign Language,2000:213-241.
[6]Ong S C W, Ranganath S. Automatic sign language analysis:a survey and the future beyond lexical meaning. IEEE TPAMI,2005,27(6):873-891.
[7]Bridges B, Metzger M. The Acquisition of Non-manual Morphology in Asl. G. Morgan & B. Woll(eds.)Current Developments in the Study of Signed Language Acquisition,2002:159-181.
[8]Hjelmas E, Low B K. Face detection:A survey. Computer Vision and Image Understanding, 2001:236-274.
[9]Zhang C, Zhang Z Y. A Survey of Recent Advances in Face Detection. Technical Report,2010.
[10]Kamruzzaman S M, Siddiqi F A, Shen C H, et al. Face Detection with Effective Feature Extraction. Computing Research Repository,2010:1-6.
[11]田捷,杨鑫.生物特征识别技术理论与应用.北京：电子工业出版社,2005.
[12]Peer P, Kovac J, Solina F. Human Skin Color Clustering for Face Detection. Eurocon 2003 International Conference on Computer as a Tool,2003:144-148.
[13]Yang G, Huang T S. Human Face Detection in Complex Background. Pattern Recognition, 199427(1):53-63.
[14]Lam K M.et al. Locating and extracting the eye in human face images. Pattern Recognition, 1996,29(5):771-779.
[15]Phung S L, Chai D and Bouzerdoum A. A Novel Skin Color Model in YCrCb Space and its Application to Human Face Detection. ICIP,2002,1:I-289-I-292.
[16]Kim H, Kang W, Shin J, Park S. Face Detection Using Template Matching and Ellipse Fitting. IEICE Trans.Inf.&Syst,2000:2008-2011.
[17]Yang M, Ahuja N. Face Detection and Gesture Recognition for Human Computer Interaction. Kluwer Academic Publishers Group,2001.
[18]Pakazad S K, Hajati F and Farahani S D. A Face Detection Framework based on Selected Face Components. Journal of Applied Science,2011,11(2):247-256.
[19]Craw I, Ellis H, Lishman J. Finding face features. Proc. Second European Conf. Computer Vision,1992:92-96.
[20]Tsukamoto A, Lee C W and Tsuji S. Detection and Tracking of Human Face with Synthesized Templates. Proc. First Asian Conf. Computer Vision,1993:183-186.
[21]Yullie A, Hallinan P. Feature exaction from faces using deformable templates. Intl. Journal of Computer Vision.1992,8(2):99-111.
[22]Kwon Y H, Lobo N V. Patten Matching as a Correlation on the Discrete Motion Group,1999: 22-35.
[23]Cootes T F et al. Active Shape Models-Their Training and Application. Computer Vision and Image Understanding,1995,61(1):38-59.
[24]Cootes T F, Edwards G J and Taylor C J. Active appearance models. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2001,23(6):681-685.
[25]Cootes T F, Twininga C J, Babalolaa K O et al. Diffeomorphic statistical shape models. Image and Vision Computing,2008.26(3):326-332.
[26]Sirohey S A. Human Face Segmentation and Identification. Technical Report CS-TR-3176, Univ.of Maryland,1993.
[27]Leung T K, Burl M C and Perona P. Finding Faces in Cluttered Scenes Using Random Labeled Graph Matching, Proc. Fifth IEEE Int'l Conf. Computer Vision,1995:637-644.
[28]Dai Y, Nakano Y. Face-Texture Model Based on SGLD and Its Application in Face Detection in a Color Scene. Pattern Recognition,1996,29(6):1007-1017.
[29]Chen D.S, Liu Z K. A Survey of Skin Color Detection. Chinese Journal of Computers,2006, 29(2):194-207.
[30]Phung S L, Bouzerdoum A, Chai D. Skin segmentation using color pixel classification: Analysis and comparison. IEEE Transactions on Pattern Analysis and Machine Intelligence,2005, 27(1):149-154.
[31]Burgin W, Pantofaru C, William D S. Using Depth Information to Improve Face Detection. Proc.of Human-Robot Interaction (HRI),2011.
[32]Chai D, Ngan K N. Face segmentation using skin-color map in videophone applications. IEEE Transactions on Circuits and Systems for Video Technology,1999,9(4):551-564.
[33]Wang Y, Yuan B. A novel approach for human face detection from color images under complex background [J]. Pattern Recognition,2001,34(10):1983-1192.
[34]黄金海.彩色图像中的正面人脸检测：(硕士论文).西安：西安电子科技大学,2005,1.
[35]Terrillon J C, Shirazi M N, Fukamachi H and Akamatsu S. Comparative performance of different skin chrominance models and chrominance spaces for the automatic detection of human faces in color images.Proc.IEEE Int'l Conf.on Face and Gesture Recognition,2000:54-61.
[36]Wen H L, Tsun L C. Skin Color-Based Face Detection in Color Images,2006:758-761.
[37]Li Q, H Ji. Face Detection in complex background based on Gaussian models and neural networks. Signal Processing,2006 8th International conference on,2006:0-7803-9736-3.
[38]Phuong P N, Hyun J K. Color-based Face Detection using Combination of Modified Local Binary Patterns and embedded Hidden Markov Models. SICE-ICASE,2006. International Joint Conference,2006:5598-5603.
[39]Turk M. A Pentland Eigenfaces for Recognition [J]. Cognitive euroscience,1991,3(1):71-86.
[40]EI-Bakry H M, Qiang F Z. Fast Neural Implementation of PCA for Face Detection. Neural Networks,2006. UCNN 06. International Joint Conference on,2006:806-811.
[41]Hamdy A, Elmabdy M and Elsabrouty M. Face detection using PCA and skin-tone extraction for drowsy driver application. Information and Communications Technology,2007:135-137.
[42]Kamruzzaman S M, Siddiqi F A, Islam S M et al. Rotation Invariant Face Detection Using Wavelet, PCA and Radial Basis Function Networks. Computing Research Repository,2010:256-258.
[43]Yujie Z et al. Fuzzy Kernel Fisher Discriminant Algorithm with Application to Face Recognition. In Intelligent Control and Automation,2006.WCICA 2006. The 6th World Congress on,2006:9669-9672.
[44]Rowley H A., S.Baluja, and T.Kanade. Neural network-based face detection.IEEE Trans. Pattern Anal. Mach. Intell.20,1998:23-38.
[45]EI-Bakry H.M, H.Stoyan. Fast neural networks for sub-matrix detection. Circuits and Systems, 2004,5:764-767.
[46]Kumar T, Singh K V, Malik S. Artificial Neural Network in face Detection. International Journal of Computer Aplications,2011,14(3):5-7.
[47]Kaushal A, Raina J P S. Face Detection using Neural Network&Garbor Wavelet Transform. International Journal of Computer Science and Technology,2011,1(1):58-36.
[48]Osuna E, Preund R, Girosi F. Training support vector machines:an application to face detection. In:Proc. of Computer Vision and Pattern Recognition, Pucrto Rico,1997:130-136.
[49]Hotta K. View independent face detection based on combination of local and global kernels. International Conference on Computer Vision Systems,2007:598-601.
[50]陈茂林,戚飞虎.自组织隐马尔可夫模型的人脸检测研究[J].计算机学报,2002,25(11)：1165-1169.
[51]Viola Paul, Jones Michael. Rapid Object Detection using a Boosted Cascade of Simple Feature. In:Proc. IEEE Conf. on Computer Vision and Pattern Recognition, Kauai, Hawaii, USA,2001.
[52]Li S Z, Zheng Q. FloatBoost learning and statistical face detection. Pattern Analysis and Machine Intelligence, IEEE Transactions on,2004,26(9):1112-1123.
[53]Shirazi H, Vasconcelos N. Asymmetric boosting. In Proc. of ICML,2007.
[54]Yang M,Crenshaw J,Augustine B,Mareachen R, Wu Y. AdaBoost-based face detection for embedded systems,2010:1116-1125.
[55]Jang J S and Kim J H. Fast and robust face detection using evolutionary pruning. IEEE Trans. on Evolutionary Computation,2008,12(5):562-571.
[56]Schapire R E. The strength of weak learns ability. Machine Learning,1990,5(2):197-227.
[57]Valiant L G. A theory of the learnable. Communications of the ACM,1984,27(11):1134-1142.
[58]Kearns M, Valiant L G. Learning Boolean Formulae or finite automata is as hard as factoring. Harvard University Alien Computation Laboratory, Aug,1988.
[59]Kearns M, Valiant L G. Cryptongraphic Limitations on Learning Boolean Formulate and Finite Automata. Journal of the ACM,1994,41(1):67-95.
[60]Freund Y. Boosting a Weak Learning Algorithm by Majority. Information and Computation, 1995,141(2):256-285.
[61]Freund Y, Schapire R E. A Decision-theoretic Generalization of Online Learning and Application to Boosting. Journal of Computer and System Sciences,1997,55(1):119-139.
[62]Rainer Lienhart and Jochen Maydt. An Extended Set of Haar-like Features for Rapid Object Detection. ICIP,2002:9000-9003.
© 2004-2018 中国地质图书馆版权所有 京ICP备05064691号 京公网安备11010802017129号 地址：北京市海淀区学院路29号 邮编：100083 电话：办公室：(+86 10)66554848；文献借阅、咨询服务、科技查新：66554700 |