摘要
随着运动感知技术的发展和普及,用户可以突破空间的限制,以身体运动的方式进行人机交互。本文综合考虑用户的外在行为运动和内在情感,探讨面向个性化学习的体感计算(Affective and Behavioral Computing,ABC)的模型与方法。
本文工作主要包括以下几个方面：
1.本文提出了体感计算的研究,探讨了基于体验性认知的个性化电子学习方式,并且分析了面向儿童的艺术创造体验,在体感虚拟化身的帮助下,用户可以通过身体运动,促进情感心智的全面发展；
3.本文归纳分析了体感计算中的交互模型和体感虚拟化身交互,探讨了运动感知识别的关键技术以及相关算法；
4.设计实现了基于运动传感的水墨画书空临摹的实验,实验的结果表明,基于运动感知的水墨画临摹对比传统的笔墨临摹方法,更加寓教于乐,可以有效培养儿童的水墨画艺术学习的兴趣,加深艺术感知体验。
With the development and widespread use of motion-sensing technology, users can interact with computer through body movement physically and break through the the 2D limitation. This paper analyzes the hybrid of external motion and internal emotion, then discusses Affective and Behavioral Computing (ABC) model and methods for personalized E-Learning.
In particular, the main contents focused by this thesis are as follows:
1. This thesis provides Affective and Behavioral Computing, and discusses the personalized E-learning style based on cognitive experience, then analyzes the creative art experience for children which can promote the all-around intellectual development of user by physical movement with the help of virtual perceptual avatar.
2. This thesis analyzes the Affective and Behavioral Computing interaction model、the Affective and Behavioral Computing avatar, and discusses the key technology of motion recognition and related algorithms.
3. This thesis proposes an interaction painting by Shu-Kong Facsimile based on motion sensors. Experiment results show that our approach can inspire the interests of the children on learning ink painting more effectively than the traditional pen-based interface, teach through lively activities and make a deeper impression.
引文
[1]董士海,王衡.人机交互[M].北京：北京大学出版社,2004：1-65.
[2]Brown.J.R.Pogramming the User Interface:Principles and Examples[M].John Wiley and Sons Inc.,1989.
[3]普里斯等,刘晓晖译.交互设计：超越人机交互[M].北京：电子工业出版社,2003：55-102.
[4]罗尔夫.俞文伟译.身体的智能：智能科学新视角[M].北京：科学出版社,2009.
[5]蔡自兴.人工智能基础[M].北京：高等教育出版社,2005：6-8.
[6]张顺.基于运动的感知用户界面模型及其应用[M].浙江大学博士学位论文,2009.
[7]Roth, I.An introduction to object perception.In I.Roth and J.B.Frisby (eds.) perception and representation:A Cognitive Approach[M].Milton Keynes:Open University.1986.
[8]Wade, N.J.,& Swanston,M.T.Visual Perception:An Introduction (2nd ED.) [M].Hove, UK:Psychology Press,2001.
[9]李鸣华.人工智能及其教育应用[M].北京：科学出版社,2008：3-44.
[10]胡馨月,梁秀波,樊儒昆,谭谞,邹文,耿卫东.基于运动传感的书空临摹交互方式[J].计算机辅助设计与图形学学报,2010,(22)：1734-1740.
[11]刘伟,袁修干.人机交互设计与评价[M].北京：科学出版社.2008：23-173.
[12]梁宁建.当代认知心理学[M].上海：上海教育出版社,2003.
[13]布丽姬特·贾艾斯.黄国强,林晓兰,徐愿译.认知心理学[M].哈尔滨：黑龙江科学技术出版社,2007：3.
[14]张耀翔.感觉心理[M].北京：工人出版社,1987：194-207.
[15]霍华德·加德纳.沈致隆译.多元智能[M].北京：新华出版社,2004：3.
[16]理查德·威廉姆斯.邓晓娥译.原动画基础教程[M].北京：中国青年出版社,2006：214-219.(Richard Williams.Imogen Sutton.The Animator's Survival Kit [M]Faber and Faber Limited,2001.)
[17]武秀波.认知科学概论[M].北京：科学出版社,2006.
[18]赵广信.人体动作语言[M].西安：西北大学出版社,1993：62-68.
[19]洛雷塔·A·马兰德罗,拉里·巴克,孟小平,单年惠,朱美德译.非语言交流[M].北京：北京语言学院出版社,1991：104-135.
[20]戴斯蒙德·莫里斯.梁豪译.身体语言[M].上海：上海三联书店,2003：71-72.
[21]戴维·兰伯特.叶婷译.身体语言使用手册[M].哈尔滨：黑龙江科学技术出版社,2008：16-22.
[22]H.A.Simon. Motivational and emotional controls of cognition.IN MODELS OF Thoughts [M].Yale university press, new haven,1967:29-38.
[23]Marvin Minsky.Society of Mind:a response to four reviews [J].Artificial Intelligence48,1991:.371-396.
[24]R.W.Picard.T.P.Minka.Vision texture for annotation. [J] Journal of Multimedia Systems.1995,3:3-14.
[25]王志良.人工心理[M].北京：机械工业出版社,2007：64-119.
[26]迪克斯.蔡利栋,方思行,周继鹏,张庆丰译.人机交互：第3版[M].北京：电子工业出版社,2006：28-29.(Alan Dix, Janet E.Finlay, Gregory D.Human-Computer Interaction, Third Edition [M].Pearson Education Limited,2004.)
[27]Alan Dix, Deconstructing experience-pulling crackers apart[C]//Funology: From Usability to Enjoyment.Kluwer,2003.
[28]Alan Dix, Managing the ecology of interaction [C]//In Proceedings of Tamodia 2002-First International Workshop on Task Models and User Interface Design, Bucharest, Romania,2005.
[29]Plutchik, Robert.Emotion, a psychoevolutionary synthesis [M].Harper & Row.1980.
[30]Kellerman, Jonathan. Psychological Effects of Illness in Adolescence. I. Anxiety, Self-Esteem, and Perception of Control[J] Journal of Pediatrics,1980,7: 126-131.
[31]R. Plutchik,H.Kellerman. Emotion [C]//In R.Plutchchik and H.Kellerman, editors, Emotion Theory, Research, and Experience.Volumne 1-5.Academic Press, Series of selected papers.1980-1990.
[32]Adrian Bullock and Bjorn Gamback. Evaluating affective interaction in gaming [C]//Adjunct Proceedings of HCI International (HCI'03), Crete,2003.
[33]Petra Fagerberg, Anna Stahl, Kristina Hook. [C]//Designing Gestures for Affective Input:An Analysis of Shape, Effort and Valence,2003:57-65.
[34]Ryberg, K. Levande farger [M]. ICA Bokforlag, Vasteras, Sweden.1991.
[35]Bordwell, D. and Thompson, K. Film Art, An Introduction, Sixth Edition [M].McGraw-Hill, New York, USA.2001.
[36]Bresin, R. and Friberg, A. Emotional Colouring of Computer-controlled Music Performances, [J].Computer Music Journal,2000,4(24):44-63.
[37]J.A. Russell.A circumplex model of affect [J] Journal of personality and social psychology,1980:1161-1178.
[38]J.A. Russell.G.A.Pratt.A description of the affective quality attributed to environments [J] Journal of personality and social psychology,1980:311-322.
[39]Ginevra Castellano, Iolanda Leite, Andr'e Pereira, Carlos Martinho, Ana Paiva, Peter W, McOwan. Affect Recognition for Interactive Companions:Challenges and Design in Real World Scenarios[J]. IN workshopNo.Environment (AFFINE) ACM International Conference on Multimodal Interface (ICMI) ACM,2008.
[40]Loic Kessous, Ginevra Castellano, George Caridakis. Multimodal emotion recognition in speech-based interaction using facial expression, body gesture and acoustic analysis[J] J Multimodal User Interfaces,2010,3:33-48.
[41]Ginevra Castellano, Santiago D. Villalba, and Antonio Camurri. Recognising Human Emotions from Body Movement and Gesture Dynamics[C]//ACII 2007.Verlag Berlin Heidelberg 2007:71-82.
[42]Risberg, J., Klein, J., Fernandez, R. and Picard, R.W. Frustrating the User on Purpose:Using Biosignals in a Pilot Study to Detect the User's Emotional State[C]//CHI'98, Los Angeles, CA.1998.
[43]Picard, R. W, and Healey, J. Affective Wearables, Personal Technologies [J].Vol 1, No.4,1997:231-240.
[44]Picard, R. W, and Healey, J., Affective Wearables [C]//In Proceedings of the First International Symposium on Wearable Computers, Cambridge, MA,1997.
[45]Ashish.Kapoor.Rosalind.W. Picard. Multimodal Affect Recognition in Learning Environments[C]//In Proceedings of MM'05, Singapore,2005.
[46]Picard.R.W. Does Hal cry digital tears? [J].Emotion and computers.MIT Press, Cambridge.1997.
[47]Healey, J., Dabek, F. and Picard, R.W. A New Affect-Perceiving Interface and its Application to Personalized Music Selection[C]//In Proc. Workshop on Perceptual User Interfaces, San Fransisco, CA.1998.
[48]Healey, J., Seger, J., and Picard, R.W. Quantifying Driver Stress:Developing a System for Collecting and Processing Bio-Metric Signals in Natural Situations[C]//In Proc. Rocky-Mt. Bio-Eng. Symp Boulder,1999.
[49]RosariaConte, Cristiano Castelfranchi. Cognitive and social action[M].UCL Press.1995.
[50]Davies, E. Beyond Dance, Laban's Legacy of Movements Analysis[M]. Brechin Books ltd., London, UK.2001.
[51]P. Ekman.An augument for basic emotions[J].Cognition and Emotion,1992,6 (3/4):169-200.
[52]P. Ekman.Are thre basic emotions? [J].Psychological Review,1992,99 (3): 550-553.
[53]P. Ekman. Commentaries:Duchenne and facial expression of emotion. [M].In R.A.Cuthbertson, editor, The Mechanism of Human Facial Expression, Cambridge University Press.1990:270-284.
[54]罗莎琳德·皮卡德.情感计算[M].罗森林.北京：北京理工大学出版社,2005.4.
[55]Clore, G. L. Ortony, A. The semantics of the affective lexicon [M].Kluwer Academic Publishers, Dordrecht, The Netherlands, Boston,1988:453.
[56]Koda T., and Maes P. Agents with Faces:The Effects of Persorfication of Agents[C]//Proceedings of HCI'96, London, UK.1996.
[57]J. Bates, A.B.Loyall, Scott Neal Reilly. An architecture for action, emotion, and social behavior[M].School of computer science CMU-CS-92-144,Carnegie Mellon University,Pittsburgh, PA,1992.
[58]F.Thomas, O.Johnson.Disney Animation:The Illusion of Life [M]. Walt Disney Productions,1981.
[59]Clark.Elliot.Components of two-way emotions communication between humans and computers using a broad, rudimenteary model of affect and personality. [J].Cognitive Studies:Bulletin of the Japanese Cognitive Science Society, 1994,1 (2):16-30.
[60]Clark.Elliot.I pick up catapia and other stories:Amultimode approach to express for emotionally intelligent agents[C]//In Proceedings of the First International Conference on Autonomous Agents.1997:451-457.
[61]I.J. RosEman,A.A.Antoniou,P.E.Jose.Appraisals determinants of emotions: Constructing a more accurate and comprehensive theory[J].Cognition and Emotion,1996.10(3):241-277.
[62]I.J. RosEman,M.S.Spindel, and P.E.Jose.Appraisals of emotion-eliciting events: Testing a theory of discreet emotions [J].Journal of Personality and Social Psychology,1990,59 (5):899-915.
[63]I.J. RosEman.Appraisals determinants of disreet emotions [J]. Cognition and Emotion,1991,5(3):161-200.
[64]J.D.Velasquez.Cathexis:A computational model for the generations in synthetic agents[C]//In AAAI97.1997:10-15.
[65]A.Sloman, M.Croucher, Why robots will have emotions. [C]//In Proceedings Seventh International Conference on AI,1981:197-202.
[66]D.Canamero.Modeling motivations and emotions as a basis for intelligent behavior. [C]//Proceeding of the First International Conference on Autonomous Agents, New York ACM Press.1997:148-155.
[67]N.H.Frijda.The Emotions.Studies in Emotion and Social Interaction. [M].Cambridge University Press, Cambridge,1986.
[68]N.H.Frijda.Jwaqerman.Can Computer feel? Theory and Design of an emotional system[J].Cognition and Emotion,1987,1 (3):235-257.
[69]Bechara.Aldiss.Super-toys last all summer long Wired[M]. First published in Happer's Bazaar,1997:135-138.
[70]G.H.Bower,P.R.Cohen.Emotional influences in memory and thinking:Data and theory[C]//In Affective and Cognition:The Seventh Annual Carnegie Symposium on Cognition.Lawrence Erlbaum Associates,N.J1982:291-333.
[71]M.G.Dyer.Emotions and their computations:Three computer models[J].Cognition and Emotion,1987,1 (3):323-347.
[72]M.G.Dyer.The role of affect in narratives[J]. Cognitive Science,1983,7: 211-242.
[73]A.F.R. Araujo.Memory, Emotions, and Neural Networks:Associative Learning and Memory Recall Influenced by Affective Evaluation and Task difficulty[M].PHD thesis, University of Sussex.1994.
[74]Kort, B.; Reilly, R.; Picard, R.W. An affective model of interplay between emotions and learning:reengineering educational pedagogy-building a learning companion[C]//Proceedings of Advanced Learning Technologies,2001:43-46
[75]Ted Selker. San Jose. COACH:a teaching agent that learns[J].Communications of the ACM 1994,7:37.
[76]Lieberman H. Selker T. Out of context:Computer systems that adapt to, and learn from, context[J].IBM Systems Journal.2000,39(3.4):617-632.
[77]Nelson Goodman.OF mind and other matters[M]. Harved University Press.1984.
[78]44拍手势的轨迹http://wenwen.soso.com/z/q139981740.htm
[79]王波.观念主义[M].内蒙古：天马出版社,2002.
[80]常俊玲.原生态艺术传承与儿童审美教育[M].北京：高等教育出版社,2007.
[81]张旭书法作品和公孙大娘舞剑http://jx3.xoyo.com/wjyh/1016996
[82]高师《外国美术史及作品鉴赏》教材编写组.外国美术史及作品鉴赏[M].北京：高等教育出版社,1997.
[83]吴飬木.中国画基础技法[M].北京：朝华出版社,1996：13.
[84]自由舞蹈摄影http://www.ppyn.cn/kmyt/PicNew/product.php?proid= 109654
[85]朝鲜族鼓舞http://www.060s.com/
[86]王琳.简笔牡丹[M].北京：西苑出版社,1999：3.
[87]蔺高管.花鸟画章法入门[M].北京：人民美术出版社,1995：5-6.
[88]董建明.人机交互：以用户为中心的设计和评估[M].北京：清华大学出版社,2007.
[89]周昌乐.心脑计算举要[M].北京：清华大学出版社,2003.
[90]Zhao, L. Synthesis and Acquisition of Laban Movement Analysis Qualitative Parameters for Communicative Gestures[M]. PhD thesis, CIS, University of Pennsylvania.2001.
[91]Laban, R.;Lawrence F.C.Effort, Economy of Human Effort[M].second edtion, Macdonald&Evans LTD.,London, UK.1974.
[92]Lester, J.C., and, Stone. B. A., Increasing believability in animated pedagogical agent[C] InProceedings of autonomous Agents'97,1997:16-21.
[93]张顺,张翔.基于运动感知的人机交互方式[C]//Conference on Intelligent CAD and Digital Entertainment, Shanghai:2007.
[94]Badler N,Hollick M,Granieri J. Real-Time Control of a Virtual Human Using Minimal Sensors. [J].Presence,1993,2(1):82-86.
[95]Marrin T.Possibilities for the digital baton as a general-purpose gestural interface[C]//CHI'97 extended abstract on human factors in computing systems: looking to the future.Atlanta,1997:312.
[96]Hinckley K, Pierce J, Sinclair M, Horvitz E.Sensing techniques for mobile interaction[C]//Proceedings of the 13th annual ACM Symposium onUser Interface Software and Technology. San Diego, California, United States: ACM,2000:92-100.
[97]Wilson A,Shafer S.Xwawnd:Ui for intelligent spaces[C]//Proceedings of the Conference on Human factors in Computing system,2003:545-552.
[98]NINTENDO OF AMERICA INC.WII 2009 http://www.nintendo.com.hk/
[99]《Wii音乐》演奏乐器http://www.cnetnews.com.cn/2008/0716/988980.shtml
[100]《Wii音乐》的指挥乐团http://www.cnetnews.com.cn/2008/0716/988980.shtml
[101]苹果智能手机iphone http://www.vgooo.com/
[102]微软xbox360 kinect来源于硅谷动力网站
[103]Kinect有几个重要传感器设备http://www.vista123.com/vista/16911.html
[104]几种游戏设备比较.http://xbox360.tgbus.com/zixun/yenei/201011/20101105143316.shtml
[105]Kinect的工作原理http://www.chiphell.com/thread-135655-1-1.html
[106]意法半导体的首款iNEMO惯性测量单元http://www.eeworld.com.cn/MEMS/2010/0226/article_406.html
[107]任天堂游戏手柄http://game.yesky.com/wii/
[108]Beak J,Jang I J,Park K H, Kang H S,Yun B J,.Human Computer Interaction for the Accelerometer-based Mobile Game[J]. Embedded and Ubiquitous Computing,2006:509-518.
[109]Turk M,Kolsch M.Perceptual Interface[C]//Emerging Topics in Computer Vision.Prentice Hall,2004.
[110]用意念控制机器人运动源自本田公司网站
[111]胡小强.虚拟现实技术基础与应用[M].北京：北京邮电大学出版社,2009：116.
[112]鲍曼.张凤军译.三维用户界面：理论与实践[M].北京：电子工业出版社,2006：176-190.
[113]Sidney Fels. Geoffrey E. Hinton. Building adaptive interfaces with neural networks:The glove-talk pilot study [C]//Proceeding INTERACT'90 Proceedings of the IFIP TC13 Third Interational Conference on Human-Computer Interaction,1990.
[114]郭亚军,金先级.人机交互[M].武汉：华中科技大学出版社,2005：121-122.
[115]K Popat, RW Picard. Novel cluster-based probability model for texture synthesis, classification, and compression 1993 [C]//SPIE Visual Communications and Image Processing'93, Boston,1993.
[116]Paul Werbos.The roots of backpropagation:from ordered derivatives to neural networks[M].New York:Wiley,1994.
[117]L.A.Cameras.Expressive development and basic emotions[J].Cognition and Emotion,1992:3-4.
[118]A. Sherstinsky, R. W. Picard, On Stability and Equilibria of Analog Feedback Networks http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.52.1802
[119]S.Grossberg,W.E.Gutowski, Neural dynamics of descision making under risk: Affective balance and cognitive emotional interactions[J]. Psychological Review, 1987,94 (3):300-318.
[120]W.J. Freeman, Societies of Brains[M]. Lawrence Erlbaum, Hillsdale, NJ,1995.
[121]Balint L.Adaptive human-computer interface for man-machine interaction in computer-integrated systems[J].Computer-Integrated Manufacturing Systems, 1995,8 (2):133-142.
[122]H.A.Simon. Nativational and emotional controls of cognition [J].Models of Thought, New Haven:Yale University Press.1967.
[123]董士海.人机交互的进展及面临的挑战[J].计算机辅助设计与图形学学报16(1)：1-13.
[124]陈忠义.写意梅兰竹菊技法[M].沈阳：辽宁美术出版社,1998：19.
[125]中国美术学院中国画系.中国画[M].北京：高等教育出版社,1990：3-4.
[126]孙其峰.中国画技法[M].北京：人民美术出版社,1986：90.
[127]戴国忠,田丰.笔式用户界面[M].合肥：中国科学技术大学出版社,2009：8-9.
[128]Takeo Igarashi,Satoshi Matsuoka(?), Hidehiko Tanaka(?) Teddy:A Sketching Interface for 3D Freeform Design[C]//Computer Graphics Proceedings, Annual Conference Series, ACM SIGGRAPH 99. New York,1999:409-416.
[129]Steve Tsang, Ravin Balakrishnan, Karan Singh, Abhishek Ranjan, A Suggestive Interface for Image Guided 3D Sketching. [C]//ACM CHI 2004 Conference Proceedings, Vienna:ACM Press,2004:591-598.
[130]Aaron Adler, Randall Davis. Speech and Sketching for Multimodal Design. [C] //Proceedings of IUI, Funchal,2004:214-216.
[131]李杰,田丰,王维信,王维信,戴国忠,面向儿童的多通道交互系统[J]软件学报,2002,13(9)：1846-1851.
[132]世纪出版集团.汉语大词典[M].汉语大词典出版社,2000：120.
[133]马颂德,张友正.计算机视觉[M].北京：科学出版社,1998：34.
[134]Liang X B, Li Q L, Zhang X, et al. Performance-driven motion choreographing with accelerometers [J]. Computer Animation and Virtual World, vol.2009,20: 89-99.
[135]Sreeram Sreedharan, Edmund S. Zurita, Beryl Plimmer.3D Input for 3D Worlds [C]//Proceedings of CHI 2007, Australia,2007.
[136]Hyun.Jean.Lee, Hyungsin.Kim, Gaurav.Gupta,Ali.Mazalek. WiiArts:Creating collaborative art experience [C]//Proceedings of TEI 2008, Bonn,2008.
[137]Gromala, D. Windows and Mirrors:Interaction Design, Digital Art and the Myth of Transparency [M]. MIT Press, Cambridge, MA,2005.
[138]Bruegge, B., Teschner, C., Lachenmaier, P., Fenzl, E.,Schmidt, D., and Bierbaum, S., Pinocchio:conducting a virtual symphony orchestra[C]//In Proc. ACE 2007, ACM Press (2007),294-295.
[139]Shirai, A., Geslin, E., and Richir, S, WiiMedia:motion analysis methods and applications using a consumer video game controller. [C]//Proceedings of Sandbox Symposium. San Diego,2007:133-140.
[140]李祯孝.工笔花鸟技法[M].重庆：西南师范大学出版社,1994：11-12.
[141]Thomas Schl " omer, Benjamin Poppinga, Niels Henze, Susanne Boll. Gesture Recognition with a Wii Controller [C]//Proceedings of the Second International Conference on Tangible and Embedded Interaction (TEI'08), Bonn,2008: 11-14.
[142]Hofmann, F., Heyer, P. and Hommel, G. Velocity profile based recognition of dynamic gestures with discrete Hidden Markov Models [C]//Proceedings of the International Gesture Workshop on Gesture and Sign Language in Human-Computer Interaction, Springer-Verlag, London,2004:81-95.
[143]梁秀波,张顺,李启雷,张翔,耿卫东.运动传感驱动的3D直观手势交互[J].计算机辅助设计与图形学学报,2010,22(3)：521-526.)
[144]Jesse James Garrett.The Elements of User Experience:User-Centered Design for the Web [M].Pearson Education.2007:13.
[145]Hyun-Jean Lee, Hyungsin Kim, Gaurav Gupta, Ali Mazalek. Wii Arts:Creating collaborative art experience [C]//Proceedings of the Second International Conference on Tangible and Embedded Interaction (TEI'08), Bonn, Germany, 2008:33-36.
[146]Darwin.C The Expression of the Emotions in Animals and Man [M].London, UK:John Murray,1872.
[147]Hu Xinyue, Wang Keyi. Personalized Recommendation for Virtual Reality [C] //Proceedings of The International Conference on Multimedia Technology, ICMTNingbo,2010.:57-65.