基于数据挖掘的入侵检测系统研究与实现
|
摘要
网络实现了包括个人、企业与政府以及全社会信息共享，随着网络应用范围的不断扩大，对网络的各类攻击与破坏也与日俱增。计算机网络安全是一个国际化的问题，每年全球因计算机网络的安全系统被破坏而造成的经济损失达数百亿美元。无论政府、商务，还是金融、媒体的网站都在不同程度上受到入侵与破坏。网络安全已成为国家与国防安全的重要组成部分，同时也是国家网络经济发展的关键。
保障计算机系统和网络系统及整个信息基础设施的安全，以及对入侵攻击的检测与防范，已经成为刻不容缓的重要课题。网络安全是一个系统的概念，有效的安全策略或方案的制定，是网络信息安全的首要目标，入侵检测技术是继“防火墙”、“数据加密”等传统安全保护措施后新一代的安全保障技术。
现行的入侵检测系统在利用用户行为特征来建立正常模式或异常模式时，由于没有很好的利用数据挖掘技术，所提取的用户行为特性和入侵模式特征不能很好的反映实际的情况。另外，所建立的正常模式或异常模式也不够完善，容易造成误警或漏警，给网络系统造成损失。更为重要的是，很多入侵检测系统要经常更新才能检测出新的攻击和适应环境的变化，此外，它们大多采用人工编造攻击特征的方式来检测入侵，这使得入侵检测系统更新比较缓慢也比较昂贵。而数据挖掘在用户行为特征提取方面具有非常大的优势，本文根据IDS的特点，将数据挖掘技术应用于IDS中，提出了采用数据挖掘的入侵检测系统的结构模型，解决了一般入侵检测系统的不足。这个模型首先从训练数据中提取并得到规则，然后用这些规则去检测新的入侵，这样规则的更新和系统的更新很快也很低廉，而且检测率很高，实验证明将数据挖掘运用到入侵检测系统中是可行的，也是有效的。
The Network makes us share all of the information, including the information of person, company and the government. With more and more using of the Network, more intrusion and more demolishment occurred. Computer Network' s safe is a world' s problem, every year we loss more than two million because of demolishment. Not only the government and commerce but also the finance and media web site are intruded and destroyed in different degree. Network Security becomes the important member in the safe of nation and national defense, and it is meaningfulness for the national network economy.
It is becoming a very important task to protect computer system-, network system and the whole Information Infrastructure, and to keep away from the intrusion. Network Security is a system conception; efficiency Security Policy is the first aim for Network Security, The Intrusion Detection is a new security technology, apart from tradition security protect technology, such as firewall and data crypt.
When many current Intrusion Detection System (IDS) using user behavior profile to build normal patterns or abnormal patterns, since it do not using Data Mining technology, the user behavior profile can't reflect the fact. Moreover, the normal pattern or abnormal pattern is not perfect and the false alarm rate and leak alarm rate are very high. The more important, there is often the need to update an installed IDS due to new attack methods or upgraded computing environments. Since many current IDS are constructed by manual encoding of expert knowledge, changes to IDS are expensive and slow. However, the data mining techniques in discovering behavior features has a big advantage, in this paper, we describe a data mining framework for adaptively building Intrusion Detection (ID) models. In this models, first we extract features and rules from
the training data, then using these rules to detect new intrusion, by using this means, updating rules and system will be more faster and cheaper, and the detection rate is high, the result proves that using data mining technology for building Intrusion Detection System is viably and availably.
保障计算机系统和网络系统及整个信息基础设施的安全，以及对入侵攻击的检测与防范，已经成为刻不容缓的重要课题。网络安全是一个系统的概念，有效的安全策略或方案的制定，是网络信息安全的首要目标，入侵检测技术是继“防火墙”、“数据加密”等传统安全保护措施后新一代的安全保障技术。
现行的入侵检测系统在利用用户行为特征来建立正常模式或异常模式时，由于没有很好的利用数据挖掘技术，所提取的用户行为特性和入侵模式特征不能很好的反映实际的情况。另外，所建立的正常模式或异常模式也不够完善，容易造成误警或漏警，给网络系统造成损失。更为重要的是，很多入侵检测系统要经常更新才能检测出新的攻击和适应环境的变化，此外，它们大多采用人工编造攻击特征的方式来检测入侵，这使得入侵检测系统更新比较缓慢也比较昂贵。而数据挖掘在用户行为特征提取方面具有非常大的优势，本文根据IDS的特点，将数据挖掘技术应用于IDS中，提出了采用数据挖掘的入侵检测系统的结构模型，解决了一般入侵检测系统的不足。这个模型首先从训练数据中提取并得到规则，然后用这些规则去检测新的入侵，这样规则的更新和系统的更新很快也很低廉，而且检测率很高，实验证明将数据挖掘运用到入侵检测系统中是可行的，也是有效的。
The Network makes us share all of the information, including the information of person, company and the government. With more and more using of the Network, more intrusion and more demolishment occurred. Computer Network' s safe is a world' s problem, every year we loss more than two million because of demolishment. Not only the government and commerce but also the finance and media web site are intruded and destroyed in different degree. Network Security becomes the important member in the safe of nation and national defense, and it is meaningfulness for the national network economy.
It is becoming a very important task to protect computer system-, network system and the whole Information Infrastructure, and to keep away from the intrusion. Network Security is a system conception; efficiency Security Policy is the first aim for Network Security, The Intrusion Detection is a new security technology, apart from tradition security protect technology, such as firewall and data crypt.
When many current Intrusion Detection System (IDS) using user behavior profile to build normal patterns or abnormal patterns, since it do not using Data Mining technology, the user behavior profile can't reflect the fact. Moreover, the normal pattern or abnormal pattern is not perfect and the false alarm rate and leak alarm rate are very high. The more important, there is often the need to update an installed IDS due to new attack methods or upgraded computing environments. Since many current IDS are constructed by manual encoding of expert knowledge, changes to IDS are expensive and slow. However, the data mining techniques in discovering behavior features has a big advantage, in this paper, we describe a data mining framework for adaptively building Intrusion Detection (ID) models. In this models, first we extract features and rules from
the training data, then using these rules to detect new intrusion, by using this means, updating rules and system will be more faster and cheaper, and the detection rate is high, the result proves that using data mining technology for building Intrusion Detection System is viably and availably.
引文
[1] S. M. Bellovin, Security problems in the TCP/IP protocol suite[J], Computer Communication Review, 1989, 19(2): 32-48.
[2] Dorothy E. Denning, An Intrusion-Detection Model, IEEE Transactions on Software Engineering, VOL. SE-13, NO. 2, February 1987, 222-232.
[3] Wenke Lee, Salvatore J. Stolfo, Kui W. Mok, A Data Mining Framework for Builiding Intrusion Detection Models, Proc of the 1999 IEEE Symp on Security and Privacy, May 1999.
[4] Weke Lee, A Data Mining Frameworke for Constructing Features and Models for Intrusion Detection Systems, PhD thesis, Columbia University, 1999.
[5] Wenke Lee, Salvatore J. Stolfo, Kui W. Mok, Mining Audit Data to Build Intrusion Detection Models, In Proceedings of the fourth internationgal conference on knowledge discovery and data mining held in New York, New Yoke, August 27-31,1998, edited by Rakesh Agrawal, and Paul Stolorz, 66-72, New York, NY: AAAI Press.
[6] Wenke Lee, Salvatore J. Stolfo, Data Mining Approaches for Intrusion Detection Models, Proc of the 7th USENIX Security Symp, San Antonio, TX, January 1998.
[7] Wenke Lee, Dong X. Information-Theoretic measures for anomaly detection. In: Needham R, Abadi M, eds. Proceedings of the 2001 IEEE Symposium on Security and Privacy. Oakland, CA: IEEE Computer Society Press, 2001. 130～143.
[8] Rebecca Gurley Bace[美]著，陈明奇、吴秋新等[译]，《入侵检测》，人民邮电出版社，2001年6月ISBN 7-115-09287／TP．2210
[9] Phil Porras, Dan Schnackenberg, Stuart Staniford-Chen. The Common Intrusion Detection Framework Architecture [EB/OL].http://www.gidos.org.
[10] 胡化平、陈海涛、黄振林、唐勇，入侵检测系统研究现状及发展趋
势。计算机工程与科学，2001年第23卷第2期。
[11] KDD99, KDD99 cup dataset http://kdd.cis.uci.edu/databases/kddcup99/kddcup99.html,1999
[12] Jiawei Han and Micheline Kamber, Simon Fraser University, Data Mining: Concepts and Techniques. Simon Fraser University, August 2000, Morgan Kaufman
[13] Mikhail Gordeev, "Intrusion Detection: Techniques and Approaches".
[14] Stephen Northcutt 著，余青霓、王晓程、周钢 等[译]，《网络入侵检测分析员手册》，人民邮电出版社，2002年。
[15] Stephen Northcu, Sark Cooper, Matt Fearnow, Karen Frederick著，林琪[译]，《入侵特征与分析》，中国电力出版社，20002年9月。
[16] Paul E．Proctor 著，邓琦皓，许鸿飞，张斌[译]，《入侵检测实用手册》，中国电力出版社，20002年10月。
[17] 韩东海，王超 李群，《入侵监测系统及实例剖析》，清华大学出版社，2002年5月
[18] 戴英侠，连一峰，王航，《系统安全与入侵检测》，清华大学出版社，2002年3月。
[19] 唐正军，《网络入侵检测系统的设计与实现》，电子工业出版社，2002年4月
[20] 冯登国，《计算机通信网络安全》，清华大学出版社，2001年3月
[21] 阮耀平，易江波，赵战生，计算机系统入侵检测模型与方法，计算机工程，1999年9月第25卷第9期
[22] 蒋建春，马恒太，任党恩，卿斯汉，网络安全入侵检测研究综述，软件学报 1000-9825／2000／11(11)1460-0
[23] J. Frank, Articial intelligence and intrusion detection: Current and future directions. In Proceedings of the 17th National Computer Security Conference, October 1994.
[24] R. Heady, G. Luger, A. Maccabe, and M. Servilla. The architecture of a network evel intrusion detection system. Technical report,Computer Science, Department, University of New Mexico, August 1990.
[25] S. McClure, J. Scambray, and J. Broderick. Test Center Comparison: Network intrusion-detection solutions. In INFOWORLD May 4, 1998.
[26] 李维民、彭新光、王全民，一种基于匹配集的入侵检测方法，太原理工大学学报，第32卷第6期，2001年11月。
[27] 卢勇、曹阳、凌军，李莉，基于数据挖掘的入侵检测系统框架，武汉大学学报(理学报)，第48卷第1期，2002年2月。
[28] 杨向荣，宋擒豹，沈钧毅，基于数据挖掘的智能化入侵检测系统，计算机工程，第27卷第9期，2001年9月。
[29] 李新远，吴宇红，狄文远，基于数据发掘的入侵检测建模，计算机工程，第28卷第2期，2002年2月。
[30] P. A. Porras and A. Valdes. Live tra. c analysis of TCP/IP gateways. In Proceedings of the Internet Society Symposium on Network and Distributed System Security, March 1998.
[31] Robert Graham, (nids-faq@RobertGraham.com), 《IDS FAQ》, March 21,2000.
[32] 《Network Based Intrusion Detection-A review of technologies》, DENMAC SYSTEMS, INC, NOVEMBER 1999.
[33] Wood, M., 《Intrusion Detection Message Exchange Requirements》 Internet-Draft Internet Engineering Task Force, Internet Security Systems, October, 1999.
[34] W．Richard Stevens著，施振川，周利民，孙宏晖等译．UNIX网络编程(第一卷)．清华大学出版社．1999．7
[35] Phil Porras, SRI, Dan Schnackenberg, Boeing, Stuart Staniford-Chen, UC, Davis, editor, Maureen Stillman, Oddysey Research, Felix Wu, NCSU 《The Common Intrusion Detection Framework Architecture》
[36] Clifford Kahn, Phillip A. Porras, Stuart Staniford-Chen,Brian Tung, 《A Common Intrusion Detection Framework》, 15 July 1998.
[37] R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In Proceedings of the 20th VLDB Conference,
Santiago, Chile, 1994.
[38] 袁春阳，柴乔林，柳忠光，网络入侵检测系统中网络实时监听程序的设计与实现，计算机工程，第28卷第10期，2002年10月。
[39] 蒋巍川，田盛丰，数据挖掘技术在入侵检测中的应用，计算机工程，第27卷第4期，2001年4月。
[40] 蒋巍川，田盛丰，入侵检测中对系统日志审计信息进行数据挖掘的研究，计算机工程，第27卷第4期，2001年4月。
[41] 陈文杰，袁超伟，入侵检测系统及其数据收集机制，微机发展，2002年第5期。
[42] Agrawal, R., T. Imielinski, and A. Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD international conference on management of data held in Washington, D. C., May 26-28, 1993,207-216.
[43] Barbara, D., S. Jajodia, and N. Wu. 2000. Mining unexpected rules in network audit trails. Personal communications. Bridges,S. and R. Vaughn. 2000. Fuzzy data mining and genetic algorithms applied to intrusion detection. In Proceedings of the 23rd National Information Systems Security Conference held in Baltimore, MA, October 16-19, 2000, 13-31.
[44] Cheung, D., S. Lee, and B. Kao. 1997. A general incremental technique for updating discovered association rules. In Proceedings of the 5th international conference on database systems for advanced applications(DASFAA'97) held in Melbourne, Australia, April 1-4, 1997, 185-194.
[45] Cheung, D., J. Han, V. Ng, and C. Wong. 1996. Maintenance of discovered association rules in large databases: An incremental updating technique. In Proceedings of the 12th IEEE international conference on data engineering (ICDE' 96) held in New Orleans,Louisiana, February 26-March 1, 1996, 106-114.
[46] Luo, J. and S. Bridges, 2000. Mining fuzzy association rules and fuzzy frequency episodes for intrusion detection.
International Journal of Intelligent Systems 15(8): 687-703.
[47] Toivonen, H, Sampling large databases for association rules.In Proceedings of 22nd international conference on very large data bases (VLDB' 96) held in Mumbai, India, September 3-6, 1996,134-145.
[2] Dorothy E. Denning, An Intrusion-Detection Model, IEEE Transactions on Software Engineering, VOL. SE-13, NO. 2, February 1987, 222-232.
[3] Wenke Lee, Salvatore J. Stolfo, Kui W. Mok, A Data Mining Framework for Builiding Intrusion Detection Models, Proc of the 1999 IEEE Symp on Security and Privacy, May 1999.
[4] Weke Lee, A Data Mining Frameworke for Constructing Features and Models for Intrusion Detection Systems, PhD thesis, Columbia University, 1999.
[5] Wenke Lee, Salvatore J. Stolfo, Kui W. Mok, Mining Audit Data to Build Intrusion Detection Models, In Proceedings of the fourth internationgal conference on knowledge discovery and data mining held in New York, New Yoke, August 27-31,1998, edited by Rakesh Agrawal, and Paul Stolorz, 66-72, New York, NY: AAAI Press.
[6] Wenke Lee, Salvatore J. Stolfo, Data Mining Approaches for Intrusion Detection Models, Proc of the 7th USENIX Security Symp, San Antonio, TX, January 1998.
[7] Wenke Lee, Dong X. Information-Theoretic measures for anomaly detection. In: Needham R, Abadi M, eds. Proceedings of the 2001 IEEE Symposium on Security and Privacy. Oakland, CA: IEEE Computer Society Press, 2001. 130～143.
[8] Rebecca Gurley Bace[美]著，陈明奇、吴秋新等[译]，《入侵检测》，人民邮电出版社，2001年6月ISBN 7-115-09287／TP．2210
[9] Phil Porras, Dan Schnackenberg, Stuart Staniford-Chen. The Common Intrusion Detection Framework Architecture [EB/OL].http://www.gidos.org.
[10] 胡化平、陈海涛、黄振林、唐勇，入侵检测系统研究现状及发展趋
势。计算机工程与科学，2001年第23卷第2期。
[11] KDD99, KDD99 cup dataset http://kdd.cis.uci.edu/databases/kddcup99/kddcup99.html,1999
[12] Jiawei Han and Micheline Kamber, Simon Fraser University, Data Mining: Concepts and Techniques. Simon Fraser University, August 2000, Morgan Kaufman
[13] Mikhail Gordeev, "Intrusion Detection: Techniques and Approaches".
[14] Stephen Northcutt 著，余青霓、王晓程、周钢 等[译]，《网络入侵检测分析员手册》，人民邮电出版社，2002年。
[15] Stephen Northcu, Sark Cooper, Matt Fearnow, Karen Frederick著，林琪[译]，《入侵特征与分析》，中国电力出版社，20002年9月。
[16] Paul E．Proctor 著，邓琦皓，许鸿飞，张斌[译]，《入侵检测实用手册》，中国电力出版社，20002年10月。
[17] 韩东海，王超 李群，《入侵监测系统及实例剖析》，清华大学出版社，2002年5月
[18] 戴英侠，连一峰，王航，《系统安全与入侵检测》，清华大学出版社，2002年3月。
[19] 唐正军，《网络入侵检测系统的设计与实现》，电子工业出版社，2002年4月
[20] 冯登国，《计算机通信网络安全》，清华大学出版社，2001年3月
[21] 阮耀平，易江波，赵战生，计算机系统入侵检测模型与方法，计算机工程，1999年9月第25卷第9期
[22] 蒋建春，马恒太，任党恩，卿斯汉，网络安全入侵检测研究综述，软件学报 1000-9825／2000／11(11)1460-0
[23] J. Frank, Articial intelligence and intrusion detection: Current and future directions. In Proceedings of the 17th National Computer Security Conference, October 1994.
[24] R. Heady, G. Luger, A. Maccabe, and M. Servilla. The architecture of a network evel intrusion detection system. Technical report,Computer Science, Department, University of New Mexico, August 1990.
[25] S. McClure, J. Scambray, and J. Broderick. Test Center Comparison: Network intrusion-detection solutions. In INFOWORLD May 4, 1998.
[26] 李维民、彭新光、王全民，一种基于匹配集的入侵检测方法，太原理工大学学报，第32卷第6期，2001年11月。
[27] 卢勇、曹阳、凌军，李莉，基于数据挖掘的入侵检测系统框架，武汉大学学报(理学报)，第48卷第1期，2002年2月。
[28] 杨向荣，宋擒豹，沈钧毅，基于数据挖掘的智能化入侵检测系统，计算机工程，第27卷第9期，2001年9月。
[29] 李新远，吴宇红，狄文远，基于数据发掘的入侵检测建模，计算机工程，第28卷第2期，2002年2月。
[30] P. A. Porras and A. Valdes. Live tra. c analysis of TCP/IP gateways. In Proceedings of the Internet Society Symposium on Network and Distributed System Security, March 1998.
[31] Robert Graham, (nids-faq@RobertGraham.com), 《IDS FAQ》, March 21,2000.
[32] 《Network Based Intrusion Detection-A review of technologies》, DENMAC SYSTEMS, INC, NOVEMBER 1999.
[33] Wood, M., 《Intrusion Detection Message Exchange Requirements》 Internet-Draft Internet Engineering Task Force, Internet Security Systems, October, 1999.
[34] W．Richard Stevens著，施振川，周利民，孙宏晖等译．UNIX网络编程(第一卷)．清华大学出版社．1999．7
[35] Phil Porras, SRI, Dan Schnackenberg, Boeing, Stuart Staniford-Chen, UC, Davis, editor, Maureen Stillman, Oddysey Research, Felix Wu, NCSU 《The Common Intrusion Detection Framework Architecture》
[36] Clifford Kahn, Phillip A. Porras, Stuart Staniford-Chen,Brian Tung, 《A Common Intrusion Detection Framework》, 15 July 1998.
[37] R. Agrawal and R. Srikant. Fast algorithms for mining association rules. In Proceedings of the 20th VLDB Conference,
Santiago, Chile, 1994.
[38] 袁春阳，柴乔林，柳忠光，网络入侵检测系统中网络实时监听程序的设计与实现，计算机工程，第28卷第10期，2002年10月。
[39] 蒋巍川，田盛丰，数据挖掘技术在入侵检测中的应用，计算机工程，第27卷第4期，2001年4月。
[40] 蒋巍川，田盛丰，入侵检测中对系统日志审计信息进行数据挖掘的研究，计算机工程，第27卷第4期，2001年4月。
[41] 陈文杰，袁超伟，入侵检测系统及其数据收集机制，微机发展，2002年第5期。
[42] Agrawal, R., T. Imielinski, and A. Swami. 1993. Mining association rules between sets of items in large databases. In Proceedings of the 1993 ACM SIGMOD international conference on management of data held in Washington, D. C., May 26-28, 1993,207-216.
[43] Barbara, D., S. Jajodia, and N. Wu. 2000. Mining unexpected rules in network audit trails. Personal communications. Bridges,S. and R. Vaughn. 2000. Fuzzy data mining and genetic algorithms applied to intrusion detection. In Proceedings of the 23rd National Information Systems Security Conference held in Baltimore, MA, October 16-19, 2000, 13-31.
[44] Cheung, D., S. Lee, and B. Kao. 1997. A general incremental technique for updating discovered association rules. In Proceedings of the 5th international conference on database systems for advanced applications(DASFAA'97) held in Melbourne, Australia, April 1-4, 1997, 185-194.
[45] Cheung, D., J. Han, V. Ng, and C. Wong. 1996. Maintenance of discovered association rules in large databases: An incremental updating technique. In Proceedings of the 12th IEEE international conference on data engineering (ICDE' 96) held in New Orleans,Louisiana, February 26-March 1, 1996, 106-114.
[46] Luo, J. and S. Bridges, 2000. Mining fuzzy association rules and fuzzy frequency episodes for intrusion detection.
International Journal of Intelligent Systems 15(8): 687-703.
[47] Toivonen, H, Sampling large databases for association rules.In Proceedings of 22nd international conference on very large data bases (VLDB' 96) held in Mumbai, India, September 3-6, 1996,134-145.