面向网络流量分析的网格系统的设计与实现
|
摘要
现今社会,科技飞速发展,信息量迅猛增加,现有计算机的计算速度,网络的通讯能力以及存储设备的存储能力越来越无法满足日益增长的需求。与此同时,随着基于互联网的新型应用的开发,网络上的协议种类也日趋增多。对网络中使用的各种协议进行快速有效的识别,成为了网络运营商和网络管理人员监控、调整、升级网络设备的重要参考依据和管理手段。
现有的流量分析系统虽然在一定程度上解决了相关问题,但是也同样存在着诸多不足。比如现有的识别系统,因为在算法上存在缺陷或是缺少对新协议的支持,使得识别结果中“其它类”的数据类型甚至可以占到50%的比例,失去了分析的意义。同时,随着网络带宽的不断增加,程序执行效率偏低且需要占用大量计算资源也变成了无法克服的问题,现有的硬件水平在速度上已经无法经受万兆带宽以上的流量识别。最后,大部分的流量分析系统往往只针对特定的平台,即使偶有支持多平台的系统出现也没能实现真正意思上的跨平台,而只是推出针对不同平台的同一种工具,其识别效果也不令人满意。
针对以上提出的各种不足,近年出现的网格平台为解决这一系列问题提供了新的思路。网格是利用互联网把地理上广泛分布的各种资源(包括计算资源、存储资源、带宽资源、软件资源、数据资源、信息资源、知识资源等)连成一个逻辑整体,形成一台超级计算机,为用户提供一体化信息和应用服务,最终实现在这个虚拟环境下进行资源共享和协同工作。网格平台可以为流量分析系统提供强大的计算资源和存储资源,同时网格平台是真正意义上的跨平台系统,可以完好的支持包括Windows,UNIX在内的各种操作系统平台。网格的这一系列特点为目前流量分析系统的不足提供了解决方案,为实现万兆以上流量分析系统奠定了基础。
本课题在分析了现有的流量分析系统和近年主流的网格体系架构后,通过查阅网格领域国内外的相关情况,详细介绍了目前网格的实际标准—Globus工具包的使用和开发应用的流程,并在此基础上编程实现了面向网络流量分析的网格系统的开发和调用。本文最后对整个系统进行了总结以及有待改善的不足之处。
With the rapid development of science and technology in the information area, the computing speed of computers, the communication capacity of networks and the storage capacity of storage devices have all become bottlenecks for nowadays implementation requirements. At the same time, with the new application development based on Internet, the types of network protocols have become increasingly day by day. To rapid and effective identify the various protocols of the networks, became the ISP and network managers to monitor, adjustment and upgrade of network equipment and management of important reference tool.
Although the existing traffic analysis system resolved some relevant issues, but there are also many inadequacies. Such as the existing identification system, because the defective of algorithm or the lack of support for the new protocols, making identification results in the "other category" type of data can even account for 50% of the proportion of the loss of significance. At the same time, with the ever increasing of network bandwidth, and the lower efficiency of the implementation of procedures need to use up a lot of computing resources have become unable to overcome the problem of existing hardware in the level of speed has been unable to withstand the flow of more than 10 Gigabit bandwidth identification. Finally, most of the traffic analysis systems are often only for a specific platform, even if the occasional support of the system of multi-platform also failed to achieve real meaning of the cross-platform, but only introduced for different platforms with a tool, not the effect of recognition satisfactory.
For the above deficiencies, and in recent years, the grid platform to address the issue of this series provides a new way of thinking. Grid is the use of the Internet on the wide geographical distribution of various resources (including computing resources, storage resources, bandwidth resources, software resources, data resources, information resources, knowledge resources, etc.) as a single logical entity, formed a supercomputer, To provide users with information and application integration services, and eventually realize in this virtual environment for the sharing of resources and work together. Grid platform for the traffic analysis system can provide a powerful computing resources and storage resources, while grid platform is the true sense of the boast platform system that can well support, including Windows, UNIX, the various operating system platforms. Grid characteristics of this series for the current lack of traffic analysis system provide solutions for the realization of more than 10 Gigabit traffic analysis systems laid the foundation.
In this issue of the existing traffic analysis system and the mainstream in recent years, the grid architecture, through access to grid areas of the relevant circumstances at home and abroad, details of the current grid of the actual standards - Globus tool kit of the use and development Application of the process, and on the basis of this programming in the face of network traffic analysis of the grid system development and call. Finally, the entire system as well as a summary of room for improvement deficiencies.
现有的流量分析系统虽然在一定程度上解决了相关问题,但是也同样存在着诸多不足。比如现有的识别系统,因为在算法上存在缺陷或是缺少对新协议的支持,使得识别结果中“其它类”的数据类型甚至可以占到50%的比例,失去了分析的意义。同时,随着网络带宽的不断增加,程序执行效率偏低且需要占用大量计算资源也变成了无法克服的问题,现有的硬件水平在速度上已经无法经受万兆带宽以上的流量识别。最后,大部分的流量分析系统往往只针对特定的平台,即使偶有支持多平台的系统出现也没能实现真正意思上的跨平台,而只是推出针对不同平台的同一种工具,其识别效果也不令人满意。
针对以上提出的各种不足,近年出现的网格平台为解决这一系列问题提供了新的思路。网格是利用互联网把地理上广泛分布的各种资源(包括计算资源、存储资源、带宽资源、软件资源、数据资源、信息资源、知识资源等)连成一个逻辑整体,形成一台超级计算机,为用户提供一体化信息和应用服务,最终实现在这个虚拟环境下进行资源共享和协同工作。网格平台可以为流量分析系统提供强大的计算资源和存储资源,同时网格平台是真正意义上的跨平台系统,可以完好的支持包括Windows,UNIX在内的各种操作系统平台。网格的这一系列特点为目前流量分析系统的不足提供了解决方案,为实现万兆以上流量分析系统奠定了基础。
本课题在分析了现有的流量分析系统和近年主流的网格体系架构后,通过查阅网格领域国内外的相关情况,详细介绍了目前网格的实际标准—Globus工具包的使用和开发应用的流程,并在此基础上编程实现了面向网络流量分析的网格系统的开发和调用。本文最后对整个系统进行了总结以及有待改善的不足之处。
With the rapid development of science and technology in the information area, the computing speed of computers, the communication capacity of networks and the storage capacity of storage devices have all become bottlenecks for nowadays implementation requirements. At the same time, with the new application development based on Internet, the types of network protocols have become increasingly day by day. To rapid and effective identify the various protocols of the networks, became the ISP and network managers to monitor, adjustment and upgrade of network equipment and management of important reference tool.
Although the existing traffic analysis system resolved some relevant issues, but there are also many inadequacies. Such as the existing identification system, because the defective of algorithm or the lack of support for the new protocols, making identification results in the "other category" type of data can even account for 50% of the proportion of the loss of significance. At the same time, with the ever increasing of network bandwidth, and the lower efficiency of the implementation of procedures need to use up a lot of computing resources have become unable to overcome the problem of existing hardware in the level of speed has been unable to withstand the flow of more than 10 Gigabit bandwidth identification. Finally, most of the traffic analysis systems are often only for a specific platform, even if the occasional support of the system of multi-platform also failed to achieve real meaning of the cross-platform, but only introduced for different platforms with a tool, not the effect of recognition satisfactory.
For the above deficiencies, and in recent years, the grid platform to address the issue of this series provides a new way of thinking. Grid is the use of the Internet on the wide geographical distribution of various resources (including computing resources, storage resources, bandwidth resources, software resources, data resources, information resources, knowledge resources, etc.) as a single logical entity, formed a supercomputer, To provide users with information and application integration services, and eventually realize in this virtual environment for the sharing of resources and work together. Grid platform for the traffic analysis system can provide a powerful computing resources and storage resources, while grid platform is the true sense of the boast platform system that can well support, including Windows, UNIX, the various operating system platforms. Grid characteristics of this series for the current lack of traffic analysis system provide solutions for the realization of more than 10 Gigabit traffic analysis systems laid the foundation.
In this issue of the existing traffic analysis system and the mainstream in recent years, the grid architecture, through access to grid areas of the relevant circumstances at home and abroad, details of the current grid of the actual standards - Globus tool kit of the use and development Application of the process, and on the basis of this programming in the face of network traffic analysis of the grid system development and call. Finally, the entire system as well as a summary of room for improvement deficiencies.
引文
[1]http://www.chinagrid.net/dvnews/show.aspx?id=645&cid=34,网格信息中转站 网格基础概念
[2]刘鹏 网格发展趋势 清华大学高性能所网格组 2005
[3]http://www.net130.com/netbass/grid/,Cisco及网格技术
[4]http://www.chinagrid.net/dvnews/show.aspx?id=1575&cid=16,IBM网格观点:网格计算--下一代分布式计算
[5]http://swig.minidx.com/exec.html SWIG Executive Summary
[6]http://www.rcuk.ac.uk/escience/default.htm,e-Science Research
[7]http://eu-datagrid.web.cern.ch/eu%2Ddatagrid,The Data Grid Project
[8]金海 中国网格技术的发展及现状,Cernet第十届学术年会 2003年
[9]http://www.ethereal.com,network protocol analyzer
[10]龚强 网格概论,金融电子化2005年
[11]龚强 网格特征详解,金融电子化 2005年
[12]Teri B(?)dwell等,吴东升等译,信息时代的个人安全策略,科学出版社,2003年
[13]丁涛 面向P2P协议流量分析的网格系统的设计与实现2008年
[14]刘成 基于网格的服务发现研究2005年
[15]刘瑛 浅谈网格技术的应用前景.电子材料与电子技术2004年
[16]都志辉,陈渝,刘鹏 网格计算 2002年
[17]http://download.zdnet.com.cn/software_zone/2007/1106/611453.shtml,Linux 下Libpcap源码分析和包过滤机制
[18]http://www.w3.org/TR,W3C Technical Reports and Publications
[19]Cay S.Horstmann,Gary Cornell著,程峰等译,Java核心技术卷I 2003年
[20]丁涛,黄小红,王振华等 基于Globus网格服务开发的研究 2006年
[2]刘鹏 网格发展趋势 清华大学高性能所网格组 2005
[3]http://www.net130.com/netbass/grid/,Cisco及网格技术
[4]http://www.chinagrid.net/dvnews/show.aspx?id=1575&cid=16,IBM网格观点:网格计算--下一代分布式计算
[5]http://swig.minidx.com/exec.html SWIG Executive Summary
[6]http://www.rcuk.ac.uk/escience/default.htm,e-Science Research
[7]http://eu-datagrid.web.cern.ch/eu%2Ddatagrid,The Data Grid Project
[8]金海 中国网格技术的发展及现状,Cernet第十届学术年会 2003年
[9]http://www.ethereal.com,network protocol analyzer
[10]龚强 网格概论,金融电子化2005年
[11]龚强 网格特征详解,金融电子化 2005年
[12]Teri B(?)dwell等,吴东升等译,信息时代的个人安全策略,科学出版社,2003年
[13]丁涛 面向P2P协议流量分析的网格系统的设计与实现2008年
[14]刘成 基于网格的服务发现研究2005年
[15]刘瑛 浅谈网格技术的应用前景.电子材料与电子技术2004年
[16]都志辉,陈渝,刘鹏 网格计算 2002年
[17]http://download.zdnet.com.cn/software_zone/2007/1106/611453.shtml,Linux 下Libpcap源码分析和包过滤机制
[18]http://www.w3.org/TR,W3C Technical Reports and Publications
[19]Cay S.Horstmann,Gary Cornell著,程峰等译,Java核心技术卷I 2003年
[20]丁涛,黄小红,王振华等 基于Globus网格服务开发的研究 2006年