摘要
由于企业信息化是一个渐进的过程,所以在企业中会出现信息孤岛的问题,它们同时构成了企业的异构数据库结构。如何解决这个问题,使得企业数据能够共享,是计算机领域研究的一个热点。
本文设计并实现了中小企业异构数据集成系统,主要的研究内容如下:
◆在对传统数据提取方法分析的基础上,针对我们开发的中小企业异构数据库集成实用系统,提出了两种新的增量数据提取方法:一种是虚拟日志,该方法利用触发器把数据库操作序列写入操作表中,通过读取操作表来获取增量数据。创新处在于我们将提取的对象缩小到了字段级,并提出了对日志的优化方法,减少了数据的冗余。另一种针对不支持触发器的数据库情况提出了SHA1法,利用SHA1算法对数据记录进行摘要运算,通过前后两次的摘要的对比来提取增量数据,该算法易于实现且速度快。
◆在异构数据转换方法的研究中,通过对常用的转换方法的比较分析,我们提出了一种基于xml模版的异构数据转换方法。在该方法中,利用xml映射来消除命名异构;利用转换表达式来消除单位异构;利用转换函数,映射表来消除精度的异构,数据类型异构以及数据表示方法的异构。这样异构转换信息都保存在Xml模版里,与业务逻辑分开,实现了可扩展性。
◆在分析现有的异构数据集成方法的基础上,针对它们对中小企业进行异构数据集成的不足,提出了一种适用于中小企业的异构数据集成方法:增量数据提取+Ftp传输+XML转换模版+签名加密。客户端程序利用增量提取程序来提取局部数据库增量数据,处理后通过Ftp把数据传递到服务器端。服务器端程序接收到数据后调用Xml转换模版进行异构消除,最后把数据到全局数据库。此方法不需要改动原有的业务系统程序,花费小,数据传输效率高,满足了中小企业异构数据集成的要求。
最后用镇江安监局系统来验证该方法。该系统是在各子系统分散的环境下,实现中小企业异构数据库集成的一个典型例子,它的成功应用表明了我们提出的中小企业异构数据集成方法的可行性及良好的功能特性,达到了预期的目标。
With the gradual procedure of Enterprise Information,information islands can be found in these enterprises. They construct the heterogeneous database structure in the enterprises. How to resolve the problem is a hotspot in computer field.
This paper designs and realizes the SME Heterogeneous Data Integration System, the main contents are as follows:
Having analyzed the traditional method of data extraction, the thesis proposes two kinds of new Incremental data extraction methodes for the Mid- and small-scale enterprise practical system we develop: One is incremental data extraction method of virtual log. By making use of trigger it writes database operation sequence into operation tables, and acquires incremental data by reading operation tables. The means of virtual log reduces the extraction object from row to fields, and optimizes log, reducing the data redundancy. The other is incremental data extraction method of SHA1 under the scene of no trigger support by DBMS, we make use of SHA1 algorithm to calculate the record's summary. According to compare the new summary and old one, we can extract the incremental data. This strategy is simple, runs rapidly.
In the research of heterogeneous data transformation method, Having analyzed and compared the traditional transformation methodes,we present a heterogeneous data transformation method basis of xml template. In this method, we make use of xml mapping to eliminate name heterogeneity,utilize transformation expression to eliminate unit heterogeneity, make use of transformation function and mapping tables to eliminate precision heterogeneity, data type heterogeneity and the heterogeneity of data expression method. We store the heterogeneity transformation information into xml template, which is separated from business system, and realizes the Scalability.
Having analyzed current data integration method, we are aware of that they don't meet the demand of heterogeneous data integration for small and medium-sized enterprises, so we present a data integration method which is feasible to medium-sized enterprises: Incremental data extraction + Ftp transmission+ xml conversion template + XML signature encryption. Client program makes use of incremental Extraction procedure to extract local database incremental data,Afer dealing with the data, transmits the data to server.After the server program receives the data,it will eliminate the data heterogeneity by making use of Xml transformation template,and writes the data into center database. This method doesn't need to modify original business system programming, spends little money, has a good data transmission speed. It meets the demand of small and medium-sized enterprises.
Lastly, the thesis presents an instance of Zhenjiang security monitor system. This system implements the data integration of heterogeneous databases in the environment of decentralized subsystems for small and medium-sized enterprises. The results show that the method of data integration for small and medium-sized which we present is available, it achieves the expectation.
引文
[1]Sunopsis.The future of data integration technologies[R].New York:A META Group White Paper,2004.2
[2]陈跃国,王京春.数据集成综述[J].计算机科学.2004,31(5):48-51
[3]Batini C,Lenzerini M.A comparative analysis of methodologies for database schema integration[J].ACM Computing Surveys.1986,18(4):323-364
[4]Spaccapietra S,Parent C,Dupont Y.Model Independent Assertions for Integration of Heterogeneous Schemas[J].VLDB Journal.1992,1(1):81-126
[5]Halevy A Y.Theory of answering queries using views[J].SIGMOD Record.2000,29(4):40-47
[6]L.M.Haas,E.T.Lin,M.A.Roth Data integration through database federation[J].IBM Systems Journal.2002,41(4):578-596
[7]A.P.Sheth,J.a.Larson.Federated database system for managing distributed,heterogeneous,and autonomous database[J].ACM Computing Surveys.1990,22(3):183-236
[8]R.Jakobovits,Integrating Autonomous Heterogeneous Information Sources[R].Washington:University of Washington Technical Report,1997
[9]Gong Jianya,Wang Yandong.Federated Spatial Databases and Interoperability[J].Wuhan University Journal.2001,6(2):164-168
[10]Convey C,Karpenko O,Tatbul N.Data Integration Services[EB/OL].http://www.cs.brown,edu/people/atbul/cs227/chapter.pdf,2001
[11]Wiederhold G.Mediators in the architecture of future information systems[J].IEEE Computer.1992,25(1):38-49
[12]Garcia-Molina H,Papakonstantion Y,Quass D.The TSIMMIS project:integration of heterogeneous information sources[J].Journal of Intelligent Information Systems.1997,8(2):117-132
[13]DataMirror Corporation.White Paper,Benefits of Transformational Data Integration[DB/OL].http://www,datamirror.com/resourcecenter/,2003-03-23
[14]Chaudhuri S,Dayal U.An overview of data warehousing and OLAP technology[J].SIGMOD Record,1997,26(1):65-74
[15]Vivek R.An Introduction to Data Warehousing[R].Chicago:System Services corporation,1997.8
[16]Widom J.Research problems in data warehousing[A].In Proceedings of the 4th International Conference on Information and Knowledge Management Baltimore[C].Maryland,1995,25-30
[17]Luc Bouganim,Franoise Fabret,C.Mohan,Patrick Valduriez.A dynamic query processing architecture for data integration systems[J].IEEE Computer.2000,23(2):42-48
[18]Hasselbring W.Information System Integration[J].Communications of the ACM.2000,43(6):33-38
[19]Alon Y.Halevy,Anand Rajaraman,Joann Ordille.Data Integration:The Teenage Years[R].Seoul,Korea:Communications of the ACM,2006.9
[20]Zhu Fujun,Turner M,Kotsiopoulos Ⅰ.Dynamic Data Integration Using Web Services[R].Proc.of ICWS'04,2004,262-269
[21]H.T.El-Khatib,M.H.Williams,L.M.MacKinnon,D.H.Marwick.Using a Distributed Approach to Retrieve and Integrate Information from Heterogeneous Distributed Databases[J].Computer Journal.2002,45(4):381-394
[22]韩伟红,贾焰.多数据库系统中的关键技术[J].计算机工程与科学.1999,21(6):49-52
[23]达飞鹏,孟广猛.分布式数据库异构消解研究[J].计算机工程与应用.2004,40(6):187-190
[24]佟俐鹃.异构数据库集成中增量数据提取问题研究[D].北京:北京机械工业学院,2005.3,17-21
[25]者敬.开放式异构数据库复制框架的研究与实现[D].北京:中国科学院,2002.
[26]刘伟,佟俐鹃.异构数据库集成中的变化捕获方案设计[J].计算机应用研究.2005.22(7):213-215
[27]W3C.Web Services Activity[EB/OL].http://www.w3.org/2002/ws/,2007-8
[28]卢正鼎,李兵,肖卫军,等.基于CORBA/XML的多数据库系统研究与实现[J].计算机研究与发展.2002,9(4):443-445
[29]W Emmerich,N Kaveh.Component Technologies:Java Beans,COM,CORBA,RMI,EJB and the CORBA Component Model[A],in Proceedings of 24rd International Conference on Software Engineering[C].Beijing,2002,691-692
[30]D Thompson,D Watkins.Comparisons between CORBA and DCOM:For Distributed computing[A],in Proceeding of the Technology of Object-Oriented Languages[C].1997,127-183
[31]W3C.SOAP Specifications[EB/OL].http://www.w3.org/TR/SOAP,2001
[32]W3C.Web Services Definition Language[EB/OL].http://www.w3.org/TR/wsdl,2001
[33]W3C.UDDI Specifications[EB/OL].http://www.uddi.org/ Specifications html,2001
[34]W3C.Extensible Mark-up Language(XML)1.0,[EB / OL].http://www.w3.org/XML/,2001
[35]孙楠楠、韩银河、许都.一种基于循环展开结构的SHA1算法实现[J].信息技术.2007,31(3):29-31
[36]朱珍.数字签名算法SHA1的C++实现[J].华南金融电脑.2007,15(4):102-104
[37]郭荷清,刘正喜.使用压缩技术改善WebService的传输性能[J].2006,计算机应用与软件.23(6):102-103
[38]杨瑞,蔡虹.提高基于Web Service的分布式系统性能的方法[J].微型机与应用.2005,24(3):8-10
[39]陈荦棋,陈克非.高性能可信Web Service研究[J].计算机工程.2006,32(17):227-229
[40]孟艳红,王育欣,倪天予.数据加密系统的设计与实现[J].沈阳工业大学学报.2007,29(3);340-343
[41]张栓记,傅和平,黄震.数字签名技术的应用研究[J].微计算机信息.2007.23(6):48-50