管道漏磁检测数据压缩技术的研究
|
摘要
为了提高现有输油管道漏磁检测系统的检测精度以及扩展系统的适用范围使其适用于天然气输送管道的检测，需要通过减小采样时间间隔和空间间隔的方法以获得更加完备的漏磁场信息。由此产生的海量数据会超出现有系统的处理能力，数据压缩是解决此问题的有效手段。由于新型管道漏磁检测器将以现场可编程门阵列器件(FPGA：Field Programmable Gates Array)作为系统核心，数据压缩算法必须适合于硬件实现。本文结合课题，在对数据压缩的原理和方法进行深入分析的基础上，分别利用霍夫曼编码和小波编码等方法实现了管道漏磁检测数据的压缩。压缩算法中不使用乘除法运算，非常适合于硬件实现并具有很好的实时性；通过对重要数据和非重要数据分别进行无损压缩和有损压缩，算法在保留所有检测相关数据的基础上可以达到10：1以上的压缩比。
本文在绪论部分介绍了课题的来源和意义以及国内外在管道漏磁检测数据压缩方向上的研究情况。第二章通过对管道漏磁检测原理和检测器结构的介绍，说明漏磁检测数据的特征并结合检测结果的分析方法提出对管道漏磁检测数据的检测无损压缩要求。第三章简介数据压缩的有关理论和方法。第四章讨论了课题中采用的霍夫曼编码、预测编码和小波变换编码的理论和方法。在第五章通过数据变化阈值和数据动态范围阈值判断数据重要性，分别采用行差分数据的霍夫曼编码方法以及提升小波变换和层次树集分割(SPIHT)编码方法实现了数据的检测无损压缩。第六章首先介绍了FPGA的结构和设计流程，然后介绍了压缩算法硬件实现的有关内容。总结部分对整个论文工作进行了总结。
In order to enhance the measure precision and expand the using area of the existing oil pipeline MFL(Magnetic Flux Leakage) detecting system to natural gas pipeline testing, detailed information of flux obtained by decreasing sampling time and space interval is needed. The huge amount of data produced by this will exceed the processing ability. Data compression is an effective way of solving this problem. The data compression algorithm must be fit to hardware implementation for the new pipeline magnetic flux detecting system will adopt FPGA (Field Programmable Gates Array) as the system kernel. This paper, combined with the task and based on the thorough analyses of the theories and methods of data compression, realize the data compression of the MFL inspection data using Huffman coding and wavelet transform coding. Those compressions algorithms are suited to implementation by hardware and have good realtime quality because they don't have multiplication and division operation. Those algorithms can achieve c
ompression ration over 10 to 1 at the same time of keeping all the important detecting correlative data by using lossless method to compress important data and using lossy method to compress insignificant data.
At the exordium, this paper introduces the task's origin and significance and introduces the research situation on pipeline MFL data compression at home and at abroad. The second chapter explains the character of MFL data by introducing the principle of pipeline MFL inspection and the structure of MFL detector, and then brings forward the diagnostically lossless compression request by combine discuss the analyzing method of inspection results. The third chapter briefly introduces the theory and methods of data compression. The fourth chapter discusses the Huffman coding method, anticipating coding method and wavelet transform coding method used in this paper. The fifth chapter use Huffman coding method of the row difference data and lifting wavelet transform and SPIHT (Set Partitioning in Hierarchical Tree) coding method to realize the diagnostically compression of MFL data by judging the significance of data using changing threshold and dynamic area threshold of MFL data. The
sixth chapter introduces the structure of FPGA and design flow at first and then introduces the related content of the hardware implementation of compression algorithms. Finally, The conclusion part summarizes the whole work of the paper.
本文在绪论部分介绍了课题的来源和意义以及国内外在管道漏磁检测数据压缩方向上的研究情况。第二章通过对管道漏磁检测原理和检测器结构的介绍，说明漏磁检测数据的特征并结合检测结果的分析方法提出对管道漏磁检测数据的检测无损压缩要求。第三章简介数据压缩的有关理论和方法。第四章讨论了课题中采用的霍夫曼编码、预测编码和小波变换编码的理论和方法。在第五章通过数据变化阈值和数据动态范围阈值判断数据重要性，分别采用行差分数据的霍夫曼编码方法以及提升小波变换和层次树集分割(SPIHT)编码方法实现了数据的检测无损压缩。第六章首先介绍了FPGA的结构和设计流程，然后介绍了压缩算法硬件实现的有关内容。总结部分对整个论文工作进行了总结。
In order to enhance the measure precision and expand the using area of the existing oil pipeline MFL(Magnetic Flux Leakage) detecting system to natural gas pipeline testing, detailed information of flux obtained by decreasing sampling time and space interval is needed. The huge amount of data produced by this will exceed the processing ability. Data compression is an effective way of solving this problem. The data compression algorithm must be fit to hardware implementation for the new pipeline magnetic flux detecting system will adopt FPGA (Field Programmable Gates Array) as the system kernel. This paper, combined with the task and based on the thorough analyses of the theories and methods of data compression, realize the data compression of the MFL inspection data using Huffman coding and wavelet transform coding. Those compressions algorithms are suited to implementation by hardware and have good realtime quality because they don't have multiplication and division operation. Those algorithms can achieve c
ompression ration over 10 to 1 at the same time of keeping all the important detecting correlative data by using lossless method to compress important data and using lossy method to compress insignificant data.
At the exordium, this paper introduces the task's origin and significance and introduces the research situation on pipeline MFL data compression at home and at abroad. The second chapter explains the character of MFL data by introducing the principle of pipeline MFL inspection and the structure of MFL detector, and then brings forward the diagnostically lossless compression request by combine discuss the analyzing method of inspection results. The third chapter briefly introduces the theory and methods of data compression. The fourth chapter discusses the Huffman coding method, anticipating coding method and wavelet transform coding method used in this paper. The fifth chapter use Huffman coding method of the row difference data and lifting wavelet transform and SPIHT (Set Partitioning in Hierarchical Tree) coding method to realize the diagnostically compression of MFL data by judging the significance of data using changing threshold and dynamic area threshold of MFL data. The
sixth chapter introduces the structure of FPGA and design flow at first and then introduces the related content of the hardware implementation of compression algorithms. Finally, The conclusion part summarizes the whole work of the paper.
引文
[1]王镛根，张西锋．长输管线泄漏检测方法综述，信息与控制，1993，22(2)，105～107
[2]余浩然，吴斌，陈丽萍．漏磁通法油气管道在役检测技术，实用测试技术，1997，第五期，1～9
[3]J. B. Nestleroth. Magnetic Flux Leakage (MFL) Technology for Gas Transmission Pipeline Inspection, Final Report GRI-00-0180. 1993 though 1999
[4]高文．多媒体数据压缩技术，电子工业出版社，1994
[5]吴乐南，徐孟侠．《数据压缩的原理与应用》，电子工业出版社，1995
[6]Celoxica 公司网页介绍。 Http://www.celoxica.com
[7]W.Tham, S.Wooley. The detection and segmentation of pipeline inspection features for diagnostically lossless data compression IST/SPIE Electronic Imaging 2001 21-26 January 2001, San Jose, California, USA
[8]杨理践，冯海英．基于双正交样条小波的管道漏磁信号数据压缩技术(沈阳工业大学学报，2001年第六期)
[9]杨理践，王玉梅，冯海英．智能化管道漏磁检测装置的研究．无损检测，2002，24(3)100～102
[10]MAGNETIC FLUX PIPE INSPECTION APPARATUS FOR ANALYZING ANOMALIES IN A PIPELINE WALL. United States Patent,Patent Number 5,864:232
[11]Battelle公司网页教程。Http://www.battelle.org/pipetechnology／MFL
[12]PII公司网页文章。 Http://www.pipetronix.de
[13]Jerry D. Gibson，多媒体数字压缩原理与标准，电子工业出版社，2000
[14]吴铮．小波编码技术及其在机载多光谱遥感图像压缩中的应用，硕士学位论文，西北工业大学，2001
[15]冯海英．管道漏磁检测中实时数据压缩技术的研究，硕士学位论文，沈阳工业大学，2002
[16]黄贤武．数字图像处理与压缩编码技术，电子科技大学出版社，2000
[17]K．R．Castleman．数字图像处理，电子工业出版社，1998
[18]姚天仁．现代数字信号处理．华中理工大学出版社，1999
[19]J.W.Woods, S.D.O.Neil, Sub band coding of images, IEEE Trans ASSP-34 (5),1278～1288, 1986。
[20]Yang X, Wang K, Shamma SA, Auditory Representation of Acoustic Signals, IEEE Trans Inform. Theory, 1992 38(2): 824～839。
[21]A.Lewis, G.Knowles. Image Compression Using the 2-D wavelet transform. IEEE Trans. on Image Processing. Apr. 1992, IP-1, 2: 244～250。
[22]Weiss L.G, Wavelets and Wideband Correlation processing, IEEE SP Jan 1994:13～32。
[23]胡昌华．基于MATLAB的系统分析与设计．西安电子科技大学出版社，2000
[24]杨福生．小波变换的工程分析与应用，科学出版社，1999．
[25]周军．小波变换理论与图像压缩编码的研究，上海交通大学研究生学位论文，1997。
[26]J.M. Shapiro, embedded image coding using zerotrees of wavelets coefficients, IEEE Trans. Signal Prossing, vol. 41,pp.3445-3462
[27]A. Said, W.A. Pearlman, “A new fast and efficient image codee based on set partitioning in hierarchical trees", IEEE Trans. On Circuits and Systems for VideoTechnology, Vol. 6, June 1996.
[28]D. Taubman, "High performance scalable image compression with EBCOT", Proc. IEEE Int Conference on Image Processing(ICIP), 24-28 October 1999, Kobe, Japan.
[29]C.M. Brislawn, J. N. Bradley, R. J. Onyshczak, and T. Hopper, "The {FBI} compression standard for digitized fingerprint images." in Appl. Digital Image Process. XIX, vol. 2847 of Proc. SPIE, (Denver, CO). pp. 344--355, SPIE, Aug. 1996.
[30]David Gibson, Michael Spann and Sandra I. Woolley, Diagnostically LOssless 3D Wavelet Compression for Digital Angiogram Video (poster) IEEE Data Compression Conference 2002, Snowbird, Utah, Proceedings of the Data Compression Conference, ISBN 0-7695-1477-4, pg 454
[31]W. Sweldens, "The lifting scheme: a custom-design construction of biothogonal wavelets," Appl. Comput. Harmonic. Analysis, vol. 3, no. 2. pp 186-200, 1996.
[32]W. Sweldens, "The lifting scheme: construction of second generation wavelets," SIAM J.Math. Anal., vol. 29, no. 2, pp 511-546, 1997.
[33]ISO/IEC, ISO/IEC 15444-1/WG1: Information technology--JPEG 2000 image coding system----Part 1: Core coding system. 2000.
[34]曾建芬．用提升格式构造二次平均插值小波，中国图形图像学报，vol7，4，2002，369—375
[35]M.D. Adams and F. Kossentini, "Reversible integer-to-integer wavelet transforms for image compression: performance evaluation and analysis," IEEE Trans. on Image Processing, 2000.
[36]A.R. Calderbank, I. Daubechies, W. Sweldens, and B.-L. Yeo, "Wavelet transforms that map integers to integers," Applied and Computational Harmonic Analysis, vol, 5, no. 3, pp. 332-369, July 1998.
[37]贺翼虎．基于快速整数提升小波变换的硬件设计 电视技术 6，2002，78—80
[38]徐志军．CPLD／FPGA的开发与应用，电子工业出版社，2002
[39]XILINX公司产品数据手册Spartan-Ⅱ 2.5V FPGA．Http://www.xilinx.com
[40]XILINX公司产品数据手册ISE Foundation．Http://www.xilinx.com
[41]FTDI公司产品数据手册 FT8U245AM．Http://www.ftdichip.conm
[42]程子敬．非定长码高速实时拼接专用集成电路的研制，电子技术应用，1，2002，53—57
[2]余浩然，吴斌，陈丽萍．漏磁通法油气管道在役检测技术，实用测试技术，1997，第五期，1～9
[3]J. B. Nestleroth. Magnetic Flux Leakage (MFL) Technology for Gas Transmission Pipeline Inspection, Final Report GRI-00-0180. 1993 though 1999
[4]高文．多媒体数据压缩技术，电子工业出版社，1994
[5]吴乐南，徐孟侠．《数据压缩的原理与应用》，电子工业出版社，1995
[6]Celoxica 公司网页介绍。 Http://www.celoxica.com
[7]W.Tham, S.Wooley. The detection and segmentation of pipeline inspection features for diagnostically lossless data compression IST/SPIE Electronic Imaging 2001 21-26 January 2001, San Jose, California, USA
[8]杨理践，冯海英．基于双正交样条小波的管道漏磁信号数据压缩技术(沈阳工业大学学报，2001年第六期)
[9]杨理践，王玉梅，冯海英．智能化管道漏磁检测装置的研究．无损检测，2002，24(3)100～102
[10]MAGNETIC FLUX PIPE INSPECTION APPARATUS FOR ANALYZING ANOMALIES IN A PIPELINE WALL. United States Patent,Patent Number 5,864:232
[11]Battelle公司网页教程。Http://www.battelle.org/pipetechnology／MFL
[12]PII公司网页文章。 Http://www.pipetronix.de
[13]Jerry D. Gibson，多媒体数字压缩原理与标准，电子工业出版社，2000
[14]吴铮．小波编码技术及其在机载多光谱遥感图像压缩中的应用，硕士学位论文，西北工业大学，2001
[15]冯海英．管道漏磁检测中实时数据压缩技术的研究，硕士学位论文，沈阳工业大学，2002
[16]黄贤武．数字图像处理与压缩编码技术，电子科技大学出版社，2000
[17]K．R．Castleman．数字图像处理，电子工业出版社，1998
[18]姚天仁．现代数字信号处理．华中理工大学出版社，1999
[19]J.W.Woods, S.D.O.Neil, Sub band coding of images, IEEE Trans ASSP-34 (5),1278～1288, 1986。
[20]Yang X, Wang K, Shamma SA, Auditory Representation of Acoustic Signals, IEEE Trans Inform. Theory, 1992 38(2): 824～839。
[21]A.Lewis, G.Knowles. Image Compression Using the 2-D wavelet transform. IEEE Trans. on Image Processing. Apr. 1992, IP-1, 2: 244～250。
[22]Weiss L.G, Wavelets and Wideband Correlation processing, IEEE SP Jan 1994:13～32。
[23]胡昌华．基于MATLAB的系统分析与设计．西安电子科技大学出版社，2000
[24]杨福生．小波变换的工程分析与应用，科学出版社，1999．
[25]周军．小波变换理论与图像压缩编码的研究，上海交通大学研究生学位论文，1997。
[26]J.M. Shapiro, embedded image coding using zerotrees of wavelets coefficients, IEEE Trans. Signal Prossing, vol. 41,pp.3445-3462
[27]A. Said, W.A. Pearlman, “A new fast and efficient image codee based on set partitioning in hierarchical trees", IEEE Trans. On Circuits and Systems for VideoTechnology, Vol. 6, June 1996.
[28]D. Taubman, "High performance scalable image compression with EBCOT", Proc. IEEE Int Conference on Image Processing(ICIP), 24-28 October 1999, Kobe, Japan.
[29]C.M. Brislawn, J. N. Bradley, R. J. Onyshczak, and T. Hopper, "The {FBI} compression standard for digitized fingerprint images." in Appl. Digital Image Process. XIX, vol. 2847 of Proc. SPIE, (Denver, CO). pp. 344--355, SPIE, Aug. 1996.
[30]David Gibson, Michael Spann and Sandra I. Woolley, Diagnostically LOssless 3D Wavelet Compression for Digital Angiogram Video (poster) IEEE Data Compression Conference 2002, Snowbird, Utah, Proceedings of the Data Compression Conference, ISBN 0-7695-1477-4, pg 454
[31]W. Sweldens, "The lifting scheme: a custom-design construction of biothogonal wavelets," Appl. Comput. Harmonic. Analysis, vol. 3, no. 2. pp 186-200, 1996.
[32]W. Sweldens, "The lifting scheme: construction of second generation wavelets," SIAM J.Math. Anal., vol. 29, no. 2, pp 511-546, 1997.
[33]ISO/IEC, ISO/IEC 15444-1/WG1: Information technology--JPEG 2000 image coding system----Part 1: Core coding system. 2000.
[34]曾建芬．用提升格式构造二次平均插值小波，中国图形图像学报，vol7，4，2002，369—375
[35]M.D. Adams and F. Kossentini, "Reversible integer-to-integer wavelet transforms for image compression: performance evaluation and analysis," IEEE Trans. on Image Processing, 2000.
[36]A.R. Calderbank, I. Daubechies, W. Sweldens, and B.-L. Yeo, "Wavelet transforms that map integers to integers," Applied and Computational Harmonic Analysis, vol, 5, no. 3, pp. 332-369, July 1998.
[37]贺翼虎．基于快速整数提升小波变换的硬件设计 电视技术 6，2002，78—80
[38]徐志军．CPLD／FPGA的开发与应用，电子工业出版社，2002
[39]XILINX公司产品数据手册Spartan-Ⅱ 2.5V FPGA．Http://www.xilinx.com
[40]XILINX公司产品数据手册ISE Foundation．Http://www.xilinx.com
[41]FTDI公司产品数据手册 FT8U245AM．Http://www.ftdichip.conm
[42]程子敬．非定长码高速实时拼接专用集成电路的研制，电子技术应用，1，2002，53—57