基于Docker的射电干涉阵软件系统敏捷封装与部署
|
摘要
随着天文技术的发展,天文数据处理软件的需求也不断更迭变化,导致软件运行环境渐趋复杂。对于开发者和使用者,急需提出一种对复杂天文数据处理软件敏捷化封装和部署的方法。我国明安图射电频谱日像仪已进入常规观测,与之配套的数据处理软件也已完成开发并投入使用。由于该软件的部署涉及操作系统环境、图形处理器运行环境及底层依赖软件等配置问题,导致安装过程既繁琐又容易出错。结合容器技术的特点,提出了一种基于Docker容器对日像仪软件系统进行敏捷封装与部署的方法,并对该方法的设计进行介绍,通过实验验证了其可用性,以及相比于传统虚拟机可获得较优异的性能表现。该方法可为未来天文数据处理软件的封装部署提供参考。可以预见,未来容器技术将成为天文海量数据处理的基础支撑技术。
With the development of astronomical technology,the demand for astronomical data processing software is changing constantly,resulting in the complexity of software running environment.For developers and users,it is urgent to find a method for agile packaging and deployment of complex astronomical data processing software.Mingantu Ultrawide Spectral Radio Heliograph(MUSER) has been used for routine observation,and the data processing software for MUSER project has also been completed and put into use.As the deployment of this software involves the configuration problems of the operating system environment,the GPU running environment and the underlying dependence on software,the installation process is cumbersome and error-prone.According to the characteristics of container technology,in this paper we present a method of agile packaging and deployment for MUSER software system based on Docker container,and the design of this method is introduced.We verify the usability of this method by experiments,and the superior performance can be obtained by comparing with the traditional virtual machine.The method proposed in this paper can provide a reference for the future deployment and encapsulation of astronomical data processing software.It is foreseeable that future container technology will become the basic supporting technology for astronomical and massive data processing.
With the development of astronomical technology,the demand for astronomical data processing software is changing constantly,resulting in the complexity of software running environment.For developers and users,it is urgent to find a method for agile packaging and deployment of complex astronomical data processing software.Mingantu Ultrawide Spectral Radio Heliograph(MUSER) has been used for routine observation,and the data processing software for MUSER project has also been completed and put into use.As the deployment of this software involves the configuration problems of the operating system environment,the GPU running environment and the underlying dependence on software,the installation process is cumbersome and error-prone.According to the characteristics of container technology,in this paper we present a method of agile packaging and deployment for MUSER software system based on Docker container,and the design of this method is introduced.We verify the usability of this method by experiments,and the superior performance can be obtained by comparing with the traditional virtual machine.The method proposed in this paper can provide a reference for the future deployment and encapsulation of astronomical data processing software.It is foreseeable that future container technology will become the basic supporting technology for astronomical and massive data processing.
引文
[1]YAN Y,ZHANG J,WANG W,et al.The Chinese Spectral Radioheliograph-CSRH[J].Earth Moon&Planets,2009,104(1/4):97-100.
[2]周鑫磊,王威,王锋,等.基于QT的MUSER观测数据多屏图形化实时显示的设计与实现[J].天文研究与技术,2015,12(4):503-509.
[3]石聪明,张晓丽,王锋,等.MUSER的负数据库接口设计与实现[J].天文研究与技术,2018,15(2):169-175.
[4]WANG F,MEI Y,DENG H,et al.Distributed data-processing pipeline for mingantu ultrawide spectral radioheliograph[J].Publications of the Astronomical Society of the Pacific,2015,127(950):383-396.
[5]蔡志平,陈彬,肖侬,等.虚拟计算环境中的虚拟网络[J].计算机工程与科学,2008(11):1-4,8.
[6]DUA R,RAJA A R,KAKADIA D.Virtualization vs containerization to support PaaS[C]//IEEE International Conference on Cloud Engineering.IEEE,2014:610-614.
[7]NGUYEN N,BEIN D.Distributed MPI cluster with Docker Swarm mode[C]//IEEE 7th Annual Computing and Communication Workshop and Conference(CCWC).IEEE,2017:1-7.
[8]WEI S L,WANG F,DENG H,et al.OpenCluster:a flexible distributed computing framework for astronomical data processing[J].Publications of the Astronomical Society of the Pacific,2016,129(972):024001(16pp).
[9]冯勇,陈坤,邓辉,等.基于OpenCL的MUSERCLEAN算法研究与实现[J].天文学报,2017,58(2):55-64.
[10]陈泰燃,王威,王锋,等.基于MPI的高性能UVFITS数据合成研究与应用[J].天文研究与技术,2016,13(2):184-189.
[11]WALTERS J P,YOUNGE A J,KANG D I,et al.GPU passthrough performance:a comparison of KVM,Xen,VMWare ESXi,and LXC for CUDA and OpenCL applications[C]//IEEE 7th International Conference on Cloud Computing.2014:636-643.
[2]周鑫磊,王威,王锋,等.基于QT的MUSER观测数据多屏图形化实时显示的设计与实现[J].天文研究与技术,2015,12(4):503-509.
[3]石聪明,张晓丽,王锋,等.MUSER的负数据库接口设计与实现[J].天文研究与技术,2018,15(2):169-175.
[4]WANG F,MEI Y,DENG H,et al.Distributed data-processing pipeline for mingantu ultrawide spectral radioheliograph[J].Publications of the Astronomical Society of the Pacific,2015,127(950):383-396.
[5]蔡志平,陈彬,肖侬,等.虚拟计算环境中的虚拟网络[J].计算机工程与科学,2008(11):1-4,8.
[6]DUA R,RAJA A R,KAKADIA D.Virtualization vs containerization to support PaaS[C]//IEEE International Conference on Cloud Engineering.IEEE,2014:610-614.
[7]NGUYEN N,BEIN D.Distributed MPI cluster with Docker Swarm mode[C]//IEEE 7th Annual Computing and Communication Workshop and Conference(CCWC).IEEE,2017:1-7.
[8]WEI S L,WANG F,DENG H,et al.OpenCluster:a flexible distributed computing framework for astronomical data processing[J].Publications of the Astronomical Society of the Pacific,2016,129(972):024001(16pp).
[9]冯勇,陈坤,邓辉,等.基于OpenCL的MUSERCLEAN算法研究与实现[J].天文学报,2017,58(2):55-64.
[10]陈泰燃,王威,王锋,等.基于MPI的高性能UVFITS数据合成研究与应用[J].天文研究与技术,2016,13(2):184-189.
[11]WALTERS J P,YOUNGE A J,KANG D I,et al.GPU passthrough performance:a comparison of KVM,Xen,VMWare ESXi,and LXC for CUDA and OpenCL applications[C]//IEEE 7th International Conference on Cloud Computing.2014:636-643.