On construction of a distributed data storage system in cloud

详细信息    查看全文

• 作者：Chao-Tung Yang ; Wen-Chung Shih ; Chih-Lin Huang ; Fuu-Cheng Jiang…
• 关键词：Cloud computing ; Distributed data storage ; Data as a service ; Distributed file system ; 68M14 ; 65Y05 ; 68P20
• 刊名：Computing
• 出版年：2016
• 出版时间：January 2016
• 年：2016
• 卷：98
• 期：1-2
• 页码：93-118
• 全文大小：2,300 KB
• 参考文献：1.Cloud computing, http://​en.​wikipedia.​org/​wiki/​Cloud_​computing#Infrastructure . Accessed 2 Apr 2014
  2.Milojičić D, Llorente IM, Montero RS (2011) OpenNebula: a cloud management tool. IEEE Internet Comput 15(2):11–14
  3.Sempolinski P, Thain D (2010) A Comparison and Critique of Eucalyptus, OpenNebula and Nimbus. Cloud Computing Technology and Science (CloudCom), 2010 IEEE Second International Conference, pp 417–426
  4.Cordeiro T, Damalio D, Pereira N, Endo P, Palhares A, Gonçalves G, Sadok D, Kelner J, Melander B, Souza V, Mångs J-E (2010) Open source cloud computing platforms. Grid and Cooperative Computing (GCC) 2010 9th International Conference, pp 366–371
  5.Truong HL, Dustdar S (2009) On analyzing and specifying concerns for data as a service. Services Computing Conference, 2009. APSCC 2009. IEEE Asia-Pacific, pp 87–94
  6.Truong HL, Dustdar S (2010) On evaluating and publishing data concerns for data as a service. Services Computing Conference (APSCC), 2010 IEEE Asia-Pacific, pp 363–370
  7.Dapeng J, Liu C, Wang D, Liu H, Tang Z (2009) Performance comparison of IP-networked storage. Tsinghua Sci Technol 14(1):29–40CrossRef
  8.Wang D, Meeting Green Computing Challenges (2007) High density packaging and microsystem integration. HDP ’07. International Symposium, pp 1–4
  9.Mackey G, Sehrish S, Jun W (2009) Improving metadata management for small files in HDFS. Cluster Computing and Workshops, 2009. CLUSTER ’09. IEEE International Conference, pp 1–4
  10.Shafer J, Rixner S, Cox AL (2010) The hadoop distributed filesystem: balancing portability and performance. IEEE, Houstan, pp 122–133
  11.Jiang L, Li B, Song M (2010) The optimization of HDFS based on small files. Broadband Network and Multimedia Technology (IC-BNMT), 2010 3rd IEEE International Conference, pp 122–133
  12.Barlet-Ros P, Iannaccone G, Sanjuas-Cuxart J, Sole-Pareta J (2011) Predictive resource management of multiple monitoring applications. Netw IEEE/ACM Trans 19(3):788–801CrossRef
  13.Cheng Guang, Gong Jian (2007) A resource-efficient flow monitoring system. Commun Lett IEEE 11(6):558–560CrossRef
  14.School of Computer Science Northwestern Polytechnical University Xi’an, China (2009) An adaptive resource monitoring method for distributed heterogeneous computing environment. Parallel and Distributed Processing with Applications, 2009 IEEE International Symposium, pp 40–44
  15.Miettinen T, Pakkala D, Hongisto M (2008) A method for the resource monitoring of OSGi-based software components. Software Engineering and Advanced Applications, 2008. SEAA ’08. 34th Euromicro Conference, pp 100–107
  16.Wang CC, Chen YM, Weng CH, Chung TY (2006) An overlay resource monitor system. Advanced Communication Technology, 2006. ICACT 2006. The 8th International Conference, vol 3, pp 5
  17.Düllmann D, Hoschek W, Jaen-Martinez J, Segal B (2001) Model for replica synchronization and consistency in a data grid. The IEEE International Symposium on High Performance Distributed Computing, San Francisco, pp 67–75
  18.Xu P, Huang X, Wu Y, Liu L, Zheng W (2009) Campus cloud for data storage and sharing. Grid and Cooperative Computing, 2009. GCC ’09. Eighth International Conference, pp 244–249
  19.Zeng W, Zhao Y, Song W (2009) Research on cloud storage architecture and key technologies. ICIS 2009, ACM, Nov 24–26
  20.Ying Z, Yong S (2009) Cloud storage management technology. In: Proceedings of the 2009 Second International Conference on Information and Computing Science, pp 309–311, May 21–22
  21.Hirofuchi T, Nakada H, Ogawa H, Itoh S, Sekiguchi S (2009) A live storage migration mechanism over wan and its performance evaluation. In: Proceedings of the 3rd international workshop on Virtualization technologies in distributed computing, June 15–15, 2009, Barcelona, Spain
  22.Bertino E, Maurino A, Scannapieco M (2010) Guest editors’ introduction: data quality in the internet aera. IEEE Internet Comput 14:11–13CrossRef
  23.Carns P, Lang S, Ross R, Vilayannur M, Kunkel J, Ludwig T (2009) Small-file access in parallel file systems. In: Proceedings of the 23rd IEEE International Parallel and Distributed Processing Symposium, pp 1–11
  24.Amazon S3, http://​en.​wikipedia.​org/​wiki/​Amazon_​S3 . Accessed 2 Apr 2014
  25.Amazon Simple Storage Service, http://​aws.​amazon.​com/​s3/​ . Accessed 2 Apr 2014
  26.EgeCast, http://​www.​edgecast.​com/​ . Accessed 2 Apr 2014
  27.Chang F, Dean J, Ghemawat S, Hsieh WC, Wallach DA, Burrows M, Chandra T, Fikes A, Gruber RE (2008) Bigtable: a distributed storage system for structured data. ACM Trans Comput Syst 26(2):4CrossRef
  28.Ghemawat S, Gobioff H, Leung ST (2003) The google file system. SOSP’03: Proceedings of the nineteenth ACM symposium on Operating systems principles. ACM Press, New York, pp 29–43
  29.Ceph, http://​ceph.​newdream.​net/​ . Accessed 2 Apr 2014
  30.Gu Y, Lu L, Robert G, Andy Y (2010) Processing massived sized graphs using sector/sphere. 3rd Workshop on Many-Task Computing on Grids and Supercomputers, co-located with SC10. LA, New Orleans 15
  31.Gu Y, Robert G (2009) Sector and sphere: the design and implementation of a high performance data cloud. Theme Issue Philos Trans R Soc A Crossing Bound Comput Sci E-Sci Glob E-Infrastruct 367(1897):2429–2445
  32.SAN, http://​en.​wikipedia.​org/​wiki/​Storage_​area_​network . Accessed 2 Apr 2014
  33.NAS, http://​en.​wikipedia.​org/​wiki/​Network-attached_​storage . Accessed 2 Apr 2014
  34.iSCSI, http://​en.​wikipedia.​org/​wiki/​ISCSI . Accessed 2 Apr 2014
  35.NFS, http://​en.​wikipedia.​org/​wiki/​Network_​File_​System_​(protocol) . Accessed 2 Apr 2014
  36.OpenNeBula, http://​opennebula.​org/​ . Accessed 2 Apr 2014
  37.Ctrix XenServer, http://​www.​citrix.​com/​ . Accessed 2 Apr 2014
  38.Lo CTD, Qian K (2010) Green computing methodology for next generation computing scientists. Computer Software and Applications Conference (COMPSAC), 2010 IEEE 34th Annual, pp 250–251
  39.Giroire F, Guinand F, Lefevre L, Torres J (2010) Energy-aware, power-aware, and green computing for large distributed systems and applications. High Performance Computing and Simulation (HPCS) 2010 International Conference, pp 4–47
  40.Zhong B, Feng M, Lung CH (2010) A green computing based architecture comparison and analysis. Green Computing and Communications (GreenCom), 2010 IEEE/ACM Int’l Conference on and Int’l Conference on Cyber, Physical and Social Computing (CPSCom), pp 386–391
  41.Richardson L, Ruby S(2007) Restful web services, 1st edn. O’Reilly Media, May 15
  42.RFC 2616, http://​tools.​ietf.​org/​html/​rfc2616 . Accessed 2 Apr 2014
  43.Roy FT, Gettys J, Mogul JC, Frystyk NH, Masinter L, Leach P, Berners-Lee J (1999) RFC 2616: Hypertext Transfer Protocol–HTTP/1.1
  44.NIO, http://​en.​wikipedia.​org/​wiki/​New_​I/​O . Accessed 2 Apr 2014
  45.JSR 203: More New I/O APIs for the JavaTM Platform (“NIO.2”) (2009) The Java Community Process(SM) Program-JSRs: Java Specification Requests. Retrieved May 23, 2009
  46.MapReduce, http://​en.​wikipedia.​org/​wiki/​MapReduce/​ . Accessed 2 Apr 2014
  47.MPI, http://​en.​wikipedia.​org/​wiki/​Message_​Passing_​Interface . Accessed 2 Apr 2014
  48.Wake on LAN, http://​en.​wikipedia.​org/​wiki/​Wake_​on_​lan . Accessed 2 Apr 2014
  49.C2DM, https://​code.​google.​com/​p/​chrometophone/​ . Accessed 2 Apr 2014
  50.Intent service, http://​developer.​android.​com/​reference/​android/​app/​IntentService.​html . Accessed 2 Apr 2014
  51.dd, http://​en.​wikipedia.​org/​wiki/​Dd_​(Unix) . Accessed 2 Apr 2014
  52.Wang L, Chen D, Hu Y, Ma Y, Wang J (2013) Towards enabling cyberinfrastructure as a service in clouds. Comput Electr Eng 39(1):3–14CrossRef
  53.Wang L, Chen D, Zhao J, Tao J (2012) Resource management of distributed virtual machines. IJAHUC 10(2):96–111CrossRef
  54.Wang L, Kunze M, Tao J, von Laszewski G (2011) Towards building a cloud for scientific applications. Adv Eng Softw 42(9):714–722CrossRef
  55.Wang L, Chen D, Huang F (2011) Virtual workflow system for distributed collaborative scientific applications on grids. Comput Electr Eng 37(3):300–310CrossRef
  56.Wang L, von Laszewski G, Kunze M, Tao J, Dayal J (2010) Provide virtual distributed environments for grid computing on demand. Adv Eng Softw 41(2):213–219MATH CrossRef
  57.Wang L, von Laszewski G, Tao J, Kunze M (2010) Virtual data system on distributed virtual machines in computational grids. IJAHUC 6(4):194–204CrossRef
  58.Wang L, Tao J, Ranjan R, Marten H, Streit A, Chen J, Chen D (2013) G-Hadoop: MapReduce across distributed data centers for data-intensive computing. Future Gener Comput Syst 29(3):739–750CrossRef
  59.Zhang W, Wang L, Liu D, Song W, Ma Y, Liu P, Chen D (2013) Towards building a multi-datacenter infrastructure for massive remote sensing image processing. Concurrency Comput Pract Exp 25(12):1798–1812CrossRef
  60.Ma Y, Wang L, Liu D, Yuan T, Liu P, Zhang W (2013) Distributed data structure templates for data-intensive remote sensing applications. Concurrency Comput Pract Exp 25(12):1784–1797CrossRef
  61. http://​en.​wikipedia.​org/​wiki/​Moore's_​law . Accessed 2 Apr 2014
  
• 作者单位：Chao-Tung Yang (1)
  Wen-Chung Shih (2)
  Chih-Lin Huang (1)
  Fuu-Cheng Jiang (1)
  William Cheng-Chung Chu (1)
  
  1. Department of Computer Science, Tunghai University, Taichung, Taiwan ROC
  2. Department of Applied Informatics and Multimedia, Asia University, Taichung, Taiwan ROC
  
• 刊物类别：Mathematics and Statistics
• 刊物主题：Mathematics
  Computational Mathematics and Numerical Analysis
  
• 出版者：Springer Wien
• ISSN：1436-5057

文摘

In the past, people have focused on cluster computing and grid computing. Now, however, this focus has shifted to cloud computing. Irrespective of what techniques are used, there are always storage requirements. The challenge people face in this area is the huge amount of data to be stored, and its complexity. People are now using many cloud applications. As a result, service providers must serve increasingly more people, causing more and more connections involving substantially more data. These problems could have been solved in the past, but in the age of cloud computing, they have become more complex. This paper focuses on cloud computing infrastructure, and especially data services. The goal of this paper is to implement a high performance and load balancing, and able-to-be-replicated system that provides data storage for private cloud users through a virtualization system. This system extends and enhances the functionality of the Hadoop distributed system. The proposed approach also implements a resource monitor of machine status factors such as CPU, memory, and network usage to help optimize the virtualization system and data storage system. To prove and extend the usability of this design, a synchronize app was also developed running on Android based on our distributed data storage. Keywords Cloud computing Distributed data storage Data as a service Distributed file system