ClowdFlows: Online workflows for distributed big data mining

详细信息    查看全文

• 作者：Janez Kranjc^a ; ^b ; ^{janez.kranjc@ijs.si}Author Vitae ; Roman Orač^cAuthor Vitae ; Vid Podpečan^aAuthor Vitae ; Nada Lavrač^a ; ^b ; ^dAuthor Vitae ; Marko Robnik-&Scaron ; ikonja^cAuthor Vitae
• 关键词：Data mining platform ; Cloud computing ; Scientific workflows ; Batch processing ; Map-reduce ; Big data
• 刊名：Future Generation Computer Systems
• 出版年：2017
• 出版时间：March 2017
• 年：2017
• 卷：68
• 期：Complete
• 页码：38-58
• 全文大小：2679 K
• 卷排序：68

文摘

We present a cloud based platform for big data and stream processing with workflows. The ClowdFlows platform enables processing of multiple concurrent data streams. Several machine learning algorithms were implemented in the map-reduce paradigm. Using all data in distributed mode is better than using a subset in non-distributed. The ClowdFlows platform handles big data sets with nearly perfect linear speedup.