daGui: A DataFlow Graphical User Interface
Loading...
Journal Title
Journal ISSN
Volume Title
Perustieteiden korkeakoulu |
Master's thesis
Unless otherwise stated, all rights belong to the author. You may download, display and print this publication for Your own personal use. Commercial use is prohibited.
Author
Date
2017-08-28
Department
Major/Subject
Cloud Computing and Services
Mcode
SCI3081
Degree programme
Master's Programme in ICT Innovation
Language
en
Pages
8+50
Series
Abstract
Big Data is a growing trend. It focuses on storing and processing a vast amount of data in a distributed environment. There are many frameworks and tools which can be used to work with this data. Many of them utilise Directed Acyclic Graphs (DAGs) in some way. A DAG is often used for expressing the dataflow of computation as it offers the possibility to optimise the execution, because it contains the overview of the whole computation. This thesis aims to create an Integrated Development Environment (IDE) like software, which is user-friendly, interactive and easily extendable. The software enables to draw a DAG which represents the dataflow of a program. The DAG is then transformed into launchable source code. Moreover, the software offers a simple way to execute the generated source code. It compiles the code (if necessary), and launches it based on the user's configuration, either on localhost or cluster. The software primarily aims to help beginners learn these technologies, but experts can also use it as visualisation for their workflow or as a prototyping tool. The software has been implemented using Electron and Web technologies, which ensure its platform independence. Its main features are code generation (i.e. translation of a DAG into source code) and code execution. It is created with extensibility in mind, to be able to plug-in support for more frameworks and tools in the future.Description
Supervisor
Heljanko, KeijoThesis advisor
Dowling, JimKeywords
big data, apache spark, DAG, dataflow, GUI