Using Kafka to build scalable and fault tolerant systems
Korhonen, Teemu (2019)
Korhonen, Teemu
2019
All rights reserved. This publication is copyrighted. You may download, display and print it for Your own personal use. Commercial use is prohibited.
Julkaisun pysyvä osoite on
https://urn.fi/URN:NBN:fi:amk-2019121025846
https://urn.fi/URN:NBN:fi:amk-2019121025846
Tiivistelmä
The objective of this study was to use Apache Kafka to improve and expand an existing messaging queue by introducing an aspect of scalability and developing its fault tolerance capabilities. The main objective of improving the messaging queue is to improve its data managing potential in terms of performance and reliability by allowing the project to be scaled to a large server system.
The theoretical basis of the study looks in to messaging concepts and more specifically how information can be shared by sending messages between software applications using a specific set of messaging standards. The study also looks closely at the structure and features of Apache Kafka and how it accomplishes this task. The study explores different alternatives to managing the way Kafka handles its messages through topics, partitions and offsets.
During the development phase the project was thoroughly tested through a large number of JUnit tests which allowed for multiple bugs to be eliminated and resulted in a better understanding of how Kafka functions at a deeper level. The end result of the study was a fully functioning messaging queue implemented into the Java project through the use of Apache Kafka.
The theoretical basis of the study looks in to messaging concepts and more specifically how information can be shared by sending messages between software applications using a specific set of messaging standards. The study also looks closely at the structure and features of Apache Kafka and how it accomplishes this task. The study explores different alternatives to managing the way Kafka handles its messages through topics, partitions and offsets.
During the development phase the project was thoroughly tested through a large number of JUnit tests which allowed for multiple bugs to be eliminated and resulted in a better understanding of how Kafka functions at a deeper level. The end result of the study was a fully functioning messaging queue implemented into the Java project through the use of Apache Kafka.