Gartner Research

Streaming Architectures With Kafka

Published: 26 September 2019

ID: G00387169

Analyst(s): Sumit Pal , Sanjeev Mohan

Summary

Organizations are shifting from traditional processing on databases and batch to a streaming-first approach. Event brokers like Kafka are becoming critical for such solutions. This research helps data and analytics technical professionals understand and implement Kafka-based solutions.

Table Of Contents

Analysis

  • What Is Kafka?
    • Detailed Kafka Architecture
    • Publishing Data to Kafka
    • Consuming Data From Kafka
    • Exactly Once and Transaction Support
    • Kafka Brokers
    • Rebalancing
    • Interfacing With Kafka
  • Kafka Ecosystem
  • Kafka Use Cases
    • ETL/ELT With Kafka
    • Change Data Capture
    • Kafka as a Database
    • Kafka for Event-Driven Architectures
  • Kafka Alternatives
    • Kafka and RabbitMQ
    • Kafka and Pulsar
    • Kafka, Amazon Kinesis, Azure Event Hubs and Google Cloud Pub/Sub — A Quick Comparison
  • Strengths
  • Weaknesses

Guidance

  • Best Practices
  • Deciding on Kafka
  • Deployment Options
    • Kafka on Kubernetes
  • Scaling Kafka
  • Performance Tuning
  • Monitoring Kafka
  • Schema Registry
  • Security in Kafka
  • Testing With Kafka
  • Tools in the Kafka Ecosystem

Gartner Recommended Reading

©2019 Gartner, Inc. and/or its affiliates. All rights reserved. Gartner is a registered trademark of Gartner, Inc. and its affiliates. This publication may not be reproduced or distributed in any form without Gartner’s prior written permission. It consists of the opinions of Gartner’s research organization, which should not be construed as statements of fact. While the information contained in this publication has been obtained from sources believed to be reliable, Gartner disclaims all warranties as to the accuracy, completeness or adequacy of such information. Although Gartner research may address legal and financial issues, Gartner does not provide legal or investment advice and its research should not be construed or used as such. Your access and use of this publication are governed by Gartner’s Usage Policy. Gartner prides itself on its reputation for independence and objectivity. Its research is produced independently by its research organization without input or influence from any third party. For further information, see Guiding Principles on Independence and Objectivity.

Already have a Gartner Account?

Become a client

Learn how to access this content as a Gartner client.