Kafka Streams Code Examples for Confluent Platform

If you’re new to Kafka Streams, here is a curated list of resources to get you started.

Confluent for VS Code

Use the Confluent for VS Code extension to generate a new Kafka Streams application that reads messages from a Kafka topic, performs a simple transformation, and writes the transformed data to another topic. This option is ideal if you’re learning about Kafka Streams.

For more information, see Confluent for VS Code with Confluent Platform.

Getting started examples

There are numerous Kafka Streams examples on Confluent Developer that provide full code examples with step–by-step instructions.

The code repository for the Kafka Streams examples is available on GitHub.

Routing

• Split a stream into substreams

• Merge many streams into one

• Choose the output topic dynamically

• Filter a stream of events

• Find distinct events in a time window

Transforming

• Uppercasing a stream of events

• Convert a KStream to a KTable

• Transform a stream of events

• Find distinct events in a time window -

Aggregations

• Count events by grouping key

• Sum a stream of events

• Find the min/max

• Calculate a running average

• Cogroup aggregates over multiple streams

Joins

• Join a KStream and a KTable

• Join a KStream and a GlobalKTable

• Join a KStream with another KStream

• Join a KTable with another KTable

• Join two KTables with a foreign key

• Multi-join expressions

• Versioned KTables for temporal join accuracy

Windowing and time operations

• Tumbling windows

• Sliding windows

• Session windows

• Emit a final result

Miscellaneous

• Reprocess input topics

• Name stateful operations

• Schedule operations

• Expire data in a KTable with TTLs

• Expire data in a KTable created from aggregate()

• Handle deserialization, processing, and production exceptions

• Handle uncaught exceptions

Interactive Queries examples

Since Confluent Platform 3.1+ and Kafka 0.10.1+, it is possible to query state stores created via the Kafka Streams DSL and the Processor API. For further information, see Kafka Streams Interactive Queries for Confluent Platform.

Java

With lambda expressions for Java 8+:

• Word Count with Kafka Streams Interactive Queries

Event-Driven Microservice example

Java

The Event-Driven Microservice example implements an Order Service that provides a REST interface to POST and GET orders. Posting an order creates an event in Kafka, which is picked up by three different validation engines: a Fraud Service, an Inventory Service, and an Order Details Service. These services validate the order in parallel, emitting a PASS or FAIL based on whether each validation succeeds.

• Self-paced Kafka Streams tutorial based on the microservices example above: deployed in an event streaming platform with all the services in Confluent Platform and interconnecting other end systems

Related content

• GitHub: Confluent Code Tutorials

• Kafka Streams Data Types and Serialization

• Kafka Streams Domain Specific Language

• Kafka Streams Processor API

Note

This website includes content developed at the Apache Software Foundation under the terms of the Apache License v2.