Kafka to ClickHouse connector. Native. Fast. Reliable.

GlassFlow Kafka to ClickHouse connector is open-source, with built-in retries and efficient batch sizes.

Managed Connectors

The Kafka and the ClickHouse connectors are built and updated by the GlassFlow team.

High Performance

The connectors are created for optimal throughput and native support.

Clean Data

You can dedupe and join Kafka streams within GlassFlow before ingesting to ClickHouse. Auto retries make sure your data is up-to-date.

Comparison

See in detail how GlassFlow performs compared to alternative solutions

Open source

Quick to start

Low maintanance

Latency

Built-in stateful processing

Error handling

Transformation support

Very low

Built in retries with backoff

Advanced

ClickHouse Kafka Table Engine

Very low

Basic

Limited

Clickpipes for

Kafka

Very low

Built-in retries and monitoring

Basic

Go
Service

Very low

Learn how to stream data from Kafka to ClickHouse using Kafka Engine, ClickPipes, or Kafka Connect. Understand when to use each.

From Kafka to ClickHouse: Get all details.

How does it work?

Supports multiple Kafka topics and partitions

GlassFlow natively supports consuming from multiple Kafka topics and partitions in parallel, ensuring high-throughput and scalable ingestion. It automatically handles partition assignment, offset tracking, and rebalancing behind the scenes. This allows you to build unified pipelines that process data from various sources without manual coordination.

Adjustable waiting times for optimal throughput

GlassFlow lets you configure wait times between batch reads from Kafka, allowing you to control how often data is flushed downstream. By adjusting this interval, you can optimize the trade-off between latency and throughput based on your workload. This flexibility helps maximize performance without overwhelming downstream systems like ClickHouse.


Configurable batch sizes

GlassFlow allows you to set configurable batch sizes for reading and processing data from Kafka, tailoring the amount of data handled in each batch. This helps balance between processing efficiency and memory usage, adapting to different workload demands. By tuning batch sizes, you can optimize pipeline throughput and reduce latency based on your system’s capacity and performance goals.

Frequently asked questions

Feel free to contact us if you have any questions after reviewing our FAQs.

Do you have a demo?

We have prepared several demo setups that you can run yourself locally or in the cloud. You can find them here.

Do you have a demo?

We have prepared several demo setups that you can run yourself locally or in the cloud. You can find them here.

Do you have a demo?

We have prepared several demo setups that you can run yourself locally or in the cloud. You can find them here.

Which datatypes are supported?

So far we support JSON but are working more data types like Avro or Protobuf.

Which datatypes are supported?

So far we support JSON but are working more data types like Avro or Protobuf.

Which datatypes are supported?

So far we support JSON but are working more data types like Avro or Protobuf.

Can you handle nested JSON?

Yes, you can read from nested JSON with GlassFlow and convert to a flat table for ClickHouse.

Can you handle nested JSON?

Yes, you can read from nested JSON with GlassFlow and convert to a flat table for ClickHouse.

Can you handle nested JSON?

Yes, you can read from nested JSON with GlassFlow and convert to a flat table for ClickHouse.

How do retries work?

GlassFlow’s ClickHouse connector includes built-in retry mechanisms that automatically attempt to resend failed data batches to ClickHouse. Retries use exponential backoff to avoid overwhelming the database during transient errors or network issues. This ensures reliable data ingestion without manual intervention, maintaining pipeline stability and data integrity.

How do retries work?

GlassFlow’s ClickHouse connector includes built-in retry mechanisms that automatically attempt to resend failed data batches to ClickHouse. Retries use exponential backoff to avoid overwhelming the database during transient errors or network issues. This ensures reliable data ingestion without manual intervention, maintaining pipeline stability and data integrity.

How do retries work?

GlassFlow’s ClickHouse connector includes built-in retry mechanisms that automatically attempt to resend failed data batches to ClickHouse. Retries use exponential backoff to avoid overwhelming the database during transient errors or network issues. This ensures reliable data ingestion without manual intervention, maintaining pipeline stability and data integrity.

How do I self-host GlassFlow?

We have several hosting options. You can find them here.

How do I self-host GlassFlow?

We have several hosting options. You can find them here.

How do I self-host GlassFlow?

We have several hosting options. You can find them here.

Cleaned Kafka Streams for ClickHouse

Clean Data. No maintenance. Less load for ClickHouse.

Cleaned Kafka Streams for ClickHouse

Clean Data. No maintenance. Less load for ClickHouse.

Cleaned Kafka Streams for ClickHouse

Clean Data. No maintenance. Less load for ClickHouse.