# GlassFlow

> GlassFlow is the only open-source streaming ingestion and data transformation system purpose-built for ClickHouse. It makes data query-ready in real time, handling TB-scale workloads with stateful and stateless transformations, native connectors, and low operational overhead.

GlassFlow sits between your data sources (Kafka, Postgres, OpenTelemetry, etc.) and ClickHouse, fixing data correctness — deduplication, joins, filtering, normalisation — before ClickHouse ever sees the data. It runs on Kubernetes and is designed for engineers who need reliable, high-throughput streaming ETL without managing complex clusters.

## Key capabilities

- Stateful transformations: deduplication, joins with time-windowed context, late event handling
- Stateless transformations: filter, normalize, reformat, map nested JSON, derive fields
- Native ClickHouse connector using the ClickHouse native protocol
- Dead Letter Queue to isolate faulty events without interrupting pipeline flow
- Built-in pipeline observability
- Linear horizontal scaling (tested to 51k RPS per replica)
- Sub-millisecond latency for 80% of transformations
- Runs on Kubernetes; no proprietary cluster to manage

## Integrations / Connectors

- Sources: Apache Kafka, OpenTelemetry (OTLP), PostgreSQL, and more
- Sink: ClickHouse (native protocol)
- Full connector list: https://www.glassflow.dev/integrations

## Pricing & Plans

- **Community (Open Source):** Free, self-managed, open-source connectors, Auth0, community Slack & docs support
- **Enterprise:** Custom connectors, RBAC, your own auth framework, 24/7/365 SLA, implementation engineer, ISO 27001 & SOC 2 Type II compliance, self-managed or deployed by GlassFlow

## Use cases

- Real-time ClickHouse ingestion from Kafka at TB scale
- Observability pipelines (OpenTelemetry → GlassFlow → ClickHouse)
- Event deduplication before ClickHouse ingestion
- PII masking and data normalisation in-stream
- Low-cost self-hosted observability stacks

## Important links

- Homepage: https://www.glassflow.dev
- Documentation: https://docs.glassflow.dev
- GitHub (clickhouse-etl): https://github.com/glassflow/clickhouse-etl
- Product overview: https://www.glassflow.dev/product
- Integrations: https://www.glassflow.dev/integrations
- Pricing: https://www.glassflow.dev/pricing
- Case studies: https://www.glassflow.dev/case-studies
- Blog: https://www.glassflow.dev/blog
- Live demo: https://demo.glassflow.dev
- Contact / Book a demo: https://www.glassflow.dev/contact
- Deployment guide: https://docs.glassflow.dev/installation
- Architecture & performance: https://docs.glassflow.dev/architecture/performance
- Transformations reference: https://docs.glassflow.dev/configuration/transformations
- Release notes: https://docs.glassflow.dev/release-notes/v2.5.0

## Company

GlassFlow is open-source and built by data engineers with backgrounds at enterprise data companies. The project is actively maintained and production-tested at enterprise scale.

- GitHub: https://github.com/glassflow