ljn

Vector.dev is a log router.



GlassFlow is built for stateful data transformations to ClickHouse

GlassFlow is an open source tool built for real-time and large-scale observability event processing - transform and ingest TBs of data with long-lasting state and enterprise support.

Is Vector suitable for stateful

stream processing?

Is Vector suitable for stateful stream processing?

What is Vector

Vector by Datadog is an open-source tool for observability data pipelines designed to collect, transform, and route observability data - logs and metrics - from various sources into ClickStack and ClickHouse, among other sinks.

What is Vector

Vector by Datadog is an open-source tool for observability data pipelines designed to collect, transform, and route observability data - logs and metrics - from various sources into ClickStack and ClickHouse, among other sinks.

The problem with Vector

Most teams start with Vector because it’s easy for shipping logs. But as volumes grow and complex transformation needs arise, Vector is often misused as a streaming engine.

The problem with Vector

Most teams start with Vector because it’s easy for shipping logs. But as volumes grow and complex transformation needs arise, Vector is often misused as a streaming engine.

The problem with Vector

Most teams start with Vector because it’s easy for shipping logs. But as volumes grow and complex transformation needs arise, Vector is often misused as a streaming engine.

When Should You Use Vector?

When Should You Use Vector?

When Vector is the right choice

Vector is a suitable option for simple log collection, routing, and lightweight event processing. If your goal is to forward logs or apply simple, stateless transformations at the edge, Vector might be efficient and easy to deploy. It works best when state, long time windows, and production guarantees are not required.


Vector is a suitable option for simple log collection, routing, and lightweight event processing. If your goal is to forward logs or apply simple, stateless transformations at the edge, Vector might be efficient and easy to deploy. It works best when state, long time windows, and production guarantees are not required.

When GlassFlow is the

better choice

GlassFlow is built for real stream processing. If you're running Kafka data transformations, multi-day aggregations, or stateful workloads that require durability and observability, GlassFlow provides the architecture, dead letter queue handling, and enterprise SLAs needed for production systems.

To summarize, there are three problems with Vector:

Vector is built primarily for log collection and routing

Vector is not designed for stateful, long-running transformations

Vector does not offer production-grade SLAs

ADDITIONAL RESOURCES

When Vector Becomes Your Streaming Engine

What happens when vertical scaling becomes your bottleneck?

ADDITIONAL RESOURCES

When Vector Becomes Your Streaming Engine

What happens when vertical scaling becomes your bottleneck?

ADDITIONAL RESOURCES

When Vector Becomes Your Streaming Engine

What happens when vertical scaling becomes your bottleneck?

How GlassFlow solves it

How GlassFlow solves it

Built for Transformations

Stateless and stateful transformations

Run real-time stateless and stateful data transformations, including complex Kafka data transformation workloads.

Long time windows (up to 7 days)

Process streaming data with extended time windows for multi-day aggregations and enrichment.

Efficient state store on file

Use a durable on-disk state store built for scalable, reliable stream processing

Built for Transformations

Stateless and stateful transformations

Run real-time stateless and stateful data transformations, including complex Kafka data transformation workloads.

Long time windows (up to 7 days)

Process streaming data with extended time windows for multi-day aggregations and enrichment.

Efficient state store on file

Use a durable on-disk state store built for scalable, reliable stream processing

Built for Transformations

Stateless and stateful transformations

Run real-time stateless and stateful data transformations, including complex Kafka data transformation workloads.

Long time windows (up to 7 days)

Process streaming data with extended time windows for multi-day aggregations and enrichment.

Efficient state store on file

Use a durable on-disk state store built for scalable, reliable stream processing

Built for Production

Enterprise SLAs

Rely on enterprise-grade SLAs for mission-critical data pipelines.

Dead-Letter-Queue

Isolate failed events with a built-in dead letter queue to protect Kafka pipelines.

OTEL pipeline monitoring

Monitor transformation performance and pipeline health with OpenTelemetry (OTEL) observability.

Built for Production

Enterprise SLAs

Rely on enterprise-grade SLAs for mission-critical data pipelines.

Dead-Letter-Queue

Isolate failed events with a built-in dead letter queue to protect Kafka pipelines.

OTEL pipeline monitoring

Monitor transformation performance and pipeline health with OpenTelemetry (OTEL) observability.

Built for Production

Enterprise SLAs

Rely on enterprise-grade SLAs for mission-critical data pipelines.

Dead-Letter-Queue

Isolate failed events with a built-in dead letter queue to protect Kafka pipelines.

OTEL pipeline monitoring

Monitor transformation performance and pipeline health with OpenTelemetry (OTEL) observability.

Built for teams ingesting 10TB to 100TBs per day.

Scales efficiently

Scales efficiently

GlassFlow is proven in real world scenarios

50 TB

Of data processed daily

50 TB

Of data processed daily

414k

Records per second

414k

Records per second

<1

Second latency

<1

Second latency

~ $2.80

Per TB infrastructure cost

~ $2.80

Per TB infrastructure cost

Multi-Pipeline

Horizontal scaling with multiple pipelines

Multi-Pipeline

Horizontal scaling with multiple pipelines

GlassFlow VS Vector.dev comparison

GlassFlow for stateful stream processing; when log routing

with vector isn't enough

Stateless processing

Stateful processing

Late event handling

DLQ

SLAs

ClickHouse

aligned ack

Pipeline Observability

Deployment Service

via file-based stat store

up to 7-day windows with durable state

Full OTEL pipeline monitoring
with metrics & traces

Vector.dev

Limited due to the in-memory store

Frequently asked questions

Frequently asked questions

If you have any questions after reviewing our FAQs, please get in touch.

Can Vector do stateful processing?
Does Vector support dead letter queues?
Is Vector suitable for Kafka transformations?
What is a Vector alternative for stream processing?

Compare Your Vector Transformations with GlassFlow

Compare Your Vector Transformations with GlassFlow

Data transformations at TB scale for ClickHouse

Get query ready data, lower ClickHouse load, and reliable pipelines at enterprise scale.

Data transformations at TB scale for ClickHouse

Get query ready data, lower ClickHouse load, and reliable pipelines at enterprise scale.

Data transformations at TB scale for ClickHouse

Get query ready data, lower ClickHouse load, and reliable pipelines at enterprise scale.