Apache Pinot Deep Dive 2026: User-Facing Analytics, Upserts, and StarTree Cloud

Apache Pinot is the real-time OLAP database that powers LinkedIn’s “Who viewed your profile,” Uber’s restaurant analytics, and Stripe’s merchant dashboards. The original engineering at LinkedIn was specifically about user-facing analytics at hyperscale — millions of users hitting analytical queries with sub-second latency expectations. In 2026, with StarTree Cloud mature, upsert tables in real production, and the comparison against Druid sharpening, Pinot has become the right pick for a specific shape of workload that ClickHouse and Druid both serve less cleanly.

What Apache Pinot actually is#

Pinot is a distributed columnar real-time OLAP database. The architecture has four components:

• Pinot Controllers — cluster metadata, segment assignment, table configuration.
• Pinot Brokers — query routing and scatter-gather across servers.
• Pinot Servers — segment storage and query execution. Split between Real-Time Servers (consuming streams) and Offline Servers (hosting batch-ingested segments).
• Pinot Minions — background tasks (segment merging, optimization, purging).

Plus a Zookeeper cluster for coordination (with Helix as the cluster manager). Plus a deep store (S3, GCS, HDFS) for segment persistence.

Tables come in three flavors: Offline (batch-ingested), Real-Time (streaming-ingested from Kafka or other sources), and Hybrid (both, with automatic stitching at query time). The hybrid model is the headline architectural choice — fresh data lives in Real-Time segments while historical data lives in Offline segments, and the broker merges results transparently.

Real-time ingestion from Kafka#

Pinot’s Stream Ingestion Pipeline reads from Kafka topics with low-latency commit. The path:

1. A Real-Time Server consumes messages from a Kafka partition.
2. Messages accumulate in an in-memory segment (the “consuming” segment).
3. Periodically, the consuming segment is flushed to disk and a new consuming segment starts.
4. Flushed segments are moved to the deep store and become queryable as completed segments.

Sub-second visibility from Kafka write to Pinot query is the default. End-to-end latency under 5 seconds is normal in production deployments we have shipped.

The Stream Ingestion Pipeline supports Kafka, Kinesis, Pulsar, and a few others. Kafka is by far the most common and the most battle-tested.

Upserts and CDC: the headline 2026 capability#

Until 2022, Apache Pinot was append-only. Adding the ability to update existing rows changed what workloads it could serve.

Pinot supports two upsert modes:

• Full upsert — the new row replaces the existing row entirely. Simple model.
• Partial upsert — only specified columns update; the rest remain. Useful for partial state updates.

Upserts use a primary key column (or composite) declared in the table config. The Real-Time Server maintains an in-memory primary-key-to-record-location map. At query time, only the latest version of each key is returned.

What this unlocks:

• Real CDC ingestion. Stream changes from Postgres or MySQL via Debezium, write to Kafka, ingest into Pinot. Pinot reflects the current state of the source tables.
• Stateful event analytics. Track the latest status of orders, deliveries, sessions — not just append events about them.
• User-facing dashboards over mutable entities. “Show me my account balance and recent activity” against a single Pinot table that mirrors the operational store.

The performance cost of upserts is real — write throughput is meaningfully lower than append-only tables, and the primary-key map consumes RAM. For workloads that need it, the trade-off is worth it. For pure event streams, stick with append-only.

User-facing analytics: the actual differentiator#

The reason LinkedIn, Uber, and Stripe picked Pinot is not raw speed — it is the ability to serve sub-second queries at user-facing concurrency. Tens of thousands of queries per second from a public-facing application, each returning in well under a second.

The Pinot features that make this work:

• StarTree index — Pinot’s specialty index that pre-aggregates across high-cardinality dimensions. A single query can hit a StarTree index instead of scanning underlying segments. Game-changer for filtered group-by queries.
• Inverted indexes on dimension columns for fast filtering.
• Range indexes for numeric column filters.
• JSON indexes for nested JSON columns.
• Text indexes (Lucene-backed) for full-text-on-columns.

The combination — pick the right indexes, materialize a StarTree on the high-cardinality dimensions, scale Brokers and Servers for the query concurrency — gets you to the LinkedIn-grade performance profile. It is not magic; it is engineering, and Pinot makes the engineering tractable.

StarTree Cloud: the managed answer#

StarTree (the commercial company behind Pinot) operates StarTree Cloud — managed Apache Pinot on AWS and GCP. The product collapses most of the Pinot operational surface:

• Managed Controllers, Brokers, Servers, Minions, Zookeeper.
• Managed deep store on S3 or GCS.
• Managed Kafka connections.
• Monitoring, alerting, automatic capacity adjustments.
• StarTree-specific tooling (the data manager UI, schema management, segment debugging).

For most teams adopting Pinot in 2026, StarTree Cloud is the right starting point. Self-hosted Pinot is supported and Apache 2.0 licensed, but the operational ramp is meaningfully steeper than ClickHouse Cloud or even self-hosted ClickHouse with the Altinity Operator.

Pinot vs Druid: the 2026 lens#

Druid and Pinot share roots — both came out of late-2000s and early-2010s real-time analytics work at large web companies. They have remained architecturally similar (segmented columnar storage, lambda-style real-time plus batch, scatter-gather queries). In 2026 the differences worth caring about:

• Pinot’s upsert support is more mature than Druid’s. If your workload needs updates, Pinot wins.
• Pinot’s join support is meaningfully better. Lookup joins, broadcast joins, and a recent v2 query engine that handles distributed joins.
• Pinot’s StarTree index is a specialty differentiator for high-cardinality multi-dimensional queries.
• Druid’s streaming ingest is slightly more flexible — better support for diverse sources and out-of-order events.
• Druid’s tooling ecosystem (Imply Polaris, the Druid UI) is more polished in some areas.
• Druid’s Kubernetes operator is more mature; Pinot’s K8s story is catching up.

For user-facing analytics specifically — high concurrency, low latency, mutable state — Pinot is the default in 2026. For time-series-shaped analytical workloads where appends dominate, Druid is still strong.

Hybrid online/offline architecture#

The Pinot hybrid table is the architectural pattern most teams should know about. The setup:

• An Offline table holds historical data, partitioned by date, batch-ingested from S3 Parquet via a Spark or Flink job.
• A Real-Time table holds fresh data, consumed from Kafka.
• A Broker query against the logical table merges results from both, transparently.

What this enables:

• Backfill historical data without touching the streaming path.
• Reprocess and replace old segments without downtime.
• Independent retention policies for hot and cold data.
• Cost optimization — Offline segments can live on cheaper storage, Real-Time on hotter.

This is the LinkedIn-pattern that has scaled to trillions of events. We have shipped a smaller version of it for a logistics analytics workload and it works as advertised.

When to pick Apache Pinot#

Pick Pinot when:

• User-facing analytics at high concurrency (hundreds to thousands of QPS from a public app).
• Sub-second P99 latency on filtered, multi-dimensional queries.
• Mutable state in the analytics workload (CDC, upserts).
• Hybrid streaming-plus-batch ingestion is the natural model.

Pick Druid when:

• Time-series-shaped append-only workloads.
• The Imply Polaris managed product fits your model.
• You already have Druid in production and the upsert/join story is not a blocker.

Pick ClickHouse when:

• Internal analytics or BI workloads (not user-facing at hyperscale).
• SQL flexibility and join performance matter more than the specific real-time ingest profile.
• You want the lightest operational footprint of the three.

The cost story#

StarTree Cloud is priced by capacity tiers; comparable to ClickHouse Cloud and Imply Polaris for similar workloads. Self-hosted Pinot is dramatically cheaper at scale but the operational surface is real.

For a user-facing analytics workload doing thousands of QPS over hundreds of GB of data, expect StarTree Cloud in the low thousands per month. Self-hosted with a competent platform team would be roughly half, but it would consume meaningful engineering attention.

Where pdpspectra fits#

We have shipped Apache Pinot on StarTree Cloud for a logistics platform that exposes real-time analytics to thousands of merchants — sub-second P99 on filtered aggregations across millions of daily shipments. The upsert capability was the reason Pinot beat ClickHouse for that workload.

If you are evaluating Pinot, Druid, or ClickHouse for a real-time analytics product, our data engineering team will benchmark all three against your query patterns and tell you which one we would deploy. The answer depends on your concurrency, your latency budget, and whether you need upserts.

Related reading#

• ClickHouse vs Druid vs Pinot in 2026
• Kafka production patterns
• ClickHouse Cloud vs self-hosted ClickHouse

User-facing analytics at scale is the workload that bought Pinot its production seat. If that is the shape you are building, it deserves a real evaluation. Tell us about the workload and we will help you decide.