The Connected Ball Is a Lesson in Event Time, and Your Pipeline Is Failing It

Buried in the reporting on this tournament’s officiating technology is one engineering sentence that deserves more attention than the trophy did:

Camera systems generally operate at around 50 frames per second, so the connected ball delivers a much more precise timestamp.

That is the entire justification for putting an inertial measurement unit inside a football and sampling it 500 times a second. Not to find where the ball is — the cameras do that better. To find when it was hit.

If you build data platforms, you have met this problem. You probably lost to it.

Two sensors, two clocks, one question#

FIFA’s offside system needs to evaluate player geometry at a single instant: the moment the ball was played. Two subsystems contribute.

The camera array gives you space. Calibrated stadium cameras, skeletal tracking, limb-accurate player positions. Sampled at roughly 50 Hz, so one sample every 20 ms.

The ball gives you time. A 500 Hz IMU capturing three-dimensional acceleration, so one sample every 2 ms. It has no idea where it is on the pitch. It knows precisely when something struck it.

Neither sensor answers the question alone. The camera knows where everyone was, but only to within a 20 ms window, and a sprinting player covers real ground in 20 ms. The ball knows the exact millisecond of contact but nothing about the defensive line. The answer only exists in the join — and the join is on time.

That is a multi-rate sensor fusion problem, and the reason I keep bringing it up in client work is that the overwhelmingly common industry response to multi-rate data is to destroy the fast signal.

The downsampling reflex#

Here is what a normal team does when handed a 500 Hz stream and a 50 Hz stream that need joining.

They bucket both into a common grain. Usually the coarser one, because the join is trivial that way and every SQL engine on earth cooperates. date_trunc, group by, average the fast signal inside the bucket, join on the key. It runs, it is cheap, it passes review.

And it has thrown away exactly the information the fast sensor was installed to capture. Averaging a 500 Hz acceleration trace into 20 ms buckets converts a sharp, unambiguous impact spike into a slightly elevated value in one bucket — and possibly smeared across two, if the contact straddled a boundary. The signal that made the hard cases decidable is now gone, and it is gone silently. No error, no null, no test failure. Just a model that mysteriously underperforms on precisely the edge cases you built it for.

I have found this pattern in production more times than any other single data defect. It is not a bug anyone writes deliberately. It is what happens when the person designing the join is optimising for the join, not for the question.

The clock skew problem nobody budgets for#

Assume you avoid the downsampling trap and keep both streams at native resolution. You now inherit the harder problem: the two devices do not agree on what time it is.

A 10 cm decision threshold at sprinting speed corresponds to single-digit milliseconds. Which means a few milliseconds of clock skew between the ball’s sensor and the camera array’s frame timestamps is enough to flip calls. Not degrade them — flip them. The system’s advertised precision is a claim about clock discipline as much as about optics.

In enterprise data platforms this shows up constantly and gets misdiagnosed as a data quality issue:

• Event time versus ingestion time. Your device stamped the event at 14:03:07.412. Your Kafka broker received it at 14:03:09.006 because of a retry. Which one is in your fact table? If the answer is “whichever the pipeline defaulted to”, you have a problem you cannot see.
• Timezone and DST poisoning. Naive local timestamps from devices in different regions, joined without normalisation, produce hour-scale skew that looks like seasonality.
• Out-of-order arrival. The fast stream is small and arrives first. The slow stream is big and arrives late. A naive stream join drops the pairing entirely because the counterpart had not landed inside the window.
• Late-arriving corrections. A record gets amended after the fact. Does your aggregate recompute, or does it silently disagree with the source forever?

Watermarking exists to handle exactly this and remains, in my experience, the least-used correct feature in stream processing. Most teams reach for a wider window and hope. A wider window is not a watermark. It trades correctness for latency without telling you the exchange rate.

What good looks like#

The pattern that works, whether the sensors are inside a football or inside a ward:

Preserve native resolution to the last possible layer. Store the raw fast stream. Let downstream consumers aggregate. If your bronze layer is already bucketed, you cannot recover the detail later, and you will need it later. This is why we default to ClickHouse for operational streams — you can keep the raw grain and still get sub-second queries over it, so nobody has a performance excuse to pre-aggregate.

Make event time a first-class column with an explicit contract. Not “there is a timestamp somewhere”. A documented column, stated timezone, stated source clock, stated expected skew bound. Then monitor the skew as a metric.

Join on the event, not the bucket. Interval joins and as-of joins exist. ASOF JOIN in ClickHouse, interval joins in Flink, merge_asof in pandas. They answer “the nearest camera frame to this exact impact” instead of “the bucket both happened to land in”. They are slightly harder to write and dramatically more correct.

Instrument the fusion, not just the inputs. Both source streams can be perfectly healthy while the join silently pairs the wrong rows. Alert on join yield and on the distribution of time deltas between matched pairs, not just on row counts.

Where we hit this outside sport#

In a Hospital Management System, bedside monitors emit vitals at second-or-finer intervals while lab results, medication administration, and clinical notes arrive minutes to hours apart, irregularly, and frequently backdated. Any deterioration model is a multi-rate fusion problem with severe clock hygiene requirements, and “we averaged the vitals hourly to join against labs” is how a promising model turns into an ignored alert.

In a School ERP, attendance is per-period, assessments are per-week, fee events are per-month, and behavioural notes are whenever a teacher gets a free minute. Roll everything to a monthly grain because that is what the report wanted and the early-warning signal disappears into the average.

In logistics, telematics at 1 Hz meets scan events at “whenever the handheld syncs”, and the interesting question — did the temperature excursion happen before or after the handoff — lives entirely in the milliseconds.

The football is a good teacher because the stakes are visible. Get the fusion wrong and 80,000 people in the stadium notice immediately. In your platform, getting it wrong produces a model that is quietly, unfalsifiably mediocre, and nobody ever finds out why.

---

Most “the model underperforms” problems are joins, clocks, and grain — not architecture and not the model. That is where we start. Send us your pipeline and we will tell you where the resolution is going.