Replay, slow-motion and multi-camera alignment in remote sports production

How it works

Where replay lives: three models

Replay at the venue. ISO recording and slow-motion servers stay at the venue; the operator either works on site or drives them remotely over a control link, with a low-bitrate preview returning to the production centre. Replay interaction is instant because the operator is working against local storage, and the contribution link carries only the programme output and selected clips rather than every ISO feed continuously.

The cost is on-site footprint and staffing. The equipment and often the operator remain at the venue, which is exactly what remote production exists to eliminate. Remote control of a replay server over a WAN also puts round-trip delay directly into the operator’s hands: a jog wheel with 200 ms of round trip is difficult to use with precision, and no amount of bandwidth fixes it.

Replay at the production centre. All ISO feeds are contributed continuously and replay servers sit centrally. This is the full REMI position: minimum on-site footprint, operators pooled across simultaneous events, replay resources shared rather than duplicated per venue.

The cost is contribution capacity. Every ISO must be carried continuously, not just the ones that end up on air, and the alignment problem described below, which does not exist when all feeds share a fabric.

Hybrid: local ISO, central selection. ISO recording stays at the venue in local circular buffers while a proxy of every angle is contributed to the centre at low bitrate. The operator scrubs and selects against proxies with acceptable responsiveness, and only the selected clip is pulled back at full quality. This is increasingly the dominant model for high-camera-count events because it decouples the two things that were fighting each other: interaction responsiveness needs low delay and low bitrate, while final quality needs high bitrate and tolerates delay.

The cost is complexity. Two representations of every feed, a clip-pull mechanism, and a metadata layer that keeps the proxy and the full-quality source referring to the same frames.

Choosing between them

The determining factors, in order:

Replay latency tolerance. A production where slow-motion appears within a few seconds of an incident constrains the architecture much harder than one where replay is used at breaks in play. Coaching and officiating applications (VAR among them) are the tightest case.

Camera count. Continuous contribution of every ISO scales linearly with camera count and is the binding constraint on centralised replay. Twelve cameras is a different problem from sixty.

Available and reliable contribution capacity. Central replay requires capacity for every ISO simultaneously, with protection, throughout the event. Hybrid requires far less for the proxy layer plus burst capacity for clip pulls.

Operator location and pooling. If the commercial reason for the architecture is pooling replay operators across events, replay interaction must be central, which rules out the venue-local model regardless of its technical merits.

The alignment problem

When every camera feeds a single ST 2110 fabric with PTP, alignment is a solved problem: each essence carries a PTP-referenced timestamp and receivers align on it. Remote production breaks that assumption in three ways.

Feeds travel different paths with different delays. A camera on managed fibre and a camera on bonded cellular arrive at the production centre tens or hundreds of milliseconds apart. The difference is not fixed; it moves with network conditions.

Delay varies per feed over time. SRT recovery buffers, cellular bonding behaviour and path changes all mean each feed’s delay drifts independently during the event.

Not every feed carries usable timing. ST 2110 flows carry PTP-referenced timestamps end to end. A compressed feed over SRT carries transport-stream timing that supports clock recovery but not sample-accurate alignment against a common facility reference unless timestamping is applied deliberately at ingest.

The consequence is that alignment must be re-established at the production centre rather than assumed, and doing so requires three things:

Timestamp at the point of capture, against a common reference. Every feed must carry, or be accompanied by, a timestamp referenced to the same epoch, ideally applied at the venue gateway at the moment of SDI-to-IP conversion or at the encoder input, before the feed enters a transport that will delay it unpredictably. A timestamp applied at ingest into the production centre records when the frame arrived, which is precisely the information alignment needs to discard.

Buffer to the slowest feed. Frame-accurate alignment across a set of feeds requires holding every feed until the slowest has arrived. The alignment buffer must therefore cover the spread between the fastest and slowest feed at their worst case, not the average delay of any individual feed. This is the calculation most often got wrong, and getting it wrong produces alignment that holds under normal conditions and fails under load.

Size against p99, not the mean. Take the 99th-percentile delay of each contributed feed, find the spread between fastest and slowest, and add margin. Per-feed jitter figures alone under-size the buffer, because independent variance on multiple feeds compounds rather than averages.

State whether a quoted variance figure is peak-to-peak or ± before using it. Two feeds each varying over a 20 ms window can sit at opposite ends of it, so the worst-case skew is the full 20 ms spread — but if each is specified as ±20 ms, the same two feeds can be 40 ms apart. The two readings differ by a factor of two, and the buffer gets sized from the wrong one more often than not.

Keeping interaction responsive

Alignment gets the right frames to the right place. It does not make the operator’s controls usable, and those are separate problems with separate solutions.

Proxy scrubbing is the primary mechanism. A low-bitrate, low-latency representation of every angle allows an operator to scrub, mark in and out points, and compare angles with a responsive interface, without carrying full-quality video for material that will never be used. The proxy must be frame-accurately correlated to the full-quality source, which is a metadata problem rather than a video one: every proxy frame must carry a reference that identifies the exact source frame.

Clip pull rather than continuous carriage follows from that. Once selection is made against the proxy, only the marked segment is transferred at full quality. Transfer is a burst (short duration, high bitrate) and the contribution link must be sized to absorb it without disturbing the continuous feeds sharing it. In practice this means either reserved burst capacity or a separate path for clip transfer.

Control-plane latency is separate from media latency and is often worse. A jog wheel, a T-bar and a mark-in keypress all traverse a control path whose round-trip delay determines how the interface feels. Budget and monitor it separately from the media path; a media chain optimised to 150 ms with a control path at 400 ms produces a system that looks good on paper and is unusable in practice.

Return paths need their own budget. Programme return, IFB, tally and talkback all travel back to the venue and are frequently treated as an afterthought. They need dedicated capacity, their own QoS treatment and their own instrumentation. Commentary IFB in particular is unforgiving: a commentator hearing their own voice delayed cannot work.

ISO buffer sizing

Local circular buffers on ISO recorders determine how far back an operator can reach without a file transfer. Sized in seconds of content held in RAM or fast storage, they are the difference between an instant replay and a retrieval operation.

The sizing question is: what is the longest interval between an incident occurring and someone deciding it should be reviewed? For fast-moving sports where replay follows within seconds, a short buffer suffices. For officiating review, where a decision may be revisited after a passage of play has continued, the buffer must reach considerably further back. Buffers in the range of tens of seconds to a couple of minutes cover most cases; the specific figure should come from the production’s own review-latency requirement rather than from a default.

Where multiple angles must be reachable simultaneously, buffer capacity scales with camera count, and the storage tier matters. A buffer that spills to slower storage stops being instant.

Why it matters in broadcast

Replay is where the production’s credibility sits. Viewers forgive a lot, but a slow-motion sequence in which angles are misaligned, or a replay that arrives after the moment has passed, is immediately visible and immediately attributed to the broadcaster.

Officiating applications raise the stakes. Where replay feeds an officiating decision, alignment across angles is not a production nicety. It determines whether the decision can be made at all, and the transport carrying those feeds is accountable in a way that programme video is not.

Replay architecture, not transport, is usually what limits camera count. Contribution capacity for continuous ISO carriage and replay-server capacity at the centre are the two ceilings a growing remote production hits first, which is why the hybrid proxy model has become dominant at high camera counts.

Alignment failures present as intermittent and are hard to diagnose. A buffer sized to average rather than worst-case delay produces a system that works in rehearsal and fails during the busiest passage of the event, and the fault presents as a replay problem rather than as a transport one.

Technical specifications and trade-offs

Replay placement compared

Model	On-site footprint	Contribution capacity required	Replay interaction responsiveness	Operator pooling	Best suited to
Venue-local replay	Highest (servers and usually operators on site)	Lowest (programme and selected clips only)	Instant against local storage	Not possible for replay roles	Latency-critical replay, limited or unreliable contribution capacity
Centralised replay	Lowest	Highest (every ISO, continuously, with protection)	Dependent on contribution and control-path round trip	Full	Well-provisioned links, moderate camera counts, strong pooling case
Hybrid proxy	Low (ISO recording only)	Moderate (proxy layer plus burst for clip pull)	Responsive against proxy	Full	High camera counts, constrained capacity, most large-scale events

Alignment buffer sizing

Required alignment buffer ≈ (p99 delay of slowest feed − p99 delay of fastest feed) + margin

Measure per-feed delay distributions over a representative period, including peak network load, rather than at set-up. Recalculate whenever a feed’s transport changes. Moving one camera from fibre to bonded cellular changes the spread, and therefore the buffer, for every feed in the set.

Separate budgets to specify

Four independent budgets, each with its own tolerance and each needing its own instrumentation:

Path	What it carries	Why it needs separate treatment
Contribution	ISO feeds venue to centre	Sized for camera count and protection; determines alignment spread
Return	Programme, IFB, tally, talkback	Different direction, different QoS, unforgiving latency tolerance for IFB
Control	Replay transport commands, camera control	Round-trip delay determines interface usability, independent of media quality
Clip transfer	Full-quality selected segments	Bursty; must not disturb continuous feeds sharing the link

Planning notes

• Timestamp at capture, not at ingest. A timestamp applied when a feed reaches the production centre records arrival time and is useless for alignment.

• Buffer to the slowest feed’s worst case, using p99 spread across the set rather than per-feed averages.

• Correlate proxy to source at frame level. Proxy scrubbing only works if every proxy frame identifies its exact source frame.

• Measure control-plane round trip separately and treat it as a first-class specification, not a consequence of the media design.

• Reserve or separate clip-transfer capacity so bursts do not disturb continuous contribution.

• Recalculate alignment buffers when any feed’s transport changes, not only at commissioning.

Related approaches and standards

• IEEE 1588 PTP and SMPTE ST 2059 provide the common time reference. Where a feed cannot participate in the PTP domain end to end, its timestamp must be applied against that reference at the venue.

• SMPTE ST 2110-40 carries ancillary data including timecode as a separate essence, which is the route by which capture-time reference survives an IP contribution path.

• SMPTE ST 2110-22 and JPEG XS provide visually lossless, sub-frame-latency contribution, which minimises the alignment spread by keeping contribution delay both low and predictable.

• SMPTE ST 2022-7 provides receiver-side path selection, protecting individual feeds without changing their timing relationship to the rest of the set.

• SRT carries feeds over unmanaged paths with a tunable recovery buffer; that buffer setting is a direct input to the alignment calculation, and changing it changes the spread.

• NMOS (IS-04, IS-05) supports discovery and connection management for the routing changes a multi-venue, multi-event replay operation generates.

How Appear addresses this

Appear’s answer to the alignment problem is to solve it in the chassis rather than leave it to the replay layer. Phase alignment across modules means feeds arriving on separate paths are presented to downstream equipment already synchronised, so a replay server receives a coherent set of angles rather than a set it has to reconcile. That is what makes super-slow-motion review across several cameras behave predictably, and it is the capability Appear leads with for this workflow.

Contribution delay being low and predictable is the other half, and both are properties of the encode and transport layer rather than of the replay system. The X Platform sits at both ends of that path: X5 as a compact edge encoder and decoder at the venue, X10 for mid-scale aggregation, and X20 in 2RU with up to twelve module slots at the production centre. Across the family, modules support AVC and HEVC with selectable latency modes, JPEG XS and JPEG 2000 where visually lossless sub-frame contribution is required, and MPEG-TS over UDP/RTP, accelerated SRT and ST 2022-7 protection on the transport side. ST 2022-1 FEC is available on the IP gateway path; on X5 it applies to output only.

The practical consequence for replay is that the same platform carries managed-network and public-internet paths, so a production mixing fibre, bonded cellular and satellite ISOs is not also mixing vendors and timing models across those paths — which is what makes the spread measurable, and therefore the alignment buffer specifiable.

Recent deployments show the pattern in service: centralised VAR signal transport in Chile with TVTEL using X20 for visually lossless contribution with sub-frame latency and full path redundancy; FOX Sports running REMI and large-scale summer soccer coverage on X20 and X5 systems that flex between compact and full configurations; and Riot Games producing League of Legends and VALORANT esports from a Dublin Remote Broadcast Center on native SMPTE ST 2110 contribution.

XM adds the estate layer across those nodes — inventory, alarm state by severity, software and licence lifecycle, and an API for teams automating their own operational workflows — so a distributed deployment is administered as one estate rather than chassis by chassis. The wider architecture is described on the Remote production page.