Summary

  • RFC 9626 exports a compact description of video frames outside the encrypted payload so an RTP switch can identify boundaries, independent frames, discardable frames and scalable layers without decoding the media.
  • SRTP can authenticate and encrypt those markings. Authentication proves which peer supplied the bits; it does not prove that the bits match the encrypted payload, the receiver's dependency state or the frame that reached the screen.
  • A defensible operating record joins negotiation, visibility policy, sender-side extraction, markings, switch decisions, delivery, decoder state and rendered outcome.

The switch dropped exactly the packets it had been told were safe to lose. Bandwidth fell. The call stayed connected. One participant's video then froze until the next independent frame.

That sequence can describe a network failure, a decoder mismatch or a false frame mark. A dashboard that records only congestion and forwarding has no way to distinguish them.

RFC 9626 addresses a real constraint in modern conferencing. A centralized RTP switch may need to select an active speaker, stop forwarding one stream, start another, or shed enhancement layers under congestion. End-to-end encryption can prevent that switch from reading the video payload. The RFC therefore places a small, codec-neutral description in an RTP header extension.

This is a narrow disclosure in service of a narrow authority. It lets the switch act on frame structure without giving it the picture. It does not turn the switch into an encoder, decoder or independent auditor of the sender's claim.

Five bits describe different kinds of knowledge

The short form carries four signals. S marks the first packet of a frame; E marks the last. I says the frame is independent of temporally prior frames. D says the sender knows the frame can be discarded while leaving a decodable stream.

The long form adds B, a base-layer synchronization point, plus TID, LID and TL0PICIDX. TID identifies the temporal layer. LID identifies a spatial or quality layer. TL0PICIDX labels a base-layer picture or states which such picture an enhancement layer depends upon.

These fields are compact because they omit more than they include. An I mark concerns temporal independence; a spatial enhancement layer can still depend on a co-timed lower layer. A higher TID or LID is a relative place in a hierarchy, not an absolute frame rate, resolution or bitrate. Without extra structure information, the identifiers do not enumerate every dependency.

Even the apparently simple B signal can cross an evidentiary boundary. RFC 9626 says it cannot always be derived reliably from simple H.264 or H.265 payload-header inspection. An implementation may need an internal codec interface. The exported bit can therefore depend on state that the switch will never see and a conformance test must deliberately capture.

The obligation lives at the sender

The RFC states that header-extension values must represent what is already in the RTP payload. For VP9 and VP8, several fields can be checked against payload descriptors. H.264 and H.265 have their own mappings and exceptions. Future codecs are expected to specify their mappings too.

That is an interoperability rule. It is not a self-verifying construction.

Once private media is encrypted, a switch cannot generally reopen the payload to prove that I really describes an independent frame or that D really describes a disposable one. It consumes an assertion produced before or alongside encryption. If the sender maps a codec signal incorrectly, uses stale internal state or contains a defect, the authenticated extension can faithfully carry the wrong description.

This distinction is easy to lose because SRTP authentication is strong evidence. It can show that an accepted peer supplied the extension and that the protected packet was not altered unnoticed. It cannot promote a sender's assertion into an independent observation of codec truth. Authentic and accurate are different properties.

The appropriate test point is therefore close to the encoder. Before encryption removes the switch's inspection ability, compare payload descriptors or internal codec events with every emitted marking. Include negative cases: a non-independent frame marked I, a referenced frame marked D, incomplete spatial layers at a proposed switching point, discontinuous indices and boundaries that disagree with timestamps or marker semantics.

Forwarding is a decision, not an outcome

RFC 9626 recommends dropping frames marked D or frames in the highest temporal and spatial/quality layers when congestion requires shedding work. It recommends beginning a newly forwarded stream at a point where I is set across all spatial layers.

Those rules are sensible only inside their evidence boundary. A D bit says what the sender believes about decodability. It does not contain the receiver's actual reference buffer, packet-loss history or implementation defects. An I bit across layers gives the switch an eligible entry point. It does not prove that every packet in those layers arrived before the playout deadline.

The same caution applies to refresh. RFC 9627 lets a receiver request a layer refresh, using layer identifiers aligned with RFC 9626. A Full Intra Request can seek a new switching point. But a request is an instruction in a control loop. It is not evidence that the encoder acted, that the refresh traversed the network or that the decoder used it.

For leadership, the distinction defines the minimum useful receipt. Preserve what the switch saw, why it chose to forward or drop, and what happened next at the receiver. “The SFU followed the marking” closes the decision question. It does not close the delivery or viewing question.

Encryption has two audiences

The extension creates a deliberate privacy choice. Under SRTP, header extensions are authenticated and may be encrypted. An RTP switch that is an SRTP peer can use an encrypted marking while remaining unable to read end-to-end encrypted private media. Other middleboxes on the path cannot read that extension.

An endpoint can instead leave the marking unencrypted so additional middleboxes can optimize dropping. The cost is reliable exposure of frame and layer timing. Keyframes can often be guessed statistically from encrypted packet sizes because they tend to be larger. RFC 9626 nevertheless draws the right distinction: an explicit unencrypted mark converts an inference into a dependable signal.

That reliability changes the privacy surface. A record of independent-frame intervals, layer changes and frame boundaries can reveal media activity patterns even when it never exposes pixels. The decision is not “encrypted video or clear video.” It is which operational metadata is visible to which party for which optimization.

RFC 9335 supplies the mechanism for completely encrypting RTP header extensions and contributing sources. RFC 8871 supplies the private-media conferencing context. Neither makes the policy choice automatically. Offer/answer direction, SRTP-peer topology, extension encryption and retention still need an accountable owner.

One URI creates a versioned dependency

The SDP URI urn:ietf:params:rtp-hdrext:framemarking negotiates the extension through the machinery of RFC 8285. A local identifier then appears on packets. That mapping can change across sessions and extension direction can change through updates.

An operator therefore needs the negotiation record, not just packet captures. The same numeric extension identifier can have no portable meaning outside the session that assigned it. A switch acting on an unnegotiated, inactive or differently mapped extension is not following RFC 9626 merely because the packet contains familiar bits.

Codec evolution adds another dependency. RFC 9628 maps VP9's scalable structure into the same vocabulary, while future payload formats must state how their layer identifiers map. An implementation update can change both the media encoder and the code that extracts markings. Treat that pair as one versioned release surface and test the correspondence again.

Sources