Summary

  • RFC 3984 separated H.264 picture data from the sequence and picture parameter sets that made the data decodable; a slice could name its rules without carrying them.
  • The operational risk was a configuration race: a set could be missing, arrive after dependent media, be overwritten while old packets remained in flight, or collide with another writer’s use of the same identifier.

The packet capture looks reassuring. RTP sequence numbers are continuous. Timestamps advance. The payload bytes match what left the sender. Yet the receiver produces no stable picture. If the investigation asks only whether the media packets arrived, it has started one layer too low.

RFC 3984 described the RTP payload format for H.264 video in February 2005. Its status record, errata record and Datatracker history show that it was later obsoleted by RFC 6184. The historical interest is not merely how it split large NAL units or aggregated small ones. It is the control state that picture packets assumed but did not necessarily contain.

H.264 moved information used by many slices into parameter sets. A sequence parameter set describes properties of a coded video sequence. A picture parameter set supplies properties used for coded pictures. Between them sit facts such as picture dimensions, coding modes and slice-group mapping. A slice header carries identifiers selecting the relevant sets. That saves repetition, but it also creates indirection: the identifier is a reference, not the referenced rules.

The decoder therefore has a timeline of its own. Before a dependent NAL unit reaches decoding order, the required parameter sets must be available under the expected identifiers. A packet can be on time according to RTP and still be too early according to decoder state. A packet can be intact while the identifier it names has acquired a new meaning. Transport correctness and interpretive readiness are separate claims.

The boundary is visible in RFC 3550. RTP sequence numbers support loss detection and reconstruction of packet order; timestamps represent sampling instants and aid playback timing and synchronization. The status page, errata page and Datatracker record do not turn either field into a version number for decoder configuration. RTP can tell an observer much about delivery without saying which SPS or PPS occupied a receiver’s slot.

RFC 3984 organized parameter-set carriage around three principles. Under Principle A, sets travel reliably out of band before the RTP session. Under Principle B, updated sets travel reliably out of band while the session is running. Under Principle C, sets travel in band with the media. The original document recommended reliable out-of-band delivery for the first two cases; in-band delivery needed defenses such as repetition, retransmission or forward error correction because losing one parameter set could damage interpretation of many later NAL units.

For initial state, the media-type parameter sprop-parameter-sets could carry base64 representations of parameter-set NAL units. Those sets had to precede NAL units that referred to them in decoding order. The attribute was not a capability declaration. It described state the stream expected to use; it did not prove every configuration the receiver could accept. Treating it as a capability list could therefore negotiate a stream on evidence the receiver never offered.

Out-of-band carriage connected the media problem to session signaling. RFC 4566 defines SDP as a description format, while its status, errata and Datatracker record preserve the specification history. RFC 3264 defines offer/answer exchanges and permits only one outstanding offer at a time; see also its status, errata and Datatracker record. Serialization helps coordinate a change. It does not make SDP a universal delivery receipt for decoder installation.

RFC 3984’s practical advice was stricter: when updated parameter sets travelled reliably out of band, the sender should wait for acknowledgment of the signaling before sending NAL units that depended on them. The order matters because acknowledgment creates a boundary before media cutover. Even then, it proves only what the signaling system says it proves. A production design still has to define when receipt becomes installation and when installation becomes active decoder state.

Identifier reuse made the race more dangerous. Suppose a picture parameter set with one identifier defines the old picture configuration. The sender overwrites that identifier with a new definition while old dependent NAL units remain in the network or the receiver’s buffers. Those older slices still carry the same small reference, but the decoder now resolves it to a different rule set. No packet checksum needs to fail. The corruption is semantic.

The specification therefore recommended selecting an identifier unused for a sufficiently long period, or adding a new identifier instead of overwriting an active one. The safe interval is not merely a network round trip. It includes packets in flight, reordering, jitter buffers and decoder retention. In a multiparty session, the ownership problem grows: two otherwise well-behaved sources can choose the same identifier unless a common controller partitions or rewrites the namespace.

Mixing delivery channels also creates stale state. RFC 3984 said Principles B and C should not be combined in one session without sufficient synchronization. A late joiner may begin with the out-of-band set and miss an in-band update. A receiver that loses the update may keep decoding against the old version while subsequent picture packets arrive cleanly. Partitioning identifiers between channels is one defense because it prevents two paths from silently assigning different meanings to the same slot.

The later standard changed the preferred mechanism without dissolving the control problem. RFC 6184, its status, errata and Datatracker history allow both in-band and out-of-band parameter-set transport and add negotiation such as in-band-parameter-sets. The 2005 recommendation for a general out-of-band preference should not be projected forward as present doctrine. What survived was the need to declare the method and coordinate configuration with dependent media.

RFC 7798 later addressed HEVC over RTP, with its own status, errata and Datatracker record. It is useful comparative evidence: compressed-video payload formats repeatedly confront state whose effect spans many media units. It is not evidence that every implementation behaved alike or that the earlier format was universally deployed.

The security implication follows the same asymmetry. Corrupting one ordinary picture packet may damage a bounded region. Corrupting or injecting a parameter set can change how many future NAL units are interpreted. RFC 3984 therefore discusses integrity protection and source authentication. The blast radius belongs to state, not merely to packet size.

For incident response, a “complete packet capture” is incomplete unless it includes the configuration story. Preserve SDP exchanges, acknowledgments, in-band parameter-set NAL units, identifier assignments, payload-type transitions and the receiver’s buffer horizon. Ask when each SPS/PPS version became active, who owned the identifier, and whether older dependants had drained. The decisive evidence may never appear in the picture packets themselves.

RFC 3984’s enduring lesson is a systems lesson. Content and the rules for interpreting content can move through different channels, under different reliability assumptions, on different clocks. A reference inside a packet does not collapse those clocks. Reliable operation begins when a system treats configuration readiness as its own state machine rather than as an optimistic side effect of media arrival.

Sources