Summary

  • The current NETCONF and RESTCONF Trace Context drafts say malformed tracing metadata should not normally make a management RPC fail; the RESTCONF example creates the resource, returns 201 Created, discards the old tracestate and begins a new unsampled trace.
  • That choice preserves the control operation at the cost of end-to-end correlation. A trace gap is not evidence that the change failed, and an unbroken trace is not evidence that the change was authorised, committed or effective.
  • The missing control is a reconciliation receipt that binds the authenticated request and protocol result to any replacement trace, server-side transaction evidence, configuration readback and observed service outcome.

The change that disappeared from its own story

Imagine the most inconveniently successful maintenance action. An orchestrator sends a RESTCONF request to create a resource. The server accepts it. The response is 201 Created, with a location and an entity tag. The new configuration is visible when the operator reads it back.

Yet the trace console shows no child span beneath the orchestration job. Searching by the original trace identifier finds the controller and then silence. The natural incident-room conclusion is that the device never saw the request. It is also wrong.

Revision 11 of the IETF NETCONF working group's RESTCONF Trace Context draft contains almost exactly this boundary as a worked example. The incoming traceparent uses a version the server cannot parse, and its tracestate is malformed. The server does not reject the resource operation. It returns 201 Created, supplies a fresh version-00 traceparent, clears the trace flags to zero and removes tracestate. The API call succeeds; the distributed trace does not continue under the caller's identity.

This is not an accidental contradiction. It is a policy choice. Both the RESTCONF draft and its NETCONF sibling say it is not recommended to reject an RPC because of Trace Context values. Tracing is meant to observe management. A defect in observational metadata should not casually acquire the authority to veto management work.

That sensible separation creates an evidence problem of its own. The resource can exist while the original trace says nothing about its creation. If the operator treats the trace as the transaction ledger, reality and the dashboard part company.

One request, several identifiers

Distributed tracing is attractive because it gives one operation a recognizable thread across an orchestrator, controllers and network elements. W3C Trace Context supplies traceparent for the trace and parent relationship and tracestate for optional vendor-specific context. The NETCONF draft carries those ideas as XML attributes; the RESTCONF draft uses HTTP headers.

Neither field is the managed object. The NETCONF draft says this directly: Trace Context is not related to the data carried by the operation, including configurations, service identifiers or state. The identifier tells a tracing system where an observation belongs. It does not say what was authorised, what bytes reached a datastore, or what the network later did.

Other identifiers answer narrower questions. A NETCONF message-id associates an RPC reply with its request on that protocol session. An HTTP status and resource location describe the RESTCONF server's response. An implementation may have a local transaction identifier. A YANG Library capability can show that the server declares support for the tracing extension. NACM can decide whether the authenticated principal may perform an operation. None of these records substitutes for the others.

The standards therefore allow a valid operational sequence with two trace identities. The caller has an old trace. The receiving server finds it unusable, treats the request as having no valid parent, and creates a new trace for local observation. W3C processing rules require orphaned tracestate to be discarded when there is no valid accompanying traceparent. The new trace can describe what happens from the reset point onward. It cannot recreate the missing parentage.

The important word is not “new” but “unlinked.” Without an additional receipt, the old trace and new trace are two internally coherent stories with no decision-grade join between them.

Fail open does not mean fail silently

The drafts leave room for a server to reject an RPC because of bad Trace Context, even though they advise against it. If a server chooses rejection, it must return a protocol operation-failed error, and the NETCONF model defines structured information for the offending metadata.

That gives operators two legitimate implementation policies:

  1. preserve the management operation, reset or omit the trace context, and make the evidence discontinuity visible; or
  2. reject the operation with an explicit protocol error because local policy treats trace continuity as a precondition.

The dangerous policy is the undocumented mixture. If one controller continues, another rejects and a third drops tracing without recording the reset, the same malformed header produces different operational outcomes along one service path. A retry layer may then turn an observability defect into duplicate configuration work. An audit system may declare a missing action while the datastore retains it. A billing process may correlate the caller's order to the wrong span or to none.

The answer is not to promote tracing into an authorisation protocol. Trace inputs themselves are untrusted. W3C warns that callers can exploit sampling to impose cost, forge collisions that damage monitoring, or leak information through correlation and opaque state. The NETCONF draft notes that trace knowledge can help an attacker map a managed network. A server needs the freedom to validate, remove, restart or decline tracing at a trust boundary.

The control obligation is narrower: whichever path the implementation takes must leave a receipt that another system can interpret without guessing.

Build the join outside the trace

A resilient evidence design gives the trace an important but bounded role. At ingress, preserve the authenticated principal, authorisation decision and exact management request identity. Record whether Trace Context was absent, accepted, reset or rejected. If the server creates a replacement traceparent, retain it beside the original request identity without copying untrusted tracestate into an authoritative field.

Next preserve the protocol result: NETCONF reply or RESTCONF status, resource location, entity tag, server time and any local transaction or datastore receipt. Then obtain a fresh readback at a named configuration epoch. Finally, observe the intended service effect from a vantage independent of the write path.

Those records form a chain:

authenticated request → trace disposition → protocol result → datastore evidence → configuration readback → service observation

The arrows are claims to be evidenced, not assumptions. A 201 can prove what one RESTCONF server reported about creating a resource; it cannot prove that every downstream subsystem converged. A successful readback can prove that named state was visible at one epoch; it cannot prove packet delivery. A service test can show an outcome without revealing which request caused it. The reconciliation receipt keeps these scopes intact while allowing an investigator to traverse them.

This also changes alert design. “Original trace has no device span” should not automatically mean “configuration failed.” It should mean “correlation incomplete; check trace-disposition and protocol receipts.” Conversely, “trace complete” should not close a change until the relevant configuration and outcome evidence arrives.

A standard can expose the boundary without operating it

The two documents are active Internet-Drafts, not final RFCs, deployment reports or interoperability results. They define carriage, capability and error behaviour. They do not prove that a particular controller exports spans, that a collector receives every span, or that a vendor preserves a reconciliation record.

That restraint matters. YANG Library may advertise the modules and still say nothing about collector custody. Mutual authentication and NACM can protect the management channel and still leave the trace reset unexplained. A well-formed trace can span every component while the configuration is denied. A malformed trace can end at the boundary while the configuration is accepted.

The drafts are valuable precisely because they make this uncomfortable state legitimate: management and observability can disagree without either subsystem necessarily being broken. The operational work begins where the wire format stops.

Sources