Summary

  • RFC 9539 is an Experimental profile for unilateral, opportunistic DoT or DoQ between recursive and authoritative DNS servers. It aims to frustrate passive monitoring while preserving ordinary resolution when encrypted transport is unavailable.
  • The profile deliberately accepts an unauthenticated certificate and can fall back to cleartext. It does not authenticate the authoritative server, replace DNSSEC, or resist an active attacker who forces downgrade or interposes on the connection.
  • The operational record must be per query as well as per server IP: attempted transports, the response actually processed, certificate-verification result, SNI, timers, failure class and cleartext fallback. “TLS succeeded” is not a privacy receipt.

The answer that won before privacy arrived

Consider a deliberately illustrative cold lookup. A recursive resolver needs an answer from authoritative address X. It has no recent encrypted-transport state for that address. It sends the question over ordinary DNS on port 53 and begins a DNS-over-TLS or DNS-over-QUIC attempt on port 853 at nearly the same time.

The cleartext response arrives first. It is well formed, matches an outstanding question and is processed. The resolver marks the copies on the encrypted paths as already handled. A few milliseconds later, the encrypted handshake succeeds. The resolver stores a successful capability result for X, perhaps for the three-day persistence interval suggested by RFC 9539. Later questions to that address can avoid port 53 while the state remains recent.

Nothing in that sequence is contradictory. The experiment is designed to learn capability without imposing a coordination ceremony or delaying resolution. But three records now describe three different claims. The DNS query was answered. The authoritative IP proved that it could complete an encrypted transport. The first query was nevertheless visible on the clear path. If a dashboard merges the records into one green label — “authoritative DNS encrypted” — it has destroyed the fact that matters.

The certificate adds a fourth distinction. Under this unilateral profile, the recursive client must accept any certificate presented by the server. Failure to verify the asserted identity may be logged, but it must not cause rejection merely to fall back to cleartext. The channel can therefore be confidential against a passive monitor while remaining vulnerable to an active machine in the middle. An accepted certificate is not an identity verdict.

A narrow experiment, not a new trust hierarchy

RFC 9539 was published in February 2024 as an Experimental RFC. Its target is one link that many encrypted-DNS discussions leave exposed: the path from a recursive resolver to the authoritative servers it consults. DoT is defined by RFC 7858, and DoQ by RFC 9250. RFC 9539 gives those transports a deployment profile for this hop.

Either side can move first. An authoritative operator may listen for DoT, DoQ or both on port 853. A recursive operator may probe addresses learned through ordinary DNS resolution. There is no prerequisite registry saying which zones have enrolled, no new record that grants permission, and no requirement that every member of the ecosystem upgrade together.

That is the strength of the design. It follows the logic of RFC 7435: use encryption when it is available even if authentication is not, because some protection against pervasive passive observation is better than cleartext everywhere. It also follows a restrained version of Heng Lu’s principle of localized future decision. The IETF specifies a compatible experiment; each running operator decides whether to implement it. Publication is not deployment, and non-adoption does not make a DNS server invalid.

The same narrowness prevents inflated claims. RFC 9539 is not a mechanism for authenticating an authoritative service. It provides no defence against an active attacker. It does not change the DNS delegation or confer new authority on the address that answers port 853. It does not replace DNSSEC, which authenticates DNS data through its own chain. It does not make the fastest transport the truest transport.

The state belongs to an address and a path

The profile encourages a resolver to remember transport capability by authoritative server IP address. That choice is operationally important. One NS hostname can resolve to several addresses. One zone can use several NS names. One address can front a load-balanced or anycast pool. If capability were stored only by zone or hostname, a success at one installation could cause an encrypted attempt to a different installation that accepts only port 53, adding a timeout before useful resolution.

The real key can be more specific still: resolver source IP, authoritative destination IP and transport. A load balancer may map different client addresses to different members. An anycast route may move one source cohort to a different site. When those paths do not share rollout state, “the server supports DoQ” is too broad. The defensible statement is that a particular source reached a particular address over a particular transport at a particular time.

RFC 9539 names the state needed to preserve that sentence. It includes when a connection was initiated and completed, whether it succeeded, failed or timed out, when the last response arrived, any usable resumption state, and which questions remain attached to a session. Live session, query-queue and last-activity data vanish across restart; capability history may persist. That split prevents a restarted resolver from pretending an old socket still exists while allowing it to retain recent experience about an address.

The suggested defaults — three days of success persistence, one day of failure damping and a four-second handshake timeout — are starting points for an experiment. They are not safety constants. A high-volume resolver, a satellite path, an anycast service in active deployment or a small authoritative host may need different values. Whatever the choice, the values belong in the public operating record because they decide how long a single success suppresses cleartext and how long a single failure suppresses another attempt at privacy.

Failure is not one event

When an encrypted handshake fails, the resolver clears the session, records failure and moves questions to Do53 if no other path covers them. It waits for the damping interval before probing that address again. A failure after establishment has a similar consequence.

A clean close is different. An authoritative server may intentionally close idle or old connections to protect memory and CPU. Outstanding questions still need another path, but the clean close does not require the same day-long memory of incapability. The next query may try encrypted transport again promptly. A query-specific timeout is different again: another outstanding encrypted transport, another authoritative address or the clear path may still complete.

Flattening these events into encrypted=false makes policy impossible to audit. A TLS alert can indicate protocol mismatch. A silent port can indicate absence, filtering or rate limiting. A clean close can be disciplined resource management. A well-formed DNS error is an application response, not a transport failure. An active attacker can imitate some of these conditions to trigger cleartext. The record must retain the event that caused downgrade, the time of the fallback, the query affected and when the next encrypted probe becomes eligible.

Availability remains the default aim. Appendix B of RFC 9539 explains why the experiment does not fail closed: an authoritative that merely happens to offer encrypted transport has made no authenticated commitment to continue doing so. A future strict profile would need a downgrade-resistant signal, server authentication and a defined scope — perhaps zone, NS name or another object. Without that commitment, refusing every answer after port 853 stops working would turn a voluntary privacy improvement into an accidental outage lever.

Encryption, identity and data are separate ledgers

Three cryptographic questions recur in DNS operations and must not be joined by vocabulary alone.

First, was the transport encrypted? A successful TLS or QUIC session answers that for the bytes carried on that session. Second, was the peer the intended authoritative server? RFC 9539 deliberately does not answer this; the client accepts any certificate. Third, was the DNS answer authentic? DNSSEC can answer that when a valid chain and signed data exist, regardless of whether the packet travelled over Do53, DoT or DoQ.

This separation explains why a DNSSEC-valid answer sent in cleartext can have authenticated data but no transport confidentiality. It also explains why an encrypted response with an unverified certificate and no valid DNSSEC chain can have confidentiality against passive observation without proving either peer identity or data origin. Both combinations are possible. Neither should be labelled simply “secure DNS.”

Server Name Indication creates another privacy ledger. The resolver normally should omit SNI for unilateral queries because a clear ClientHello field may reveal which nameserver name — and therefore which zone relationship — motivated traffic to a shared address. If an implementation has another reason to send SNI, Encrypted ClientHello can reduce that leak. This does not erase address, timing or size information, and it does not convert the certificate into an authenticated identity under this profile.

Padding addresses size leakage. RFC 7830 defines the EDNS padding option, while RFC 8467 provides policy guidance. QNAME minimisation reduces the name material disclosed to each authoritative step. These are complementary controls: encryption conceals content from a passive path observer, minimisation limits what each server needs to receive, and padding makes size inference harder. None authenticates the other endpoint.

The authoritative side must converge too

An authoritative service implementing the experiment must answer from the same zone data on encrypted and unencrypted transports. Response size, EDNS behaviour and truncation can differ because transports have different characteristics. The substantive DNS view must not split merely because the listener changed.

Operationally, the harder split can occur behind one IP. A load balancer may send successive connections to different backend members, some upgraded and some not. Anycast can expose the same address at sites on different rollout schedules. RFC 9539 recommends enabling the pool within a short interval, mapping clients consistently or teaching the load balancer which members support each encrypted transport.

That recommendation creates a measurable obligation, not a claim of perfect simultaneity. Operators should compare results by resolver source cohort, anycast site and time. They should record handshakes that succeed but are followed by no DNS response. They should verify that encrypted and unencrypted answers come from equivalent zone generations. One successful probe is evidence about one path, not a certificate for the fleet.

What the evidence row must preserve

A useful record begins with the question: query ID, QNAME, QTYPE, QCLASS and creation time. It names the resolver source address, authoritative destination address, NS and zone context, and observation point. It lists every attempted transport, destination port, ALPN, initiation and completion time.

It then records which outstanding queue held the question and which response was processed. For the encrypted attempt, it retains the certificate fingerprint and form, identity-verification outcome, SNI or ECH state, session or resumption identifier, early-data status and failure classification. For policy, it retains the active persistence, damping and timeout values. For the DNS result, it retains response bytes or a hash, RCODE, relevant flags, validation result and any comparison with another transport.

Finally, it records consequences: whether this question crossed Do53, when fallback occurred, when encrypted probing becomes eligible again, which pool or anycast cohort answered, and what review trigger will change configuration. A fleet summary should report the percentage of questions actually carried by each transport, not just the percentage of server IPs that once completed a handshake.

That last denominator is decisive. The experiment’s privacy value exists in protected questions, not in configured listeners.

Sources