DNS Infrastructure

A service provider's DNS management API may return success, yet users in a specific region still resolve to an old address. Troubleshooting requires checking the control plane publish, authoritative replicas, recursive caches, and application caches layer by layer, rather than just checking the management page. DNS's distributed mechanisms improve scalability, but they also create multiple observable stages between "change submitted" and "user switched."

Root Services: 13 Names, Not 13 Physical Machines

The root zone configures 13 named authoritative services, from a to m, with numerous instances deployed by various operators. The number of instances and sites changes continuously, making it inappropriate to write a dateless quantity as an architectural constant. Root hints provide root service names and addresses to help resolvers bootstrap; they are not a complete root zone, nor are they DNSSEC trust anchors. IANA: Root Name Servers

The root zone contains TLD delegations, corresponding addresses, and DNSSEC data. The root hints file is only used to locate root servers and does not equal a complete root zone. When size is needed, the actual version/format should be measured explicitly; do not treat the hints file size as the scale of the complete zone. IANA: Root Zone and Files

Root servers do not provide general recursive services, but they can answer for data they are responsible for within the root zone or for corresponding non-existent cases; they do not only provide referrals. Receiving RD=1 does not mean they must recurse on behalf of the requester. Resolvers obtain current information such as root NS via priming. Stale startup information and root zone trust chain errors are two distinct types of problems. RFC 8109: Root Hints

Anycast: Routing Selection Is Not Geographical Distance Sorting

Multiple locations announce the same service prefix. The network selects paths based on reachability, routing policies, and various metrics. AS_PATH is only one of the BGP candidate factors; LOCAL_PREF and others may determine the result earlier. "Nearest" cannot be directly equated to the fewest AS hops, shortest kilometers, or lowest RTT. RFC 4786: Anycast Operations

Accessing the same IP from one observation point may switch to another instance as routing changes. Multiple instances consistently announcing the same /24 does not guarantee permanent session stickiness; precisely because the same prefix is announced in multiple locations, selection depends on upstream routing state.

Short UDP queries are easier to retry, but DNS also uses TCP, TLS, HTTPS, QUIC, and other transports that hold connection state. When the path switches suddenly, the new instance may not have the old connection state, requiring appropriate routing stability, state handling, or reconnection strategies. It is incorrect to broadly say "DNS is independent each time, so Anycast is perfectly cost-free."

Service failures do not necessarily lead to link or BGP session failures. If an instance cannot answer correctly but still announces routes, traffic may continue to fall into a black hole; the coordination between health checks and route withdrawal, withdrawal timing, and covered prefixes needs design. On the other hand, frequent withdrawals cause flapping; do not let every brief timeout trigger a full-site route withdrawal.

Authoritative Redundancy: Multiple NS Names Do Not Equal Independent Failure Domains

If two NS share the same data center, upstream, account, or publishing pipeline, they may still fail together. Design must consider shared dependencies in the service and management planes and ensure that each announced authority can correctly answer the required data.

Primary/secondary can synchronize via AXFR/IXFR, and managed services can publish via internal databases; regardless of the mechanism, observe the actual results of each authority. SOA serial helps check versions in some deployments, but signature validity, record content, and full-node load results must also be verified.

External authorities should generally not open recursion to everyone. BIND's views, access controls, and recursion options can express different policies, but their matching order and specific semantics must be verified against the version used; "listening on 53" does not indicate correct role isolation. BIND Configuration Reference

Recursive Capacity: Cache Hit Rate Determines Upstream Pressure

Assume the service receives 10,000 queries/s, with a 90% hit rate at the cache layer in question, then approximately 1,000 queries/s need further processing. If each miss triggers an average of 3 upstream transactions, the average upstream load is about 3,000 transactions/s; if the hit rate drops to 50%, it can rise to 15,000 transactions/s. The "3" here is an example measurement assumption; actual aliases, delegation caching, DNSSEC, and failure retries will change it.

With an average of 3,000 upstream transactions/s and an average in-flight time of 20 ms, according to Little's Law for a stable system, the average in-flight volume is about 60. This cannot be used as the sole capacity for connection pools or memory, as cold starts, authority timeouts, and retry tails may be much larger than the mean.

Restarting multiple recursive instances simultaneously loses hot caches, potentially amplifying upstream pressure. Negative caching, request coalescing, prefetching, and serve-stale each have their uses and correctness boundaries; do not just pursue maximizing the hit rate number.

How Much Busier Will the Upstream Be If the Hit Rate Drops by Ten Percentage Points?

Keeping the workload per miss constant, dropping the hit rate from 90% to 50% increases upstream traffic from 3,000 to 15,000 transactions/s, which is five times, not "only 40% more." Subsequently, only latency stretches while the upstream rate remains unchanged, and the in-flight volume continues to increase. This is exactly the division of labor between the two factors in Little's Law.

$${\lambda }_u={\lambda }_q(1-h)k,L_u={\lambda }_u{W}_u$$

CDN and ECS: Authorities Usually See the Resolver First

Ordinary clients hand queries to resolvers. The source IP seen by CDN authorities usually belongs to the resolver, not the end user. Geolocation strategies use address databases and operational data for judgment; latency strategies may also use historical measurements or aggregate estimates. Do not assume every DNS query pings all candidate edges in real-time.

ECS allows resolvers to carry client subnet information. Authorities can refine answers based on this and use rules such as scope to constrain cache reuse. Source prefix length is not protocol-mandated to be /24 for all IPv4 or /56 for all IPv6; exposure precision, service support, and configuration all affect results. RFC 7871: EDNS Client Subnet

Assume originally only one answer needed to be cached for a name/type. After introducing 100 related subnet ranges, multiple cached entries with ranges may be needed, reducing the reuse rate. ECS does not just add "more accurate geolocation" without other costs; it also changes privacy and caching behavior.

Route 53's latency routing selects resource configurations in regions based on its latency data, rather than guaranteeing the instantaneous minimum latency from the requester to each specific application endpoint; geographical proximity may be slower due to poorer network interconnection. AWS: Latency Routing

DNS Load Balancing: Selecting Answers Does Not Equal Scheduling Each Request

Listing two IPs in an A RRset only expresses multiple addresses; there is no weight field for weighted round-robin. Managed services' weighted strategies select records or record groups based on configuration when generating answers; how clients use multiple addresses, whether connections are reused, and failure retries are next-layer behaviors. AWS: Weighted Routing

For example, resolver R1 caches A and brings 10,000 business requests, while R2 caches B and brings only 100. Even if these two DNS selections are 1:1 by count, the application load is approximately 100:1. Therefore, DNS weights cannot guarantee precise proportional distribution of byte volume, connection count, or transaction count.

After a health check fails, new answers can avoid faulty endpoints, but old caches and existing long connections will not be immediately "recalled." Shortening TTL changes future cache windows but also increases query pressure and cannot replace client retry, application health, and connection migration design.

Split DNS: View Selection and Upstream Routing Must Be Clear

The same name can return private addresses internally and public addresses externally, or specific suffixes can be handed only to internal DNS. Server-side views and client-side upstream selection based on suffixes are implemented in different locations; both need to clarify which type of request enters which policy.

For example, dashboard.example.com. returns 192.168.31.8 internally and the document example public address externally. If internal users use independent browser DoH to bypass system splitting, they may get external results; unreachable addresses may also occur if internal caches are retained after VPN disconnect. This is not an A record syntax error, but a problem with query path and cache context.

Caches cannot indiscriminately provide results from one view to another. When using the same DNSSEC-signed name space internally and externally, internal authorization and validation relationships must be designed to prevent local arbitrary overwrites from conflicting with the public DS chain.

Search domains expand short names into candidate absolute names, while routing domains help select upstreams; concepts and specific implementation semantics must be verified separately. Configuring internal DNS does not mean all applications will use it; verify from the application's actual path.

Publishing One Change: Observing in Stages

Modifying ordinary records focuses on authoritative publishing, signatures, and record caching; changing NS delegations also requires attention to parent zone NS/glue and overlap with old/new authoritative services; updating DS involves the DNSSEC parent/child zone chain and key caching. The three cannot be summarized by the same "just wait for TTL."

Diagnosis can use dig @<authoritative address> <name> A +norecurse to compare with authorities, then query using the resolver in use. dig +trace shows the path iterated by the tool itself, which does not equal the application-configured recursive, DoH, or VPN path; changing the observation method also changes the experimental conditions. This article did not run real external probes.

Local Capacity and Load Model

query_rate = 10000
upstream_per_miss = 3
warm = query_rate*(1-.9)*upstream_per_miss
cold = query_rate*(1-.5)*upstream_per_miss
assert round(warm)==3000 and cold==15000
assert round(warm*.020)==60
# Two caches each make one selection, but the application volume behind them is different.
requests = {'A':10000,'B':100}
assert requests['A']/requests['B']==100
assert 10000/(10000+100)>.99
# Load under limited state for example only, does not represent measured capacity or fixed cache model.
print('Cache miss load, average in-flight transactions, and DNS/business ratio difference checks passed')

These are examples under explicit assumptions, not service provider performance measurements; root hints, public delegations, authoritative records, or resolver views were not modified.

Continue Reading

• DNS Protocol and Resolution: TTL, negative caching, and delegation.
• DNSSEC and DNS Security: DS and signature chain publishing dependencies.
• DNS Privacy: Resolution paths and trust for different applications.
• Routing and BGP: Anycast path selection and convergence.