A widely shared engineering write-up describes reclaiming roughly 100 terabytes of memory across a large public DNS resolver by changing how cached records are stored. The gain came from data-structure and expiry choices, not new hardware.
Key takeaways
- A public DNS resolver keeps enormous numbers of cached answers in memory, so small per-record overheads multiply into very large aggregate costs.
- Reported savings on the order of 100 terabytes refer to memory summed across a large fleet of servers, not a single machine.
- Most of the win in this kind of work comes from removing per-entry overhead — pointers, padding, allocator slack and duplicated strings — rather than from compressing the answers themselves.
- Cache tuning involves a trade-off: holding fewer or smaller entries can raise the miss rate, which increases upstream queries and can slow some lookups.
- The engineering pattern generalises: any system caching billions of small objects tends to spend more memory on bookkeeping than on the data.
What is actually happening here
The item circulating on Hacker News is an engineering account of optimising the cache behind 1.1.1.1, a public recursive DNS resolver operated by Cloudflare. A recursive resolver answers questions like “which address does this hostname map to” on behalf of users, and it stores the answers it has already fetched so that repeated questions can be served locally instead of being sent back out to authoritative name servers.
The claimed result is a reduction of around 100 terabytes of memory. That figure is best read as a fleet-wide total: a resolver of this scale runs in many data centres, with many processes on many machines, and each one holds its own cache. A saving of a few hundred bytes per cached record, multiplied by billions of records across hundreds of locations, arrives at very large numbers quickly. The precise breakdown of where the savings came from, and the exact measurement method, are details that sit in the original write-up and are not independently verifiable from the trend signal alone.
Why it is being discussed now
The discussion is driven by publication rather than by any incident. Infrastructure operators periodically publish accounts of internal optimisation work, and this one attracted a large comment thread because it combines a familiar piece of infrastructure with a concrete, unusually large number.
Posts of this kind tend to travel because they are legible to a broad technical audience. Nearly every working programmer has written a cache, and almost none have written one at this scale, so the gap between the familiar problem and the unfamiliar magnitude is where the interest lies. The comment volume suggests debate about method as much as agreement about the result.
The background a newcomer needs
DNS translates human-readable names into the numeric addresses machines use. When a device asks a resolver for a name, the resolver may need to consult a chain of authoritative servers to find the answer. That round trip costs time, so resolvers cache results.
Each cached record carries a time-to-live, a value set by the domain’s owner that says how long the answer may be reused. When the TTL expires, the entry must be refetched or revalidated. A resolver therefore maintains a large map from names and query types to answers, with expiry attached.
The memory cost of such a map is not only the answers. It includes the key strings, hash table slots, per-entry metadata such as timestamps and flags, pointers linking entries into eviction structures, alignment padding inside each object, and the slack that a memory allocator leaves when it rounds allocations up to size classes. DNS answers are small — often a few dozen bytes — so the bookkeeping around each one can exceed the payload. This is the structural reason large caches of small objects are so responsive to layout changes.
Typical remedies are well known in the field: packing fields tightly, replacing pointers with smaller offsets or indices, interning repeated strings such as shared domain suffixes, using open-addressed tables instead of pointer-chasing ones, arena allocation to avoid per-object allocator overhead, and expiring entries more aggressively where the hit rate does not justify keeping them. Which of these produced the reported saving, and in what proportion, is not something that can be stated without the source document.
Who is affected and how
For ordinary internet users, the intended effect is nothing at all. A resolver optimisation of this kind is meant to be invisible: the same answers, at broadly the same speed. Any user-facing change would show up as slightly different latency on uncommon lookups, which is below the threshold of everyday notice.
For the operator, the effect is direct. Memory is one of the more expensive components in a server fleet and is often the limiting factor in how many requests a machine can handle. Freeing memory can mean deferring hardware purchases, fitting more capacity into existing machines, or using the reclaimed space for other services on the same hardware.
For domain owners and network operators, the relevant question is whether the resolver’s caching behaviour still respects their TTLs. Shorter effective caching can increase load on authoritative servers; longer caching can delay the propagation of record changes such as failovers. Nothing in a memory optimisation necessarily changes TTL handling, but it is the aspect that would matter to third parties if it did.
For engineers elsewhere, the value is transferable. The lesson that per-entry overhead dominates in small-object caches applies to session stores, rate limiters, feature flag caches and metrics pipelines alike.
Where informed people disagree
The first disagreement is about how to present a headline figure. Aggregating memory across a fleet produces a large number that is technically accurate but hard to compare against anything. Critics of this framing argue that percentage reduction per node, or memory per cached record, communicates the engineering achievement more honestly. Defenders point out that fleet-wide totals are exactly what determines cost and capacity planning, so the aggregate is the operationally meaningful figure.
The second is about the source of the savings. There is a real difference between making each entry smaller and simply keeping fewer entries. Both reduce memory; only the first is free. If a meaningful share of the reduction came from tighter expiry or more aggressive eviction, then the hit rate fell somewhere, and some queries now travel further. Whether that trade was made, and how it was measured, is the sort of question a detailed write-up can answer and a summary cannot.
The third is about language choice and runtime. Debates on threads like this reliably turn to whether the overheads being removed were inherent to the problem or artefacts of a particular implementation — garbage-collected heaps, standard library containers, or default allocator behaviour. That argument rarely resolves, because the answer depends on constraints the outside reader cannot see.
The practical implications
For teams running caches at any scale, the actionable point is to measure where memory actually goes before optimising. Profiling that reports only total heap usage will hide the distinction between payload and overhead, and it is usually the overhead that offers the larger win.
Second, cache sizing should be evaluated against hit rate rather than against memory alone. A cache that is 30 per cent smaller and loses two percentage points of hit rate may be a poor trade if the miss path is expensive; the same change may be excellent if misses are cheap. The metrics have to be considered together.
Third, allocator behaviour deserves attention. Rounding to size classes can waste a substantial fraction of a small allocation, and choosing object sizes that align with those classes is a low-effort improvement that requires no algorithmic change.
Finally, the ecological and cost argument is real but should not be overstated. Memory reclaimed on existing servers reduces the need for new ones, though translating that into an energy or emissions figure requires assumptions about utilisation that are rarely published.
What to watch next
Watch for whether the operator publishes follow-up detail on hit rates and query latency alongside the memory figure, since that combination is what shows the optimisation was genuinely free rather than a shifted cost. Watch also for whether the techniques appear in open-source resolver software, where other operators could adopt them and where the implementation would be open to inspection. More broadly, expect this pattern — careful attention to per-object overhead in very large caches — to recur as infrastructure operators face memory costs that have risen faster than the cost of computation.
Frequently asked questions
What is 1.1.1.1?
1.1.1.1 is the address of a public recursive DNS resolver operated by Cloudflare. Devices configured to use it send their name-lookup queries there, and the service finds and returns the corresponding addresses. It is one of several free public resolvers available to anyone who chooses to configure it, alongside similar services run by other large providers.
How does a DNS cache save memory without losing data?
Most savings come from reducing overhead rather than discarding entries. A cached record consists of a small answer surrounded by metadata, pointers, padding and allocator slack. Packing fields more tightly, sharing repeated strings, and choosing data structures with less per-entry cost can shrink total memory substantially while storing exactly the same answers for the same duration.
Does this optimisation make DNS lookups slower?
Not necessarily. If the savings come from making each cached entry smaller, the number of cached answers is unchanged and lookup speed should be unaffected. If savings come partly from holding fewer entries or expiring them sooner, the cache miss rate rises and some queries take longer. Which applies here cannot be determined from the summary alone.
Why is 100 terabytes such a large number for a cache?
Because it is a fleet-wide total rather than one machine’s usage. A global resolver runs across many data centres, with independent caches on each server. Even a modest per-record saving multiplies across billions of records and hundreds of locations. No single machine holds anything approaching that amount of DNS cache in memory.
What is a TTL in DNS?
TTL stands for time-to-live. It is a value attached to each DNS record by the domain’s owner, specifying how many seconds a resolver may reuse a cached answer before checking again. Short TTLs allow rapid changes to take effect but increase query load; long TTLs reduce load but delay the propagation of updates.
Can smaller organisations apply the same techniques?
Yes, in principle. The underlying ideas — measuring overhead separately from payload, interning repeated strings, packing structures, using arena allocation and aligning object sizes to allocator size classes — apply to any cache of many small objects. The absolute savings scale with size, but the proportional improvement and the profiling method are the same at any scale.
Sources and further reading
- Cloudflare’s engineering blog, which publishes technical accounts of changes to its network and resolver infrastructure.
- Hacker News, where the discussion thread surfaced the item and contains community analysis and criticism of the approach.
- The IETF’s published DNS specifications, which define caching behaviour, record types and time-to-live semantics.
- General systems-programming literature on memory allocators, hash table design and cache eviction policies.
Surfaced from the hackernews signal “DNS cache memory optimisation”. AI-assisted draft, editorially reviewed.
