Cloudflare Prototypes Cache Transcoding with Zstandard and Pingora to Save Storage

Cloudflare Prototypes Cache Transcoding with Zstandard and Pingora to Save Storage In response to surging memory and disk prices, Cloudflare prototyped Cache Transcoding, a system integrated into its Pingora proxy that compresses uncompressed cacheable text with Zstandard (zstd). The approach trades a modest increase in CPU overhead for significant storage density gains and lower inter-datacenter bandwidth usage. Eligible uncompressed assets are compressed on disk with zstd level 3, reducing their size to roughly one-third (2.834x ratio) in testing. Text assets (HTML, JSON, CSS, JavaScript) accounted for 67.3% of sampled requests and 22.3% of bytes, with around 71% arriving uncompressed. Transcoding encodes once per cache fill (4.31 ns/byte, ~232 MB/s) and decodes on each client serve (1.56 ns/byte, ~641 MB/s), keeping the asset compressed across Tiered Cache transfers. The system enforces eligibility checks, requiring 200 OK status, compressible Content-Type, unset Content-Encoding, and a known Content-Length of at least 4 KiB. Architecture and Workflow Inside Cloudflare's Pingora proxy, Cache Transcoding encodes eligible bodies into zstd before writing them to disk upon a cache miss, while metadata tracks compression and original content length. In multi-tier caching architectures, the asset is transferred between upper and lower tiers in its compressed format, minimizing inter-datacenter backbone bandwidth. Decoding to the original identity representation only occurs on the client-facing hop before final delivery. Scope and Eligibility Rules The prototype avoids already compressed media—which accounted for 63.3% of sampled bytes across 21.4% of requests—to prevent wasteful CPU consumption. Eligibility requires a 200 OK response with unset Content-Encoding, compressible text types, and a known Content-Length of at least 4 KiB. This minimum size threshold filters out tiny objects while retaining approximately 99% of compressible bytes, avoiding per-object overhead without sacrificing significant space savings. Performance and Future Testing Benchmark testing over one million requests across 10 cache servers demonstrated a 2.834x compression ratio on test corpus assets, holding the overall CPU cost increase to a few percent. Cloudflare plans further evaluations to test higher zstd levels, evaluate larger and more varied content pools, and investigate skipping decoding entirely when downstream components can natively ingest the compressed payload.

Cloudflare 原型设计 Cache Transcoding：借助 Zstandard 与 Pingora 节省 PB 级缓存存储 面对内存与硬盘硬件成本剧增，Cloudflare 在其 Pingora 代理中基于 Zstandard 算法开发了 Cache Transcoding 原型系统，通过小幅增加 CPU 开销将符合条件的内容压缩至原大小的约三分之一，大幅提升有效缓存容量并削减跨数据中心传输带宽。 在测试语料中，该方案将符合条件的文本资产平均压缩至原盘体积的约三分之一（压缩比约 2.834x）。 原型采用 zstd 级别 3，编码速度约为 232 MB/s（仅在写入时执行一次），解码速度约为 641 MB/s（每次服务时执行）。 系统设定严格的筛选标准，仅对未压缩且体积在 4 KiB 以上的 200 OK 文本内容进行转码，规避已压缩多媒体内容造成的 CPU 浪费。 在分层缓存（Tiered Cache）中，资产在上下层节点间直接以压缩形态传输，仅在面向客户端的最后一跳才执行解码。 背景与系统设计逻辑 面对持续高涨的存储硬件成本，Cloudflare 原型设计了 Cache Transcoding 系统。传统架构会保留源站原始内容编码直接存盘，而新技术在资产进入缓存且符合条件时使用 zstd 进行无损编码。虽然编码与解码需要消耗少量 CPU，但编码仅在填入缓存时执行一次，而压缩形态在本地存储和跨数据中心分层缓存中复用，显著降低了数据逐出率并节省了骨干网带宽。 筛选规则与运作机制 流量分析显示，图片、视频等不可压缩媒体占据了 63.3% 的字节量，二次压缩毫无意义；而 HTML、JSON、CSS 等文本内容中约 71% 以未压缩形态到达。系统制定了简化策略，仅针对未设置 Content-Encoding、Content-Type 为文本且长度不低于 4 KiB 的 200 OK 响应进行转码。4 KiB 阈值排除了大量微小对象，同时保留了约 99% 的可节省字节。 测试验证与后续展望 团队通过 10 台缓存服务器发送了超百万次请求进行验证，在分层缓存与独立缓存场景下均证实了架构的正确性，受测资产压缩比达 2.8 倍左右。后续工作将计划评估更高的 zstd 压缩等级、测试更广泛的内容类型与体积，并探索支持范围请求（range requests）以及直接向支持下游透传压缩内容而无需解码的可能性。

Memory costs are increasing dramatically. Both RAM and hard disk drive prices have exploded over the past year. At Cloudflare, we run several massively distributed storage products (including our famous CDN) that rely on making efficient use of the memory we have deployed so we can continue to serve all of our customers. With this in mind, we prototyped a way to expand effective cache capacity. By encoding eligible assets with [Zstandard](https://github.com/facebook/zstd) inside [Pingora](https://blog.cloudflare.com/how-we-built-pingora-the-proxy-that-connects-cloudflare-to-the-internet/), the architecture trades a minor CPU increase for significant storage and cross-data center bandwidth savings. We have been prototyping a system called Cache Transcoding, which I built during my internship at Cloudflare as part of the [1.1.1.1 Intern Program](https://blog.cloudflare.com/cloudflare-1111-intern-program/). When an eligible response enters the cache, we encode it using Zstandard, or zstd, before writing it to disk. We keep that compressed form while the asset lives in the cache and moves between data centers via [Tiered Cache](https://blog.cloudflare.com/tiered-cache-smart-topology/#tiered-cache), then decode it before serving the response to the client. In our initial testing, this encoding shrunk eligible assets to ⅓ of their original on-disk size on average. The estimated extra CPU cost in our origin-facing proxy was small, but that is the trade. A small increase in CPU gives Cloudflare petabytes of effective cache capacity and reduces the data transferred between our data centers. The encoding cost is paid once when an asset enters the cache. The storage and bandwidth savings continue every single time that asset is reused. Zstandard, or zstd, is a lossless compression algorithm developed by [Yann Collet at Facebook](https://engineering.fb.com/2018/12/19/core-infra/zstandard/) and open sourced in 2016. Lossless means that after compressed data is decoded, every byte is identical to the original. We can change how an asset is represented on disk without changing the asset itself. Zstd is designed to balance compression ratio with speed. In our [earlier browser compression testing](https://blog.cloudflare.com/new-standards/), it compressed data 42% faster than [Brotli](https://research.google/pubs/brotli-a-general-purpose-data-compressor/) while producing nearly the same file size, and produced files 11.3% smaller than gzip at a comparable speed. That balance matters because Cache Transcoding would touch a large amount of traffic, so both encoding and decoding need to stay fast. The prototype uses zstd level 3, giving us most of the compression benefit without turning cache fills into a CPU bottleneck. Cloudflare traditionally stores an asset using the content encoding supplied by its origin. If an origin sends an uncompressed response, we store those uncompressed bytes on disk and transfer them between data centers in the same form. Cache Transcoding adds compression inside the cache itself. Transcoding does not mean compressing everything. Images, video, and fonts are usually compressed already. In our traffic sample, this media slice represented 21.4% of requests but 63.3% of bytes. Compressing it again would burn CPU for nothing. Compressible text is different. HTML, JSON, CSS, and JavaScript represented 67.3% of requests and 22.3% of bytes. Within that text slice, approximately 71% arrived uncompressed with `Content-Encoding` unset and it compresses well. In our controlled test corpus, the eligible assets compressed by roughly 2.8 times. | **Measure** | **Value** | |---|---| | Compression ratio | 2.834x | | Encode cost | 4.31 ns per byte, approximately 232 MB/s, paid once per fill | | Decode cost | 1.56 ns per byte, approximately 641 MB/s, paid on every serve | Encoding is more expensive per byte, but assets are served far more often than they are filled. By changing how assets are represented, existing hardware could store more customer content. Fewer bytes on disk mean each server can retain more objects. This increases cache density and reduces the likelihood that useful content is evicted because an uncompressed representation consumed more space than necessary. The smaller representation also helps as an asset moves through Tiered Cache because it reduces the data transferred between Cloudflare data centers, making backbone usage more efficient. Compression is never free. Encoding and decoding both use CPU, so the important question is whether the byte savings are worth the processing cost. At zstd level 3 (often the default balance of speed and compression size output), our model kept the extra CPU cost to a few percent under the traffic and reuse assumptions we tested. We initially considered limiting transcoding to popular content, since hot assets are reused more, but it did not help. Decoding happens every time an asset is served, so limiting the feature to only the hottest content reduced the storage saving without cutting CPU by the same amount. The simpler policy performed better. Transcoding all eligible compressible text at or above 4 kibibytes (KiB) captured nearly all of the measured storage benefit, while remaining within the CPU budget. On a cache miss, our Pingora-based proxy **encodes the body using zstd** before writing it to disk. The cache metadata records that the stored representation is compressed and preserves the original content length. Before the response leaves the proxy, the body is **decoded back to its original identity representation**. On a cache hit, the stored zstd object is read from disk and decoded. With Tiered Cache, the compressed representation is transferred from the upper tier to the lower tier in the compressed form. Decoding only happens on the client-facing hop. On a full cache miss, the upper tier fetches identity bytes from the origin. Those bytes are encoded once, stored as zstd, and transferred to the lower tier in their compressed form. The lower tier also stores the zstd representation, then decodes it for the request path. If the lower tier misses but the upper tier already has the object, the origin is not involved. The compressed object moves directly between the cache tiers. It remains compressed on the wire and on disk, then is decoded once at the lower tier. If the lower tier already has the object, no network transfer or encoding is needed. The lower tier reads the zstd bytes from disk, decodes them, and passes the original asset onward. The storage encoding marker prevents an object from being encoded more than once. A cache layer receiving an object from another tier can see that it is already stored using zstd, and preserve it in that form. The fastest compression operation is the one we do not need to perform. Cache Transcoding therefore uses a series of eligibility checks to avoid content that is unlikely to benefit. The prototype only transcodes a `200 OK` response when `Content-Encoding` is unset, the `Content-Type` is compressible text, and the response has a known `Content-Length` of at least 4 KiB. Slice subrequests, responses using active upstream compression, range requests, precompressed responses, unknown length bodies, and binary content remain unchanged. The 4 KiB threshold removed a large number of tiny requests while leaving out only about 1% of the otherwise eligible bytes. Lowering it would add per-object overhead without saving much more storage. The threshold and zstd level are both parameters rather than permanent limits. We started with zstd level 3 and a 4 KiB minimum because they gave us a conservative way to measure the architecture. With the initial CPU budget understood, we can test whether higher compression levels improve the ratio enough to justify their additional cost. We exercised the prototype against a controlled test zone and correlated each request across request logs, Prometheus metrics, and Jaeger traces. The correctness campaign covered cache misses, cache hits, single-hop fills, Tiered Cache fills, and more. We varied cache keys to make each request follow a specific path and used traces to confirm where encoding and decoding occurred. One performance campaign sent more than a million requests across 10 cache servers. Half of the campaign ran with Tiered Cache disabled and the other half with it enabled. This allowed us to measure local cache behavior separately from transfers between cache tiers. The two assets were approximately 195 KiB and 272 KiB, and both compressed by roughly 2.8 times. This was deliberately a compressible test corpus. It gave us a clear signal for validating the architecture, but it does not represent every text object on the Internet. A broader corpus is required before treating the measured compression ratio as a fleet-wide constant. What this experiment showed us is that there are significant efficiencies we can still deploy across our caching service that can benefit all of our customers. What we built for Cache Transcoding shows that the trade is favorable under the conditions we tested. The architecture preserved the content and remained within the CPU budget. For next steps, we plan to evaluate higher `zstd` levels, test a broader range of content types and object sizes, tune different parameters from the eligibility criteria and more. Future work can also examine range requests, pre-compressed origin responses, and passing the compressed object directly to downstream components that already support it without decoding. Throughout my internship, I’ve had the wonderful opportunity to work alongside Cloudflare's engineering teams on the real infrastructure that stores and serves content across our global network. If you want to start your career by helping build a better Internet, explore our [internship opportunities](https://www.cloudflare.com/careers/early-talent/) and [job openings](https://www.cloudflare.com/careers/jobs/).