Cloudflare Remediates Cross-Tenant Residual Data Exposure Vulnerability in Containers

Cloudflare Remediates Cross-Tenant Residual Data Exposure Vulnerability in Containers Cloudflare resolved a vulnerability in Cloudflare Containers and Sandboxes that could have allowed a tenant to read residual storage blocks left behind by workloads previously hosted on the same physical server. Security researcher Oren Yomtov of Accomplish reported the issue via Cloudflare's bug bounty program on September 4, 2026. The vulnerability stemmed from the `skip_block_zeroing` option in Linux device mapper thin provisioning (dm-thin), allowing unwritten portions of reassigned 64 KiB blocks to retain prior data. Researchers observed residual data across 18 of 24 placements on 20 nodes, but could not target specific customers or modify active data, and securely deleted all recovered test data. Cloudflare disabled the setting, retired running container disks, cleared pre-mitigation image snapshot caches, and found no evidence of malicious exploitation in historical disk-I/O telemetry. Storage Mechanism and Exploitation Technique Cloudflare Containers utilize dm-thin to provide writable root disks inside Firecracker virtual machines, relying on 64 KiB thin blocks returned to a shared pool upon deletion. Because the storage pool was configured with `skip_block_zeroing`, dm-thin did not clear reassigned blocks prior to container access. By writing a 4 KiB block to free ext4 filesystem space in an unmapped region, a tenant could trigger allocation of a 64 KiB physical block where the remaining 60 KiB retained residual bytes from prior owners, which could subsequently be read via raw-device reads. Validation and Potential Impact Using ext4 directory block checksums to isolate external blocks from their own test data, the researchers detected residual materials across 20 of 22 underlying nodes spanning four continents. Recovered artifacts included directory structures, database pages, and complete SQLite databases. However, an attacker could not choose specific targets or hosts, nor read actively mounted disks or modify active customer data, and the researchers' submissions contained no third-party identifiers or customer content. Fleet-Wide Remediation and Telemetry Review Cloudflare removed `skip_block_zeroing` across its fleet to restore automatic block zeroing upon allocation. To sanitize blocks already mapped into active containers or cached OCI image snapshots, engineers drained hosts during off-peak hours, rebooted virtual machines, and wiped cached image layers. A review of historical disk-I/O telemetry using signatures derived from the proof of concept showed no indications of unauthorized or malicious exploitation beyond the researchers and internal validation teams.

Cloudflare修复Containers跨租户数据残留漏洞并完成全量排查 2026年9月4日，安全研究员通过漏洞赏金计划报告了Cloudflare Containers及沙箱环境中的跨租户数据暴露漏洞。Cloudflare已在全舰队完成修复，历史遥测数据未发现恶意利用迹象，客户无需进行任何操作。 漏洞根源在于底层Linux dm-thin存储池配置了跳过块清零，导致小额写入触发块重新分配时，同宿主机新容器可读取前任租户遗留的未覆盖数据。 研究人员在四大洲的20个物理节点上验证了该技术，但该攻击无法定向指定受害目标、主机或活跃数据，且未对数据修改或可用性造成影响。 Cloudflare通过恢复分配前清零、下线存量容器磁盘、清理镜像缓存并重放历史磁盘I/O特征，完成了整改与排查。 漏洞成因与概念验证 Cloudflare Containers基于Firecracker虚拟机并使用Linux dm-thin提供64 KiB块大小的可写根磁盘。受影响存储池配置了skip_block_zeroing选项，使得释放回收的物理块重新分配给新容器时不预先清零。研究人员通过在ext4文件系统空闲区域写入对齐的4 KiB块触发新物理块分配，再通过直接读取裸设备/dev/vdc，成功读取到了该块内未被覆盖的剩余60 KiB历史残留数据。在测试中，研究人员在24次部署中的18次、22个节点中的20个节点观察到了残留数据，类型包括目录结构、数据库页面和完整的SQLite数据库；测试数据已依据漏洞披露政策安全删除。 潜在风险与攻击限制 该漏洞理论上允许持有Workers Paid账户的攻击者跨越租户隔离边界，获取同宿主机先前容器的文件系统元数据、目录及应用数据。但该攻击具有明确限制：Containers由系统自动调度，攻击者无法选定特定目标客户、主机或正在活跃挂载的磁盘，残余数据的出现缺乏确定性。此外，研究人员未证明可以修改活跃客户数据或干扰工作负载可用性。 修复措施与遥测排查 Cloudflare已从全舰队配置中移除了skip_block_zeroing，恢复dm-thin分配前强制清零的机制。由于存量运行磁盘和镜像层缓存仍可能映射未清零块，团队在业务低峰期排空宿主机、重启虚拟机并清空各主机镜像缓存，完成了全量刷新。同时，安全团队基于4 KiB写入与大额读取的特征开发了检测特征码，排查了可用的历史磁盘I/O遥测数据，确认除研究人员和官方验证人员的活动外，未见其他利用痕迹。

On September 4, 2026, Oren Yomtov, a security researcher from [Accomplish](https://accomplish.ai/), responsibly reported a vulnerability affecting [Cloudflare Containers](https://developers.cloudflare.com/containers/) and [Cloudflare Sandboxes](https://developers.cloudflare.com/sandbox/) (which is built on Containers), through Cloudflare’s [bug bounty program](https://hackerone.com/cloudflare?type=team). Cloudflare has fully remediated the vulnerability, and we have no evidence that customer data has been compromised. This post was prepared in collaboration with Oren Yomtov and the Accomplish security research team, whose detailed report and controlled testing helped us validate the issue and respond quickly. Cloudflare Containers run workloads on multi-tenant infrastructure and automatically assign them to eligible servers; customers cannot select the underlying host. The researchers demonstrated that a customer with a Workers Paid account could recover residual disk blocks previously used by Containers on the same host. The technique could not target a particular customer, workload, host, or data, and residual data was not guaranteed to be present. Cloudflare applied a fix across the Containers fleet, with no customer-side configuration changes required. Within the historical disk-I/O telemetry available to us, we identified no evidence of malicious exploitation. Activity we could attribute to the reported technique came from the researchers and Cloudflare engineers conducting authorized validation. Here, we explain the underlying storage behavior, its potential impact, our investigation, and the actions we took in response. Cloudflare Containers use Linux device mapper thin provisioning (dm-thin) to provide each container with a writable root disk. Each container lives inside a dedicated virtual machine powered by the Firecracker virtual machine monitor. Firecracker presents this disk to the virtual machine as /dev/vdc. Thin provisioning allocates physical storage only when a virtual disk writes to a previously unmapped region. The affected storage pools used a 64 KiB thin-block size. When the thin volume backing a container's root disk was deleted, its physical blocks were returned to a pool that served workloads belonging to multiple customer accounts. The affected pool configuration included the following option: `skip_block_zeroing` With this option configured, dm-thin skips zeroing newly allocated blocks before making them accessible. Consequently, when a previously-used 64 KiB block was reassigned, a full-block write replaced its previous contents, but a smaller write changed only the written portion. The remainder could retain data from the block’s previous owner. Reading an unmapped region of a new thin disk did not reveal residual data. For an unmapped region of the thin device, `dm-thin` returned zeroes without allocating a physical block. The proof of concept identified 64 KiB-aligned regions corresponding to free space in the guest’s ext4 filesystem and wrote one aligned 4 KiB block into each region. When such a write reached an unmapped thin block, dm-thin allocated a physical 64 KiB block from the shared pool. The 4 KiB write replaced only that portion of the block, and because block zeroing was disabled, the remaining 60 KiB could retain data from a previous container. A subsequent raw-device read could therefore observe bytes that the new container had never written. The proof of concept performed the following steps: The submission included counts, block offsets, sizes, checksum results, and truncated hash prefixes. Although the researchers recovered raw blocks to validate the issue, the materials provided to Cloudflare contained no third-party filenames, identifiers, credentials, hostnames, addresses, or recovered content values. As described below, the researchers have also confirmed that they securely deleted the recovered data. The researchers used ext4 directory block checksums to distinguish blocks belonging to their own test filesystem created for the proof of concept from blocks originating from other filesystems. When ext4 uses the `metadata_csum` feature, directory block checksums incorporate values associated with the filesystem and inode. Across six production placements, the researchers reported: To validate the method, the researchers tested it against blocks they had deliberately created and deleted in the controlled test filesystem used for the proof of concept. The method correctly attributed all 162 blocks to that filesystem. The researchers ultimately observed residual material on 18 of 24 placements and 20 of 22 underlying nodes across four continents. The recovered block types included directory structures, database pages, and structurally complete SQLite databases. The researchers reported using scripts that output only aggregate counts and format checks, not recovered file contents. The materials submitted to Cloudflare contained no recovered content values or third-party identifiers. The researchers subsequently confirmed that recovered data under their control remained confidential and was securely deleted following submission, consistent with Cloudflare’s HackerOne disclosure policy. The vulnerability would potentially have allowed for a customer with a Workers Paid account to recover residual data from storage blocks previously used by other customers’ Containers on the same underlying host. A successful exploitation would have crossed the tenant-isolation boundary and could disclose filesystem metadata, directory structures, database pages, and application data. However, an attacker could not select a particular victim or access an actively attached disk. Exposure depended on Cloudflare’s workload placement and which previously released blocks `dm-thin` reassigned. Moreover, the researchers **did not** demonstrate modification of another customer’s active data or impact to workload availability. Our first mitigation was to remove `skip_block_zeroing` from the `dm-thin` pool configuration across the fleet. This restored `dm-thin`’s default behavior of clearing newly allocated blocks before exposing them to a container. It stopped the reported technique, in which a small write triggered allocation and a larger read recovered residual data from the remainder of the block. The researchers independently confirmed that their proof of concept no longer worked after this change. Zeroing new allocations did not sanitize blocks already mapped into existing thin devices. These mappings existed in running container disks and in each host’s cache of prepared dm-thin snapshots for OCI image layers. A new container could inherit mappings from a cached layer without allocating those blocks again, allowing residual bytes in unused regions, including ext4 free space, to remain readable through raw reads of `/dev/vdc`. We therefore also retired all running container disks and removed cached image snapshots created before the mitigation. We drained hosts during off-peak hours, restarted the VMs on each host, and cleared each host's image cache so that disks and cached layers were recreated using zeroed allocations. We have completed this cleanup across the Containers fleet. As part of our response, we investigated whether other workloads showed activity consistent with the reported exploitation technique. We reviewed retained historical disk-I/O telemetry from our container infrastructure, using the researchers’ proof of concept and our internal reproduction as reference activity. The proof of concept produced a characteristic relationship between writes and reads. When a 4 KiB write reached a previously unmapped region, it could trigger allocation of a reused 64 KiB storage block. With zeroing disabled, the remaining 60 KiB could retain data from a previous container. Subsequent reads could therefore recover substantially more data than the new container had overwritten. Using these characteristics, we developed detection signatures and applied them to the historical telemetry available to us. We identified activity attributable to the researchers and Cloudflare engineers conducting authorized validation, and did not identify additional activity consistent with the reported technique. We saw no evidence that this specific attack vector was exploited by anyone else. As we noted above, Cloudflare has patched this vulnerability and remediation does not require any further action by Cloudflare customers. In addition, we found no evidence of any malicious actor abusing this vulnerability. We thank Oren Yomtov and the Accomplish security research team for their thorough research, responsible disclosure, and collaboration on this post. We encourage the Cloudflare community to submit any identified vulnerabilities to help us continually improve the security posture of our products and platform. We also recognize that the trust you place in us is paramount to the success of your infrastructure on Cloudflare. We take these vulnerabilities very seriously and will continue to do everything in our power to mitigate impact. We deeply appreciate your continued support and trust in our platform, and remain committed not only to prioritizing security in all we do, but also acting swiftly and transparently whenever an issue arises.