Databricks
Decoupling Postgres compute from data lake storage is necessary to deliver the sub-second branching and automated cross-cloud disaster recovery required by autonomous AI agents.
AI-assisted summary of Inside Lakebase: fully-managed serverless Postgres – Nikita Shamgunov, VP, Engineering, Databricks — original source. Check the original for full context.
Source published:
Key ideas
- Databricks has launched Lakebase, a fully-managed serverless Postgres platform built to power the surge of AI-generated applications and autonomous agents by decoupling compute from lake storage. (1:53)
- To overcome the inherent latency and consistency limitations of object lakes, Lakebase splits intermediate storage into Paxos-driven Safekeepers for writes and low-latency Page Servers for reads. (1:53)
- The platform provisions compute instances and branches complete databases in under 500 milliseconds, featuring automatic scale-to-zero and instant snapshot rollbacks for agent sandboxing. (3:44)
- In TPC-C benchmark testing, Lakebase sustained over 600,000 operations per second with sub-10ms latencies, comfortably outpacing major incumbent cloud databases. (6:03)
- Lakebase introduced fully-managed cross-cloud disaster recovery between providers like AWS and Azure, surpassing 3,500 enterprise customers within its first year. (7:44)
Detailed notes
## Background Published by Databricks on YouTube on June 24, 2026, this keynote session from the Data + AI Summit 2026 features Nikita Shamgunov, VP of Engineering at Databricks. The presentation details the architecture and features of Lakebase, a fully-managed, serverless Postgres database designed to support AI applications and autonomous agents with built-in cloud disaster recovery.
## TL;DR Addressing the surge of AI-generated software, Nikita Shamgunov introduces Lakebase, a serverless, decoupled Postgres database tailored for modern agents and mission-critical enterprise workloads. Databricks split Postgres compute from storage by engineering intermediate consensus safekeepers and page-serving nodes backed by lake storage, enabling sub-500ms provisioning, scale-to-zero, instant branching, and quick snapshot-restores. In TPC-C benchmark testing, Lakebase sustained over 600,000 operations per second with sub-10ms latencies, well outperforming legacy cloud competitors. The system also introduces fully-managed cross-cloud disaster recovery, allowing seamless failover between major cloud providers like AWS and Azure.
## Atmosphere & style The presentation carries the brisk, authoritative momentum typical of a flagship tech summit keynote, blending ambitious industry forecasting with deep infrastructure architecture. Shamgunov speaks with confident precision and enthusiasm, shifting smoothly from the macro realities of AI code generation to the technical mechanics of Paxos consensus and page servers. The delivery is focused and persuasive, punctuated by clear visual architectural diagrams and performance charts that frame Lakebase as an indispensable pillar for the next era of computing.
## Bulletin - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Databricks has launched Lakebase, a fully-managed serverless Postgres platform built to power the surge of AI-generated applications and autonomous agents by decoupling compute from lake storage. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) To overcome the inherent latency and consistency limitations of object lakes, Lakebase splits intermediate storage into Paxos-driven Safekeepers for writes and low-latency Page Servers for reads. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) The platform provisions compute instances and branches complete databases in under 500 milliseconds, featuring automatic scale-to-zero and instant snapshot rollbacks for agent sandboxing. - [06:03](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=363s) In TPC-C benchmark testing, Lakebase sustained over 600,000 operations per second with sub-10ms latencies, comfortably outpacing major incumbent cloud databases. - [07:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=464s) Lakebase introduced fully-managed cross-cloud disaster recovery between providers like AWS and Azure, surpassing 3,500 enterprise customers within its first year.
## Key takeaways - [00:05](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=5s) Software engineering is undergoing an AI transition, with more code projected to be generated over the next year than in all prior human history. - [00:05](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=5s) Next-generation autonomous agents and AI applications impose three essential database requirements: ecosystem familiarity, infrastructure nimbleness, and mission-critical reliability. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) PostgreSQL was selected as the engine of choice because its open-source adoption and vast web presence make it uniquely familiar to AI agents. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Monolithic Postgres couples compute and storage, making it ill-suited for cloud-native agility without a fundamental architectural decoupling. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Persisting operational database data directly into data lakes resolves storage costs and scale, but introduces latency and transactional consistency challenges. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Databricks introduced Safekeeper nodes running the Paxos consensus protocol to guarantee low-latency writes and ACID consistency on top of lake storage. - [01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Page Servers act as an intermediate caching layer that serves requested database pages directly to decoupled compute instances with ultra-low latency. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) Postgres compute instances can be spun up in less than 500 milliseconds and automatically scale to zero when idle to prevent runaway cloud bills. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) Instant branching enables engineers and agents to spin up fully isolated sandboxes for testing and pull requests in under 500 milliseconds. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) One-click snapshot and restore capabilities provide a safety net against destructive schema alterations or data corruption introduced by autonomous agents. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) Dynamic autoscaling lets administrators configure minimum and maximum compute error bars, adjusting capacity automatically between peak hours and off-peak periods. - [03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) Direct persistence to lake storage delivers infinite capacity, entirely eliminating manual disk expansion and out-of-space operational incidents. - [06:03](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=363s) In TPC-C benchmark evaluations, Lakebase scaled past 600,000 operations per second while preserving transaction latencies below 10 milliseconds. - [06:03](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=363s) Incumbent cloud database offerings exhibited capacity plateaus or severe latency spikes well below Lakebase's throughput ceiling. - [07:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=464s) As enterprises automate operational workflows with agents, single-cloud vendor outages become potentially catastrophic single points of failure. - [07:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=464s) Lakebase provides the first fully-managed cross-cloud disaster recovery, supporting active replication and instant failover between clouds such as AWS and Azure. - [08:57](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=537s) Over 3,500 enterprise customers adopted Lakebase for mission-critical production workloads within its first year of release.
## Detailed walkthrough ### [00:05](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=5s) What Next-Generation AI Applications and Agents Require from a Database
Software development has undergone a fundamental shift: code is increasingly being generated by artificial intelligence rather than handwritten by human developers. Reflecting on the momentum from early 2026, Nikita Shamgunov observes: “And now we can confidently say we're going to have more software generated in the next year than in the history of humanity.” Yet despite this automated deluge, the core architecture remains rooted in fundamental infrastructure: every application still requires a database.
Next-generation AI applications and autonomous agents impose three non-negotiable requirements on their operational databases: - **Familiarity**: Agents operate significantly better when interacting with platforms they understand through their web crawls of internet forums, Stack Overflow, and Reddit. The database needs to be open source, widely adopted, and extensible so organizations can consolidate diverse primary and niche workloads onto a single platform. - **Nimbleness**: To accommodate the flood of AI-generated applications, testing, staging, and development environments, systems must be fully serverless, support instant branching for agent isolation, and maintain strict cost-effectiveness at volume. - **Mission-Critical Reliability**: Operational systems run core business logic. They must offer infinite scalability, low-latency performance, and enterprise-grade reliability.
[01:53](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=113s) Decoupling Postgres and Re-Architecting Storage for the Lake
To build on a proven standard, Databricks selected PostgreSQL, recognized as the world's most advanced open-source database, with an unmatched extension ecosystem and ubiquitous agent familiarity. However, standard Postgres is a traditional monolithic database where storage and compute are tightly coupled. To make it nimble and cloud-native, Databricks had to decouple compute from storage and move the persistence layer directly into the data lake.
Moving operational storage to data lake storage is notoriously challenging. While lake storage is inexpensive and easily scalable, it is inherently slow and transactionally inconsistent. To reconcile standard Postgres ACID guarantees with lake persistence, Databricks split the storage layer into two purpose-built intermediate services: - **Safekeepers**: Dedicated write-path consensus nodes that implement the Paxos consensus protocol to guarantee low-latency writes and strict transactional consistency. - **Page Servers**: Dedicated read-path caching and retrieval nodes designed to serve requested database pages directly to Postgres compute instances with ultra-low latency.
The combination of this decoupled architecture, Paxos consensus write nodes, page-serving read nodes, and underlying lake storage constitutes **Lakebase**: a fully-managed serverless Postgres platform.
```mermaid flowchart TD accTitle: Lakebase Decoupled Architecture accDescr: Diagram showing the decoupled compute, consensus safekeepers, page servers, and lake storage layers of Lakebase.
subgraph ComputeLayer["Compute Layer"] PC["Postgres Compute Engine"] end
subgraph ServiceLayer["Storage Intermediate Services"] SK["Safekeepers (Paxos Consensus / Low-Latency Writes)"] PS["Page Servers (Page Serving / Low-Latency Reads)"] end
subgraph StorageLayer["Persistence Layer"] Lake["Lake Storage (Cost-Effective / Infinitely Scalable)"] end
PC -->|"Write WAL / Transactions"| SK SK -->|"Replicate & Persist"| Lake PC -->|"Request Data Pages"| PS PS -->|"Fetch / Cache Data"| Lake ```
[03:44](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=224s) The Transformative Capabilities of Lakebase
By decoupling compute and storage, Lakebase achieves operational speeds and lifecycle controls tailored directly to AI agent workflows. Postgres compute instances can be provisioned in under 500 milliseconds. To prevent runaway cloud expenditures across dozens of ad-hoc environments created per GitHub pull request, Lakebase automatically scales compute to zero, shutting down inactive instances when not in use.
Shamgunov highlights two database patterns essential for agentic workflows: - **Instant Branching**: Any database can be branched within 500 milliseconds, allowing engineers or autonomous agents to spawn isolated sandbox environments for dev, test, or staging without manual cloning. - **Instant Snapshot and Restore**: Developers or pipelines can take a snapshot via a single API call or mouse click before delegating a task to an autonomous agent. If the agent makes improper schema adjustments or corrupts data, the entire database can be instantly rolled back to the prior snapshot.
From a mission-critical capacity standpoint, administrators only configure guardrail "error bars"—specifying minimum and maximum compute limits for budget and operational bounds. Lakebase automatically scales compute up during peak traffic hours and scales down over weekends and nights. Because storage resides directly in the lake, it is infinitely scalable, completely eliminating manual disk-expansion procedures and out-of-space alerts.
[06:03](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=363s) Industry Benchmarks and Fully-Managed Cross-Cloud Disaster Recovery
Operational database workloads require executing small, highly concurrent queries while sustaining minimal latencies. When tested against standard industry benchmarks such as TPC-C, Lakebase significantly outperformed incumbent cloud solutions: - **Cloud Vendor 1**: Reached capacity at roughly 130,000 operations per minute. - **Cloud Vendor 2**: Sustained operations up to approximately 350,000 operations per second before latencies spiked severely. - **Databricks Lakebase**: Maintained transaction latencies below 10 milliseconds per operation, scaling past 600,000 operations per second.
Beyond standard compliance, encryption, and enterprise security features, Shamgunov unveiled Lakebase's flagship capability: fully-managed cross-cloud disaster recovery. As businesses automate mission-critical operations with autonomous agents, outages at a single cloud provider can halt entire operations. Shamgunov notes: “Cloud outages don't happen every day. Uh but when they do happen, they're they're devastating to your business.”
To eliminate this vulnerability, Lakebase supports instant cross-cloud failover. Users can provision a primary Lakebase instance in AWS US West with an active replica hosted in Azure US East; in the event of an AWS outage, workloads instantly fail over to Azure to maintain uninterrupted business operations. Within one year on the market, over 3,500 enterprise customers have adopted Lakebase for their production workloads.
## Notable quotes > [00:05](https://www.youtube.com/watch?v=NqJIyP9rIW8&t=5s) And now we can confidently say we're going to have more software generated in the next year than in the history of humanity. — Nikita Shamgunov
## Mentioned AWS, Azure, Data + AI Summit 2026, Databricks, GitHub, Lakebase, Nikita Shamgunov, Paxos, PostgreSQL, Reddit, Stack Overflow, TPC-C
## Video description More software will be built in the next year than throughout the whole history of humanity. And every one of these new systems requires an database designed for AI.
It's why Databricks built Lakebase, a fully-managed, serverless Postgres database for AI apps and agents. At Data + AI Summit 2026, Nikita Shamgunov, VP, Engineering of Databricks, explained how the team re-architected Postgres and outlined the features that help Lakebase out perform industry rivals, including the first fully-managed cloud disaster recovery.
00:00 — How AI is changing software development 01:56 — The pros and cons of Postgres 02:28 — Re-architecting Postgres 03:45 — The transformative power of Lakebase 07:27 — Why Lakebase is ready for any workload 07:44 — The first fully-managed cloud DR 08:57 — Lakebase momentum
[Source](https://www.youtube.com/watch?v=NqJIyP9rIW8)