Flapping Airplanes Fights AI Monopolies Via Data Efficiency

Flapping Airplanes借助数据效率对抗AI垄断 除非算法将数据效率提升千倍，否则前沿模型的训练仍将集中在少数顶尖实验室手中。 在大会上，联合创始人Ben Spector和Asher Spector从经济与哲学角度论证指出，数据效率是AI未来的终极瓶颈与竞争护城河。尽管目前的前沿模型在搜索和编程等数据丰富的环境中表现出色，但现实经济中的绝大多数领域都属于数据匮乏领域。为了弥合这一差距，他们的AI实验室Flappy通过构建直接与GPU硬件交互的定制系统和虚拟机，绕开了PyTorch等标准框架的限制。这种“系统+算法”的方法使他们能够运行高度并行的定制工作负载，旨在大幅提高数据效率。 Flapping Airplanes（Flappy）联合创始人 Ben 和 Asher Spector 指出，当前 AI 在搜索与代码等万亿美元级场景的成功，主要依赖于整个互联网等极丰富的数据支撑，而人类学习编程所需的数据量比现有大模型少 10,000 到 100,000 倍。 (1:43) 广阔的现实经济由机器人、量化交易和各行业长尾等“数据匮乏”领域构成；算力能够随着成本指数级下降与市场同质化更容易扩展，但前沿高质量数据收集极度分散且受监管制约，数据效率提高 1,000 倍将使模型部署难度降低 1,000 倍。 (2:46) 数据已成为当前 AI 领域的核心护城河，甚至促使前沿实验室收购二手书店和稀有图书馆以获取利基数据；提升数据效率能够打破少数公司的垄断，促进前沿模型研发的民主化竞争。 (4:20) Flappy 打破 PyTorch 等主流框架的限制，构建了直接接管 GPU 的虚拟机内部框架，通过底层硬件原语深度发掘 GPU 算力极限，以支持小批次、深流水线等非常规并行算法来攻克数据效率难题。 (5:15)

Flappy co-founders Ben and Asher Spector argue that data, rather than compute, represents the primary bottleneck in artificial intelligence, requiring hardware-level co-optimization and data-efficient architectures to unlock the broader economy. The Economic Limits of Data-Rich Pretraining (1:43) Ben Spector observes that frontier large language models excel primarily at search and coding—a trillion-dollar software market—because those sectors offer abundant internet-scale text and synthetic data. In contrast, humans learn to program using 10,000 to 100,000 times less data than current AI systems require. Spector argues that the remainder of the economy, including robotics, scientific research, financial trading, and niche industrial operations like the end-to-end toaster supply chain, lacks vast centralized data repositories. Applying AI across these diverse real-world tasks depends on achieving comparable competence with radically smaller datasets. Compute Scalability Versus Data Centralization (1:43) Asher Spector contrasts the economics of compute with data acquisition. While floating-point operations (FLOPs) benefit from predictable exponential cost declines in a centralized, homogeneous hardware market, collecting frontier-quality data remains operationally fragmented. Sourcing specialized information requires negotiating commercial terms, navigating legal jurisdictions, and in extreme cases, acquiring distressed bookstores or rare libraries. Because data serves as the primary competitive moat for frontier AI labs, Asher warns that relying on data-intensive training concentrates model development within a few well-capitalized firms. Developing models that are orders of magnitude more data-efficient reduces operational deployment hurdles and broadens participation across the AI economy. Bypassing PyTorch for Low-Level GPU Optimization (5:15) To build data-efficient models, Flappy focuses on co-optimizing hardware execution with new algorithms. Asher and Ben Spector argue that mainstream research is constrained by frameworks like PyTorch, which enforce a sequential, single-threaded programming model—such as sequential matrix multiplications and attention layers—over fundamentally parallel GPU architectures. These high-level abstractions struggle to express fine-grained operations or non-linear, deeply pipelined parallel execution loops. Drawing on Ben Spector's background writing GPU mega-kernels, Flappy operates outside standard framework boundaries by running an internal virtual machine that takes direct control of the GPU. Managing hardware execution directly allows the team to run custom, non-standard parallel workloads—such as small-batch, deeply pipelined training loops—which they claim unlock algorithmic primitives necessary for extreme data efficiency. The Spector brothers conclude that circumventing data limits requires co-designing unconventional machine learning algorithms alongside custom low-level GPU infrastructure, moving AI beyond brute-force web-scale data pretraining.

Flappy联合创始人Ben Spector与Asher Spector主张，高质量数据的获取壁垒已取代算力成为人工智能的核心瓶颈，未来的突破将取决于数据利用效率的提升。他们指出，通过深入GPU硬件底层重构执行架构以绕过主流开发框架的局限，是攻克数据稀缺性并打破模型集中化垄断的关键途径。 长尾经济与人类学习的数据效率鸿沟 (1:43) 当前以搜索和代码生成为代表的大语言模型市场规模至少达万亿美元，但这类任务高度依赖从公开互联网抓取的全量数据以及低成本生成的合成数据。Spector兄弟指出，人类在掌握编程等能力时所消耗的数据量，比现有大语言模型少10,000倍至100,000倍，这表明现有架构存在极大的效率损耗。 更为关键的是，真实的宏观经济由海量缺乏完备数据的长尾领域构成。诸如机器人、金融交易、科学探索以及被称为“多士炉端到端供应链”的各类细分实体产业，均无法复制搜索和编程领域的数据富集条件。若无法大幅提升数据效率，人工智能将难以拓展至这类具有无限潜在价值的长尾经济场景。 数据扩展瓶颈与AI生态的去中心化 (1:43) 从资源供给属性来看，算力与数据存在本质差异。浮点运算（FLOPs）成本遵循技术演进呈指数级下降，且算力属于同质化资源，企业可通过直接采购快速扩充。相反，高前沿质量的数据不具备统一交易市场，采集长尾任务数据需要逐一与各商业主体进行商务谈判并遵守繁复的监管合规要求，甚至迫使部分前沿实验室收购破产实体书店和珍稀图书馆以搜寻小众语料。 Asher Spector分析认为，若能开发出数据效率高出1,000倍的模型，技术的实际部署难度也将同步降低1,000倍。在与会者普遍将数据认定为AI行业核心护城河的背景下，克服数据稀缺瓶颈是阻止少数头部机构通过数据独占实现权力集中、进而促进市场多元竞争与技术民主化的必要条件。 接管GPU底层的系统与算法协同优化 (5:15) 在技术实现路径上，Flappy认为当前的机器学习研究普遍受困于主流软件框架的表达边界。物理GPU所能高效执行的计算范围，远大于PyTorch等主流框架所能高效表达的操作子集。现存框架往往将高并行度的底层硬件抽象为矩阵乘法、注意力机制（Attention）和RMS Norm等单线程、顺序执行的编程流，一旦面对高度非线性或定制化的并行工作负载便会出现性能瓶颈。 为突破这一限制，Flappy选择对其核心算法知识产权保持保密，但透露其研发了一套基于虚拟机的内部系统框架，以完全接管GPU的底层执行控制权。Ben Spector以极小批量、深度流水线化的“Hogwild”风格异步训练循环为例，说明这类在传统框架中渐进效率极差的负载，可以通过重构底层执行原语重新变得可行，从而以系统级协同优化解锁服务于数据效率的新算法。 Spector兄弟认为机器学习的技术突破历来倚重对既有硬件极限性能的挖掘而非单纯等待新芯片，但其主张的定制化虚拟机构想完全摒弃了通用的框架抽象，其算法细节与工程可靠性目前尚未完全公开，这种通过低层级工程绕过数据依赖的范式能否在通用任务中兑现数万倍的效率提升，仍有待实际应用验证。

[0:03] Asher: we are not an airplane, we're an AI lab. [1:43] Asher: Compute is easier to scale than data [5:15] Ben: trying to kind of abuse the crap out of GPUs in ways that haven't been done before.

[0:03] Bringing up two frontier spotlights, the [0:04] first is Ben and Asher, two brilliant [0:07] brothers. Yes, they are come on up. Uh [0:09] founders of swimming submarines. Wait, [0:11] no. Wait a second, flapping airplanes. [0:14] Um and we're excited to hear about data [0:16] efficiency. Thanks so much. Thanks, [0:18] Konstantin. [0:20] >> [applause] [0:25] >> All right, yeah. It's uh really great to [0:27] be here. Thanks for having us. Uh we're [0:28] very excited to tell you about why the [0:29] future is data efficient. Um so, let's [0:32] get into it. So, uh first I think [0:34] introductions are in order. Uh my [0:36] background, I spent 3 years in the PhD [0:38] kind of deep in the kernel mines uh [0:41] writing very low-level GPU systems uh [0:43] before helping to start Flappy. And uh I [0:46] I also should say I I at one point [0:47] helped start an incubator called Prod [0:48] that worked with a bunch of companies [0:49] that did well. So, that's also part of [0:51] my background. [0:52] Cool. And so, I'm Asher. Um if you can't [0:54] tell, I'm Ben's older, um wiser, and [0:56] slightly less handsome brother. Um I [0:58] previously was also a Stanford PhD, [1:00] spent time at Cursor, Mercor. [1:02] And then lastly, our third co-founder is [1:03] Aiden Smith. Uh he's a Thiel fellow. [1:05] He's busy working right now. Uh but he's [1:07] spent uh 3 years super commuting between [1:08] his college and our lab. So, he knows a [1:10] lot both about the brain and also about [1:11] machine learning. [1:14] Um so, before we get started in earnest, [1:15] I do want to clarify some confusion. We [1:17] launched the company 3 months ago. Um we [1:20] are very excited to have received a lot [1:21] of inbound, but a surprising amount of [1:23] it has been from the aviation industry. [1:26] Um you know, people trying to sell us [1:28] things like runways, like not not runway [1:30] in the venture capital sense, like [1:31] literal runways. Uh you know, airplane [1:33] parts, wind tunnels, etc. So, I'd like [1:35] to disclaim once and for all, uh we are [1:37] not an airplane, we're an AI lab, and [1:39] hopefully that will become clear by the [1:40] end of this talk. [1:43] Okay, great. Um so, first a quick [1:45] outline. In this talk, we're going to [1:46] tell you about two things. First of all, [1:48] our thesis of why this is a really [1:49] important problem. And second, we'll [1:50] tell you a bit about our approach that [1:52] fuses both systems work and algorithmic [1:54] work. So, uh first on the thesis side. [1:57] So, if we look at the current state of [1:58] the world, LLMs have gotten incredibly [2:00] good at a couple of really valuable [2:02] tasks. So, for example, search, coding. [2:04] Together, these are, you know, at least [2:05] a trillion-dollar market that's probably [2:06] accessible here. A big part of the [2:08] reason why they've gotten so good at [2:09] these tasks, and this is related to what [2:10] uh Andre I think talked about this [2:11] morning, is that these are incredibly [2:13] well-resourced tasks in terms of the [2:15] amount of data that's available for [2:16] them. For search, this is basically the [2:18] entire internet. For coding, this is a [2:20] big fraction of the internet. Coding is [2:21] also a very friendly environment in that [2:23] it's very easy to produce mountains of [2:24] synthetic data if you want to. [2:26] Um so, when we talk about data [2:28] efficiency, what we're asking the [2:29] question of is, is it possible to get [2:31] these kinds of capabilities with much [2:33] less data? Intuitively, it seems like it [2:35] should be possible. Uh you know, humans [2:37] are able to become quite good at coding [2:38] uh with, you know, maybe 10,000 times or [2:41] 100,000 times less data on this than [2:43] these current models take. So, this is [2:45] kind of the core question. And a reason [2:47] why this matters is that if we look [2:48] towards the future, there's a bunch of [2:50] other domains of the economy [2:52] where this uh where there's there's much [2:54] less data. So, for example, you know, [2:55] Jim talked this morning about how in [2:57] robotics, there's a ton of work that has [2:58] to go on to try to generate the data. [3:00] It's much more complicated than it is uh [3:02] in in coding and search. In trading, [3:04] there's only so much financial data. [3:06] Scientific discoveries, obviously very [3:07] little data. Uh but the the uh the [3:10] potential there is sort of unbounded. [3:11] And most important of all is, of course, [3:13] the end-to-end toaster supply chain. [3:15] Now, the reason I bring this up is not [3:16] because of what it is, it's because of [3:17] what it represents, namely that there [3:19] are tens of thousands of things like [3:21] this that really make up the broad [3:22] economy. And the broad economy doesn't [3:24] just look like search and coding, it [3:25] looks like a lot of things that are [3:26] actually really not well-resourced. [3:28] Yeah, and so [3:29] Ben's given you one economic reason to [3:31] believe that the future is data [3:32] efficient. I think a second claim is [3:34] that compute is actually easier to scale [3:35] than data. [3:36] I mean, we know that flops get [3:37] exponentially cheaper over time. And you [3:39] know, I think it's probably true that [3:40] data is not getting cheap as quickly as [3:42] compute is. Although, it is getting [3:43] cheaper. Um I think, you know, the [3:45] second reason is that collecting [3:47] frontier quality is complicated. The the [3:49] compute market is homogeneous in a or is [3:51] more homogeneous than the the data [3:52] market is. Like, you know, Greg was [3:53] telling us after, you know, GPT [3:55] launched, they could just try to buy all [3:57] the compute. And there's no centralized [3:58] data provider. Like, if you want to go [4:00] into the economy and collect the long [4:01] tail of tasks, you have to like deal [4:03] with regulations, you have to negotiate [4:05] with businesses about terms of use. It's [4:07] actually very annoying. [4:08] So, I think if you could As a result, if [4:10] you can make a model that's a thousand [4:11] times more data efficient, I think it'd [4:13] be a thousand times easier to deploy. [4:15] So, [4:16] those are two sort of economic reasons [4:18] that we think data efficiency is [4:19] important. The last reason is actually [4:21] kind of philosophical. [4:23] I think that, you know, if you look at [4:24] the world today, [4:26] um [4:27] there's not that many companies that can [4:28] train AI models. And And you know, part [4:30] of this is because of uh of [4:31] centralization of compute. But I think [4:33] it's also because of centralization of [4:34] data. Like, you know, I've heard news [4:36] about Neo Labs who, when trying to [4:38] create new capabilities, they like [4:39] literally buy out distressed bookstores [4:41] to find all the the data they can find. [4:43] They like go to like rare libraries to [4:44] find all the tiny niches of data that [4:46] you need to actually make a frontier [4:48] model. And I think um in a world that's [4:50] data efficient, I think that companies [4:52] can actually more broadly participate in [4:54] the AI revolution. Like, you know, we [4:55] heard earlier today that uh you know, in [4:58] in the poll about what the moat is in [4:59] the AI world, the the most common answer [5:01] was data. If that's true, then data [5:03] efficiency is the thing that actually [5:04] enables broader competition. So, I think [5:06] if you care about the shape of the world [5:07] to come, I think you really should care [5:09] about data efficiency because it it [5:11] modulates who can actually participate [5:12] in which parts of the AI economy. [5:15] Cool. So, now we're going to talk about [5:17] our approach. Um so, our goal is to [5:19] design data efficient AI. Um we design [5:21] algorithms for this. We're not going to [5:23] talk about them cuz they're our core IP. [5:24] But I will tell you about an important [5:26] facet of our approach, which is how we [5:27] try to look in new places. Like, if you [5:29] want new capabilities, where can you [5:30] find them? [5:32] So, our claim is that essentially, if [5:34] you want to develop new algorithms, you [5:36] should look at new primitives for [5:37] interacting with hardware. There's some [5:39] set of things that GPUs can do [5:40] efficiently. [5:41] And then there's a smaller set of things [5:43] that current frameworks like PyTorch, [5:44] for example, can actually express [5:45] efficiently. And, you know, that that [5:47] gray circle, we've seen a lot of [5:49] research into that circle. We actually [5:50] know what kinds of algorithms work. [5:52] But if you're looking for new [5:53] capabilities, you might want to look in [5:55] these new places. And that's where we [5:57] live at Flappy Airplanes. That's what we [5:58] try to do. [6:00] And you know, Ben will talk for a little [6:01] bit later, for example, about things [6:02] like fine-grainedness, which actually [6:03] are hard to do under current frameworks, [6:04] but a GPU actually can do. [6:08] Um you know, history suggests that this [6:09] is a good bet, for what it's worth. I [6:10] mean, a lot of the development of [6:11] machine learning over the past 15, 20, [6:13] even 100 years has actually been new [6:15] primitives for interacting with [6:16] hardware. Not necessarily designing new [6:18] chips, which is also important, but [6:20] merely figuring out how to squeeze more [6:21] out of the existing technology that has [6:23] proliferated. [6:25] Yeah, so I guess a bit of context. Um in [6:28] my PhD, I did some of this work on on [6:30] early work on mega kernels, uh which was [6:32] trying to make GPUs do weird stuff. Um [6:35] at Flappy, I would say that we are going [6:37] further in spiritually similar [6:38] directions, uh namely trying to kind of [6:40] abuse the crap out of GPUs in ways that [6:42] haven't been done before. Um this has [6:44] been quite a lot of fun. So, I actually [6:45] would recommend it if you if you like [6:46] playing with hardware. And I guess part [6:48] of what I'm excited about in this uh [6:50] talk is that I, you know, this is in [6:53] general I think a fairly high-level [6:54] venue, but I actually want to get into [6:55] the weeds of some stuff because I think [6:56] it's fun. Um [6:58] So, if we look at how kind of current [7:00] systems work, and this will be a little [7:01] bit abstract, but I hope that the the [7:03] concept will come across. What makes [7:05] current machine learning frameworks easy [7:06] to use is that they synthesize a [7:08] single-threaded programming model out of [7:10] very parallel processors. So, basically, [7:12] when you write PyTorch, you write [7:14] matmul, and then attention, and then [7:16] matmul, and then RMS norm. And under the [7:18] hood, there's all kinds of contortions [7:19] that happen where the software [7:20] dispatches it in parallel across a very [7:22] parallel set of processors. But [7:24] nonetheless, to the programmer, this [7:25] looks very natural. [7:26] But what happens if you want to run [7:28] things that look like this? [7:30] Or maybe you want to run things that [7:32] look like this. [7:34] All of a sudden, these kinds of things [7:36] are not easily expressed in current [7:37] frameworks. [7:39] So, uh one thing I'm just very excited [7:40] to show sort of a little teaser of is [7:42] our internal framework that we use, [7:44] which is built off a virtual machine uh [7:47] that kind of just takes over the whole [7:48] GPU, and we just do everything we want [7:49] ourselves on it. Uh in this particular [7:51] case, this is not a real workload I'm [7:52] showing. This is kind of a stylized [7:54] thing that is uh similar, but uh gives [7:57] is an example of something that is also [7:59] asymptotically inefficient to run in [8:00] PyTorch. This is a very small batch kind [8:03] of deeply pipelined hogwild-style [8:05] training loop. Uh this is the kind of [8:06] thing that's very hard to do with with [8:07] current frameworks. So, [8:09] uh the the thing that I sort of like to [8:11] get across here is that when you build [8:13] these new kinds of systems, you enable [8:15] new algorithms. Many of these algorithms [8:16] we think are actually very relevant to [8:18] the data efficiency problems. This is [8:19] why we care about it. Um [8:21] and that it is the kind of [8:22] co-optimization of those things that we [8:24] find really interesting. [8:26] Cool. So, um we're almost out of time. I [8:28] I just really have one more thing to [8:29] say, which is that if you find this [8:30] exciting, please come talk to us. Um you [8:32] know, we we work with lots of folks who [8:34] have trained big models before, but [8:35] we're also trying to do creative new [8:36] things. We're trying to change the [8:37] paradigm. And you know, certainly our [8:39] experience watching uh some of the the [8:41] companies Ben has has helped incubate is [8:43] that, you know, creative people with [8:45] unconventional backgrounds can actually [8:46] do amazing things and change the world. [8:47] So, if you are of that background, [8:49] please come chat with us. [8:51] Okay, so I guess maybe a very quick [8:52] recap, and then I think we probably have [8:53] time for one question or so. Uh [8:56] first of all, data efficiency. If you [8:58] care about the shape of the world to [8:59] come, if you care about the broad [9:00] deployment of AI into the economy, you [9:03] should care about making models much [9:04] more data efficient. And that uh [9:06] you know, our finding has been that [9:08] building new systems to enable more [9:10] fine-grained use of GPUs is actually [9:12] very relevant to building these kinds of [9:13] systems. Um and as Asher said, if you [9:15] find this exciting, please come talk to [9:17] us for a round. [9:19] Thanks, everyone. Thank you. [9:20] >> [applause]