接棒主讲的鸭哥（Yan）是 Superlinear Academy 的联合创始人，曾在 CVPR、NeurIPS、KDD 等顶会发表论文，先后担任过 Microsoft Bing 图像搜索与 Pinterest 搜索的核心技术贡献者，并奠定了 Samsara AI 平台的技术基石。他强调工程研究必须坚持“先测量，再决策”，因为“测量不了的东西，你就没法改进”。
每周都有各种大语言模型发布，Jev 凭什么值得一整节课深入拆解？鸭哥指出关键在于一个震撼业界的数字：“13%”。在 Jev 发布当天，Vercel CEO 宣布其 AI Gateway 上约 13% 的活跃团队开启了 Jev，证明了极高的大盘采用率。TypeSafe 将 Jev 定位为现代软件产品的“System 1”（快思考模块），专门用于又快又便宜的确定性多选决策（如工单分发路由、规则审查等）。从工程指标来看，Jev 的确是个扎实的产品：
- **极低延迟**：通过公网 API 往返决策，耗时仅约 100 毫秒左右。
- **极致廉价**：玩完一整局吃豆人游戏，消耗的 Token 成本不到 1 美分。
- **开箱即用**：纯托管 API 形式交付，开发者侧完全零部署负担。
```mermaid
flowchart TD
accTitle: Real-time Pac-Man Game Engine with Decision Model
accDescr: Architecture of the adaptation layer turning visual game state into 380 tokens sent to the LLM at 30 FPS.
subgraph GameEngine["实时游戏引擎 (30 FPS)"]
GS["游戏内部状态<br/>(生命/剩余豆子/距离)"]
Clock["实时时钟<br/>(超期直接丢弃决策)"]
end
subgraph Adapter["适配层 (Adaptation Layer)"]
PromptGen["生成方向事实描述<br/>(~380 Tokens)"]
end
```mermaid
flowchart TD
accTitle: Decision Flow between Hosted and Self-Owned Models
accDescr: A decision flowchart showing whether to choose hosted models like Jev or train custom lightweight models.
subgraph Evaluation["核心评估维度"]
A["评估业务刚性需求"] --> B{"是否有硬性限制？"}
end
subgraph Options["方案分流"]
B -->|"通用判断 / 零配置 / 接受 ~100ms"| C["选择托管模型 (如 Jev)<br/>成本低廉、即开即用"]
B -->|"超低延迟 (~55ms) / 离线设备 / 单任务压倒一切"| D["拥有并蒸馏自有模型<br/>掌控取舍自由度"]
end
```
[0:00] 上次我们讲到，评价一个技术
[0:01] 最重要的就是找对标
[0:04] 找到它的 best alternative
[0:06] 然后我们在那个视频里边也去评价了 Jev
[0:09] 并且说我们要把 Jev 的这个对标做出来给大家看
[0:13] 这样的话，才能让大家正确地理解
[0:15] Jev 这件事情到底值不值得跟
[0:17] 我的答案是不值得跟的
[0:19] 那鸭哥就把这个做出来了
[0:21] 并且给大家开源了
[0:22] 他用了两个小时的时间
[0:23] 就用一个本地模型完全取代了
[0:25] Jev 的能力、Jev 的性能
[0:27] 而且在 Jev 最核心的那个案例
[0:29] 就是用来打那个吃豆人的那个游戏上
[0:32] 取得了更好的效果
[0:33] 再用了一个小时的时间
[0:35] 去把这个模型蒸馏
[0:36] 变成一个更小的模型
[0:37] 在手机上就可以使用
[0:39] 那它整个的讲解和背后的道理
[0:41] 我们也做成了一个小课程
[0:44] 免费开放给大家
[0:45] 大家感兴趣的话
[0:46] 我们在 10 月 12 号的那期直播课
[0:49] 也会有更加详细、更加系统的讲解
[0:52] 我现在发了以后
[0:53] 这个课程还有两天的报名时间
[0:55] 如果说你错过报名时间的话
[0:56] 也欢迎给 support@maven.com 发邮件
[1:01] 应该还有一个周的时间是可以报名的
[1:03] 接下来我们就放鸭哥的迷你课程
[1:06] 那我们开始吧
And we can get started.
[1:08] 大家好，感谢大家来参加
And hello, everyone, and thanks for joining here.
[1:13] 如果你关注 AI 新闻，大概已经在信息流里刷到过 Jev 了
If you follow AI news at all, you've probably already seen Jev in your feed.
[1:18] 大家看到一个语言模型在玩它从没见过的游戏
Like, people watched a language model playing games it has never seen before.
[1:23] 而且是实时的，速度还快到跟得上游戏
Live, and also fast enough to keep up with it.
[1:27] 今天我们就来拆解这个 demo，用 3 个实验来拆
So today, we're going to take that demo apart, and we'll do it with 3 experiments.
[1:33] 今天你看到的每个数字，都来自我们自己实测的真实运行
Every number you will see today comes from the real runs we measured by ourselves,
[1:38] 除非我特别说明
unless I tell you otherwise.
[1:39] 不过在第一局游戏之前，先简单说说是谁在做这些实验
But before the first game, let me quickly tell you who's running those experiments.
[1:44] 就是我自己
That is myself.
[1:45] 我是 Yan，Superlinear Academy 的联合创始人
I am Yan, and I am a co-founder of Superlinear Academy.
[1:50] 我在 AI 的两端都做了很多年：做研究，也做产品落地
I have spent many years on both sides of the AI, like research and also shipping products.
[1:57] 我的论文发表在 CVPR、NeurIPS 和 KDD 等会议上
My papers have appeared at venues like CVPR, NeurIPS, and KDD.
[2:02] 产品方面，我为 Samsara 的 AI 平台打下了科学基础
And on the product side, I laid the scientific foundation of Samsara's AI platform,
[2:08] 在那之前，我是核心技术贡献者
and before that, I was a key technical contributor
[2:11] 参与过 Microsoft Bing 图片搜索和 Pinterest 搜索
to Microsoft Bing Image Search and Pinterest Search.
[2:16] 今天我还是按老习惯来：先测量，再决策
Today I will work as I always do, measure first and then decide
[2:20] 因为测量不了的东西，你就没法改进
because you cannot improve anything you cannot measure.
[2:24] 我们进入正题。大家知道每周都有新模型发布
And let's go to the meat. We know that new models launch every week,
[2:30] 那为什么 Jev 值得专门讲一整节课？
so why does Jev deserve a whole lesson?
[2:34] 答案就在这个数字里
And the answer lies in this number.
[2:37] 13%
13%.
[2:39] 发布当天，Vercel 的 CEO 说
On launch day, Vercel's CEO said that, oh,
[2:42] 他们 AI Gateway 上约 13% 的活跃团队开启了 Jev
about 13% of the active teams on its AI Gateway turned Jev on.
[2:48] 想想这意味着什么：很多团队第一天就把它用上了
And think about what that means. That's a lot of teams putting it to work on day one.
[2:53] 那他们开启的到底是什么？
So, what exactly were they turning on?
[2:57] 它是一个决策模型
Well, it's a decision model.
[2:59] 更具体地说，TypeSafe 做 Jev
And more specifically, TypeSafe built Jev
[3:02] 是为了让又快又便宜的多选决策变得实用
to make fast and cheap multi-choice decisions practical.
[3:07] 他们把它定位为你产品的 System 1
And they pitch it as a System 1 for your product.
[3:12] 你给它一个情境和几个选项，它选一个，就这样
You give it a situation and a few options, and it picks one. That's it.
[3:16] 比如把工单分给对的团队，或者对照规则检查一个请求
Think of routing a support ticket to the right team, or checking a request against a policy.
[3:22] 老实说，它确实做到了
And honestly, it delivers.
[3:24] 一次决策大约 100 毫秒返回，而且是走公网
So, a decision comes back in about 100 milliseconds, and that's over the public internet.
[3:31] 一整局 吃豆人 下来，花费不到 1 美分
And a whole game of Pac-Man, it will cost less than 1 cent.
[3:35] 你那边也不用做任何部署，因为它就是个 API
And there's nothing to set up on your side, because it's an API.
[3:40] 所以光看成本，没有理由弃用 Jev
So, cost alone is no reason to move off Jev.
[3:44] 得承认，它……它是个扎实的产品
And let's admit, it's… it is a solid product.
[3:47] 但如果它只是分派工单，是火不起来的
But if all it did was just routing support tickets, it won't have gone viral.
[3:54] 那它是怎么火起来的？
So, what made it go viral?
[3:57] 让 Jev 爆火的其实是游戏。发布帖
What made Jev go viral was actually games. The launch post
[4:01] 在 X 上的浏览量超过了 3500 万
passed like 35 million views on X.
[4:04] 大家转发的 Jev 片段，不是它在分派工单
And all the clips people share about Jev were not like Jev triaging support tickets.
[4:11] 而是它在玩从没见过的游戏
They were Jev playing games it has never seen.
[4:15] 为什么这件事这么有意思？
And why is that so interesting?
[4:17] 因为它打破了我们大多数人的一个直觉
Because it breaks an intuition most of us had.
[4:21] 我们觉得语言模型又慢又啰嗦
Like, we think of language models as slow and chatty.
[4:24] 你问一个问题，它给你写一大段
You ask a question, it writes you a paragraph.
[4:28] 但游戏不会等你。错过一步，时机就没了
But a game doesn't wait for you. You miss a turn and the moment is gone.
[4:32] 一个语言模型实时操控角色，这让大家很惊讶
So, a language model steering a character live, that's what surprised the people.
[4:38] 于是这些片段就传开了
And the clips traveled.
[4:40] 你现在看到的是 Jev 在我们的浏览器里玩 吃豆人
And what you're watching here is Jev playing Pac-Man in our browser,
[4:45] 而且是按实时时钟
and on a real-time clock.
[4:48] 这个游戏是我们自己写的，已经开源了
We wrote this game ourselves, and open sourced it.
[4:51] 稍后会贴出 GitHub 链接。所以你可以……
We'll paste the GitHub link later. So, you can…
[4:54] 我们能读取它的内部状态，能控制时钟，能做实验
We can read it, like, the internal state, we can control its clock, we can do experiments.
[5:00] 这里我们选了 吃豆人
And here, we pick Pac-Man.
[5:02] 因为没见过的游戏正是 Jev 自己的招牌
Because unseen games are Jev's own showcase,
[5:06] 所以这确实是在它主场上的公平测试
so this is indeed a fair test on its home turf.
[5:11] 爆火的 demo 往往会给背后的机制罩上光环
And now, a viral demo turns out to wrap the mechanism in a halo.
[5:16] 那我们怎么看清这个 demo 背后的东西？
And how do we look behind this one?
[5:20] 为此我们会做 3 个实验，每个都建立在前一个之上
And to figure this out, we'll run 3 experiments, and each one built on the one before.
[5:25] 第一，接上它
First, we'll hook it up.
[5:27] 我们会演示怎么把 Jev 接到实时 吃豆人 游戏上
We will show how to connect Jev to a live Pac-Man game,
[5:31] 看看它到底读到了什么
and look at exactly what it reads.
[5:34] 第二，替换它。我们换上一个普通 LLM
And second, we replace it. We swap in an ordinary LLM.
[5:37] 说清楚一点，Jev 也是 LLM，只是专门为决策训练过
And to be clear, Jev is an LLM too, but this one has been trained specifically for decisions.
[5:43] 我说的普通 LLM，是指那种
And when I say an ordinary LLM, I mean this kind of LLM
[5:47] 今天就能下载、完全没做过决策训练的 LLM
you can download today with no decision training at all.
[5:51] 第三，打败它
And then, third, we beat it.
[5:54] 模型到了我们手里，我们就训练一个小模型
Once the model is ours, we'll train a small model
[5:57] 把这个游戏玩得更好，甚至比 Jev 还好
that plays this game better, even better than Jev.
[6:01] 那么，实验一
So, experiment one.
[6:03] 怎么把语言模型接到游戏上？
How do you connect the language model to a game?
[6:07] 设置是这样的
Here's a setup.
[6:08] Jev 从来看不到屏幕
So Jev never sees the screen.
[6:10] 它没有多模态能力
It doesn't have multimodality capabilities.
[6:13] 它的 API 接收数据形式的情境，加上一组选项
Its API takes the situation as data, and plus a list of options.
[6:17] 今天测的每个模型都是这样
And that's true for every model we test today.
[6:21] 然后游戏不停地问模型同一个问题
And then the game keeps asking the model one question.
[6:24] 下个路口往哪走？下个路口往哪走？
Oh, at the next junction, which way? At the next junction, which way?
[6:28] 所以得有个东西把屏幕上的画面变成文字
So, something has to turn the picture on the screen into words.
[6:32] 这就是我们说的适配层
And that's what we call the adaptation layer.
[6:35] 对 吃豆人 能走的每个方向，它会写下几条事实
For each direction the Pac-Man could take, it writes down a few facts.
[6:41] 比如那个方向有没有鬼？多远？有几颗豆？
Like, is there a ghost that way? How far? How many pellets?
[6:45] 最近的大力豆有多远？整段描述大约 380 个 token
How far is the nearest power pellet? This whole description is about 380 tokens,
[6:51] 老实说，这层是 Claude Code 帮我们写的，并不难
and honestly, Claude Code wrote this layer for us, and it's not that hard.
[6:57] 任何接入 Jev 的人，也都得写
And anyone who wires up Jev also has to write
[7:01] 一个这样的东西
One like this, too.
[7:03] 为了公平，所有实验里每个玩家读到的输入都完全一样
And for fairness, for all the experiments, every player reads exactly the same input.
[7:10] 再来说时钟
And now, the clock.
[7:11] 游戏以固定的 30……
It's a game because it runs at a fixed 30…
[7:15] FPS 运行，不等任何人。如果答案来晚了
FPS, and it doesn't wait for anybody. If the answer arrives late,
[7:20] 吃豆人 已经过了路口，这个决策就会被丢弃
And Pac-Man has already passed the junction, that decision, we'll drop it.
[7:25] 我们用固定种子玩 10 局，每局最长 5 分钟，数吃了几颗豆
So, we play 10 games on fixed seeds, and each up to 5 minutes, and count the pellets.
[7:32] 那从模型那边看，这段文本长什么样？
So what does this text look like from the model side?
[7:36] 这些是 Jev 某一局里的真实决策，是实时记录下来的
These are the real decisions from one of Jev's games, logged as, when they happened.
[7:41] 上半部分是 Jev 收到的游戏状态，比如剩几条命、剩几颗豆
The top half is the game state that Jev received, like the lives, the pellets left.
[7:48] 然后是到下个路口的距离
And then, the distance to the next junction,
[7:51] 再是它能走的每个方向各一行
and then one line for each direction it can take.
[7:54] 每个数字都是以迷宫格为单位的距离
Like, every number is a distance in maze tiles,
[7:57] 从 吃豆人 当前位置算起
so it's counted from Pac-Man's current position.
[8:01] 比如这一行说，如果 吃豆人 掉头，6.3 格外有只鬼
So, for example, this one says, there is a ghost 6.3 tiles away if Pac-Man turns back.
[8:09] 游戏什么时候发问？
And when does the game ask?
[8:12] 答案一回来，它就接着问前面的下一个路口
As soon as an answer comes back, it asks about the next junction ahead.
[8:16] 下半部分是 Jev 的回答
So, the bottom half, this is Jev's answer.
[8:19] 它选一个方向，并给每个选项一个概率，而且常常不太确定
It picks up one direction, plus a probability for each option. And it's often not very sure.
[8:26] 比如第一个决策里，「上」只拿到大约一半的概率
And in this first, decision, the up just wins about half of the probability.
[8:32] 跟第二个选项差不多
Compared with the second.
[8:34] 右边是调用托管模型的往返时间，单位毫秒
And on the right, it's the round trip to the hosted model in milliseconds.
[8:40] 比如这次往返调用是 140 毫秒
And, for example, this is 140 milliseconds for this round trip call.
[8:45] 这次的总开销大约是 600 个输入 token
The total cost for this is about 600 input tokens.
[8:49] 前面说过，我们用 380 个 token 来描述情境
We mentioned that we had 380 tokens for… to describe the situation,
[8:54] 多出来的是 Jev 自己加的，大概是固定指令
and the extra were added by Jev itself. It probably is fixed instructions.
[9:01] 我们来看几个路口。这是 吃豆人 的位置
So, let's watch a few junctions go by. This is where the Pac-Man is.
[9:06] 它问……它构造出这段文本，问 Jev 当前情况
It asks… it constructs this and asks Jev what is the situation,
[9:10] Jev 返回结果。根据这个结果
and Jev returned the result. Based on the result.
[9:14] 它沿着路走，继续玩
It goes along the way and play the game.
[9:17] 可以看到，Jev 基本上是先奔着大力豆去
And we can see that essentially Jev heads for the power pellet,
[9:21] 等鬼变成可以吃的，它就去追一只
and then once the ghosts turn edible, it chases one.
[9:24] 这就是幕后发生的事
This is what happened behind the scene.
[9:28] 就模型所知，这就是整个世界
And that's the whole world, as far as the model knows.
[9:31] 没有画面，也没有记忆
And no, there's no picture, no memory.
[9:34] 那仅凭这些，Jev 玩得怎么样？
So, how well does Jev play from just this?
[9:38] 结果是 178
And it turns out it's 178.
[9:42] 一关有 244 颗豆，Jev 一局通常就是吃 178 颗
A level has 244 pellets, so 178 pellets is what a typical Jev game looks like.
[9:48] 吃豆人 会吃掉这么多豆
The Pac-Man would eat this many pellets.
[9:52] 这是实时时钟下 10 局的平均值
And that's the average over 10 games on the real-time clock.
[9:57] 走公网，每次决策大约 115 毫秒
At about 115 milliseconds per decision across the public internet.
[10:03] 我觉得这是个不错的基线，也是我们要超越的标杆
I would say it's a respectable baseline, and it's the bar to beat.
[10:07] 真正的问题是：这件事是只有 Jev 能做
So here's the real question. Is this something only Jev can do,
[10:12] 还是普通 LLM 也能做？
or can an ordinary LLM do it too?
[10:16] 结果是，它们也能
Turns out, they can.
[10:17] 我们把完全相同的文本给了 5 个不同的模型
So we give the exact same text to five different models.
[10:21] 其中 2 个是专门的决策模型
Two of them are dedicated decision models.
[10:23] 比如 Together AI 有个决策模型叫 Tev1
Like Together AI, they have a decision model called Tev1.
[10:27] 它有 40 亿
And it has 4 billion
[10:30] 参数。它吃到 148 颗豆，Jev 前面看到是 178，差得不远
parameters. This one gets 148 pellets, and Jev, as we saw, is 178. So, not too far away.
[10:40] 另外 3 个是没做过决策训练的普通 LLM
And the other three are ordinary LLMs with no decision training.
[10:46] 我们测了 Qwen3.5 4B，它吃到 197 颗
And we tested the Qwen3.5 4 billion, it eats 197,
[10:52] Google 的 Gemma 4 E4B 是 230 颗
and Google's Gemma 4 E4B, it's, 230.
[10:56] Microsoft 的 Phi-4-mini 吃到 232 颗
And Microsoft's Phi-4-mini eats 232.
[11:00] 所以在这个任务上，没有什么是 Jev 独有的
So, on this task, nothing here is unique to Jev.
[11:05] 普通 LLM 的得分其实和决策模型一样好
The ordinary LLM actually scored as well as the decision models,
[11:08] 最好的那个还更高
and the best one actually scored higher.
[11:12] 我们让最好的那个和 Jev 正面对决
So, let's put the best one head-to-head with Jev.
[11:16] 这是 Phi-4-mini，一个 38 亿参数的普通 LLM
This is a Phi-4-mini, and it's an ordinary 3.8 billion parameter LLM
[11:21] 没做过任何决策训练
with no decision training.
[11:22] 我们只加了 JSON 约束，类似 GPT API 的 JSON mode
All we added was, like, JSON constraint, pretty much like a JSON mode in GPT API.
[11:28] 所以它的回答格式总是对的
So, its answer always come back in the right shape.
[11:31] 可以当普通 JSON 来解析
We can parse it as a regular JSON blob.
[11:33] 左边 Phi-4-mini 平均 232 颗豆，右边 Jev 是 178 颗
And on the left, Phi-4-mini averages 232 pellets, and on the right, Jev achieved 178.
[11:42] 但对 Jev 公平地说，它更快，确实更快
But to be fair to Jev, it's faster. It's indeed faster.
[11:46] 每次决策 100 毫……115 毫秒
It's 100 milli… and 15 millisecond per decision,
[11:50] 而 Phi-4 每次决策要 187 毫秒
and while, this Phi-4 uses, 187 milliseconds per decision,
[11:55] 但有个前提：我们是在 Mac 上测的
but there's a caveat. We measure this on a Mac.
[11:59] M3 Mac。我们也在 NVIDIA 显卡上试过，如 RTX 5090
M3 Mac. We also tried that out on NVIDIA cards, like RTX 5090,
[12:04] 在 NVIDIA 显卡上，4B 模型读同样的状态快约 7 倍
and on the NVIDIA cards, a 4B model reads the same state about 7 times faster,
[12:10] 所以它其实……甚至可能比 Jev 快很多
so it's actually… it could be a lot faster compared with even Jev.
[12:16] 为了公平，我们还按种子逐局配对，让两个玩家
And to be fair, we also paired the game seed by seed, so both players are
[12:21] 面对完全相同的迷宫。这样配对后
face exactly the same mazes and matched that way,
[12:25] Phi-4-mini 每局多吃 54 颗豆，标准误为 24
Phi-4-mini would eat 54 more pellets per game with a standard error bar of 24.
[12:32] 不过……在你把 Jev 换掉之前，有两个合理的质疑
But… Before you rip Jev out, there are two fair objections.
[12:37] 第一个质疑是可靠性
The first objection is reliability.
[12:39] 普通 LLM 能不能连续几千次都保持输出干净？
Can an ordinary LLM keep its output clean, like, thousands of times in a row?
[12:46] 答案其实是能
And here the answer is actually yes.
[12:49] 在近 6000 次决策里，3 个普通 LLM 的无效输出为 0
Across nearly 6,000 decisions, three ordinary LLMs produce zero invalid outputs.
[12:55] 所以格式问题不是非用专门模型的理由
So, formatting is no reason to need a special model.
[12:59] 第二个质疑是通用性
And the second objection is the generality.
[13:02] 吃豆人 只是一个任务，而外面有几百万种不同的任务
We know that Pac-Man is just one task, and there are millions of different tasks there.
[13:07] 所以我们来看看外部证据
So, let's look at outside evidence.
[13:10] 有个开源项目
There is an open source project.
[13:12] 以前叫 OpenJev，后来不知怎么改名叫 SemIf 了
Previously, it was called OpenJev, and then somehow it changed its name to SemIf.
[13:18] 他们复现了 Jev 的 102 个官方判断任务
They rebuilt 102 of Jev's official judgment tasks
[13:21] 底座是冻结的普通 Qwen3.5 4B，完全没训练
on top of a regular frozen Qwen3.5 4B, with no training at all.
[13:26] 就是我们说的普通 LLM 加 JSON mode
So it's just an ordinary LLM, as we said, with JSON mode.
[13:31] 这个普通的 Qwen3.5 拿了 84.5%，Jev 是 88.3%
And this mere Qwen3.5 model scored 84.5%, and Jev scored 88.3%,
[13:38] 相当接近，而且它没做任何后训练
so that's pretty close, and there's no post-training on it.
[13:45] 但事情还有另一面
But there's another side of the story.
[13:48] 有个叫 JevBench 的基准，里面有 220 道难题
There are 220 hard questions on a benchmark called JevBench.
[13:54] 这里就有差距了
There is a gap.
[13:56] Jev 得 74.1%，SemIf 只有 59.5%
Jev scores 74.1%, and SemIf only scored 59.5%.
[14:02] 所以我觉得结论是这样的
So I would say the takeaway is this.
[14:05] 吃豆人 只是一个任务。很多任务上，没训练过的普通开源模型
Pac-Man is one task. There are a lot of tasks that just a regular, untrained open-source model
[14:11] 就能做得很好，跟 Jev 差不多，可能更快、更便宜
would do really well, pretty much like Jev. It could be faster, it could be cheaper,
[14:16] 但也有些任务，它们很难达到 Jev 的水平
but there are also other tasks that they struggle to match Jev's performance.
[14:21] 所以，换之前先在你自己的任务上测一测
So, measure on your task before you switch.
[14:24] 现在假设普通 LLM 在你的任务上过关了
And now, suppose the ordinary LLM does pass on your task.
[14:28] 你能用它做哪些托管 API 做不到的事？
What can you do with it that you cannot do with a hosted API?
[14:33] 其实能做的挺多
And quite a lot, actually.
[14:36] 模型一旦归你，你就可以继续往前推
Once the model is yours, you can keep pushing it.
[14:39] 首先，让它变快
And first, we make it fast.
[14:41] 我们会演示怎么让它变快，再演示怎么让它更聪明
We'll show you how to make it fast, and then we'll show you how to make it smarter,
[14:46] 然后试着两者兼得，看看要付出什么代价
and then we'll try to get both at once, and look at what that costs us.
[14:51] 先从刚才看到的差距说起：速度
So let's start with the gap we just saw, the speed.
[14:54] Jev 比 Phi-4-mini 快。这速度是从哪来的？
Jev was faster than the Phi-4-mini. So, what does that speed come from?
[15:00] 其实发布后 72 小时内
Actually, within 72 hours of the launch,
[15:03] 开源社区就已经有 4 个能跑的复现版本，它们能……
the open source community already had 4 working reproductions, and they can…
[15:08] 在一定……在相当程度上追平 Jev 的速度
to some… to quite some extent, they can match Jev's speed.
[15:12] 这是推理工程，谁都能做
And it's inference engineering, so anyone can do it.
[15:15] 还记得刚才说的 SemIf 吗？它的作者给这些技巧计了时
Remember SemIf, that from a minute ago? Its authors timed those tricks.
[15:21] 他们有个基准，包含 700 多次决策
They have a benchmark with more than 700 decisions,
[15:25] 如果每个问题都从头重算一遍
and they're recomputing everything from scratch for every question,
[15:30] 要花 333 秒
and it would take 333 seconds.
[15:33] 怎么优化呢？
And how to optimize for that?
[15:35] 第一个思路：关于同一情境的每个问题
The first idea is, oh, every question about the situation,
[15:39] 开头都是同一段描述
they start with the same description.
[15:41] 这正是前缀缓存的经典用法
And that's the classic use of a prefix cache.
[15:45] 共享的部分只算一次，然后复用
So, they will compute the shared part once and reuse it.
[15:49] 在同一基准上，这把时间降到了 72 秒
That brought down to 72 seconds on the same benchmark.
[15:54] 第二个思路：在真实产品里
And the second idea is, in a real product,
[15:56] 你常常会就同一情境问好几个问题，比如一张工单
you often ask several questions about the same situation, like, this is a ticket.
[16:01] 急不急？该给哪个团队？需不需要人工？
Is it urgent? Which team should get it? Does it need a human?
[16:05] 那就一起问，而不是一个个问，放进一个 batch
So, you ask them together instead of one by one, and in one batch.
[16:09] 这把时间降到了 38.8 秒
That brought down to 38.8 seconds,
[16:12] 比起点快了将近一个数量级
nearly an order of magnitude faster than where we started.
[16:16] 还有第三个思路：干脆不写答案
And there is even a third idea, that just don't write an answer at all.
[16:21] LLM 要逐个 token 地写答案，这才是最耗时的部分
LLMs try to ask to write an answer token by token, and that's the time-consuming part.
[16:27] 如果给每个选项一个字母
What if we just give each option a letter,
[16:30] 只让 LLM 给我一个字母，就这样
and just ask LLM, just give me a letter, that's it.
[16:33] 然后读出模型偏向哪个字母，这也能省很多时间
And then we read which letter the model favors. That could also save a lot of time.
[16:39] 只要 1.02 秒，而写 JSON 要 5.33 秒
That took only 1.02 seconds against, like, 5.33 seconds for writing JSON.
[16:44] 当然……
And…
[16:46] 这也是有代价的
We also pay price for that.
[16:48] 跟 Jev 的做法差不多，就是输出很窄
It's pretty much the same as what Jev did, it's just a narrow output.
[16:52] 模型只能从你给的字母里选一个
So the model can only pick one from your letters.
[16:55] 但这样一来，速度基本就够得着了
But then, the speed is pretty much within reach.
[16:59] 那……在这个速度下，能不能也让它玩得更好？
So… Can we also make it play much better with this speed?
[17:04] 可能不是在这个速度下，而是做到这个速度之后
Probably not with this speed, after we achieve this speed.
[17:08] 答案是可以，但要用更大的模型，或者让它思考
And the answer is yes, but with a bigger model or by letting it think.
[17:13] 我们已经知道，在实时时钟下，模型大小本身就有帮助
So, we already know that on the real-time clock, size already helps.
[17:19] 0.8B 的 Tev1 吃 69 颗豆，4B 版吃 148，Jev 是 178
Like, Tev1 at 0.8 billion eats 69 pellets, the 4B version eats 148, and Jev sits at 178.
[17:27] 现在换个大得多的：Qwen3.8-27B
And now let's go much bigger to Qwen3.8, 27 billion.
[17:31] 这里有个值得知道的技巧。当模型太慢跟不上时钟
And here's the trick worth knowing. When a model is too slow for the clock,
[17:36] 或者你想让每次运行都能精确复现
or if you want to want runs that are exactly reproducible,
[17:39] 或者只是想收集数据
or if you want to just collect the data,
[17:41] 就可以让游戏在每个路口等模型。我们管这叫 lockstep
you can let the game wait for the model at every junction. We call that lockstep.
[17:46] 这是个选项，不是必需的
It's an option, not a requirement,
[17:48] 它能让你看到，不受时间限制时决策能有多好
and it is able to show you how good decisions can get when the time is not a constraint.
[17:54] 所以这些柱子要和实时的分开来看
So, reading these bars as a separate track from the real-time ones.
[17:58] 不开思考，27B 吃到 301 颗豆！
Without thinking, 27 billion is 301 pellets!
[18:03] 等一下，一关只有 244 颗豆，怎么能吃 301 颗？
But now, wait. A level only has 244 pellets, so how do you eat 301?
[18:08] 吃豆人 清完一关后，我们让游戏
So, when a Pac-Man clears a level, we ask the game,
[18:12] 进入下一关，换一张新地图
move on to the next one with a fresh board.
[18:15] 所以超过 244 就说明它清完第一关又接着玩了
So, anything above 244 means it just cleared the first level and kept going.
[18:21] 打开思考后是 531
And with thinking turning on, it is 531.
[18:24] 比两整张地图的豆还多
That's more than two full boards worth of pellets.
[18:29] 但问题来了
But here's the catch.
[18:30] 开思考后，即使在 NVIDIA 显卡上，每次决策也要约 2.4 秒
With thinking, each decision takes about 2.4 seconds, even on an NVIDIA card.
[18:36] 按实时时钟，吃豆人 早就跑没影了
So, on the real clock, Pac-Man would be long gone.
[18:41] 怎么办？我们想要大模型的水平、小模型的速度
So what do we do? We want the big models play at a small model speed.
[18:46] 而这正是微调的用武之地
And that's exactly what fine tuning is for.
[18:49] 还记得 lockstep 吗？我们就是这样收集数据的
And remember lockstep? That's how we collect the data.
[18:53] 我们是这么做的
So here's what we did.
[18:54] 我们让 27B 模型离线标注了一大批游戏状态
We let the 27 billion model label a large set of game states offline.
[19:00] 想花多少时间就花多少时间
And then taking all of the time it needed.
[19:02] 然后训练了一个 0.8B 的学生模型去模仿这些标注
Then, we trained a 0.8 billion student to copy those labels,
[19:07] 用机器学习的术语说，就是蒸馏
or, in machine learning terms, to distill them.
[19:10] 玩的时候它只读当前状态，和大家一样的 380 个 token
At playtime, it reads only the current state, and the same 380 tokens everyone else gets.
[19:16] 其实这部分不需要博士学位，也不需要什么配方
Actually, you don't need a PhD degree or a recipe for this part,
[19:20] 因为现在的 coding agent 已经会做了
because modern coding agents already know how to do it.
[19:23] 跟 Claude Code 说一句：训个小模型蒸馏这些标注，就行
You just tell Claude Code, oh, train a small model to distill those labels, that's it.
[19:28] 它知道该怎么做
And it knows what to do.
[19:29] 如果你想看它具体怎么做的，我们的代码也开源了
And if you want to see exactly how it did it, our code is also open source,
[19:34] 链接会在课后邮件里
and the link will be in the email after this lesson.
[19:38] 那它玩得怎么样？
So, how does it play?
[19:39] 实时时钟下，它吃到 425 颗豆，是 3 轮实时运行的平均
On the real-time clock, it eats 425 pellets, average over three real-time rounds
[19:46] 每次决策 55 毫秒。在这个任务上，是 Jev 178 的两倍多
at 55 milliseconds per decision. And at this task, that's more than twice of Jev's 178.
[19:53] 更妙的是，它小到可以离开数据中心
And what's better is it's small enough to leave the data center.
[19:57] 能部署到 iPhone 上。视频里它就跑在我的 iPhone 上
Like, we can deploy it on an iPhone. Here's… this video shows it's running on my iPhone,
[20:03] iPhone 16 Pro Max，app 也是让 Claude Code 写的
iPhone 16 Pro Max, and in an app we build, we also ask Claude Code to do it,
[20:08] 每次决策大约半秒
at about half a second per decision.
[20:11] 又小又快，吃豆人 还玩得更好。像是免费午餐，对吧？
So, it's small, it's fast, it's better at Pac-Man. Looks like a free lunch, right?
[20:17] 那账单在哪？
So, where's the bill?
[20:19] 账单在这。我们给学生模型出了 194 道通用判断题
Here's the bill. We give the student 194 general judgment questions.
[20:23] 涵盖意图、工单分派、规则检查、严重度、信息抽取等
They cover intent, ticket routing, policy checks, severity, extraction, everything,
[20:29] 没有一道和 吃豆人 有关
and there's no Pac-Man in any of them.
[20:32] 得分是这样的
And here's the score.
[20:34] 同样大小的 Tev1 0.8B 得 0.66
The same size, Tev1, 0.8 billion, score 0.66,
[20:39] 我们蒸馏的 0.8B 只有 0.53
and our distilled 0.8 billion scores only 0.53.
[20:45] 而 Jev 在这些题上公布的得分是 0.89
And Jev, they published scores on those questions on 0.89.
[20:51] 但有一点要说明
But there's one caveat.
[20:53] 我们的学生模型不是从 Tev1 起步的
Our student started not from Tev1.
[20:56] 而是从普通的 Qwen3.5 0.8B 基座起步
Our student, started from the regular Qwen3.5, 0.8 billion base.
[21:01] 所以部分差距在于它从没接受过通用训练
So part of the gap is it never receives the general training.
[21:06] 抱歉，是决策训练
Sorry, receives the decision training.
[21:09] 另一部分是它只擅长这一个任务
And another part is it's only good at this task.
[21:15] 然后退一步看
And then step back.
[21:16] 这就是我们所做一切的规律：处处都有取舍
That's the pattern for everything we did. We see trade-offs here.
[21:21] 速度的代价是输出变窄，智能的代价是等待时间
We see speed causes narrow output, we see intelligence causes waiting time,
[21:25] 小而专的模型让我们付出了广度
and we also see that small specialists cause us breadth here.
[21:30] 那托管模型和自有模型之间，该怎么选？
So how do you decide between a hosted model and your own?
[21:34] 我的建议是，从你产品不能妥协的那一行出发
And my suggestion is, start from one row your product can't compromise on.
[21:38] 如果你要零配置、开箱即用的广泛判断能力
If you want a zero setup, you want broad judgment out of the box,
[21:42] 托管的 Jev 完全是个好选择。走公网约 100 毫秒
hosted Jev is a perfectly good answer. It's about 100 milliseconds over internet,
[21:47] 每局不到 1 美分，所以别只为了省钱而切换
it's under a cent per game, so please don't switch just to save on price.
[21:52] 但当其中某一行真的卡住你时，就该切换
But switch when one of these rows really bites.
[21:55] 也许你需要更低延迟，比如我们 0.8B 在 Mac 上的 55 毫秒
Maybe you need a lower latency, like the 55 milliseconds of our 0.8 billion on a Mac.
[22:01] 也许得在设备上离线跑，或者某个任务远比其他重要
Maybe it has to run offline on a device, or maybe one task matters far more than others.
[22:07] 自己拥有模型，就能自己调整这些行
And owning the model by yourself allows you to move these rows by yourself.
[22:13] 这就是关于 Jev 的决策
And that's the decision for Jev.
[22:15] 但还有一个更大的道理
But here's a bigger lesson that
[22:18] 比 Jev 本身还重要
Even more important than Jev itself.
[22:21] 每个热门技术都捆绑了两样东西
Every hot technology bundles two things.
[22:23] 一个长期存在的问题，和应时的包装
A problem that lasts, and the packaging from its moment.
[22:27] Jev 只是今天的热门名字
So Jev is just today's hot name.
[22:29] 没人会帮你拆开它，因为拆开就摘掉了光环
Nobody's going to split that bundle for you, because splitting it takes off the halo.
[22:34] 那我们自己来拆。什么是长期的？
So let's split it by ourselves. What lasts?
[22:38] 产品需要又快又便宜的决策
Products need fast and cheap decisions.
[22:41] 让决策变快的技巧，比如复用共享前缀、读答案字母
The tricks that make them fast, like reusing the shared prefix or reading answer letters,
[22:46] 对任何模型都管用
they work with any model.
[22:48] 在自己的任务上做配对测量，清楚每个取舍的代价
You measure on your own task, with paired rounds, and you know what each trade costs.
[22:54] 这些都能带到下一个模型上。而在你的产品里
All of it carries over to the next model. And in your product.
[22:57] 你的系统本来就掌握状态，所以不需要游戏引擎
Your system already holds the state, so you don't need a game engine.
[23:01] 包装呢？品牌名、System 1 标签、爆火的游戏片段
And the packaging? A brand name. A System 1 label. The viral game clips.
[23:06] 明年又会换个名字
Next year, there will be a different name.
[23:10] 所以下次有东西爆火时，问自己两个问题
So, next time something goes viral, ask yourself two questions.
[23:14] 如果模型强 10 倍，这件事还重要吗？
If the model were 10 times better, would this still matter?
[23:17] 它到底在解决什么问题，我还有这个问题吗？
And what problem is it actually solving, and do I still need to still have it?
[23:23] 怎么拆分是一种判断
Making that split is a judgment call,
[23:25] 判断力来自练习，还要有人帮你把关
and judgment comes from practice, with someone checking your work.
[23:29] 下面说说怎么获得这种练习
So, let me tell you how to get that practice.
[23:32] 今天你看到的一切都遵循同一个循环
Everything you saw today followed one loop.
[23:34] 向模型描述情境，在真实任务上测量
We described the situation to the model, we measured on the real task,
[23:38] 再决定拿什么去换
and we decided what to trade.
[23:40] 这个循环说起来简单
That loop is simple to describe,
[23:42] 但看我们做一遍，和你在自己的问题上亲手做
but watching us do it once and doing it yourself on your own problem
[23:47] 是完全不同的两回事
are two very different things.
[23:49] 这正是我们的课程 AI Builders 2027 要解决的
And that's what our course, the AI Builders 2027, is built for.
[23:53] 这门课有 2 次直播课，每次 2 小时
This course has two live sessions, two hours each,
[23:57] Yuzheng Sun 博士和我每次各讲一半
and Dr. Yuzheng Sun and I would teach half of every session.
[24:01] 第一次课，Yuzheng 先讲到底该怎么学 AI
In the first one, Yuzheng starts off how to learn AI at all.
[24:05] 他的答案是动手做，因为具体需求能过滤掉炒作
His answer is to build, because a concrete need filters out the hype.
[24:11] 然后我接着讲上下文
Then, I will take over with the context.
[24:13] 今天模型只知道适配层告诉它的东西
Today, the model only knew what our adapter told it,
[24:16] 所以我们从一个简单的聊天框出发，现场搭建
so we start from a plain chat box and build it up, live,
[24:20] 直到变成 AI 能用你真实数据和技能的工作空间
to a workspace where the AI works with your real data and your real skills.
[24:25] 第二次直播课，我讲测量
And in the second live session, I cover the measuring,
[24:28] 这也是今天这节课的核心
which is the heart of today's lesson too.
[24:31] 我们现场做一个小产品，在写第一个 prompt 之前
We'll build a small product live, and the first thing we build before a single prompt
[24:36] 最先搭的是评测：要有数据集、指标和基线
is the evaluation. We need a dataset, we need a metric, and a baseline.
[24:40] 然后 Yuzheng 会用一个问题收尾，正好呼应今天
Then, Yuzheng will close with a question that connects right back to the day.
[24:45] 当你几乎什么都能做时，怎么判断什么值得做？
Once you can build almost everything, how do you decide what's worth building?
[24:51] 围绕直播课，你先做 5 个引导项目
And around the sessions, you build 5 guided projects first,
[24:55] 再围绕你工作中的问题做一个毕业项目
then a capstone on a problem from your own work.
[24:59] 那你具体会做什么？
So what will you actually build?
[25:01] 路径是这样的
Here's the path.
[25:03] 首先是 5 个引导项目，它们有同一个主题
First, five guided projects. They share one theme,
[25:06] 让 AI 读取和操作为人设计的东西，比如网页和 GUI
getting AI to read and operate things built for humans, like web pages and GUIs.
[25:11] 为什么？因为在工作中做 AI 功能，卡点通常不是模型
Why? Because when you build an AI feature at work, the blocker usually isn't the model.
[25:17] 而是拿到数据。过程中我们还会带你看一个常见陷阱
It's getting to the data. And along the way, we show you a very common trap.
[25:22] 在一次答疑中，agent 得意地报告画画游戏近乎满分
In one office hour, the agent proudly reported a near-perfect score in a drawing game
[25:28] 还附了截图作证
with a screenshot to prove it.
[25:30] 但截图跟真实游戏长得不一样，结果是它自己编的
But the screenshot didn't look like the real game. It had made up its own result.
[25:35] 我们会教你怎么识别这种情况，以及怎么设置才能避免再发生
So, we will show you how to spot that, and how to set things up so it doesn't happen again.
[25:41] 第二，你在 AI Builder Space 上搭建，这是学员平台
And second, you build on our AI Builder Space, our platforms for students.
[25:46] 一个 API key 就能用模型、语音识别、图像和搜索
One API key would give you models, speech recognition, images, search,
[25:50] app 做好后部署成公开链接，可以写进简历
and when your app works, you deploy to a public link so you can put on your resume.
[25:55] 接入工作由你的 coding agent 替你完成
Your coding agent wires up for you.
[25:58] 第三，毕业项目。你从工作或生活中挑一个问题
And third, your capstone. You pick a problem from your own work or life,
[26:03] 围绕它做一个应用，过程中我们会给反馈
and build an application around it, with feedbacks from us along the way.
[26:07] 我来讲讲其中一个
And let me tell you about this one.
[26:09] 我们有位学员 Ye，是经济学家
So, one of our students, Ye, is an economist.
[26:13] 她会写代码分析数据，但从没做过软件
She writes code to analyze data, but she has never built software.
[26:17] 她的祖母听力受损，听力测试显示
Her grandmother has hearing loss, and a hearing test showed that, oh,
[26:21] 她只能听到低频。陷阱就在这里
she can only hear low frequencies. And here's the trap.
[26:24] 跟听力不好的人说话时，你会自然地提高音量
When you talk to someone who's hard of hearing, you naturally raise your voice,
[26:29] 音调也会跟着升高，结果反而更糟
and your pitch will also go up, which actually makes it worse.
[26:33] 你能把音量降下来，但很难把音调降下来
You can lower your volume, but it's very hard for you to lower the pitch.
[26:38] 所以 Ye 想要一个 app：你对它说话，它用低沉的声音放出来
So, Ye wanted an app, like you talk to it and it plays your words back in a low voice.
[26:45] 她去问 AI，AI 上来就是大动作：语音模型、声音克隆
So she asked the AI, and it went for the big moves like speech models, voice cloning,
[26:50] 每家公司都推荐自家产品
every company recommending its own product.
[26:52] 几个小时过去，还是什么都没成
And hours later, still, nothing worked.
[26:55] 她来找我们，我的建议是：这事其实用不着 LLM
So, she comes to us, and my advice was, you actually don't need an LLM for this.
[27:00] 有个标准的变调技术正好能做这件事
There's a standard pitch shift technique doing exactly this,
[27:04] 大概几十行 Python 就够了
and probably in a few dozen lines of Python.
[27:07] 结果成功了
And so that worked.
[27:08] 她提交了 42 次 commit，部署到 AI Builder Space
She came back and with 42 commits and deployed it to AI Builder Space
[27:14] 家人现在都在用
and her family is using it.
[27:16] 她自己的体会是：真正有价值的是判断力
And her own takeaway was, what's valuable is the judgment.
[27:20] 而这种判断力，正是这门课要培养的
And that judgment is exactly what we spent on this course building.
[27:24] 下一期马上就要开始了
And the next cohort is about to start.
[27:27] 下一期下周一 10 月 12 日开课
Actually, the next cohort starts next Monday, October 12th,
[27:31] 开课时报名就截止
and enrollment closes when it opens.
[27:33] 第一周你会拿到资料和 Builder Space 的权限
That first week is when you get your materials and your Builder Space access
[27:38] 开始阅读，然后 10 月 17 日周六第一次直播
and started reading, then we'll meet live on Saturday, October 17th.
[27:43] 第二次直播课在 10 月 25 日，中间有一次可选的答疑
Then second live session is October 25th, with an optional office hour in between,
[27:47] 你可以把没跑通的部分带来
where you can bring the parts that didn't work.
[27:50] 如果没法参加直播，也有录像
If you can't make a session live, the recording is also there,
[27:54] 而直播课只是其中一部分
and the live sessions are only part of it.
[27:56] 你还会拿到 200 多页的教材，终身更新
You also get our textbook, more than 200 pages, with lifetime updates.
[28:01] 报名一次，以后每个新版本都能拿到
So you enroll once, and you get every future revision.
[28:04] 我们其实已经重写过好几次
We have actually already rewritten it several times,
[28:07] 现在正在进行又一次重写
and we're in the middle of another one right now.
[28:10] 此外，你还能在我们的社区 Stay Superlinear 待一年
And on top of that, you also get a year in our community, Stay Superlinear.
[28:15] 这门课已经打磨了 13 期，评分 5.0
We have refined this course over 13 cohorts, and it rated 5.0 from
[28:20] 来自 84 个评价。这里能看到几条评价
84 ratings. You can see a few of reviews here
[28:24] 来自 OpenAI、Anthropic 等公司的 builder
from builders at places like OpenAI and Anthropic,
[28:27] 他们反复提到的，是免费课里得不到的部分
and they keep pointing to the part is that you can't get from the free lesson.
[28:33] 在有反馈的情况下做出真东西
You build real things with feedback.
[28:36] 想加入的话，扫二维码或打开我们贴在 Zoom 聊天里的链接
And if you'd like to join, scan the QR code or open the link, we'll paste in Zoom chat.
[28:43] 为了感谢大家参加直播，优惠码 beyond-jev
As a thank you note for joining live, the code beyond-jev
[28:47] 可以减 200 美元
takes $200 off.
[28:48] 链接里已经带上了优惠码
The link already has the code applied,
[28:50] 扫二维码就能直接看到折后价
so you will see the discounted price right away if you scan the QR code,
[28:54] 后续邮件里也会有
and it will also be in the follow-up email.
[28:58] 感谢大家一直听到这里
So, thank you for staying with me.
[29:00] 关于 Jev、实验或课程，有什么问题都欢迎提
I'm happy to take any questions about Jev, about the experiments, and about the course.
[29:05] 另外，课后请留意一封邮件，里面有幻灯片
And, by the way, after the lesson, look out for an email. It will have the slide deck,
[29:09] 录像、开源代码、优惠券和报名链接
the recording, the open source code, your coupon, and the sign-up link.
