OpenAI Partners Deploy Agent Kits to Automate High-Stakes Sectors

OpenAI合作伙伴部署智能体套件，推动关键领域自动化 先进AI智能体在教育、医疗和软件开发领域的深度融合，正促使开发者的重心从传统工程转向自主、自我改进型生产系统的管理。 本期内容展示了OpenAI全新Agents SDK、Model Context Protocol以及使用成本大幅降低等前沿AI进展如何重塑核心产业。SchoolAI展示了具备安全护栏的AI导师如何在课堂上为教师提供实时的“教学成效GPS”；Jam.dev通过推出“Please Fix”功能降低软件修改门槛，开启意识流式浏览的“Web 4”时代；Abridge借助环境监听技术捕捉关键医患对话以缓解医生职业倦怠；Cursor则探讨了AI辅助编程从简单的Tab键自动补全到自主代码库重构及“氛围编程（vibe coding）”的演进。这些开发者共同印证了一个重大转变：技术重心已不再是串联简单工具，而是转向构建以人为本、具备自我改进能力的智能体系统。 SchoolAI 的 Caleb Hicks 指出，教育界对 AI 的态度经历了全面禁止、教师生产力辅助到视 AI 素养为学生必备技能的三阶段演进；随着模型智能跃升与成本大幅降低，AI 辅导正变得普惠且可规模化。 (2:35) Jam.dev 推出“Please Fix”功能，允许产品经理和设计师无需编写代码即可直接在浏览器中修改网页并自动生成符合设计系统的 PR，推动网络交互步入“阅读、编写与思考”的 Web 4 时代。 (13:47) 创业团队普遍经历从内部“吃狗粮”工具到核心产品的转型，AI 与无代码工具正在开启软件开发的“寒武纪大爆发”，未来软件将分化为人机协作精细维护的长期软件与 AI 按需生成的单次一次性软件。 (18:35) Abridge 通过环境声学 AI 技术将医患对话直接转录为病历，显著缓解了医生的“睡衣时间”文书负担，并将医疗 AI 幻觉重新定义为任何未获许可的信息，采用专属评估管线实现约 97% 的召回率。 (27:18)

At OpenAI DevDay, founders building in education, healthcare, and developer tooling argued that AI's primary value is shifting from generic chat interfaces to specialized agents that resolve severe administrative bottlenecks and democratize software creation. They emphasized that sustaining these systems in mission-critical environments requires strict domain-specific evaluations and architectural discipline rather than basic prompt completion. Alleviating Clerical Burnout in High-Stakes Professions (25:52) In professions facing acute workload constraints, practitioners frequently sacrifice direct human interaction for administrative upkeep. Abridge co-founder Zach Lipton highlighted that physicians historically spent roughly two hours on electronic health record documentation for every hour of direct patient care, driving after-hours work known as "pajama time." By translating ambient doctor-patient conversations into medical documentation, Abridge reported saving physicians an hour or more per day across 10 to 15 daily patient encounters, significantly easing retention and burnout concerns. Similarly, SchoolAI founder Caleb Hicks described the classroom challenge where a single teacher supervising up to 300 students must split attention between the top 10%, the struggling bottom 10%, and an overlooked 80% in the middle. SchoolAI deploys automated formative exit quizzes, social-emotional check-ins, and live comprehension dashboards to give teachers a "GPS for impact." This setup alerts educators to specific students requiring immediate help while abstracting prompt engineering behind tailored grade-level configurations. Democratizing Software Creation for Non-Engineers (13:47) Modern AI tooling is increasingly designed to bypass traditional engineering bottlenecks by enabling non-technical stakeholders to modify software directly. Danny Grant presented Jam.dev’s "Please Fix" browser extension, which allows product managers and designers to edit live website copy and layout elements visually, automatically converting those changes into pull requests that adhere to established design systems. Grant characterized this shift as "Web 4"—progressing from read-only and read-write models to an agentic "read, write, think" browsing environment where disposable, single-use tools can be spun up on demand. At Cursor, developer relations lead Lee Robinson observed a parallel evolution in code editors. To accommodate product managers, designers, and non-programmers entering codebases, Cursor introduced chat-centric agent interfaces that hide traditional, intimidating file trees. While professional developers typically onboard by using autocomplete before delegating complex refactors, non-engineers invert this path: they begin by describing software in natural language, relying on the agent to generate syntax and explain programming concepts interactively. Guardrails and Domain-Specific Evaluation Suites (31:36) Deploying foundation models at scale introduces operational risks when error rates compound across large user bases. Hicks noted that across five million students, a marginal 2% to 3% error or hallucination rate produces widespread classroom issues, requiring developers to adopt standardized Model Context Protocol (MCP) servers and continuous meta-prompt evaluations to guarantee safe interactions. In clinical documentation, Lipton emphasized that conventional natural language evaluations are insufficient because medical hallucinations include "unlicensed" information—content that may be medically correct in isolation but was never stated by the clinician during the visit. To counteract this, Abridge developed a specialized ontology of errors and parallel remediation models that evaluate every generated sentence against the clinical transcript context, operating at an estimated 97% recall rate. Autonomous Coding Loops and the Engineering Iceberg (44:08) AI-assisted development is transitioning from basic autocomplete toward autonomous agents that diagnose bugs, execute codebase-wide refactors, and plan architectures using files like `agent.md`. Cursor pairs large foundation models with custom, latency-sensitive autocomplete models that are updated every 30 minutes using online reinforcement learning derived from user acceptance and rejection signals. Despite viral interest in "vibe coding"—exploratory, rapid prototyping using natural language prompts—Robinson warned that intuitive front-end generation represents only the visible tip of an engineering iceberg. Robust production systems still require deep computer science fundamentals, observability, and data structure design. Both Robinson and Grant anticipated that while agents will increasingly automate mundane engineering chores like triaging customer bugs and submitting overnight pull requests, durable software still hinges on deliberate human architectural judgment. Speakers emphasized that despite rapid tooling advances within Silicon Valley, global adoption remains in its infancy, with university computer science programs lagging behind agentic development practices. The central ongoing challenge is moving models beyond narrow technical execution and retrieval into independent systems capable of handling abstract, real-world reasoning while preserving institutional trust.

在 OpenAI DevDay 现场，来自 SchoolAI、Jam.dev、Abridge 和 Cursor 的团队阐述了 AI 应用从基础补全走向深度专业垂直领域与自主 Agent 的实践历程。他们强调，真正的产品竞争力并非仅由底层基础模型决定，更依赖于严格的评估体系、垂直业务认知以及对生产级工作流的细致打磨。 垂直行业落地与严苛评估要求 (10:01) 在教育与医疗等关键领域，AI 落地依赖于将基础模型适配至极其严格的工作流中。SchoolAI 创始人 Caleb Hicks 指出，学校受限于紧缩的软件预算，必须依赖智能飞跃与推理成本剧降，同时其系统必须在后台丰富教师提示词以适应年级与学科，避免教师成为提示词工程师。在评估方面，当平台面向 500 万学生规模时，哪怕仅有 2% 到 3% 的错误或幻觉率，也会演变为每天海量的严重教学事故，使得端到端评估体系成为不可或缺的基础设施。 Abridge 联合创始人 Zach Lipton 同样强调了高风险专业场景中评估的特殊性。在医疗场景下，幻觉被重新定义为任何“未获授权”的信息：哪怕生成的医疗事实客观准确，但只要医生在当次问诊中并未实际提及，就会被标记为违规幻觉。为此，Abridge 建立了专用的错误本体分类与纠错模型管线，在上下文比对中实现了约 97% 的召回率。Lipton 强调，基础模型往往擅长在事后辨认已发生的“违规”，却难以完全自发避免犯错，开发者必须确立高度清晰的评估标准以保障临床安全。 消除非技术瓶颈与无代码赋能 (13:47) AI 工具的发展正在重塑软件迭代流程，打破工程团队与业务职能部门之间的协作壁垒。Jam.dev 推出的“Please Fix”浏览器插件允许产品经理、设计师与市场人员在前端页面直接修改文案或按钮交互，并自动生成符合现有设计系统的 GitHub Pull Request。Jam.dev 创始人 Danny Grant 表示，传统开发中最耗时的并非编写核心功能，而是排查复现 Bug 以及跨职能沟通；通过赋能非技术人员直接产出符合工程规范的代码变更，消除了以往上线一篇博客比发布一个模型还慢的效率瓶颈。 这种低代码和 Agent 工具正在降低软件构建门槛，催生所谓软件领域的“寒武纪大爆发”。Grant 援引消防员为同行开发专用工具、社区成员为教会打造软件的应用实例，说明没有技术背景的群体已能直接创造生产力工具。同时，未来软件生态可能分化为两类：一类是经过长期精细打磨的核心系统（如 Canva 或 Zillow），另一类则是由 Agent 依据临时需求即时生成、用后即弃的轻量化一次性软件。 从单点辅助迈向全流程自主 Agent (44:08) 与会者普遍认为，软件开发与专业工作流正在从被动式单点辅助演变为具有自主规划能力的 Agent 系统。Cursor 团队的 Lee Robinson 指出，AI 辅助编程已从早期的单行或单块代码补全，演进为能够在复杂代码库中自我纠错、主动检索信息的自主 Agent。为了支持复杂重构，Cursor 引入了“规划模式”（Planning Mode）与 `agent.md` 上下文机制，让 Agent 在实际编写代码前先遍历仓库梳理系统架构，以高质量的上下文输入保证最终生成代码的可靠性。 Abridge 同样在拓展 AI 在临床全流程中的角色，不再局限于问诊结束后的被动电子病历生成，而是延伸到问诊前的患者病历梳理、问诊中针对保险预授权条件的实时提示与决策支持。多位开发者一致看好 OpenAI 发布的 Agent 工具与标准化协议（如 MCP 服务器），这些框架将开发者从手动编排 Fast API 与底层脚手架中解放出来，使 Agent 能更自主地处理报错诊断与夜间 Bug 自动修复等枯燥任务。 深度自用研发与专业门槛并存 (21:04) 高频的内部“吃狗粮”（Dogfooding）成为工具团队打磨产品的核心方法论。Jam.dev 团队内部通过全面自用其浏览器插件，在短时间内独立上线了新版定价页面与深色模式，并观察到后台排练的四家初创公司中有三家都是从解决自身痛点的内部工具转型而来；OpenAI 自身也有 70% 的 Pull Request 由 Codex 生成。Cursor 全体产研团队亦全面使用 Cursor Agent 编写代码，通过在 Slack 内部发布主分支更新、统计同事的留存与流失率来检验功能是否达成内部产品市场契合度（Internal PMF）。 尽管“Vibe Coding”（氛围编码）通过降低门槛吸引了大量新手与非技术人员，但与会嘉宾提醒，这仅是软件工程的“冰山一角”。Lee Robinson 指出，虽然用户现在可以通过自然语言快速生成前端原型，但在构建高可靠、可观测的企业级系统时，底层计算机科学基础依然具有不可替代的决定性作用。目前硅谷与外部高校的实际教学存在明显脱节，传统教育体系尚未普及 AI 编程工具，新一代工程师需要兼备扎实的底层系统原理理解与熟练调动现代 AI 工具的双重能力。 本期播客展示了 AI 工具从实验室演示向教育、医疗与核心工程全面落地的演化图景。尽管模型推理能力提升与 Agent 框架加速了工作流自动化，但高敏环境对零差错评估的刚性需求，以及隐藏在简易原型背后的复杂工程底层，仍是开发者将生成式 AI 转化为可靠生产力必须应对的长期挑战。

[4:02] Caleb Hicks: teachers should never have to become prompt engineers [0:43] Caleb Hicks: always on expert hyper senior engineer [6:54] Caleb Hicks: these four students really need you today [10:01] Caleb Hicks: Two or three percent may not seem like a lot, but can make all the difference particularly with a student. [15:32] Danny Grant: Read, write, think [17:12] Danny Grant: Cambrian explosion of software [18:35] Danny Grant: disposable, one-time-use software [24:15] Danny Grant: If your startup works out to your wildest dreams, you're going to work on it for like 10 years [27:18] I actually got to have dinner with my family every night this week for the first time in like 10 years. [27:18] Abridge is saving my marriage. [30:52] Zach Lipton: Learning to Execute [31:36] Zach Lipton: unlicensed [37:48] Zach Lipton: they're not very funny yet [50:05] There's like all this file tree on the left is very overwhelming for people who are just getting started versus opening up something that looks closer to a chatbt. You're like, okay, I know this experience.

[0:11] Welcome to the OpenAI podcast where [0:11] Welcome to the OpenAI podcast where we're live from OpenAI Dev Day. Here [0:14] we're live from OpenAI Dev Day. Here [0:14] we're live from OpenAI Dev Day. Here sitting with me from school AI is Caleb [0:17] sitting with me from school AI is Caleb [0:18] sitting with me from school AI is Caleb Hicks. Caleb, hello. [0:19] Hicks. Caleb, hello. [0:19] Hicks. Caleb, hello. >> Hi. Thanks for having me. This will be [0:21] >> Hi. Thanks for having me. This will be [0:21] >> Hi. Thanks for having me. This will be fun. [0:21] fun. [0:21] fun. >> So Caleb, you're working on tools for [0:23] >> So Caleb, you're working on tools for [0:23] >> So Caleb, you're working on tools for helping educators and helping people [0:25] helping educators and helping people [0:25] helping educators and helping people basically in the classroom understand [0:27] basically in the classroom understand [0:27] basically in the classroom understand progress of students. [0:28] progress of students. [0:28] progress of students. >> That's right. Yeah. So, first off, what [0:31] >> That's right. Yeah. So, first off, what [0:31] >> That's right. Yeah. So, first off, what was your reaction so far to Dev Day? [0:33] was your reaction so far to Dev Day? [0:33] was your reaction so far to Dev Day? >> Uh, a ton of fun. I think makes it uh a [0:37] >> Uh, a ton of fun. I think makes it uh a [0:37] >> Uh, a ton of fun. I think makes it uh a lot of things to be excited about that [0:38] lot of things to be excited about that [0:38] lot of things to be excited about that help us build, but also help students [0:41] help us build, but also help students [0:41] help us build, but also help students and teachers be more creative as well. [0:42] and teachers be more creative as well. [0:42] and teachers be more creative as well. So, that'll be fun. [0:43] So, that'll be fun. [0:43] So, that'll be fun. >> So, what have you been working on over [0:45] >> So, what have you been working on over [0:45] >> So, what have you been working on over the last year? What has changed with AI [0:47] the last year? What has changed with AI [0:47] the last year? What has changed with AI that's accelerated what you've been [0:49] that's accelerated what you've been [0:49] that's accelerated what you've been doing? [0:49] doing? [0:49] doing? >> Oo. Um, I think probably the biggest [0:51] >> Oo. Um, I think probably the biggest [0:52] >> Oo. Um, I think probably the biggest advancement over the last year for us, [0:53] advancement over the last year for us, [0:53] advancement over the last year for us, so we we uh put AI in students hands. [0:56] so we we uh put AI in students hands. [0:56] so we we uh put AI in students hands. That's the main uh the main thing that [0:59] That's the main uh the main thing that [0:59] That's the main uh the main thing that we focus on is safe managed uh AI that [1:03] we focus on is safe managed uh AI that [1:03] we focus on is safe managed uh AI that can act as kind of one-time personal [1:05] can act as kind of one-time personal [1:05] can act as kind of one-time personal tutors for students. [1:06] tutors for students. [1:06] tutors for students. >> Mh. [1:06] >> Mh. [1:06] >> Mh. >> And so probably the biggest change from [1:10] >> And so probably the biggest change from [1:10] >> And so probably the biggest change from open AI has been model progression. I [1:12] open AI has been model progression. I [1:12] open AI has been model progression. I think we get two advantages from that. [1:15] think we get two advantages from that. [1:15] think we get two advantages from that. One is uh significant leaps in [1:17] One is uh significant leaps in [1:17] One is uh significant leaps in intelligence [1:19] intelligence [1:19] intelligence >> uh and the other one is uh you know [1:21] >> uh and the other one is uh you know [1:21] >> uh and the other one is uh you know improvements in cost because we are [1:23] improvements in cost because we are [1:23] improvements in cost because we are working with uh an industry that isn't [1:26] working with uh an industry that isn't [1:26] working with uh an industry that isn't known for paying big dollars for [1:28] known for paying big dollars for [1:28] known for paying big dollars for software. Uh it's been important for us [1:30] software. Uh it's been important for us [1:30] software. Uh it's been important for us to be able to manage students using this [1:33] to be able to manage students using this [1:33] to be able to manage students using this in a cost-ffective way. So those have [1:34] in a cost-ffective way. So those have [1:34] in a cost-ffective way. So those have been the two areas that AI progression [1:37] been the two areas that AI progression [1:37] been the two areas that AI progression has helped from our work. has been a lot [1:40] has helped from our work. has been a lot [1:40] has helped from our work. has been a lot of uh orchestration which I'm sure we'll [1:42] of uh orchestration which I'm sure we'll [1:42] of uh orchestration which I'm sure we'll talk about a little bit just just [1:44] talk about a little bit just just [1:44] talk about a little bit just just getting different AI agents and models [1:46] getting different AI agents and models [1:46] getting different AI agents and models to work together uh for the best outputs [1:50] to work together uh for the best outputs [1:50] to work together uh for the best outputs for students in particular. So a couple [1:53] for students in particular. So a couple [1:53] for students in particular. So a couple of the releases we saw today, one was [1:54] of the releases we saw today, one was [1:54] of the releases we saw today, one was the agents SDK, you know, and you've [1:57] the agents SDK, you know, and you've [1:57] the agents SDK, you know, and you've talked about that. [1:58] talked about that. [1:58] talked about that. >> How much have one tools changed the [2:01] >> How much have one tools changed the [2:01] >> How much have one tools changed the ability of one to work faster into the [2:03] ability of one to work faster into the [2:03] ability of one to work faster into the scope of what you find is capable now? [2:06] scope of what you find is capable now? [2:06] scope of what you find is capable now? >> Yeah, I think we're seeing teams across [2:09] >> Yeah, I think we're seeing teams across [2:09] >> Yeah, I think we're seeing teams across industries work way faster and building [2:12] industries work way faster and building [2:12] industries work way faster and building better software because they've got kind [2:14] better software because they've got kind [2:14] better software because they've got kind of this always on expert hyper senior uh [2:18] of this always on expert hyper senior uh [2:18] of this always on expert hyper senior uh engineer next to them that they're pair [2:20] engineer next to them that they're pair [2:20] engineer next to them that they're pair programming with, right? And uh so we [2:22] programming with, right? And uh so we [2:22] programming with, right? And uh so we see that with our teams as well and that [2:26] see that with our teams as well and that [2:26] see that with our teams as well and that just allows us to to build better [2:29] just allows us to to build better [2:29] just allows us to to build better software faster uh and get it in the [2:31] software faster uh and get it in the [2:31] software faster uh and get it in the hands of teachers and students which is [2:33] hands of teachers and students which is [2:33] hands of teachers and students which is what we're here to do. [2:35] what we're here to do. [2:35] what we're here to do. >> What has been the biggest shift you've [2:37] >> What has been the biggest shift you've [2:37] >> What has been the biggest shift you've seen in talking to educators or people [2:39] seen in talking to educators or people [2:39] seen in talking to educators or people like that in regards to AI in general? [2:42] like that in regards to AI in general? [2:42] like that in regards to AI in general? >> Yeah, great question. So every teacher, [2:44] >> Yeah, great question. So every teacher, [2:44] >> Yeah, great question. So every teacher, school, and district is on a very [2:46] school, and district is on a very [2:46] school, and district is on a very similar journey. It starts with [2:47] similar journey. It starts with [2:47] similar journey. It starts with permission, right? Two and a half years [2:49] permission, right? Two and a half years [2:49] permission, right? Two and a half years ago, it was everyone under the sun was [2:51] ago, it was everyone under the sun was [2:51] ago, it was everyone under the sun was just banning AI altogether. Uh we've [2:54] just banning AI altogether. Uh we've [2:54] just banning AI altogether. Uh we've we've moved past that into productivity [2:57] we've moved past that into productivity [2:57] we've moved past that into productivity as teachers and and school leaders [2:59] as teachers and and school leaders [2:59] as teachers and and school leaders realizing, hey, this helps me in my job. [3:02] realizing, hey, this helps me in my job. [3:02] realizing, hey, this helps me in my job. >> Uh the leading schools are starting to [3:04] >> Uh the leading schools are starting to [3:04] >> Uh the leading schools are starting to get into that that really important [3:06] get into that that really important [3:06] get into that that really important spot, which is recognizing that every [3:08] spot, which is recognizing that every [3:08] spot, which is recognizing that every student has to know how to use this [3:10] student has to know how to use this [3:10] student has to know how to use this stuff. M [3:11] stuff. M [3:11] stuff. M >> if you're going into if you're [3:13] >> if you're going into if you're [3:13] >> if you're going into if you're graduating from high school and you're [3:14] graduating from high school and you're [3:14] graduating from high school and you're competing for colleges or jobs and you [3:17] competing for colleges or jobs and you [3:17] competing for colleges or jobs and you are don't know how to use AI yourself um [3:20] are don't know how to use AI yourself um [3:20] are don't know how to use AI yourself um you're at a severe disadvantage. So um [3:25] you're at a severe disadvantage. So um [3:25] you're at a severe disadvantage. So um most people are now orienting to like [3:27] most people are now orienting to like [3:27] most people are now orienting to like yeah we have to teach this. Uh we think [3:29] yeah we have to teach this. Uh we think [3:29] yeah we have to teach this. Uh we think there's a couple of special steps beyond [3:31] there's a couple of special steps beyond [3:31] there's a couple of special steps beyond those two uh where where you get to [3:33] those two uh where where you get to [3:33] those two uh where where you get to better support students. The AI tutor in [3:36] better support students. The AI tutor in [3:36] better support students. The AI tutor in every kid's pocket, right? But we think [3:38] every kid's pocket, right? But we think [3:38] every kid's pocket, right? But we think that is it's got to be classroom [3:40] that is it's got to be classroom [3:40] that is it's got to be classroom connected. It's got to know what you're [3:42] connected. It's got to know what you're [3:42] connected. It's got to know what you're doing in class and it's got to know um [3:45] doing in class and it's got to know um [3:45] doing in class and it's got to know um you know where you're trying to go and [3:48] you know where you're trying to go and [3:48] you know where you're trying to go and help you move that direction. Um, that [3:50] help you move that direction. Um, that [3:50] help you move that direction. Um, that last step we get really excited about is [3:52] last step we get really excited about is [3:52] last step we get really excited about is how you can put AI to work with [3:54] how you can put AI to work with [3:54] how you can put AI to work with teachers, uh, families at home, school [3:57] teachers, uh, families at home, school [3:57] teachers, uh, families at home, school leaders, kind of the system at large, [3:59] leaders, kind of the system at large, [3:59] leaders, kind of the system at large, uh, to really make school awesome for [4:01] uh, to really make school awesome for [4:01] uh, to really make school awesome for students. [4:02] students. [4:02] students. >> Could you tell me a little bit about the [4:03] >> Could you tell me a little bit about the [4:03] >> Could you tell me a little bit about the kind of stack you're using from both the [4:05] kind of stack you're using from both the [4:06] kind of stack you're using from both the teacher facing, student facing, backend [4:08] teacher facing, student facing, backend [4:08] teacher facing, student facing, backend or what you're working on in that [4:09] or what you're working on in that [4:09] or what you're working on in that regard? Yeah, from from the product [4:13] regard? Yeah, from from the product [4:13] regard? Yeah, from from the product side, we have kind of three different uh [4:16] side, we have kind of three different uh [4:16] side, we have kind of three different uh parts of the product that the that [4:19] parts of the product that the that [4:19] parts of the product that the that students, teachers, and school leaders [4:21] students, teachers, and school leaders [4:21] students, teachers, and school leaders use. So, there's uh kind of just a a [4:23] use. So, there's uh kind of just a a [4:24] use. So, there's uh kind of just a a pretty basic uh AI assistant, right? Um [4:27] pretty basic uh AI assistant, right? Um [4:27] pretty basic uh AI assistant, right? Um the the the GPT rapper as they say, [4:30] the the the GPT rapper as they say, [4:30] the the the GPT rapper as they say, right? Uh but but tuned to use cases in [4:33] right? Uh but but tuned to use cases in [4:33] right? Uh but but tuned to use cases in schools. I think a big thing that we [4:35] schools. I think a big thing that we [4:35] schools. I think a big thing that we felt like was really important is [4:37] felt like was really important is [4:37] felt like was really important is teachers should never have to become [4:39] teachers should never have to become [4:39] teachers should never have to become prompt engineers, [4:40] prompt engineers, [4:40] prompt engineers, >> right? [4:40] >> right? [4:40] >> right? >> Uh so [4:41] >> Uh so [4:41] >> Uh so >> you know I was a prompt engineer. [4:42] >> you know I was a prompt engineer. [4:42] >> you know I was a prompt engineer. >> Yes. Yeah. So so we do a lot of that uh [4:46] >> Yes. Yeah. So so we do a lot of that uh [4:46] >> Yes. Yeah. So so we do a lot of that uh kind of extra orchestration [4:48] kind of extra orchestration [4:48] kind of extra orchestration uh to we essentially enrich every prompt [4:51] uh to we essentially enrich every prompt [4:51] uh to we essentially enrich every prompt that that a teacher writes to get better [4:52] that that a teacher writes to get better [4:52] that that a teacher writes to get better output for them for their what they're [4:55] output for them for their what they're [4:55] output for them for their what they're teaching uh their grade level all that [4:57] teaching uh their grade level all that [4:57] teaching uh their grade level all that stuff. Um so there's there's that uh we [5:00] stuff. Um so there's there's that uh we [5:00] stuff. Um so there's there's that uh we call that dot. It's fun little blue [5:02] call that dot. It's fun little blue [5:02] call that dot. It's fun little blue animated character. Uh, and then we have [5:05] animated character. Uh, and then we have [5:05] animated character. Uh, and then we have tools. It's a form. You fill it out and [5:07] tools. It's a form. You fill it out and [5:07] tools. It's a form. You fill it out and it gives you an output, a lesson plan, [5:09] it gives you an output, a lesson plan, [5:09] it gives you an output, a lesson plan, adapted reading content, uh, things like [5:11] adapted reading content, uh, things like [5:11] adapted reading content, uh, things like that. Those we we like to call those the [5:14] that. Those we we like to call those the [5:14] that. Those we we like to call those the 101, like the the checkers level [5:15] 101, like the the checkers level [5:16] 101, like the the checkers level features that kind of table stakes that [5:18] features that kind of table stakes that [5:18] features that kind of table stakes that you've got to give to to teachers for [5:20] you've got to give to to teachers for [5:20] you've got to give to to teachers for them to move from can I even allow this [5:22] them to move from can I even allow this [5:22] them to move from can I even allow this to this is useful for me. Uh but the [5:25] to this is useful for me. Uh but the [5:25] to this is useful for me. Uh but the special part is when you start uh doing [5:28] special part is when you start uh doing [5:28] special part is when you start uh doing kind of these one-time guard railed safe [5:31] kind of these one-time guard railed safe [5:31] kind of these one-time guard railed safe managed AI tutors that the teacher can [5:34] managed AI tutors that the teacher can [5:34] managed AI tutors that the teacher can create [5:35] create [5:35] create >> give to their students students are [5:37] >> give to their students students are [5:37] >> give to their students students are interacting and then the teacher gets a [5:38] interacting and then the teacher gets a [5:38] interacting and then the teacher gets a real time dashboard of how the students [5:42] real time dashboard of how the students [5:42] real time dashboard of how the students are doing what they're doing with the AI [5:44] are doing what they're doing with the AI [5:44] are doing what they're doing with the AI and uh to make that very concrete uh the [5:48] and uh to make that very concrete uh the [5:48] and uh to make that very concrete uh the last 5 10 minutes of class a teacher may [5:51] last 5 10 minutes of class a teacher may [5:51] last 5 10 minutes of class a teacher may give what's called an exit ticket to [5:53] give what's called an exit ticket to [5:53] give what's called an exit ticket to their students Um, so they've got a [5:56] their students Um, so they've got a [5:56] their students Um, so they've got a recording of everything that they did [5:57] recording of everything that they did [5:57] recording of everything that they did during class that day and it loads up [5:59] during class that day and it loads up [5:59] during class that day and it loads up and says like, "Hey, how did the content [6:01] and says like, "Hey, how did the content [6:01] and says like, "Hey, how did the content go today?" And it does almost a what's [6:03] go today?" And it does almost a what's [6:03] go today?" And it does almost a what's called a a formative quiz. It it asks [6:07] called a a formative quiz. It it asks [6:07] called a a formative quiz. It it asks them questions and then it's coaching [6:08] them questions and then it's coaching [6:08] them questions and then it's coaching them on the an on on what they learned, [6:12] them on the an on on what they learned, [6:12] them on the an on on what they learned, what they want to learn, where they [6:13] what they want to learn, where they [6:13] what they want to learn, where they might go next, tease them up for [6:15] might go next, tease them up for [6:15] might go next, tease them up for whatever homework they might have. Um, [6:18] whatever homework they might have. Um, [6:18] whatever homework they might have. Um, and then it will do just kind of an emot [6:20] and then it will do just kind of an emot [6:20] and then it will do just kind of an emot kind of a social emotional check-in. and [6:21] kind of a social emotional check-in. and [6:21] kind of a social emotional check-in. and it will say like, hey, how like how was [6:23] it will say like, hey, how like how was [6:23] it will say like, hey, how like how was class? Um what feedback do you have for [6:26] class? Um what feedback do you have for [6:26] class? Um what feedback do you have for the teacher? Uh what are what are you [6:28] the teacher? Uh what are what are you [6:28] the teacher? Uh what are what are you looking for out of using this content? [6:30] looking for out of using this content? [6:30] looking for out of using this content? And rolling all of that up to the [6:33] And rolling all of that up to the [6:33] And rolling all of that up to the teacher so that they know how to better [6:34] teacher so that they know how to better [6:34] teacher so that they know how to better support those students in the future. Um [6:37] support those students in the future. Um [6:37] support those students in the future. Um a thing a lot of us don't recognize [6:39] a thing a lot of us don't recognize [6:39] a thing a lot of us don't recognize about teachers is they might be working [6:42] about teachers is they might be working [6:42] about teachers is they might be working with 300 students at a time. [6:44] with 300 students at a time. [6:44] with 300 students at a time. >> I had 42 desks in my classroom. [6:47] >> I had 42 desks in my classroom. [6:47] >> I had 42 desks in my classroom. >> So you started off as a teacher before [6:48] >> So you started off as a teacher before [6:48] >> So you started off as a teacher before all this. That informed a lot what [6:49] all this. That informed a lot what [6:49] all this. That informed a lot what you're doing. [6:50] you're doing. [6:50] you're doing. >> Yeah. So, I had 42 desks in my classroom [6:52] >> Yeah. So, I had 42 desks in my classroom [6:52] >> Yeah. So, I had 42 desks in my classroom and I was teaching seven or eight [6:53] and I was teaching seven or eight [6:53] and I was teaching seven or eight periods a day. [6:54] periods a day. [6:54] periods a day. >> Every day, teachers have to make this [6:56] >> Every day, teachers have to make this [6:56] >> Every day, teachers have to make this impossible choice. Do I work with uh the [6:59] impossible choice. Do I work with uh the [7:00] impossible choice. Do I work with uh the top 10% of students who love this? They [7:02] top 10% of students who love this? They [7:02] top 10% of students who love this? They get it. They want more of it. They'd [7:03] get it. They want more of it. They'd [7:03] get it. They want more of it. They'd stay after school if I let them. The 10% [7:06] stay after school if I let them. The 10% [7:06] stay after school if I let them. The 10% of students that are really struggling, [7:07] of students that are really struggling, [7:07] of students that are really struggling, maybe that's because they don't [7:09] maybe that's because they don't [7:09] maybe that's because they don't understand it. Maybe it's a learning [7:10] understand it. Maybe it's a learning [7:10] understand it. Maybe it's a learning disability. Maybe it's a problem at [7:12] disability. Maybe it's a problem at [7:12] disability. Maybe it's a problem at home. Maybe they got bullied in the last [7:14] home. Maybe they got bullied in the last [7:14] home. Maybe they got bullied in the last class. [7:16] class. [7:16] class. Or everyone else. [7:18] Or everyone else. [7:18] Or everyone else. And I care a lot about that. everyone [7:20] And I care a lot about that. everyone [7:20] And I care a lot about that. everyone else that middle 80% [7:22] else that middle 80% [7:22] else that middle 80% >> because I was one of those students and [7:24] >> because I was one of those students and [7:24] >> because I was one of those students and most of us were definitionally and [7:28] most of us were definitionally and [7:28] most of us were definitionally and we think about what we're building uh [7:29] we think about what we're building uh [7:30] we think about what we're building uh and what we've been able to build with [7:31] and what we've been able to build with [7:31] and what we've been able to build with with OpenAI and some of the tools that [7:33] with OpenAI and some of the tools that [7:33] with OpenAI and some of the tools that were announced today will be able to do [7:34] were announced today will be able to do [7:34] were announced today will be able to do even more of is uh give teachers almost [7:37] even more of is uh give teachers almost [7:37] even more of is uh give teachers almost a GPS for impact [7:39] a GPS for impact [7:39] a GPS for impact >> like these four students really need you [7:42] >> like these four students really need you [7:42] >> like these four students really need you today [7:43] today [7:44] today and you can jump in and support those [7:46] and you can jump in and support those [7:46] and you can jump in and support those four students in a way that you maybe [7:47] four students in a way that you maybe [7:47] four students in a way that you maybe wouldn't have even known that they had a [7:50] wouldn't have even known that they had a [7:50] wouldn't have even known that they had a concern [7:50] concern [7:50] concern >> that it brings up a very I think [7:52] >> that it brings up a very I think [7:52] >> that it brings up a very I think interesting point when I talk to [7:53] interesting point when I talk to [7:53] interesting point when I talk to developers often people confuse the [7:55] developers often people confuse the [7:55] developers often people confuse the tools for the product and what you have [7:57] tools for the product and what you have [7:58] tools for the product and what you have to do is you have to both understand the [7:59] to do is you have to both understand the [7:59] to do is you have to both understand the needs of the educator that comes from [8:01] needs of the educator that comes from [8:01] needs of the educator that comes from you working in the classroom and your [8:02] you working in the classroom and your [8:02] you working in the classroom and your peers working at school and [8:03] peers working at school and [8:03] peers working at school and understanding that and I think that's [8:05] understanding that and I think that's [8:05] understanding that and I think that's the thing you're able to bring to it [8:06] the thing you're able to bring to it [8:06] the thing you're able to bring to it where you look at this is a platform to [8:08] where you look at this is a platform to [8:08] where you look at this is a platform to build on top of it and that's something [8:10] build on top of it and that's something [8:10] build on top of it and that's something where you've identified all these areas [8:12] where you've identified all these areas [8:12] where you've identified all these areas that you can bring into to specialize it [8:14] that you can bring into to specialize it [8:14] that you can bring into to specialize it and make it very custom what you're [8:16] and make it very custom what you're [8:16] and make it very custom what you're doing. Yeah, I think that's something [8:18] doing. Yeah, I think that's something [8:18] doing. Yeah, I think that's something exciting we saw today in uh in the [8:21] exciting we saw today in uh in the [8:21] exciting we saw today in uh in the announcements was uh opportunities for [8:25] announcements was uh opportunities for [8:25] announcements was uh opportunities for uh people like me with subject matter [8:27] uh people like me with subject matter [8:27] uh people like me with subject matter expertise with that that really know a [8:30] expertise with that that really know a [8:30] expertise with that that really know a domain and we're able to build kind of [8:32] domain and we're able to build kind of [8:32] domain and we're able to build kind of with [8:34] with [8:34] with >> you all uh with OpenAI not just on top [8:37] >> you all uh with OpenAI not just on top [8:37] >> you all uh with OpenAI not just on top of OpenAI which is which is uh a really [8:39] of OpenAI which is which is uh a really [8:39] of OpenAI which is which is uh a really cool unlock for a lot of people. [8:41] cool unlock for a lot of people. [8:41] cool unlock for a lot of people. >> What are you most excited about that you [8:42] >> What are you most excited about that you [8:42] >> What are you most excited about that you saw today? [8:43] saw today? [8:43] saw today? >> The agent builder for sure. [8:44] >> The agent builder for sure. [8:44] >> The agent builder for sure. >> Agent builder. Yeah. Yeah. [8:45] >> Agent builder. Yeah. Yeah. [8:45] >> Agent builder. Yeah. Yeah. >> Yeah. That looks like uh it could be a [8:48] >> Yeah. That looks like uh it could be a [8:48] >> Yeah. That looks like uh it could be a lot of fun. And going back to when you [8:51] lot of fun. And going back to when you [8:51] lot of fun. And going back to when you know you you know how it was when we [8:52] know you you know how it was when we [8:52] know you you know how it was when we first started you had to just wire up a [8:54] first started you had to just wire up a [8:54] first started you had to just wire up a lot of that yourself and code tools [8:56] lot of that yourself and code tools [8:56] lot of that yourself and code tools certainly make that easier but just [8:58] certainly make that easier but just [8:58] certainly make that easier but just being able to drag and drop think [8:59] being able to drag and drop think [8:59] being able to drag and drop think something like file search the [9:01] something like file search the [9:01] something like file search the permission structure seemed really well [9:02] permission structure seemed really well [9:02] permission structure seemed really well thought out and particularly what you're [9:04] thought out and particularly what you're [9:04] thought out and particularly what you're doing in a classroom situation where you [9:05] doing in a classroom situation where you [9:05] doing in a classroom situation where you really have to have those safeguards in [9:07] really have to have those safeguards in [9:07] really have to have those safeguards in there. [9:07] there. [9:07] there. >> Yeah. uh we actually built our own and [9:10] >> Yeah. uh we actually built our own and [9:10] >> Yeah. uh we actually built our own and have for a number of years and I think [9:11] have for a number of years and I think [9:11] have for a number of years and I think we're excited to go and get our hands on [9:13] we're excited to go and get our hands on [9:13] we're excited to go and get our hands on on it and see what do we get to [9:15] on it and see what do we get to [9:15] on it and see what do we get to >> get rid of. Yeah. What do we get to uh [9:18] >> get rid of. Yeah. What do we get to uh [9:18] >> get rid of. Yeah. What do we get to uh make even better now that we have access [9:20] make even better now that we have access [9:20] make even better now that we have access to it ourselves? [9:21] to it ourselves? [9:21] to it ourselves? >> Um or or from you all. I think that's uh [9:27] >> Um or or from you all. I think that's uh [9:27] >> Um or or from you all. I think that's uh >> either way we're going to learn a lot [9:28] >> either way we're going to learn a lot [9:28] >> either way we're going to learn a lot from working with it and and that makes [9:30] from working with it and and that makes [9:30] from working with it and and that makes it easier for us to use but also again [9:33] it easier for us to use but also again [9:33] it easier for us to use but also again bring it into the hands of these people [9:34] bring it into the hands of these people [9:34] bring it into the hands of these people that aren't technical. They're not [9:36] that aren't technical. They're not [9:36] that aren't technical. They're not developers. They're not they're not [9:38] developers. They're not they're not [9:38] developers. They're not they're not thinking about, you know, which model [9:40] thinking about, you know, which model [9:40] thinking about, you know, which model they're using, but they just want to get [9:41] they're using, but they just want to get [9:41] they're using, but they just want to get their stuff done. [9:42] their stuff done. [9:42] their stuff done. >> Yeah, I think that's a great point in [9:43] >> Yeah, I think that's a great point in [9:44] >> Yeah, I think that's a great point in that the more you can have the people [9:46] that the more you can have the people [9:46] that the more you can have the people who understand who your customer is, in [9:48] who understand who your customer is, in [9:48] who understand who your customer is, in this case, students and teachers, [9:50] this case, students and teachers, [9:50] this case, students and teachers, understand what that problem is, and [9:52] understand what that problem is, and [9:52] understand what that problem is, and less time they're spending trying to [9:54] less time they're spending trying to [9:54] less time they're spending trying to wire things up and run, you know, fast [9:56] wire things up and run, you know, fast [9:56] wire things up and run, you know, fast API servers and stuff to do this. Seems [9:58] API servers and stuff to do this. Seems [9:58] API servers and stuff to do this. Seems like it can make the product get a lot [9:59] like it can make the product get a lot [10:00] like it can make the product get a lot better faster. [10:00] better faster. [10:00] better faster. >> Absolutely. [10:01] >> Absolutely. [10:02] >> Absolutely. >> What else have you been excited about [10:03] >> What else have you been excited about [10:03] >> What else have you been excited about you've seen? [10:05] you've seen? [10:05] you've seen? Um I think uh we've we've built some [10:09] Um I think uh we've we've built some [10:09] Um I think uh we've we've built some similar things and in our product we [10:11] similar things and in our product we [10:11] similar things and in our product we have this concept called powerups which [10:12] have this concept called powerups which [10:12] have this concept called powerups which are basically apps that you can use with [10:14] are basically apps that you can use with [10:14] are basically apps that you can use with dot our our character. I think we got to [10:17] dot our our character. I think we got to [10:17] dot our our character. I think we got to see some good patterns and examples of [10:18] see some good patterns and examples of [10:18] see some good patterns and examples of that today with the new apps that were [10:20] that today with the new apps that were [10:20] that today with the new apps that were that were announced. And I think one of [10:23] that were announced. And I think one of [10:23] that were announced. And I think one of the things we're really excited about is [10:24] the things we're really excited about is [10:24] the things we're really excited about is these partners that we've been [10:26] these partners that we've been [10:26] these partners that we've been developing uh that everyone is really [10:29] developing uh that everyone is really [10:29] developing uh that everyone is really solidified around MCP servers as a way [10:32] solidified around MCP servers as a way [10:32] solidified around MCP servers as a way to communicate with uh AI. And so um a [10:36] to communicate with uh AI. And so um a [10:36] to communicate with uh AI. And so um a thing that will definitely benefit us is [10:38] thing that will definitely benefit us is [10:38] thing that will definitely benefit us is OpenAI kind of drawing a line in the [10:40] OpenAI kind of drawing a line in the [10:40] OpenAI kind of drawing a line in the sand saying we're doubling down on this. [10:42] sand saying we're doubling down on this. [10:42] sand saying we're doubling down on this. Yeah. Uh and so now when we go to a [10:44] Yeah. Uh and so now when we go to a [10:44] Yeah. Uh and so now when we go to a partner that's already building for [10:46] partner that's already building for [10:46] partner that's already building for integration with Chad GBT, they can also [10:50] integration with Chad GBT, they can also [10:50] integration with Chad GBT, they can also bring that directly into school AI again [10:52] bring that directly into school AI again [10:52] bring that directly into school AI again for that safe managed guardrail [10:55] for that safe managed guardrail [10:55] for that safe managed guardrail experience [10:56] experience [10:56] experience >> um that uh that teachers and school [10:58] >> um that uh that teachers and school [10:58] >> um that uh that teachers and school leaders are looking for. [10:59] leaders are looking for. [10:59] leaders are looking for. >> Yeah. Yeah, one of the things I think [11:00] >> Yeah. Yeah, one of the things I think [11:00] >> Yeah. Yeah, one of the things I think it's going to be helpful to and you [11:01] it's going to be helpful to and you [11:01] it's going to be helpful to and you mentioned this before too is eval and [11:03] mentioned this before too is eval and [11:03] mentioned this before too is eval and the ability that as you run these [11:05] the ability that as you run these [11:05] the ability that as you run these systems and you run the models to be [11:06] systems and you run the models to be [11:06] systems and you run the models to be able to know how well they're performing [11:08] able to know how well they're performing [11:08] able to know how well they're performing because two or 3% may not seem like a [11:10] because two or 3% may not seem like a [11:10] because two or 3% may not seem like a lot but can make all the difference [11:11] lot but can make all the difference [11:11] lot but can make all the difference particularly with a student when you [11:14] particularly with a student when you [11:14] particularly with a student when you have five million students using uh your [11:16] have five million students using uh your [11:16] have five million students using uh your platform. Two to three% means a whole [11:20] platform. Two to three% means a whole [11:20] platform. Two to three% means a whole ton of issues every day. [11:21] ton of issues every day. [11:21] ton of issues every day. >> And it's one of those things it's often [11:23] >> And it's one of those things it's often [11:23] >> And it's one of those things it's often every company I talk to knows it's [11:25] every company I talk to knows it's [11:25] every company I talk to knows it's important. they want to do that, but to [11:27] important. they want to do that, but to [11:27] important. they want to do that, but to have the the opportunity and the time to [11:29] have the the opportunity and the time to [11:29] have the the opportunity and the time to spend building an eval suite to build [11:31] spend building an eval suite to build [11:31] spend building an eval suite to build something in there is just often kind of [11:33] something in there is just often kind of [11:33] something in there is just often kind of a thing they'll do in the future, but [11:35] a thing they'll do in the future, but [11:35] a thing they'll do in the future, but having that built into a system seems [11:36] having that built into a system seems [11:36] having that built into a system seems like a big [11:37] like a big [11:37] like a big >> we kind of saw it today, right? I don't [11:38] >> we kind of saw it today, right? I don't [11:38] >> we kind of saw it today, right? I don't have time to to do the eval. [11:41] have time to to do the eval. [11:41] have time to to do the eval. >> Exactly. Well, yeah. So, they did an [11:43] >> Exactly. Well, yeah. So, they did an [11:43] >> Exactly. Well, yeah. So, they did an eight minutes building an agent. Very [11:45] eight minutes building an agent. Very [11:45] eight minutes building an agent. Very impressive. Yeah. Take 10 and do the [11:47] impressive. Yeah. Take 10 and do the [11:47] impressive. Yeah. Take 10 and do the evals and that'll be fun. [11:48] evals and that'll be fun. [11:48] evals and that'll be fun. >> Yeah. That's it's kind of exciting too [11:50] >> Yeah. That's it's kind of exciting too [11:50] >> Yeah. That's it's kind of exciting too because for prototyping and spinning [11:53] because for prototyping and spinning [11:53] because for prototyping and spinning things out because I think a lot of what [11:54] things out because I think a lot of what [11:54] things out because I think a lot of what you're doing is probably going to be [11:55] you're doing is probably going to be [11:55] you're doing is probably going to be experimental and trying things with [11:56] experimental and trying things with [11:56] experimental and trying things with teachers to be able to sort of see if [11:57] teachers to be able to sort of see if [11:57] teachers to be able to sort of see if these things work [11:59] these things work [11:59] these things work >> and do you see that accelerating? [12:01] >> and do you see that accelerating? [12:01] >> and do you see that accelerating? >> Yeah. So we um one one of the things we [12:04] >> Yeah. So we um one one of the things we [12:04] >> Yeah. So we um one one of the things we do teachers when they're creating these [12:06] do teachers when they're creating these [12:06] do teachers when they're creating these like custom AI tutors they they do them [12:08] like custom AI tutors they they do them [12:08] like custom AI tutors they they do them lesson by lesson. It's like, hey, I'm [12:09] lesson by lesson. It's like, hey, I'm [12:09] lesson by lesson. It's like, hey, I'm teaching about the water cycle and I'm [12:11] teaching about the water cycle and I'm [12:11] teaching about the water cycle and I'm going to create this activity that's a [12:13] going to create this activity that's a [12:14] going to create this activity that's a uh starts out as a tutor and then turns [12:16] uh starts out as a tutor and then turns [12:16] uh starts out as a tutor and then turns into a game and then turns into a quiz. [12:18] into a game and then turns into a quiz. [12:18] into a game and then turns into a quiz. Uh just being able to preview that [12:20] Uh just being able to preview that [12:20] Uh just being able to preview that faster, but but almost having adaptive [12:22] faster, but but almost having adaptive [12:22] faster, but but almost having adaptive evals in the moment is one of the things [12:24] evals in the moment is one of the things [12:24] evals in the moment is one of the things that we've gotten into is [12:26] that we've gotten into is [12:26] that we've gotten into is >> uh really the meta prompting. How do we [12:29] >> uh really the meta prompting. How do we [12:29] >> uh really the meta prompting. How do we how do we not just tell the AI what to [12:30] how do we not just tell the AI what to [12:30] how do we not just tell the AI what to do, but tell it how to do the thing that [12:32] do, but tell it how to do the thing that [12:32] do, but tell it how to do the thing that it decides to do. [12:33] it decides to do. [12:33] it decides to do. >> Right. [12:33] >> Right. [12:34] >> Right. >> Right. Yeah. That's a it's a very [12:35] >> Right. Yeah. That's a it's a very [12:35] >> Right. Yeah. That's a it's a very interesting space to be in where the AI [12:37] interesting space to be in where the AI [12:37] interesting space to be in where the AI can help you build the prompt to tell it [12:38] can help you build the prompt to tell it [12:38] can help you build the prompt to tell it what to do, but knowing where you're [12:40] what to do, but knowing where you're [12:40] what to do, but knowing where you're going to start from has been super [12:43] going to start from has been super [12:43] going to start from has been super helpful. Um, where can we look forward [12:46] helpful. Um, where can we look forward [12:46] helpful. Um, where can we look forward to finding out more about School AI, [12:47] to finding out more about School AI, [12:47] to finding out more about School AI, what you're working on? [12:48] what you're working on? [12:48] what you're working on? >> Yeah. Uh, you can uh follow us on on X [12:52] >> Yeah. Uh, you can uh follow us on on X [12:52] >> Yeah. Uh, you can uh follow us on on X uh get school AAI. You can check out [12:54] uh get school AAI. You can check out [12:54] uh get school AAI. You can check out schoolai.com. Um, I think this is this [12:57] schoolai.com. Um, I think this is this [12:58] schoolai.com. Um, I think this is this is a fun time of year with back to [12:59] is a fun time of year with back to [12:59] is a fun time of year with back to school and seeing a ton of how how [13:01] school and seeing a ton of how how [13:01] school and seeing a ton of how how teachers and students are building [13:03] teachers and students are building [13:03] teachers and students are building >> um different uh different holiday stuff [13:06] >> um different uh different holiday stuff [13:06] >> um different uh different holiday stuff that gets really fun when when teachers [13:08] that gets really fun when when teachers [13:08] that gets really fun when when teachers are sharing with each other. So yeah, I [13:10] are sharing with each other. So yeah, I [13:10] are sharing with each other. So yeah, I would say uh on uh on X uh or Instagram [13:15] would say uh on uh on X uh or Instagram [13:15] would say uh on uh on X uh or Instagram or schooli.com. [13:16] or schooli.com. [13:16] or schooli.com. >> Last question. Any advice you give to [13:18] >> Last question. Any advice you give to [13:18] >> Last question. Any advice you give to developers about where to start with the [13:20] developers about where to start with the [13:20] developers about where to start with the tools what you've played with so far? [13:21] tools what you've played with so far? [13:21] tools what you've played with so far? Oh, I think um you can probably just [13:24] Oh, I think um you can probably just [13:24] Oh, I think um you can probably just start in the in like the GBT builder for [13:26] start in the in like the GBT builder for [13:26] start in the in like the GBT builder for most of the ideas that I hear is uh just [13:29] most of the ideas that I hear is uh just [13:29] most of the ideas that I hear is uh just start with the GBT builder and then [13:31] start with the GBT builder and then [13:31] start with the GBT builder and then expand from there. Um you've got uh you [13:34] expand from there. Um you've got uh you [13:34] expand from there. Um you've got uh you know in the developer portal you've got [13:36] know in the developer portal you've got [13:36] know in the developer portal you've got a ton of other tools to just start [13:37] a ton of other tools to just start [13:37] a ton of other tools to just start playing. Um and then I think the the [13:40] playing. Um and then I think the the [13:40] playing. Um and then I think the the agent builder that we saw today is going [13:42] agent builder that we saw today is going [13:42] agent builder that we saw today is going to be another fun one to start [13:43] to be another fun one to start [13:43] to be another fun one to start connecting the dots between all the [13:45] connecting the dots between all the [13:45] connecting the dots between all the different tools. [13:45] different tools. [13:46] different tools. >> Awesome. Thank you so much. [13:47] >> Awesome. Thank you so much. [13:47] >> Awesome. Thank you so much. >> Yeah, thank you Caleb. Hope you enjoy [13:49] >> Yeah, thank you Caleb. Hope you enjoy [13:49] >> Yeah, thank you Caleb. Hope you enjoy the rest of Dev Day. So, we're going to [13:51] the rest of Dev Day. So, we're going to [13:51] the rest of Dev Day. So, we're going to be talking to a few other developers [13:52] be talking to a few other developers [13:52] be talking to a few other developers about some of their different [13:53] about some of their different [13:53] about some of their different experiences and I'm always fascinated to [13:56] experiences and I'm always fascinated to [13:56] experiences and I'm always fascinated to see what it gets accelerated and how [13:58] see what it gets accelerated and how [13:58] see what it gets accelerated and how they're able to focus when a new tool [14:00] they're able to focus when a new tool [14:00] they're able to focus when a new tool comes out on really what is their core [14:02] comes out on really what is their core [14:02] comes out on really what is their core thing they're working on. So, up next we [14:04] thing they're working on. So, up next we [14:04] thing they're working on. So, up next we have Danny Grant who is with jam.dev [14:07] have Danny Grant who is with jam.dev [14:07] have Danny Grant who is with jam.dev which is an in browser tool for helping [14:09] which is an in browser tool for helping [14:09] which is an in browser tool for helping you evaluate your site and to figure out [14:10] you evaluate your site and to figure out [14:10] you evaluate your site and to figure out basically how to improve it. Is that [14:13] basically how to improve it. Is that [14:13] basically how to improve it. Is that correct? [14:13] correct? [14:13] correct? >> I'm so happy to be here and actually [14:15] >> I'm so happy to be here and actually [14:15] >> I'm so happy to be here and actually later today on stage we're going to [14:16] later today on stage we're going to [14:16] later today on stage we're going to announce a brand new tool. [14:18] announce a brand new tool. [14:18] announce a brand new tool. >> Oh boy. It lets any PM, designer, [14:21] >> Oh boy. It lets any PM, designer, [14:21] >> Oh boy. It lets any PM, designer, marketer Oh, I'm a little short. [14:23] marketer Oh, I'm a little short. [14:23] marketer Oh, I'm a little short. >> No, no, no. Micron's too tall. It's all [14:25] >> No, no, no. Micron's too tall. It's all [14:25] >> No, no, no. Micron's too tall. It's all good. [14:25] good. [14:25] good. >> I agree. Um, well, just like you just [14:28] >> I agree. Um, well, just like you just [14:28] >> I agree. Um, well, just like you just did, it helps anyone fix what's broken [14:30] did, it helps anyone fix what's broken [14:30] did, it helps anyone fix what's broken instantly without writing code. Um, it's [14:32] instantly without writing code. Um, it's [14:32] instantly without writing code. Um, it's called please fix. [14:34] called please fix. [14:34] called please fix. >> Okay. So, I would use just like my [14:36] >> Okay. So, I would use just like my [14:36] >> Okay. So, I would use just like my website, go into it and say just please [14:39] website, go into it and say just please [14:39] website, go into it and say just please fix or [14:41] fix or [14:41] fix or >> Yeah, I mean like yesterday before this [14:43] >> Yeah, I mean like yesterday before this [14:43] >> Yeah, I mean like yesterday before this tool was announced. Um, if you wanted to [14:46] tool was announced. Um, if you wanted to [14:46] tool was announced. Um, if you wanted to change like some copy on your site or [14:48] change like some copy on your site or [14:48] change like some copy on your site or like a like a button didn't look good or [14:51] like a like a button didn't look good or [14:51] like a like a button didn't look good or missed a hover state, you'd have to [14:53] missed a hover state, you'd have to [14:53] missed a hover state, you'd have to probably ask an engineer to do that for [14:54] probably ask an engineer to do that for [14:54] probably ask an engineer to do that for you. [14:55] you. [14:55] you. >> And the engineer, [14:56] >> And the engineer, [14:56] >> And the engineer, >> nobody wants to do that. No. [14:57] >> nobody wants to do that. No. [14:57] >> nobody wants to do that. No. >> Like, okay. And then they'd be like, can [14:58] >> Like, okay. And then they'd be like, can [14:58] >> Like, okay. And then they'd be like, can you make a ticket? You like, okay. You [14:59] you make a ticket? You like, okay. You [14:59] you make a ticket? You like, okay. You know, and then the the ticket gets [15:01] know, and then the the ticket gets [15:01] know, and then the the ticket gets prioritized and then like maybe they get [15:03] prioritized and then like maybe they get [15:03] prioritized and then like maybe they get to it. [15:03] to it. [15:03] to it. >> I worked for a company. I won't name it [15:05] >> I worked for a company. I won't name it [15:05] >> I worked for a company. I won't name it where we joked it was easier to release [15:06] where we joked it was easier to release [15:06] where we joked it was easier to release a model than put up a blog post. [15:08] a model than put up a blog post. [15:08] a model than put up a blog post. >> That's hilarious. [15:09] >> That's hilarious. [15:09] >> That's hilarious. >> Yeah, we won't name it. [15:10] >> Yeah, we won't name it. [15:10] >> Yeah, we won't name it. >> Um well, so so now as of today, what you [15:13] >> Um well, so so now as of today, what you [15:13] >> Um well, so so now as of today, what you can do is while you're looking at your [15:14] can do is while you're looking at your [15:14] can do is while you're looking at your site, you click on the please fix [15:16] site, you click on the please fix [15:16] site, you click on the please fix browser extension and then it lets you [15:18] browser extension and then it lets you [15:18] browser extension and then it lets you edit your site right there, like it's a [15:20] edit your site right there, like it's a [15:20] edit your site right there, like it's a Google doc, like it's a Figma. And then [15:22] Google doc, like it's a Figma. And then [15:22] Google doc, like it's a Figma. And then when you're ready and you like how it [15:23] when you're ready and you like how it [15:23] when you're ready and you like how it looks, you just click submit and it [15:26] looks, you just click submit and it [15:26] looks, you just click submit and it creates the PR for you. And it uses your [15:28] creates the PR for you. And it uses your [15:28] creates the PR for you. And it uses your design system. So engineers like the [15:29] design system. So engineers like the [15:29] design system. So engineers like the PRs. They're very clean. They're very [15:31] PRs. They're very clean. They're very [15:31] PRs. They're very clean. They're very tight. [15:32] tight. [15:32] tight. >> Very cool. What were you most excited [15:33] >> Very cool. What were you most excited [15:33] >> Very cool. What were you most excited about? about what you saw today. [15:35] about? about what you saw today. [15:35] about? about what you saw today. >> I mean, I think we just saw a new way to [15:36] >> I mean, I think we just saw a new way to [15:36] >> I mean, I think we just saw a new way to browse the web. Like I think OpenAI [15:38] browse the web. Like I think OpenAI [15:38] browse the web. Like I think OpenAI maybe just changed what we mean by the [15:41] maybe just changed what we mean by the [15:41] maybe just changed what we mean by the web, what we mean by a browser. Like if [15:43] web, what we mean by a browser. Like if [15:43] web, what we mean by a browser. Like if you think like web one is read, web two [15:47] you think like web one is read, web two [15:47] you think like web one is read, web two is read, write. Okay, there's web three. [15:49] is read, write. Okay, there's web three. [15:49] is read, write. Okay, there's web three. And then maybe this is like web four [15:52] And then maybe this is like web four [15:52] And then maybe this is like web four like read write think. I think we just [15:53] like read write think. I think we just [15:53] like read write think. I think we just saw a whole new way of experiencing the [15:55] saw a whole new way of experiencing the [15:55] saw a whole new way of experiencing the web where it's a lot less mechanical and [15:59] web where it's a lot less mechanical and [15:59] web where it's a lot less mechanical and it's a lot more stream of consciousness. [16:01] it's a lot more stream of consciousness. [16:01] it's a lot more stream of consciousness. So you talk about the apps inside chat [16:02] So you talk about the apps inside chat [16:02] So you talk about the apps inside chat GPT, right? [16:03] GPT, right? [16:03] GPT, right? >> It's freaking cool. [16:04] >> It's freaking cool. [16:04] >> It's freaking cool. >> Yeah, it is. It's it makes you think a [16:07] >> Yeah, it is. It's it makes you think a [16:07] >> Yeah, it is. It's it makes you think a lot about when you're building [16:09] lot about when you're building [16:09] lot about when you're building something, what the core functionality [16:10] something, what the core functionality [16:10] something, what the core functionality is, like what the purpose of it is. Any [16:12] is, like what the purpose of it is. Any [16:12] is, like what the purpose of it is. Any idea that if it's going to be presented [16:14] idea that if it's going to be presented [16:14] idea that if it's going to be presented inside chat GPT and so watching the demo [16:16] inside chat GPT and so watching the demo [16:16] inside chat GPT and so watching the demo where they interacted with with the [16:18] where they interacted with with the [16:18] where they interacted with with the Zillow website and actually having to [16:19] Zillow website and actually having to [16:20] Zillow website and actually having to have data about that and be able to [16:21] have data about that and be able to [16:21] have data about that and be able to drill down into it and it's it's [16:23] drill down into it and it's it's [16:24] drill down into it and it's it's exciting to think about the [16:24] exciting to think about the [16:24] exciting to think about the possibilities there. And so I could see [16:26] possibilities there. And so I could see [16:26] possibilities there. And so I could see for you all that seems like a pretty [16:28] for you all that seems like a pretty [16:28] for you all that seems like a pretty interesting area because people as they [16:29] interesting area because people as they [16:29] interesting area because people as they focus on usability, they're gonna want [16:30] focus on usability, they're gonna want [16:30] focus on usability, they're gonna want to make sure those things work really [16:32] to make sure those things work really [16:32] to make sure those things work really well. [16:32] well. [16:32] well. >> Yeah. Like now when you build your [16:34] >> Yeah. Like now when you build your [16:34] >> Yeah. Like now when you build your inside of chat GPT app, if you're the [16:37] inside of chat GPT app, if you're the [16:37] inside of chat GPT app, if you're the PM, if you're the designer and you want [16:38] PM, if you're the designer and you want [16:38] PM, if you're the designer and you want to tweak some things and make make the [16:40] to tweak some things and make make the [16:40] to tweak some things and make make the app look a little nicer, you can just [16:42] app look a little nicer, you can just [16:42] app look a little nicer, you can just use our browser extension, change it [16:44] use our browser extension, change it [16:44] use our browser extension, change it from the chat GPT interface and make the [16:46] from the chat GPT interface and make the [16:46] from the chat GPT interface and make the PR in GitHub. It's so cool. And I think, [16:48] PR in GitHub. It's so cool. And I think, [16:48] PR in GitHub. It's so cool. And I think, you know, these tiny tweaks, they often [16:50] you know, these tiny tweaks, they often [16:50] you know, these tiny tweaks, they often don't get prioritized today, but they [16:52] don't get prioritized today, but they [16:52] don't get prioritized today, but they should because the difference between a [16:54] should because the difference between a [16:54] should because the difference between a fine designed product and a [16:56] fine designed product and a [16:56] fine designed product and a well-designed product is that the [16:57] well-designed product is that the [16:57] well-designed product is that the well-designed product changes the world. [16:58] well-designed product changes the world. [16:58] well-designed product changes the world. Like the iPhone changes the world [17:00] Like the iPhone changes the world [17:00] Like the iPhone changes the world because it's usable. And there were [17:02] because it's usable. And there were [17:02] because it's usable. And there were actually attempts before that that I [17:04] actually attempts before that that I [17:04] actually attempts before that that I won't name that many people under the [17:05] won't name that many people under the [17:06] won't name that many people under the age of 30 have never heard of um because [17:08] age of 30 have never heard of um because [17:08] age of 30 have never heard of um because they just they didn't have that [17:10] they just they didn't have that [17:10] they just they didn't have that attention to detail. [17:12] attention to detail. [17:12] attention to detail. >> Have you seen your development process [17:14] >> Have you seen your development process [17:14] >> Have you seen your development process change with tools that have coming out, [17:16] change with tools that have coming out, [17:16] change with tools that have coming out, particularly AI tools? Yeah, I mean I [17:19] particularly AI tools? Yeah, I mean I [17:19] particularly AI tools? Yeah, I mean I like I'm biased. We've been using [17:21] like I'm biased. We've been using [17:21] like I'm biased. We've been using PleaseFix as we've been building [17:22] PleaseFix as we've been building [17:22] PleaseFix as we've been building PleaseFix and it like even today our [17:25] PleaseFix and it like even today our [17:25] PleaseFix and it like even today our pricing page changed dramatically [17:27] pricing page changed dramatically [17:27] pricing page changed dramatically because a PM could just go in and test a [17:29] because a PM could just go in and test a [17:29] because a PM could just go in and test a bunch of stuff without asking an [17:30] bunch of stuff without asking an [17:30] bunch of stuff without asking an engineer. I think the thing I'm most [17:32] engineer. I think the thing I'm most [17:32] engineer. I think the thing I'm most excited about is this idea that you can [17:34] excited about is this idea that you can [17:34] excited about is this idea that you can move as fast as your entire creative [17:36] move as fast as your entire creative [17:36] move as fast as your entire creative team altogether. Like no disruptions for [17:38] team altogether. Like no disruptions for [17:38] team altogether. Like no disruptions for engineers. It's like people can move [17:41] engineers. It's like people can move [17:41] engineers. It's like people can move without having to have bottlenecks on a [17:45] without having to have bottlenecks on a [17:45] without having to have bottlenecks on a few people. So it seems kind of cool [17:47] few people. So it seems kind of cool [17:47] few people. So it seems kind of cool because what you're talking about is the [17:48] because what you're talking about is the [17:48] because what you're talking about is the idea that people who aren't specifically [17:50] idea that people who aren't specifically [17:50] idea that people who aren't specifically engineer background are able to make [17:52] engineer background are able to make [17:52] engineer background are able to make those changes and has that been [17:53] those changes and has that been [17:53] those changes and has that been something you've seen basically within [17:55] something you've seen basically within [17:55] something you've seen basically within your company the adoption of tools like [17:56] your company the adoption of tools like [17:56] your company the adoption of tools like vibequ and stuff like this for people to [17:58] vibequ and stuff like this for people to [17:58] vibequ and stuff like this for people to contribute and be able to come up with [17:59] contribute and be able to come up with [17:59] contribute and be able to come up with ideas. [18:00] ideas. [18:00] ideas. >> Yeah, it's really cool. Um when we we [18:03] >> Yeah, it's really cool. Um when we we [18:03] >> Yeah, it's really cool. Um when we we talk to our users every day and some of [18:04] talk to our users every day and some of [18:04] talk to our users every day and some of the stories we hear are awesome. Like [18:06] the stories we hear are awesome. Like [18:06] the stories we hear are awesome. Like last week we talked to a user who's a [18:07] last week we talked to a user who's a [18:07] last week we talked to a user who's a firefighter and building software for [18:09] firefighter and building software for [18:09] firefighter and building software for firefighters. That's cool. um talked to [18:11] firefighters. That's cool. um talked to [18:12] firefighters. That's cool. um talked to a user a couple weeks ago who um grew up [18:14] a user a couple weeks ago who um grew up [18:14] a user a couple weeks ago who um grew up in the church system and is building [18:15] in the church system and is building [18:15] in the church system and is building software for churches has no software [18:17] software for churches has no software [18:17] software for churches has no software experience but is able to now make [18:19] experience but is able to now make [18:19] experience but is able to now make something that's very impactful to their [18:21] something that's very impactful to their [18:21] something that's very impactful to their community. I think that is awesome. I [18:23] community. I think that is awesome. I [18:23] community. I think that is awesome. I think we are about to see the Cambrian [18:24] think we are about to see the Cambrian [18:24] think we are about to see the Cambrian explosion of software the same way that [18:26] explosion of software the same way that [18:26] explosion of software the same way that with the web there's the Cambrian [18:28] with the web there's the Cambrian [18:28] with the web there's the Cambrian explosion of like news sources with you [18:29] explosion of like news sources with you [18:30] explosion of like news sources with you know Substack and Twitter. I I think [18:31] know Substack and Twitter. I I think [18:31] know Substack and Twitter. I I think this is about to happen for software and [18:33] this is about to happen for software and [18:33] this is about to happen for software and it's I think it's one of the best things [18:34] it's I think it's one of the best things [18:34] it's I think it's one of the best things for humanity. [18:35] for humanity. [18:35] for humanity. >> So how do things work at jam.dev dev as [18:38] >> So how do things work at jam.dev dev as [18:38] >> So how do things work at jam.dev dev as far as ideation, testing things with [18:41] far as ideation, testing things with [18:41] far as ideation, testing things with customers, releasing, and then figuring [18:43] customers, releasing, and then figuring [18:43] customers, releasing, and then figuring out what's best features and whatnot. [18:45] out what's best features and whatnot. [18:45] out what's best features and whatnot. >> The only thing we care about is does [18:48] >> The only thing we care about is does [18:48] >> The only thing we care about is does this deliver a wow for our users when [18:50] this deliver a wow for our users when [18:50] this deliver a wow for our users when they use it. And so that's what we're [18:52] they use it. And so that's what we're [18:52] they use it. And so that's what we're focused on. There's an emotional [18:53] focused on. There's an emotional [18:53] focused on. There's an emotional response we want people to have when [18:54] response we want people to have when [18:54] response we want people to have when they use our product because we work on [18:56] they use our product because we work on [18:56] they use our product because we work on the worst part of software development. [18:58] the worst part of software development. [18:58] the worst part of software development. Like I don't know anyone who is like [19:00] Like I don't know anyone who is like [19:00] Like I don't know anyone who is like today I want to fix some bugs, right? [19:02] today I want to fix some bugs, right? [19:02] today I want to fix some bugs, right? >> And so our job is just to make that [19:04] >> And so our job is just to make that [19:04] >> And so our job is just to make that whole part of the software experience a [19:06] whole part of the software experience a [19:06] whole part of the software experience a lot better. And so we're laser focused [19:07] lot better. And so we're laser focused [19:07] lot better. And so we're laser focused on that. We're just constantly talking [19:09] on that. We're just constantly talking [19:09] on that. We're just constantly talking to users. Every user that signs up from [19:10] to users. Every user that signs up from [19:10] to users. Every user that signs up from Jam hears from a co-founder. Every user [19:12] Jam hears from a co-founder. Every user [19:12] Jam hears from a co-founder. Every user who uses Jam hears from a PM. We are [19:14] who uses Jam hears from a PM. We are [19:14] who uses Jam hears from a PM. We are just in constant contact. [19:16] just in constant contact. [19:16] just in constant contact. >> When I worked at OpenAn, one of the most [19:18] >> When I worked at OpenAn, one of the most [19:18] >> When I worked at OpenAn, one of the most exciting things, and it sounds [19:20] exciting things, and it sounds [19:20] exciting things, and it sounds absolutely silly today was when we [19:22] absolutely silly today was when we [19:22] absolutely silly today was when we watched GPD3 spit out a React button. [19:24] watched GPD3 spit out a React button. [19:24] watched GPD3 spit out a React button. That was like, oh my gosh, I could do [19:25] That was like, oh my gosh, I could do [19:26] That was like, oh my gosh, I could do that. [19:26] that. [19:26] that. >> I remember that. I I don't remember it [19:27] >> I remember that. I I don't remember it [19:27] >> I remember that. I I don't remember it inside, but I remember from the outside. [19:29] inside, but I remember from the outside. [19:29] inside, but I remember from the outside. I remember seeing demos of that being [19:30] I remember seeing demos of that being [19:30] I remember seeing demos of that being like, the world just changed. That was [19:32] like, the world just changed. That was [19:32] like, the world just changed. That was that was the threshold for us being [19:34] that was the threshold for us being [19:34] that was the threshold for us being impressed was literally like four lines [19:36] impressed was literally like four lines [19:36] impressed was literally like four lines of code and to be able to oh that's [19:37] of code and to be able to oh that's [19:37] of code and to be able to oh that's really cool it can do that. What has [19:39] really cool it can do that. What has [19:39] really cool it can do that. What has been a big aha moment for you? [19:41] been a big aha moment for you? [19:41] been a big aha moment for you? >> I think look it's it's it's similar to [19:44] >> I think look it's it's it's similar to [19:44] >> I think look it's it's it's similar to what you're saying but when you think of [19:46] what you're saying but when you think of [19:46] what you're saying but when you think of what you just described coupled with [19:48] what you just described coupled with [19:48] what you just described coupled with what was just announced with the apps [19:49] what was just announced with the apps [19:49] what was just announced with the apps SDK. You can imagine that in the future [19:52] SDK. You can imagine that in the future [19:52] SDK. You can imagine that in the future yes humans are going to be building a [19:54] yes humans are going to be building a [19:54] yes humans are going to be building a lot of apps that are shown dynamically [19:56] lot of apps that are shown dynamically [19:56] lot of apps that are shown dynamically as they browse the web. But also I think [19:58] as they browse the web. But also I think [19:58] as they browse the web. But also I think that agents will dynamically build apps [20:00] that agents will dynamically build apps [20:00] that agents will dynamically build apps for you as you browse the web and that [20:03] for you as you browse the web and that [20:03] for you as you browse the web and that opens up a whole world of possibilities [20:05] opens up a whole world of possibilities [20:05] opens up a whole world of possibilities because you can have dynamic software [20:06] because you can have dynamic software [20:06] because you can have dynamic software right there. Like imagine inside of an [20:08] right there. Like imagine inside of an [20:08] right there. Like imagine inside of an organization right now if a PM wants to [20:11] organization right now if a PM wants to [20:11] organization right now if a PM wants to see how a product is doing they have to [20:12] see how a product is doing they have to [20:12] see how a product is doing they have to build a dashboard and it's a lot faster [20:14] build a dashboard and it's a lot faster [20:14] build a dashboard and it's a lot faster to do that today than even six months [20:15] to do that today than even six months [20:15] to do that today than even six months ago. [20:16] ago. [20:16] ago. >> But imagine the PM needs a dashboard and [20:19] >> But imagine the PM needs a dashboard and [20:19] >> But imagine the PM needs a dashboard and they're in chat GBT and chatgbt just [20:22] they're in chat GBT and chatgbt just [20:22] they're in chat GBT and chatgbt just gives it to them and no human had to do [20:24] gives it to them and no human had to do [20:24] gives it to them and no human had to do that. It means that there are two types [20:26] that. It means that there are two types [20:26] that. It means that there are two types of software. There's sort of long-term [20:28] of software. There's sort of long-term [20:28] of software. There's sort of long-term software that humans are going to work [20:30] software that humans are going to work [20:30] software that humans are going to work on. They're going to like really [20:31] on. They're going to like really [20:31] on. They're going to like really fine-tune, make it great, like Zillow, [20:33] fine-tune, make it great, like Zillow, [20:33] fine-tune, make it great, like Zillow, Canva, and then there's going to be sort [20:35] Canva, and then there's going to be sort [20:35] Canva, and then there's going to be sort of disposable one-time use software that [20:36] of disposable one-time use software that [20:36] of disposable one-time use software that an agent can just whip up. And that is [20:38] an agent can just whip up. And that is [20:38] an agent can just whip up. And that is just freaking cool. [20:39] just freaking cool. [20:39] just freaking cool. >> Yeah, it's it's a thing where, you know, [20:42] >> Yeah, it's it's a thing where, you know, [20:42] >> Yeah, it's it's a thing where, you know, as a developer, you're used to sometimes [20:43] as a developer, you're used to sometimes [20:43] as a developer, you're used to sometimes spinning up a tool one time to use that [20:45] spinning up a tool one time to use that [20:45] spinning up a tool one time to use that and then you're done with it. But it's [20:47] and then you're done with it. But it's [20:47] and then you're done with it. But it's another thing to think about that could [20:48] another thing to think about that could [20:48] another thing to think about that could be a new modality. You know, we get kind [20:49] be a new modality. You know, we get kind [20:50] be a new modality. You know, we get kind of fixated in sort of like the app store [20:51] of fixated in sort of like the app store [20:51] of fixated in sort of like the app store sort of idea. And I think that there's [20:53] sort of idea. And I think that there's [20:53] sort of idea. And I think that there's going to be, like you said, long-term [20:54] going to be, like you said, long-term [20:54] going to be, like you said, long-term stuff that you deploy inside a chat GPT, [20:56] stuff that you deploy inside a chat GPT, [20:56] stuff that you deploy inside a chat GPT, but then the ability for somebody just [20:58] but then the ability for somebody just [20:58] but then the ability for somebody just to spin up a thing they're only going to [20:59] to spin up a thing they're only going to [20:59] to spin up a thing they're only going to use once and that's the end of it. [21:01] use once and that's the end of it. [21:01] use once and that's the end of it. >> Yeah. [21:01] >> Yeah. [21:01] >> Yeah. >> And have you seen as far as things that [21:04] >> And have you seen as far as things that [21:04] >> And have you seen as far as things that sort of start off as hobby projects or [21:06] sort of start off as hobby projects or [21:06] sort of start off as hobby projects or things like that or maybe inside [21:07] things like that or maybe inside [21:07] things like that or maybe inside jam.dev, people said, "Hey, I have this [21:09] jam.dev, people said, "Hey, I have this [21:09] jam.dev, people said, "Hey, I have this idea. I wanted to solve for this thing, [21:10] idea. I wanted to solve for this thing, [21:10] idea. I wanted to solve for this thing, maybe even something from finance or [21:11] maybe even something from finance or [21:12] maybe even something from finance or comms or something that's turned out to [21:13] comms or something that's turned out to [21:13] comms or something that's turned out to be something useful." [21:14] be something useful." [21:14] be something useful." >> Yesterday we were um at the rehearsal [21:16] >> Yesterday we were um at the rehearsal [21:16] >> Yesterday we were um at the rehearsal for our 10x codebase session. It's four [21:18] for our 10x codebase session. It's four [21:18] for our 10x codebase session. It's four startups. We're all demoing and we're [21:20] startups. We're all demoing and we're [21:20] startups. We're all demoing and we're just sitting around kind of backstage [21:21] just sitting around kind of backstage [21:21] just sitting around kind of backstage talking and we're like, "Oh, how did you [21:22] talking and we're like, "Oh, how did you [21:22] talking and we're like, "Oh, how did you start your startup?" And almost [21:24] start your startup?" And almost [21:24] start your startup?" And almost everyone, three out of four, I'm the [21:27] everyone, three out of four, I'm the [21:27] everyone, three out of four, I'm the only one um where their startup started [21:30] only one um where their startup started [21:30] only one um where their startup started as an internal tool at their company [21:32] as an internal tool at their company [21:32] as an internal tool at their company that they needed and actually these [21:33] that they needed and actually these [21:33] that they needed and actually these startups pivoted to the internal tool [21:35] startups pivoted to the internal tool [21:35] startups pivoted to the internal tool because they found it so tremendously [21:37] because they found it so tremendously [21:38] because they found it so tremendously valuable to how they build software. [21:40] valuable to how they build software. [21:40] valuable to how they build software. >> What's you know how Slack got it start [21:42] >> What's you know how Slack got it start [21:42] >> What's you know how Slack got it start and other companies? It's a very [21:43] and other companies? It's a very [21:43] and other companies? It's a very interesting thing where the thing you [21:45] interesting thing where the thing you [21:45] interesting thing where the thing you spend the most time on is often the [21:47] spend the most time on is often the [21:47] spend the most time on is often the thing that's going to be the best [21:48] thing that's going to be the best [21:48] thing that's going to be the best product. And you know, one of the things [21:49] product. And you know, one of the things [21:49] product. And you know, one of the things they talked about OpenAI on here today [21:51] they talked about OpenAI on here today [21:51] they talked about OpenAI on here today was like how 70% of the codes coming [21:54] was like how 70% of the codes coming [21:54] was like how 70% of the codes coming from, you know, the PRs are coming from, [21:55] from, you know, the PRs are coming from, [21:55] from, you know, the PRs are coming from, you know, generated by codeex. And it's [21:57] you know, generated by codeex. And it's [21:58] you know, generated by codeex. And it's a very interesting thing to see when a [21:59] a very interesting thing to see when a [21:59] a very interesting thing to see when a company's using their own tool to build [22:01] company's using their own tool to build [22:01] company's using their own tool to build the tool, it seems to iterate much [22:02] the tool, it seems to iterate much [22:02] the tool, it seems to iterate much faster. [22:03] faster. [22:03] faster. >> It's actually so funny because if you [22:04] >> It's actually so funny because if you [22:04] >> It's actually so funny because if you listen to standard startup advice, like [22:06] listen to standard startup advice, like [22:06] listen to standard startup advice, like what would Paul Paul Graham tell you? [22:08] what would Paul Paul Graham tell you? [22:08] what would Paul Paul Graham tell you? It's hey, don't optimize the internal [22:10] It's hey, don't optimize the internal [22:10] It's hey, don't optimize the internal processes. Like just do things that [22:11] processes. Like just do things that [22:11] processes. Like just do things that don't scale. do them poorly. But [22:13] don't scale. do them poorly. But [22:13] don't scale. do them poorly. But actually, it's when you take a lot of [22:15] actually, it's when you take a lot of [22:15] actually, it's when you take a lot of care in your own processes that you can [22:17] care in your own processes that you can [22:17] care in your own processes that you can develop these products that can be their [22:18] develop these products that can be their [22:18] develop these products that can be their own companies and can help a lot of [22:20] own companies and can help a lot of [22:20] own companies and can help a lot of people. [22:20] people. [22:20] people. >> Okay. It might be that that is going to [22:21] >> Okay. It might be that that is going to [22:21] >> Okay. It might be that that is going to be the advantage is how quickly [22:23] be the advantage is how quickly [22:23] be the advantage is how quickly internally you make something because I [22:24] internally you make something because I [22:24] internally you make something because I think you're going to see a leveling [22:26] think you're going to see a leveling [22:26] think you're going to see a leveling effect with tools like the agent kit and [22:29] effect with tools like the agent kit and [22:29] effect with tools like the agent kit and everything else because it's going to be [22:31] everything else because it's going to be [22:31] everything else because it's going to be probably having a big technical bench [22:33] probably having a big technical bench [22:33] probably having a big technical bench isn't going to matter as much as having [22:35] isn't going to matter as much as having [22:35] isn't going to matter as much as having good product depth or understanding of [22:37] good product depth or understanding of [22:37] good product depth or understanding of your customer. [22:38] your customer. [22:38] your customer. >> Yeah. At the end of the day, a human has [22:40] >> Yeah. At the end of the day, a human has [22:40] >> Yeah. At the end of the day, a human has to use the software. And if a human has [22:42] to use the software. And if a human has [22:42] to use the software. And if a human has to use it, it has to be easy for the [22:43] to use it, it has to be easy for the [22:43] to use it, it has to be easy for the human to use it. And I do think design [22:45] human to use it. And I do think design [22:45] human to use it. And I do think design still today and maybe forever makes or [22:47] still today and maybe forever makes or [22:47] still today and maybe forever makes or breaks a difference. [22:48] breaks a difference. [22:48] breaks a difference. >> So you use jam.dev on the jam.dev site. [22:50] >> So you use jam.dev on the jam.dev site. [22:50] >> So you use jam.dev on the jam.dev site. >> Yeah. Yeah. And if you want to see [22:51] >> Yeah. Yeah. And if you want to see [22:51] >> Yeah. Yeah. And if you want to see what's brand new, go to jam.devfix. [22:54] what's brand new, go to jam.devfix. [22:54] what's brand new, go to jam.devfix. >> Okay. How often do you guys making [22:56] >> Okay. How often do you guys making [22:56] >> Okay. How often do you guys making updates on your own site? [22:57] updates on your own site? [22:57] updates on your own site? >> Um, right now we have an [22:59] >> Um, right now we have an [22:59] >> Um, right now we have an >> You did your pricing page. You did that. [23:01] >> You did your pricing page. You did that. [23:01] >> You did your pricing page. You did that. So yeah, [23:02] So yeah, [23:02] So yeah, >> too often. Is it become does that become [23:04] >> too often. Is it become does that become [23:04] >> too often. Is it become does that become sort of thing like well if it's so easy [23:05] sort of thing like well if it's so easy [23:05] sort of thing like well if it's so easy to change it you know [23:07] to change it you know [23:07] to change it you know >> we just added dark mode we added pricing [23:09] >> we just added dark mode we added pricing [23:09] >> we just added dark mode we added pricing page uh we do copy updates yeah now it's [23:12] page uh we do copy updates yeah now it's [23:12] page uh we do copy updates yeah now it's too easy [23:12] too easy [23:12] too easy >> is there going to be like a please fix [23:14] >> is there going to be like a please fix [23:14] >> is there going to be like a please fix wait you know [23:15] wait you know [23:15] wait you know >> oh please fix with a polite delay [23:17] >> oh please fix with a polite delay [23:17] >> oh please fix with a polite delay >> yeah or like maybe think about this like [23:19] >> yeah or like maybe think about this like [23:19] >> yeah or like maybe think about this like you know [23:19] you know [23:19] you know >> yeah we should add as a feature for [23:20] >> yeah we should add as a feature for [23:20] >> yeah we should add as a feature for engineers that if you made too many [23:22] engineers that if you made too many [23:22] engineers that if you made too many changes it like arbitrarily waits a few [23:25] changes it like arbitrarily waits a few [23:25] changes it like arbitrarily waits a few days and then asks for a change again [23:27] days and then asks for a change again [23:27] days and then asks for a change again >> do you really want this yeah [23:29] >> do you really want this yeah [23:29] >> do you really want this yeah >> what are you looking forward to what [23:31] >> what are you looking forward to what [23:31] >> what are you looking forward to what kind of tools would you like to see that [23:32] kind of tools would you like to see that [23:32] kind of tools would you like to see that don't exist yet. [23:33] don't exist yet. [23:33] don't exist yet. >> Um I well I was I was talking to the [23:35] >> Um I well I was I was talking to the [23:35] >> Um I well I was I was talking to the engineers sitting around me during the [23:37] engineers sitting around me during the [23:37] engineers sitting around me during the keynote [23:37] keynote [23:37] keynote >> and they were really excited about the [23:40] >> and they were really excited about the [23:40] >> and they were really excited about the optimizer and the evals as part of the [23:43] optimizer and the evals as part of the [23:43] optimizer and the evals as part of the agent kit [23:44] agent kit [23:44] agent kit >> and um and what they wanted to see was [23:47] >> and um and what they wanted to see was [23:47] >> and um and what they wanted to see was well like what if the eval could be [23:50] well like what if the eval could be [23:50] well like what if the eval could be written automatically for you using your [23:52] written automatically for you using your [23:52] written automatically for you using your own data and then use that to [23:54] own data and then use that to [23:54] own data and then use that to automatically optimize your prompt. So [23:56] automatically optimize your prompt. So [23:56] automatically optimize your prompt. So rather than people sort of sweating the [23:57] rather than people sort of sweating the [23:57] rather than people sort of sweating the details about these agents, what if the [23:59] details about these agents, what if the [23:59] details about these agents, what if the agents could improve themselves? And I [24:00] agents could improve themselves? And I [24:00] agents could improve themselves? And I think if we had that at Jim, like we [24:02] think if we had that at Jim, like we [24:02] think if we had that at Jim, like we could move a lot faster and we can make [24:03] could move a lot faster and we can make [24:03] could move a lot faster and we can make a lot more powerful software a lot [24:05] a lot more powerful software a lot [24:05] a lot more powerful software a lot sooner. [24:06] sooner. [24:06] sooner. >> Uh what advice do you have to founders [24:09] >> Uh what advice do you have to founders [24:09] >> Uh what advice do you have to founders or developers right now? [24:10] or developers right now? [24:10] or developers right now? >> It's it's never been a more fun time to [24:13] >> It's it's never been a more fun time to [24:13] >> It's it's never been a more fun time to build. I think we all just get to enjoy [24:14] build. I think we all just get to enjoy [24:14] build. I think we all just get to enjoy it. [24:15] it. [24:15] it. >> So how did you figure out that this your [24:17] >> So how did you figure out that this your [24:17] >> So how did you figure out that this your team wanted to work on with jam.dev? How [24:19] team wanted to work on with jam.dev? How [24:19] team wanted to work on with jam.dev? How did you decide this was the problem you [24:21] did you decide this was the problem you [24:21] did you decide this was the problem you wanted to solve? [24:22] wanted to solve? [24:22] wanted to solve? >> When uh my co-founder and I were product [24:24] >> When uh my co-founder and I were product [24:24] >> When uh my co-founder and I were product managers together at Cloudflare, we we [24:26] managers together at Cloudflare, we we [24:26] managers together at Cloudflare, we we worked on the fastest moving team in the [24:29] worked on the fastest moving team in the [24:29] worked on the fastest moving team in the company. It's this sort of skunk works [24:31] company. It's this sort of skunk works [24:31] company. It's this sort of skunk works team that would try to do big new [24:33] team that would try to do big new [24:33] team that would try to do big new things. It's the team that shipped [24:35] things. It's the team that shipped [24:35] things. It's the team that shipped Cloudflare workers um their cloud [24:37] Cloudflare workers um their cloud [24:37] Cloudflare workers um their cloud compute platform. It shipped 1.1.1.1 [24:39] compute platform. It shipped 1.1.1.1 [24:39] compute platform. It shipped 1.1.1.1 which now fields a trillion DNS queries [24:41] which now fields a trillion DNS queries [24:41] which now fields a trillion DNS queries a month. Um and it was just trying to [24:45] a month. Um and it was just trying to [24:45] a month. Um and it was just trying to move fast on this team that we realized [24:46] move fast on this team that we realized [24:46] move fast on this team that we realized a lot of the frustrations and [24:48] a lot of the frustrations and [24:48] a lot of the frustrations and bottlenecks come from just reproducing [24:50] bottlenecks come from just reproducing [24:50] bottlenecks come from just reproducing issues and there's no tooling out there [24:52] issues and there's no tooling out there [24:52] issues and there's no tooling out there and no user, no PM, no one in sales [24:55] and no user, no PM, no one in sales [24:55] and no user, no PM, no one in sales knows how to communicate to an engineer [24:57] knows how to communicate to an engineer [24:57] knows how to communicate to an engineer what the engineer needs to know. And [24:59] what the engineer needs to know. And [24:59] what the engineer needs to know. And we're like, we can't believe that [25:00] we're like, we can't believe that [25:00] we're like, we can't believe that there's no way to help the engineer get [25:03] there's no way to help the engineer get [25:03] there's no way to help the engineer get things done faster. We're spending all [25:04] things done faster. We're spending all [25:04] things done faster. We're spending all this like great brain time on [25:07] this like great brain time on [25:07] this like great brain time on communicating bugs and hopping in calls [25:09] communicating bugs and hopping in calls [25:09] communicating bugs and hopping in calls and screen sharing and and not enough on [25:11] and screen sharing and and not enough on [25:11] and screen sharing and and not enough on fixing them. And we thought that that [25:12] fixing them. And we thought that that [25:12] fixing them. And we thought that that that we can solve. [25:14] that we can solve. [25:14] that we can solve. >> It seems uh like a lot of things sort of [25:17] >> It seems uh like a lot of things sort of [25:17] >> It seems uh like a lot of things sort of in front of you as far as what you can [25:19] in front of you as far as what you can [25:19] in front of you as far as what you can do. How do you decide what you're going [25:20] do. How do you decide what you're going [25:20] do. How do you decide what you're going to do next? [25:27] >> Okay. I think I think there's something [25:27] >> Okay. I think I think there's something that we undertalk about in like startup [25:29] that we undertalk about in like startup [25:29] that we undertalk about in like startup world which is if if your startup works [25:32] world which is if if your startup works [25:32] world which is if if your startup works up works out to your wildest dreams [25:35] up works out to your wildest dreams [25:35] up works out to your wildest dreams you're going to work on it for like 10 [25:36] you're going to work on it for like 10 [25:36] you're going to work on it for like 10 years [25:37] years [25:37] years >> and that's a really long time and so you [25:39] >> and that's a really long time and so you [25:39] >> and that's a really long time and so you better just love the problem because [25:40] better just love the problem because [25:40] better just love the problem because that makes it really really fun and so I [25:42] that makes it really really fun and so I [25:42] that makes it really really fun and so I think you work on the thing where if you [25:44] think you work on the thing where if you [25:44] think you work on the thing where if you get to wake up every day work on it and [25:46] get to wake up every day work on it and [25:46] get to wake up every day work on it and talk to users of the thing you're going [25:48] talk to users of the thing you're going [25:48] talk to users of the thing you're going to be pretty darn happy about that. [25:50] to be pretty darn happy about that. [25:50] to be pretty darn happy about that. >> That's pretty awesome. Uh how can people [25:51] >> That's pretty awesome. Uh how can people [25:51] >> That's pretty awesome. Uh how can people get started at jam.dev [25:52] get started at jam.dev [25:52] get started at jam.dev >> jam.dev dev/pleasfix and we'll please [25:55] >> jam.dev dev/pleasfix and we'll please [25:55] >> jam.dev dev/pleasfix and we'll please fix some code for you. [25:56] fix some code for you. [25:56] fix some code for you. >> Awesome. Thank you so much. [25:57] >> Awesome. Thank you so much. [25:57] >> Awesome. Thank you so much. >> Thank you. [25:59] >> Thank you. [25:59] >> Thank you. >> Uh [26:00] >> Uh [26:00] >> Uh so it's interesting to see what each [26:03] so it's interesting to see what each [26:03] so it's interesting to see what each kind of developer what they're looking [26:04] kind of developer what they're looking [26:04] kind of developer what they're looking forward to, what kind of excites them as [26:07] forward to, what kind of excites them as [26:07] forward to, what kind of excites them as far as that. A lot of that's based upon [26:08] far as that. A lot of that's based upon [26:08] far as that. A lot of that's based upon what they've been working on. I know a [26:10] what they've been working on. I know a [26:10] what they've been working on. I know a lot of devs have to build a lot of [26:12] lot of devs have to build a lot of [26:12] lot of devs have to build a lot of internal tooling and when you can solve [26:13] internal tooling and when you can solve [26:14] internal tooling and when you can solve for that with an SDK, it makes your life [26:16] for that with an SDK, it makes your life [26:16] for that with an SDK, it makes your life a lot easier because you can focus on [26:17] a lot easier because you can focus on [26:17] a lot easier because you can focus on the thing you want to work with. Up next [26:19] the thing you want to work with. Up next [26:19] the thing you want to work with. Up next we have Zach Lipton who is with a [26:21] we have Zach Lipton who is with a [26:21] we have Zach Lipton who is with a bridge. How you doing Zach? [26:23] bridge. How you doing Zach? [26:23] bridge. How you doing Zach? >> Not too bad. How you doing? [26:25] >> Not too bad. How you doing? [26:25] >> Not too bad. How you doing? >> Fantastic. So a bridge you're working on [26:27] >> Fantastic. So a bridge you're working on [26:27] >> Fantastic. So a bridge you're working on tools for helping the medical community [26:28] tools for helping the medical community [26:28] tools for helping the medical community people doing like transcription and that [26:31] people doing like transcription and that [26:31] people doing like transcription and that kind of area. Is that how you [26:32] kind of area. Is that how you [26:32] kind of area. Is that how you >> API platform for doctor patient [26:34] >> API platform for doctor patient [26:34] >> API platform for doctor patient conversations? Like if we were to like [26:36] conversations? Like if we were to like [26:36] conversations? Like if we were to like rewind to like the state of affairs, you [26:38] rewind to like the state of affairs, you [26:38] rewind to like the state of affairs, you know, uh post adoption of electronic [26:41] know, uh post adoption of electronic [26:41] know, uh post adoption of electronic health records but pre-ambient [26:42] health records but pre-ambient [26:42] health records but pre-ambient listening. um doctors were spending [26:45] listening. um doctors were spending [26:45] listening. um doctors were spending about two hours doing paperwork for [26:47] about two hours doing paperwork for [26:47] about two hours doing paperwork for every one hour of direct patient care. [26:49] every one hour of direct patient care. [26:49] every one hour of direct patient care. So it was this kind of like clerical uh [26:52] So it was this kind of like clerical uh [26:52] So it was this kind of like clerical uh burden crisis. Um and it was pulling it [26:55] burden crisis. Um and it was pulling it [26:55] burden crisis. Um and it was pulling it was a situation where technology was [26:56] was a situation where technology was [26:56] was a situation where technology was pulling doctors away from patients [26:58] pulling doctors away from patients [26:58] pulling doctors away from patients rather than bringing them closer. [26:59] rather than bringing them closer. [26:59] rather than bringing them closer. >> Um what we do is we provide this [27:01] >> Um what we do is we provide this [27:01] >> Um what we do is we provide this platform that helps kind of gives [27:03] platform that helps kind of gives [27:03] platform that helps kind of gives doctors superpowers, helps them with [27:04] doctors superpowers, helps them with [27:04] doctors superpowers, helps them with their paperwork, does all the [27:06] their paperwork, does all the [27:06] their paperwork, does all the note-taking in the background, preps [27:07] note-taking in the background, preps [27:07] note-taking in the background, preps them everything. that all these kind of [27:09] them everything. that all these kind of [27:09] them everything. that all these kind of documentation artifacts are ready for [27:10] documentation artifacts are ready for [27:10] documentation artifacts are ready for them the moment the visit's over in [27:12] them the moment the visit's over in [27:12] them the moment the visit's over in exactly the form they need so they can [27:14] exactly the form they need so they can [27:14] exactly the form they need so they can be fully present with the patient [27:15] be fully present with the patient [27:15] be fully present with the patient instead of spending all their time [27:16] instead of spending all their time [27:16] instead of spending all their time staring at the computer. [27:18] staring at the computer. [27:18] staring at the computer. >> What kind of metrics do you have so far [27:19] >> What kind of metrics do you have so far [27:19] >> What kind of metrics do you have so far as far as like time saved for doctors? [27:22] as far as like time saved for doctors? [27:22] as far as like time saved for doctors? >> Sure. Interesting story um and a [27:25] >> Sure. Interesting story um and a [27:26] >> Sure. Interesting story um and a difficult to track down in part because [27:27] difficult to track down in part because [27:27] difficult to track down in part because there's a time that doctors spend uh [27:30] there's a time that doctors spend uh [27:30] there's a time that doctors spend uh documenting during the day but the [27:32] documenting during the day but the [27:32] documenting during the day but the reality of the status quo before was [27:34] reality of the status quo before was [27:34] reality of the status quo before was that most doctors actually didn't finish [27:36] that most doctors actually didn't finish [27:36] that most doctors actually didn't finish their notes during the workday. So, they [27:38] their notes during the workday. So, they [27:38] their notes during the workday. So, they were home after hours logging into the [27:40] were home after hours logging into the [27:40] were home after hours logging into the EHR and doing what we called um pajama [27:43] EHR and doing what we called um pajama [27:43] EHR and doing what we called um pajama time. They're basically sitting there [27:45] time. They're basically sitting there [27:45] time. They're basically sitting there like pulled away from dinner with their [27:46] like pulled away from dinner with their [27:46] like pulled away from dinner with their families or logged in after hours, you [27:48] families or logged in after hours, you [27:48] families or logged in after hours, you know, sitting in sitting in bed [27:50] know, sitting in sitting in bed [27:50] know, sitting in sitting in bed finishing up their notes. Um our report, [27:53] finishing up their notes. Um our report, [27:53] finishing up their notes. Um our report, so we we we cobble together this [27:55] so we we we cobble together this [27:55] so we we we cobble together this information from a bunch of different [27:55] information from a bunch of different [27:56] information from a bunch of different sources, but we see doctors saving as [27:57] sources, but we see doctors saving as [27:57] sources, but we see doctors saving as much as an hour or more a day. Um we [28:00] much as an hour or more a day. Um we [28:00] much as an hour or more a day. Um we see, you know, doctors some doctors [28:03] see, you know, doctors some doctors [28:03] see, you know, doctors some doctors seeing like 10 15 patients in a day. [28:05] seeing like 10 15 patients in a day. [28:05] seeing like 10 15 patients in a day. we're talking about like you know 5 to [28:07] we're talking about like you know 5 to [28:07] we're talking about like you know 5 to 10 minutes often in note takingaking. So [28:09] 10 minutes often in note takingaking. So [28:10] 10 minutes often in note takingaking. So it's a it's a tremendous um [28:12] it's a it's a tremendous um [28:12] it's a it's a tremendous um >> kind of relief but even beyond the um [28:16] >> kind of relief but even beyond the um [28:16] >> kind of relief but even beyond the um actual time saves oftentimes there's an [28:18] actual time saves oftentimes there's an [28:18] actual time saves oftentimes there's an even larger sort of like perception of [28:20] even larger sort of like perception of [28:20] even larger sort of like perception of burden lift and that's because the [28:22] burden lift and that's because the [28:22] burden lift and that's because the doctor's worrying about less and able to [28:23] doctor's worrying about less and able to [28:23] doctor's worrying about less and able to focus on their patient. [28:24] focus on their patient. [28:24] focus on their patient. >> Yeah. I mean that's great because like [28:26] >> Yeah. I mean that's great because like [28:26] >> Yeah. I mean that's great because like you you know look at with an hour a day [28:28] you you know look at with an hour a day [28:28] you you know look at with an hour a day is either an hour they can spend with [28:29] is either an hour they can spend with [28:29] is either an hour they can spend with patients or just not have burnout and [28:31] patients or just not have burnout and [28:31] patients or just not have burnout and you know have more focus on that. You [28:33] you know have more focus on that. You [28:33] you know have more focus on that. You know that's we talk to before school.ai [28:35] know that's we talk to before school.ai [28:35] know that's we talk to before school.ai AI and looking at how they're basically [28:38] AI and looking at how they're basically [28:38] AI and looking at how they're basically able to help teachers spend more time in [28:39] able to help teachers spend more time in [28:39] able to help teachers spend more time in the classroom with students and that's [28:40] the classroom with students and that's [28:40] the classroom with students and that's the same problem you're dealing with [28:41] the same problem you're dealing with [28:41] the same problem you're dealing with here. How can doctors be there for [28:43] here. How can doctors be there for [28:43] here. How can doctors be there for students or student patients and also [28:46] students or student patients and also [28:46] students or student patients and also >> yeah well so we have a channel in Slack [28:48] >> yeah well so we have a channel in Slack [28:48] >> yeah well so we have a channel in Slack and it's what we call love stories and [28:50] and it's what we call love stories and [28:50] and it's what we call love stories and it's where we hear like uh kind of like [28:51] it's where we hear like uh kind of like [28:52] it's where we hear like uh kind of like all the feedback from the field from [28:53] all the feedback from the field from [28:53] all the feedback from the field from doctors and one of the one of the wild [28:55] doctors and one of the one of the wild [28:55] doctors and one of the one of the wild indicators that like we had landed on [28:56] indicators that like we had landed on [28:56] indicators that like we had landed on something big and this is relatively [28:58] something big and this is relatively [28:58] something big and this is relatively early um maybe like a couple months [29:00] early um maybe like a couple months [29:00] early um maybe like a couple months after we launched the first like [29:02] after we launched the first like [29:02] after we launched the first like enterprise pilot at a hospital system [29:04] enterprise pilot at a hospital system [29:04] enterprise pilot at a hospital system was when we started getting stories [29:05] was when we started getting stories [29:05] was when we started getting stories coming in and they weren't just talking [29:07] coming in and they weren't just talking [29:07] coming in and they weren't just talking about the clinical experience but they [29:09] about the clinical experience but they [29:09] about the clinical experience but they were talking about like uh I I spent and [29:11] were talking about like uh I I spent and [29:11] were talking about like uh I I spent and uh actually got to have dinner with my [29:13] uh actually got to have dinner with my [29:13] uh actually got to have dinner with my family every night this week for the [29:14] family every night this week for the [29:14] family every night this week for the first time in like 10 years or like a [29:17] first time in like 10 years or like a [29:17] first time in like 10 years or like a bridge is saving my marriage and that [29:18] bridge is saving my marriage and that [29:18] bridge is saving my marriage and that was that was not like where I was [29:20] was that was not like where I was [29:20] was that was not like where I was expecting things to land so quickly but [29:22] expecting things to land so quickly but [29:22] expecting things to land so quickly but uh you know that the problem is that [29:24] uh you know that the problem is that [29:24] uh you know that the problem is that big. [29:25] big. [29:25] big. >> What did you see today dev day that has [29:27] >> What did you see today dev day that has [29:27] >> What did you see today dev day that has you excited? [29:28] you excited? [29:28] you excited? >> Oh um so many things but maybe like two [29:31] >> Oh um so many things but maybe like two [29:31] >> Oh um so many things but maybe like two that are top of mind. Um one a lot of [29:33] that are top of mind. Um one a lot of [29:33] that are top of mind. Um one a lot of people have already talked about the [29:35] people have already talked about the [29:35] people have already talked about the agent developer kit [29:37] agent developer kit [29:37] agent developer kit >> and I think that's extremely exciting. [29:39] >> and I think that's extremely exciting. [29:39] >> and I think that's extremely exciting. There's this moment right now where I [29:41] There's this moment right now where I [29:41] There's this moment right now where I think everyone it's sort of a proto [29:43] think everyone it's sort of a proto [29:43] think everyone it's sort of a proto discipline. So everyone's been rolling [29:45] discipline. So everyone's been rolling [29:45] discipline. So everyone's been rolling their own tools in the hope of like kind [29:47] their own tools in the hope of like kind [29:47] their own tools in the hope of like kind of trying to figure out like what is the [29:48] of trying to figure out like what is the [29:48] of trying to figure out like what is the paradigm? How does this work? And [29:49] paradigm? How does this work? And [29:49] paradigm? How does this work? And there's so many things that have to work [29:50] there's so many things that have to work [29:50] there's so many things that have to work together. Um there's the context [29:53] together. Um there's the context [29:53] together. Um there's the context engineering, there's the prototyping, [29:54] engineering, there's the prototyping, [29:54] engineering, there's the prototyping, there's a sanenity checking, um there's [29:57] there's a sanenity checking, um there's [29:57] there's a sanenity checking, um there's evaluation, there's also down the road, [29:59] evaluation, there's also down the road, [29:59] evaluation, there's also down the road, you know, everything around production [30:00] you know, everything around production [30:00] you know, everything around production and monitoring. So, I'm really excited [30:02] and monitoring. So, I'm really excited [30:02] and monitoring. So, I'm really excited to see where these tools um ultimately [30:04] to see where these tools um ultimately [30:04] to see where these tools um ultimately go, but seeing OpenAI like take a strong [30:07] go, but seeing OpenAI like take a strong [30:07] go, but seeing OpenAI like take a strong position and put a a kind of [30:09] position and put a a kind of [30:09] position and put a a kind of comprehensive offering that brings [30:11] comprehensive offering that brings [30:11] comprehensive offering that brings together a lot of these things that you [30:12] together a lot of these things that you [30:12] together a lot of these things that you know people have been rolling their own [30:14] know people have been rolling their own [30:14] know people have been rolling their own orchestration tools, their own [30:15] orchestration tools, their own [30:15] orchestration tools, their own evaluation platforms, we certainly have [30:18] evaluation platforms, we certainly have [30:18] evaluation platforms, we certainly have um and seeing how much how much this is [30:20] um and seeing how much how much this is [30:20] um and seeing how much how much this is going to create a common platform and [30:21] going to create a common platform and [30:22] going to create a common platform and allow us to sort of lift off and focus [30:24] allow us to sort of lift off and focus [30:24] allow us to sort of lift off and focus more on the content um is something I'm [30:26] more on the content um is something I'm [30:26] more on the content um is something I'm super excited about. Um, and then in [30:29] super excited about. Um, and then in [30:29] super excited about. Um, and then in general, I've just been extremely [30:30] general, I've just been extremely [30:30] general, I've just been extremely excited and drawn a lot of inspiration [30:32] excited and drawn a lot of inspiration [30:32] excited and drawn a lot of inspiration from all the work that's gone on in [30:35] from all the work that's gone on in [30:35] from all the work that's gone on in terms of software developer tooling. And [30:37] terms of software developer tooling. And [30:37] terms of software developer tooling. And so, we are we are developing AI powered [30:39] so, we are we are developing AI powered [30:39] so, we are we are developing AI powered products for our customers, but we're [30:40] products for our customers, but we're [30:40] products for our customers, but we're also big consumers as uh, you know, [30:42] also big consumers as uh, you know, [30:42] also big consumers as uh, you know, productivity tools like codeex um, um, [30:45] productivity tools like codeex um, um, [30:45] productivity tools like codeex um, um, play a big role for us and just seeing [30:48] play a big role for us and just seeing [30:48] play a big role for us and just seeing how far we've come there. I mean, I [30:50] how far we've come there. I mean, I [30:50] how far we've come there. I mean, I remember um, so my background is [30:52] remember um, so my background is [30:52] remember um, so my background is academia. I was AI researcher. I am an [30:54] academia. I was AI researcher. I am an [30:54] academia. I was AI researcher. I am an researcher, a professor at Carnegie [30:56] researcher, a professor at Carnegie [30:56] researcher, a professor at Carnegie Melon and I did my PhD back in uh 2010s [30:59] Melon and I did my PhD back in uh 2010s [30:59] Melon and I did my PhD back in uh 2010s and I remember when Ilia had a paper, [31:02] and I remember when Ilia had a paper, [31:02] and I remember when Ilia had a paper, Iliach had a paper called learning to [31:04] Iliach had a paper called learning to [31:04] Iliach had a paper called learning to execute and it was just having like you [31:06] execute and it was just having like you [31:06] execute and it was just having like you know the idea was like code going in and [31:08] know the idea was like code going in and [31:08] know the idea was like code going in and like a model kind of anticipating output [31:11] like a model kind of anticipating output [31:11] like a model kind of anticipating output was like such a the fact that the model [31:13] was like such a the fact that the model [31:13] was like such a the fact that the model is doing anything at all in the space of [31:14] is doing anything at all in the space of [31:14] is doing anything at all in the space of code was kind of revolutionary. and to [31:17] code was kind of revolutionary. and to [31:17] code was kind of revolutionary. and to to see where we are now going from like [31:19] to see where we are now going from like [31:19] to see where we are now going from like maybe two years ago having our minds [31:21] maybe two years ago having our minds [31:21] maybe two years ago having our minds blown by just code completion and right [31:23] blown by just code completion and right [31:23] blown by just code completion and right now seeing these like larger like full [31:26] now seeing these like larger like full [31:26] now seeing these like larger like full codebased refactors taking place um you [31:30] codebased refactors taking place um you [31:30] codebased refactors taking place um you know it's it's kind of amazing to see [31:32] know it's it's kind of amazing to see [31:32] know it's it's kind of amazing to see the progress in the space and um I've [31:34] the progress in the space and um I've [31:34] the progress in the space and um I've been excited to to follow along. So when [31:36] been excited to to follow along. So when [31:36] been excited to to follow along. So when you work in medical it's a very high [31:38] you work in medical it's a very high [31:38] you work in medical it's a very high stakes area and so that's got to be [31:39] stakes area and so that's got to be [31:40] stakes area and so that's got to be something you think a lot about about [31:41] something you think a lot about about [31:41] something you think a lot about about how you deal with hallucination and also [31:43] how you deal with hallucination and also [31:43] how you deal with hallucination and also how you deal with customer concerns [31:45] how you deal with customer concerns [31:45] how you deal with customer concerns about these things [31:46] about these things [31:46] about these things >> 100%. [31:46] >> 100%. [31:46] >> 100%. >> So what do you look forward to in tools? [31:48] >> So what do you look forward to in tools? [31:48] >> So what do you look forward to in tools? What have you seen the biggest help in [31:49] What have you seen the biggest help in [31:50] What have you seen the biggest help in that area? [31:50] that area? [31:50] that area? >> Um that's an area where we've had to [31:52] >> Um that's an area where we've had to [31:52] >> Um that's an area where we've had to develop a lot of our own technology. Um [31:55] develop a lot of our own technology. Um [31:55] develop a lot of our own technology. Um >> you know what is a hallucination? It's [31:57] >> you know what is a hallucination? It's [31:57] >> you know what is a hallucination? It's kind of like like back in the old days [31:58] kind of like like back in the old days [31:58] kind of like like back in the old days of like developing simple classifiers. [32:00] of like developing simple classifiers. [32:00] of like developing simple classifiers. We had we had false negatives and false [32:01] We had we had false negatives and false [32:02] We had we had false negatives and false positives and now kind of like [32:03] positives and now kind of like [32:03] positives and now kind of like everything that's like if it's there and [32:05] everything that's like if it's there and [32:05] everything that's like if it's there and you don't want it, it's a hallucination. [32:06] you don't want it, it's a hallucination. [32:06] you don't want it, it's a hallucination. And the question is, well, what is a [32:07] And the question is, well, what is a [32:07] And the question is, well, what is a hallucination? Sometimes it's [32:09] hallucination? Sometimes it's [32:09] hallucination? Sometimes it's >> completely confabulated information that [32:11] >> completely confabulated information that [32:11] >> completely confabulated information that is asserting facts about the world that [32:13] is asserting facts about the world that [32:13] is asserting facts about the world that are not real. But in the context of [32:15] are not real. But in the context of [32:15] are not real. But in the context of medical note-taking, medical [32:16] medical note-taking, medical [32:16] medical note-taking, medical documentation, [32:18] documentation, [32:18] documentation, order placement, um it's there there's a [32:21] order placement, um it's there there's a [32:21] order placement, um it's there there's a kind of particular situated notion of [32:23] kind of particular situated notion of [32:23] kind of particular situated notion of what we really mean. It's something [32:25] what we really mean. It's something [32:25] what we really mean. It's something that's sort of unlicensed, you know, by [32:27] that's sort of unlicensed, you know, by [32:27] that's sort of unlicensed, you know, by the sort of surrounding context, the [32:30] the sort of surrounding context, the [32:30] the sort of surrounding context, the substantiating evidence, even if it [32:32] substantiating evidence, even if it [32:32] substantiating evidence, even if it might be true, you know, if uh if a [32:35] might be true, you know, if uh if a [32:35] might be true, you know, if uh if a doctor doesn't um if a kind of [32:37] doctor doesn't um if a kind of [32:37] doctor doesn't um if a kind of explanation of a disease like shows up [32:40] explanation of a disease like shows up [32:40] explanation of a disease like shows up in a generated patientf facing summary [32:43] in a generated patientf facing summary [32:43] in a generated patientf facing summary that the doctor never said. [32:46] that the doctor never said. [32:46] that the doctor never said. >> Yeah. you know, that's that's kind of [32:47] >> Yeah. you know, that's that's kind of [32:48] >> Yeah. you know, that's that's kind of like even if some of the information [32:49] like even if some of the information [32:50] like even if some of the information might be like either correct or [32:51] might be like either correct or [32:51] might be like either correct or plausible, like that's not within our so [32:54] plausible, like that's not within our so [32:54] plausible, like that's not within our so we have a kind of bespoke notion of what [32:55] we have a kind of bespoke notion of what [32:56] we have a kind of bespoke notion of what constitutes a hallucination. [32:57] constitutes a hallucination. [32:57] constitutes a hallucination. >> Um, but we able to like kind of [33:00] >> Um, but we able to like kind of [33:00] >> Um, but we able to like kind of >> often draw a lot of inspiration from [33:02] >> often draw a lot of inspiration from [33:02] >> often draw a lot of inspiration from what can like the frontier models [33:03] what can like the frontier models [33:03] what can like the frontier models already do out of the box. If we define [33:06] already do out of the box. If we define [33:06] already do out of the box. If we define our ontology of like these are the types [33:09] our ontology of like these are the types [33:09] our ontology of like these are the types of of errors we're concerned with, then [33:11] of of errors we're concerned with, then [33:11] of of errors we're concerned with, then we can go and say what is the ability of [33:13] we can go and say what is the ability of [33:13] we can go and say what is the ability of an out-of-the-box model given like each [33:15] an out-of-the-box model given like each [33:16] an out-of-the-box model given like each each documentation sort of sentence [33:18] each documentation sort of sentence [33:18] each documentation sort of sentence allocart to correctly designate them as [33:21] allocart to correctly designate them as [33:21] allocart to correctly designate them as belonging to the right category. And we [33:22] belonging to the right category. And we [33:22] belonging to the right category. And we find okay, we're already within the [33:24] find okay, we're already within the [33:24] find okay, we're already within the realm of like the model is able to judge [33:26] realm of like the model is able to judge [33:26] realm of like the model is able to judge even if even if it's not able to sort of [33:29] even if even if it's not able to sort of [33:29] even if even if it's not able to sort of never commit the crime in the first [33:30] never commit the crime in the first [33:30] never commit the crime in the first place, it's able to recognize when the [33:32] place, it's able to recognize when the [33:32] place, it's able to recognize when the crime's been committed. That gives us a [33:33] crime's been committed. That gives us a [33:33] crime's been committed. That gives us a sort of like proof of concept and now [33:35] sort of like proof of concept and now [33:35] sort of like proof of concept and now what we need to do is make it better, [33:37] what we need to do is make it better, [33:37] what we need to do is make it better, more accurate, cheaper, faster. So [33:39] more accurate, cheaper, faster. So [33:39] more accurate, cheaper, faster. So ultimately create our own special [33:41] ultimately create our own special [33:41] ultimately create our own special purpose models that are able to take in [33:43] purpose models that are able to take in [33:43] purpose models that are able to take in parallel every single sentence in all [33:45] parallel every single sentence in all [33:45] parallel every single sentence in all the generated documentation and [33:47] the generated documentation and [33:47] the generated documentation and surrounding artifacts and process for [33:49] surrounding artifacts and process for [33:49] surrounding artifacts and process for each one like sort of does it contain an [33:52] each one like sort of does it contain an [33:52] each one like sort of does it contain an error of of a unacceptable variety like [33:55] error of of a unacceptable variety like [33:55] error of of a unacceptable variety like of of what kind and then a kind of [33:57] of of what kind and then a kind of [33:58] of of what kind and then a kind of pipeline downstream for remediating. and [33:59] pipeline downstream for remediating. and [33:59] pipeline downstream for remediating. and we're able to do that um with about 97% [34:02] we're able to do that um with about 97% [34:02] we're able to do that um with about 97% recall at this point. [34:04] recall at this point. [34:04] recall at this point. >> So, do you have any advice for people [34:06] >> So, do you have any advice for people [34:06] >> So, do you have any advice for people who are trying to work on basically [34:07] who are trying to work on basically [34:07] who are trying to work on basically developing their own evals for [34:09] developing their own evals for [34:09] developing their own evals for hallucination or just sort of a good [34:10] hallucination or just sort of a good [34:10] hallucination or just sort of a good starting point? I [34:11] starting point? I [34:11] starting point? I >> I think it all starts with getting [34:12] >> I think it all starts with getting [34:12] >> I think it all starts with getting really crisp about what you really mean. [34:15] really crisp about what you really mean. [34:15] really crisp about what you really mean. And I think that's what we've seen [34:16] And I think that's what we've seen [34:16] And I think that's what we've seen before is that what is a hallucination [34:18] before is that what is a hallucination [34:18] before is that what is a hallucination for us is a little bit different from [34:20] for us is a little bit different from [34:20] for us is a little bit different from what is a hallucination for like a [34:22] what is a hallucination for like a [34:22] what is a hallucination for like a general um sort of like open world QA [34:25] general um sort of like open world QA [34:25] general um sort of like open world QA system. Mhm. So is there kind of like a [34:29] system. Mhm. So is there kind of like a [34:29] system. Mhm. So is there kind of like a uh boundary at which you keep expanding? [34:31] uh boundary at which you keep expanding? [34:31] uh boundary at which you keep expanding? For instance, you talk about medical [34:32] For instance, you talk about medical [34:32] For instance, you talk about medical say, okay, we feel maybe working in [34:34] say, okay, we feel maybe working in [34:34] say, okay, we feel maybe working in scribing right now is is an area that we [34:36] scribing right now is is an area that we [34:36] scribing right now is is an area that we can probably solve for and produce a [34:37] can probably solve for and produce a [34:37] can probably solve for and produce a pretty good product that's at or better [34:39] pretty good product that's at or better [34:39] pretty good product that's at or better than human level. Then do you look at [34:40] than human level. Then do you look at [34:40] than human level. Then do you look at like there's areas in which you would [34:42] like there's areas in which you would [34:42] like there's areas in which you would expand out to as you feel more [34:43] expand out to as you feel more [34:43] expand out to as you feel more confident? [34:44] confident? [34:44] confident? >> Absolutely. So for us, um, we've always [34:47] >> Absolutely. So for us, um, we've always [34:47] >> Absolutely. So for us, um, we've always bristled a little bit when, you know, [34:48] bristled a little bit when, you know, [34:48] bristled a little bit when, you know, like VCs put up this chart and they're [34:50] like VCs put up this chart and they're [34:50] like VCs put up this chart and they're like, "These guys over here are the the [34:51] like, "These guys over here are the the [34:51] like, "These guys over here are the the conversation agents and these guys here [34:53] conversation agents and these guys here [34:53] conversation agents and these guys here do are the coding startups and these [34:54] do are the coding startups and these [34:54] do are the coding startups and these guys are the scribing companies." And we [34:57] guys are the scribing companies." And we [34:57] guys are the scribing companies." And we we've never liked getting pigeonholed as [34:59] we've never liked getting pigeonholed as [34:59] we've never liked getting pigeonholed as a scribing company. That's because from [35:00] a scribing company. That's because from [35:00] a scribing company. That's because from the very outset [35:02] the very outset [35:02] the very outset >> um the central thesis wasn't just about [35:03] >> um the central thesis wasn't just about [35:03] >> um the central thesis wasn't just about scribbing. The central thesis was sort [35:05] scribbing. The central thesis was sort [35:05] scribbing. The central thesis was sort of [35:05] of [35:05] of >> um about medical conversations about [35:08] >> um about medical conversations about [35:08] >> um about medical conversations about this being this moment where um you know [35:11] this being this moment where um you know [35:11] this being this moment where um you know this is this is this magical spot. It's [35:13] this is this is this magical spot. It's [35:13] this is this is this magical spot. It's these 15 minutes that the patient waited [35:15] these 15 minutes that the patient waited [35:15] these 15 minutes that the patient waited maybe six months for are where the [35:16] maybe six months for are where the [35:16] maybe six months for are where the patient tells their entire story where [35:18] patient tells their entire story where [35:18] patient tells their entire story where the doctor goes through their entire [35:20] the doctor goes through their entire [35:20] the doctor goes through their entire reasoning process and within minutes [35:22] reasoning process and within minutes [35:22] reasoning process and within minutes after it's over the patient's forgotten [35:24] after it's over the patient's forgotten [35:24] after it's over the patient's forgotten 80% of what happened the doctor is like [35:26] 80% of what happened the doctor is like [35:26] 80% of what happened the doctor is like you know hours behind on their [35:27] you know hours behind on their [35:27] you know hours behind on their note-taking and so for us we've had this [35:30] note-taking and so for us we've had this [35:30] note-taking and so for us we've had this feeling that you know and we I've also [35:33] feeling that you know and we I've also [35:33] feeling that you know and we I've also like as an academic been working in [35:34] like as an academic been working in [35:34] like as an academic been working in actually applications in healthcare is [35:36] actually applications in healthcare is [35:36] actually applications in healthcare is like my kind of passion area for about a [35:39] like my kind of passion area for about a [35:39] like my kind of passion area for about a decade and I've been watching so many [35:40] decade and I've been watching so many [35:40] decade and I've been watching so many interesting machine learning ideas get [35:42] interesting machine learning ideas get [35:42] interesting machine learning ideas get developed as a proof of concept only to [35:45] developed as a proof of concept only to [35:45] developed as a proof of concept only to only to kind of sit on the floor and not [35:47] only to kind of sit on the floor and not [35:47] only to kind of sit on the floor and not get used. And so what we realized that [35:49] get used. And so what we realized that [35:50] get used. And so what we realized that the conversation was was this weigh in. [35:52] the conversation was was this weigh in. [35:52] the conversation was was this weigh in. It was this important arena where it was [35:54] It was this important arena where it was [35:54] It was this important arena where it was in some ways it's the most important [35:56] in some ways it's the most important [35:56] in some ways it's the most important moment in the entire experience of [35:57] moment in the entire experience of [35:57] moment in the entire experience of healthcare and sort of no one was [35:59] healthcare and sort of no one was [35:59] healthcare and sort of no one was providing value in that moment. And [36:01] providing value in that moment. And [36:01] providing value in that moment. And scribing we already knew was going to be [36:03] scribing we already knew was going to be [36:03] scribing we already knew was going to be a killer application because you know it [36:05] a killer application because you know it [36:05] a killer application because you know it was never going to scale. human scribing [36:07] was never going to scale. human scribing [36:07] was never going to scale. human scribing was never going to scale to all doctors. [36:09] was never going to scale to all doctors. [36:09] was never going to scale to all doctors. But those who could afford it were [36:10] But those who could afford it were [36:10] But those who could afford it were willing to pay tens of thousands of [36:12] willing to pay tens of thousands of [36:12] willing to pay tens of thousands of dollars per doctor per year to have a [36:14] dollars per doctor per year to have a [36:14] dollars per doctor per year to have a sort of like offshore scribe. And so [36:16] sort of like offshore scribe. And so [36:16] sort of like offshore scribe. And so that already kind of told us like there [36:18] that already kind of told us like there [36:18] that already kind of told us like there was this clamor for it. There was this [36:19] was this clamor for it. There was this [36:20] was this clamor for it. There was this need for it, but we could get in. But [36:21] need for it, but we could get in. But [36:21] need for it, but we could get in. But now that we're in um I I I'd view like [36:25] now that we're in um I I I'd view like [36:25] now that we're in um I I I'd view like take this broader view of like what what [36:27] take this broader view of like what what [36:27] take this broader view of like what what is like the entire picture rather than [36:29] is like the entire picture rather than [36:29] is like the entire picture rather than sort of being like we're we're quietly [36:30] sort of being like we're we're quietly [36:30] sort of being like we're we're quietly in the background just like minding our [36:32] in the background just like minding our [36:32] in the background just like minding our own business and then at the very end of [36:33] own business and then at the very end of [36:33] own business and then at the very end of the visit, boom, the magic happens. We [36:36] the visit, boom, the magic happens. We [36:36] the visit, boom, the magic happens. We don't want to go so far in the other [36:38] don't want to go so far in the other [36:38] don't want to go so far in the other direction that we become interruptive. [36:39] direction that we become interruptive. [36:39] direction that we become interruptive. But from a p perspective of just sort of [36:41] But from a p perspective of just sort of [36:41] But from a p perspective of just sort of like how do we support a doctor for the [36:43] like how do we support a doctor for the [36:44] like how do we support a doctor for the entirety of the visit from sort of their [36:46] entirety of the visit from sort of their [36:46] entirety of the visit from sort of their pre-charting experience, you know, [36:47] pre-charting experience, you know, [36:48] pre-charting experience, you know, before the patient even comes in the [36:49] before the patient even comes in the [36:49] before the patient even comes in the room through sort of every kind of cue [36:51] room through sort of every kind of cue [36:51] room through sort of every kind of cue or nudge that they might need during the [36:53] or nudge that they might need during the [36:53] or nudge that they might need during the visit to help them make the best [36:55] visit to help them make the best [36:55] visit to help them make the best possible decisions, help them uh tick [36:58] possible decisions, help them uh tick [36:58] possible decisions, help them uh tick all the boxes to make sure that like [37:00] all the boxes to make sure that like [37:00] all the boxes to make sure that like insurance is going to preapprove the [37:02] insurance is going to preapprove the [37:02] insurance is going to preapprove the particular test or treatment that [37:03] particular test or treatment that [37:03] particular test or treatment that they're going to do. So patient end [37:05] they're going to do. So patient end [37:05] they're going to do. So patient end doesn't end up wait wasting like uh you [37:07] doesn't end up wait wasting like uh you [37:07] doesn't end up wait wasting like uh you know a month waiting for care. Um, so we [37:11] know a month waiting for care. Um, so we [37:11] know a month waiting for care. Um, so we kind of like zoom out. we kind of see [37:13] kind of like zoom out. we kind of see [37:13] kind of like zoom out. we kind of see like this space of the conversation like [37:15] like this space of the conversation like [37:15] like this space of the conversation like the point of care is like now that we [37:16] the point of care is like now that we [37:16] the point of care is like now that we have ears in the visit and now that you [37:18] have ears in the visit and now that you [37:18] have ears in the visit and now that you have a sort of AI workforce to sort of [37:20] have a sort of AI workforce to sort of [37:20] have a sort of AI workforce to sort of do your bidding um what what are all [37:23] do your bidding um what what are all [37:23] do your bidding um what what are all these other jobs to be done that could [37:24] these other jobs to be done that could [37:24] these other jobs to be done that could be addressed in the moment and that [37:26] be addressed in the moment and that [37:26] be addressed in the moment and that includes everything from from from the [37:29] includes everything from from from the [37:29] includes everything from from from the sort of previsit experience through to [37:32] sort of previsit experience through to [37:32] sort of previsit experience through to real-time clinical decision support [37:34] real-time clinical decision support [37:34] real-time clinical decision support through all the kind of um anticipating [37:37] through all the kind of um anticipating [37:37] through all the kind of um anticipating and getting in front of all of the uh [37:40] and getting in front of all of the uh [37:40] and getting in front of all of the uh sort of like financial related ated [37:42] sort of like financial related ated [37:42] sort of like financial related ated documentation that needs to be done to [37:43] documentation that needs to be done to [37:44] documentation that needs to be done to ensure that that the doctor gets paid [37:45] ensure that that the doctor gets paid [37:45] ensure that that the doctor gets paid and that the patient gets their care in [37:47] and that the patient gets their care in [37:47] and that the patient gets their care in a timely fashion. [37:48] a timely fashion. [37:48] a timely fashion. >> Yeah. You know, on one end you have [37:49] >> Yeah. You know, on one end you have [37:49] >> Yeah. You know, on one end you have people working on, you know, the AI [37:52] people working on, you know, the AI [37:52] people working on, you know, the AI scientists and tools and trying to solve [37:53] scientists and tools and trying to solve [37:53] scientists and tools and trying to solve frontier problems. I've talked to other [37:55] frontier problems. I've talked to other [37:55] frontier problems. I've talked to other people who have told me that if you can [37:56] people who have told me that if you can [37:56] people who have told me that if you can have better intake in hospitals, you [37:58] have better intake in hospitals, you [37:58] have better intake in hospitals, you might be able to get rid of hepatitis [37:59] might be able to get rid of hepatitis [37:59] might be able to get rid of hepatitis that there's some very lowhanging fruits [38:01] that there's some very lowhanging fruits [38:01] that there's some very lowhanging fruits there. And is that something you've [38:02] there. And is that something you've [38:02] there. And is that something you've looked into or you sort of see a lot of [38:04] looked into or you sort of see a lot of [38:04] looked into or you sort of see a lot of opportunity there? [38:05] opportunity there? [38:05] opportunity there? >> Absolutely. [38:06] >> Absolutely. [38:06] >> Absolutely. >> Any particular area you'd like to see [38:08] >> Any particular area you'd like to see [38:08] >> Any particular area you'd like to see the tools get better sooner at? Oh. Um, [38:13] the tools get better sooner at? Oh. Um, [38:14] the tools get better sooner at? Oh. Um, so many. Uh, I'd say on a personal note, [38:17] so many. Uh, I'd say on a personal note, [38:17] so many. Uh, I'd say on a personal note, they're not very funny yet. [38:19] they're not very funny yet. [38:19] they're not very funny yet. >> Okay. [38:19] >> Okay. [38:19] >> Okay. >> So, I don't know if you've had this [38:20] >> So, I don't know if you've had this [38:20] >> So, I don't know if you've had this experience at Chat DVT that it's, uh, [38:22] experience at Chat DVT that it's, uh, [38:22] experience at Chat DVT that it's, uh, it's far better at like solving hard [38:24] it's far better at like solving hard [38:24] it's far better at like solving hard math problems than it is making a joke. [38:25] math problems than it is making a joke. [38:26] math problems than it is making a joke. >> Uh, Sora though is very good at jokes. I [38:28] >> Uh, Sora though is very good at jokes. I [38:28] >> Uh, Sora though is very good at jokes. I don't know if you, you know, tried that [38:29] don't know if you, you know, tried that [38:29] don't know if you, you know, tried that yet, but [38:30] yet, but [38:30] yet, but >> yeah. Um I think there's a tremendous [38:34] >> yeah. Um I think there's a tremendous [38:34] >> yeah. Um I think there's a tremendous amount of work that one still has to do [38:37] amount of work that one still has to do [38:37] amount of work that one still has to do to like um crisply define every single [38:42] to like um crisply define every single [38:42] to like um crisply define every single task for the model. And I think that [38:44] task for the model. And I think that [38:44] task for the model. And I think that like just how high models can come and I [38:49] like just how high models can come and I [38:49] like just how high models can come and I think you know like they're when it [38:50] think you know like they're when it [38:50] think you know like they're when it comes to like a crisply defined [38:51] comes to like a crisply defined [38:51] comes to like a crisply defined technical task they're they've gotten [38:54] technical task they're they've gotten [38:54] technical task they're they've gotten very high in the abstraction chain about [38:56] very high in the abstraction chain about [38:56] very high in the abstraction chain about breaking it down. Um but I think when [38:59] breaking it down. Um but I think when [38:59] breaking it down. Um but I think when you get outside the you know and start [39:02] you get outside the you know and start [39:02] you get outside the you know and start like addressing problems at like [39:04] like addressing problems at like [39:04] like addressing problems at like interacting with the system kind of like [39:06] interacting with the system kind of like [39:06] interacting with the system kind of like tackling the like the the the more like [39:10] tackling the like the the the more like [39:10] tackling the like the the the more like world problem you're you're discussing [39:11] world problem you're you're discussing [39:11] world problem you're you're discussing you find that like you kind of have to [39:13] you find that like you kind of have to [39:13] you find that like you kind of have to do all the driving and the system is a [39:15] do all the driving and the system is a [39:15] do all the driving and the system is a little bit more of a it's an information [39:17] little bit more of a it's an information [39:18] little bit more of a it's an information retrieval system. It is it is the [39:20] retrieval system. It is it is the [39:20] retrieval system. It is it is the world's knowledge at your fingertips but [39:22] world's knowledge at your fingertips but [39:22] world's knowledge at your fingertips but it is not kind of connecting dots at a [39:25] it is not kind of connecting dots at a [39:25] it is not kind of connecting dots at a more abstract level. And so I think you [39:27] more abstract level. And so I think you [39:28] more abstract level. And so I think you know in the in the coming you know [39:29] know in the in the coming you know [39:29] know in the in the coming you know months and years I'm excited to see uh [39:32] months and years I'm excited to see uh [39:32] months and years I'm excited to see uh to what extent does a model go from um [39:35] to what extent does a model go from um [39:35] to what extent does a model go from um more technical problem solver to uh a [39:40] more technical problem solver to uh a [39:40] more technical problem solver to uh a more independent interlocutor and the [39:42] more independent interlocutor and the [39:42] more independent interlocutor and the like normative side of problem solving. [39:44] like normative side of problem solving. [39:44] like normative side of problem solving. >> Going back how did you all decide this [39:46] >> Going back how did you all decide this [39:46] >> Going back how did you all decide this was the space you wanted to start with? [39:50] was the space you wanted to start with? [39:50] was the space you wanted to start with? >> Um you know I think we saw a few trends [39:53] >> Um you know I think we saw a few trends [39:53] >> Um you know I think we saw a few trends that were happening all at once. um at [39:56] that were happening all at once. um at [39:56] that were happening all at once. um at once like my research background was in [39:59] once like my research background was in [39:59] once like my research background was in deep learning um and any one given [40:03] deep learning um and any one given [40:03] deep learning um and any one given approach we kept running into a little [40:05] approach we kept running into a little [40:05] approach we kept running into a little plateaus here and there but if you zoom [40:07] plateaus here and there but if you zoom [40:07] plateaus here and there but if you zoom back and like look at the arc from 2012 [40:10] back and like look at the arc from 2012 [40:10] back and like look at the arc from 2012 through to you know maybe 2018 19 when [40:13] through to you know maybe 2018 19 when [40:13] through to you know maybe 2018 19 when the company was founded you saw there [40:15] the company was founded you saw there [40:15] the company was founded you saw there was there was a there was a a path of [40:17] was there was a there was a a path of [40:17] was there was a there was a a path of advances in speech recognition and path [40:19] advances in speech recognition and path [40:19] advances in speech recognition and path and advances in natural language [40:21] and advances in natural language [40:21] and advances in natural language processing that was you know preceding [40:23] processing that was you know preceding [40:23] processing that was you know preceding if anything accelerating [40:25] if anything accelerating [40:25] if anything accelerating And simultaneously [40:28] And simultaneously [40:28] And simultaneously um there was a crisis around physician [40:30] um there was a crisis around physician [40:30] um there was a crisis around physician burnout that maybe in 2018 wasn't the [40:34] burnout that maybe in 2018 wasn't the [40:34] burnout that maybe in 2018 wasn't the like number one burning priority on the [40:37] like number one burning priority on the [40:37] like number one burning priority on the minds of like CMIOS and like hospital [40:39] minds of like CMIOS and like hospital [40:39] minds of like CMIOS and like hospital system CFOs across the country but it [40:42] system CFOs across the country but it [40:42] system CFOs across the country but it was you know it might have been number [40:43] was you know it might have been number [40:44] was you know it might have been number five on their priorities and it was like [40:46] five on their priorities and it was like [40:46] five on their priorities and it was like rising up the charts and so we kind of [40:49] rising up the charts and so we kind of [40:49] rising up the charts and so we kind of knew there was this coming crisis of [40:50] knew there was this coming crisis of [40:50] knew there was this coming crisis of like doctors were burning out they were [40:52] like doctors were burning out they were [40:52] like doctors were burning out they were spending more and more time on [40:53] spending more and more time on [40:53] spending more and more time on documentation [40:55] documentation [40:55] documentation They were dropping out of med school. [40:56] They were dropping out of med school. [40:56] They were dropping out of med school. They were graduating med school with no [40:58] They were graduating med school with no [40:58] They were graduating med school with no intention to practice medicine, leaving [40:59] intention to practice medicine, leaving [40:59] intention to practice medicine, leaving to join to join tech companies, to join [41:01] to join to join tech companies, to join [41:01] to join to join tech companies, to join pharma, but to do anything but practice. [41:03] pharma, but to do anything but practice. [41:04] pharma, but to do anything but practice. Um, and so there was a turn problem or [41:06] Um, and so there was a turn problem or [41:06] Um, and so there was a turn problem or attention problem. And at the same time, [41:08] attention problem. And at the same time, [41:08] attention problem. And at the same time, like we we we kind of knew that there [41:10] like we we we kind of knew that there [41:10] like we we we kind of knew that there was like the the right family of tools [41:13] was like the the right family of tools [41:13] was like the the right family of tools were coming into fruition [41:14] were coming into fruition [41:14] were coming into fruition simultaneously. [41:15] simultaneously. [41:16] simultaneously. Um and you know for us uh I think me [41:21] Um and you know for us uh I think me [41:21] Um and you know for us uh I think me coming from um I was saying before this [41:25] coming from um I was saying before this [41:25] coming from um I was saying before this kind of academic perspective having you [41:27] kind of academic perspective having you [41:27] kind of academic perspective having you know a lot of us before had maybe [41:29] know a lot of us before had maybe [41:29] know a lot of us before had maybe operated in the machine learning for [41:31] operated in the machine learning for [41:31] operated in the machine learning for healthcare community a little bit on [41:32] healthcare community a little bit on [41:32] healthcare community a little bit on like [41:33] like [41:33] like >> what feels like a cool or important [41:34] >> what feels like a cool or important [41:34] >> what feels like a cool or important predictive problem but without maybe [41:37] predictive problem but without maybe [41:37] predictive problem but without maybe connecting all the dots when it came to [41:39] connecting all the dots when it came to [41:39] connecting all the dots when it came to like what were the priorities of the [41:41] like what were the priorities of the [41:41] like what were the priorities of the health system, what were the pain points [41:42] health system, what were the pain points [41:42] health system, what were the pain points of the health system, what were really [41:44] of the health system, what were really [41:44] of the health system, what were really like the choke points in [41:45] like the choke points in [41:45] like the choke points in maybe coming a little bit more from like [41:47] maybe coming a little bit more from like [41:47] maybe coming a little bit more from like what seems like an interesting clinical [41:49] what seems like an interesting clinical [41:49] what seems like an interesting clinical predictive problem. And I think at that [41:52] predictive problem. And I think at that [41:52] predictive problem. And I think at that moment we had a little bit of flash of [41:53] moment we had a little bit of flash of [41:53] moment we had a little bit of flash of insight that you know we didn't know if [41:55] insight that you know we didn't know if [41:55] insight that you know we didn't know if our timing would be right. Keep in mind [41:56] our timing would be right. Keep in mind [41:56] our timing would be right. Keep in mind that in 2018 like the typical like [41:58] that in 2018 like the typical like [41:58] that in 2018 like the typical like context length for a language model was [42:00] context length for a language model was [42:00] context length for a language model was maybe [42:01] maybe [42:01] maybe >> I don't know 256 words and these [42:03] >> I don't know 256 words and these [42:03] >> I don't know 256 words and these conversations are like 4 to 8,000 words [42:05] conversations are like 4 to 8,000 words [42:05] conversations are like 4 to 8,000 words like a median. Um but uh I think we just [42:09] like a median. Um but uh I think we just [42:09] like a median. Um but uh I think we just saw a lot of those convergence of like a [42:12] saw a lot of those convergence of like a [42:12] saw a lot of those convergence of like a few trends that that all spell that [42:13] few trends that that all spell that [42:13] few trends that that all spell that there was there there was this real [42:15] there was there there was this real [42:15] there was there there was this real opportunity to um to save time for [42:18] opportunity to um to save time for [42:18] opportunity to um to save time for doctors and ultimately hopefully you [42:21] doctors and ultimately hopefully you [42:21] doctors and ultimately hopefully you know um save money and also save lives. [42:23] know um save money and also save lives. [42:23] know um save money and also save lives. So in an area like medicine which is [42:25] So in an area like medicine which is [42:26] So in an area like medicine which is very high stakes what advice do you have [42:27] very high stakes what advice do you have [42:27] very high stakes what advice do you have for developers that are trying to build [42:29] for developers that are trying to build [42:29] for developers that are trying to build trust with their product because as you [42:30] trust with their product because as you [42:30] trust with their product because as you know probably better at anybody that [42:32] know probably better at anybody that [42:32] know probably better at anybody that there's been kind of a a road of broken [42:35] there's been kind of a a road of broken [42:35] there's been kind of a a road of broken promises of people who were very [42:36] promises of people who were very [42:36] promises of people who were very frustrated by you know oh this is going [42:38] frustrated by you know oh this is going [42:38] frustrated by you know oh this is going to do this this didn't work and when you [42:39] to do this this didn't work and when you [42:39] to do this this didn't work and when you come in with something that says hey it [42:41] come in with something that says hey it [42:41] come in with something that says hey it really works how do you win them over [42:46] really works how do you win them over [42:46] really works how do you win them over um [42:52] I think it's uh it's it's it's a [42:52] I think it's uh it's it's it's a neverending I I think trust is trust is [42:54] neverending I I think trust is trust is [42:54] neverending I I think trust is trust is earned every single day. Trust is [42:55] earned every single day. Trust is [42:55] earned every single day. Trust is earned. I mean there's [42:57] earned. I mean there's [42:57] earned. I mean there's >> there's like the initial trust of like [42:59] >> there's like the initial trust of like [42:59] >> there's like the initial trust of like we've been talking about this vision for [43:00] we've been talking about this vision for [43:00] we've been talking about this vision for a long time and then we actually built [43:02] a long time and then we actually built [43:02] a long time and then we actually built the product and got it to work. But [43:04] the product and got it to work. But [43:04] the product and got it to work. But there's also the trust that's built [43:06] there's also the trust that's built [43:06] there's also the trust that's built through, you know, um, working with [43:08] through, you know, um, working with [43:08] through, you know, um, working with hospital systems is kind of like a [43:10] hospital systems is kind of like a [43:10] hospital systems is kind of like a hightouch like white glove enterprise. [43:12] hightouch like white glove enterprise. [43:12] hightouch like white glove enterprise. And I think you know through continued [43:16] And I think you know through continued [43:16] And I think you know through continued um delivery on everything from like our [43:19] um delivery on everything from like our [43:19] um delivery on everything from like our product commitments, our data security [43:20] product commitments, our data security [43:20] product commitments, our data security commitments, um you know the the kind of [43:23] commitments, um you know the the kind of [43:23] commitments, um you know the the kind of service we give people um the the [43:26] service we give people um the the [43:26] service we give people um the the continued fulfillment of of of every [43:29] continued fulfillment of of of every [43:29] continued fulfillment of of of every kind of promise and like continue to [43:31] kind of promise and like continue to [43:31] kind of promise and like continue to expand and serve not just you know [43:33] expand and serve not just you know [43:33] expand and serve not just you know initially maybe more primary care [43:35] initially maybe more primary care [43:35] initially maybe more primary care ambulatory now emergency inpatient [43:38] ambulatory now emergency inpatient [43:38] ambulatory now emergency inpatient nursing stakeholders. I think this this [43:40] nursing stakeholders. I think this this [43:40] nursing stakeholders. I think this this trust kind of acrus through this um [43:43] trust kind of acrus through this um [43:43] trust kind of acrus through this um continued delivery of everything from [43:46] continued delivery of everything from [43:46] continued delivery of everything from like the product through the experience [43:49] like the product through the experience [43:49] like the product through the experience of the the medical teams that are [43:50] of the the medical teams that are [43:50] of the the medical teams that are working with us in partnership over the [43:52] working with us in partnership over the [43:52] working with us in partnership over the course of now we're talking about you [43:55] course of now we're talking about you [43:55] course of now we're talking about you know many years. [43:57] know many years. [43:57] know many years. >> Awesome. Well, thank you very much. I [43:59] >> Awesome. Well, thank you very much. I [43:59] >> Awesome. Well, thank you very much. I appreciate it, Zach. And it's a bridge. [44:00] appreciate it, Zach. And it's a bridge. [44:00] appreciate it, Zach. And it's a bridge. People want to find out more. It sounds [44:02] People want to find out more. It sounds [44:02] People want to find out more. It sounds like a very exciting space to see where [44:03] like a very exciting space to see where [44:03] like a very exciting space to see where you guys are headed. [44:04] you guys are headed. [44:04] you guys are headed. >> Yeah. Thanks for having me. [44:05] >> Yeah. Thanks for having me. [44:05] >> Yeah. Thanks for having me. >> Enjoy the rest of Dev Day. [44:06] >> Enjoy the rest of Dev Day. [44:06] >> Enjoy the rest of Dev Day. >> See you. [44:08] >> See you. [44:08] >> See you. It's interesting to see where you have [44:10] It's interesting to see where you have [44:10] It's interesting to see where you have companies that are dealing with very [44:11] companies that are dealing with very [44:11] companies that are dealing with very high stake stuff like medical or [44:13] high stake stuff like medical or [44:13] high stake stuff like medical or education and a lot of it is the trust [44:15] education and a lot of it is the trust [44:15] education and a lot of it is the trust building. It's not a thing where you [44:16] building. It's not a thing where you [44:16] building. It's not a thing where you just pop out with a product and you say, [44:17] just pop out with a product and you say, [44:18] just pop out with a product and you say, "Hey, we're ready. We're done." You can [44:19] "Hey, we're ready. We're done." You can [44:19] "Hey, we're ready. We're done." You can see from the examples here between Danny [44:22] see from the examples here between Danny [44:22] see from the examples here between Danny and Zach and how they have to basically [44:24] and Zach and how they have to basically [44:24] and Zach and how they have to basically and um Caleb of basically just trying to [44:27] and um Caleb of basically just trying to [44:27] and um Caleb of basically just trying to sort of show the customers one step at a [44:29] sort of show the customers one step at a [44:29] sort of show the customers one step at a time, iterate on the product, improve [44:31] time, iterate on the product, improve [44:31] time, iterate on the product, improve it. And here to talk to us about tools [44:33] it. And here to talk to us about tools [44:33] it. And here to talk to us about tools for helping work on this is Lee Robinson [44:35] for helping work on this is Lee Robinson [44:35] for helping work on this is Lee Robinson from Curser. How you doing? I'm doing [44:37] from Curser. How you doing? I'm doing [44:37] from Curser. How you doing? I'm doing well. Thanks for having me. [44:38] well. Thanks for having me. [44:38] well. Thanks for having me. >> Lee, it is great to have you here. Uh, [44:40] >> Lee, it is great to have you here. Uh, [44:40] >> Lee, it is great to have you here. Uh, so cursor, um, I probably have about [44:43] so cursor, um, I probably have about [44:43] so cursor, um, I probably have about three cursor windows open right now on [44:45] three cursor windows open right now on [44:45] three cursor windows open right now on my computer that I'm thinking about [44:46] my computer that I'm thinking about [44:46] my computer that I'm thinking about right now. Uh, it has been an incredible [44:49] right now. Uh, it has been an incredible [44:49] right now. Uh, it has been an incredible product evolution for you all here. And [44:52] product evolution for you all here. And [44:52] product evolution for you all here. And just a little backstory, I was at open [44:54] just a little backstory, I was at open [44:54] just a little backstory, I was at open eye when we worked on the earliest [44:55] eye when we worked on the earliest [44:55] eye when we worked on the earliest version of codecs and code completions. [44:57] version of codecs and code completions. [44:57] version of codecs and code completions. >> Mh. [44:57] >> Mh. [44:57] >> Mh. >> And I kind of naively thought that, oh [45:00] >> And I kind of naively thought that, oh [45:00] >> And I kind of naively thought that, oh well, I just ask, you know, GPT3 at the [45:02] well, I just ask, you know, GPT3 at the [45:02] well, I just ask, you know, GPT3 at the 3.5 at this time or the codeex model or [45:05] 3.5 at this time or the codeex model or [45:05] 3.5 at this time or the codeex model or Da Vinci Code, whatever, and say just [45:06] Da Vinci Code, whatever, and say just [45:06] Da Vinci Code, whatever, and say just complete it and it's done and I got my [45:07] complete it and it's done and I got my [45:07] complete it and it's done and I got my code. I'm like, that's it. Code is [45:09] code. I'm like, that's it. Code is [45:09] code. I'm like, that's it. Code is solved with AI and we're done. [45:11] solved with AI and we're done. [45:11] solved with AI and we're done. >> Yep. Turned out that's not the case. [45:13] >> Yep. Turned out that's not the case. [45:13] >> Yep. Turned out that's not the case. Yeah, it turns out there's a lot that [45:15] Yeah, it turns out there's a lot that [45:15] Yeah, it turns out there's a lot that goes into it, especially from maybe [45:17] goes into it, especially from maybe [45:17] goes into it, especially from maybe simple text autocomplete, but to where [45:19] simple text autocomplete, but to where [45:19] simple text autocomplete, but to where we're at now with fully autonomous [45:21] we're at now with fully autonomous [45:21] we're at now with fully autonomous coding agents who can self-correct and [45:23] coding agents who can self-correct and [45:23] coding agents who can self-correct and fix their own errors and pull in [45:25] fix their own errors and pull in [45:25] fix their own errors and pull in information from the outside world. And [45:27] information from the outside world. And [45:27] information from the outside world. And it's wild how much better the coding [45:30] it's wild how much better the coding [45:30] it's wild how much better the coding with AI space has gotten just in the [45:31] with AI space has gotten just in the [45:31] with AI space has gotten just in the past year, I would say. [45:34] past year, I would say. [45:34] past year, I would say. >> And it's a tool very much what I [45:36] >> And it's a tool very much what I [45:36] >> And it's a tool very much what I appreciate is the fact that you guys are [45:37] appreciate is the fact that you guys are [45:37] appreciate is the fact that you guys are using cursor to make cursor better. [45:40] using cursor to make cursor better. [45:40] using cursor to make cursor better. >> Definitely. Yeah. One part of our [45:42] >> Definitely. Yeah. One part of our [45:42] >> Definitely. Yeah. One part of our culture that I think helps us produce a [45:44] culture that I think helps us produce a [45:44] culture that I think helps us produce a better product is not only of course dog [45:47] better product is not only of course dog [45:47] better product is not only of course dog fooding how we build cursor by all of [45:48] fooding how we build cursor by all of [45:48] fooding how we build cursor by all of our engineering teams, all of our design [45:50] our engineering teams, all of our design [45:50] our engineering teams, all of our design teams, product management teams using [45:52] teams, product management teams using [45:52] teams, product management teams using the product and using cursor agent to [45:54] the product and using cursor agent to [45:54] the product and using cursor agent to build the actual experience and for [45:56] build the actual experience and for [45:56] build the actual experience and for every part of their daily work [45:58] every part of their daily work [45:58] every part of their daily work >> but then it allows us to give not only [46:01] >> but then it allows us to give not only [46:01] >> but then it allows us to give not only hard evals and you know feels on how a [46:04] hard evals and you know feels on how a [46:04] hard evals and you know feels on how a model works but just what it what it [46:06] model works but just what it what it [46:06] model works but just what it what it feels like every day to use the product. [46:07] feels like every day to use the product. [46:07] feels like every day to use the product. You know sometimes the feedback is this [46:09] You know sometimes the feedback is this [46:09] You know sometimes the feedback is this just didn't feel right. I don't know [46:10] just didn't feel right. I don't know [46:10] just didn't feel right. I don't know what it was with this model experience, [46:12] what it was with this model experience, [46:12] what it was with this model experience, but we we need to tweak this a little [46:13] but we we need to tweak this a little [46:13] but we we need to tweak this a little bit here. And that is actually just as [46:15] bit here. And that is actually just as [46:16] bit here. And that is actually just as valuable as, you know, an eval that [46:17] valuable as, you know, an eval that [46:18] valuable as, you know, an eval that shows that maybe we need to tweak this [46:19] shows that maybe we need to tweak this [46:19] shows that maybe we need to tweak this number one a little bit. So the vibes [46:21] number one a little bit. So the vibes [46:21] number one a little bit. So the vibes are the vibes are important, too. [46:22] are the vibes are important, too. [46:22] are the vibes are important, too. >> How do things work internally as far as [46:24] >> How do things work internally as far as [46:24] >> How do things work internally as far as adding feature? I imagine when you're a [46:26] adding feature? I imagine when you're a [46:26] adding feature? I imagine when you're a company filled with developers, building [46:27] company filled with developers, building [46:27] company filled with developers, building a tool for developers, there's a lot of [46:29] a tool for developers, there's a lot of [46:29] a tool for developers, there's a lot of suggestions on what you should be doing, [46:30] suggestions on what you should be doing, [46:30] suggestions on what you should be doing, but how do you prioritize? Yeah, I think [46:33] but how do you prioritize? Yeah, I think [46:33] but how do you prioritize? Yeah, I think across the entire company, anyone has [46:36] across the entire company, anyone has [46:36] across the entire company, anyone has the ability to contribute features and [46:38] the ability to contribute features and [46:38] the ability to contribute features and we kind of measure if a feature gets [46:41] we kind of measure if a feature gets [46:41] we kind of measure if a feature gets product market fit internally. So people [46:43] product market fit internally. So people [46:43] product market fit internally. So people make changes, they push them to main and [46:45] make changes, they push them to main and [46:45] make changes, they push them to main and they see if it gets adoption. They kind [46:47] they see if it gets adoption. They kind [46:48] they see if it gets adoption. They kind of push it out to company Slack and say, [46:49] of push it out to company Slack and say, [46:50] of push it out to company Slack and say, "Hey, we built this feature. Um, here's [46:52] "Hey, we built this feature. Um, here's [46:52] "Hey, we built this feature. Um, here's what it's good for. We'd love for you to [46:53] what it's good for. We'd love for you to [46:53] what it's good for. We'd love for you to try it out." We've even seen people [46:54] try it out." We've even seen people [46:54] try it out." We've even seen people incentivize with bagels and other [46:56] incentivize with bagels and other [46:56] incentivize with bagels and other pastries to get people to try out the [46:59] pastries to get people to try out the [46:59] pastries to get people to try out the feature. and they get a lot of feedback [47:01] feature. and they get a lot of feedback [47:01] feature. and they get a lot of feedback internally on if it's good, where they [47:02] internally on if it's good, where they [47:02] internally on if it's good, where they should change. We also have weekly demo [47:05] should change. We also have weekly demo [47:05] should change. We also have weekly demo sessions where people show off the cool [47:06] sessions where people show off the cool [47:06] sessions where people show off the cool things that they're working on. And some [47:08] things that they're working on. And some [47:08] things that they're working on. And some features don't work out and we end up [47:11] features don't work out and we end up [47:11] features don't work out and we end up removing them. And others like really [47:12] removing them. And others like really [47:12] removing them. And others like really take off like people have been wanting [47:14] take off like people have been wanting [47:14] take off like people have been wanting this for a while. We build it [47:15] this for a while. We build it [47:15] this for a while. We build it internally. All of a sudden there's we [47:17] internally. All of a sudden there's we [47:17] internally. All of a sudden there's we look at metrics and we see how many [47:18] look at metrics and we see how many [47:18] look at metrics and we see how many internal devs are actually adopting the [47:20] internal devs are actually adopting the [47:20] internal devs are actually adopting the feature and what the churn rate is. And [47:22] feature and what the churn rate is. And [47:22] feature and what the churn rate is. And if it reaches a certain threshold, we [47:24] if it reaches a certain threshold, we [47:24] if it reaches a certain threshold, we have the conversation maybe we should [47:25] have the conversation maybe we should [47:25] have the conversation maybe we should actually release this externally. So [47:27] actually release this externally. So [47:27] actually release this externally. So then we go through kind of a series of [47:29] then we go through kind of a series of [47:29] then we go through kind of a series of steps where starts internal then maybe [47:31] steps where starts internal then maybe [47:31] steps where starts internal then maybe rolls out to some of our ambassadors [47:33] rolls out to some of our ambassadors [47:33] rolls out to some of our ambassadors rolls out to some of the people who opt [47:34] rolls out to some of the people who opt [47:34] rolls out to some of the people who opt into our nightly channel which is like [47:36] into our nightly channel which is like [47:36] into our nightly channel which is like you know releases all the time and then [47:38] you know releases all the time and then [47:38] you know releases all the time and then eventually there's the path towards [47:39] eventually there's the path towards [47:39] eventually there's the path towards getting it out to everybody. So one of [47:42] getting it out to everybody. So one of [47:42] getting it out to everybody. So one of the challenges in AI now is evaluating [47:44] the challenges in AI now is evaluating [47:44] the challenges in AI now is evaluating models and we can put something into one [47:47] models and we can put something into one [47:47] models and we can put something into one of the the eval one of the benches or [47:49] of the the eval one of the benches or [47:49] of the the eval one of the benches or whatever but say oh it performs this [47:50] whatever but say oh it performs this [47:50] whatever but say oh it performs this well but it's entirely different when [47:52] well but it's entirely different when [47:52] well but it's entirely different when you put it inside of an IDE and you have [47:53] you put it inside of an IDE and you have [47:53] you put it inside of an IDE and you have to basically put a harness around it [47:55] to basically put a harness around it [47:55] to basically put a harness around it >> right [47:55] >> right [47:56] >> right >> so how much time do you all have to [47:57] >> so how much time do you all have to [47:57] >> so how much time do you all have to spend really evaluating these things to [47:59] spend really evaluating these things to [47:59] spend really evaluating these things to really figure out if this is going to be [48:00] really figure out if this is going to be [48:00] really figure out if this is going to be a good fit or not [48:01] a good fit or not [48:01] a good fit or not >> yeah quite a bit I mean one of the [48:03] >> yeah quite a bit I mean one of the [48:03] >> yeah quite a bit I mean one of the things that you know we love working [48:04] things that you know we love working [48:04] things that you know we love working with open AI on is getting access to [48:07] with open AI on is getting access to [48:07] with open AI on is getting access to integrate the model early work with your [48:08] integrate the model early work with your [48:08] integrate the model early work with your team on tweaking the prompts getting the [48:10] team on tweaking the prompts getting the [48:10] team on tweaking the prompts getting the harness updated as newer model comes [48:12] harness updated as newer model comes [48:12] harness updated as newer model comes out. I know we spent a lot of time on [48:13] out. I know we spent a lot of time on [48:13] out. I know we spent a lot of time on this with GPT5 to actually we're able to [48:16] this with GPT5 to actually we're able to [48:16] this with GPT5 to actually we're able to delete some of our system prompt where [48:18] delete some of our system prompt where [48:18] delete some of our system prompt where as the models get better, you don't have [48:19] as the models get better, you don't have [48:20] as the models get better, you don't have to be as precise with the instructions [48:21] to be as precise with the instructions [48:21] to be as precise with the instructions that you're giving it and the tools are [48:23] that you're giving it and the tools are [48:23] that you're giving it and the tools are getting better too. So we we spend a lot [48:26] getting better too. So we we spend a lot [48:26] getting better too. So we we spend a lot of time not only dog fooding and just [48:27] of time not only dog fooding and just [48:27] of time not only dog fooding and just trying it for a bunch of tasks [48:29] trying it for a bunch of tasks [48:29] trying it for a bunch of tasks internally, but also having many of the [48:31] internally, but also having many of the [48:31] internally, but also having many of the engineers internally all try to build [48:33] engineers internally all try to build [48:33] engineers internally all try to build new features on the core product [48:34] new features on the core product [48:34] new features on the core product experience with new models as they come [48:36] experience with new models as they come [48:36] experience with new models as they come out. So then you get a good range of, [48:38] out. So then you get a good range of, [48:38] out. So then you get a good range of, you know, maybe smaller tasks, but also [48:40] you know, maybe smaller tasks, but also [48:40] you know, maybe smaller tasks, but also real tricky engineering problems, like [48:42] real tricky engineering problems, like [48:42] real tricky engineering problems, like real gnarly bugs in a very large [48:44] real gnarly bugs in a very large [48:44] real gnarly bugs in a very large codebase. [48:45] codebase. [48:45] codebase. >> Yeah, it's interesting in that um a [48:47] >> Yeah, it's interesting in that um a [48:47] >> Yeah, it's interesting in that um a couple years ago, people sort of [48:48] couple years ago, people sort of [48:48] couple years ago, people sort of wondered like what was going to happen [48:49] wondered like what was going to happen [48:50] wondered like what was going to happen in the model space. Was everything going [48:51] in the model space. Was everything going [48:51] in the model space. Was everything going to be using big foundational models, [48:53] to be using big foundational models, [48:53] to be using big foundational models, whatever. But you guys have sort of an [48:54] whatever. But you guys have sort of an [48:54] whatever. But you guys have sort of an all the above. You're using you can have [48:56] all the above. You're using you can have [48:56] all the above. You're using you can have I can use if I want I can choose variety [48:58] I can use if I want I can choose variety [48:58] I can use if I want I can choose variety of different models in there. I can use [48:59] of different models in there. I can use [48:59] of different models in there. I can use codecs, whatever inside of there. But [49:00] codecs, whatever inside of there. But [49:00] codecs, whatever inside of there. But also you guys like training your own [49:02] also you guys like training your own [49:02] also you guys like training your own models for the smaller for like tab [49:03] models for the smaller for like tab [49:03] models for the smaller for like tab completions. Yeah, it's been an [49:05] completions. Yeah, it's been an [49:05] completions. Yeah, it's been an interesting journey to watch. The first [49:07] interesting journey to watch. The first [49:07] interesting journey to watch. The first versions of cursor were really focused [49:09] versions of cursor were really focused [49:09] versions of cursor were really focused just on code autocomp completion and it [49:11] just on code autocomp completion and it [49:11] just on code autocomp completion and it was predicting the next line or maybe [49:13] was predicting the next line or maybe [49:13] was predicting the next line or maybe the next action. And now we kind of [49:15] the next action. And now we kind of [49:15] the next action. And now we kind of moved from a world of using an [49:16] moved from a world of using an [49:16] moved from a world of using an off-the-shelf model to training our own [49:18] off-the-shelf model to training our own [49:18] off-the-shelf model to training our own model for autocomplete to now we do [49:20] model for autocomplete to now we do [49:20] model for autocomplete to now we do online reinforcement learning where we [49:22] online reinforcement learning where we [49:22] online reinforcement learning where we can actually roll updates every 30 [49:23] can actually roll updates every 30 [49:24] can actually roll updates every 30 minutes to the model which is pretty [49:25] minutes to the model which is pretty [49:25] minutes to the model which is pretty amazing just based on the signals we get [49:27] amazing just based on the signals we get [49:27] amazing just based on the signals we get back from developers whether they're [49:29] back from developers whether they're [49:29] back from developers whether they're accepting or rejecting changes that the [49:31] accepting or rejecting changes that the [49:31] accepting or rejecting changes that the autocomplete um suggests. And that's [49:34] autocomplete um suggests. And that's [49:34] autocomplete um suggests. And that's been really helpful where you can have [49:36] been really helpful where you can have [49:36] been really helpful where you can have both these really focused and [49:38] both these really focused and [49:38] both these really focused and intentional models for very specific [49:40] intentional models for very specific [49:40] intentional models for very specific parts of tasks in coding. And then as [49:43] parts of tasks in coding. And then as [49:43] parts of tasks in coding. And then as you all make better models, these [49:45] you all make better models, these [49:45] you all make better models, these foundational models, we can integrate [49:47] foundational models, we can integrate [49:47] foundational models, we can integrate them and make that experience really [49:48] them and make that experience really [49:48] them and make that experience really great too. I'm seeing more and more of [49:51] great too. I'm seeing more and more of [49:51] great too. I'm seeing more and more of my friends who never even thought about [49:53] my friends who never even thought about [49:53] my friends who never even thought about coding start off with writing something [49:55] coding start off with writing something [49:55] coding start off with writing something maybe in canvas inside of chat GBT and [49:57] maybe in canvas inside of chat GBT and [49:58] maybe in canvas inside of chat GBT and then all of a sudden go, "Oh, how would [49:59] then all of a sudden go, "Oh, how would [49:59] then all of a sudden go, "Oh, how would I deploy this?" And then they go try [50:00] I deploy this?" And then they go try [50:00] I deploy this?" And then they go try cursor and try to do this. And how much [50:02] cursor and try to do this. And how much [50:02] cursor and try to do this. And how much have you seen the demographic of the [50:04] have you seen the demographic of the [50:04] have you seen the demographic of the people using your tools change? [50:05] people using your tools change? [50:05] people using your tools change? >> Yeah, first and foremost because we dog [50:08] >> Yeah, first and foremost because we dog [50:08] >> Yeah, first and foremost because we dog food cursor to build cursor like we're [50:10] food cursor to build cursor like we're [50:10] food cursor to build cursor like we're always building for the professional [50:12] always building for the professional [50:12] always building for the professional software engineer and we're trying to [50:13] software engineer and we're trying to [50:13] software engineer and we're trying to make that experience really great. But [50:15] make that experience really great. But [50:15] make that experience really great. But as a side effect of making the product [50:18] as a side effect of making the product [50:18] as a side effect of making the product easier and more accessible for [50:20] easier and more accessible for [50:20] easier and more accessible for engineers, it kind of welcomes in this [50:22] engineers, it kind of welcomes in this [50:22] engineers, it kind of welcomes in this whole new generation of people who maybe [50:24] whole new generation of people who maybe [50:24] whole new generation of people who maybe were on the fringes of coding or had [50:26] were on the fringes of coding or had [50:26] were on the fringes of coding or had tried coding in the past and now it's [50:28] tried coding in the past and now it's [50:28] tried coding in the past and now it's it's much more accessible for them to do [50:30] it's much more accessible for them to do [50:30] it's much more accessible for them to do that. So, a lot more product managers, a [50:32] that. So, a lot more product managers, a [50:32] that. So, a lot more product managers, a lot more designers, uh, our support team [50:34] lot more designers, uh, our support team [50:34] lot more designers, uh, our support team uses cursor quite a bit. And as more of [50:37] uses cursor quite a bit. And as more of [50:38] uses cursor quite a bit. And as more of these people have came in and started to [50:39] these people have came in and started to [50:39] these people have came in and started to use the product, it's allowed us to [50:41] use the product, it's allowed us to [50:41] use the product, it's allowed us to change actually some of the core [50:42] change actually some of the core [50:42] change actually some of the core features we're building. So, one of the [50:44] features we're building. So, one of the [50:44] features we're building. So, one of the things that we're releasing right now is [50:46] things that we're releasing right now is [50:46] things that we're releasing right now is a new view of viewing the cursor [50:48] a new view of viewing the cursor [50:48] a new view of viewing the cursor experience where it's less like a [50:50] experience where it's less like a [50:50] experience where it's less like a traditional code editor and it's more [50:52] traditional code editor and it's more [50:52] traditional code editor and it's more about working with the agents. Because [50:54] about working with the agents. Because [50:54] about working with the agents. Because for a lot of people who haven't coded [50:56] for a lot of people who haven't coded [50:56] for a lot of people who haven't coded before, you open up an IDE and you're [50:59] before, you open up an IDE and you're [50:59] before, you open up an IDE and you're like, what am I looking at? [51:00] like, what am I looking at? [51:00] like, what am I looking at? >> All this stuff. [51:01] >> All this stuff. [51:01] >> All this stuff. >> There's like all this file tree on the [51:03] >> There's like all this file tree on the [51:03] >> There's like all this file tree on the left is very overwhelming for people who [51:04] left is very overwhelming for people who [51:04] left is very overwhelming for people who are just getting started versus opening [51:06] are just getting started versus opening [51:06] are just getting started versus opening up something that looks closer to a [51:08] up something that looks closer to a [51:08] up something that looks closer to a chatbt. You're like, okay, I know this [51:10] chatbt. You're like, okay, I know this [51:10] chatbt. You're like, okay, I know this experience. I've got my agents on the [51:11] experience. I've got my agents on the [51:12] experience. I've got my agents on the left. I can type into my input text box. [51:14] left. I can type into my input text box. [51:14] left. I can type into my input text box. And so far, we've seen this really [51:16] And so far, we've seen this really [51:16] And so far, we've seen this really resonate with that type of persona where [51:19] resonate with that type of persona where [51:19] resonate with that type of persona where they're graduating into becoming a [51:21] they're graduating into becoming a [51:22] they're graduating into becoming a developer. I went from somebody who was [51:25] developer. I went from somebody who was [51:25] developer. I went from somebody who was first copy pasting before we even had [51:27] first copy pasting before we even had [51:27] first copy pasting before we even had chat GBT from the playground or whatever [51:29] chat GBT from the playground or whatever [51:29] chat GBT from the playground or whatever building some simple little custom tools [51:30] building some simple little custom tools [51:30] building some simple little custom tools to then using IDE and like in case of [51:32] to then using IDE and like in case of [51:32] to then using IDE and like in case of this case using cursor and using tab [51:34] this case using cursor and using tab [51:34] this case using cursor and using tab completions and now really just sitting [51:37] completions and now really just sitting [51:37] completions and now really just sitting down and saying I just need to write a [51:38] down and saying I just need to write a [51:38] down and saying I just need to write a really good agents.mmd or really [51:40] really good agents.mmd or really [51:40] really good agents.mmd or really description. Have you seen those [51:42] description. Have you seen those [51:42] description. Have you seen those patterns shift you know? Yeah, I think [51:45] patterns shift you know? Yeah, I think [51:45] patterns shift you know? Yeah, I think as people get more advanced with working [51:48] as people get more advanced with working [51:48] as people get more advanced with working with coding agents, they really start to [51:50] with coding agents, they really start to [51:50] with coding agents, they really start to realize that high quality context in is [51:52] realize that high quality context in is [51:52] realize that high quality context in is going to give you much higher quality [51:55] going to give you much higher quality [51:55] going to give you much higher quality responses from the models. And we can [51:57] responses from the models. And we can [51:57] responses from the models. And we can build a lot of this into the core [51:59] build a lot of this into the core [51:59] build a lot of this into the core product itself. But some of it require [52:01] product itself. But some of it require [52:01] product itself. But some of it require just it depends on your use case of what [52:04] just it depends on your use case of what [52:04] just it depends on your use case of what you're trying to build the system [52:05] you're trying to build the system [52:05] you're trying to build the system requirements and a lot of this stuff [52:07] requirements and a lot of this stuff [52:07] requirements and a lot of this stuff gets put into an agent. MD file or gets [52:10] gets put into an agent. MD file or gets [52:10] gets put into an agent. MD file or gets put into newer features we have like [52:12] put into newer features we have like [52:12] put into newer features we have like planning. So you can actually work [52:13] planning. So you can actually work [52:13] planning. So you can actually work through a plan mode, go back and forth [52:15] through a plan mode, go back and forth [52:16] through a plan mode, go back and forth with the agent to do some research on [52:17] with the agent to do some research on [52:18] with the agent to do some research on what you're trying to build and it can [52:19] what you're trying to build and it can [52:19] what you're trying to build and it can search your codebase and kind of help [52:21] search your codebase and kind of help [52:21] search your codebase and kind of help paint a picture and then when you're [52:23] paint a picture and then when you're [52:23] paint a picture and then when you're handing it off to the model to do [52:24] handing it off to the model to do [52:24] handing it off to the model to do coding, it's already got all of this [52:26] coding, it's already got all of this [52:26] coding, it's already got all of this amazing input context and the quality of [52:28] amazing input context and the quality of [52:28] amazing input context and the quality of the code generated is just significantly [52:30] the code generated is just significantly [52:30] the code generated is just significantly better at that point. So we've seen that [52:32] better at that point. So we've seen that [52:32] better at that point. So we've seen that be very helpful too. [52:33] be very helpful too. [52:33] be very helpful too. >> What would you advise for best practices [52:36] >> What would you advise for best practices [52:36] >> What would you advise for best practices for general just adopting cursor [52:38] for general just adopting cursor [52:38] for general just adopting cursor >> starting there? Yeah. [52:39] >> starting there? Yeah. [52:39] >> starting there? Yeah. >> Yeah. I think that for people who are [52:41] >> Yeah. I think that for people who are [52:41] >> Yeah. I think that for people who are completely fresh to using cursor and [52:43] completely fresh to using cursor and [52:44] completely fresh to using cursor and we'll start with more of the [52:45] we'll start with more of the [52:45] we'll start with more of the professional engineer use case then we [52:46] professional engineer use case then we [52:46] professional engineer use case then we can talk about others but for [52:48] can talk about others but for [52:48] can talk about others but for professional engineers I think what we [52:49] professional engineers I think what we [52:49] professional engineers I think what we see for most people is their on-ramp is [52:51] see for most people is their on-ramp is [52:51] see for most people is their on-ramp is they start using the editor just like [52:53] they start using the editor just like [52:53] they start using the editor just like they would traditional editors or idees [52:56] they would traditional editors or idees [52:56] they would traditional editors or idees where mostly they're using tab and [52:57] where mostly they're using tab and [52:57] where mostly they're using tab and they're doing their normal coding and [52:59] they're doing their normal coding and [52:59] they're doing their normal coding and they're still getting AI suggestions as [53:01] they're still getting AI suggestions as [53:01] they're still getting AI suggestions as they ride along and they're using agent [53:03] they ride along and they're using agent [53:03] they ride along and they're using agent to somewhat augment that experience but [53:05] to somewhat augment that experience but [53:05] to somewhat augment that experience but they're not coding maybe agent first [53:08] they're not coding maybe agent first [53:08] they're not coding maybe agent first >> but I think slowly Over time, as the [53:10] >> but I think slowly Over time, as the [53:10] >> but I think slowly Over time, as the models get better and the tools get [53:11] models get better and the tools get [53:11] models get better and the tools get better, they're graduating then into [53:13] better, they're graduating then into [53:13] better, they're graduating then into what would it look like if I can hand [53:15] what would it look like if I can hand [53:15] what would it look like if I can hand off more gnarly task to the agents to [53:18] off more gnarly task to the agents to [53:18] off more gnarly task to the agents to maybe run in the background or go [53:20] maybe run in the background or go [53:20] maybe run in the background or go refactor some part of the codebase. So, [53:21] refactor some part of the codebase. So, [53:22] refactor some part of the codebase. So, I usually recommend that progression for [53:24] I usually recommend that progression for [53:24] I usually recommend that progression for professional engineers. It's kind of the [53:26] professional engineers. It's kind of the [53:26] professional engineers. It's kind of the inverse for those who are not [53:28] inverse for those who are not [53:28] inverse for those who are not professional engineers because it's very [53:30] professional engineers because it's very [53:30] professional engineers because it's very hard for them to drop into code and then [53:33] hard for them to drop into code and then [53:33] hard for them to drop into code and then know, okay, this is JavaScript. Oh, [53:34] know, okay, this is JavaScript. Oh, [53:34] know, okay, this is JavaScript. Oh, actually, this is Typescript and this is [53:35] actually, this is Typescript and this is [53:36] actually, this is Typescript and this is what a type is and this is what all [53:37] what a type is and this is what all [53:37] what a type is and this is what all these fancy words mean. It's much easier [53:39] these fancy words mean. It's much easier [53:39] these fancy words mean. It's much easier for them to take the other approach [53:40] for them to take the other approach [53:40] for them to take the other approach where they start with the agent view. [53:42] where they start with the agent view. [53:42] where they start with the agent view. >> They talk to the agent in natural [53:44] >> They talk to the agent in natural [53:44] >> They talk to the agent in natural language and the agent outputs code and [53:46] language and the agent outputs code and [53:46] language and the agent outputs code and then they can ask the agent what does [53:48] then they can ask the agent what does [53:48] then they can ask the agent what does that mean? What is a const versus a let [53:50] that mean? What is a const versus a let [53:50] that mean? What is a const versus a let you know what do these things mean? And [53:52] you know what do these things mean? And [53:52] you know what do these things mean? And you know or something in Python for [53:54] you know or something in Python for [53:54] you know or something in Python for example. So generally that's what I [53:56] example. So generally that's what I [53:56] example. So generally that's what I recommend. [53:57] recommend. [53:57] recommend. >> Where do you think this is going to be [53:58] >> Where do you think this is going to be [53:58] >> Where do you think this is going to be headed? We're we're five years in to [54:00] headed? We're we're five years in to [54:00] headed? We're we're five years in to GPD3. You know, the first time I [54:02] GPD3. You know, the first time I [54:02] GPD3. You know, the first time I mentioned it earlier was that, you know, [54:04] mentioned it earlier was that, you know, [54:04] mentioned it earlier was that, you know, seeing somebody make a react button was [54:06] seeing somebody make a react button was [54:06] seeing somebody make a react button was exciting. Yeah. And here we are now [54:07] exciting. Yeah. And here we are now [54:07] exciting. Yeah. And here we are now where you have an entire codebase going [54:09] where you have an entire codebase going [54:09] where you have an entire codebase going through they're doing this. Where's it [54:10] through they're doing this. Where's it [54:10] through they're doing this. Where's it going to be five years from now? [54:12] going to be five years from now? [54:12] going to be five years from now? >> There's still so much of software [54:14] >> There's still so much of software [54:14] >> There's still so much of software engineering, not coding, but software [54:17] engineering, not coding, but software [54:17] engineering, not coding, but software engineering, the professional job that [54:19] engineering, the professional job that [54:19] engineering, the professional job that is a lot of mundane, repetitive tasks [54:22] is a lot of mundane, repetitive tasks [54:22] is a lot of mundane, repetitive tasks that engineers are not excited by. [54:24] that engineers are not excited by. [54:24] that engineers are not excited by. >> There's being on call and having to go [54:26] >> There's being on call and having to go [54:26] >> There's being on call and having to go through the fire hose of data to figure [54:28] through the fire hose of data to figure [54:28] through the fire hose of data to figure out how to solve an issue. There's [54:30] out how to solve an issue. There's [54:30] out how to solve an issue. There's managing all of the incoming bugs that [54:32] managing all of the incoming bugs that [54:32] managing all of the incoming bugs that users are reporting and trying to figure [54:34] users are reporting and trying to figure [54:34] users are reporting and trying to figure out the right ones to work on and then [54:35] out the right ones to work on and then [54:35] out the right ones to work on and then actually getting them addressed. There's [54:37] actually getting them addressed. There's [54:37] actually getting them addressed. There's so much more that goes into how the [54:39] so much more that goes into how the [54:39] so much more that goes into how the actual software gets packaged and [54:41] actual software gets packaged and [54:41] actual software gets packaged and delivered and shipped than just [54:42] delivered and shipped than just [54:42] delivered and shipped than just producing code. And I think the next, [54:45] producing code. And I think the next, [54:45] producing code. And I think the next, let's call it year to two years. [54:46] let's call it year to two years. [54:46] let's call it year to two years. Hopefully, there will be a lot better [54:48] Hopefully, there will be a lot better [54:48] Hopefully, there will be a lot better solutions across the industry for making [54:50] solutions across the industry for making [54:50] solutions across the industry for making that part of software engineering [54:52] that part of software engineering [54:52] that part of software engineering easier. Like I imagine a world where you [54:54] easier. Like I imagine a world where you [54:54] easier. Like I imagine a world where you wake up in the morning and you're able [54:55] wake up in the morning and you're able [54:55] wake up in the morning and you're able to review code that's already been [54:59] to review code that's already been [54:59] to review code that's already been tested, generated. You had customers [55:01] tested, generated. You had customers [55:01] tested, generated. You had customers report issues and they actually got [55:02] report issues and they actually got [55:02] report issues and they actually got fixed overnight and you could just [55:04] fixed overnight and you could just [55:04] fixed overnight and you could just review the output. Yes, this looks okay. [55:05] review the output. Yes, this looks okay. [55:06] review the output. Yes, this looks okay. It passes all of our tests except in [55:07] It passes all of our tests except in [55:07] It passes all of our tests except in merge. And a world where maybe on call [55:10] merge. And a world where maybe on call [55:10] merge. And a world where maybe on call isn't as painful and maybe code reviews [55:12] isn't as painful and maybe code reviews [55:12] isn't as painful and maybe code reviews are actually fun to use. You know, I I [55:15] are actually fun to use. You know, I I [55:15] are actually fun to use. You know, I I think we can get there faster than what [55:17] think we can get there faster than what [55:17] think we can get there faster than what people might expect. in I think both in [55:20] people might expect. in I think both in [55:20] people might expect. in I think both in exploring things just working with [55:22] exploring things just working with [55:22] exploring things just working with models directly and then the products [55:23] models directly and then the products [55:23] models directly and then the products around them getting better. [55:24] around them getting better. [55:24] around them getting better. >> What surprised you the most? What has [55:26] >> What surprised you the most? What has [55:26] >> What surprised you the most? What has been your big aha moment [55:27] been your big aha moment [55:28] been your big aha moment >> in working with cursor [55:29] >> in working with cursor [55:29] >> in working with cursor >> in cursor? Anything? [55:30] >> in cursor? Anything? [55:30] >> in cursor? Anything? >> Yeah, I think for me it is incredible [55:34] >> Yeah, I think for me it is incredible [55:34] >> Yeah, I think for me it is incredible how when you're in the San Francisco [55:36] how when you're in the San Francisco [55:36] how when you're in the San Francisco tech world, it seems like everyone's [55:39] tech world, it seems like everyone's [55:39] tech world, it seems like everyone's coding with AI, right? Like everyone's [55:41] coding with AI, right? Like everyone's [55:41] coding with AI, right? Like everyone's using these tools and then you talk to [55:43] using these tools and then you talk to [55:43] using these tools and then you talk to people around the world and you realize [55:44] people around the world and you realize [55:44] people around the world and you realize we're still so early. We are so early. I [55:47] we're still so early. We are so early. I [55:47] we're still so early. We are so early. I mean 5 years in from that first React [55:49] mean 5 years in from that first React [55:49] mean 5 years in from that first React component generated and for a lot of [55:51] component generated and for a lot of [55:51] component generated and for a lot of people this is still you know this year [55:52] people this is still you know this year [55:52] people this is still you know this year is the first time they're really coding [55:54] is the first time they're really coding [55:54] is the first time they're really coding with AI. So for the long tale of the [55:57] with AI. So for the long tale of the [55:57] with AI. So for the long tale of the industry and then the next however many [55:59] industry and then the next however many [55:59] industry and then the next however many million of people who become developers [56:01] million of people who become developers [56:01] million of people who become developers it's never been a better time to [56:03] it's never been a better time to [56:03] it's never been a better time to actually start learning how to code in [56:05] actually start learning how to code in [56:05] actually start learning how to code in general but then also how to use AI to [56:07] general but then also how to use AI to [56:07] general but then also how to use AI to write code and review code. I was at a [56:09] write code and review code. I was at a [56:09] write code and review code. I was at a at a college campus bar. I won't name [56:11] at a college campus bar. I won't name [56:11] at a college campus bar. I won't name the campus and this was last semester [56:15] the campus and this was last semester [56:15] the campus and this was last semester and I asked the students in the CS [56:17] and I asked the students in the CS [56:17] and I asked the students in the CS program. Um, what are they teaching you [56:19] program. Um, what are they teaching you [56:19] program. Um, what are they teaching you about agent coding? What are they [56:20] about agent coding? What are they [56:20] about agent coding? What are they teaching about about AI code completion? [56:23] teaching about about AI code completion? [56:23] teaching about about AI code completion? Nothing. [56:23] Nothing. [56:23] Nothing. >> Nothing. Nothing. Not even not even a [56:25] >> Nothing. Nothing. Not even not even a [56:25] >> Nothing. Nothing. Not even not even a one-day class on it. And and it was [56:27] one-day class on it. And and it was [56:27] one-day class on it. And and it was weird because like I get learn the [56:29] weird because like I get learn the [56:29] weird because like I get learn the fundamentals. If you understand, you [56:31] fundamentals. If you understand, you [56:31] fundamentals. If you understand, you know, a deeper understanding of Python, [56:32] know, a deeper understanding of Python, [56:32] know, a deeper understanding of Python, these other languages, you're going to [56:34] these other languages, you're going to [56:34] these other languages, you're going to really be excellent this stuff. [56:35] really be excellent this stuff. [56:35] really be excellent this stuff. >> Absolutely. [56:35] >> Absolutely. [56:36] >> Absolutely. >> But there was none of that. [56:37] >> But there was none of that. [56:37] >> But there was none of that. >> Yeah. The education system is going to [56:40] >> Yeah. The education system is going to [56:40] >> Yeah. The education system is going to have to change a bit specifically around [56:42] have to change a bit specifically around [56:42] have to change a bit specifically around coding because it's evolving so fast. [56:45] coding because it's evolving so fast. [56:45] coding because it's evolving so fast. It's being speedrun and you 100% need to [56:48] It's being speedrun and you 100% need to [56:48] It's being speedrun and you 100% need to learn the foundations of CS. That's how [56:50] learn the foundations of CS. That's how [56:50] learn the foundations of CS. That's how you're going to have a successful career [56:51] you're going to have a successful career [56:51] you're going to have a successful career as a software engineer, but you need to [56:53] as a software engineer, but you need to [56:54] as a software engineer, but you need to know the new tools that are changing at [56:56] know the new tools that are changing at [56:56] know the new tools that are changing at this point. Every few months, they're [56:57] this point. Every few months, they're [56:57] this point. Every few months, they're just getting better and better and [56:58] just getting better and better and [56:58] just getting better and better and better. So, this is one of the things I [56:59] better. So, this is one of the things I [56:59] better. So, this is one of the things I work on full-time at Kurser is teaching [57:01] work on full-time at Kurser is teaching [57:01] work on full-time at Kurser is teaching developers and putting out educational [57:02] developers and putting out educational [57:02] developers and putting out educational material. So, we just put out a course [57:04] material. So, we just put out a course [57:04] material. So, we just put out a course teaching devs and hopefully new devs a [57:06] teaching devs and hopefully new devs a [57:06] teaching devs and hopefully new devs a lot of these fundamentals. What What is [57:08] lot of these fundamentals. What What is [57:08] lot of these fundamentals. What What is context? What are tokens? What's a [57:10] context? What are tokens? What's a [57:10] context? What are tokens? What's a what's a context window? Like, for a lot [57:12] what's a context window? Like, for a lot [57:12] what's a context window? Like, for a lot of people, this is, you know, gibberish. [57:14] of people, this is, you know, gibberish. [57:14] of people, this is, you know, gibberish. They don't know what this means yet. [57:15] They don't know what this means yet. [57:15] They don't know what this means yet. >> Yeah. It's it's still it took me as an [57:18] >> Yeah. It's it's still it took me as an [57:18] >> Yeah. It's it's still it took me as an aside, though, because it was just [57:19] aside, though, because it was just [57:19] aside, though, because it was just thinking about, you know, students [57:21] thinking about, you know, students [57:21] thinking about, you know, students coming out are going to want to go work [57:23] coming out are going to want to go work [57:23] coming out are going to want to go work in companies and the companies that are [57:24] in companies and the companies that are [57:24] in companies and the companies that are moving really fast, [57:26] moving really fast, [57:26] moving really fast, >> you know, are the ones where you [57:27] >> you know, are the ones where you [57:27] >> you know, are the ones where you understand that. And you know my friends [57:29] understand that. And you know my friends [57:29] understand that. And you know my friends who were brilliant computer scientists [57:32] who were brilliant computer scientists [57:32] who were brilliant computer scientists who understand it really deeply who are [57:33] who understand it really deeply who are [57:34] who understand it really deeply who are using these things get the best of both [57:35] using these things get the best of both [57:35] using these things get the best of both worlds. They understand deeply how the [57:37] worlds. They understand deeply how the [57:37] worlds. They understand deeply how the stuff works but they also understand how [57:38] stuff works but they also understand how [57:38] stuff works but they also understand how to apply this. [57:39] to apply this. [57:39] to apply this. >> Yeah. [57:40] >> Yeah. [57:40] >> Yeah. >> So you know whatever we can do to [57:41] >> So you know whatever we can do to [57:41] >> So you know whatever we can do to encourage more of that I think in [57:43] encourage more of that I think in [57:43] encourage more of that I think in education would be great. [57:44] education would be great. [57:44] education would be great. >> Well just like how Chad GBD has the [57:46] >> Well just like how Chad GBD has the [57:46] >> Well just like how Chad GBD has the learn mode built in. I would love like [57:48] learn mode built in. I would love like [57:48] learn mode built in. I would love like this is a side project maybe but I [57:50] this is a side project maybe but I [57:50] this is a side project maybe but I really want to build this into cursor. [57:51] really want to build this into cursor. [57:51] really want to build this into cursor. That would be amazing. Like I want [57:53] That would be amazing. Like I want [57:53] That would be amazing. Like I want engineers, especially those just getting [57:54] engineers, especially those just getting [57:54] engineers, especially those just getting started, maybe a freshman in college, to [57:57] started, maybe a freshman in college, to [57:57] started, maybe a freshman in college, to be able to use the tool and learn as [57:59] be able to use the tool and learn as [57:59] be able to use the tool and learn as they're building because for me [58:00] they're building because for me [58:00] they're building because for me personally, I think for most people who [58:02] personally, I think for most people who [58:02] personally, I think for most people who write code, applied application use of [58:05] write code, applied application use of [58:05] write code, applied application use of coding is actually how I learn the best. [58:07] coding is actually how I learn the best. [58:07] coding is actually how I learn the best. Like I can't just read all of the CS [58:09] Like I can't just read all of the CS [58:09] Like I can't just read all of the CS textbooks. I actually need to build [58:10] textbooks. I actually need to build [58:10] textbooks. I actually need to build something with the data structures or [58:11] something with the data structures or [58:11] something with the data structures or algorithms. It's like, oh, then it [58:13] algorithms. It's like, oh, then it [58:13] algorithms. It's like, oh, then it clicks. So if we can build more of that [58:15] clicks. So if we can build more of that [58:15] clicks. So if we can build more of that into the core product experience, I [58:17] into the core product experience, I [58:17] into the core product experience, I think that would be really helpful. [58:18] think that would be really helpful. [58:18] think that would be really helpful. >> Yeah. two of my close friends who were [58:20] >> Yeah. two of my close friends who were [58:20] >> Yeah. two of my close friends who were never coders, their first introduction [58:22] never coders, their first introduction [58:22] never coders, their first introduction after playing around chat GPT was to go [58:24] after playing around chat GPT was to go [58:24] after playing around chat GPT was to go into cursor and to learn inside of there [58:27] into cursor and to learn inside of there [58:27] into cursor and to learn inside of there and also understand that you know once [58:29] and also understand that you know once [58:29] and also understand that you know once you even using it as is right now having [58:31] you even using it as is right now having [58:31] you even using it as is right now having something that can explain to you what a [58:32] something that can explain to you what a [58:32] something that can explain to you what a server does, how to run it locally and [58:34] server does, how to run it locally and [58:34] server does, how to run it locally and do that is amazing lift and it seems [58:36] do that is amazing lift and it seems [58:36] do that is amazing lift and it seems like that would be kind of a really you [58:37] like that would be kind of a really you [58:37] like that would be kind of a really you know great addition to it. Do you feel [58:40] know great addition to it. Do you feel [58:40] know great addition to it. Do you feel like we're going to see kind of the vibe [58:44] like we're going to see kind of the vibe [58:44] like we're going to see kind of the vibe coding error sort of just evolve into [58:46] coding error sort of just evolve into [58:46] coding error sort of just evolve into where we just call it coding? [58:48] where we just call it coding? [58:48] where we just call it coding? >> Yeah, probably. I think one of the [58:50] >> Yeah, probably. I think one of the [58:50] >> Yeah, probably. I think one of the things with vibe coding that's [58:51] things with vibe coding that's [58:51] things with vibe coding that's interesting is certain words just catch [58:54] interesting is certain words just catch [58:54] interesting is certain words just catch fire because they put a phrase to [58:56] fire because they put a phrase to [58:56] fire because they put a phrase to something people have been trying to [58:57] something people have been trying to [58:57] something people have been trying to explain for a while. And to me, vibe [58:59] explain for a while. And to me, vibe [58:59] explain for a while. And to me, vibe coding is this idea of it's never been [59:01] coding is this idea of it's never been [59:01] coding is this idea of it's never been easier to create prototypes. You can [59:03] easier to create prototypes. You can [59:04] easier to create prototypes. You can just try out ideas and put it into chat [59:06] just try out ideas and put it into chat [59:06] just try out ideas and put it into chat GPT and get a canvas and like see what [59:08] GPT and get a canvas and like see what [59:08] GPT and get a canvas and like see what it looks like or open up cursor and ask [59:09] it looks like or open up cursor and ask [59:10] it looks like or open up cursor and ask it to build an idea and that doesn't [59:11] it to build an idea and that doesn't [59:12] it to build an idea and that doesn't necessarily mean that you have to ship [59:13] necessarily mean that you have to ship [59:13] necessarily mean that you have to ship that code because there there is a long [59:15] that code because there there is a long [59:15] that code because there there is a long tale of software engineering where it's [59:17] tale of software engineering where it's [59:17] tale of software engineering where it's kind of like the iceberg iceberg memes [59:19] kind of like the iceberg iceberg memes [59:19] kind of like the iceberg iceberg memes where it's like on the top it's vibe [59:21] where it's like on the top it's vibe [59:21] where it's like on the top it's vibe coding and then you're like oh wait [59:23] coding and then you're like oh wait [59:23] coding and then you're like oh wait there's actually a lot more to building [59:25] there's actually a lot more to building [59:25] there's actually a lot more to building delivering complicated software like we [59:26] delivering complicated software like we [59:26] delivering complicated software like we saw today with agent kit like there's a [59:29] saw today with agent kit like there's a [59:29] saw today with agent kit like there's a lot that goes into building successful [59:31] lot that goes into building successful [59:31] lot that goes into building successful reliable observable agents. So, I love [59:35] reliable observable agents. So, I love [59:35] reliable observable agents. So, I love that it's kind of opening up the top of [59:36] that it's kind of opening up the top of [59:36] that it's kind of opening up the top of the funnel and making software creation [59:38] the funnel and making software creation [59:38] the funnel and making software creation accessible to more people. And then as [59:40] accessible to more people. And then as [59:40] accessible to more people. And then as they go down, they realize, okay, [59:43] they go down, they realize, okay, [59:43] they go down, they realize, okay, there's there's more complexity here. [59:44] there's there's more complexity here. [59:44] there's there's more complexity here. And I think over time that just evolves [59:46] And I think over time that just evolves [59:46] And I think over time that just evolves into this new world of software that [59:49] into this new world of software that [59:49] into this new world of software that will probably look quite a bit different [59:50] will probably look quite a bit different [59:50] will probably look quite a bit different than the past 5 years. Yeah. As a [59:51] than the past 5 years. Yeah. As a [59:52] than the past 5 years. Yeah. As a writer, we use terms seat of the pancer [59:54] writer, we use terms seat of the pancer [59:54] writer, we use terms seat of the pancer was somebody who would just sit down in [59:55] was somebody who would just sit down in [59:55] was somebody who would just sit down in front of the typewriter and just go and [59:56] front of the typewriter and just go and [59:56] front of the typewriter and just go and tell maybe find a story or if they're [59:59] tell maybe find a story or if they're [59:59] tell maybe find a story or if they're famous horror author have to kill [60:00] famous horror author have to kill [60:00] famous horror author have to kill everybody off halfway into it and then [60:02] everybody off halfway into it and then [60:02] everybody off halfway into it and then come to an ending or you know planners [60:04] come to an ending or you know planners [60:04] come to an ending or you know planners you know and I think that sometimes see [60:05] you know and I think that sometimes see [60:05] you know and I think that sometimes see the pancing is a great way to explore [60:07] the pancing is a great way to explore [60:07] the pancing is a great way to explore and find out new things go I never would [60:09] and find out new things go I never would [60:09] and find out new things go I never would have tried that and did it and then when [60:10] have tried that and did it and then when [60:10] have tried that and did it and then when you sit down and say no I know what I [60:11] you sit down and say no I know what I [60:11] you sit down and say no I know what I want to do I'm going to go proceed this [60:13] want to do I'm going to go proceed this [60:13] want to do I'm going to go proceed this way. [60:13] way. [60:14] way. >> Uh this has been awesome. Thank you so [60:15] >> Uh this has been awesome. Thank you so [60:15] >> Uh this has been awesome. Thank you so much for joining us Lee and people can [60:17] much for joining us Lee and people can [60:17] much for joining us Lee and people can find you know cursor everywhere. It's [60:18] find you know cursor everywhere. It's [60:18] find you know cursor everywhere. It's pretty awesome. highly recommend people [60:20] pretty awesome. highly recommend people [60:20] pretty awesome. highly recommend people play with this and it's been just like I [60:22] play with this and it's been just like I [60:22] play with this and it's been just like I said it's like my favorite thing to do [60:23] said it's like my favorite thing to do [60:23] said it's like my favorite thing to do is just open up some tabs and just throw [60:25] is just open up some tabs and just throw [60:25] is just open up some tabs and just throw some crazy thing in and see if I can [60:27] some crazy thing in and see if I can [60:27] some crazy thing in and see if I can make it. [60:27] make it. [60:28] make it. >> Yeah. Yeah. Thank you. [60:28] >> Yeah. Yeah. Thank you. [60:28] >> Yeah. Yeah. Thank you. >> Have a great dev day. [60:29] >> Have a great dev day. [60:29] >> Have a great dev day. >> Yeah. [60:30] >> Yeah. [60:30] >> Yeah. >> Thank you everybody for joining us. [60:46] [Music] [60:46] [Music] Heat. Heat.