Brex CEO Replaces SaaS Seats With Token Budgets and AI Employees

Brex CEO Franceschi 摒弃软件席位制，转向按量计费的 AI 员工 随着智能体工作流使企业能够销售劳动力成果而非单纯工具，传统软件订阅模式将会瓦解，企业人力薪酬和 SaaS 预算也将直接转向 Token 消耗。 Pedro Franceschi 详细介绍了 Brex 的实践路径，即用专注于执行招聘等特定工作流的“虚拟员工”，取代标准产品研发周期和通用 AI 智能体。他展示了 Brex 内部招聘智能体 Jim、用于防止数据泄露的 CrabTrap 网络代理，以及旨在直接对接企业财务账簿以追踪和对账 Token 支出的 Magpie 分析引擎。Franceschi 认为，现代软件变现模式正从按席位订阅转向销售以 Token 计量的已完成工作成果，同时他还展示了如何通过纯文本 Markdown 文件自主编排其个人日常高管日程与公司项目。 Pedro Franceschi 认为，初创企业应默认构建具有明确角色、技能和管理者的专属“虚拟员工”，而不是构建通用的开放式 AI 智能体。 (0:00) Brex 开发并开源了 CrabTrap，这是一种出站 HTTP 代理，利用静态规则和 LLM 裁判（Claude Sonnet）来检查传输载荷，防止失控智能体泄露企业数据。 (22:33) Brex 构建了 Magpie 以追踪企业、运营和产品层面的 AI Token 消耗，直接将 API 元数据与已清算的企业银行账簿交易进行关联绑定。 (27:30) Pedro Franceschi 透露了基于 OpenClaw 构建的个人高管操作系统“Autopilot”，该系统完全运行在提交至 Git 仓库的纯 Markdown 文件上，可将 Slack、电子邮件和会议纪要汇总为结构化的人员与项目档案。 (40:45)

Brex co-founder Pedro Franceschi argues that the software industry is transitioning from selling tools to delivering completed work through role-specific "virtual employees." He contends that scaling this model requires replacing bureaucratic product management with AI-assisted review loops, securing agent execution at the network perimeter, and actively managing enterprise token budgets. Deploying Role-Specific Employees to Deliver Work (12:32) Franceschi distinguishes generic, open-ended conversational bots from dedicated "virtual employees" that maintain state, integrate into standard communication channels, and execute end-to-end responsibilities. At Brex, this concept is demonstrated by "Jim," an internal AI recruiter built on the OpenClaw framework. Rather than acting as a standard software interface, Jim operates inside Slack and a custom dashboard, automatically syncing with Greenhouse, evaluating inbound applications, and running weekly outbound sourcing loops enriched by applicant data from GitHub, LinkedIn, and X. Candidate evaluations are scored against specific hiring manager rubrics, while human-agent agreement dashboards monitor false-positive and false-negative screening rates. Franceschi asserts that this architecture reflects a broader economic transformation where software vendors stop selling seat-based tools for humans to operate and instead monetize direct work output. By wrapping frontier language models in iterative tool-calling loops, virtual workers complete entire operational workflows, shifting software value toward token consumption markups and direct business outcomes. Compressing Product Delivery and Codifying Organizational Precedent (0:37) Franceschi declares traditional product management practices—such as drafting static Product Requirement Documents (PRDs) and acting as bureaucratic "mini CEOs"—obsolete. Because AI tools enable designers and engineers to prototype and code directly, the organizational mandate shifts to compressing the latency between absorbing customer signals and shipping functional features. Brex product managers now rely on language models to synthesize unstructured data across months of sales calls, RFP responses, and customer support transcripts, validating hypotheses through rapid interactive prototypes rather than speculative roadmaps. To balance rapid iteration with strategic coherence without creating management bottlenecks, Brex treats executive evaluations as a "case law system" where leadership decisions set repeatable institutional precedents. Franceschi and his Head of Product codified their product principles and review heuristics into a centralized GitHub repository of AI "skills." Product managers run their initiatives through these automated pre-review evaluations to verify problem clarity and strategic trade-offs before meeting executives. Similarly, technical hiring has moved away from LeetCode whiteboarding to timed, high-scope "builder loops," where candidates are evaluated on orchestrating AI tools to ship working software under intentionally constrained deadlines. Securing Agent Execution and Governing Token Telemetry (22:33) Deploying autonomous agents with code-execution privileges introduces major operational security risks. Franceschi argues that application-layer guardrails and system prompts are insufficient because arbitrary code execution can bypass framework constraints to initiate unauthorized outbound calls. To establish an enforceable boundary, Brex developed and open-sourced CrabTrap, an HTTP network proxy that intercepts agent traffic. CrabTrap evaluates requests through a two-tiered filter using deterministic static rules alongside an LLM-as-a-judge policy engine. Network policies can be synthesized automatically by replaying approximately 30 minutes of historical staging traffic to enforce semantic least-privilege access, such as granting an agent read-only access to GitHub while blocking repository writes, pirate websites, Okta configuration changes, or access to sensitive meeting notes. Alongside security, Franceschi identifies token spend unpredictability as a primary barrier to enterprise AI adoption. To track cost drift across disparate model providers, Brex created an internal tool, Magpie, which categorizes usage into Corporate AI (internal productivity), Operational AI (internal automation like fraud transaction tagging), and Product AI (customer-facing features). Franceschi outlines a two-phase adoption philosophy: organizations should initially permit unconstrained exploration to drive adoption, followed by surgical spend optimization that targets prompt caching and routing tasks to cheaper models. Brex is integrating Magpie directly into its financial rails to let companies reconcile API token telemetry directly against cleared bank card transactions. Token Budgets, Role Conflation, and Plaintext Workspaces (35:43) Franceschi projects that corporate expenditure will increasingly migrate from traditional employee payroll to token allocation budgets as solo founders and lean teams deploy agent networks. Rather than predicting net job destruction, he envisions a broad "conflation of roles" where high-performing employees leverage coding harnesses to become versatile builders across multiple domains. To illustrate how early this shift remains, Franceschi notes that despite rapid progress among power users, the proportion of the global population effectively using coding harnesses and advanced agents remains an exceptionally small fraction. Demonstrating personal workflow automation, Franceschi runs his executive operations through Autopilot, an OpenClaw system that continuously ingests daily signals from Slack, email inboxes, meeting notes, and internal documents. The system structures context entirely as plain Markdown files stored in a continuous Git repository without a relational database or vector store. Background aggregation jobs update tracking files organized by "People" and "Programs," generating interaction logs, project summaries, and automated draft action items that can be inspected and edited manually. Franceschi frames autonomous agents as viable virtual coworkers only when organizations support them with deterministic network safeguards, granular unit-cost monitoring, and rigorous problem evaluation. A central question remains how effectively non-technical enterprises will adopt these complex frameworks, especially as frameworks like OpenClaw continue to require heavy engineering overhead to operate dependably.

[0:00] Pedro Franceschi: why shouldn't it just be you and agents? [5:23] Pedro Franceschi: we manage the work not the people and we manage people around the work versus the other way around. [8:42] Pedro Franceschi: winning starts way upstream. [18:22] Peter Yang: This is like a better Greenhouse, dude. This is what Greenhouse should have built, [21:24] Pedro Franceschi: The presence of it is a signal. The absence of it isn't a signal. [27:04] Pedro Franceschi: the wall that every company has when adopting AI. [27:04] Pedro Franceschi: Well, it depends on what they're doing, which is what this helps you do. [31:05] Pedro Franceschi: The same way you have spend management, you need to have token spend management... spend management has to understand AI natively, which is what we want to go build with this thing. [35:43] Pedro Franceschi: Why shouldn't it just be you, right? And a bunch of... and a big token budget and agents. [37:33] Pedro Franceschi: everybody's going to be a builder to some degree. [44:27] Pedro Franceschi: to consume the entire state of my life into one system. [44:27] Pedro Franceschi: acquired taste when it comes to making work reliably. [45:33] Pedro Franceschi: sell the work itself versus just sell the software.

[0:00] If someone's starting a company today, I [0:01] think the question is like, why [0:02] shouldn't it just be you and agents? The [0:04] best way to go about it is to focus on [0:06] this idea of building virtual employees [0:09] versus open-ended agents that could do a [0:11] million things. The fundamental thing [0:13] that we spent time on was how to make it [0:15] behave, look, and feel like a real [0:17] person. Jim is our open cloud recruiter. [0:20] It does like effectively three things. [0:21] It sources people, it filters all [0:23] inbound, and third, it serves as an [0:25] analytics layer. It's just like actually [0:27] behaving like a coworker, which is like [0:29] coolest thing I think here. You're now [0:31] able to sell the work itself versus just [0:33] sell the tool. [0:37] All right. Hey everyone, my guest today [0:38] is Pedro, co-founder and CEO of Brex. [0:40] Pedro is one of the most AI pill CEOs [0:42] that I've ever met. So really excited to [0:44] talk to him about PMing and also how [0:46] Brex uses agents internally and also how [0:48] he personally uses AI. Welcome, sir. [0:50] >> Thanks for having me. Excited to be [0:52] here. So, so dude, like a couple months [0:53] ago while I still have my PM job, uh you [0:55] tweeted this, you tweeted that the PM [0:58] playbook of writing PRDs and aligning [0:59] stakeholders is dead [laughter] and and [1:02] I think many PMs wish this was true, but [1:04] like [1:04] >> yeah, [1:05] >> is this true that can you unpack this a [1:07] little bit or [1:07] >> Yeah, I mean look, I think I think we've [1:09] seen that over and over again the the [1:13] job of a traditional PM actually taking [1:15] a step back the way I describe the [1:17] quality of an engineering uh in a [1:20] product organization is how quickly it [1:23] takes between ingesting some signal out [1:26] there in the world about your customer [1:28] about a painoint about an opportunity [1:31] internalizing that into uh a product or [1:35] or a feature or whatever it is and then [1:37] shipping something that works and [1:38] addresses the pain point and put it in [1:40] front of a customer. So what you're [1:41] trying to do is compress cycles in that [1:43] loop and and of course a lot of the job [1:46] of a PM uh traditionally was spend [1:49] ingesting tons of information uh making [1:52] trade-off decisions and and just like [1:54] actually prototyping things right and [1:56] building things and we we bra a lot of [1:59] the role of PM evolved over the past [2:02] couple of years where for a long time I [2:04] would say maybe five years ago I think [2:06] there was this view that PMs ran the [2:08] show and PMs were you the ones designing [2:11] the product. They were thinking about [2:12] how the trade-offs of how things work. [2:14] They were debating which opportunities [2:16] were. There was this whole like PM is [2:17] the mini CEO. Uh and we actually don't [2:20] believe in that like the way we see the [2:22] world is like you need you need the [2:24] craft of engineering and design to be [2:26] equal footing with PM uh in order for to [2:28] make the model work. And so so so a lot [2:31] of what we've done at Brex is so this is [2:33] maybe the backdrop and then a lot of [2:34] what we're seeing with AI is the [2:36] conflation of these roles. So the lines [2:39] between what was traditionally [2:40] engineering or product or design I think [2:43] is getting more and more blurred and you [2:45] see PMs writing code you see designers [2:47] you know writing code and building [2:48] things you see engineers making [2:50] decisions that with data and context [2:52] that would traditionally be in the PM [2:54] realm so so we're seeing a lot of that [2:56] and I would say PM specifically at Brex [2:59] became a lot more about how do you use a [3:01] lot of the tooling that we've built [3:02] internally to just ingest context about [3:05] customers and and and and really what's [3:07] happening happening on the ground, [3:08] especially with the sales team. Um, and [3:10] then based on that information, make [3:12] trade-off and prioritization decisions [3:14] that affect what we build. Um, and then [3:16] of course, this is maybe the sort of the [3:18] first half of the job. The second half [3:19] of the job is making sure the things you [3:20] build uh are actually great and address [3:23] the the very pain that you were you're [3:25] addressing to solve. So a lot of the [3:27] things that we see PMs doing now is just [3:30] you know like ingesting I don't know six [3:32] months years of u gong calls for example [3:36] making sense of that in a very clean way [3:39] um unearthing insights from unstructured [3:41] data uh presenting and validating those [3:44] hypothesis with very cheap and easy to [3:46] do prototypes and then whenever we go [3:48] build something we have so much [3:49] conviction it's the right thing that you [3:52] just compress the cycles a lot and you [3:53] end up shipping the right thing in the [3:54] beginning. [3:55] >> Got it. Yeah, that makes sense. Yeah, I [3:56] was talking to a PM on the Codex team [3:58] and he said, you know, the these days [4:00] engineers are empowered to just like [4:01] build their own features and actually [4:02] talk to customers too. So, yeah, he [4:04] spends a lot of time with the enterprise [4:05] uh folks just like understanding what [4:07] what they want and it's just trying to [4:09] manage all the stuff that the [4:10] engineering actually wants to ship. It's [4:11] like it's like so much more stuff [4:13] shipping now, right? Because it's like [4:14] so much easier to build. [4:15] >> 100%. And and and I think it's funny [4:17] because I I I actually think because [4:20] it's so easy to build things, uh having [4:23] discernment on what matters I think is [4:25] like more important than ever. Um and [4:27] and I think a lot of what we we we push [4:31] on the teams is is just this idea of [4:33] like [4:35] what is a true lever to make this thing [4:38] move uh and and actually deliver value [4:40] in a unique way and let's go do this one [4:42] thing even though we could do five or [4:43] 10. Um we really believe in this idea of [4:46] like minimizing the surface area going [4:48] really deep in solving one thing really [4:50] well um and then sort of building around [4:52] that. So so for us it's been a it's been [4:54] a journey of just uh seeing this over [4:56] and over again and uh you know at the [4:58] end of the day even though you can build [4:59] a lot more things uh I think leadership [5:02] bandwidth is something that concentrated [5:05] always yields like better outcomes I [5:06] think. And how do you let me just uh [5:08] push a little bit on the stakeholder [5:09] alignment thing because I I I still [5:11] think like after a certain size it's [5:13] kind of hard not to like you don't want [5:15] you don't want your PM just like ships [5:16] of themselves, right? There's probably [5:18] some sort of review process but like at [5:20] the same time you don't want people to [5:21] wait like a month to meet with you. So [5:22] how do you [5:23] >> Yeah, I [clears throat and laughter] [5:24] mean a lot of it for us was was I would [5:26] say was was two things. one is um I I [5:31] really believe that the I would say I [5:34] probably spend half of my job my time [5:36] just reviewing work. So, so, so this is [5:39] just the nature of the job because I [5:41] think the way you set a high quality bar [5:43] is by, you know, effectively building a [5:45] case law system inside your company [5:47] where you have, you know, I would say [5:49] the Supreme Court, which is when [5:51] something comes from me to review where [5:53] you're setting the standard for what [5:55] great looks like and then you have [5:56] reviews that teams are doing internally [5:58] all the way down to like an IC building [6:00] something themselves and understanding [6:02] ultimately what the task that this thing [6:03] needs to needs to needs to sort of, you [6:05] know, withstand. And I would say I think [6:07] that the biggest thing is you know you [6:09] just have to engage with the work [6:11] directly right I think like one of the [6:13] mantras that we have at Brex is like you [6:15] know we manage the work not the people [6:17] and we manage people around the work [6:18] versus the other way around. And I think [6:20] a lot of reviews for us now they you [6:23] know when we do a review we have super [6:24] two sort of types of reviews at Brex uh [6:26] the things we call problem alignment and [6:27] solution alignment. And problem [6:29] alignment is like what is the problem [6:30] we're solving and there's like a lot of [6:32] things that we could go about solving uh [6:33] in a space. So let's say you see like [6:36] retention is maybe less strong than we [6:38] would like on a specific feature or a [6:39] product. Then there's a like a lot of [6:41] the 50% of the problem is like which [6:43] problem is worth solving and let's make [6:44] sure we're align on that. And and a lot [6:46] of what AI helped us is is make sense of [6:49] like immense amounts of unstructured [6:51] data. So you know when you know when a [6:54] rep is in front of a customer and you [6:57] know we may be losing a deal for example [6:59] like why are we losing the deal? Like [7:00] what are the features that are [7:01] mentioned? what are the things that are [7:02] not mentioned but you know we can [7:04] understand that the customer cared about [7:07] based on something else they asked based [7:08] on an RFP based on like a lot of [7:10] different signals um and and and that is [7:13] something that PM I think before didn't [7:15] have access to and and now obviously do [7:17] um so so that that's like a big part and [7:19] then when you are uh uh on a review a [7:22] lot of the work is just u how do you how [7:24] do you build a hypothesis around that [7:25] and of course LLMs help on that but but [7:28] I think the judgment plays a little bit [7:31] of role and saying here's how this [7:33] aligns to what we're trying to do as a [7:35] company more broadly uh versus you know [7:38] doing the sort of paper pushing of [7:39] understanding and digesting data and [7:41] making it suit. [7:42] >> Some of the exacts have like skills for [7:44] themselves to that their PMs can run to [7:46] get the initial pass. Do you have [7:48] something like that? [7:48] >> Yeah, we we do we do we have a we have a [7:50] repo of like um I mean it's it's a it's [7:54] a we have all the skills that a product [7:55] or uses in a repo and we try to keep it [7:58] pretty updated. So me and my head of [8:00] product, we spend a bunch of time making [8:02] sure those things are correct. So when [8:04] we when someone gets a review, they [8:06] already on that skill. Um, and it's like [8:08] it's it's it's I would say relatively [8:10] digested already. [8:11] >> Yeah. [8:12] >> To a point where the things that really [8:14] spend spending time are things that are [8:16] idiosyncratic to that problem versus [8:18] okay, let me just apply the same [8:20] algorithm I would apply on anything that [8:22] shows up in front of me. Uh so [8:24] absolutely. So, so and like uh the skill [8:27] is basically like basic stuff like um [8:28] what your principles are like is the [8:30] problem clearly stated like that kind of [8:32] stuff right like PM1 stuff. [8:33] >> Um I think I think I think it's like I [8:35] think it's a little bit of that. I think [8:37] it's like also how do you codify the [8:40] ethos of what makes a good product from [8:42] the perspective of your company and your [8:45] customer. A lot of it for us is honestly [8:48] about go to market is like I think like [8:50] I think PM at Brex like plays less of a [8:53] role as like the actual craft of [8:56] building and probably a little bit more [8:57] of a role as being the bridge of go to [9:00] market and understanding which problems [9:02] we're solving and and for example a lot [9:04] of the times we spend time on you know [9:08] even understanding how how is this [9:10] affecting the reality on the ground of a [9:13] goto market team much more than what's [9:15] happening inside the product. Got it? [9:17] >> Because you know at the end of the day [9:19] the [9:20] winning is winning starts way upstream, [9:23] right? It's not just something that [9:24] happens when you're using a product. So, [9:26] so we spend a bunch of time there. I [9:28] would say more more specifically um we [9:33] do have a bunch of things that we always [9:34] ask people right it's like you know how [9:37] did you think of trade-offs on this [9:39] thing like you know what what were the [9:41] alternatives like what's the actual [9:43] bottleneck limiting the rate of progress [9:44] of this initiative why why why are we [9:48] how is this creating more throughput on [9:50] the bottleneck there are things like [9:52] that that we ask but um but you know we [9:55] we spend a lot of time just clarifying [9:56] what are the common patterns that we [9:58] based on, you know, a bunch of meetings [9:59] and a bunch of reviews that we have, you [10:01] know, transcripts on. [10:03] >> This episode is brought to you by [10:05] Riverside. I've used Riverside for years [10:07] to record my podcast because it records [10:10] in 4K [music] [10:11] resolution each person locally. So, the [10:14] audio and video still comes through [10:15] clean, even if a guest's Wi-Fi gets [10:17] shaky. But the reason I love it now more [10:19] than ever is what happens after we stop [10:22] recording. If I go in here that I can [10:24] use these AI tools to remove pauses, [10:26] remove filler words and just clean up [10:28] the recording. And I I can also edit the [10:30] transcript directly and it will [10:32] automatically generate clips with [10:34] captions ready to publish to YouTube, [10:36] Spotify, and all types of social media [10:39] platforms all from one place. As a [10:41] oneperson creator business, that matters [10:43] a lot. Riverside is the upgrade your [10:45] content workflow needs. Try it at [10:48] creators.side.com/peryang [10:49] riverside.com/peteryang [10:52] and use code peteryang at checkout to [10:54] get one month completely free. That's [10:57] creators. [10:59] Peter Yang. Now, back to our episode. As [11:02] a PM, I just want to talk to users and [11:04] like, you know, welcome engineers and [11:06] craft something great, right? But like [11:07] there's a lot of other stuff, dude. Like [11:08] you got to hire PMs, you got to write [11:10] performance reviews, you got to do all [11:11] this other crap. [laughter] [11:12] And like you actually have some really [11:14] cool agents or tools maybe you can show [11:16] us like that does some of this or helps [11:17] on some of the stuff. Yeah, happy to. So [11:20] we we spent a lot of time um thinking [11:22] about how to actually um go about 10xing [11:26] the productivity of non-engineering [11:28] teams of Rex and because I feel like [11:30] where where most people are in AI now is [11:32] you have like the sort of token masters [11:35] 10x engineers that are spending a lot of [11:37] time in a coding harness uh and and [11:40] becoming sort of hyperproductive and [11:42] then you have you know people that are [11:44] using you know basically like you know [11:48] cloud code or codeax X and or SHP with a [11:51] bunch of MCPS and the question is like [11:53] what is how do you bridge the gap [11:55] between the non-engineering teams and [11:56] the engineers and from a productivity [11:59] standpoint and and and a lot of the [12:01] belief that we had Rex is like [12:03] effectively that the best way to go [12:06] about it is to focus on this idea of [12:09] building virtual employees versus like [12:12] you know sort of open-ended agents that [12:14] could do a million things and and and [12:16] and really the I would say the [12:18] fundamental thing that we spent time on [12:21] when we when we sort of went down this [12:23] route was how to make it really easy to [12:26] make it behave, look, and feel like a [12:29] real person. Um, so I can show you like [12:32] an example here of uh the way this [12:35] works. So this is a this is Jim. Jim is [12:38] our um openclaw recruiter. So basically [12:41] it runs I mean he he posted we have a [12:44] channel called welcome to bra where any [12:45] new employee posts you know their their [12:47] their sort of welcome to the company and [12:50] this is what Jim posted. [laughter] [12:52] Uh but anyway Jim is like an actual like [12:55] it looks and feels like an actual [12:56] person. So uh so you can actually like [12:59] go and talk to Jim um you know and and [13:02] effectively interact with it like like a [13:03] normal recruiter. and um and you know [13:06] it's been running since February [13:07] actually uh and it runs on effectively [13:10] like a three-stage pipeline where you [13:12] know it first syncs the greenhouse uh [13:14] which is our ATS uh then we have of [13:16] course you know uh notes and roles and [13:20] applications in that system um then it [13:22] ingests candidates and parses resumes [13:24] and rich LinkedIn profiles and fetches [13:25] even GitHub data to complete a full [13:27] picture on the applicant and manager [13:29] evaluates every candidate against like [13:31] you know ro specific criteria um So, let [13:34] me see if I can show you this just the [13:36] way uh the UI works on Jim. Yeah, while [13:40] it's loading. Uh, but the cool thing is [13:42] like because it's been running for a [13:43] while, we can actually just ask [13:45] questions about it um and see what [13:48] happens. So, like for example, like if I [13:50] ask like uh you know Jim, you know, what [13:54] criteria changes have you made to the [13:56] you know, senior software engineer AI [13:58] role based on feedback. [13:59] >> Uhhuh. [14:00] >> And this is all just open claw running [14:02] on a harness. And think of it as like a [14:04] set of skills deployed with like state [14:07] right so you have like state of [14:08] candidates that are you know in process [14:10] you have connections greenhouse but it [14:12] it's just a collection of skills and [14:15] a fair amount of infrastructure that we [14:17] build around eh halls to make sure this [14:19] is actually behaves correctly. Uh let's [14:22] see what Jim finds. I don't know what [14:25] this warning is but certainly [14:27] interesting. [14:28] >> Yeah because I I really dislike using [14:30] the greenhouse UI. So anything that [14:32] [laughter] that I don't have to use it [14:34] >> green greenhouse UI is not not something [14:36] we like. Um well well while it does that [14:39] I can just show you the UI so you can [14:40] get a sense for the way this actually [14:42] works in practice. This is like [14:44] basically the web UI that Jim exposes [14:47] and you can basically see here everyone [14:49] that needs action um and how they found [14:52] Brex. You can see you can go here and [14:53] see applicants uh and and you can also [14:56] see job posting. So it all starts with a [14:58] job posting and you can see here like a [15:00] staff rock designer role or you can go [15:02] into like let's say an engineering role [15:06] uh let me see [15:09] uh like this role for example uh there's [15:12] people advanced a lot that we rejected [15:14] um and we can see here you know sourcers [15:17] we can also create a custom criteria for [15:19] a candidate let me find someone that has [15:20] a custom criteria [15:22] uh here but you can basically come in [15:24] here set a criteria and then we're going [15:26] to evaluate candidates it's against this [15:28] uh this spec. And if I go here into uh [15:32] applicants, I can effectively filter for [15:34] uh let me get let me actually source [15:36] this by uh confidence score. [15:40] Um and you know, for example, this [15:43] person here, you know, it's an [15:44] exceptional fit for senior front end [15:46] because they've been, you know, [15:47] Microsoft for a long time and Coinbase. [15:50] Uh and we can effectively see, you know, [15:52] the criteria score, where the career [15:54] timeline came from. we can see the [15:56] original uh resume from the person where [16:00] they applied uh and uh you know this [16:03] basically just saves a bunch of time [16:04] from recruiting and the agent is doing a [16:06] bunch of sourcing uh and flagging things [16:09] that recruiting should do. So the idea [16:10] is you can just create a job posting and [16:13] say hey Jim go about it and then Jim's [16:14] going to come back with uh with [16:16] candidates to go in and spend and spend [16:18] time doing it. We we spent a lot of the [16:20] time just you know we put a little bit [16:21] of the infrastructure around it but a [16:23] lot of the idea of this thing was how [16:25] can we make this like mostly self [16:27] bootstrapped u so we spend some some [16:29] time building an infrastructure around [16:31] you know defining you know what it means [16:33] to actually score a candidate against [16:35] against a job posting for example um [16:38] >> but other than that like a lot of this [16:40] was just like evolves and telling the [16:43] model and prompting and saying hey like [16:44] this is the wrong decision you should go [16:46] and change this and change that and uh [16:48] and if you go here into the Slack [16:50] channel. You can see people interacting [16:53] with Jim in a pretty clean way. So, uh [16:56] here's a recruiter talking to Jim. Uh [16:58] and you can see here like, hey, which [17:00] role has the highest average candidate [17:02] score? Uh you know, are there any [17:04] outliers? And he's like showing us like [17:05] the roles. Or I can see top 30 [17:08] applicants for this role. And this is [17:10] just like giving us like a bunch of [17:11] context on people. Um and it also posts [17:14] like anyone that's like a standout [17:16] candidate here with like LinkedIn links. [17:18] So it's just like actually behaving like [17:20] a coworker which is like kind of the the [17:22] the coolest thing I think here. It also [17:25] has like a self improvement loop where [17:28] you know it just goes through like [17:29] things that we should be building to [17:31] automate more of the process and then [17:34] you can go in and say you know here's a [17:35] step to go and implement this. So the [17:38] idea is like you can just you can just [17:40] do a lot of that by building the right [17:42] crown jobs and skills and and we do use [17:44] open claw for this but the harness [17:45] doesn't really matter as much as just [17:47] the skills. So you can see how this [17:49] creates a bunch of leverage for [17:50] recruiting. [17:51] >> Okay. So recruiting can just like uh [17:52] live in Slack instead of green [17:53] greenhouse. They can just like [17:55] >> exactly [17:55] >> ask questions. Yeah. [17:57] >> Exactly. And these are like people that [17:58] apply now for example. [18:00] >> So Jim is basically doing the upper [18:02] funnel pass like uh filtering inbound. [18:04] Does Jim source people too? Jim does [18:07] source people too. Uh it does it does [18:09] like effectively three things. It [18:10] sources people. Um it filters all [18:14] inbound. It it started by filtering [18:15] inbound and finding signal there. Uh and [18:18] third it serves as an analytics layer. [18:22] So you can just effectively go in and uh [18:25] and uh you know enable autosourcing on [18:27] the role. So let me just see if I can do [18:29] that now. So if you go into job posting, [18:32] so you can just come in here for example [18:34] and say yeah like enable I'm not going [18:36] to disable it but you can just like [18:38] source candidates weekly for this role [18:40] and this is going to go in and like find [18:42] people using the job description [18:43] evaluation criteria and sourcing [18:44] criteria. So we're going to go run the [18:46] evaluation criteria against uh [18:49] candidates and uh make sure that uh we [18:52] are only reaching out to people that [18:54] actually feel like a fit for example. So [18:57] this is just one example and again we're [18:58] not in the business of building [18:59] recruiting tools but this is just [19:01] something that we build that is in our [19:03] opinion what a virtual employee should [19:04] look like. [19:05] >> Okay. So Jim will actually DM people on [19:07] LinkedIn like hey you know uni from [19:09] page. [19:10] >> Yes. Yes. Uh and and you know we there [19:14] there's a bunch of things we do to [19:16] >> make that email be and feel like a a [19:20] person. But you know, yes, we we do [19:22] spend a lot of time uh [19:24] >> you know, making sure it can send on [19:26] respon um it's emailing on behalf of a [19:28] recruiter or sourcer. You know, there's [19:30] ways to you know, tell who is the [19:31] sorcerer, who is a recruiter here. So, [19:33] um anyway, there's like there's like [19:35] ways you can just see how this thing all [19:36] like compounds and you know, becomes uh [19:39] yeah, you can see here metrics as well. [19:40] So, anyway, it's a pretty [19:42] >> Yeah, this is like a better greenhouse, [19:44] dude. This is what greenhouse should [19:45] have built. [laughter] [19:47] >> Yeah. Anyway, this is just this is just [19:48] one example, but I think it shows like [19:50] the the power of what you can do with AI [19:52] and like and obviously like you know we [19:54] have a team of like agent builders that [19:56] builds things for like a lot of teams in [19:57] the company but this uh oh nice [19:59] >> you know creates creates leverage for [20:02] one single use case right but uh but it [20:04] all you can sort of see it all playing [20:06] out with uh with you know it's sending [20:09] emails it's doing a bunch of things that [20:11] most people would be like okay this [20:12] feels like you know it's pretty uh uh [20:15] risky um which actually gets us to how [20:18] we make it secure to deploy this thing [20:20] because most people would say, "Oh gosh, [20:22] you have like a thing emailing [20:24] candidates and like doing things that [20:26] feel pretty risky on behalf of real [20:28] people and uh that's uh that's like, you [20:31] know, a lot worse than time of actually [20:32] crab trap." [20:33] >> I feel like LinkedIn is basically like [20:35] all AI now anyway. So, it's not [20:37] [laughter] big a deal. The gym is really [20:39] >> Well, it definitely helps. I mean, to be [20:41] honest, it doesn't actually use LinkedIn [20:42] as much. It uses a bunch of other data [20:44] sources. GitHub, Twitter, like is it's [20:46] it's LinkedIn is like more where we get [20:49] a profile versus where we go about, you [20:52] know. [20:52] >> Yeah. I want to ask you this actually [20:53] like just to close this hiring thing. I [20:55] I feel like a lot of people these days [20:57] are a lot of AI builders I talk to, [20:59] they're making a bunch of open source [21:00] contributions on GitHub and stuff. [21:02] >> Yeah. [21:02] >> Like like when you hire engineers, do [21:04] you [21:05] >> because LinkedIn is just like people can [21:06] make up on LinkedIn, right? [21:08] Like do you think the open source stuff [21:09] is very important when you hire? [21:11] >> I don't think I don't think open source [21:12] is that important. I I mean I think it [21:13] can be important but like I would say [21:16] the presence of it is a signal. The [21:17] absence of it isn't a signal, right? So [21:19] it's like so I would say people that [21:21] don't have it I think we our our [21:24] interview process is pretty much a [21:26] builder loop like you just have to go [21:27] build something uh when you interview as [21:29] an engineer. So if you don't know how to [21:31] use tools and have a good sense for you [21:34] know how to just be 10x more productive [21:35] you just cannot pass the interview like [21:37] you're going to fail because it's just [21:38] too much work to do manually or do [21:40] without AI proficiency. [21:42] >> Okay. Okay. So, no more elite code is [21:44] it's actually building stuff with AI. [21:46] >> That's right. Yeah. And this is just one [21:48] example of like a lot of a lot of where [21:49] we spend time is like how how do you [21:51] create like measurement between like [21:53] what humans are doing and the agents [21:55] doing. So, this is like you know how [21:58] many like how many missed recruiters [22:00] like you know uh uh recruiters actually [22:02] passed the candidate and Jim rejected [22:05] and this is like people that Jim said [22:07] that were actually standouts that [22:09] recruiters actually didn't reach out. So [22:10] you're basically monitoring the [22:11] agreement rate between both. So anyway, [22:13] you can just see how this thing [22:14] compounds. But all these things are [22:16] things that we built uh in service of [22:18] you know creating leverage for uh [22:19] internal employees. [22:20] >> Yeah. Yeah. You can make it better over [22:22] time. Yeah. [22:22] >> Yeah. [22:23] >> That that's amazing too. That that [22:24] that's amazing. Um why don't we talk [22:26] about um some of the other problems of [22:29] enterprises? [22:30] >> Yeah, I was talking about security. [22:31] >> Yeah. Because that that's actually well [22:33] there's actually two things we should [22:34] talk about. One is security, the other [22:35] is cost controls because this is all you [22:37] know fun and games until you know you [22:40] have a security issue and uh and [22:42] actually that's where we build crab [22:44] trap. So the idea of crab trap and and [22:46] actually I can just show you the uh the [22:49] GitHub repo uh because [22:51] >> yeah crab trap is open source right so [22:52] any company can use it. So, so approp's [22:54] open source and really what it does like [22:56] is this like proxy that like intercepts [22:58] uh requests um and and and the idea is [23:01] like when you when you have an agent in [23:03] production the only way to secure the [23:05] agent in our opinion is to actually do [23:07] something the network boundary because [23:09] you know you see a lot of like Nemo claw [23:11] and a lot of these like you know [23:12] initiatives that happen in in the [23:14] community around like constraining tools [23:16] and things like that but at the end of [23:17] the day if if the if the agent has the [23:19] capability of running code which we [23:21] think it should um it can always like [23:24] cross a network boundary and do [23:25] something wrong. So, so what we've done [23:27] is like how do you actually create you [23:29] know uh uh an HTTP proxy that sits [23:31] around the agent and controls all the [23:34] network traffic and actually that's [23:35] actually how a gym works. So the way Jim [23:37] works is we have a crab trap here and [23:40] I'm just going to show you in uh in uh [23:42] audit trail mode first where there's a [23:45] bunch of users. You can just like filter [23:46] for Jim's user and this is actually [23:48] showing like basically all the network [23:50] traffic that is going through Jim. And [23:53] you see here that there's like a bunch [23:54] of like requests and this is actually [23:55] live. So you can see this actually [23:56] happening now. Um, and you can see here [23:59] like the approver for these transactions [24:01] like an LM static rule, [24:03] >> but I can go here and say I just want to [24:05] see see things that actually went [24:06] through the actual uh an actual LM as a [24:09] gateway and and the LM here what it is [24:12] doing is actually um behaving as a judge [24:16] and we're screening a request against um [24:20] you know the LM. So basically this is a [24:22] get request to list emails account and [24:25] this is consistent with the policy that [24:26] we created for Jim and uh and basically [24:29] what Jim is operating under is uh this [24:33] policy. So let me just find Jim here and [24:36] uh there is this production policy in [24:38] place and this policy is actually like [24:41] how we control the traffic of the agent. [24:43] So on every request we're first applying [24:46] the set of static rules of here are [24:48] things that we believe should be always [24:50] allowed for the agent to go and do and [24:52] then here are here is the prompt of what [24:56] everything that is not a static rule [24:57] should go through this prompt and we're [24:58] actually running every request against [25:00] an LLM and actually measuring uh whether [25:03] it passes or not. So the way it starts [25:05] you you don't have to come in and write [25:07] the policy yourself. What you can do is [25:09] you can you can just come in here and [25:11] actually go on Jim's user and say, you [25:15] know, suggest a policy based on the HTTP [25:17] traffic. And what it's going to do is [25:18] actually replay the traffic and uh and [25:22] use another LLM to summarize all the [25:25] network traffic that went through this [25:26] agent uh and build a policy based on [25:29] that. And what you're going to end up [25:31] with and this is going to take like [25:32] probably 30 minutes to run. Uh you're [25:34] going to end up with something like this [25:36] which is you know it observed the [25:37] traffic and created all these rules. So [25:40] this is just effectively based on the [25:42] network traffic that this agent [25:44] observed. So you can see how this thing [25:46] becomes pretty robust as you start to [25:49] have evolves to start to replay network [25:51] traffic and block things that are not [25:53] allowed in a way that you wouldn't be [25:56] able to do uh traditionally just by [25:58] deploying an agent and saying yeah you [26:00] know the world's your oyster go crazy [26:02] >> and I'm reading the prompt right now. So [26:03] is a lot of it just like blocking [26:04] certain domains or like allow listing. [26:06] Uh yeah and and and you know and for [26:08] example saying you know here's what you [26:10] can do in GitHub you can like you can [26:12] research candidates but you cannot like [26:14] go in and [26:15] >> post on any other repo you cannot like [26:18] push on a repo that is not like the claw [26:20] right and and traditionally you cannot [26:22] do this if allow list because that's [26:24] actually reasoning about the nature of [26:26] the request [26:27] >> uh so that's that's like a lot of the [26:29] things that we we end up spending a lot [26:31] of time doing [26:33] >> um you know here's like an example what [26:34] it cannot do you cannot like delete [26:35] GitHub repost cannot change octa [26:37] configurations. It cannot change [26:38] policies and if it tries to do any of [26:40] these things are just going to block [26:41] them. [26:42] >> So if Jim goes to the piratebay.com it [26:44] will get blocked. [26:46] >> That's right. [26:47] >> Yeah. [26:47] >> Yeah. So for example, it was trying to [26:48] access granola nodes and uh granola is [26:51] blocked. [26:52] >> So it can go in here and actually go in [26:54] and allow granola. But um for some [26:57] reason it's not allowed and I don't know [26:58] why we would allow it or not. But this [27:00] is something that like you know it ran [27:02] against sonnet. It took us, you know, [27:04] two seconds to run it, thousand tokens [27:06] in, 104 tokens out, and uh, you know, it [27:08] blocked the request. So, this is what we [27:10] do to like actually secure agents in [27:12] production. Um, which is a huge part of [27:14] what it takes to make this work. [27:16] >> Is is this whole interface also open [27:18] source? So, I can set this up too. [27:19] >> Yeah, this is open source. You [27:20] >> want to? [27:20] >> Okay, great. [27:22] >> Yeah, this is great, dude. Yeah. [27:23] >> Yeah. So, so anyway, this is this is the [27:25] maybe the second thing we had to go [27:27] build, which is like a way to secure [27:28] agents in production. Um and then the [27:30] third thing we had to build is a way to [27:32] track costs [laughter] [27:33] and this is funny because uh tracking [27:36] costs is part of our business right like [27:38] Brex is a spend management company and [27:40] you know fundamentally I would say we [27:43] think that tokens are a massive uh well [27:46] will be even more a massive uh lever of [27:50] cost in a company and uh like many [27:52] customers are spending a lot of money [27:54] across you know anthropic openi bedrock [27:58] talk, cursor, codecs, like you name it, [28:00] right? Like all the all the sort of [28:02] harnesses and and and models and uh and [28:05] so what we had to do is like we just [28:07] couldn't really understand how pricing [28:10] had drifted on tokens based on the use [28:13] cases that we had. So and and we think [28:15] this is the wall that every company has [28:16] when adopting AI. So so really what we [28:19] did is we ended up building a tool uh [28:22] that we call Magpby that actually helps [28:25] you control uh token costs. Um so I'm [28:29] going to show you what we've done. [28:31] >> What's your general philosophy on this? [28:32] Like do you think uh every token maxing [28:33] engineer is like doing a 10x job or like [28:35] what's your [28:36] >> uh well it depends on what they're doing [28:37] which is what this helps you [laughter] [28:38] do. [28:39] >> Okay. [28:39] >> So so so the so the question is like how [28:41] do you actually go about understanding [28:44] token consumption in your company? [28:46] >> So this is a this is like a very this is [28:50] some of this is demo data. We should [28:51] figure out how toize this but uh I'll [28:53] just give you a sense for for how this [28:55] thing works. So basically the idea is [28:56] you can see here like effectively like [28:58] three pillars of AI usage. Uh you also [29:01] have a leaderboard. If you love [29:03] leaderboards yes you can you can go and [29:04] do that. Uh but the three pillars that [29:06] we have first one is like corporate AI [29:08] and this is like how are people using AI [29:10] internally at Brex to be better at their [29:12] jobs. Operation AI is how we use AI to [29:15] better serve customers when it comes to [29:17] driving more growth or automating [29:19] processes. And product AI is the actual [29:22] product we ship and uh how AI um affects [29:26] like literal features in the product. Uh [29:28] and a lot of what we've done is is [29:30] actually like think about how to create [29:33] the ability to drill down really deep [29:35] into token consumption in a granular [29:37] way. So for example, let's say I want to [29:39] go here in operational AI and understand [29:41] what are different callers that are [29:43] using models and what are these things [29:45] are doing. So, for example, if I want to [29:47] go into like transaction tagging is a [29:48] very important thing we do, you can see [29:50] that this thing costs 10 cents a call [29:53] today and we spend like 28,000 roughly [29:55] on this thing a month. Um, and then you [29:58] can go here and disputes and see that [29:59] this cost like two bucks uh [30:01] transactions. It's actually pretty [30:02] expensive. It's 240x difference. So, uh [30:05] you can go in here and try to [30:06] understand, okay, what are things that [30:08] actually driving this thing up and down [30:09] and and you can see here a chart. Uh [30:12] there's another way here I forgot how to [30:13] do it where you can see like caching uh [30:16] which affects like performance and cost [30:18] pretty aggressively. [30:19] >> Okay. So so so this is actually tagging [30:20] customer spend variables right? [30:22] >> No this is transaction tagging for like [30:23] underwriting for example uh in fraud. [30:25] Okay. [30:25] >> Uh is it making sense of like you know a [30:28] very unstructured transaction. Um, but I [30:30] think that's cool is that like you can [30:31] see here that like the number of calls [30:33] that we're making is like we're making [30:35] 329 calls and each call is costing us [30:37] like you know roughly 15 bucks or or [30:40] nine bucks and this cost like really [30:42] skyrocketed over the past like two weeks [30:44] for this specific thing. If you just go [30:46] through like the the cost per call which [30:47] is this line and you can see here that [30:49] like if you go through transaction [30:51] tagging this has been pretty stable over [30:53] time. So something changed here that is [30:55] driving this cost to go up. And I can [30:57] imagine doing this across like [30:58] everything you have and actually being [31:00] able to plot and see which services are [31:02] consuming the most amount of traffic and [31:03] why. Um [31:05] >> and uh and and and and this is like [31:08] something that we're actually really [31:09] excited about because this is uh this is [31:12] actually something that will evolve into [31:14] a breast product versus just an internal [31:16] tools because um the thing is like this [31:19] is just a usage side but then the [31:20] important side is actually reconciling [31:22] to the dollars that go out of your [31:24] company's bank account and we own that [31:26] data right because we are the card and [31:27] we are the rail. So when you put both [31:29] together, the idea is like how to get AI [31:32] costs that's grounded in the money that [31:34] actually clears and leaves your account. [31:36] Um, so the idea is like every company [31:38] that spends on Brex will very soon be [31:39] able to have this whole thing for their [31:42] own AI spend. And of course you can [31:43] still go in here and compare, you know, [31:46] usage by, you know, model and and [31:48] employee and all that, which, you know, [31:50] we think is really cool. You can see [31:52] who's token maxing and all that, which [31:54] is [snorts] great. But I would say at [31:56] the same time the the real value is just [31:59] building this like entire layer of [32:00] analytics where you can see the entire [32:02] thing together into model use case uh [32:06] you know you know can see which models [32:07] are performing the best and you know and [32:09] and so on so on in a way that allows you [32:11] to actually track and do something about [32:13] cost because we think this is going to [32:15] be the biggest cost of companies moving [32:16] forward. [32:17] >> And without this like uh what kind of [32:19] stuff can they track? You can just track [32:20] like spend per model maybe per employee. [32:22] >> Uh you can track spend per model. You [32:24] can track spend per employee. You can [32:25] you can you can track spend by caller. [32:28] Um you know so you can see here like [32:29] every single thing we have that uses AI [32:31] in the company and what is actually [32:33] using the most amount. So you can see [32:35] here you know cloud code is at the top [32:37] codeex uh is second cursor and then you [32:40] have audit agent which is like a product [32:41] we have uh transaction tagging in [32:43] production. you can see like literally [32:45] everything. Uh and you can go here in [32:47] product and be like, okay, how much is [32:49] this thing also costing per customer. So [32:50] you can see here, you know, at the same [32:53] time, you know, which customers are the [32:55] most expensive to serve. Uh and uh this [32:58] data is of course highly confidential, [32:59] but you can see here, you know, a lot of [33:01] games that take a bunch of time for us [33:02] to and and dollars for us to go in and [33:04] and serve. [33:05] >> So um so you can just sort of see how [33:07] this whole thing stacks together. So the [33:09] problem we're trying to solve is like [33:10] you know if my if my token costs get out [33:12] of control I I can actually isolate the [33:14] root cause and try to like optimize [33:16] >> yeah you can just have like full token [33:17] analytics the same way you have spend [33:19] management you need to have like token [33:20] spend management and this is really you [33:22] know what we're doing what we're doing [33:23] at bra for for us internally but you [33:25] know at the same time will be very soon [33:28] a reality that we had to build this for [33:30] ourselves and then it became you know [33:32] spend management has to understand AI [33:34] natively which is what we want to go [33:35] build with this thing [33:36] >> what I thought do you give your [33:38] engineers a limit limited token budget [33:40] or is there like a cap? [33:41] >> Uh we we do we do give them unlimited [33:43] token budget. Um we we have a few caps [33:47] uh because some people uh of course have [33:49] used the process but we try to be pretty [33:51] unconstrained if we're actually using it [33:53] to deliver productivity. [33:54] >> Yeah. Cuz because I I I feel like I feel [33:56] like there's a difference between like [33:57] um somebody's uh like uh you know like [34:00] like a Gary Tan or like even my myself [34:02] like we have like free to tokens, right? [34:04] just just took max [34:07] >> versus like actually people spending [34:08] tokens on API and then it gets very [34:10] expensive if you just like token max. So [34:13] >> 100%. Yeah, I think I think I think my I [34:16] think there's a couple things, right? [34:17] Obviously, you know, not all like I I [34:20] think one of the learnings that we've [34:21] had at Brex is um you you you really you [34:26] really probably have to [34:28] there's a first phase which is how do [34:29] you create the the ability for people to [34:33] go pretty deep and and and then you [34:36] optimize it, right? And I think like if [34:37] we had tried to optimize it before, it [34:39] would probably not have really driven [34:42] the outcomes we want. Uh, and I would [34:44] say we're glad we did that that way. So, [34:48] >> oh, go very deep. I think just [34:50] understand the usage overall and then I [34:51] think [34:52] >> just just just let people go really deep [34:54] on AI first and get AI build and then [34:56] you can always optimize it later. It [34:58] hasn't spent a lot of time trying to [35:00] control costs in the beginning and now I [35:02] think we have the tools to actually go [35:03] in and very surgically say where does it [35:06] have lower productivity and go in and do [35:08] something about it because also a lot of [35:10] what drives like cost savings is like [35:12] for example are you using like caching [35:14] efficiently when you go like have a [35:16] product using a bunch of tokens uh if [35:17] you're using corporate AI and and and [35:19] you know and and and have like a bunch [35:20] of adoption are you running everything [35:22] on like Opus 4.8 eight or maybe table [35:25] five soon or are you just uh you know or [35:28] or or or you know can you use some [35:30] things that can run on a cheaper model. [35:32] So answering these things is really hard [35:34] unless you actually have a layer of [35:35] analytics that can give you these [35:36] answers which is why we build MacPI for [35:38] us and we're going to soon have it [35:40] available for uh for BEX customers. [35:42] >> Okay. Yeah, that that was my next [35:43] question for you. It's probably not [35:44] smart for a company just be like a [35:46] anthropic shop or like a open eye shop, [35:48] right? like you kind of want to have [35:50] multiple models in play so you can like [35:53] orchestrate stuff [35:54] >> and and the thing is like it's there's [35:56] certainly the multiple models at play [35:57] but there's also the fact that a lot of [35:59] products will become uh metered on a [36:02] token basis right so in the past you [36:04] used to have like [36:06] paying per [clears throat] seat uh and [36:07] then over time you're going to start to [36:09] see more and more products that are like [36:11] cursor for example that at the end of [36:12] the day they're really wrapping tokens [36:15] as the pricing mechanism and so so so [36:18] even if you're using, you know, [36:19] anthropic and open AI, you're still [36:21] going to have a bunch of products that [36:22] may be indirectly using tokens, uh, and [36:25] and and and pricing per token. And [36:28] making sense of usage, um, in these new [36:31] patterns of consumption, I think, is [36:32] something that is is much harder than [36:35] than people realize. And so, so we think [36:38] it's like absolutely shouldn't be a [36:40] single model shop, but most importantly, [36:42] you're going to have a lot of different [36:43] things consuming tokens versus just, you [36:44] know, opening eye onropic. [36:46] >> It's funny. What do you think? Do you [36:47] think it's going to be like half the [36:48] budget for a company is going to tokens [36:50] versus humans? Or I guess it depends on [36:52] the company. [36:53] >> I mean, I think I think I think like I [36:54] think it depends on the company, but the [36:56] way I think about it is like, you know, [36:58] if someone's starting a company today, I [36:59] think the question is like why shouldn't [37:00] it just be [clears throat] you, right? [37:02] And and a bunch of and a bunch of uh and [37:05] a big token budget and agents. Um and [37:08] and and obviously I think there's still [37:10] you know I'm going to talk about the [37:11] role that that people should play but [37:13] you know more and more we think a huge [37:15] percentage of payroll is going to go to [37:17] tokens and uh that's okay as long as you [37:20] can manage it and make sure it's like [37:21] the ROI is there. [37:22] >> Yeah. I I keep wondering if I should I [37:24] mean I have like a very small company [37:25] but I keep wondering if I should hire [37:26] someone but now I just figured out how [37:27] to use codeex to do what whatever I want [37:29] that person to do. Just just figure out [37:31] how to do it. Yeah. [37:33] >> It's hard. [37:33] >> Cool man. Just just a couple more [37:34] questions. Why don't we point that [37:36] thread a little bit? So, do you actually [37:38] believe that uh there's like people, you [37:39] know, these big companies are like [37:40] laying people off, dude? So, like do you [37:42] think there's be less people working in [37:43] tech now or do you think maybe smaller [37:44] companies will fill the gap? Like, what [37:45] do you think is going to happen to our [37:46] employment? [37:47] >> I think you're going to have there's [37:49] going to be a shift in in in jobs, [37:51] right? And and I, you know, there's a [37:53] lot of the sort of, you know, huge [37:56] unemployment [37:58] duorisks and all that. And and my sense [38:00] is you're gonna have some of that, but [38:02] you're also going to have like a lot [38:03] more uh jobs that we may not even know [38:06] about yet. Uh my my personal thesis is I [38:10] think I think what's going to happen is [38:12] you're going to have a much bigger [38:13] conflation of roles and everybody's [38:15] going to be a builder to some degree. [38:17] So, and I can show you like the way I [38:20] use AI personally uh uh and show you a [38:23] little bit of my own setup, but my sense [38:25] is like I think what you're really going [38:27] to be in the market for is like 10x [38:30] employees and I think great people are [38:32] going to learn how to use AI and and [38:36] generally I think good people are going [38:38] to see what great people are doing and [38:39] replicate some of that. So you're going [38:41] to have like a dispersion in [38:42] productivity as you always have in [38:44] performance like you know your 10x [38:45] engineer is 10x better than your 1x [38:48] engineer obviously and I think you're [38:49] going to have a similar thing with AI [38:50] but I think you're going to raise the [38:52] floor and probably raise the se raise [38:55] the ceiling and raise the floor at the [38:56] same time. uh although I think you're [38:58] going to raise the ceiling a lot more [39:00] than the floor, you're going to have a [39:01] general productivity increase. And and [39:04] the second thing is I think we're so [39:06] early in AI still, right? So [39:09] realistically, you know, we there's a [39:12] really good image that I like which is [39:14] like and I can show you is like [39:17] basically splitting the world in 2500 [39:19] boxes. So like when when someone asks [39:22] like oh god like what's what's going to [39:23] happen in AI? I'm like, I just showed [39:25] them this, which is like each dot in [39:27] this thing is 3.2 million people, right? [39:29] Every red box is like someone that never [39:31] used AI in the world. Green box is like [39:34] free chat users. Orange box is like [39:36] you're paying 20 bucks a month for AI. [39:38] >> And this tiny red box is people that are [39:41] using agents effectively. So, this is [39:44] still so early, right? And I think like [39:46] people don't necessarily make sense of [39:48] that when they when they talk about [39:50] like, you know, everything is going to [39:52] go shift. And and the reality is like [39:54] most businesses, most people, most [39:55] companies are going to need to go [39:57] through that transformation and it's not [39:58] going to happen alone. You're going to [39:59] need people to go do that. Yeah. [40:00] >> Uh but in general, I think there's just [40:02] going to be a lot more productivity per [40:04] person. the same way that you know I I I [40:07] kind of joke with you know a lot of uh a [40:09] lot of founders that I think I I build [40:11] that you know and you never been so [40:14] productive and never worked so hard [40:15] because you just feel so much more now [40:18] that you can do more and you're just [40:19] going to go and tackle these things [40:21] versus uh you know uh uh uh sort of keep [40:24] them in the corner and say oh you know I [40:26] have more time or free time let me go to [40:27] the beach now you know you're just gonna [40:29] I think it's just the nature of being a [40:30] builder you just get energized by the [40:32] process so [40:33] >> it's If if you're in that little cording [40:35] harness box, you can literally be like [40:37] 100x more productive than someone's not [40:38] using AI. So, yeah, it's like got to [40:40] capture the moment [40:42] >> 100%. [40:42] >> Uh do you want to like quickly show us [40:43] some of your personal stuff and then we [40:45] can [40:45] >> uh I can Yeah, I mean I I have I have [40:47] this this system and obviously, you [40:50] know, we should redact this, but I have [40:52] a system that I I call autopilot and [40:55] basically what it does is like it's an [40:56] open claw harness that basically runs my [41:00] my life. And the idea is I have like [41:04] basically two building blocks like [41:06] people and programs and these are like [41:08] programs that I care about. These are [41:09] people that I actually spend a lot of [41:10] time with on my team. Um and then the [41:13] idea is it all starts with like signals [41:15] and signals are like literally like [41:19] Slack messages, emails, meeting notes [41:22] and signals just exist in the context of [41:25] like everything that uh I am doing uh [41:28] and and they're reading think of it as [41:30] reading my Slack messages and [41:32] >> you know uh you can see here like [41:34] reading my email and basically finding [41:37] things that match um either a program or [41:41] a [41:42] Uh and you can see here this job uh it [41:44] effectively like runs this like signal [41:46] collector that's like collecting my [41:48] email things on my inbox collecting [41:49] meetings uh and collecting slack and [41:52] then the whole magic is this thing [41:54] called aggregate where really what it's [41:55] doing and you can see here like the [41:57] output of this thing really what it's [41:58] doing is it is basically going through [42:02] every uh person and every program I have [42:05] and updating um these files um and and [42:09] basically you can go here in PR and [42:10] social for example and just see, you [42:13] know, this is how I actually prepared [42:14] for this podcast. So, you can see here [42:17] that, you know, uh you know, someone [42:19] helped me prep for uh for for this uh my [42:22] camera was acting up. So, you can see [42:24] here there's a few marketing campaigns [42:26] we're working on. Um and uh you know, [42:30] this is uh this is the this is the sort [42:32] of status of the thing. And there's [42:33] blockers and risks. And then the cool [42:35] thing here is you can see here like [42:37] effectively this like list of action [42:38] items of everything that I have to go do [42:40] based on uh based on this program. So uh [42:44] I had Kobe on my team write a script for [42:47] me to demo macpie. He's done that [42:48] already. So I can actually check that. [42:50] Um and I have to go do this. Uh I [42:53] already done this. And these things here [42:55] that are in yellow are drafts that this [42:57] that that autopilot automatically [42:59] created from signaling injection. So, I [43:01] can click in here and say, you know, [43:03] there's this DM that someone messaged me [43:06] asking if I want to go to this [43:07] conference. It ingested my Slack and [43:09] this is like literally the DM. So, I was [43:12] like, okay, like here's the DM. And it [43:15] just like figured out that I have to go [43:16] in and make a decision about whether I [43:18] want to go and join this thing. Um, so [43:20] you can see how this whole thing [43:21] compounds into like a pretty sort of [43:24] robust task system. And I also have an [43:26] interaction log. So, I can come in here [43:27] and say, let's see, if I just want to [43:29] see what happened today, I can come in [43:31] here and filter. Well, sounds like [43:32] nothing happened today, but today and [43:34] yesterday, these are all the things that [43:35] happened. And I can go in here and be [43:37] like literally like, hey, here's just a [43:41] DM. I can go in here and like give a [43:44] give a like a view on the signal and [43:46] actually see exactly where that came [43:48] from. And this this whole thing just [43:50] like runs in the context of that [43:52] program. And I have this for like a [43:54] bunch of other programs and a bunch of [43:55] other people. So I can do this and have [43:57] like effectively a download of [43:58] everything that's happening on this and [44:00] every task that I have to go track and [44:02] every interaction log. So it's very easy [44:04] for me to say okay I want to see [44:05] everything that happened in one place in [44:07] one glance and not have to go through [44:09] slack email meeting notes and etc. Uh [44:11] and also docs it can come in here and [44:13] summarize uh all the docs that you know [44:16] we I had interactive to make this thing [44:18] work. [44:19] >> Yeah. Okay. Okay. So, so basically you [44:20] have a bunch of inputs from like Slack, [44:21] uh, Granola and everything else. And [44:23] then and then are are each of these [44:25] programs like a prompt or something like [44:26] to kind of extract? [44:27] >> Uh, yeah. Each of these programs is is [44:28] just a markdown file. So I can go in [44:30] here and like literally define what it [44:32] is and then the signal injection is [44:34] going to go in and take care of like [44:36] filling it all out. I can this is all [44:38] markdown. So I can go in and edit each [44:39] of the files. I can edit the interaction [44:41] log and then there's a bunch of uh [44:43] scripts that just uh create nice UIs and [44:46] make this like editable and searchable. [44:48] Uh, but it's all markdown file [44:50] >> and then you have like a database or [44:51] like a QMD thing or like [44:52] >> it's all markdown files. [44:53] >> It's all markdown files and they all get [44:55] committed to a repo um that you run. So [44:58] it's so it's literally just text files [45:00] uh and and a bunch of UI and and sort of [45:04] >> you know nice things to be able to to go [45:06] in and have this like job management [45:08] system here. Uh but you know it's [45:10] effectively just like a way to consume [45:12] the entire state of my life into one [45:14] system. [45:15] >> That's amazing dude. I can't even get [45:16] open call to work reliably. You have [45:18] this whole thing set up. [45:19] >> Uh yeah, I mean Open CL is uh open C is [45:22] a learned uh acquired taste when it [45:25] comes to making work reliably, but uh [45:27] you know I think it's worth it, but uh [45:30] definitely takes a bunch of time to to [45:31] do it well. [45:33] >> Okay. All right, dude. Well, let me ask [45:34] you one last question. I feel like the [45:36] good old days of like SAS we can just [45:37] charge a subscription for a niche [45:39] product are are kind of like depends on [45:40] what kind of SAS it is but it's kind of [45:41] like dying a little bit because I feel a [45:43] lot of stuff I can just use skills and [45:44] like hook up to a bunch of APIs and do [45:46] the same result. [45:47] >> So like personally even I want to build [45:49] businesses I want to build products that [45:51] make money but I feel like it's kind of [45:52] hard to assassinate anymore. What are [45:54] your thoughts on this? I mean, I think I [45:56] think it's like we just changed the [45:57] currency of things, right? But the [45:58] reality is you're still paying for [46:00] solving problems for people, whether it [46:02] was like go build a SAS to do it or [46:05] build an agent that is consuming tokens [46:08] and you're charging a markup on the [46:09] tokens. Like I think I think that's like [46:12] fundamentally still true. The question [46:14] is like I think the bar for how to build [46:17] good AI products uh is just a lot higher [46:19] now because you can just sell the work [46:21] itself versus just sell the software. [46:23] And and I think like the reality of all [46:26] good AI products is they're all the same [46:27] thing which is um you know an agentic [46:30] loop and a measure of tools like that [46:32] that that's really all it is. It's like [46:34] you expose the tools to the model you [46:36] run it on aentic loop and just let it do [46:38] its thing. Um and and and fundamentally [46:40] I think that applies to a lot of [46:43] different uh problems and domains and [46:46] you can go pretty far of that. I think [46:47] the but I think the fundamental thing is [46:50] like you're now able to sell the work [46:51] itself versus just sell um you know the [46:55] the the tool. [46:56] >> Okay. So you have to sell the output of [46:58] Okay. Got it. So it's like you go do it [47:00] yourself. You can be like hey this agent [47:01] will do it for you. [47:02] >> Yeah. Exactly. [47:04] >> Okay. All right Pedro. Well thank thanks [47:05] so much man. This was this is like uh [47:07] pretty eye opening. Uh I guess where can [47:09] people find you online or do or do you [47:10] want to be found online? [47:11] >> Uh well go to X. Uh I'm Pedro age 96. Uh [47:15] so uh yeah you should be able to find me [47:17] there and uh I try to keep uh the world [47:20] you know updated to some degree on the [47:21] things that I'm building and uh a lot of [47:23] the stuff we're building at bra because [47:25] you know we at the end of the day the [47:27] the most valuable thing we can do is [47:28] help companies better manage their AI [47:30] spend and you know build great things [47:31] with AI token [47:32] >> max responsibly that that's the model. [47:34] >> Exactly. Exactly. Very responsibly. [47:37] >> Cool. Nice.