Ras Mic Tames AI Coding Agents With Git Worktrees and Greptile Loops

Ras Mic利用隔离Git worktree阻断自主AI代码失效 不受约束的软件工场不可避免地会退化为破损代码，因为有缺陷的初始假设会在多次运行中不断累积，迫使工程师将任务隔离在Git worktree中，并在人工签发前通过证据闭环验证功能。 为了防止自主AI编程配置将错误假设累积为混乱的“垃圾代码炮”，Micky将他的工程规范整理为一套由`agents.md`和模块化技能驱动、且与具体模型无关的四步工作流。首先，他将并行的智能体任务隔离在专用的Git worktree中（`new-feature`）；其次，他强制代码遵循严格的编排与服务层架构（`code-structure`）；第三，他要求智能体通过前后对比的FFmpeg视频、截图或延迟基准测试来证明其工作（`evidence-driven-testing`）；第四，他引入自动化Greptile PR审查循环，直到代码评分达到5/5，最后再进行必要的人工检查。Micky没有编写正式的PRD，而是使用Wispr Flow通过10到30分钟的语音“闲聊会”来规划功能，依靠对话历史或针对性的`setup.md`文件作为上下文。最后，他通过个人基准测试对比了前沿模型——更青睐OpenAI的Astra用于性能审计、不受限的安全审计和计算机使用，同时保留Anthropic的Fable用于原始代码质量和UI设计——并倡导战略性模型路由、裸机家庭实验室，以及对来自AI实验室的高消耗token建议保持怀疑态度。 不受约束的自主“软件工场”就像“垃圾代码炮”，因为哪怕是一个微小的错误假设也会在整个构建过程中不断累积，导致智能体最初提交的Pull Request评分最高只能达到4分（满分5分）。 (0:00) Michael的四步智能体工作流——隔离（Isolate）、构建（Build）、求证（Prove）和发布（Ship）——通过将每个任务锁定在独立的Git worktree中并强制执行严格的服务层架构，能够安全地并行运行5个以上的智能体。 (2:06) 自动化验证闭环要求智能体在人工审查前证明其成果，包括使用FFmpeg视频录制重现Bug，或通过量化基准测试展示延迟从850毫秒降至60毫秒。 (10:44) 在第4步（发布）中，自动化的`greptile-loop`技能会迫使智能体对Pull Request进行迭代（必要时最多10次），直到代码获得5/5的审查置信度评分且无未解决的评审意见。 (21:30) 尽管Fable编写的人类级别代码更整洁且在无参考UI设计方面表现出色，但Astra在性能和安全审计方面占据主导地位，且不会触发内容过滤降级。 (33:06)

Michael argues that fully autonomous software generation repeatedly fails because unverified initial assumptions compound into severe code drift. Instead of relying on open-ended loops, he advocates a constrained four-stage workflow—Isolate, Build, Prove, and Ship—that enforces modular architecture, deterministic proof artifacts, and automated remediation while keeping human oversight in the loop. Worktree Isolation and Controlled Parallelism (2:06) Michael and Peter Yang reject the mainstream vision of an unattended "software factory," observing that unconstrained coding agents regularly produce flawed pull requests that plateau around an 80% (4/5) quality ceiling. Michael warns that autonomous setups quickly turn into an unchecked "slop cannon" when models make unverified assumptions early in development. His process avoids unguided black-box loops by codifying developer practices and constraints into an `agents.md` configuration file and specialized agent skills. To ensure workspace hygiene, the first step prohibits agents from touching working branches directly. An agent triggers the `new-feature` skill, which fetches the latest `main` branch and spins up an isolated Git worktree dedicated to a single task. This isolation allows Michael to run five or more Codex agent sessions concurrently inside Ghostty terminal tabs without repository collisions. He highlights an operational caveat: while worktrees isolate the filesystem and version control, they share system infrastructure, meaning active development server ports and local databases remain shared across parallel tasks. Modular Architecture and Deterministic Proofs (7:18) In the build phase, Michael's `code-structure` skill enforces a strict Service Layer Architecture. Code is separated into an orchestration layer that dictates the "why" and "when" (business rules, authorization, and state transitions) and a service layer that manages the reusable "how" (typed SDK wrappers and low-level execution scripts). According to Michael, this modular organization not only keeps the codebase readable for humans but also stops agents from their common habit of reinventing the wheel by writing duplicate utility functions across different files. The subsequent "Prove" phase requires agents to produce deterministic, empirical evidence before proposing changes. For bug tickets dispatched through Linear, cloud agents on Linux or macOS use FFmpeg to record video reproducing the bug on `main`, followed by a video demonstrating the fix on the worktree. For non-video tasks, agents provide side-by-side screenshot diffs or quantifiable benchmark numbers—such as an analytics optimization where Astra reduced route latency from 850 milliseconds down to 60 milliseconds—allowing models to inspect their own results and iterate autonomously. Automated Review Loops and Human Governance (21:30) To deliver features, Michael deploys an automated pull request review loop using Greptile and a custom `grep-loop` skill. Because baseline model PRs rarely receive a perfect evaluation on the first attempt, the skill forces the agent into a closed remediation loop: the model parses actionable feedback, refactors code to architectural standards, regenerates validation artifacts, and pushes new commits until Greptile issues a 5/5 score or reaches a hard iteration cap (configured between 5 and 10 attempts). Michael also incorporates an `unslop` skill authored by Cursor engineer Potato to strip AI-generated boilerplate, noting that models like Astra often attempt to skip cleanup steps unless constrained. Michael draws a strict boundary between disposable weekend hacks and software built for real users. While throwaway utilities can be generated rapidly without formal reviews, production-grade applications require empathy for the end user and rigorous manual checks. He dismisses unattended overnight generation, arguing that unmonitored agent assumptions derail codebases far faster than human errors. Model Routing, Planning, and System Maintenance (33:06) Rather than writing formal requirement specifications, Michael uses Wispr Flow voice dictation for 25-to-30-minute conversational planning sessions with Codex, treating the dialogue history as the functional spec and creating `setup.md` files only when postponing complex setups. Evaluating models under his personal benchmark, Michael prefers OpenAI's Astra for performance audits, computer use, and security evaluations—recounting how Astra uncovered a critical receipt vulnerability in a church payment application—while Anthropic's Fable excels at unprompted UI design and generating cleaner, more human-like pull requests. Michael cautions against deploying expensive flagship models for routine execution, recommending model routers that delegate high-volume tasks (like calculating tax returns) to cheaper models paired with auditing sub-agents. Furthermore, running multi-agent workflows locally requires dedicated maintenance: Michael notes that his Linux setup once accumulated over 3,000 abandoned Git worktrees that threatened system memory, requiring automated janitor cron jobs to prune idle branches and processes. While AI lab creators frequently promote continuous, unconstrained agent loops, Michael maintains that dependable engineering workflows depend on explicit architectural boundaries, reproducible evidence, and strict cost and maintenance controls. As model capabilities expand, developers must evaluate baseline model behavior without prompt wrappers before deciding which process constraints remain necessary.

[9:51] Micky: Agents love to write the same function again and call it in different places [15:45] Micky: The one thing I would say that makes this software factory not like other software factories is I test my features... I'm not just going feature after feature after feature and then now I have 10 other features that don't work. I think when you're building an application for people, a little bit of empathy is necessary for the user... It would break my heart if someone used my app and it was terrible. It was trash. It was disgusting. [26:22] Micky: When there's a lack of communication and things are not discussed, stuff hits the fan. [29:16] Codex Agent: Oh dude, you could use Cloudflare Email and you could basically build the same thing. It wouldn't be too hard. [29:16] Micky: Planning for me is me and the agent discussing and I'm having a yap session and then I'll tell the agent what I want and then I'll go build it [29:16] Peter Yang: The conversation history is the spec [33:06] Micky: Boy oh boy, I'm glad I did that. [33:06] Micky: If you have any sort of app that is being used by actual people and money is involved in particular, I beg, please run a security audit [33:06] Micky: Fable writes better code. [38:36] Micky: There's no need to use state-of-the-art for everything. [43:08] Micky: I found a lot of the founders and certain people in the lab companies... their advice just sounds like someone who has a bajillion tokens to burn telling me exactly how I'm supposed to do things.

[0:00] software factory as it's explained and [0:02] broadly known I'm just not a fan of [0:04] right because one bad wrong assumption [0:06] in the beginning just destroys [0:08] everything I'm not at a point where I [0:10] trust the agent most models your fables [0:13] even Astra usually the first PR they [0:16] make is a four out of five this is [0:18] essentially the crux of the workflow [0:20] isolate new work tree write code to a [0:23] specific [music] standard prove that a [0:24] bug exists even though this is [0:26] supposedly software factory it's very [0:28] much human in the loop I am testing my [0:29] features. [0:33] >> Hey everyone, today I'm really excited [0:34] to welcome Michael, a full stack [0:36] engineer who also has a popular YouTube [0:38] channel where he teaches 100,000 people [0:40] how to code with AI. And today, [0:42] Michael's going to show us how he set up [0:43] his software factory and agentic [0:45] engineering system. Personally, I'm a [0:47] little bit skeptical of software [0:48] factories because when I try to do stuff [0:49] like that, it tends to think Michael has [0:52] found a way. So, really excited. So, so [0:54] welcome, Michael. [0:55] >> Yeah, thank you for having me, Peter. [0:57] I'm really excited and I agree with you, [0:58] too. I think the uh general term and [1:01] what you hear of a software factory, it [1:02] usually is a slob cannon. Um I have a [1:05] different version a different sorry [1:06] definition of what a software factory [1:08] is. And I'm I'm happy to share that. To [1:11] me, a software factory is more about [1:14] your workflow skills and domain [1:15] knowledge and sort of packaging that in [1:18] either an agent.mmd cloud MD file and a [1:20] bunch of skills, right? You know, going [1:22] back to core engineering principles, [1:24] like many I've I've worked with many [1:25] engineers, we all kind of work [1:28] differently. And to me, a software [1:29] factory is is being able to take that [1:31] domain knowledge and sort of have a [1:33] system where you can easily go through [1:36] the flow. So a software factory in my [1:38] humble respectful opinion, I know [1:40] there's a lot of startups that are doing [1:41] this, is model and harness agnostic, [1:43] right? So it doesn't matter what harness [1:46] you use or what model you use. [1:47] Obviously, you want to use the best [1:48] model. That being Astro right now, in my [1:50] humble opinion, although the Fable 5.1 [1:52] is great at writing code, that's what a [1:54] software factory is. And I think I'll [1:56] probably give you like a brief breakdown [1:57] of how mine works. It's just exactly how [2:00] I would write code normally. I have I'll [2:03] just delete some of these examples so we [2:05] can walk through them together. I have [2:07] basically one agents.md file and a [2:10] couple skills. And I think the best way [2:12] to explain this is sort of show my [2:14] agents.md file. and my agents.mmd file. [2:18] The purpose of it is it just explains my [2:20] workflow. It doesn't really talk about [2:22] how the codebase is structured and stuff [2:23] like that because the models are so good [2:26] now they can use their read tools and [2:28] figure out how things are structured. [2:29] But what my agents MD file sort of [2:31] explains how I want my agent to work, [2:33] right? And there's essentially four [2:35] steps. The first step is to isolate, [2:38] right? So whenever I go and I prompt for [2:41] a new feature or I'm in like discovery [2:43] mode or I'm telling the agent, okay, I [2:45] want you to build, you know, analytics [2:47] or I want you to improve email tracking [2:50] or I want you to, you know, I want us to [2:52] have an IM iMessage service run on my [2:55] MacBook that's on my network. Right? So [2:58] whenever we're working on a new feature [3:00] or we're fixing a bug that exists just a [3:03] fresh fresh new work what is going to [3:06] happen is this skill called new feature [3:09] is going to be called and basically what [3:11] new feature does is it branches off a [3:15] git work tree off of the main branch [3:17] right so if I were to think of this is [3:21] my project right this is like the start [3:23] and this is the finish and this is my [3:25] main branch every time I'm working on a [3:28] new feature, what essentially happens is [3:32] um a new git work tree is created [3:35] branching off main. And what this allows [3:37] me to do is I'm usually working on [3:39] multiple features at a time. Agents have [3:42] allowed for me to explore multiple ideas [3:44] at once. So I'll have multiple features [3:48] and they're all on separate work trees [3:51] that, you know, the agent's not going to [3:53] overstep on each other. And a perfect [3:55] example here is I have a ghosty terminal [3:57] um running herder and I don't know if [4:00] you can see it on the bottom left. I can [4:02] zoom in. I have one, two, three, four, [4:05] five tabs running codeex agents. Four [4:09] four of them just finished. One of them [4:10] is continuing to go. And the reason why [4:13] I'm able to do this safely is the [4:16] isolate uh step which basically calls my [4:20] new feature skill which if I show you [4:22] the skill it's pretty simple [4:24] self-explanatory. It basically just [4:26] tells the agent start a new task in an [4:28] isolated git work tree branched off main [4:30] so multiple agents can work on the same [4:32] repo in parallel without conflicts. Use [4:34] at the beginning of every new feature [4:36] fix or task before writing any code. [4:39] Right? So this gives me a true parallel [4:42] experience where I can have multiple [4:44] agents working on different features. [4:46] This is step number one. Now notice [4:49] there's you hear software factory and [4:51] you just think give it a task and let it [4:53] just rip till the end. Right? Mine is [4:56] more so a workflow that I tell I give [5:00] the agent which skills to call at which [5:02] step. And to me, this is not a slop [5:06] cannon versus when we were running Ralph [5:08] loops and all those things or even [5:10] slashgo to a certain extent. You're [5:12] leaving a lot of gaps for the agent to [5:15] fill itself and sometimes they'll make [5:17] good assumptions, right? Especially the [5:18] latest and greatest models. They they [5:20] can make good assumptions, but sometimes [5:22] it could be terribly wrong, right? So, I [5:24] would say this is the first step of my [5:26] workflow. I'll stop here, Peter, if you [5:28] need me to fix or explain anything. [5:30] >> This episode is brought to you by [5:31] Whisperflow. Whisper Flow saves me at [5:33] least three hours a week and is one of [5:35] my favorite AI apps by far. It's just so [5:38] much faster to dictate to AI using your [5:40] voice than to type. You just talk [5:42] naturally and it outputs clean, ready to [5:44] send text. Whisper Flow even removes [5:46] filler words and formats your sentences [5:48] for you. I use Whisperflow for [5:50] everything including drafting newsletter [5:52] posts, writing product specs, replying [5:54] on Slack, and more. It works on Mac, [5:56] Windows, iPhone, and Android across all [5:58] of your favorite apps. Try a free at [6:00] whisperflow.com and use my code peter [6:04] whisperflow to get six months free. [6:06] That's Peter whisperflow. Now back to [6:08] our episode. So what what is the output [6:10] of the new features scale? Like is this [6:12] like a spec or or something? Basically, [6:15] it says every task gets its own work [6:16] tree and branch created off. It's just [6:18] explaining exactly how to use git. Um [6:22] how to create the work tree. And again, [6:24] this was generated with AI, right? I [6:26] previous to AI, I wasn't the best at [6:29] git. I was always on Stack Overflow or [6:33] even Google figuring out, okay, I have [6:35] this issue. I need to rebase. So, git [6:37] was primarily never my strong suit. [6:39] Yeah, I use AI and basically this skill [6:41] just explains it how to create the uh [6:45] work trees, right? First of all, fetch [6:47] origin, make sure you're in sync with [6:49] your repo and creates the work tree from [6:52] the repo route, verifies it, and then it [6:55] tells it here work trees do not isolate [6:57] shared resources, dev dev server ports, [6:59] shared database. So just again some [7:02] basic information for it to understand [7:03] how to properly create a work tree and [7:06] to understand that yes the work is [7:07] isolated but the resources are not. [7:09] >> Okay. So basically the new feature uh [7:11] primarily creates a new work tree but in [7:14] terms of actually starting to build the [7:15] feature just like a separate scale. [7:17] >> Yeah. So this is this is just step one [7:19] where we're isolating you can think of [7:21] them as workspaces. Right. when it comes [7:23] to actually building uh we get to step [7:25] two right and the build step calls a [7:28] skill called code structure and [7:31] basically what code structure is this is [7:34] a skill that I made that basically tells [7:37] the agent to write code following a [7:40] service layer architecture right there's [7:43] a pattern that I explain here have an [7:46] orchestration layer for actions have [7:48] shared mechanics service layer basically [7:50] an example of how this should look how [7:53] functions should be written right this [7:55] is and and the reason why I have this is [7:57] there's two thought process when [7:59] creating this skill and the first one is [8:01] if I for some reason have to get in and [8:05] read and write code like it's 2024 then [8:08] this should make it at least easy for me [8:11] to understand because this is how I was [8:13] taught to write code right so [8:16] >> this skill basically gets called when [8:19] it's time for the agent to write code. [8:21] So in the build step it says write code [8:24] to the service layer architecture action [8:26] and boundaries orchestrate the why and [8:28] when a service layer owns the reusable [8:30] how with explicit inputs and structured [8:32] returns. [8:33] >> Can you just briefly explain a couple [8:35] lines for people who don't know what [8:36] service layer architecture is? [8:38] >> Yeah. So basically the and it's in the [8:40] skill here. The premise of a service [8:43] layer architecture is actually it's [8:45] defined right here. the actions and [8:47] boundaries that orchestrate the why and [8:50] the when and the how are sort of [8:52] separate. And a a good example here I [8:55] think is for example just read the [8:58] function names create manage sandbox [9:01] prepare repo detect package manager [9:04] install dependencies run build command [9:07] start sandbox runtime. So the way [9:10] functions actions and all different [9:12] resources are defined they're defined in [9:15] a specific pattern. This pattern allows [9:17] for another developer when they're [9:19] reading to understand this right to [9:21] understand the core pattern really [9:23] there's two places there's two spaces [9:25] there's an orchestration layer meaning [9:27] this owns business rules state [9:29] transitions if you have a state machine [9:31] you know off checks um you know service [9:33] functions all that stuff and then the [9:34] service layer owns reusable you know [9:37] functions this is where you would [9:39] probably define your SDK it owns all [9:41] that type of stuff right so any sort of [9:43] structured result lives here The reason [9:46] why this is this architecture in my [9:48] humble opinion again a lot of people [9:50] debate this there's some developers who [9:51] probably wouldn't be a fan of this but [9:52] in my opinion the reason why this is [9:54] great not only for myself but for agents [9:56] is this limits duplicate functions which [9:59] agents love to write the same function [10:02] again and call it in different places [10:04] and this is readable right so I might [10:06] not have to read every line but I can [10:08] read create manage sandbox and what do I [10:11] understand that this function does it [10:13] creates a manic sandbox right So it [10:15] helps not only myself or if another [10:17] developer were to jump on this project [10:20] but it also helps in my humble opinion [10:22] the AI agents understand okay this is [10:24] what this function does. [10:26] >> Okay. [10:27] >> Yeah. So that's what the build step does [10:29] right it writes code and it follows this [10:32] architecture. So you can almost imagine [10:35] if I go back to this diagram let me just [10:38] clear some of these lines out. We [10:40] isolated a new work tree and I've [10:43] started to build. Now in the process of [10:46] building [10:47] there is this proof step and this skill [10:52] it only works for some machines right um [10:55] I think if you have a Linux machine with [10:57] ffmpeg and all that stuff installed. [10:59] Basically what this does is it captures [11:02] the before state while reproducing the [11:04] issue prior to fixing it and then it [11:07] will show the after state. This works [11:09] great with cursor cloud agents. And I [11:12] can show an example of what this is. Let [11:17] me just pull this up. Yeah. So, what [11:20] this skill allows with like a cursor or [11:23] a Devon Cloud agent is it basically [11:26] forces not forces but it tells my agent [11:30] to actually produce evidence of said bug [11:35] not working and said bug working. Right? [11:38] So it records a before and after state. [11:41] And if I were to show you the skill, [11:42] it's a bunch of uh scripts as you will [11:45] see. [11:46] >> And then just to people who who don't [11:48] know, FFmpeg is basically just like a [11:50] video recording [11:51] >> basically, right? So if you have a [11:52] Linux, Mac OS, I haven't tested the [11:54] Windows one. I don't have a Windows [11:56] machine unfortunately. So this is why I [11:58] say this skill might work, might not [12:00] work. But basically with a cloud agent, [12:02] this is great because I'm getting video [12:04] evidence of the agent testing my skill [12:07] out. This works great for bugs because [12:09] like for example for some of the [12:12] projects I'm working on where we [12:13] actually have users, a product person [12:15] will say, "Oh, we found this bug and [12:17] live leave a linear ticket, right?" And [12:20] what happens now is on linear, I wish I [12:23] could show, but some of it is is private [12:24] information. With linear, you can assign [12:27] work to a person, but you can also [12:29] assign work to an agent, right? And [12:31] basically what this skill does is it [12:34] forces the agent to prove that this bug [12:37] actually happens, right? Cuz cuz [12:38] sometimes it just might have been user [12:40] error. Sometimes it just something might [12:42] have went down, right? We want to make [12:44] sure that we prove that a bug is [12:46] actually a bug. And what this skill also [12:48] does is it proves the fix as well, [12:51] right? I think in this case this is [12:53] proving that it can get a response and [12:56] stream properly, right? And it streamed [12:57] properly, right? There was a streaming [12:58] issue. But let's say that doesn't work [13:01] for you. There's another skill called [13:03] before and after. This is a skill I [13:05] stole from Versel. So, shout out to the [13:07] Versell team for this. And basically [13:09] what this skill does is it takes a [13:11] before and after screenshot of said [13:14] feature build, right? And I can show an [13:16] example of this so people can have an [13:19] idea. Let me go to a project I've been [13:23] working on and just pull up one maybe. [13:26] Let's try this right here. [13:29] So, this is a great example. Um I I [13:33] think this is a tracking feature for [13:34] email delivery that I was working on. [13:38] And before this is existing mailbox and [13:41] then the after it shows the result that [13:44] oh whatever file this was malware, [13:47] right? So I get a before and after [13:50] screenshot sort of showing the change [13:52] that was made. But for let's say for a [13:55] feature that's not really UI specific or [13:58] doesn't have like you know something [14:00] like UI to screenshot then what the [14:02] skill will do and this is pretty cool. [14:05] Performance I just spelled performance [14:07] wrong in front of thousands of people. I [14:09] apologize. And for something that's not [14:12] screenshotable it will give you numbers. [14:14] Right. So in this case, this was um the [14:17] dashboard I was I I had built um using [14:20] GPT 5.6 soul. The navigation was just [14:22] terribly slow. Um and I asked Astra to [14:26] do a performance update and it got it [14:28] from 850 milliseconds to 60, 817 to 61. [14:32] You could see considerable performance [14:34] update, but this isn't necessarily [14:36] something you could show with a [14:37] screenshot. So that's what before and [14:41] after will do, right? again proving the [14:44] work of the agent. [14:45] >> So okay so evidence-driven testing is it [14:48] records a video of the main branch [14:50] without the new feature and the work [14:52] tree branch. So it kind of compares the [14:54] two. [14:54] >> Yeah. So basically if it more so the [14:57] before is more so useful if a bug was [15:00] reported. Right. So say a bug was [15:02] reported what it will do is it will like [15:04] reproduce the bug right? So let's say [15:06] there was a streaming bug when you [15:08] refresh the app after you sent the [15:10] message right at the moment. So it will [15:12] record that, right? If there is no bug [15:15] per se, there's no before for it to [15:16] record, it will just record the after [15:18] state, right? So evidence-driven testing [15:20] is really good for an app that's already [15:23] in production and you're getting a bug [15:25] report. But before and after always [15:27] works, and I find this to be very, very [15:29] great because it'll show you the [15:30] improvements made, right? And in my [15:32] opinion, it sort of forces the agent to [15:35] sort of prove that the solution it made [15:37] work cuz I've seen traces before where [15:40] it'll do the before and after and be [15:41] like, "Oh, the changes aren't really [15:43] good." And then it'll continue on [15:44] working. [15:45] >> And let me ask you this before you move [15:46] on to or maybe you this is a part of [15:48] Grap Loop. Like do you trigger each of [15:50] these phases manually or or like it kind [15:53] of triggers by itself? So, I will say [15:55] with Astra, and this is uh something [15:58] I've noticed with 5.1 and 5.6 Soul, [16:02] these have not been triggered uh man, [16:05] like I haven't been triggering these [16:06] manually, but I find I'm trying to find [16:08] an example here, but I found that with [16:11] Astra, and I don't know if this is a [16:13] model flaw or maybe I have to make some [16:16] updates to how my age.md is written. I [16:18] find that sometimes I have to tell it, [16:20] oh, I got a PR with some uh feedback [16:23] from Gretile. You didn't run a GP loop. [16:25] And it'll be like, oh, I forgot. Like [16:26] it'll be like, oh, my apologies. And [16:28] then it will continue to work. So, I [16:29] find with Astra, I've had to do a little [16:32] bit of reminding and I plan on trying to [16:34] figure out why. But with 5.1 and GBT [16:38] 5.6, like the like when I run this, it [16:42] literally goes from, okay, I've [16:43] explained the feature to I get a PR that [16:47] I test. Now, the one thing I would say [16:50] that makes this software factory not [16:53] like other software factories is I test [16:55] my features, right? I test them out. I I [16:57] see I use I'm not just going feature [17:00] after feature after feature and then now [17:01] I have 10 other features that don't [17:03] work. I think when you're building an [17:04] application for people, a little bit of [17:07] empathy is necessary for the user. I'm a [17:10] very like I like I it would break my [17:12] heart if someone used my app and it was [17:14] terrible. It was trash. It was [17:16] disgusting. Right? So I do even though [17:18] this is supposed to be software factory [17:20] it's very much human in the loop I am [17:22] testing my features. [17:23] >> Got it. Okay. So basically and just you [17:25] working on this app right or is there [17:27] other people? [17:27] >> So I I like a little side note I don't [17:29] really talk about this online. I do run [17:31] uh a consultancy where we do build uh [17:34] apps and agents for third party [17:36] companies for other companies right [17:38] different sizes. And obviously when [17:39] you're building for yourself then yeah [17:41] you can have the slot machine on run and [17:43] it's fun but like when you're building [17:44] for other businesses you know there is a [17:46] responsibility uh that you have to have [17:48] right which is why again this is there's [17:50] a code structure skill there's a before [17:51] and after and we do have I'm not the one [17:53] to review but I do have developers on [17:55] the team that actually review the code [17:57] right shout out to them but for software [17:59] that I'm building or side projects or [18:01] apps that I'm launching then yeah 100% [18:03] this is just me and I'm following this [18:04] flow [18:05] >> I yeah I guess um the before and after [18:08] And evidence-driven testing is happens [18:10] before you do the manual review. The [18:12] manual review is almost like the final [18:13] step, right? Like like the agent tells [18:14] you it's ready to go, then you test it [18:15] yourself. [18:16] >> Basically, yeah, the manual review is [18:18] like all the work's done and I have as [18:20] much evidence from the agent as [18:21] possible. Essentially, what I'm trying [18:23] to do with evidence- driven testing and [18:25] before and after and even loop is we're [18:27] trying to build trust, right? U so [18:30] there's evidence that the agent can [18:32] present to say, okay, this is the work [18:33] that's done and this is why it's good, [18:35] right? So it makes manual review as [18:38] little as as short of a time as [18:40] possible. For anyone who's been a [18:41] developer for any time, manual review is [18:43] probably the worst part of your job. [18:44] This sort of helps at least minimize the [18:46] amount of uh friction uh a developer has [18:48] to go through for manual review. [18:50] >> Got it. Okay. Yeah. I also do find that [18:53] Astra is a little bit timid in just [18:55] doing the work. And I don't know about [18:58] you, but I also find it not very [19:00] reliable in following my skills. [19:02] >> Yeah. So I I've been using it again [19:05] extensively the last couple days. All my [19:07] agents right now are literally codeex. [19:10] And one thing that I find it's yeah it's [19:13] very smart but it almost needs like [19:15] assurance like it will say it's going to [19:17] do something and it'll do it but it [19:19] hasn't done it all the way and I'll tell [19:21] it oh continue and it'll be like oh yes [19:23] I will continue the work I started. And [19:24] yes to your point like there's sometimes [19:26] where it doesn't follow skills right. [19:27] So, and we can talk about this later [19:29] with the the model uh stuff, but I still [19:33] find, especially if people have the [19:35] budget for it, I still find a great [19:37] place and value for Fable 5.1. I I don't [19:40] think it's down and dead. I do think [19:42] Astra as an overall model is very much [19:45] impressive and powerful and the computer [19:48] use. And I can even share some of the [19:51] hardware things I've been working on. [19:52] and a bunch of hard I bought I bought a [19:54] bunch of hardware stuff thanks to [19:56] >> Astra and Blender. Uh but yes, it is a [20:00] little to your point timid sometimes has [20:02] to follow instructions and sometimes [20:04] needs to be reminded to keep on working [20:06] which I'm not a fan of [20:08] >> but yeah 100%. [20:09] >> Related to that I'm curious so do you [20:11] mostly do your coding from the terminal [20:13] or because if you use the codeex app [20:15] some of this evidence-driven testing [20:17] stuff it can actually do through the [20:18] browser use and through some of the [20:20] stuff that is included in the app. [20:21] >> Yeah. Yeah. So I I it depends, right? So [20:25] I will use um the terminal. I will also [20:29] use the guey. And the way I basically [20:31] divide work is to your point. For [20:34] example, with this app called bezel I'm [20:36] working on, I'm really doing a lot of [20:38] backendy type work right now. Not a lot [20:41] of visual stuff. So I will just use the [20:43] terminal. It's connected to a Linux box [20:45] that I have that's very very powerful. [20:47] So I'll let it run and do that. But [20:50] whenever I'm working on userfaced stuff, [20:52] I have it synced to my Mac. I'll just [20:54] jump on the codeex app. And to your [20:56] point, with browser, with the computer [20:58] use and stuff like that, evidence-driven [21:00] testing just becomes super super [21:01] powerful, right? So I split up work that [21:04] way. Also, the reason why this exists is [21:07] because we were talking about this [21:08] earlier. I I I do have some new [21:10] responsibilities now. So I'm not really [21:12] at the office and I can work from my [21:14] phone. And I know Codex has the remote [21:16] control as well, but I like to use again [21:18] Fable and other models especially when [21:20] they come out because we have to make [21:21] you and me have to make content like [21:22] this. So I'm sort of forced to use [21:24] everything to figure out what's the best [21:26] thing right now. [21:27] >> Got it. Okay, awesome. Well, let's [21:30] finish explaining the workflow. So, can [21:32] you explain what Grap Loop is? [21:33] >> Yeah, so Grap Loop is a skill by [21:36] Gravile, which is a code review agent [21:38] company. I am a big fan of them. There [21:40] are other great code review agents as [21:42] well. I just like reptile. And what the [21:44] GP loop skill does is basically it's [21:47] this loop. And maybe I can give an [21:49] example with a PR. Uh whenever you file [21:52] a PR with when you have GPA installed, [21:55] maybe I can find one that doesn't have a [21:58] five out of five. But essentially, just [22:01] to explain this, every time you make a [22:04] PR, if you have Grapile installed, [22:06] you're going to get a GP summary and a [22:09] confidence score and some feedback, [22:11] right? feedback, right? For example, it [22:13] says do not assume cash funding. This is [22:15] feedback. This is a P1. It explains and [22:18] it even gives you some artifacts to [22:20] check out. What I'm really interested in [22:22] though is this confidence score. The way [22:23] they have this confidence score [22:25] calculated is really, really good. Most [22:27] models, your Fables, even Astra, usually [22:32] the first PR they make is a four out of [22:35] five. Sometimes it'll be a three out of [22:37] five, but with the state-of-the-art [22:38] models, you'll get a four out of five. [22:40] and you'll get one piece of feedback. [22:42] What the GP loop skill allows u your [22:46] agent to do is and I'll um actually why [22:50] don't we read the skill itself because [22:51] it will explain it properly. What this [22:54] skill basically does it says I'll read [22:56] it here. Iteratively improves PR until [22:59] grapile gives a five out of five [23:01] confidence score with zero unresolved [23:03] comments. Triggers gravile review fixes [23:06] all actionable comments. pushes, [23:09] re-triggers review, and repeats. Use [23:11] when user wants to fully optimize PR [23:14] against Gravile's code review standard. [23:15] So basically to draw this cuz this is [23:18] when I heard of loops at first wasn't [23:20] really a fan, but loops in this sense [23:21] are great. So I'm ready to merge my [23:24] feature, but I am now running Gravile. [23:27] What Grav will basically do is it gives [23:30] me a score. My agent checks the score [23:33] and if the score, let's say, is a three [23:36] out of five, right? What essentially [23:38] happens is if I go back to my skills so [23:41] we can visually understand this. What [23:43] essentially happens is we go back to [23:45] step two. So it starts to it takes the [23:47] feedback in. It starts to write code. It [23:50] will follow step three. It will follow [23:52] before and after. And then it'll push a [23:54] change. The agent will then wait for [23:56] Gravile's new review. If it's a five out [23:59] of five, it stops and it tells me the PR [24:02] is ready. This is a five out of five. [24:04] What's essentially good? What's [24:06] essentially the time for me now is I can [24:08] look at the code if I want to or I can [24:10] merge. I basically have all the evidence [24:12] I need to merge. But what this skill [24:14] does is it takes Gravile, which is a [24:16] code review agent, uses the score as a [24:20] way to check if the code generated by my [24:23] agent was good. Right? Anytime Gravile [24:26] gives any sort of feedback or a score [24:28] under five out of five, the GP loop [24:30] fires. It takes in the feedback. It [24:32] writes the code. It pushes to GitHub. [24:35] Grab does its check again. And this loop [24:38] doesn't stop until you leave reach five [24:41] iterations. My version is 10 iterations [24:44] or you get a five out of five. [24:46] >> Okay. So basically where and all these [24:48] skills are open source and free, right? [24:50] They're available on GitHub. [24:51] >> Yeah, this is all available on my repo. [24:53] Gre loop again is their skill. I just I [24:55] literally brought it in and you might [24:57] have seen a GP loop apps. This one is [24:59] basically for PRs with 3,000 4,000 plus [25:02] lines. These are their skills, right? Um [25:04] the only skills that are mine that I [25:06] made are code structure and [25:08] evidence-driven testing and new feature. [25:10] I even have the unslopped skill. This is [25:12] from potato, one of the engineers from [25:15] uh cursor now. And this just, you know, [25:18] unslops the PR. Again, to your point, um [25:21] Astro doesn't really follow this. It it [25:24] tends to skip unslop. [25:28] This is essentially the crux of the [25:30] workflow, right? Isolate new work tree, [25:34] write code to a specific standard, prove [25:36] that a bug exists if a bug is what's [25:38] being fixed. If not, prove that the [25:40] solution you made actually works via [25:42] video. If that doesn't work, before and [25:44] after screenshot, if it's not [25:46] screenshotable material, give me some [25:48] stats, write some tests, and give me [25:49] some stats to actually prove. Push this [25:51] to GitHub. If the gravile code review [25:54] agent gives a score less than five out [25:56] of five, we run a loop until we get a [25:58] five out of five. And then what I end up [26:00] getting is responses like this. Updated [26:03] PR 155 documentation checks passile [26:07] remains pending after 10-minute wait. So [26:09] I get basically at the end I get PRs to [26:12] review, right? So this is a very human [26:16] in the loop but my workflows injected um [26:20] software factory. So just to summarize, [26:22] it sounds like basically what you're [26:24] doing is you have this shipping workflow [26:25] that chains together a bunch of skills, [26:27] right? That that that you kind of either [26:29] copied or built yourself. And it seems [26:31] like both step three and four are like [26:33] really focused on verification, right? [26:34] So like and where the loop comes in is [26:37] really around verification, just like [26:39] having it self-improve and make sure the [26:40] thing actually works. [26:41] >> Exactly. Right. I'm not at a point where [26:44] I trust the agent, right? Especially [26:45] with stuff that people are using. For [26:48] example, I have this um I I have these [26:50] two little mini apps, right? I have this [26:52] image service that I built. Um and it's [26:55] so that I can share images between [26:57] machines, right? I didn't need to read [26:59] the code. I just wrote / Go. It worked. [27:02] It was good enough for me. I have [27:03] another tool called Still that just [27:06] takes really good-looking screenshots. I [27:08] don't really care how the code looks. I [27:10] don't care for the reptile score. To me, [27:12] what matters is this works. I can copy [27:14] this. I can go to Twitter. I can paste [27:16] my screenshot. I can tweet hopefully get [27:18] some likes, right? So for software like [27:20] this um this workflow doesn't really [27:23] matter to me, right? But this I employ I [27:27] use when I I I want other people to use [27:30] it, right? And I don't trust the agent [27:32] uh to do all the right things all at [27:34] once. This sort of gives me a [27:36] verification process where I can sort of [27:37] build trust with it. [27:39] >> That makes sense. Yeah. Yeah. If you're [27:40] doing it for a client or something like [27:42] if you're doing for personal software, [27:43] maybe you can slopify a little bit. But [27:45] >> yeah, you'll have fun. [27:47] >> I think even for personal software dude, [27:48] like I've tried like leaving it running [27:50] for the whole night to build a game for [27:52] myself or something and then if it makes [27:53] like one wrong assumption at the [27:55] beginning, the whole thing is just a [27:56] waste of token. [27:56] >> Yeah. It just falls apart. Right. And [27:58] this is why software factory as it's [28:00] explained and broadly known. I'm just [28:02] not a fan of right because one bad wrong [28:05] assumption in the beginning just [28:06] destroys everything. Right. Um and and [28:09] this is true with real engineering and [28:11] real orgs, right? when there's a lack of [28:12] communication and things are not [28:14] discussed, stuff hits the fan, right? [28:16] So, it's just a lot faster with agents [28:18] now versus with humans. [28:20] >> Uh, one thing that I think you didn't [28:21] quite cover is how do you let let's say [28:23] you want to ship a new feature? Do you [28:25] write some sort of a spec or do you just [28:26] kind of vi vive voice like hey built [28:29] this feature here's like stream [28:30] conscious [28:31] >> my my planning and this is just how I am [28:33] as a person. I like to the way I think [28:36] is I talk through, right? So, the way my [28:39] planning sessions really work and I [28:41] could write this down is I literally yap [28:43] to the to the agent for I would say a [28:46] good amount of time like 5 to 10 minutes [28:48] where and I guess this is just how my [28:50] mind is wired. I'm even for personal [28:52] notes and stuff like that or the way I [28:55] sort of train or how I study or even [28:57] like how I read books and stuff like I [28:59] have to read out loud like I have to [29:01] hear it. So the reason why I I I've [29:03] never really done a detailed I've have [29:05] like one video on planning maybe is [29:07] because I understand conventionally most [29:08] people might not think like this but the [29:10] way I plan is I'll have something in my [29:12] mind and the only way I can get it out [29:14] and explain it thoroughly is if I [29:16] continue to talk. So, I'm pressing Fn on [29:19] my Mac with whisper flow and I'm having [29:22] back and forth like and I maybe can [29:25] bring and this here's the thing with [29:26] codeex it loves to do all this tool [29:30] calling but I essentially will go back [29:32] and forth with my agent discussing even [29:35] random ideas like for example um I think [29:38] you might find this interesting I have [29:39] this app called Bezelo and I was using [29:42] email service for agents and I hit the [29:44] free tier. I hit the free tier and you [29:47] know I it got a little expensive to [29:49] upgrade. So what [clears throat] I did [29:51] was I literally had a back and forth [29:53] with my agent and this is another [29:55] example. I'll explain this example in a [29:57] second. I went back and forth with my [29:58] agent and it was like, "Oh dude, you [30:00] could use Cloudflare email and you could [30:03] basically build the same thing. It [30:04] wouldn't be too hard. There's a couple [30:05] things that we missed." And I I looked [30:07] at the Cloudflare email docs. I read [30:08] about it and we continued to have this [30:10] back and forth. It took about like 27 [30:12] minutes or 30 minutes of back and forth [30:14] till I finally realized, okay, this is [30:16] the feature I want. I wanted to have [30:17] tracking. I wanted to have custom [30:18] domains. So, planning for me isn't the [30:22] agent writing the spec and me reviewing [30:24] the spec and me saying, "Go build it." [30:27] Planning for me is me and the agent [30:28] discussing [30:30] and I'm having a yap session and then [30:32] I'll tell the agent what I want and then [30:34] I'll go build it. Right? So again, not [30:36] the most conventional way for most [30:38] people, but I will find if you are the [30:40] type of person who thinks who who needs [30:43] to talk when they think, this is perfect [30:45] for you. [30:46] >> Okay. So you don't actually make a spec [30:48] document. You just kind of like the [30:49] conversation history is the spec. [30:52] >> Yes. The conversation history is a spec. [30:53] Sometimes like as a reminder, for [30:55] example, I think I might have a PR to [30:58] show this. Um, [31:02] so I have a Mac machine and I'm paying [31:04] for a service that allows my agent to [31:07] connect to iMessage and and they're [31:09] pretty expensive. It works well. SDK is [31:11] pretty nice. Um, but it's a little [31:13] expensive. So I was like, "Oh, what if [31:15] we built our own?" And it wrote, we had [31:18] a discussion, wrote a plan. But then [31:20] what I asked it to do was I asked it to [31:24] write a markdown file called setup.mmd [31:26] because it was like there's this whole [31:27] setup and things I had to install and I [31:29] didn't want it to do it right now. So I [31:32] asked it to make the setup a setup.md [31:34] file and to put it there. So for me, [31:37] files like this are important for [31:40] myself, right? The agent has context of [31:42] the conversation. So if you're going to [31:44] build right there and then, you can just [31:45] tell it to build. But for certain things [31:47] that I want to remember, I want to come [31:49] back to, I want the agent to come back [31:51] to, I will ask it to create files within [31:54] the codebase itself. So right here, this [31:55] is a monor repo. There's an iMessage [31:57] folder. In the iMessage folder, there's [31:59] a setup.md. So when I come back to this [32:01] and I ask the agent, oh, how do I set [32:03] this up? It'll know exactly how to do it [32:04] because we had the plan and it was all [32:06] written here. Right? So that's kind of [32:08] how I use the agent to create files. [32:11] It's partly also for me because I'm a [32:13] very forgetful person and this helps jog [32:16] my memory and the agents can pick it up [32:18] pretty quickly. [32:19] >> Got it. Okay. So, let me ask you uh [32:21] Okay, so let me ask you this. So, if I'm [32:23] just building my own personal apps, is [32:25] it still best practice to use work trees [32:27] and even have PRs and stuff or should I [32:28] just ship to prod? [32:30] >> No, honestly, if you're building a [32:31] personal project and you you don't [32:33] really care, there's no really high [32:34] standard for it. Just have fun, right? I [32:36] think there's something fun about speed, [32:38] right? I remember when was it Composer [32:40] when it first came out? When Composer [32:41] first came out, it wasn't really the [32:42] best model, but it was so fast that I [32:45] don't know, Peter, if you share the same [32:47] experience, but I loved the speed. It [32:49] was so fun. It felt like a video game, [32:51] right? So, I think there's something fun [32:53] about speed. And if you're building a [32:55] personal app, I I think going fast just [32:58] makes it fun. Um, you don't want to make [32:59] it tedious. You don't want to have [33:01] unless you're trying to sharpen your [33:02] skills. I would just go guns blazing. [33:04] That's what I do for some personal apps [33:06] that I have here. [33:06] >> Got it. Okay. And and on the model [33:08] front, I think you had a diagram for it, [33:10] but like basically you're using Astra [33:11] for everything. You're using Astra for [33:13] planning and then so to execute. [33:15] >> So the the way I have it like a couple [33:17] things, my benchmark, the Michael [33:19] benchmark is usually running a [33:21] performance audit, a security audit, [33:23] some UI and computer use for all OpenAI [33:26] models, right? [33:27] >> In terms of performance, if you are [33:30] using any sort of modern framework like [33:32] Nex.js, JS, felt kit or any sort of [33:34] React framework, Astra is almost n out [33:38] of 10 times great at finding some sort [33:40] of speed improvement when it comes to [33:42] either navigations, loading things, [33:44] caching, pre-fetching. It really is good [33:47] at that type of stuff, right? So, if you [33:49] have an existing app, the two things I [33:51] would highly, highly recommend is [33:53] running a performance update, checking [33:55] out your navigation, seeing if things [33:57] are a little slow or something's load [33:59] every single time you click back at it. [34:01] Maybe there's some caching that could be [34:02] added, some pre-fetching that could be [34:04] done. Astra is really, really good at [34:06] that. Uh, number two, and this is the [34:08] big reason why if I had to pick between [34:10] Astra and Fable, I would stick with [34:12] Astra, as much as I love Fable, is [34:14] because I can run security audits [34:16] without getting stopped. and being [34:19] forwarded to a lesser model, right? With [34:21] Fable, I mean, they said they make [34:22] improvements to 5.1, but I'm every time [34:25] anything I talk security related, I'm [34:27] getting booted to Opus 5, and I'm not [34:29] really a big fan of that. One app that I [34:31] I build that actual users use is my [34:34] church's payment system. I I built a [34:36] custom payment system for my church, [34:37] partly because I'm a I'm a developer, [34:40] but partly also because a lot of the [34:42] payment services in Canada, the fees are [34:44] exorbitant. They're in incredibly high. [34:47] So, I built one and lots of money has [34:49] flowed through. I'm I'm actually going [34:51] to onboard other churches as well um to [34:53] use it. I'm not charging them anything. [34:55] But here's the thing. I asked Astra to [34:56] run a security audit and boy oh boy, I'm [34:59] glad I did that because there were some [35:02] things that it caught some receipt [35:04] mechanisms that were an issue where if [35:06] you know people knew how to read u you [35:09] know the network tab and did like there [35:10] was a lot of like there was one [35:12] particular bug that security issue that [35:13] I found that would have been [35:15] detrimental. Right? So if you have any [35:17] sort of app that is being used by actual [35:19] people and money is involved in [35:22] particular I beg please uh run a [35:24] security audit. It's really get great [35:26] with that. [35:27] >> Got it. [35:27] >> UI [35:30] questionable. It's good if you give it [35:32] references. For example, this app I'm [35:35] working on bezel. The UI was designed by [35:38] Astra, but and I, you know, it it has [35:41] this Windows 95 XP feel, but the reason [35:44] why it has that is because I gave it the [35:46] Windows XP network uh logo. The examples [35:50] it generated before it it had that open [35:54] AI smell where it does the headers with [35:56] the small text but all caps. Um, it [35:59] loves random copies. Uh, the buttons are [36:02] exaggerated. So you know cable is miles [36:05] better in my opinion when it comes to [36:06] UI. [36:07] >> I I I think if you use the if you use [36:09] chat images to generate image first. [36:11] >> Yeah. So and that's another thing right. [36:12] So images is a great way to uh give [36:15] references. I find that if you give a [36:17] reference image again I gave it [36:19] reference images and it made this is [36:20] great right but I find fable off rip is [36:24] just better with no references right but [36:26] again if you're building software you [36:28] should give references. I agree using [36:29] images helps. And last but not least [36:32] computer use. I mean, we've all seen the [36:33] demos. Fantastic, incredible, amazing. [36:37] This is to me a leap in technology when [36:40] it comes to this. So, this is my thought [36:41] process when it comes to Astra. Now, why [36:44] do I still have my Fable subscription? [36:47] Fable writes better code. Fable writes [36:50] like if if I wanted to join a company [36:54] and I wanted to LAR as an engineer even [36:57] though I let's say I wasn't and I wanted [37:00] to ship PRs and a human would review my [37:02] code. I find that you would get away [37:04] with more code being merged with Fable [37:06] than you would with Astra, [37:08] >> right? So I I still think Fable is the [37:11] better uh programmer uh per se, which is [37:15] why I don't think I would cancel my [37:16] anthropic subscription. Uh but I I would [37:20] say overall Astra is definitely the [37:23] better model especially when it comes to [37:25] other things again performance updates [37:27] security audit and the computer use [37:29] stuff and if you want to get into [37:30] hardware I can also talk about that if [37:32] you'd like. Astra is definitely leaps [37:34] and bounds when it comes to that [37:35] >> and maybe that's why you used to use the [37:36] terminal so you can use both Astra and [37:38] Fable at the same time. [37:40] >> Yes. Right. And that's the thing too [37:42] being able to switch on terminal versus [37:44] and I also find when I was using the [37:46] codeex guey for a while I really loved [37:49] the codeex guey and then when I tried to [37:52] use cloud codes like claude's gooey it [37:54] just I I it it was really hard and [37:56] difficult for me to get used to it. So [37:58] all the terminal interfaces are pretty [37:59] much the same except open code has just [38:01] the best one but the rest are pretty [38:03] much the same. So switching between [38:04] models, switching between cloud code or [38:06] codecs or even cursor um it doesn't [38:09] really there's not much of a dread [38:11] between switching. [38:12] >> Got it. Okay. So basically you're using [38:14] Astra to like find issues and fable to [38:16] code, right? I guess maybe most people [38:18] can't afford using the two best models. [38:21] >> Yeah, it is an expensive thing, right? [38:22] Which is why I would say I would just [38:23] stick to Austra um and and to codeex and [38:26] the reason being is the resets are also [38:28] very generous, right? So if prices are a [38:31] little too much, I will say OpenAI is a [38:34] lot more generous um than Anthropic, [38:36] although they've started to do resets a [38:38] little bit more. But I know people who [38:39] have like three resets ready to use, [38:41] right? So yeah, you have to be smart [38:43] with your money and I would just go with [38:44] the one subsidizing you the most. [38:46] >> I think I I'm I'm not obviously a [38:48] programmer like you, but I also find [38:49] Soul still pretty reliable just so [38:52] medium effort. It's like for routine [38:54] stuff is pretty good at just getting [38:55] stuff done. So [38:56] >> yeah, Soul is pretty good. what other [38:57] models actually kind of slept on in my [38:59] opinion for maybe knowledge work type [39:01] stuff. Luna is really good like uh and [39:04] and it's it's very cheap, right? So for [39:06] my personal agent type work, I I'm using [39:09] Luna and it's pretty good at computer [39:11] use stuff. It's pretty good at any sort [39:14] of knowledge work stuff. I would I I [39:16] definitely think the way we should think [39:18] about models is there's no need to use [39:21] state-of-the-art for everything. [39:23] >> Yeah, [39:23] >> I think there's a place for [39:24] state-of-the-art. obviously take [39:26] advantage of the subsidies if you're [39:28] within the higher tier uh subscriptions, [39:30] but to your point like yeah, GBD 5.6 [39:32] Soul for computer use is still really [39:35] good, right? Um Luna for a lot of like [39:38] knowledge work. Like for example, I had [39:40] some tax work I had to do. I've been [39:41] married for like a year and a half now, [39:43] so our taxes are joint. So doing my [39:45] wife's taxes and my taxes, and there's a [39:47] lot of numbers and and expenses to [39:49] crunch. And the way Luna just blew [39:52] through that and then I used Soul as a [39:54] sub agent to track Luna's work. It was [39:56] just fantastic. Right. So don't fall [39:59] into this trap of having to use the [40:02] state-of-the-art for everything. I find [40:04] in fact the really big models can do [40:06] really dumb things for minuscule tasks. [40:09] So I I think model routers will be a big [40:12] thing in the next couple of weeks [40:14] because we don't need state-of-the-art [40:15] for everything, right? If if anything, [40:17] state-of-the-art should probably stick [40:18] to orchestrating and reviewing because a [40:21] lot of the smaller models are getting [40:22] really really good. Grock 46 is really [40:24] really good model, right? So, this is [40:26] why I think again take advantage of [40:28] subsidies, but don't think there's a [40:30] need to use state-of-the-art for [40:32] everything. I I just don't think that's [40:33] necessary. [40:34] >> Yeah, it could totally backfire if you [40:35] use Astra Ultra to like ship some simple [40:37] PR. It'll just like overthink [40:39] everything. [40:40] >> Yeah. No. And it'll take a lot longer. [40:41] It'll burn tokens. It's just it's just [40:43] not a good day. [40:44] >> And Okay, so let me ask this. So my last [40:46] question is you walk through the [40:48] software factory the steps. I'm really [40:50] curious if you have any schedule task or [40:52] cron jobs to make this thing a little [40:54] bit more proactive. So for example like [40:55] the performance security audit thing [40:57] like maybe you have it run every week or [40:59] something because it probably finds [41:00] something each time. [41:01] >> I let me I I just built streamer mode [41:04] for this. So I'll show this right here. [41:07] So this is an overview of my home lab [41:11] setup [41:12] >> and one thing that I recently built. So, [41:14] I used to use cursor cloud agents for [41:16] that. Um, and I still do. But one thing [41:19] that I just recently built is my own [41:21] sort of agents workspace and because I [41:24] have a Linux box, I have uh a Mac [41:27] machine, I have a couple other machines [41:29] on the way and yes, that's what I plan [41:32] on building on my own infrastructure. [41:34] So, currently I have that on cursor [41:36] cloud. It's more so for security stuff. [41:39] All right. Uh, especially for client [41:40] apps. One in particular is in the works [41:44] with spec government companies and the [41:46] standard is extremely extremely high. So [41:47] we have a consistent schedule job [41:50] running on cursor cloud agent that [41:52] always checks every single day at a [41:54] specific time for uh those issues. But [41:57] if you have a couple old dusty computers [41:59] lying around, I highly suggest people [42:01] connect them up, use codeex. By the way, [42:03] I'm not a network person. All this is [42:05] Fable uh GBT 5.6 Soul and a little bit [42:08] of Astronaut. So, in my like home lab, I [42:11] plan on running stuff like this. But to [42:13] answer your question, simply like I [42:15] said, I like the app. Yes, I do that. I [42:17] think for production apps, running just [42:19] a simple security audit is 100%. Should [42:22] be mandatory. [42:23] >> Got it. Yeah. I I have problems where I [42:25] only have my main laptop and the Mac [42:28] Mini and some some of the stuff is in [42:29] the cloud, right? Like like the Gro [42:30] Crockbot stuff is in a cloud and then [42:32] I'm at a point where I don't even know [42:33] what kind of crown jobs I have anymore [42:34] because [laughter] they're like spread [42:36] across three different things, you know? [42:40] consolidate everything the dashboard to [42:42] track everything too because like I [42:43] remember I think a week ago or two weeks [42:44] ago I have a pretty beefy Linux box and [42:47] it was like we're out of close to out of [42:48] memory. I'm like how? And then I asked [42:51] Codex to look and it's like oh you have [42:53] 3,000 something work trees you have that [42:55] you have this running these dependencies [42:57] and yeah I asked it all to clean up. So [42:59] it it can be cumbersome. [43:01] >> Yeah. I I I think if anything it's good [43:02] to have a crown job where just cleans up [43:04] all all your It's like hey [43:05] >> and that's a good thing to have. Yep. [43:07] Yep. [43:07] >> Yeah. Awesome dude. Well, how how do you [43:09] think this software factory that you set [43:11] up is going to evolve like you know [43:13] [snorts] with the newer models and stuff [43:14] or do you think it's going to be pretty [43:15] consistent? [43:16] >> I see both sides, right? I found as the [43:19] models get better, we tend to use less [43:21] and less skills, right? But I also sort [43:24] of disagree with that statement because [43:26] I find skills before um were uh [43:30] descriptors of the things that the model [43:32] had access to versus specific workflows, [43:35] right? So [43:36] >> I think my software factory will [43:39] somewhat exist as the models make leaps [43:42] and leaps because this is more about how [43:44] I work versus u like a specific [43:47] explanation of how the code is right so [43:49] let's say the next model is really great [43:52] at writing code the one thing I would [43:54] say gets tweaked for here is maybe code [43:56] structure gets destroyed right because [43:57] it just writes better code and it just [43:59] knows how to write really really elegant [44:02] code I still wanted to create a new [44:04] worksh Let's say they they the the [44:06] harnesses start to do that, right? We'll [44:09] nuke work tree and maybe the agent after [44:12] pushing a feature starts to loop, right? [44:14] So I do see a future where maybe not the [44:17] model but the harness will start to [44:19] incorporate better engineering practices [44:22] and these things start to get removed. [44:24] Right? So one thing that I do is every [44:26] time I use a new model comes out I like [44:29] I start fresh with no agents.mmd with no [44:32] skills just to see how it is and then [44:34] I'll incorporate my software factory. So [44:36] yeah at some point this will be this [44:38] right but for now it's doing a pretty [44:41] good job. [44:42] >> I I think uh the most durable skills are [44:44] already kind of skills personalized for [44:46] your own taste and preferences and like [44:49] unless there's some god model that just [44:51] like does the best job for everything. I [44:52] think everyone has different tastes and [44:53] preferences right? So like I don't see [44:55] skills going away entirely. [44:57] >> Agreed. Agreed. I think explaining stuff [44:59] to the agent that it already has access [45:01] to that's for sure not going that that's [45:03] gone. You shouldn't do that. But yeah, I [45:05] think we're all our domain knowledge and [45:07] expertise can be different and skills [45:10] influenced by how you work. I think yeah [45:12] to your point unless the to your point [45:15] the new model in harness interviews how [45:17] you work you explain to it and then it [45:18] just knows how to work like that for [45:20] forever. [45:21] >> Yeah. Yeah. or maybe it's a memory thing [45:23] or something. It's like just really got [45:25] that. But I also think just to kind of [45:26] wrap this up, I feel like you kind of [45:28] have to be a little bit skeptical about [45:30] some of the best practices coming from [45:31] the labs because the engineers who work [45:33] in the labs are are just kind of they [45:34] have unlimited tokens, right? So they [45:35] can just do they can just do whatever [45:37] the hell they want. And for for the rest [45:39] of us, we have to like be a little bit [45:40] more careful. We just run loops all day [45:42] long, [45:42] >> you know? [45:43] >> Yeah. So there was this company I used [45:44] to work at and whenever we shipped a [45:47] feature, we actually used to have older [45:50] laptops in the office. Yeah. [45:52] >> And the reason why we have that is when [45:54] we were testing the features, we were [45:55] forced to test it on older laptop [45:57] because the thing is we're testing it [45:58] out on these beautiful MacBooks with all [46:01] this power and yeah, everything feels [46:03] fast, right? But most people don't have [46:05] a machine like this, right? So to your [46:07] point, this is why I found a lot of the [46:10] open claw founders and certain people in [46:12] the lab companies their advice because [46:14] it just sounds like someone who has [46:15] bajillion tokens to burn telling me [46:18] exactly how I'm supposed to do things. [46:20] saying even with the subsidies great we [46:22] still have limits right so I 100% agree [46:25] sometimes some advice from the labs can [46:27] be a little bit cumbersome and also at [46:29] the end of the day they are a business [46:30] and they want your business and they [46:32] want you to spend more money right so [46:34] this is why philosophically I'm super [46:36] supportive of open source models and [46:38] open weight models especially ones we [46:40] can run on our machines because I feel [46:42] like there will be a time where that [46:43] stuff will be super super important [46:45] >> yeah you don't want to put all your eggs [46:46] in one basket so yeah awesome Michael [46:48] well I will link your YouTube channel in [46:50] the description and thanks for sharing [46:51] your software factory and being real [46:53] about it and like not hyping it up. [46:55] >> I appreciate you again, Peter, for [46:56] bringing me on and thank you so much. I [46:58] It really means a lot. Awesome, dude.