Sixfold Replaces Human Code Review With AI Agents

Sixfold 用 AI 智能体取代人工代码审查 由于自主编程智能体生成了公司 90% 的代码库，Sixfold 取消了人工拉取请求审查，将工程重心从语法彻底转向系统架构。 Joanna Wang 介绍了 Sixfold 的多语言架构：其中 Ruby on Rails 后端作为数据库的唯一网关，而 Python 微服务则负责管理保险核保的 AI 评估。在并行扇出和扇入的 Web 数据摄取管道遭遇严重的竞态条件后，团队开始用 Hatchet 的 DAG 驱动工作流引擎取代脆弱的 ActiveRecord 回调和 Faktory 队列。Wang 还透露，Sixfold 已过渡到完全智能体化的编程模式，AI 智能体生成了大约 80% 到 90% 以上的代码，并在结构化的 agent.md 框架引导下通过 CodeRabbit 处理拉取请求审查。Wang 和 Russell 共同探讨了 AI 助手如何帮助非 Ruby 工程师揭开 Rails 元编程的神秘面纱，并指出尽管代码生成的成本已变得低廉，但开发者在系统架构和边界设定方面的专业能力比以往任何时候都更加重要。 Sixfold 的工程团队已迁移至完全智能体化的编程模式，由 AI 智能体生成 80% 至 90% 以上的所有代码并处理自动化的 PR 审查。 (24:55) 为解决 Web 数据摄取期间复杂的异步同步和竞态条件问题，Sixfold 将工作流编排从 ActiveRecord 回调迁移到了 Hatchet。 (13:58) 核心应用采用多语言架构，由 Rails REST API 管理数据库访问和后端逻辑，并与 Python 微服务交互以进行 AI 模型调用。 (4:17)

Software engineer Joanna Wang explains that as AI agents take over 80% to 90%+ of manual code writing, software engineering is pivoting toward architecting robust harness systems, scaling polyglot backends, and managing orchestration complexity. Redefining Developer Roles in an Agentic Paradigm Sixfold has transitioned to an almost entirely agentic development environment where human engineers write virtually zero syntax by hand and no longer perform manual pull request reviews. Instead, tools like CodeRabbit conduct automated PR reviews against structured repository harnesses and `agent.md` documentation files that lay out architecture patterns and pitfalls. Because AI agents make code generation cheap, Wang contends that the core value of software engineers has migrated away from syntax proficiency and into high-level system architecture, problem framing, and building resilient systems. While agentic tools like Claude Code excel at demystifying Rails' metaprogramming conventions—allowing Python and Go engineers to ship full-stack features independently—they present new operational tradeoffs. AI agents often generate excessively defensive validation logic unless explicitly corralled. Moreover, podcast host Robby Russell cautions that neglecting architectural discipline risks substituting human maintenance debt with massive token burn, a significant risk given that today's development workflows operate in a temporarily subsidized compute environment. Polyglot Architecture and Overcoming Callback Fragility (13:58) Sixfold’s AI-assisted insurance underwriting platform relies on Ruby on Rails as a strict, sole gatekeeper to the database and API backend, while Python microservices handle generative AI workloads and external model provider APIs. In the bootstrap phase, Rails accelerated development by eliminating boilerplate for an engineering team hand-crafting every line of code without AI assistants. However, managing complex asynchronous workflows through model callbacks like `after_create` and state machine gems became an architectural failure point under scale. The system reached breaking points during asynchronous web ingestion tasks requiring parallel fan-out and fan-in coordination. When scraping websites to assess risk, parallel background jobs had to fan out to Python services for relevance checks while matching user-uploaded documents, then synchronize before triggering final underwriting analysis. Coordinating this sequence through Active Record callbacks and the Faktory background job queue produced persistent race conditions, transaction dependencies, and severe queue bottlenecks, prompting Sixfold to pilot and migrate workflows to Hatchet to declare explicit directed acyclic graphs (DAGs). The Tension of Rails Magic and Custom Extensions (18:36) Sixfold's REST API layer relies heavily on convention via the `jsonapi-resources` gem, resulting in controllers that are virtually empty on disk while serving full CRUD functionality. However, this level of abstraction introduced severe rigidity. Surfacing custom Active Record validation errors proved difficult because the gem dictates its own error formatting. Because the upstream open-source repository became unmaintained, Sixfold was forced to maintain an internal fork to patch issues. This experience shaped Sixfold's dependency strategy: relying on open-source gems for small tools while focusing proprietary engineering on their core underwriting platform. Given supply chain security risks—amplified by npm vulnerabilities on their TypeScript frontend—the team uses automated vulnerability scanning and rejects automated Dependabot updates for major Rails and Active Record versions in favor of strictly supervised, manual migrations. Multi-Tenant Isolation and Staged Enterprise Deployments (35:28) Because insurance underwriting involves sensitive personal and corporate records subject to HIPAA, SOC 2, and international data residency laws, Sixfold implements strict tenant compartmentalization. To prevent customer uploads and AI inferences from leaving designated geographical boundaries such as the European Union, the team mirrors infrastructure across regional boundaries, deploying independent, hosted Hatchet workflow clusters alongside regionally isolated databases. Continuous deployment pipelines deploy in parallel across regions, while runtime feature delivery is controlled via an internal configuration platform rather than third-party SaaS like PostHog. Rather than using fractional canary deployments, Sixfold toggles features at 100% for targeted alpha and beta enterprise cohorts after staging changes through sandbox and User Acceptance Testing tiers. End-to-end regression testing has transitioned from manual human QA to automated verification using QA Wolf and Capybara. As automated code generation commoditizes implementation syntax, engineering teams face unresolved questions regarding whether classic clean code principles remain practical or if messy agentic outputs are acceptable so long as workflow harnesses, regulatory boundaries, and underlying data orchestration remain stable.

[1:17] Joanna Wang: you're probably not the only one or you know you're not the first one to encounter almost anything. [6:53] Joanna Wang: what's the opposite of verbose?... it's really not verbose and like you can write very little code. [13:58] Joanna Wang: at every possible place there is a race condition. [18:00] Robby Russell: This episode of On Rails is brought to you by ApplicationController, where all roads lead and no logic escapes. [18:00] Robby Russell: Because if you don't know where it goes, it probably goes here. [19:58] Joanna Wang: with the magic comes the rigidness of it. [19:58] Joanna Wang: a very hard thing to then surface in the way you want it [21:55] Joanna Wang: our value is not in creating those like in like smaller tools, but in creating this like overall system for insurance underwriting. [24:55] Joanna Wang: We are migrating to like well at this point I think the migration to fully agentic coding is done. [28:19] Joanna Wang: putting in the harnesses for the agents instead of like just writing code. [28:19] Joanna Wang: code is cheap now. [40:18] Joanna Wang: We have a lot of race conditions in the current state of workflows. [40:45] Joanna Wang: messy code is okay as long as it works [40:45] Robby Russell: burning a lot more tokens in the future to do things. [40:45] Robby Russell: slightly subsidized token era of development

[0:00] [music] [0:05] Welcome to on rails, the podcast where [0:06] we dig into the technical decisions [0:08] behind building and maintaining [0:09] production rub on rails apps. And I'm [0:11] your host, Robbie Russell, and I run [0:12] planet Argon. And for over 21 years, [0:14] we've helped teams maintain and evolve [0:16] their long live Rails apps. So I tend to [0:18] approach these conversations through [0:19] that [music] lens. In this episode, I'm [0:22] joined by Joanna Juan, senior software [0:24] engineer at Six-Fold AI. [music] Sixfold [0:26] helps insurance teams analyze complex [0:28] cases using a mix of document [0:30] processing, automated workflows, and AI [0:32] powered tools. [music] Joanna has worked [0:34] across Java, Go, Python, Node, and Ruby [0:36] throughout her career, which gives her a [0:38] unique perspective on what Rails does [0:40] well, and where teams can run into [0:42] challenges as their applications grow. [0:44] We'll talk about what Rails has enabled [0:46] for her team, the realities of managing [0:48] increasingly complex workflows, and [0:50] whether agentic coding tools are [0:51] changing how developers think about [0:53] Rails famous magic. Joanna joins us from [0:56] New York City. All right, check for your [0:58] belongings all aboard. [1:02] Joanna Juan, welcome to On Rails. [1:05] >> Thank you. Thank you. It's great to be [1:07] here. [1:08] >> I'm so glad that we were able to finally [1:09] make this happen and we think we had a [1:11] conversation maybe three or four months [1:13] ago and we're like, "Oh yeah, this would [1:14] be a great conversation." So, we're [1:16] here. Let's do this. All right. So, [1:18] Joanna, let's start with this. What [1:20] keeps you on Rails? [1:22] Well, Rails has just a very almost [1:25] completed ecosystem. There just a lot of [1:28] maintainers and it just has a really big [1:30] community and dev base so that if you [1:33] have any problem with it or anything, [1:36] you're probably not the only one or you [1:38] know you're not the first one to [1:39] encounter almost anything. So, I think [1:42] that's a really great thing. [1:44] >> Now, we'll talk about sixfold in a [1:46] moment, but if I recall, you didn't work [1:48] with Rails prior to your current [1:50] position. Is that correct? Correct. [1:52] >> Just to set the scene a little bit, what [1:53] technology, programming languages, [1:55] stacks were you kind of working with [1:56] prior to Ruby on Rails? [1:58] >> I started with Java in college. I was a [2:01] computer science major and then I worked [2:04] on Python for about a year or so and [2:08] then I was back on Groovy on Grails, [2:11] which is actually basically it just also [2:14] a Java JVM language suite. [2:17] >> Did you ever out of curiosity work with [2:19] J Ruby at all? [2:20] >> No. Okay. I don't Do you know what that [2:22] is? No. Okay. Uh well, it's actually a [2:24] way it's it's a way that you can run [2:26] your Rails like app within the JVM. So, [2:28] it's like a it's a Java based Ruby [2:32] interpreter. So, it's uh [2:33] >> interesting. [2:34] >> It was a way that a lot of organization [2:35] back maybe like 15 20 years ago were [2:37] able to kind of like sneak Rails into [2:39] their Java environment and deploy it. I [2:42] don't fully understand how it all works, [2:43] but it's still it's still a thing. Um [2:45] so, for for those listening out there, [2:47] but all right. Well, great. So, you have [2:48] set the scene. to work with Java, some [2:51] Python and things like that. So for our [2:54] listeners who might not be familiar with [2:55] Sixfold, can you give us a sense of what [2:57] the company does and the kinds of [2:59] problems you're helping solve with as an [3:01] organization? [3:02] >> Yeah, we are an AI underwriter. So [3:06] basically when insurance underwriters [3:08] have to underwrite cases, they need to [3:10] do a lot of you know like basically data [3:12] points gathering. They look at the [3:14] documents that are sent to them by [3:16] brokers. Then they make a decision of [3:19] like what the risk of this insurance [3:21] this company or person is. We kind of [3:25] replace like a lot of the human process [3:27] in there. We basically streamline it [3:30] from you know you have a case like with [3:33] a name with some maybe some documents. [3:36] It could be from like a email. You can [3:38] forward us an email. Then we'll [3:39] basically just take it all the way to [3:42] the end where we give you okay this is [3:45] how much risk we think there is for this [3:47] company [3:48] >> with that workflow is that is like [3:50] you're ingesting a lot of data it sounds [3:52] like kind of a lot of complex things to [3:54] try to make sense of and I would imagine [3:57] with insurance I'm still trying to wrap [3:59] my head around a little bit around the [4:00] AI and how to describe this to to like [4:03] non I'm air quoting technical people in [4:06] the sense of like non-deterministic [4:08] output and like be able to like say [4:10] what's the risk level here? What sort of [4:13] technology and tooling have you been [4:14] able to use there to to make that kind [4:16] of happen there? And then and then maybe [4:18] tell us about how Rails then kind of [4:19] fits into that. [4:20] >> Yeah. So the main app right now is [4:23] actually on Rails. We're in this [4:25] transitional phase where we were using [4:26] the state machine which is actually a [4:29] gem on on Ruby and then we like are [4:32] moving to hatchet for basically like [4:35] workflow management and then on the AI [4:37] side for every kind of like evaluation [4:41] we use different models we do evaluation [4:44] with ground truths to like make sure [4:47] your questions are being answered [4:48] properly like this model is actually [4:50] good. We do tune our prompts with [4:53] evaluation to to like try to give the [4:56] most accurate result as possible. [4:58] >> Can you tell us a little bit about [4:59] what's Hatchet? [5:00] >> It's a um workflow orchestration tool. [5:05] >> And so is that basically a way to move [5:08] your state machine type logic into a [5:11] different platform? Is that [5:13] >> Yeah. Now, is that like some sort of [5:14] wizzywig pull map it all out that way or [5:17] is a lot of like code orchestr you like [5:20] write code to orchestrate that in their [5:22] tool, but then you're kind of letting [5:23] them manage that workflow. [5:25] >> Let me just give you some background on [5:26] how we did it. We have our code that [5:29] orchestrates all the workflows with the [5:31] state machine, the real like callback [5:34] hooks like on create and stuff like [5:36] that. Now we're moving to hatchet where [5:39] you just get to define you know this DAG [5:42] where step after step and then like when [5:46] does this trigger another workflow when [5:48] does this end what's the condition of a [5:51] certain next step runs and then hatchet [5:54] itself will be the engine that actually [5:56] orchestrates it [5:57] >> I see out of curiosity what does the [5:59] engineering organization look like today [6:01] like how roughly how large is the team [6:04] on the engineering side [6:05] >> uh I think we have about 20 to 30 people [6:09] on engineering right now. [6:11] >> I think you mentioned in our prior [6:13] conversation that there's a lot of [6:14] Python developers there I believe [6:16] >> and working on Python things and there's [6:18] some folks working on the you know [6:19] maintaining and working on the Rails app [6:20] and stuff. So how much crossover has [6:23] historically needed to kind of happen [6:25] between the teams? [6:26] >> Um quite a lot. We we actually work very [6:29] closely because like basically the Rails [6:32] app is like the web application the [6:34] worker for like orchestrating the jobs [6:37] for like database access where the [6:40] Python codebase is the micros service [6:42] basically for any like AI interaction. [6:44] So they're in charge of like making a [6:46] request to open AI to anthropic it will [6:49] then report back to the Rails app. What [6:53] do you think Rails has enabled your team [6:54] to do particularly well? [6:57] >> It's a very what's the opposite of [6:59] verbose I guess like [laughter] so it [7:02] it's really not verbose and like you can [7:04] write very little code which is really [7:07] nice like it's a double-sided sword. It [7:10] has its benefits which is like you know [7:12] yes we write a lot of a lot less of code [7:15] but then I think historically it had [7:18] also created some problems maybe [7:21] >> when you started working with Ruby on [7:22] Rails how people throw this word around [7:25] a little bit maybe a little overused but [7:27] did it feel like there was a little too [7:29] much magic at all or was that how would [7:31] you have described it at the time when [7:32] you say speak to the the verboseness [7:34] aspect of it or the lack of it? It was a [7:38] lot of magic. [laughter] I mean to me it [7:40] was a lot of magic because I came from [7:42] Java which is a very verbose language. [7:46] So for a little while it was I wouldn't [7:49] say difficult necessarily but it was a [7:51] lot of like oh what is happening? I have [7:53] to like keep jumping from this line to [7:56] that line. Jump all the way to to maybe [7:59] the library code to see oh okay so then [8:03] there is a helper because you know like [8:06] you don't actually need to define it but [8:08] it's just like under the hood it's [8:10] already defined and then you kind of see [8:12] that then you understand oh what where [8:14] this like specific method call is coming [8:16] from. [8:17] >> Yeah. What about for like your your [8:20] peers? What was that? Did did many of [8:23] them have a Ruby or Rails background [8:25] prior to [8:26] >> some I think they expect or they kind of [8:30] know a lot of the under the hood stuff. [8:32] Um but I also think especially for the [8:35] team that works mostly on Python when [8:37] they ever have to touch the real side of [8:40] code they like have a lot of troubles [8:43] because like a lot of it or at least [8:45] before like agentic coding like you know [8:47] like we can't avoid that now but like be [8:50] prior to it like they were definitely [8:52] having a lot of trouble understanding [8:54] what is going on because almost nothing [8:56] is like explicitly defined. What parts [8:59] of Ruby on Rails do you feel like have [9:01] been most beneficial or where did you [9:04] appreciate the most out of like relative [9:06] to other platforms and things you'd [9:07] worked with in the past? [9:09] >> I do like how dynamic it can be. As an [9:13] example, like one thing is like with [9:15] Ruby, you could literally dynamically [9:19] construct a class and then make a class [9:21] method call. [9:23] That is really really cool. and like a [9:27] lot of things like that and then like [9:28] you can just be having like oh do [9:31] compact and array and then it does [9:34] everything for you. The easiness of [9:36] using it is really great and then the [9:39] easiness to make things dynamic is also [9:42] really great you know like I think that [9:44] that again like I feel like it's a it [9:47] has two sides to it. I think it [9:48] benefited us a lot, but it also creates [9:51] some like confusions and like [9:53] frustration when people are not so [9:55] familiar with the codebase or with like [9:57] Ruby on Rails. [9:59] >> So maybe for folks that are kind of just [10:00] jumping in and out of it and aren't [10:02] working on a regular like they don't do [10:03] you feel like it's one of those things [10:04] where like where's this defined in the [10:06] codebase and and not being super obvious [10:08] or something? [10:09] >> Yeah, [10:09] >> I can appreciate that. So, what has [10:11] Rails allowed your team to accomplish [10:14] during this important phase of your [10:16] company's growth over the last few [10:17] years? [10:18] >> I think not having to write code like as [10:21] much code was very important. Being able [10:24] to make things more dynamic that [10:26] accelerates like an early just setup of [10:29] an initial app where we can sell where [10:32] we can like actually enable our [10:34] customers to do the AI underwriting. [10:36] That was very important because we had [10:38] very few people. It wasn't an a agentic [10:42] coding era of time. You actually had to [10:45] handw write code. Do not have to like [10:48] for example list out every line in your [10:51] database, every column in your database. [10:54] Don't have to list out every like [10:56] response to the front end. That was [10:59] actually really like acceleration. Now [11:02] has Rails been used for like a lot of [11:05] the customerf facing views and things [11:08] like that or is it kind of more of the [11:10] behind the scenes tool? [11:12] >> Yeah, so we use it for the rest API as [11:15] well. Uh we don't use it necessarily for [11:18] views. We do use React for front end but [11:21] the rest API responses all of that is [11:24] still handled by Rails. [11:26] >> What about from like a database [11:28] perspective? What have you been running [11:29] that in? And then is everything kind of [11:31] running historically through Rails or [11:32] does everything touch their own in [11:34] databases? [11:35] >> Yeah, the Rails app is also the entrance [11:38] to the database like the only entrance [11:40] to the database. Other microservices do [11:42] not actually touch the database. [11:44] >> Interesting. Can you give us a little [11:46] bit of background more on the the [11:48] migration to using something like [11:50] Hatchet, but like historically you [11:51] mentioned you're using like state [11:52] machines. So if you were going to be [11:55] migrating to like a new platform, could [11:56] you just could you tell us a little bit [11:58] about like some of the pain points that [11:59] you found by using and out of which gem [12:02] were you using? Was it just like access [12:04] state machine type of thing or [12:05] >> Yeah, it was yeah it was mostly just [12:07] state machine. I mean we were also using [12:10] active jobs. uh we were using factory [12:13] which is like a basically sidekick but [12:16] can be used in like other languages [12:19] because then we also started using [12:20] factory in Python for like the job [12:23] handling. [12:23] >> Interesting. What were some of the [12:25] limitations of like using the gem? Was [12:27] it just that it was too consolidated [12:28] within the Rails app or did you want to [12:30] like kind of leverage that elsewhere or [12:32] were you felt like maybe from an [12:34] architecture perspective was it were you [12:36] relying on feeding too many jobs to the [12:38] Rails app to then have it do something [12:40] when maybe the microservices could be [12:42] just dispatching their own jobs so to [12:45] speak? [12:46] >> Yeah, a couple things. One is like on [12:48] the factory side I I think under like [12:51] severe load we were having problem with [12:53] scaling. it gets backed up but like it's [12:56] hard to see or to like quickly resolve [13:00] it when it's backed up. So that's like [13:03] that's one problem that's more or less [13:05] on factory but then on the other hand we [13:08] built our inh home like orchestration [13:11] with the state machine and the callbacks [13:13] um for like you know after grade stuff [13:15] like that. So that becomes very brittle [13:18] as the workflow becomes more and more [13:21] complicated. It's very very scattered [13:24] how we like manage states. It's very [13:27] reliant on you know your database [13:30] transaction is actually you know [13:32] completed all of that is working well to [13:35] be able to like transition then that [13:37] transition itself then we have some [13:40] other side effects that comes with the [13:41] transition so that you know we can push [13:44] for next workflow to start but then that [13:47] also makes it very brittle because like [13:50] sometimes you create a lot of racing [13:51] issues and then to handle those you you [13:54] add more logic than making the logic [13:56] even more scattered. [13:58] >> Out of curiosity, off top of your head, [14:00] are you able to kind of like walk us [14:01] through like an example of some of that [14:04] at a high level of what the [14:05] orchestration might look like in for [14:07] like an actual use case just for our [14:09] listeners? Yeah, for example, we do like [14:13] web ingestion for like a company, which [14:15] means like we would go search for [14:18] information about the company online. [14:20] Then we like script those websites, [14:23] create those documents and then process [14:26] those like web content. But the problem [14:29] came as we realized like sometimes when [14:32] we were scraping the web it's very hard [14:33] to to say like this is actually the [14:35] correct website because there is [14:37] sometimes different names for the same [14:40] company or like same name company but [14:42] they're different actually not really [14:45] related. So we started doing this thing [14:48] which is like after we scrape the web we [14:51] ingest the documents then we're going to [14:53] check if like every one of them is [14:55] relevant. That means there is a fan out [14:58] then there is a fan backing while we [15:01] could also have documents for this [15:03] company like user uploaded. So these are [15:05] all running parallel. So now by the time [15:08] where you want to fan all of them back [15:11] in to say okay now we're done with this [15:14] like document ingestion or like web [15:16] ingestion we can like move on to the [15:18] analysis. That step became very brittle [15:22] like we see so much like racing with it [15:25] because they're all spread out they are [15:28] fanning out we are starting a job on the [15:31] micros service that's doing you know [15:33] like the the AI stuff to decide if it's [15:36] relevant then it fans back in it it's it [15:39] become a nightmare like [laughter] it's [15:41] just every at every possible place there [15:43] is a race condition [15:46] >> that's that sounds like a fun challenge [15:48] like to reach out for like a a platform [15:50] to maybe assist with that was part of [15:52] that. Do you feel like uh like just like [15:54] outside of like the capacity of what [15:56] your team could try to solve and keep [15:58] figuring out amongst yourself without [16:00] needing to bring in some more external [16:02] or additional help out for a while and [16:04] that sort of thing because I know [16:06] there's companies that are doing really [16:07] complex things within their apps from [16:09] time to time. But also, it's it's an [16:11] interesting thing to be like, maybe we [16:13] should reach out for a platform that [16:15] might be able to assist with that and [16:16] like let them figure out some of that [16:19] stuff. I would imagine it doesn't solve [16:20] all of the problems, right? But it maybe [16:22] addresses like enough of it that it [16:24] becomes like, oh, let's let's experiment [16:26] with that. Do you remember some of the [16:27] conversations or was there any like [16:29] hesitation like why would we pass this [16:31] off to someone else? we can figure this [16:32] out ourselves type of thing or or or [16:35] what does your team have like a culture [16:37] of like no we're gonna like use the best [16:39] tool for the job and we'll move off and [16:40] try something else if that's going to [16:42] solve the problem for us [16:43] >> um I think both are true for us one [16:46] being like I I think we do try to buy [16:49] over built if there is a better tool out [16:52] there we would try to use the better [16:54] tool on the other hand we did have the [16:57] whole conversation about maybe we just [16:59] rewrite this whole thing because you [17:01] know to some extent It was also because [17:03] as we add more complexity, we maybe [17:05] didn't think through the architecture [17:07] that well. So there was also the side [17:11] you know maybe we don't have to do this. [17:13] We just like rewrite like to be fair. [17:17] The rewrite of these workflows has been [17:19] on the table for a while even before we [17:22] even knew or like even before Hatchet [17:24] became like available at all. But at the [17:27] time it was also like okay there is [17:28] hatchet let's like maybe just try we did [17:32] have some other internal tooling that [17:34] started using hatchet as like a pilot to [17:36] see like how well they do with workflows [17:39] and then we started like moving smaller [17:41] things onto Hatchet to like solve some [17:44] of the problems and then it's proven to [17:46] be like possibly more efficient and [17:48] easier to maintain because you know they [17:50] give you this whole dashboard where you [17:52] can like cancel jobs like look at jobs [17:55] and Like you know that decision was made [17:57] on like both ends. [17:58] >> Nice. [18:00] This episode of on rails is brought to [18:02] you by application controller [music] [18:03] where all roads lead and no logic [18:05] escapes. Do you need to share a method [18:08] across all controllers? Add it here. [18:10] Want to run a call back before every [18:12] action? You know where to put it. Do you [18:14] need a place for O logic, flash helpers, [18:17] and a little panic code? It's waiting [18:19] for you. application controller is the [18:21] one file that's holding your app [18:23] together [18:24] >> and also maybe holding it a little [18:25] hostage. [music] [18:26] >> Side effects may include confusion, long [18:28] scroll sessions, and blaming something [18:29] in here for half of your bugs. [18:30] >> Application controller because if you [18:32] don't know where it goes, it probably [18:33] goes here. [18:36] When we were preparing for this [18:37] conversation, you mentioned uh I think [18:40] it was JSON API resources. Could you [18:42] tell us a little bit about that and like [18:44] what type of problems has that been [18:45] helping your team solve since it was [18:47] introduced? It's a gem that gives you [18:51] JSON API spec response for your like [18:55] REST endpoints. It hooks almost directly [18:58] to your database to your controller [19:01] layer. It's all the way. So you don't [19:03] need to write anything that's like like [19:05] a create endpoint. Our controllers are [19:07] like almost 100% empty. The files are [19:10] there and you still need to create those [19:12] files. But those magic [19:14] >> Yes, it's a lot of magic. [laughter] [19:18] It's great. It's great in the sense that [19:19] like whenever you have a new model, you [19:22] have a new entity or data model, you [19:25] barely need to write anything and it [19:28] already gives you back a very, you know, [19:30] like it's JSON API standard response. [19:33] >> Interesting. [19:33] >> You don't need to write serializer [19:35] either. So [19:36] >> is it everything on the like within the [19:39] active record and active model where [19:41] your business logic can kind of live and [19:43] kind of protect everything or is there [19:45] some some interesting layers where you [19:47] do have to is there like a DSL that it [19:49] has to say like only these things are [19:52] exposed so you can't like update certain [19:54] columns or attributes or something like [19:56] that. So we do use action controller. We [19:58] do use pundit for authentication and [20:01] some of the authorization. We do define [20:06] rows of like readrite and then we do [20:09] check for you know if they have the [20:11] readr access and then with the JSON API [20:15] resource you also get to define if you [20:18] want a create you want a read you know [20:23] like index list anything like that. Was [20:26] that introduced prior to you joining the [20:28] organization or was that [20:30] >> did you run into any interesting [20:32] challenges with that or has that in [20:33] general been actually been done its job [20:35] pretty well? [20:36] >> With the magic comes the rigidness of [20:41] it. It's like it's very rigid in like [20:44] what you can do with it like how the [20:46] errors can be handled. For example, if [20:49] you want specific errors when you're [20:52] doing, for example, you do a create and [20:55] then you do like model checks, right? [20:58] So, generally with JSON API resource, [21:01] you don't need to do anything like you [21:02] don't even write like a query. You don't [21:04] write the OM query either. You don't [21:07] write none of that. But you can still [21:09] define, you know, like my error classes [21:11] with active record, right? Like the [21:13] validation errors. But then with JSON [21:16] API resource, those errors are very hard [21:19] to then surface in the way you want it [21:22] because it has its own very defined [21:26] list and way of how it populates errors. [21:30] >> I see. [21:30] >> So then yeah, whenever you want [21:32] customization to it, it's a little bit [21:35] of a pain because you try to add to it. [21:37] On the other hand, the other thing is [21:39] this gem is now not maintained anymore. [21:43] So issues that comes with it just isn't [21:48] going to be solved. So like we kind of [21:50] have our own forked version of it where [21:52] we solve some issues. [21:55] >> What's your team's approach to picking [21:58] and and deciding when you'll bring in [22:00] like an external dependency? Do you feel [22:03] like your team's philosophy has changed [22:05] much in the last six months or so now [22:07] that presumably that LOM is maybe [22:10] agentic development is a little bit more [22:12] prevalent in in teams. Now [22:14] >> the introduction of hatchet is actually [22:16] kind of an example of how we deal with [22:18] like vendors like I think we generally [22:21] would like to find especially if like [22:24] for example hatchet is like a bigger [22:26] tool right like you got to pay but if [22:28] there is like open source gems that do [22:31] things for us we would definitely try to [22:34] you know look for tools that are already [22:35] out there like I guess the mentality is [22:37] that our value is not in creating those [22:40] like in like smaller tools but in creat [22:43] creating this like overall system for [22:45] insurance underwriting. [22:46] >> The reason I asked about that is I feel [22:48] like I've talked to some people where [22:50] they feel like their teams are trying to [22:51] are are revisiting that approach a [22:54] little bit and especially when it comes [22:55] to like bringing in a let's say a Ruby [22:57] gem or some open source library like and [22:59] you use that example of like JSON API [23:01] resources maybe not as actively [23:03] maintained and I just looked at their [23:05] web their GitHub it was updated three [23:07] months ago but it was to fix a Ruby 2.6 [23:10] logger compatibility thing. So [23:13] uh for and for Rails 6.0. So yeah uh [23:16] it's a little behind um all anyhow but [23:20] the I think the um but when it comes to [23:22] like with security in particular has [23:24] been a thing that like where where we're [23:27] pulling in external dependencies in [23:29] particular there's a lot of other uh [23:31] attack uh surface levels things that [23:33] everybody's being a little bit more [23:34] mindful like Ruby gems like they just [23:36] announced that new version of bundler [23:38] and ruby gems will have like a cool down [23:40] feature so that we're not automatically [23:42] upgrading things too quickly and [23:44] bringing in a potential security risk [23:45] because maybe an LLM just like insult [23:48] the gem or what have you. Hopefully [23:50] people are more thoughtful and catch [23:51] that stuff. But yeah, I was just curious [23:53] if that has been at all a discussion [23:55] there about how your team's thinking [23:56] about that even maybe beyond just your [23:59] Rails app. [24:00] >> Uh for sure the security issue has been [24:02] a problem. I mean not on Ruby or Rails [24:05] but like you know the npm attacks that [24:08] because we also you uh we use like [24:10] Typescript for for front end. So like [24:12] the npm attacks has definitely raised [24:14] our like awareness on like security [24:16] vulnerabilities. So we do have a lot of [24:19] like automatic tools that are monitoring [24:21] it, you know, new security [24:23] vulnerabilities and we put on a lot of [24:26] emphasis on like fixing them as soon as [24:28] possible. And then with like bigger [24:31] larger updates, especially for example, [24:33] Rails or like any Rails related like [24:36] active record got a major update or [24:39] anything like that, we actually don't do [24:41] the general dependabot like upgrade. We [24:43] like make sure like we do a proper [24:46] >> like upgrade or migration like having [24:48] someone to like even like in the age of [24:50] AI like we're like having someone to [24:52] like make sure that everything works. [24:54] >> Nice. Well, out of curiosity, what sort [24:56] of a testing tooling has your team [24:59] historically been using from the rail [25:00] side? [25:01] >> We use RSpec and then factory bot for [25:05] factory items and then we also use copy [25:08] bar for some end to end testing. [25:10] >> Are there any other interesting tooling? [25:12] How are you like APMs or observability [25:15] tools that your team tends to rely on? [25:17] >> Yeah, we use data dog for like basically [25:20] all monitoring. [25:21] >> Has your team been able to like leverage [25:23] some of like the LOM tools to with data [25:27] dog at this point? And out of curiosity, [25:29] are you using much LOM tooling [25:31] day-to-day in your coding activities? Uh [25:34] yeah so much we have basically every MCP [25:37] we are migrating to like well at this [25:40] point I think the migration to fully [25:42] agentic coding is done but uh at one [25:45] point we were like just like really [25:47] being really aware of this like change [25:50] and like really trying to bring everyone [25:52] in the organization up to speed with [25:53] agentic coding. [25:55] >> What were some of the challenges that [25:57] you found through that process [26:00] regardless of the Rails part of it? I'm [26:02] just kind of curious about that. [26:04] >> Yeah, sometimes it doesn't do what you [26:07] ask it to do, or rather you have to ask [26:09] it in a very specific way for it to do [26:12] what you need it to do. And um I do see [26:15] it also sometimes being overly defensive [26:18] when it's coding. [26:19] >> Tell me more like what do you mean by [26:20] that? Um sometimes there are things that [26:25] it's like oh we need to prevent for like [26:29] this string being not not empty or like [26:34] not having any weird characters but we [26:37] know if that string is fed from an [26:39] upstream of code [26:41] >> it's never going to happen. [26:44] But then claw would be like sometimes [26:46] overdefensive and just try to be like, [26:49] hey, writing like, you know, just 20 [26:52] extra lines just to be like, oh, let's [26:54] just make sure that like we catch every [26:56] possibility of like this the format of [26:58] this string. [27:00] >> I can appreciate that. And you when you [27:02] mentioned you're like fully agentic at [27:04] this point, what does that mean for you [27:07] right now at sixfold? [27:10] Um we almost well not almost we pretty [27:13] much we don't write code anymore. We [27:17] don't write code anymore basically [27:19] >> like like in the 80 90% plus perspective [27:22] of most of the code or even more so is [27:25] almost entirely right like are you [27:26] orchestrating a bunch of agents right [27:28] now at this point in parallel or is it [27:30] is that all happening kind of in your [27:31] local environment? I know by the time [27:33] this we record this might change by the [27:35] time it gets published but just kind of [27:37] getting a sense of where your folks are [27:38] at at at the moment. [27:39] >> Yeah, we are fully agentic or like we [27:42] are basically not writing any code. Um I [27:46] think we're at a point where some people [27:48] are still reading the code but some [27:51] people like are fully on the point where [27:54] like if it works then it's good. I don't [27:57] care if it I don't care like to read the [28:00] code anymore. I I guess like this is [28:02] where the org is at, but like in general [28:04] like nobody's writing code anymore. We [28:06] don't review other people's PRs mostly. [28:09] We do have agents also review the PRs. [28:12] So a lot of the work is actually putting [28:14] in the harnesses for the agents instead [28:17] of like just writing code. [28:19] >> Can you tell us a little bit about like [28:21] what's maybe what some of the what those [28:22] harnesses look like? [28:24] >> Yeah, so one we have code rabbit that [28:27] like is reviewing every PR. We do tune [28:30] code rabbit. We do give a guidelines of [28:32] like you know what you need to be aware [28:35] of and then we do like ask everyone to [28:38] like basically have documentation have [28:41] like a agent.md file it which is for you [28:44] know these are the pitfalls I've had [28:47] this is how this is designed this is the [28:49] general architecture this is a pattern [28:51] so that then when an agent is coming to [28:54] like make more addition to a feature it [28:57] has all these guidelines that it that it [28:59] needs to model. [29:01] >> How has that been for you as a software [29:04] developer to are are you embracing it? [29:07] You're excited, optimistic about this or [29:08] has it been kind of an [29:10] >> I think it's it's an exciting time. [29:13] [laughter] [29:14] >> It's like I I to me it's like oh wow [29:16] what a time to be alive. Um, and what a [29:19] time to be to be working in this [29:21] industry actually cuz I I think it's it [29:25] completely changed what it means to be a [29:28] software developer. In the past like [29:31] writing code is like you know some [29:33] people might say it's like an elitist [29:35] thing almost [29:36] >> um but nowadays code is cheap. [29:39] >> I don't think the value of a software [29:42] developer is gone. I actually think it's [29:45] more like know the value of you is not [29:48] about writing code. It's about thinking [29:50] through a solution to a problem. It's [29:53] about how you architect something so [29:54] that it is it is you know like it is not [29:58] brittle that that it is robust and you [30:00] are solving a problem. You know how to [30:02] scale it. I think that's really where [30:05] now the value of a software engineer is [30:07] at. It's not so much so about code like [30:10] I I to be honest I myself never thought [30:13] it was about writing code because like [30:15] syntax is always online documentation [30:18] was always online code is like you know [30:20] like your library is always there but I [30:23] think now it's very explicit in that way [30:25] such that you know like code is cheap [30:27] it's easy to write [30:29] >> do you think that AI assisted [30:31] development changes how [30:34] teams might be thinking about those [30:35] Rails abstractions do you feel like [30:37] you've noticed much with how it's m like [30:39] the using AI to help maintain the Rails [30:42] app in particular has has been a a [30:44] positive thing you've seen and like oh [30:46] it seems to actually work really well [30:47] with Rails or what's that what's your [30:48] experience like [30:49] >> yeah I think it's especially useful for [30:51] Rails because of the magic you know in [30:54] the past you have to like go through the [30:56] code you have to go through like very [30:58] deep in the line because like you know [31:00] other robust language you see it like [31:02] you see it you do a global search you [31:03] see that method you see the definition [31:05] but like with rails [31:07] A lot of times you don't see it. [31:09] Sometimes it's maybe like constructed. [31:11] Sometimes it may be just like a real, [31:13] you know, like helper method that it's [31:15] not obvious. But with claw code that's [31:18] not a problem anymore, you know, like it [31:20] knows like you ask it, it can actually [31:22] trace for you. [31:24] >> That's interesting. You mentioned [31:25] earlier that there's people that kind of [31:27] jump in and out of Rails with other tool [31:30] sets that you're working with there that [31:31] agentic development has been able to [31:32] make that a little bit easier for them [31:34] because they didn't understand the [31:36] magic. [31:37] >> For sure. For sure. Like they can [31:39] actually do the real stuff. Like in the [31:41] past it was more like okay we have like [31:43] you know like we kind of have this ask [31:45] of like how we want the rail site to be [31:48] and then they kind of hand it off to [31:50] like a rails you know like the engineers [31:52] that are working on the rail site. But [31:53] now they can actually take it to done. [31:56] >> Out of curiosity with with the Rails [31:58] app, has your team built any interesting [32:00] tooling to help historically with like [32:03] onboarding or developer experience like [32:06] any interesting little tools that are [32:08] very very specific to like ramping up [32:10] things. So you can have like a let's say [32:12] for example like do you have a pretty [32:14] healthy seed system or how does your [32:16] team debug production level type things [32:19] in a local environment with the rails [32:22] application part of your platform or is [32:24] that a kind of a challenging thing for [32:25] you? I think we did have some tooling [32:29] for just getting things up like we did [32:32] have you know clear instructions and [32:35] like commands that you just run to start [32:38] your dev environment and then we also [32:41] did do a lot of like CLI commands to be [32:46] like okay you know you're it's [32:48] production like there are some specific [32:50] things you need to do you can just like [32:53] run this you don't need to necessarily [32:55] go into the Rails console So and then we [32:57] have sometimes have like customer [33:00] support requests where they need a [33:02] specific type of export or something [33:05] like that. So like we had a lot of like [33:06] CLI commands that were built for this [33:08] reason. Now we kind of like also have a [33:12] set of like clawed skills to be like you [33:15] know especially for setting up dev like [33:18] there is a whole claw set of things that [33:21] you can just run and then it'll take [33:23] care of like setting up dev environment [33:25] for you. [33:26] That's cool. Is your your Rails codebase [33:29] have a bunch of like uh dead pieces of [33:33] like like rake tasks or you mentioned [33:35] like you're doing customer support and [33:36] needing to generate some weird reports [33:38] like how is your team typically manage [33:40] that type of work historically and do [33:42] you feel like that's a little bit [33:42] different now in your codebase? [33:45] >> So in the very beginning we use Thor for [33:48] writing CI commands. So like most of the [33:51] time we would like you know if a [33:53] customer request keeps coming we just we [33:55] then write a CLI command you just type [33:57] in what you need then it you know like [33:59] it does what you need to do for it. Then [34:01] we actually started this internal tool [34:04] completely for handling like the mass [34:07] majority of like the customer requests [34:10] which is just completely like its own [34:12] it's like hosted um and it does a lot of [34:15] like explores like the playground for [34:18] the the customer support. [34:20] >> Interesting. [34:21] >> Yeah. [34:21] >> And then is that just connecting back [34:24] straight to the same database then kind [34:25] of bypassing rails or does it go go [34:27] through Rails at all? [34:28] >> It actually just goes through Rails. [34:30] Okay, that's cool. Yeah, I was talking [34:32] to a couple people recently and they [34:34] were talking about how they've been been [34:36] building I don't know if your team your [34:38] organization done that but they've been [34:39] building tools for like cloud skills [34:41] that allow their customer support people [34:43] to actually run cloud skills against the [34:46] Rails like Rails console and they have [34:48] like a protected version of the Rails [34:49] console other things like that so they [34:51] can kind of like let it be a more [34:53] natural language and stuff like that. [34:54] Seems like there's a lot of teams trying [34:55] to figure out how to like take advantage [34:57] of some stuff. And I think Rails [34:58] recently came out with a was it Rails [35:01] query or something? Forget the name of [35:02] it, but it basically uh kind of a little [35:06] bit of a wrapper for like the uh Rails [35:08] Runner things. So you can pass it'll [35:10] generate queries for you or something [35:12] like that. Like that's kind of kind of [35:13] neat and interesting. Right now I'm [35:15] curious when it comes to like AI and the [35:18] state of things at Sixfold. Uh I'm when [35:20] it comes to you know the Rails app and [35:23] you you talked you shared a little bit [35:24] about the orchestration and the the uh [35:26] underwriting process out of curiosity is [35:29] is there anything about the data that uh [35:32] sixfold manages that's is there any [35:34] interesting challenges you've had to do [35:36] with like keeping segmenting data or do [35:38] you have to do like a multi-tenant type [35:39] thing or is like is the data that's [35:41] being shared into the system very very [35:43] important that you need to [35:44] compartmentalize it from different [35:46] customers and different clients and that [35:48] beyond just being like here's a row in a [35:50] database with a column or tell us a [35:51] little bit more about that. [35:53] >> Yeah, we are multi-tenant because we [35:55] deal with insurance. We have to be HIPPA [35:58] compliant. We have to be sock 2 [36:00] compliant. Our clients sometimes if [36:02] they're in a specific region they [36:03] actually have compliance where like you [36:06] your data can only be retained in that [36:09] specific region. M [36:11] >> um so with all of that it just made more [36:14] sense and also we do have a pretty big [36:17] load in terms of like data processing. [36:19] So with all of that I think like the [36:21] multi-tenant just makes more sense [36:24] >> is it the infrastructure then [36:26] distributed across several regions then [36:28] at that point then [36:29] >> to make that make that work [36:31] >> that's interesting. Okay like how does [36:33] that then work with like uh something [36:34] like hatchet is that then also needing [36:37] to be isolated in different regions as [36:38] well? Yeah, we have specific like [36:41] basically for every like for example EU [36:44] like for every region like that where it [36:47] needs to be there is like one for every [36:50] like one of those hosted things [36:52] >> and is the the app basically then deploy [36:55] it out to each of those regions in [36:56] parallel and is it the same deploy or do [36:58] you do you phase deployments out to [37:01] different regions at different points [37:02] and it's kind of like a one by one or [37:04] you can be selective about that [37:06] approach. We could be selective, but [37:08] like in general, we just like deploy all [37:11] at once. Yeah. [37:12] >> Okay. Do you lean on a lot of like [37:14] feature flags or anything? [37:17] >> Yeah, that's that's actually the other [37:19] thing is like a lot of like our [37:20] different clients actually want to use [37:22] different features. Um, so that also [37:25] plays into the multi-tenency is like, [37:28] you know, we get to turn only certain [37:30] things on for specific client. [37:33] >> And what sort of tooling do you use for [37:35] that? We started with post hawk. [37:40] >> Yeah. [37:41] >> But then we actually started doing our [37:43] own internal. We're just like envir and [37:46] then we have this internal tool for [37:48] flipping them. [37:50] >> Have you found like a a good pattern for [37:52] how to do that? Is that across your [37:53] different stacks as well? So it's not [37:56] just like something in your Rails app, [37:57] but if you're using environmental [37:59] variables and flipping switches [38:00] somewhere, like what does that kind of [38:03] what's the architecture of that kind of [38:05] look like a little bit if you if you [38:06] don't mind sharing? [38:07] >> Generally, most of the features are [38:09] maintained on the rail side. If it's a [38:11] feature, you know, that we want to turn [38:12] on for some some clients but not the [38:14] others. for example, maybe like a prompt [38:17] or like some specific things are on the [38:20] AI side, then it would be an envir on [38:24] that micro service. Uh we do have an [38:27] internal platform that handles it. No, [38:30] maybe a set of customers have a specific [38:33] feature. So, we also have that to be [38:36] able to apply to just a set. Do you use [38:38] any of any any of that tooling to like [38:40] roll out new feature ideas to a subset [38:42] of clients before you roll them out to [38:44] everybody or to test things out before [38:46] you fully roll it out to everybody? [38:48] >> Yeah, we don't do the, you know, like [38:50] 50% of customers on a place. We don't [38:53] really do that. We generally roll it out [38:56] 100% to a group of like maybe alpha or [38:59] beta customers or we roll it out into [39:02] like the UAT and sandbox environments [39:04] first and then we would roll out to like [39:06] the actual production. [39:07] >> You mentioned like things like Cappy [39:08] Bar, but does your team have like a like [39:10] QA type team there that is always some [39:14] humans doing that or is that somewhat [39:15] automated at this point? [39:17] >> It's somewhat automated. We do have [39:20] frontend testing as well. Um and then [39:23] but then the full really like you know [39:26] QA testing used to be completely human. [39:30] We went through a couple iterations. I [39:33] guess we had like a dedicated QA person. [39:36] We're now using QA Wolf um to do the QA [39:40] for us. [39:41] >> Nice. So Joanna, I want to ask you [39:43] something. So let's let's pretend we're [39:44] do a little exercise here. Let's pretend [39:46] it's your last day at Sixfold and [39:48] someone else is going to come join the [39:50] organization and you had five minutes [39:53] with them and you needed to quickly say, [39:55] "Hey, this is a typical Rails app." [39:57] Well, you mentioned you touched on the [39:59] JSON API resources. You don't need to go [40:01] too deep into that, but let's say here's [40:04] two or three things you should probably [40:05] know about this Ruby on Rails [40:07] application that might be a little [40:08] atypical from other Rails apps. What [40:11] would you tell them as you walk out the [40:13] door [40:14] >> and say, "Good luck. [40:15] >> [laughter] [40:18] >> We have a lot of race conditions in the [40:21] current state of workflows. Things are [40:24] moving to a better state is our goal. Be [40:27] on the watch out. [40:30] And then most things are documented [40:34] somewhere in notion. If you have your cl [40:40] um will be good. I think that's it. [40:43] >> Okay. All right. That's that's fair. All [40:45] right, Joanna, is there a technical [40:49] programmingesque type book that you've [40:51] historically found yourself recommending [40:53] to your your colleagues? [40:55] >> Yeah, I historically I would usually [40:58] recommend clan code and clan [40:59] architecture by Uncle Uncle Bob. [41:02] >> Okay. [41:03] >> But I I don't know if it's still [41:05] relevant as of today with the agentic [41:08] coding, you know, like maybe messy code [41:11] is okay as long as it works. Do you [41:13] think there's still some value in trying [41:15] to corral the agents into [41:20] It's interesting because it is so much [41:22] about I think it's an interesting thing [41:24] about like when are we opening up and [41:26] having to look under the hood and do [41:27] these things ourselves versus relying on [41:29] the agents to sort these things out but [41:31] also needing to know how to corral them [41:33] in cohesive ways so that we can keep [41:35] things maintainable. Right? So otherwise [41:37] we'll be [41:37] >> potentially burning a lot more tokens in [41:39] the future to do things in the same way [41:41] we'd burn human hours on being like this [41:45] is a mess. This is taking too long to do [41:46] this and we're still paying for this [41:48] stuff supposedly. And we're also living [41:50] in this uh slightly subsidized token era [41:53] of development. So see what that looks [41:56] like. All right. Um but great [41:59] recommendations. I'll include links to [42:00] both of those in the the show notes. uh [42:02] join it. And with that, I want to thank [42:05] you so much for stopping by to talk shop [42:07] with us on Rails. [42:09] >> Thank you for having me. [42:10] >> Yeah, it's been a delight talking with [42:11] you as well. Thanks so much. [42:16] That's it for this episode of On Rails. [42:18] This podcast is produced by the Rails [42:20] Foundation with support from its core [42:22] and contributing members. If you enjoyed [42:24] the ride, leave a quick review on Apple [42:25] Podcast, Spotify, or YouTube. It helps [42:28] more folks find the show. Again, I'm [42:30] Robbie Russell. Thanks for writing [42:32] along. See you next time.