Why AI Projects Break After Launch and How to Address the 'Day 2' Problem

Why AI Projects Break After Launch and How to Address the 'Day 2' Problem While modern AI tools enable non-technical users to quickly build functional automations, projects frequently fail post-launch due to classic software engineering challenges known as 'Day 2' problems. Overcoming these maintenance, scalability, and observability issues requires addressing structural requirements during initial planning. A 'Day 2' problem refers to long-term challenges in growing and maintaining software after planning (Day 0) and initial building (Day 1) are complete. Common post-launch pitfalls include lack of observability, absent version control, missing documentation, shared-access lockouts, hardwired configurations, and silent failures. Large language models build only what they are asked to create and do not automatically incorporate architectural best practices unless prompted. Project creators can mitigate post-launch failures by asking architectural questions on Day 0 or prompting AI to interview them regarding maintainability, security, and scalability. The Post-Launch Reality of AI Building Shipping an initial AI tool or automation often leads to unexpected breakdowns as the system expands. Non-technical creators frequently lack visibility into underlying infrastructure, leaving them unable to diagnose why an automation failed. In software engineering, issues occurring after initial launch belong to the 'Day 2' phase, which encompasses ongoing operations, maintenance, and growth rather than preliminary development. Failure Modes in Unprepared Systems A hypothetical scenario involving a non-engineer named Dave illustrates standard Day 2 pitfalls in AI automations. After building an invoice-processing workflow, Dave encounters incorrect outputs with zero logging, overwrites his only working version due to a lack of version control, and causes workflow lockouts because credentials and documentation are not shared. Later, expanding the workflow to a new office requires a total rebuild due to hardwired settings, while unparsed vendor invoices fail silently without generating alerts. Applying Engineering Principles on Day 0 Software development encompasses dozens of system quality attributes—often dubbed 'the ilities'—such as maintainability, portability, and scalability. Addressing these demands requires clear ownership, following principles like Amazon's 'you build it, you run it.' Because LLMs are optimized solely to deliver requested outputs, they omit infrastructure requirements unless instructed. Creators can steer models by asking them to interview the user about software engineering best practices before launch.

构建AI项目面临的“第二天问题”及应对建议 许多非技术背景的构建者在使用AI工具快速搭建项目后，往往会在后续维护和扩展中遭遇系统崩溃。这种在规划（Day 0）和构建（Day 1）之后出现的长期运维与扩展挑战，在软件工程中被称为“第二天问题”（Day 2 Problem）。 “第二天问题”指产品在初始构建上线后，在维护、扩展和故障排查等长期运行中暴露的问题。 非技术人员借助AI自动化常因缺乏可观测性、版本控制、文档和可扩展性设计而陷入困境。 LLM仅按输入指令生成基础结果，默认不会自动补全日志记录、权限管理等工程最佳实践。 防范此类问题的关键是在规划阶段（Day 0）确立所有权并主动提示AI按照软件工程规范进行审查。 案例分析：忽视运维设计带来的连锁故障 财务人员Dave利用AI构建了发票自动解析和上传工具，但上线后迅速遇到一系列挑战：由于缺乏分步日志记录，无法核查错误解析原因；因缺少版本控制，修改代码失败后无法回退；系统未留下文档且绑定个人凭证，导致同事无法协助处理故障；系统硬编码限制了扩展至其他分部的能力；此外，格式无法解析时程序静默跳过发票，未产生任何报错或标记。 工程经验与软件质量考量 软件工程师长期通过维护性、可移植性与可扩展性等系统质量属性（ilities）应对此类挑战。虽然低门槛项目不需要过度设计，但高可靠性系统需要明确所有权，践行“谁构建、谁运行”的原则，并在必要时引入专业工程师介入。 在规划阶段引导AI规避潜在隐患 大语言模型主要根据直接指令交付结果，无法预知用户未提及的隐式需求。构建者需在Day 0阶段主动引导AI，例如要求模型围绕维护性、安全性、可观测性与可扩展性对自己进行提问访谈，从而在上线前排查并补充必要的工程保障架构。

Despite the pure bliss of shipping something you’ve built with AI, if you’ve actually launched a few things on your own, you probably know the feeling in the pit in your stomach when something breaks. Or just the looming sense of dread when something you’ve created starts to expand and you’re not really sure how to handle it. *Yikes*. If you come from a non-technical background, this can be fatal for your momentum. There’s so much possibility for what LLMs can help us create, but most AI tools are not designed to teach you how the underlying infrastructure works. So when things fall apart, it’s not always clear what happened or why. Just to be clear: You don’t need to feel bad about this! Jumping in to start building is often the best way to learn. You’re probably trying to solve problems that you didn’t previously know existed, and you likely don’t even have a name for what you’re trying to do. That’s where I’d like to help. I’m currently a marketer, but I came up through the world of software engineering, getting my degree in CS and working as a dev for years before making the switch. So I have a unique perspective on where this stuff is going wrong. And this particular challenge - where you build something, but it breaks down the road - is a perfect example. Because this isn’t a new problem for software builders. In fact, it’s a classic. We even have a name for it: This, my friends, is ***a Day 2 Problem***. What’s that, exactly? **Day 2 Problems are the long-term issues and challenges that show up when growing and maintaining a product, after the planning (Day 0) and initial building work (Day 1) are done.** If you’re having trouble imagining what I mean, here’s an example: Meet Dave. He's on the finance team at a 100+ person company and has a can-do attitude. Everyone loves Dave. He’s not an engineer, but he’s always trying out new software and he’s already showing other people on the team how to build things with AI. This week, things went wrong. **Very** wrong. Dave’s team is drowning in invoices. Vendors send in PDFs by email, but then someone has to manually type the information into the system by hand. It takes forever, so everyone is several weeks behind. So after thinking about it over the weekend, Dave hits the ground running on Monday morning. With the help of an AI tool, he’s built an automation: Invoice comes in, AI scans the PDF, key information is extracted and uploaded. It even flags if something looks off. Hurray, Dave! Everyone is so impressed and his boss tells him, “This is the kind of thing we can use to help you land that promotion we’ve been trying to get for you.” Dave is practically dancing in his car on the commute home. First thing in the morning, the accounts payable team pings Dave: 3 invoices posted with the wrong amounts. And not by a small amount. Dave scrambles to figure out what happened: Did the AI misread the PDFs? Did it not properly upload it to the system? He opens his automation and… there’s no way to know. There’s no record of what the AI did for each step - it just says “Automation complete.” So Dave starts poking around. He tweaks a few things and finds the issue. Fortunately, it’s minor, so he fixes it and updates the automation. Once it’s live, he does a test run. … and it doesn’t work. Uh oh. This is worse. Time for Control-Alt-Delete! He goes back to the system to try to go back to the previous version, but that’s not a thing. He edited the only version and there’s nothing else there. Dave spends the next few hours rebuilding his original version, and then adding in the new fix. Then the rest of the afternoon triple-checking to make sure it all works before updating right before going home. Hopefully everything is fixed now? Dave is stuck in back-to-back meetings all afternoon. At 2pm, the automation program requires an update. It just needs to be reset, the error message says. His teammate Marco offers to jump in. But… where? Dave is the one who always does it. How do you even get to the editor? There’s no documentation. And when Marco finally finds someone who knows, the login credentials are tied to Dave’s email and there’s a password that is not Password123. Marco texts Dave. Dave, who is currently pretending to pay attention in a quarterly planning meeting, tries to text back instructions under the table. It doesn’t go well. The invoices sit until Dave is free at 4:30. At least the problem is resolved by 4:31! Dave’s boss still believes! She sees the vision and knows that it’s already saving their team significant time. So she asks the natural next question: “Can we roll this out to the Austin office, too?” Dave pulls up the automation and his stomach sinks. Everything is hardwired: one email inbox, one set of vendor formats, one connection to one accounting system. To make it work for Austin, he’ll have to do more than just tweak some settings. He’ll pretty much have to rebuild it from scratch. Again. He tells his boss, “Sure, no problem!” And cancels his after-work plans to get it done. Smooth sailing! After all the bumps, everything seems to be running correctly. Until 3pm, when a vendor calls: Their invoice hadn’t been paid this week. Dave checks: the automation never processed them. It couldn't parse the format, so it just… skipped them. No error. No flag. Nothing. Even worse: That’s not the only one. Dave starts digging and finds 11 more invoices from the week that were silently skipped the same way. Every day this week, while he was troubleshooting the problems he could see, there were more problems that he couldn't. His boss stops by. Dave does not feel like dancing on the drive home. Now, most “Day 2” problems don’t literally happen on the second day. But the experience of things breaking in unexpected ways, or requiring an update after the initial launch is unavoidable. Here’s the bad news: There are way more things that could have gone wrong. What about security? Or the AI model getting updated and suddenly giving very different outputs? The longer the system runs, the more vulnerable it is to falling apart. But fortunately, I also have very good news for you: This is not a new problem! Dealing with Day 2 issues is something software engineers have been dealing with for decades. You can learn from their experience and save yourself a lot of heartache. So how do you do that? My #1 piece of advice for all my friends, coworkers, and internet strangers is this: You solve Day 2 problems by asking yourself the right questions on Day 0. The real underlying issue with Day 2 problems is lack of awareness. Most of Dave’s problems weren’t even that hard to fix. He just didn’t realize they even could be problems until after they happened and he was scrambling to fix them. Like I said at the top: I don’t blame Dave, and I wouldn’t blame you, either. You don’t know what you don’t know. Dave and his team actually got one thing right: They knew who owned it. Before everything else, make sure that’s clear. Back in 2006, Amazon's Werner Vogels famously introduced this principle:  *"You build it, you run it."*  If you’re *not* the right person to actually maintain and sustain something you’re building, you need to be very clear about who will? So whether it’s you, or you plus some other person who will be owning in the future, ask yourself these questions, before you have a week like Dave’s: Dave got unlucky, but he also got lucky. Even in his disaster of a week, more could have gone wrong. Here are a few more questions you should consider: As you’d imagine, experienced software engineers have a much more comprehensive set of areas to review. One popular way to describe them is ***the “ilities,”*** since so many of these topics end with that suffix: maintainability, portability, scalability, etc. As I was learning all of this stuff in school, we were taught that [there are 61 of them](https://en.wikipedia.org/wiki/List_of_system_quality_attributes)! Knowing how deep to go really depends on what you’re trying to build: If you were building a, well, building, what would this be: a shed, a 2-story house, or a 20-story building? If you’re building a quick, simple app, you don’t need to be so exhaustive. But if you’re trying to build something that needs to run reliably and deal with complex needs, prepare accordingly. Maybe you need to bring in a professional? Just like some people build their own outdoor sheds, but not their own nuclear reactors or industrial bridges, there are times when bringing in an engineer is the right move. You may be asking yourself: Wait, why doesn’t AI solve this, already? Given that it’s been trained on massive sets of software engineering data, why doesn’t it answer (or even ask) these questions, out of the box? The problem is that LLMs are optimized to give you the output you’re looking for, and if you don’t know what you want, then it doesn’t either! So, when you say “Build me an app that does X,” it won’t assume that what you *actually* mean is “Build me an app that does X, with the ability to provide third-party access, and the ability to expand to adjacent use-cases, and make sure we know when it fails or does something wrong.” You just need a bit of nudging. If you ask, “How should I handle version control?” in my experience, it will often respond with something like, “Good point! We should actually add that, like this.” But you have to guide it in that direction. And you have to know what you actually want! Not sure where to start? Just use a prompt like this to give the AI the correct context and get it to ask you what you really need: *“I'm not a professional software developer. I want to make sure this project follows best practices for software engineering: things like maintainability, security, observability, and scalability. Interview me about my project and help me identify what I should be thinking about before I launch.”*  So Dave learned his lesson. Building software is not something you do one time and then forget about it. Once the thing is live, there’s just as much work keeping it going. It turns out, Day 2 is the longest day! But that’s not a bad thing: It’s actually kind of a mark of honor. Products only make it to Day 2 if people enjoy using them or find enough value to keep coming back. So if you’re going to start building, give your product a chance at a long and happy life. **Prepare for Day 2 by asking the right questions on Day 0!**