Hey, it's Danny.
A few weeks back my wife looked up from her laptop and said, "My Claude is dumb today." She runs the books for one of our businesses, and some days her automation just misbehaved. She'd fix it, the fix would seem to work, and then the next morning it would do the dumb thing again.
Spoiler: Claude was not dumb. The way the whole thing was built had three quiet problems, and not one of them throws an error. I want to walk you through it, because the woman who built this is the most competent person in the story, and what she pulled off is the entire reason Claude Actually exists.
First, the stakes, so you know this isn't a toy.
Around 6 million dollars a year in vendor bills run through this automation. Real invoices, real money, dozens of different vendors. Here's the flow: the automation reads her email, pulls out the vendor invoices, reads each one (every vendor names files and lays out invoices differently, and it handles all of them), and posts them as bills in our CRM. The CRM pushes those to QuickBooks. Then my wife audits and reconciles every single bill by hand. No AI agent ever touches QuickBooks. The books are human verified, full stop. The AI does the boring fetch-and-sort. The human does the judgment.
Now here's the part I need you to sit with.
My wife was scared of APIs. The technical side intimidated her. So instead of writing code against an API, she built the whole thing using Claude computer use on a remote desktop. She had the AI operate the software with a mouse and keyboard, clicking through our CRM the exact way a person would. Think about that. A non-developer who was scared of the technical part automated millions of dollars of real work by teaching an AI to use the computer for her. That is the entire Claude Actually thesis in one sentence. You do not have to be a coder to build something real.
And what she built is real software. Custom Python scripts. Actual if-this-then-that logic. An email-triage step that scans her inbox and pulls the vendor invoices out of the noise. A pile of built-in knowledge about how all these different vendors format things. You could rip the AI out of the middle and it would still stand up as software. This was not a clever prompt. This was a build.
So why did it keep going "dumb"? Three reasons, and they're sneaky.
One: context contamination, but in the fixing, not the running. The production side was done right. Every run spins up a fresh agent on a clean slate, and she knew to do that. The trap was the repair work. When something broke, she'd fix it inside one long, continuous chat, figuring a single agent that understood the whole goal would make smarter fixes. Reasonable instinct. But that long thread still held every earlier draft of the skill, so her good fixes kept getting muddied by old, dead versions sitting further up the conversation. Lesson: even your fixing thread is memory. The installed skill is the one true copy. Fix against a clean slate, not a long thread.
Two, and this was the big one: two sources of truth. The rules ended up living in two places at once, a scheduled-task prompt AND the skill. Over months, those two drifted apart. So she'd fix a rule in the skill and the fix wouldn't take, because that same rule still lived, in an older form, in the scheduled prompt she wasn't even editing. Two rulebooks, one fresh and one stale, fighting each other on every single run. Fix: every rule lives in exactly one place. The trigger just says "run the skill," with no logic of its own.
Three: snapshot sprawl. Every time she made a "new version," it saved another downloadable copy. v1 through v7, sitting in a folder. None of them actually run (there's one installed skill), but a folder full of look-alikes makes you certain the duplicates are the bug, when they're just a red herring eating your afternoon. Fix: keep one copy and write over it.
Now the war story, because it grows straight out of how she built this.
Since the whole thing ran on computer use, it was literally clicking through the CRM in a browser. Clicking through software is twitchy even when an AI is the one doing the clicking. Misclicks, stale pages, laggy grids. That flakiness reads as "dumb" too, but it's really just a robot fighting a website. The sturdier fix is the API, the exact thing that scared her in the first place. And here's the payoff: this CRM's API turned out to be a genuine powerhouse, a serious data platform with great endpoints. Facing that fear was worth it.
When we went to wire it up, the published docs were stale and contradicted each other. So we wrote a probe, a tiny agent that hits the live endpoints one at a time and writes down what actually works. It settled the argument fast. The catch: it ran against the live system, not a sandbox, and it created a one-cent receipt that the API cannot delete. It is in there. Forever. Lesson: don't assume "it's probably a sandbox." Test live on purpose, with your eyes open, and know that some buttons are one-way doors.
Here's the meta-lesson I keep coming back to. "Claude is dumb today" is a clue, not a verdict. Nine times out of ten it's pointing at how the thing was built (the fixing thread, the competing rulebooks, the snapshots), not at the model.
So here's the checklist. Steal all of it:
Doing your building or fixing in one endless chat? Start fresh. The installed skill is your source of truth, not the thread history.
More than one thing describing the same behavior (a prompt AND a skill)? One source of truth per rule, and let the trigger just say "run the skill."
Saving new versioned files instead of writing over one copy? Stop. Keep one.
Agent wrestling a website when there's an API? Use the API, and test it live. Don't trust the docs.
Asking it to make a judgment call it shouldn't make alone? Have it flag for a human. (Like our books: the AI never touches QuickBooks, my wife reconciles every bill.)
That's it. A woman who was scared of the technical side built software that runs millions of dollars of work, by teaching an AI to use a computer the way she would. If she can do that, you can build the thing you've been putting off.
This is exactly the kind of thing we figure out together, every day.
Now I want yours. Hit reply and tell me about the time your AI "got dumb." I'll bet money it was the setup, not the model, and I'd love to help you find it.
Danny
