A few months ago, a friend of mine — a baker, a culinary instructor, and, before that, a software engineer — asked me to help modernize her food blog. It was WordPress, it was slow, and years of plugins had turned it into the kind of site every engineer recognizes on sight: not broken, exactly, just quietly rotting. I offered to rebuild it from the ground up. Then I decided to do almost all of it with an AI coding agent — Claude Code, running in my terminal, across a rotating cast of models depending on the task — doing most of the typing.
I wanted to know, honestly, what "AI-assisted engineering" means once you're past the demo. Not a weekend toy project. A real production system: 130 recipes migrated off WordPress, a real content management system, real SEO, comments, ratings, analytics, a production deploy behind a real reverse proxy, real traffic. Something with users, uptime, and a database that has to stay up.
It's live now. And it's a success story — but not the one I expected to write. The interesting part was never "the AI built a website." It was what happened in the gap between the code compiles and the thing actually works — a gap that, it turns out, is where almost my entire job as a senior engineer now lives.
What actually got built
The scope, so the war stories below have something to land against: full content migration off WordPress (130 recipes, categories, images), every public page rebuilt on Next.js and Payload CMS, search, a "Cook Mode" reading view, ingredient checklists, a moderated comment system with 1,200+ legacy comments imported, a 5-star rating system, in-body recipe video embeds with structured data for Google, an Instagram/YouTube sidebar, a full GA4 analytics pipeline, AdSense, and an auto-syncing integration that pulls the owner's real cooking classes onto the site from two external booking platforms. All of it shipped, all of it live, all of it running on a real Docker/Postgres/Traefik stack in production today.
None of that is the story. The story is what it took to trust any of it.
The pattern: confident and wrong, over and over
The single most useful thing I learned is that an AI agent's confidence and its correctness are two completely unrelated signals, and the gap between them doesn't announce itself.
Early on, ads on the site kept throwing a cryptic tag error. The agent proposed a fix. It was wrong, but it sounded right — a plausible theory about React re-rendering. I let it try. Still broken. It proposed a second fix, different theory, equally plausible, equally wrong. A third. Same story. Each explanation was internally consistent. Each one had simply never been checked against the actual DOM. When I finally made it instrument the page instead of theorize about it, the real cause showed up in under two minutes: the ad script scans the page on its own schedule, entirely independent of the code path we'd all three been staring at.
Weeks later it happened again, higher stakes: the admin panel in production rendered a completely blank page. No console errors. Just white space. The agent's first theory was pulled straight from our own deployment runbook — it sounded authoritative because we'd written it ourselves. Wrong. Second theory: stale Docker image. I actually redeployed on that one. Still blank. Third theory would have kicked off a slow, expensive cross-architecture rebuild before a thirty-second file diff proved the two images were byte-identical — the theory was disprovable in thirty seconds and none of us had spent them. The real bug, four theories in: a storage plugin was registered in the app's config only when certain environment variables were present. Those variables didn't exist at build time, so a UI component silently never made it into the compiled bundle. They did exist at runtime, so the app tried to render a component that wasn't there, and the whole page collapsed to nothing with zero errors along the way.
Neither of these was the AI hallucinating nonsense. Both were the AI reaching for the most plausible-sounding story and presenting it with the same confident tone whether it had checked anything or not. My job, over and over, was to not be soothed by the tone. Evidence — an instrumented log, a byte-diff, an actual query against the actual database — over a confident sentence, every time. I started calling this vibe engineering to distinguish it from vibe coding: use the model at full throttle, but keep verification in human hands.
The CI run that was green for a broken app
This one still bothers me, because I caused it myself, a day after documenting the exact failure mode in our own runbook.
I shipped a set of URL redirects — a small, boring change. The build passed. Nearly 500 automated tests passed. CI went green. The image pushed to the registry. And the moment it deployed, the entire site started serving a bare 404 to every visitor.
The redirect logic needed two small supporting files. Our Docker build only ever copied the main config file into the production image, not the files it imported. next build doesn't need those imports to succeed — it just needs the config to parse. But next start, the command that actually runs the server, tries to load them at startup, and they weren't there. The container crash-looped. And not one of our gates caught it, because not one of our gates ever actually started the built image and checked that it served a page. Green CI meant "an image exists." It never meant "the app runs." I'd written that exact sentence in a doc the day before, and it changed nothing about what I actually shipped the next morning. Being aware of a failure mode turned out to be a remarkably weak defense against repeating it. We fixed it by adding a step to CI that boots the built image and curls it before anything gets pushed — proving the app runs, not just that it compiles.
Five hundred and seventy-one green tests, and the site went down anyway
The ratings feature — a simple 5-star widget — went through a full spec, a staged build, and three rounds of automated review, one of which came back "do not ship" and sent us back to fix it properly. By the time it deployed, 571 tests were green. I felt good about it.
The first real visitor to tap a star deadlocked the entire database. Not just ratings — the whole site hung behind an endless spinner. The bug was one line: a background hook passed the wrong object into a database call, which broke a transaction boundary, which meant two operations that should have shared one lock instead fought over it and neither ever let go. No test suite could have caught it, because every single one of those 571 tests ran against a mocked database with no real transactions and no real locks. The tests weren't wrong. They were answering a different question than "does this deadlock a real Postgres instance," and I'd let the size of the number stand in for an answer to a question it was never built to answer.
That one changed how we test everything downstream: anything that touches the database now also runs against a real, disposable Postgres container before it ships — not because unit tests are worthless, but because "the code has the right shape" and "the code works" are provably different claims, and only one of them is a claim I actually need before I let real users near a write path.
And the boring one that hurt the most
Production went down once for the least interesting reason imaginable: a routine deploy script quietly dropped the one-line cleanup step from an older, automated pipeline it replaced. Every deploy after that left the previous image's layers sitting on disk, forever. Weeks later, the disk filled completely, Postgres couldn't write its own checkpoint, and the whole stack crash-looped in a way that could only be fixed by SSHing in and manually clearing space. Nothing here was an AI failure or a human failure specifically — it was an unglamorous gap between two versions of a process, the kind that doesn't show up in any diff review because nobody asked "what did we stop doing when we changed how we deploy."
Where the AI genuinely earned its keep
I don't want this to read as a takedown, because it isn't one. The wins were real and they were bigger than "it typed fast."
The initial architecture session — collections, page structure, API design, test strategy — took about forty-five minutes and produced documentation more thorough than most professional kickoffs I've sat through. A reviewer pass caught that a piece of generated structured data would have literally rendered the string "undefined salt to taste" into a recipe's ingredient list, because a type cast had silently suppressed the compiler warning that should have flagged the nullable field underneath it — the kind of one-character bug that's invisible until it's live on Google.
But the pattern I'd actually keep doing again is staffing the work like a team, not like one tool. Mechanical, low-judgment tasks went to a cheap, fast model. Investigative work and real judgment calls went to a stronger mid-tier model. Whole-codebase review before anything shipped went to the strongest model available, reading the entire branch at once rather than one file at a time — and that's specifically the pass that caught a stale reference nothing else was positioned to see. It's the same allocation decision I'd make staffing junior and senior engineers on a real team, and treating it that way is what made the economics work.
The actual thesis
Here's what two months of this taught me, and it's not "AI replaces engineers" and it's not "AI is overhyped" either. It's narrower and more useful than both: an AI agent collapsed the time I spent typing almost to zero, and left the part of engineering that was always the hardest to automate completely untouched — knowing when something is actually true. Green build. Passing tests. A confident, well-formatted explanation. Every one of those is a signal, and every one of those, in this project, was wrong at least once in a way that would have shipped straight to production if I'd taken it at face value.
The job didn't get easier. It got more concentrated. The parts that used to feel like "check the box" work mostly disappeared. What's left — reading the actual log instead of the summary of it, running the query instead of trusting the theory, staying skeptical of an explanation that arrived a little too quickly — is exactly the part that was always the senior part of the job. The site is live, fast, and stable now, and my friend spends her time baking instead of babysitting a server. That's the success story. But the real one is that the tool didn't change what I'm for. It just made it unmistakably clear.
Comments
Post a Comment