Jordan was PM at a Series A fintech under serious pressure. The CEO wanted an AI expense categorizer live before the next board meeting — “two weeks, max.” Jordan and one sharp junior engineer fired up Cursor and Claude Code. By Friday night they had a demo that looked production-ready: drag in a receipt photo, watch the categorization appear, clean dashboard, even a confidence score. Investors saw it Monday. “This is exactly what we need. Ship it.”
Six weeks later the thing was still in staging. Real users uploaded PDFs from banks the parser had never seen. LLM costs tripled — no rate limits, no caching. The new backend engineer refused to touch the codebase. “This needs a full rewrite,” she said. “There's no schema, the data layer is a JSON file that'll collapse under concurrency, secrets are scattered in three places, and the tests the agent wrote mostly check themselves.”
Jordan learned the expensive way that “it worked in the chat” is not a production metric.
AI gave everyone superpowers in the demo phase. It did not remove the need for judgment, architecture, and discipline when real money enters the picture.
Why the Demo Feels Magic and Production Hurts
In the demo you control the inputs. Happy path only. Tiny or mocked data. One user at a time. No one trying to break auth or spam the model with garbage. Failures are free — you tweak the prompt or refresh. Scope is whatever you decided to show that day.
Production is the opposite. Real PDFs arrive with weird formats. Fifty users hit the endpoint at once and your JSON “database” starts corrupting itself. An attacker — or a curious user — pokes at the API and finds the key you left in the frontend. Inference and egress costs show up on the AWS bill at 3 a.m. A new engineer spends two weeks understanding a 400-line function with three nested “TODO: handle edge case” comments.
The gap isn't a few bugs. It's structural. And AI-generated code often makes it worse because the code looks plausible even when the foundations are missing.
Axel Molist Cordina, July 2026: “I vibe coded the start of an app. The developer who took it on rewrote it from scratch — easier than adapting AI-generated code. The gap between a demo and production software is still very real.”
The Most Common Gaps (and Why They Bite)
The prototype stores everything in a local JSON file or pickle because “it worked.” Then real traffic hits. Writes collide, queries slow, no transactions, backups, migrations, or indexing. One team watched their preference store corrupt during a small marketing push. Moving to Postgres took three weeks and forced a near-rewrite. Persistence was never solved — just postponed.
LLMs love to drop API keys, tokens, and connection strings straight into code. In a demo it feels harmless. In production it's an audit failure or a breach. One startup found keys committed across three services in the AI-generated repo. The fix wasn't one line — it was ripping out the pattern everywhere.
“It's just me and the AI,” so no version control. Later a real engineer joins or you need to roll back — and there's nothing to roll back to. One founder shipped a 400-file codebase with zero schema docs and components that broke the moment they connected. Alive in demos, structurally sick underneath.
You ask the model to review its own output. It often “fixes” the problem by weakening the test or changing the assertion instead of fixing the logic. The code passes its own tests and still falls over. Treat agent output as a first draft that needs independent review — another model, a human, or tools with real test data.
Generated endpoints work for the exact demo payload. Real traffic brings malformed input, missing fields, unexpected types. No proper auth, rate limiting, error handling that doesn't leak stack traces, versioning, or observability. One team ended up with a single 800-line route because every edge case got another if statement.
Vercel + Supabase is incredible for demos — deploy in seconds, generous free tier. Then traffic spikes, data grows, you need SOC2 or GDPR, multi-environment deploys, or predictable inference costs. Many teams migrate to AWS/GCP anyway and discover surprise egress and inference bills. The demo never showed the second bill.
How Experienced Teams Bridge the Gap
The teams that ship reliably don't pretend the demo is most of the work. They treat vibe-coding as a superpower for exploration and validation only. Then they pause.
They run an architecture review. They add real tests not written by the same agent that wrote the code. They run security scans and secret detection from day one. They model costs at 10x and 100x scale before the first real user arrives. Then they decide deliberately: harden the core, or rewrite the critical paths? Often the latter — several founders have said publicly a clean rewrite beat salvaging a messy AI-generated foundation.
They use agents in production, but with guardrails, human-in-the-loop for high-stakes flows, and proper observability. The agent becomes a force multiplier, not the architect.
“When 100+ lines appear in one go, you stop having a model of the problem in your head.”— Paras Chopra
Good AI-assisted engineering keeps you in the loop — line by line when it matters — so the codebase stays understandable to humans, not just to the model that generated it.
Framework: Demo-to-Production Red Flag Audit
Run this before anyone says “it's ready to ship.” More than two red? Treat the prototype as learning, not the foundation.
- 1
Secrets or keys hardcoded anywhere in the repo or frontend?
- 2
Persistent data in local JSON/pickle/files instead of a real database with migrations, transactions, backups?
- 3
Proper Git history, branches, PR process — or a single “final_v3.py”?
- 4
Independent review (human or separate model + real test data), or same-agent self-QA?
- 5
Cost modeled at 10x and 100x usage — inference, storage, egress, database?
- 6
Rate limiting, safe error handling, structured logging, monitoring in place?
- 7
New senior engineer onboards in under a week — or needs a month to understand the shape?
- 8
Concurrency and realistic data volumes tested — or only happy-path single-user flows?
Common Mistakes PMs Make Here
| Mistake | Reality |
|---|---|
| Treating the demo as 80% of the work. | The last 20–50% is the real product. The demo validated the idea, not the system. |
| Letting the same agent review its own output. | It changes the test, not the bug. Independent verification is non-negotiable. |
| Skipping cost + scale modeling. | Inference and egress are easy to underestimate in happy-path mode. The bill finds you. |
| No architecture/security checkpoint. | Debt compounds faster with AI code — it's harder for humans to reason about later. |
| Shipping brittle systems because the CEO loved the demo. | Prototypes de-risk the idea. Process de-risks the system. Protect both. |
Key Takeaways
“It worked in the demo” is not a KPI. Surviving real users, data, attacks, costs, and new engineers joining is the bar.
Vibe-coding is extraordinary for exploration. It's not a substitute for engineering discipline in production.
The gap is smaller when the prototype is disposable learning, not the foundation you must live with.
Create space for hardening without killing velocity. De-risk the idea fast, then harden or rewrite deliberately.
The teams winning in 2026 aren't shipping the fastest demos — they know exactly when the demo has done its job.
Ask Your Engineering / Data Team
1. “If a new senior engineer joined tomorrow, how long until they could safely ship a meaningful change to this module without breaking something else?”
2. “What's our modeled p99 latency and monthly cost per user at 10x and 100x traffic — and have we tested it with realistic data volumes?”
3. “Where would a security audit or SOC2 review find the biggest gaps right now — and how long to close them?”
early ship products that survive. The ones that don't
learn it the hard way — usually at 2 a.m., staring
at a dashboard that used to work in the chat.
CHAPTER 14 AT A GLANCE
| Core question | Has the demo actually built the system, or just validated the idea? |
| Frameworks | Demo-to-Production Red Flag Audit (8Q), Harden/Rewrite/Buy Decision Flow |
| Stories | Jordan (fintech categorizer), Axel Cordina (rewrite), Paras Chopra (loss of control) |
| Key rule | Prototypes de-risk the idea. Process de-risks the system. Decide deliberately. |