Case study: anatomy of a failed build
Same prompt - build a complete, production-quality SaaS landing site - sent once to 10 different models. 50% actually finished. This isn't about which model is "best" - it's about the specific, different ways the rest failed. Some of them looked like they worked.
This task scaffolds a real multi-file Next.js app, not a single HTML file, so there's no screenshot for every run - but we rendered the one clean success below to see what it actually looks like.
The one clean success, rendered
50%
Finished with a real, working scaffold
1
Claimed their own todo list was 100% complete while shipping zero files
The exact prompt used
Build a single-page SaaS landing site using Next.js (App Router), TypeScript, and Tailwind CSS. It should feel like a premium startup site (Linear, Vercel, Stripe) - not a generic template. ## Design Dark mode, near-black background (#080808), white text, electric lime + cyan + purple accents, soft glows. Glassmorphism cards, soft borders, subtle glow effects, rounded-xl/2xl corners. Tasteful hover/fade motion - alive but not distracting. Fully responsive, semantic HTML. ## Single page (/) - all in one file plus a few components - Sticky navbar (logo, nav links, Sign In, Get Started CTA) - Hero: bold gradient headline, subheading, primary + secondary CTA - Feature grid (6 features) - 3 testimonials with names/avatars/companies - Pricing section (3 tiers: Free/Pro/Team, cards only, no separate page) - FAQ (4-5 questions) - Final CTA + footer ## Components Build 4-5 small reusable components (Navbar, Footer, GlassCard, PricingCard or similar) - not one giant file, but keep it tight. ## Copy Real, compelling SaaS marketing copy - no lorem ipsum. ## Goal Should look and feel like it could launch on Product Hunt today - polished and visually impressive, built efficiently in one focused page rather than a full multi-page site.
What actually happened - 5 distinct failure modes, not one
cohere/north-mini-code:free
FREEText-dump (whole app as prose)41sWrote the entire multi-file app as a fenced code block inside its chat reply, narrating file-by-file, but never called the edit tool once. Finished fast and looks superficially like a real answer.
I'll create a premium single-page SaaS landing site with Next.js, TypeScript, and Tailwind CSS. Here's the complete implementation: ## Project Structure [... full file contents follow as plain text, never staged as edits ...]
poolside/laguna-m.1:free
FREEClaimed-complete, shipped nothing5.0mIts own todo list shows all 5 items marked "completed" - yet zero files were ever staged. The turn simply timed out with a todo list that had quietly lied about its own progress.
todos: ["completed", "completed", "completed", "completed", "completed"], edits: []
tencent/hy3:free
FREEPlain timeout, no recovery10.0mTimed out at 300s on both the base attempt and the automatic retry. No todo list, no text, no edits either time.
poolside/laguna-xs.2:free
FREEPlain timeout, no recovery10.0mSame as hy3:free - timed out on both attempts with nothing produced.
deepseek/deepseek-v4-pro
CHEAP-PAIDInconclusive (rate-limited)5.0mBase attempt timed out; the retry came back in 0.3s with an OpenRouter rate-limit error rather than a real second attempt. Not a fair test of this model - infrastructure got in the way before it could really try.
The 5 that finished, fastest first
Every one of these failure modes is now something codelazr actively detects and shows the user - the text-dump and claimed-complete patterns are caught by the client's uselessResponse/emptyResponse detectors, and a timeout with real partial progress offers Continue instead of silence.
Full testing history and methodology: /stats.