AI coding case studies: free vs paid models
Real prompts, real models, no cherry-picking - every result shown, including the failures.
Each case study sends the same coding prompt, unchanged, to several AI models inside CodeLazr - cheap paid models and OpenRouter free models side by side - then checks the actual result: does the page render, does the build pass, did the model finish the task. Where a model failed, you can see how it failed and whether CodeLazr's automatic retries and fixes rescued it. For every run across all tasks rolled up into success rates, see Stats.
Results at a glance
| Case study | What was tested | Key result |
|---|---|---|
| 10 most common coding tasks | 6 models (3 free), 60 runs | 100% after fixing a retry-detector gap (92% before) |
| Overnight free-model marathon | 13 tasks x 4 free models, 52 runs | 2% first try, 51 of 52 rescued by auto-fix |
| Kanban board + auto-fix | Real browser check, multi-model fixer | 33% first try, 100% after auto-fix |
| Does planning first help? | 8 models x 2 prompt styles, 48 runs | Plan-first was slightly worse |
| Guided vs plain prompt | apple.com clone, 6 models | Guidance made successful runs faster and cheaper |
| SaaS landing site | One prompt, 10+ models, deployed | Free models shipped working sites for $0 |
| Multi-API landing page | 3 live APIs, 4 models | All 4 finished, one needed a fixer swap |
| Anatomy of failed builds | Same task, many models | Five distinct failure patterns, some looked like success |
Building a SaaS landing page
Same prompt, 10+ models, real deploys - who actually ships a working site.
Anatomy of a failed build
Half the models failed on the same task - five different ways, some of them looked like success.
Cloning apple.com, guided vs. plain prompt
A little extra guidance in the prompt made every model that succeeded noticeably faster and cheaper.
Kanban board, guided prompt + multi-model auto-fix
Only 33% worked on the first try - a real browser check plus a second opinion from another model got that to 100%.
A multi-step, multi-API landing page
Research a topic, then build a page that actually fetches from 3 real APIs - all 4 models finished, one needed a fixer swap.
An overnight free-model marathon
13 fun tasks, 4 free models, 52 runs - only 2% landed on the first try, but the auto-fixer saved nearly every one.
Does planning first actually help?
Counterintuitive: a plan-first step made success rate slightly worse, not better, across 48 runs.
The 10 most common coding tasks
6 models, 10 everyday task types - a real detector gap was found, fixed, then re-run to 100%.