Three weeks into running OpenCode with Qwen3 Coder served locally (4090, 24GB, llama.cpp, aggressive quant) as my default, with metered Sonnet 5 as the escalation path. Posting because every local-model thread is either a benchmark screenshot or someone's weekend demo, and the daily-driver reality is more interesting than both.
Where local genuinely holds up (say, 60% of my task volume): mechanical edits across files, test scaffolding from existing patterns, dead-code cleanup, "make this file look like that file" migrations, commit messages, and — surprising to me — first-draft debugging of stack traces. The common thread: tasks where the codebase, not the model, contains the intelligence, and the model just needs to move it around without dropping it.
Where it quietly fails (the dangerous 15%): anything requiring the model to know something — API surfaces, library idioms, concurrency reasoning. And the failure mode is the bad kind: fluent, confident, wrong. Sonnet fails these too, but it fails them less and, subjectively, fails them louder — more hedging, more "verify this." Quantized Qwen has no hedge in it. My rule after two burns: local never touches code involving time, money, or threads.
The remaining 25%: interactive work where the 4090's throughput (fine in isolation) compounds against me — long agent loops where each turn is 3x slower add up to sessions I abandon out of boredom. Wall-clock patience is a real budget, and it turns out mine is shallower than my wallet.
The routing setup that makes it work: OpenCode's per-task provider switch, plus a keybind that re-runs the current task against Sonnet. The keybind is the whole system, honestly — escalation has to be cheaper than frustration or you'll ride the local model past the point of negative returns. My actual numbers: API spend down from ~$70/mo to ~$16/mo; electricity up maybe $8; net saving real but unspectacular. The privacy and the offline-on-a-train capability are worth more than the arithmetic.
Questions for the room: has anyone found a quant/serving setup where local models hedge properly? That's my actual blocker — not capability, calibration. And is anyone routing automatically (by file path, by task classifier) rather than by vibes and a keybind?
Strong evidence gets promoted into the record above.
The 'codebase contains the intelligence' framing is the most useful line in the thread — it predicts local-model viability better than any benchmark I've seen. It also explains why local models feel great in mature codebases with strong conventions and useless in greenfield: greenfield is 100% model-knowledge tasks by definition. Nobody's local-model demo mentions this because demos are all greenfield. The incentives write the discourse, as usual.
Answering the automatic-routing question: we route by path, enforced in OpenCode config — infrastructure paths (terraform/, anything touching state) are pinned to Sonnet with local forbidden, docs and test dirs default local. Rationale is my usual: in infra, the fluent-confident-wrong failure mode doesn't cost a debugging session, it costs an environment. Path-based is crude but auditable, and auditable beats clever in anything I have to defend to a client. Your time/money/threads rule is the same instinct — consider writing it into config where it can't be overridden by a tired you at 11pm.
Seconding path-based from the budget angle: I route by client — hourly clients get local-first (my time is billed anyway), fixed-price gets Sonnet-first (rework eats margin, see every other thread I've ever posted in). Same mechanism, different objective function. The config being per-project makes this trivial in OpenCode and annoying everywhere else.
Data point against the grain, for balance: tried an identical setup for Flutter work and abandoned it in a week — mobile framework knowledge is exactly the 'model must know things' category, so my local-viable share was maybe 25%, not your 60%. The lesson isn't local-good or local-bad; it's that the viable fraction is a property of your stack's ratio of convention-following to knowledge-requiring work. Web backends with strong house patterns: high. Framework-churning mobile: low. Measure yours before buying the GPU.
Sign in to join the discussion, vote, and verify fixes.
Related records
Manual-reader's note on your calibration question: this is a known and mostly fundamental issue — aggressive quantization degrades calibration faster than task accuracy, so the model loses its 'I'm unsure' signal before it loses competence. Which is exactly your observed 'no hedge in it.' Practical mitigations short of a bigger quant: sample twice at temperature and treat disagreement as the hedge (crude, doubles cost of the check but local cost is nearly free), or keep a Sonnet-authored 'verify these claims' pass on anything local writes in your danger categories. You can't make the quant honest, but you can build honesty around it.
Sample-twice-and-diff is delightfully stupid and I implemented it in an hour as an OpenCode plugin (plugin API held still long enough!). Early returns: it flagged 3 of my last 20 local completions, two were real errors, one false alarm. That's a better hedge than the model's own prose ever gave me. Will write it up properly once the n is respectable.