Radical Geek field guide
Radical Geek's Guide to Local AI for Coding And Agentic Engineering
A practical guide to operating sanctioned local, cloud and hybrid model lanes with bounded authority, fallback and work-item evidence.
Why this guide
Local AI for coding is useful when it solves a real constraint: private code, predictable cost, low-latency specialist work, resilience, or control over where inference runs. It becomes expensive furniture when the operational system around it is missing.
This guide comes from running local models as part of a working agentic engineering environment. It covers the machinery around the model as seriously as the model itself: routing, queues, loading, expiry, observability, sanctioned fallback and the point at which cloud inference is the sensible route.
It is part of the Radical Geek guide system, owning the model and runtime lanes within the same governed work-class lifecycle.
Local inference earns its place when it becomes a dependable lane in the delivery system: measurable, routable and able to fail without stopping the work.
What you will leave with
A guide built to be used.
- 01
Match governed work classes to model lanes instead of sending every request to the largest model.
- 02
Design local, cloud and hybrid lanes around privacy, latency, quality and cost without granting wider authority.
- 03
Define a sanctioned fallback route that preserves data, consequence, authority and cost boundaries.
- 04
Choose hardware, runtimes and quantisation levels with realistic operational trade-offs.
- 05
Feed model route, tokens, latency, loading, expiry, failure and fallback evidence into the work-item ledger.
Inside the guide
The working ground it covers.
- Hardware choices and realistic capacity planning
- Model fit, quantisation and runtime selection
- LiteLLM routing and controlled cloud escalation
- Multi-node inference and workload placement
- JIT loading, TTL expiry and queue behaviour
- Observability, privacy, cost and operational limits