Radical Geek field guide
Radical Geek's Guide to Local AI for Coding And Agentic Engineering
A practical guide to choosing, running, and routing local AI models for coding assistants, agentic engineering workflows, private code, and cost control.
Why this guide
Local AI for coding is useful when it solves a real constraint: private code, predictable cost, low-latency specialist work, resilience, or control over where inference runs. It becomes expensive furniture when the operational system around it is missing.
This guide comes from running local models as part of a working agentic engineering environment. It covers the machinery around the model as seriously as the model itself: routing, queues, loading, expiry, observability, fallback and the point at which cloud inference is the sensible route.
Local inference earns its place when it becomes a dependable lane in the delivery system: measurable, routable and able to fail without stopping the work.
What you will leave with
A guide built to be used.
- 01
Match coding and agentic workloads to models instead of sending everything to the largest model.
- 02
Design local, cloud and hybrid routing lanes around privacy, latency, quality and cost.
- 03
Choose hardware, runtimes and quantisation levels with realistic operational trade-offs.
- 04
Run local inference as an observable service with loading, expiry and failure behaviour.
Inside the guide
The working ground it covers.
- Hardware choices and realistic capacity planning
- Model fit, quantisation and runtime selection
- LiteLLM routing and controlled cloud escalation
- Multi-node inference and workload placement
- JIT loading, TTL expiry and queue behaviour
- Observability, privacy, cost and operational limits