From Raspberry Pis to a Private AI Development Cloud

This started because I wanted somewhere real to build.

Not another laptop Docker Compose file, and not a cloud account where every experiment left a standing bill. I wanted an environment with separate machines, network boundaries, persistent storage, ingress, certificates, deployment pipelines, logs, metrics and failure modes. Something close enough to production to make architecture decisions earn their keep.

The first version was a ten-node Raspberry Pi cluster. It is still there more than three years later, but the system around it has changed considerably. The cluster now sits between a Synology storage layer and two very different AI inference machines: a dedicated NVIDIA GPU server for fast work and a high-memory AMD Evo node for long-context models.

That evolution is the useful part of the case study. This was not a weekend cluster assembled for a photograph. It became the private infrastructure behind Radical Geek experiments, products, observability, delivery tooling, Rembr services and agentic engineering work.

The platform at a glance

Layer	What it does
Network and power	OpenWRT routing, an isolated cluster network, managed PoE switching and UPS-backed core equipment
Compute cluster	Ten Raspberry Pi 4 8GB nodes running 64-bit K3s: three control-plane nodes and seven workers
Storage	Synology NAS providing PXE/TFTP boot files, NFS, per-node iSCSI root disks and dynamically provisioned Kubernetes volumes
Fast AI lane	Dedicated Linux server with an NVIDIA RTX 4070 Ti SUPER and 16GB VRAM
Long-context AI lane	128GB AMD Ryzen AI Max/Strix Halo Evo workstation using high-capacity shared memory
Service layer	Traefik, certificate management, Git-based delivery, metrics, logs, model routing and stable APIs for applications and agents

The original Pi estate provides 40 ARM CPU cores and 80GB of RAM. That is modest by public-cloud standards, but it is enough to run a surprisingly broad service layer when workloads are separated properly and state is kept out of the nodes.

Why I built it

I spend a lot of time making decisions about deployment, service boundaries, resilience and operability. Those decisions are difficult to test honestly if the only environment available is a developer laptop.

I wanted to be able to answer practical questions:

• What happens when a worker disappears halfway through a deployment?
• Can a stateful application move between nodes without taking its data with it?
• Is a certificate or ingress change repeatable, or did it work once by accident?
• Can I recover a database volume from a point-in-time snapshot?
• What does a useful observability stack look like when it has to monitor itself?
• How should local model inference be exposed to applications without binding every client to one machine?

A private cloud is useful here because the constraints are visible. There is no managed service quietly repairing the hard parts. If storage, routing, scheduling or monitoring are badly designed, the platform tells you.

The original cluster

The compute layer began with ten Raspberry Pi 4 boards, each with 8GB of RAM, PoE power and an additional USB 3 gigabit network adaptor. Three nodes formed the K3s control plane and the remaining seven were workers.

The PoE design was more important than it sounds. One managed 24-port Netgear switch carried data and powered the nodes, so a Pi could be power-cycled at the switch without reaching into the rack. The router, NAS and switch were protected by a UPS; because the Pis drew power from the switch, the whole core platform shared that protection.

K3s was the right Kubernetes distribution for this shape of hardware. It kept the control-plane footprint sensible, supplied Traefik for ingress, and behaved enough like the larger Kubernetes environments I work with for the lessons to transfer.

The live estate still exposes all ten original Pi nodes. At the latest check, nine were reporting Ready and one was offline; several were deliberately cordoned for maintenance. That is a more honest proof point than a perfectly green diagram. This is long-running working infrastructure, with failed components and maintenance states made visible rather than hidden.

The NAS is the centre of the design

The most important machine in the original platform was not a Kubernetes node. It was the Synology NAS.

Earlier storage experiments used whatever spare hardware I had available. They worked until more nodes and more state arrived, at which point storage became the source of nearly every awkward failure. Moving to a Synology DS920+ with four 4TB IronWolf Pro disks in RAID 10 gave the cluster a proper shared storage layer and removed local disks from the compute design.

The NAS performs several distinct jobs:

1. It hosts the TFTP files used to network-boot each Pi.
2. It provides an individual iSCSI LUN as the root filesystem for each node.
3. It exports NFS where shared filesystem semantics are useful.
4. It supplies Kubernetes persistent volumes through the Synology CSI driver.
5. It supports volume snapshots for recovery and for creating new environments from known points in time.

The Pis therefore do not depend on SD cards for their working operating systems. OpenWRT supplies the PXE boot options, each Pi retrieves its kernel and initial filesystem from the NAS, then attaches its own iSCSI root disk. The nodes are cheap and replaceable; their identity and state live elsewhere.

That approach adds network and storage dependencies to the boot path, so it is not automatically the right design for every cluster. Here it was intentional. I wanted consistent node images, centralised storage operations and the ability to replace a compute board without treating the board itself as precious.

For application state, the official Synology CSI driver can create an iSCSI LUN when a workload requests a persistent volume. NFS-backed provisioning is also available for workloads better suited to shared files. Current cluster state still shows active Synology iSCSI and NFS storage classes, with volume expansion and retained data policies configured for the shared storage paths.

Networking without pretending it is simple

The cluster has its own routed network behind OpenWRT. DHCP reservations give infrastructure stable identities, local DNS provides predictable service names, and the router handles traffic between the main LAN, the cluster segment and external ingress.

The original build also used a separate physical backbone on the Pis for cluster traffic. That prevented storage and inter-node communication from competing blindly with ordinary client traffic. The managed switch provided the VLAN and PoE controls needed to operate the hardware as a small platform rather than a pile of boards.

Traefik handles Kubernetes ingress. Certificate management automates TLS issuance and renewal, while the router provides the outer routing and reverse-proxy boundary. The division is deliberate: Kubernetes owns service discovery and workload ingress; the network edge owns how traffic reaches the cluster.

Making it a development cloud

Compute and storage were only the foundation. To become useful, the cluster needed the services around application delivery.

Over time I added Git-based source and package management, CI runners, PostgreSQL, Redis, object storage, Prometheus, Grafana, log aggregation, exporters, alerting and certificate automation. Workloads could be built, deployed, observed and recovered without inventing a different path for every project.

The platform now runs a mixture of product and engineering services. Rembr, for example, uses replicated API and UI services, PostgreSQL, connection pooling and Redis on the cluster. Agent gateways and model-routing components run alongside normal web and data workloads. The same infrastructure can support a conventional service one minute and a team of software agents the next.

That is exactly what I wanted from the lab: a place where new architecture has to coexist with databases, storage limits, network policy, certificates, restarts and old decisions.

Adding the dedicated GPU node

The next major change came when local AI work outgrew laptop hardware.

I added a Linux server built from a repurposed desktop and an NVIDIA RTX 4070 Ti SUPER with 16GB of VRAM. The machine was configured with the NVIDIA container runtime and originally joined to K3s as a GPU-capable worker.

Kubernetes needed more than a driver installation. The container runtime had to expose the GPU, an NVIDIA RuntimeClass had to be available, and the device plugin had to advertise nvidia.com/gpu capacity to the scheduler. I labelled the node for AI workloads and constrained the device plugin so it did not try to initialise on every ARM worker.

The result was a heterogeneous cluster: ARM nodes for ordinary services and an x86/NVIDIA machine for containers that explicitly requested a GPU. The same pattern later proved useful even when inference moved towards host-managed model servers. Kubernetes could expose the capability through a stable service without forcing the model runtime itself to look like every other pod.

Today that machine is better understood as the fast inference lane. Its 16GB GPU is well suited to responsive, shorter-context coding and embedding workloads, provided model size and context are controlled.

The Evo changed the shape of local inference

Fast VRAM is only one side of local AI. Agentic engineering often needs much more context than a 16GB card can hold comfortably.

The second inference machine is an Evo-style workstation built around AMD Ryzen AI Max/Strix Halo hardware with 128GB of shared memory. In practice, roughly 96GB can be made available to the integrated Radeon GPU, depending on firmware and runtime configuration.

That changes the trade-off. The Evo is not there to beat a discrete NVIDIA card on raw token speed. It is there to keep larger quantised models and long context windows local. It can carry repository-scale context—up to the 200K profiles used in Radical Geek testing—while also keeping an embeddings model available.

The two machines therefore have different jobs:

• NVIDIA fast lane: shorter-context, latency-sensitive coding and interactive work;
• Evo long-context lane: repository analysis, planning, context construction, review and larger model fits.

Trying to make either box handle every request would waste its strengths. A routing layer presents a stable OpenAI-compatible API to applications and agents, then chooses a lane based on context size, model availability and the type of work. Compatibility and guard services isolate callers from machine-specific endpoints, and cloud fallback remains an explicit option rather than a hidden default.

The important architectural decision was to route by workload, not by brand loyalty to one model or one piece of hardware.

From homelab to agentic infrastructure

The cluster now supports the layers needed for agentic engineering:

• model routing across local inference lanes;
• persistent workspaces and shared storage;
• agent gateways with stable service endpoints;
• Rembr memory and context services;
• PostgreSQL, Redis and object storage;
• metrics, GPU exporters, logs and dashboards; and
• ingress and certificates for product and control-plane interfaces.

This does not make the platform a miniature public cloud. It makes it a controlled environment where the full path can be inspected—from an agent request, through model routing and memory, into a workload, onto persistent storage, and back through logs and metrics.

For work involving proprietary repositories or long-lived agent context, local inference also provides a useful privacy boundary. Data can remain on infrastructure I operate, while selected requests can still be routed elsewhere when a cloud model is the right engineering choice.

What has held up

Several decisions proved durable:

• Separate compute from state. Replaceable nodes are only replaceable when data is not trapped on them.
• Use boring interfaces. NFS, iSCSI, HTTP and OpenAI-compatible APIs made it possible to change implementations without rebuilding every consumer.
• Keep the edge distinct from the cluster. OpenWRT and the managed switch own network boundaries; Kubernetes owns workloads and services.
• Route heterogeneous compute deliberately. ARM, NVIDIA and AMD shared-memory machines all belong in the platform, but not in the same scheduling bucket.
• Instrument the infrastructure itself. A private platform without logs, metrics and alerts is just a collection of machines waiting to surprise you.
• Expect maintenance debt. Nodes fail, certificates expire, images disappear and old operating systems need attention. The platform is valuable because those problems are encountered here before they become a client incident.

Where the design is deliberately limited

The Synology NAS and main router remain important failure domains. The Pi control plane is distributed, but this is not a geographically redundant platform. Some nodes are old, the hardware is mixed, and not every workload deserves to run here.

Those limits are acceptable because the purpose is development, product incubation, local AI and architecture proof—not pretending a rack in my office has the resilience of a multi-region managed service.

The platform helps me decide what should stay private, what should move to managed infrastructure, and what operational work a design will create before I recommend it to someone else.

The outcome

What began as ten Raspberry Pis became a long-running private development cloud, then a private AI platform.

It has provided a realistic place to build and operate Kubernetes services, test storage and recovery patterns, run delivery pipelines, develop Radical Geek products, host memory and agent services, and learn where local inference is genuinely useful.

The value is not the novelty of the hardware. It is the fact that the architecture has kept accepting new capabilities without requiring the whole platform to be discarded.

For the original implementation detail, start with the Raspberry Pi, Synology and OpenWRT build overview and the GPU-node build notes.

Book a call if you need a pragmatic platform architecture your team can actually operate.