Building a Kubernetes Development Cloud with Raspberry Pi 4, Synology NAS and OpenWRT – Part 2 – Preparing the Network and Storage

This is the second article in a series covering detailing building a Raspberry Pi Kubernetes development cluster. In the first part we covered what was going to be built and the hardware choices. In this post we will cover the initial installation steps to get the cluster up and running.

Preparing the Router

The fist device we need to configure is the router. I am using a Linksys WRT flashed with the OpenWRT. If you have a different device you will need to adapt these instructions accordingly.

While strictly no longer necessary in my setup,one of the things I did to my router was to expand the root file system to allow me to install more software and store more files. If you also want to expand your root file system follow these instructions to do so.

https://openwrt.org/docs/guide-user/additional-software/extroot_configuration

If you are happy with the capacity of your router then the first step is to log in and set a password. Next navigate to System > System and set your hostname

Now lets create a VLAN for the cluster. Ensure the switch that powers the Pi cluster is plugged into the router (I have it in LAN port 1).

Navigate to Network > Switch. Click the “Add VLAN” button and then select “tagged” in the drop down for “CPU (eth0)”.

If you have your switch plugged into “LAN 1” like me then select the “LAN 1” drop down in the “VLAN 1” row then change it from “untagged” to “off”. In the “VLAN 3” row you just created, set “LAN 1” to be “untagged”. Your VLANs should now look something like this.

Next navigate to Network > Interfaces and click the “Add new interface” button. I called mine “DMZ”, call yours what you like. In the Protocol drop down select “Static address” and in the Interfaces drop down select the VLAN you just created. for me that is “eth0.3”

When you click “create interface” you will be taken to the next screen to configure further settings. The first field you need to enter a value for is the IPv4 address. This will be the IP of the router on this interface. I have selected to go with “10.1.0.1”. I have used the 10 range because it is one of three ranges reserved for internal network use. The existing LAN interface already uses the 192 range and in the past I have had conflicts with docker containers when I have used the 172 range.

When the time comes we will create static DHCP reservations for each node and the NAS. The IP addresses and hostnames I will use are:

Router : 10.1.0.1 rdg-gtway-01
NAS: 10.1.0.20 rdg-strg-01
Cluster: 10.1.0.[21:30] rdg-clust-01

Next select an IPv4 netmask. I went with a class B network mask.

Select the “DHCP Server” tab and ensure that “Dynamic DHCP” is checked. Here we are also going to add some DHCP-Options that are sent to our Raspberry Pi clients to PXE boot them. Add the following 4 DHCP-Options

• “43,Raspberry Pi Boot”
• “60,PXEClient”
• “67,bootcode.bin”
• “66,10.1.0.20”

Option 66 is the IP address of the TFTP server that hosts the boot files. In my case this is the IP address I will give the NAS, where the TFTP service will be running.

Now click the “Firewall Settings” tab and enter a new firewall zone name. I went with “DMZ”.

You can now click save on the interface dialog, and navigate to Network > Firewall

TODO: Fill in section on correct firewall settings.

That will do on the router for now. We will need to come back here later to create DHCP reservations and set up dynamic DNS and Nginx, but for now we need to prepare the NAS.

Preparing the NAS

I am going to assume you are using a fresh NAS and, and you have plugged both LAN ports into your network switch.

First, install the latest Synology DSM software and do any initial configuration it requires during installation. Once set up the first thing to do is to bond the network interfaces. Go to Control Panel > Network. Add in a hostname (mine is rdg-strg-01) click apply and then select the “Network Interfaces” tab. Click the “Create” button and select “Create Bond”. Accept all the defaults (adaptive load balancing, use both interfaces and use DHCP). Once the network adaptor is set up make a note of the IP address that it has been assigned. We will switch this for a static IP.

Go back to the Router and navigate to Network > DHCP and DNS and select the static Leases Tab. Click the Add button and select the MAC address of the Synology NAS bonded interface, and enter a name a give it the IP address you would like it to have.

Make sure you click “Save” then “Save and Apply” and then go back to the NAS.

We are now ready to begin setting up our storage volumes. You might choose a slightly different setup for your own needs, but this is how I have done it.

First set up a new storage pool using all 4 disks in RAID 10. With that done I created 4 ext4 volumes.

• Volume 1 – 10GB (min volume size) This will be used for hosting the TFTP boot files for the Pis.
• Volume 2 – 700GB This volume will host the iSCSI root file systems for the Raspbery Pis. It needs to be big enough to host a 64GB disk for each of your Pis plus a bit of additional space so it reports as healthy (32GB would be enough)
• Volume 3 – 10GB This volume will be used for running some docker images on the NAS later
• Volume 4 – 4TB This is the volume that we will use for hosting kubernetes persistent volumes

We now need to create the iSCSI LUNs that that the Raspery Pi nodes will use as their root file system. Repeat this procedure for each Raspberry Pi node in your cluster. Navigate to SAN Manager and select the “LUN” tab then click the “Create” button. Give the LUN a name. “LUN-1” is fine. Ensure it is on volume 2, give it 64GB which is more than enough for our the operating system disk and select thick provisioning.

Click next and accept the default iSCSI Target and click next again. Here we need to grant permissions to our new LUN. Select the “Custom” checkbox and then click “Add New Host”. Enter the hostname you intend to use for the node and select Linux as the operating system. You also need to add an initiator, give this the same name as the hostname

Finish the wizard then repeat for each of your other nodes. Once you have finished you should have a 64GB LUN for each Pi, and you should have defined a host for that LUN which has read/write access. Only one host should have access to each LUN.

Before leaving San Manager navigate to the iSCSI tab on the left click the “edit” button on the “Synology iSCSI Target”. Select the “Advanced” tad in the dialog that pops up and check the box for allowing multiple sessions.

Now that we have our root file system volumes in place, lets move on to the TFTP boot directory.

Go to Control Panel > Shared Folders and create a new folder called “tftp” on Volume 1. Go through the wizard accepting the defaults

Next go to Control Panel > File services. Make sure NFS is enabled on the NFS tab, and on the Advanced tab enable TFTP and select the directory you just created on Volume 1. Make sure you apply your settings,

Before we are done go to Terminal & SNMP and make sure SSH is enabled so we can shell into the NAS later to copy files.

We are now ready to begin installing the Raspberry Pi nodes!