projectsgallery

Adding a NixOS server to my deployment

Purpose

I'm repurposing a vps running at Hetzner, created before 1st of September 2021 so it's a little bit cheaper :). It will be called via, and run various Matrix bridges, to IRC networks, Telegram, for Twitter DM's, and any future bridging needs. The process for adding a new server is basically the same every time though, installing a bare NixOS from ISO, then adding it to my morph configuration and branching out from there.

Installation

To start, I boot from the NixOS 22.05 ISO provided by Hetzner. Using the webconsole I log in as root and set a password, so I can do the rest of the installation over SSH from my local machine.

Partitioning

With fdisk I repartition the disk:

fdisk /dev/sda
o # generate a new DOS partition table
n p # new primary partition 1, sized with +256M (for boot)
n p # new primary partition 2, rest of disk (for ZFS)
w # write, exit

ZFS

Next I create the ZFS pool with a combination of flags I copied from various sources over time:

1
2
3
4
5
6
7
8
zpool create \
    -O mountpoint=none \
    -O atime=off \
    -O compression=zstd \
    -O xattr=sa \
    -o ashift=12 \
    -o autotrim=on \
    pool /dev/sda2

The partition layout is modelled after Erase your darlings, with a root filesystem that gets reset to a blank snapshot every boot. Filesystems under /volatile are not backed up, filesystems under /safe/ are:

1
2
3
4
5
6
7
8
9
10
11
12
13
14
# Root filesystem, with blank snapshot
zfs create -p -o mountpoint=legacy pool/volatile/root
zfs snapshot pool/volatile/root@blank
mount -t zfs pool/volatile/root /mnt

# /nix, also volatile
zfs create -p -o mountpoint=legacy pool/volatile/nix
mkdir /mnt/nix
mount -t zfs pool/volatile/nix /mnt/nix

# /safe/persist, backed up
zfs create -p -o mountpoint=legacy pool/safe/persist
mkdir /mnt/persist
mount -t zfs pool/safe/persist /mnt/persist

Boot partition

There's a 256MB partition for booting, last VPS I set up only had 64MB and it's a continuous pain...
Format the boot partition as FAT and mount it:

mkfs.fat -F 32 -n boot /dev/sda1
mkdir -p /mnt/boot
mount /dev/sda1 /mnt/boot

Base configuration

Running nixos-generate-config --root /mnt gets the barebones configuration, I'll overwrite configuration.nix with an adaptation of new-node.nix, while the hardware-configuration.nix is good as-is. Notable changes:

• update hostName
• update hostId, used by ZFS and for the internal network v6 ip: head -c 8 /etc/machine-id
• update the ipv6.addresses address to the one Hetzner has assigned to the vps.
• system.stateVersion because I haven't updated new-node.nix to the 22.05 release yet

Copy it to a place where it's actually persisted, just in case we need it in a future accident recovery type situation: cp -r /mnt/etc/nixos /mnt/persist/nixos.

Now the actual installation begins, which is just running nixos-install --no-root-passwd. There's no need for a root password since I'll always connect with my SSH keys, and besides the /etc/shadow file would just get deleted on the next boot anyways :)

Actual setup

With everything installed, it's time to unmount the ISO and reboot into the new server. I'll create a new folder for the server in my nixos/nodes/ and copy over the generated configuration.nix and hardware-configuration.nix. It then has to be added to nixos/nodes.nix and nixos/nodes/nodeProperties.nix. It has to get some secrets, like a Wireguard keypair to participate in the mesh with the other nodes, and a password for it's backups.

With a morph deploy --on via nodes.nix switch --upload-secrets (don't forget the upload-secrets!) and git commit, the new server is now ready for use!