I had some kind of home server / network attached storage since I've started working with Linux. The devices and their tasks have changed over the decades. My last iteration was a Synology DS213J which I have been running on it's original spinning rust disks for the last 11 years at least.

As you can imagine with a computer that's over a decade old, it's not the most performant device anymore, so I was running an old Celeron based mini PC as a small Docker server as well.

But it was really time to replace the Synology and merge the docker server with the NAS server.

My Synology had 2TB of disk space, which I only recently filled ¹⁾. I only have a couple of movies and TV episodes to store. Most of my data is backups, photos etc. So I don't needed a system with a huge capacity.

I knew I wanted to run Docker on the machine for a handful of services and to easily try out software that looks interesting. So a somewhat decent CPU and as much RAM as possible was important.

However, thanks to my HomeAssistant energy monitoring, I know that my “basement electronics” (Synology, router, switch, camera NVR, etc) are the biggest contributors to the base electricity use in our home. Since the NAS will be running 24/7, I wanted to keep the power usage low for the new system.

I decided to build a purely NVME based system around the 6W Intel N150 processor and bought the following hardware:

The above prices were from the Amazon Prime Days, but normal prices aren't too far off. A system using HDDs would reduce the storage price about 200 €, but I'm not sure you could find a N150 based system with enough space for full hard disks at the same price.

I bought the cheapest NVMEs I could find, using the old wisdom to not use the same brand of disks for a mirror array lest they might fail at the same time. However, looking at the disks, I have a strong suspicion they came from the same factory anyway.

Not included in the list is a 500GB Samsung 970 EVO Plus NVME which I used as the boot device and which I still had laying around (it used to be in my desktop).

The CWWK P6²⁾ has a nice build quality and reviews on YouTube seemed favorable. If you pay 100€ more, you can also get it with the more powerful N355 processor, but that also comes with a 15W power draw and more heat, so I picked the N150.

The nice thing about this system is that you can install your own RAM. The maximum supported size is 48GB.

To install it, pop off the lid (it's held with just pogo pins). Next you need to unscrew the visible screws using a Phillips 00 screwdriver. Theoretically only the 4 outer screws need to be removed, but I found it easier to remove all 8, then gently pull up the NVME plate.

With the screws removed, The full outer case can be pulled upwards to be removed (careful with the power button).

The RAM is simply slotted into the SODIMM slot. There is a M2 slot for a Wifi card under the NVME adapter board, but I am not sure there is enough space to actually fit a card there.

There are also two proprietary SATA ports, but only one is easily reachable. The other is also covered by the NVME adapter board. Probably the reason why the system comes with only one SATA cable.

On the underside of the board towards the back is a 4pin fan connector. The P6 has a matching cable included. They also provide an optional fan to mount on top of the NVMEs, but this one is powered by USB.

To actually route the fan and SATA cables out of the case, simply peel off the sticker from the back - it reveals two large openings.

As mentioned above, for now I only used three of the four NVME slots. Two as storage RAID and one for the boot device. Should I want add another set of mirrored disks in the future, I would move the boot disk to an external SSD connected via the mentioned SATA port.

The next decision was which OS to install. For a while I considered using one of the ready-made NAS systems like UnRaid, TrueNAS or OpenMediaVault. But I felt like those systems did more than what I needed and might get more in the way than they would help.

So instead I decided to roll my own. Since the boot device is not secured by a RAID, it could fail any time. When that happens I want to be able to easily restore it again.

So I decided to give NixOS a go. Thanks to it's declarative approach, I can easily restore the system to a new disk by simply applying the configuration again (of course I need to back that up).

I won't bore you with how to install NixOS. I roughly followed this video

But there are a few things you might wanna know when installing NixOS on a CWWK P6.

The default installer will not work. It took me a while and this thread to figure out that you need to disable the graphical output of the boot manager.

I built a Docker image to create an installer ISO that does that and also enables SSH for a headless installation.

The NVME slots are labelled 1 to 4 on the board, but the OS will order them exactly reverse. So your NVME in slot 4 will be nvme0n1 🤷‍♂️.

Here's how I configured my boot device' partitions:

                                     Disk: /dev/nvme2n1
                  Size: 465.76 GiB, 500107862016 bytes, 976773168 sectors
                Label: gpt, identifier: 3A802A61-347C-420A-9E50-F5AF9FFB507D

    Device                  Start          End       Sectors       Size Type
    /dev/nvme2n1p1           2048      2099199       2097152         1G EFI System
    /dev/nvme2n1p2        2099200     18876415      16777216         8G Linux swap
>>  /dev/nvme2n1p3       18876416    976773119     957896704     456.8G Linux filesystem    

I went conservative and used EXT4 for my root partition:

mkfs.ext4 -L NIXOS /dev/nvme2n1p3
mkswap -L SWAP /dev/nvme2n1p2
mkfs.fat -F32 -n BOOT /dev/nvme2n1p1

My complete NixOS configuration is available in a Github Repository. It should tell you all you need to know.

This is my first time using ZFS, so I am still learning. At first I followed this blog post. But later decided to not use the legacy mount option.

Basically there are two ways to mount datapools and its datasets:

1. Set the data pool's mount point to “legacy”, then mount each dataset via fstab, respectively NixOS' fileSystems configuration
2. Set the mountpoint as a pool property and let ZFS auto mount everything

Option 1 is the proper NixOS declarative way. However option 2 makes it much easier to add new datasets on a whim.

When switching to option 2 I had the issue that my mounts got lost after a nixos-rebuild switch but after doing a manual zfs mount -a and rebooting again the mounts were properly restored. Not sure what happened there.

So to create a zfs pool with the two disks, using mirroring use the following command:

zpool create -f -o ashift=12 -m /data data mirror /dev/disk/by-id/nvme-FIKWOT_FX991_4TB_AA251340954 /dev/disk/by-id/nvme-Fanxiang_S880E_4TB_FXS880E251530499

NVME IDs (as reported in lsblk -o NAME,SIZE,LABEL,MODEL,ID) are prefixed with nvme- in /dev/disk/by-id/.

ashift aligns things with internal physical sectors of the disk (which are not exactly known but most likely 4k or larger, not the 512b the drives report) - it's important to set it on pool creation as you can not change it later. I named my pool data and mounted it to /data.

I also disabled access time creation to reduce wear on the NVME disks:

zfs set atime=off data

Data sets can be created like this:

zfs create data/docker
zfs create -o mountpoint=/home data/home

Data sets inherit settings and mount points from the pool. So you only need to specify it when you want something different. In my case I mount the home dataset to /home instead of /data/home.

Have a look at my full nix configuration for more details.

Providing storage via the network is the main task of a NAS. I'm using Samba and the CIFS protocol for that. I need it for Kaddi's Windows system anyway and I found it to be good enough on Linux, so I don't bother with NFS anymore.

The setup itself is relatively simple using the Nix service definition. However initializing user passwords via agenix required a bit of trickery involving a postStart systemd service.

Details in my nix configuration.

As mentioned before, I am using docker to run most other services on the machine, independent from the Nix configuration. I am basically repeating what I did on my personal server.

Instead of including all docker compose files in a single top level file, I am trying Dockge as a lightweight manager to edit compose files and stop and start services.

My docker setup can be found in another Github Repository

Similar posts:

• NixOS Agenix (for secrets management)
• Postfix Relay to SMTP2GO on NixOS
• Some thoughts on NixOS