Skip to content
wordpress meta

title: 'Systemctl With Docker and ZFS'
date: '2020-05-20T18:54:21-05:00'
status: publish
permalink: /systemctl-with-docker-and-zfs
author: admin
excerpt: ''
type: post
id: 1653
category:
    - Docker
    - ZFS
tag: []
post_format: []

I previously wrote about Ubuntu 20.04 as a rpool(boot volume) on OCI (Oracle Cloud Infrastucture). If using a ZFS rpool you probably wont have this silly race condition I am writing about here.

So for this POC I was using Docker and an isci mounted disk for the Docker root folder. Unfortunately there are a couple issues. The first one not related to Docker just booting up and the zpool not being imported. Fix A is for that. The second issue is that Docker may not wait for the zpool to be ready before it starts and just automatically lay down its docker folder you specified in daemon.json. And of course then zfs will not mount even if it was imported with fix A.

Fix A

If you don't know yet please create your zpool with the by-id device name not for example /dev/sdb. If this zpool was already created you can fix this after the fact with export and import and updating the cache.

You can look at systemctl status zfs-import-cache.service to see what happened at boot with this zpool. There are many opinions on how to fix this; suffice to say this is what I used and it works reliably for me so far.

Create service

```bash

cat /etc/systemd/system/tank01-pool.service

[Unit] Description=Zpool start service After=dev-disk-by\x2did-wwn\x2d0x6081a22b818449d287b13b59a47bc407.device

[Service] Type=simple ExecStart=/usr/sbin/zpool import tank01 ExecStartPost=/usr/bin/logger "started ZFS pool tank01"

[Install] WantedBy=dev-disk-by\x2did-wwn\x2d0x6081a22b818449d287b13b59a47bc407.device

systemctl daemon-reload

systemctl enable tank01-pool.service

systemctl status tank01-pool.service

‚óŹ tank01-pool.service - Zpool start service Loaded: loaded (/etc/systemd/system/tank01-pool.service; enabled; vendor preset: enabled) Active: inactive (dead) since Tue 2020-05-19 02:18:05 UTC; 5min ago Main PID: 1018 (code=exited, status=0/SUCCESS)

May 19 02:18:01 usph-vmli-do01 systemd[1]: Starting Zpool start service... May 19 02:18:01 usph-vmli-do01 root[1019]: started ZFS pool tank01 May 19 02:18:01 usph-vmli-do01 systemd[1]: Started Zpool start service. May 19 02:18:05 usph-vmli-do01 systemd[1]: tank01-pool.service: Succeeded. ````

To find your exact device

```bash

systemctl list-units --all --full | grep disk | grep tank01

  dev-disk-by\x2did-scsi\x2d36081a22b818449d287b13b59a47bc407\x2dpart1.device                                                                                       loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2did-wwn\x2d0x6081a22b818449d287b13b59a47bc407\x2dpart1.device                                                                                       loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2dlabel-tank01.device                                                                                                                                loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2dpartlabel-zfs\x2d9eb05ecca4da97f6.device                                                                                                           loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2dpartuuid-d7d69ee0\x2d4e45\x2d3148\x2daa7a\x2d7cf375782813.device                                                                                   loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2dpath-ip\x2d169.254.2.2:3260\x2discsi\x2diqn.2015\x2d12.com.oracleiaas:16bca793\x2dc861\x2d49e8\x2da903\x2dd6b3809fe694\x2dlun\x2d1\x2dpart1.device loaded    active   plugged   BlockVolume tank01                                                           
  dev-disk-by\x2duuid-9554707573611221628.device                                                                                                                    loaded    active   plugged   BlockVolume tank01

ls -l /dev/disk/by-id/ | grep sdb

lrwxrwxrwx 1 root root  9 May 18 22:32 scsi-36081a22b818449d287b13b59a47bc407 -> ../../sdb
lrwxrwxrwx 1 root root 10 May 18 22:32 scsi-36081a22b818449d287b13b59a47bc407-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 May 18 22:33 scsi-36081a22b818449d287b13b59a47bc407-part9 -> ../../sdb9
lrwxrwxrwx 1 root root  9 May 18 22:32 wwn-0x6081a22b818449d287b13b59a47bc407 -> ../../sdb
lrwxrwxrwx 1 root root 10 May 18 22:32 wwn-0x6081a22b818449d287b13b59a47bc407-part1 -> ../../sdb1
lrwxrwxrwx 1 root root 10 May 18 22:33 wwn-0x6081a22b818449d287b13b59a47bc407-part9 -> ../../sdb9

````

Fix B

This was done before and just showing for reference how you enable the docker zfs storage.

```bash

cat /etc/docker/daemon.json

{ "storage-driver": "zfs", "data-root": "/tank01/docker" } ````

For the timing issue you have many options in systemctl and probably better than this. For me just delaying a little until isci and zpool import/mount is done works OK.

```bash

grep sleep /etc/systemd/system/multi-user.target.wants/docker.service

ExecStartPre=/bin/sleep 60

systemctl daemon-reload

````