tips:ubuntu:zfs-on-root [Galileo Labs]

This page is read only. You can view the source, but not change it. Ask your administrator if you think this is wrong.
====== Ubuntu on ZFS root ======

{{tag>[zfs ubuntu]}}

  * https://github.com/zfsonlinux/zfs/wiki/Ubuntu-16.10-Root-on-ZFS

===== Livecd =====

boot live cd and 
<code bash>
sudo su
passwd ubuntu
apt-add-repository universe
apt update
apt install openssh-server
# connect from another PC via ssh
</code>

<code>
apt install --yes debootstrap gdisk zfs-initramfs
</code>

===== Disk =====

Reset disk and create GPT partition schema
<code>
DISK=/dev/disk/by-id/ata-HFS512G39MND-3510A_FJ64N5235113A4S13
sgdisk --clear -g $DISK
</code>

Prepare partitions
<code>
# Run this if you need legacy (BIOS) booting:
sgdisk -a1 -n2:34:2047  -t2:EF02 $DISK

# Run this for UEFI booting or raidz pool:
sgdisk     -n3:1M:+512M -t3:EF00 $DISK
parted $DISK set 3 bios_grub on # needed for embedding grub in this partition 

# and after run these:
sgdisk     -n9:-8M:0    -t9:BF07 $DISK
sgdisk     -n1:0:0      -t1:BF01 $DISK
</code>

Create pool or ...
<code bash>
zpool create -o ashift=12 \
      -O atime=off -O canmount=off -O compression=lz4 -O normalization=formD \
      -O mountpoint=/ -R /mnt -f \
      rpool ${DISK}-part1
</code>

... import existing pool
<code bash>
zpool export rpool
zpool import -R /mnt rpool
... umount everything inside /mnt and remove subfolders
zfs mount rpool/ROOT/ubuntu
</code>

or create raidz pool
<code bash>
zpool create -O atime=off -O mountpoint=/ -R /mnt -O canmount=off -O compression=lz4 -O normalization=formD -o ashift=12 \
  rpool raidz /dev/disk/by-id/ata-ST4000NM0035-1V4107_ZC10????-part1
</code>
===== System install =====

create filesystem dataset to act as container
<code>
zfs create -o canmount=off -o mountpoint=none rpool/ROOT
</code>

create filesystem for root
<code>
zfs create -o canmount=noauto -o mountpoint=/ rpool/ROOT/ubuntu
zfs mount rpool/ROOT/ubuntu
</code>

create other filesystems
<code bash>
zfs create -o setuid=off rpool/home

#zfs create -o canmount=off -o setuid=off  -o exec=off rpool/var
#zfs create -o com.sun:auto-snapshot=false -o exec=on  rpool/var/tmp


If this system will use NFS (locking):
# zfs create -o com.sun:auto-snapshot=false \
             -o mountpoint=/var/lib/nfs                 rpool/var/nfs

</code>

deboostrap
<code>
DISTRO=xenial
mount -o remount,dev /mnt
debootstrap $DISTRO /mnt

zfs set devices=off rpool
mount --rbind /dev  /mnt/dev
mount --rbind /proc /mnt/proc
mount --rbind /sys  /mnt/sys
chroot /mnt /bin/bash --login
</code>

customization
<code bash>
echo 'LANG="en_US.UTF-8"' > /etc/default/locale
dpkg-reconfigure tzdata

dpkg-reconfigure keyboard-configuration

locale-gen it_IT.UTF-8
locale-gen en_US.UTF-8
update-locale LANG=it_IT.UTF-8 LC_MESSAGES=POSIX

DISTRO=xenial
cat > /etc/apt/sources.list <<EOF
deb http://archive.ubuntu.com/ubuntu $DISTRO main universe
#deb-src http://archive.ubuntu.com/ubuntu $DISTRO main universe

deb http://security.ubuntu.com/ubuntu $DISTRO-security main universe
#deb-src http://security.ubuntu.com/ubuntu $DISTRO-security main universe

deb http://archive.ubuntu.com/ubuntu $DISTRO-updates main universe
#deb-src http://archive.ubuntu.com/ubuntu $DISTRO-updates main universe
EOF

ln -s /proc/self/mounts /etc/mtab
apt update

apt install --yes --no-install-recommends linux-image-generic zfs-initramfs

cat > /etc/fstab <<EOF
rpool/ROOT/ubuntu / zfs defaults,noatime 0 0
EOF

</code>

==== GRUB ====

<code bash>
export DISK=/dev/disk/by-id/ata-HFS512G39MND-3510A_FJ64N5235113A4S13
</code>

=== UEFI ===

<code bash>
apt install dosfstools
mkdosfs -F 32 -n EFI ${DISK}-part3
mkdir /boot/efi
echo PARTUUID=$(blkid -s PARTUUID -o value ${DISK}-part3) /boot/efi vfat defaults 0 1 >> /etc/fstab
      
mount /boot/efi
apt install --yes grub-efi-amd64

</code>      

check
<code>
grub-probe /

---> zfs
</code>

if error occur upgrade grub to 2.02-beta3 from https://repo.morph027.de/grub/pool/main/g/grub2/

<code>
grub-install --target=x86_64-efi --efi-directory=/boot/efi \
      --bootloader-id=ubuntu --recheck --no-floppy
</code>

verify if "ls /boot/grub/*/zfs.mod"

If you are creating a mirror, repeat the grub-install command for each disk in the pool.


=== legacy MBR ===

<code>
apt install --yes grub-pc

</code>

<code>
grub-install ${DISK}

</code>

=== manual grub menu ===

<file txt /etc/grub.d/40_custom>
#!/bin/sh
exec tail -n +3 $0
# This file provides an easy way to add custom menu entries.  Simply type the
# menu entries you want to add after this comment.  Be careful not to change
# the 'exec tail' line above.

menuentry 'Ubuntu artful 17.10' {
	recordfail
	load_video
	gfxmode $linux_gfx_mode
	insmod gzio
	insmod part_gpt
	insmod zfs
	set root='hd1,gpt0'
	search --no-floppy --fs-uuid --set=root 3f1e7de17907507c
        linux	/ROOT/artful@/boot/vmlinuz-4.13.0-16-generic root=ZFS=rpool/ROOT/artful ro 
	initrd	/ROOT/artful@/boot/initrd.img-4.13.0-16-generic
}
</file>


=== update grub ===

<code>
update-initramfs -c -k all
update-grub
</code>


==== exit ====

**disable root password** (re-enabled in phase two below)
<code bash>
passwd --delete root
</code>

<code>
zfs snapshot rpool/ROOT/ubuntu@install
exit

mount | grep -v zfs | tac | awk '/\/mnt/ {print $3}' | xargs -i{} umount -lf {}
zpool export rpool
reboot
</code>

===== Complete system =====

create home filesystem
<code>
zfs create -o setuid=off rpool/home
</code>

add swap
<code>
zfs create -V 4G -b $(getconf PAGESIZE) -o compression=zle \
      -o logbias=throughput -o sync=always \
      -o primarycache=metadata -o secondarycache=none \
      -o com.sun:auto-snapshot=false rpool/swap
mkswap -f /dev/zvol/rpool/swap
echo /dev/zvol/rpool/swap none swap defaults 0 0 >> /etc/fstab
swapon -av
</code>

enable power button decomment in /etc/systemd/logind.conf
<code>
HandlePowerKey=poweroff
</code>

add user
<code>
useradd -m scipio
usermod -a -G adm,cdrom,dip,plugdev,sudo,video scipio
passwd scipio
</code>

ubuntu standard
<code bash>
apt install --yes ubuntu-standard
</code>

ubuntu desktop
<code bash>
apt-get install -y software-properties-common
add-apt-repository ppa:gnome3-team/gnome3-staging
sudo apt install -y ubuntu-gnome-desktop
#apt install --yes ubuntu-desktop
apt-get -y install $(check-language-support -l it)

# HWE support for XENIAL
sudo apt-get install --install-recommends linux-generic-hwe-16.04 xserver-xorg-hwe-16.04
</code>

enable network manager on all interfaces (see [[https://bugs.launchpad.net/ubuntu/+source/network-manager/+bug/1638842|this]]
<code>
touch /etc/NetworkManager/conf.d/10-globally-managed-devices.conf
</code>

disable compression on logs
<code>
for file in /etc/logrotate.d/* ; do
    if grep -Eq "(^|[^#y])compress" "$file" ; then
        sed -i -r "s/(^|[^#y])(compress)/\1#\2/" "$file"
    fi
done
</code>

after reboot destroy ubuntu@install snapshot
<code bash>
zfs destroy rpool/ROOT/ubuntu@install
</code>

disable root password
<code bash>
sudo usermod -p '*' root
</code>