Setting up Gentoo Linux on UTM on Apple Silicon
These are my notes on setting up a ‘headless’ Gentoo Linux installation on UTM on my Apple Silicon Mac, with the Apple Virtualization ‘backend’.
I followed the Gentoo handbook.
I’ve never used Gentoo before and I’ve come away pretty impressed. There’s something about having the cruft that other distros imposed stripped away, and doing all the setup yourself, that somehow makes things seem easier to me because I ended up understanding things a lot more. And the installed system ‘feels’ pared down and efficient.
There is not yet an official handbook for arm64 (“The arm and arm64 architectures are supported by the Gentoo project but do not yet have Handbooks at their disposal due to too many variations in SoCs.”), but I found that the steps - for UTM and Apple Virtualization at least - are basically exactly the same as the ones in the AMD64 (64-bit Intel) handbook.
This is a condensed transcript mostly for myself. If you choose to follow this, and need to know more, check out the handbook!
Without further ado, my notes:
Setting up Gentoo arm64:
Download arm64 “Minimal Installation CD” from https://www.gentoo.org/downloads/
UTM: File -> New, “Virtualize”, “Linux”, “Use Apple Virtualization”
Select the Gentoo “Minimal Installation CD” as the boot iso. Set memory and leave CPU cores at default. Specify a hard drive size.
Boot.
Set up root password, user, and turn on SSH - this is so that we can use the macOS terminal instead of the virtual console to control installation (so e.g. copy/paste works).
livecd ~ # passwd
livecd ~ # rc-service sshd start
livecd ~ # ip address
Now, minimize the UTM window and SSH in in a regular Mac terminal:
jamie@Jamies-MacBook-Air ~> ssh root@192.168.64.8
Work out what the hard drive is:
livecd ~ # lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
loop0 7:0 0 446.2M 1 loop /mnt/livecd
sda 8:0 0 692.4M 1 disk /mnt/cdrom
├─sda1 8:1 0 104K 1 part
├─sda2 8:2 0 2.8M 1 part
└─sda3 8:3 0 689.4M 1 part
vda 252:0 0 50G 0 disk
Format the drive:
livecd ~ # fdisk /dev/vda
Create a GPT:
Command (m for help): g
Created a new GPT disklabel (GUID: 1A47D934-85E8-594A-BF02-1F39C5CE6B2F).
Set up partitions. No swap (for now?), so just an EFI boot partition and a Linux partition:
Command (m for help): p
Disk /dev/vda: 50 GiB, 53687091200 bytes, 104857600 sectors
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: 1A47D934-85E8-594A-BF02-1F39C5CE6B2F
Command (m for help): n
Partition number (1-128, default 1): 1
First sector (2048-104857566, default 2048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (2048-104857566, default 104855551): +500M
Created a new partition 1 of type 'Linux filesystem' and of size 500 MiB.
Command (m for help): t
Selected partition 1
Partition type or alias (type L to list all): 1
Changed type of partition 'Linux filesystem' to 'EFI System'.
Command (m for help): n
Partition number (2-128, default 2): 2
First sector (1026048-104857566, default 1026048):
Last sector, +/-sectors or +/-size{K,M,G,T,P} (1026048-104857566, default 104855551): 104857566
Created a new partition 2 of type 'Linux filesystem' and of size 49.5 GiB.
Command (m for help): w
The partition table has been altered.
Calling ioctl() to re-read partition table.
Syncing disks.
Set up the filesystems:
livecd ~ # mkfs.vfat -F 32 /dev/vda1
mkfs.fat 4.2 (2021-01-31)
livecd ~ # mkfs.xfs /dev/vda2
meta-data=/dev/vda2 isize=512 agcount=4, agsize=3244735 blks
= sectsz=512 attr=2, projid32bit=1
= crc=1 finobt=1, sparse=1, rmapbt=0
= reflink=1 bigtime=1 inobtcount=1 nrext64=0
data = bsize=4096 blocks=12978939, imaxpct=25
= sunit=0 swidth=0 blks
naming =version 2 bsize=4096 ascii-ci=0, ftype=1
log =internal log bsize=4096 blocks=16384, version=2
= sectsz=512 sunit=0 blks, lazy-count=1
realtime =none extsz=4096 blocks=0, rtextents=0
Discarding blocks...Done.
Mount the new partitions:
livecd ~ # mount /dev/vda2 /mnt/gentoo
livecd / # mkdir /gentoo/efi
livecd / # mount /dev/vda1 /gentoo/efi
Check the date, handbook says it must be correct (don’t worry about timezone):
livecd ~ # date
Mon Oct 30 18:25:27 UTC 2023
Download the Stage 3 tarball for arm64 from https://www.gentoo.org/downloads/ using wget:
livecd ~ # cd /mnt/gentoo
livecd /mnt/gentoo # wget <TARBALL URL>
Untar the stage3 tarball:
livecd /mnt/gentoo # tar xpvf stage3-*.tar.xz --xattrs-include='*.*' --numeric-owner
Set compile options:
livecd /mnt/gentoo # vi /mnt/gentoo/etc/portage/make.conf
I set:
COMMON_FLAGS="-march=native -Os -pipe"
ACCEPT_LICENSE="*"
[No need to set MAKEOPTS - Portage now defaults to setting the number of threads to the value returned by nproc.]
Select mirrors:
livecd ~ # mirrorselect -6 -i -o >> /mnt/gentoo/etc/portage/make.conf
Set up DNS for chroot (this is necessary because the DHCP was done in the non-chroot environment)
livecd ~ # cp --dereference /etc/resolv.conf /mnt/gentoo/etc/
Chroot to new partition:
livecd ~ # mount --types proc /proc /mnt/gentoo/proc
livecd ~ # mount --rbind /sys /mnt/gentoo/sys
livecd ~ # mount --make-rslave /mnt/gentoo/sys
livecd ~ # mount --rbind /dev /mnt/gentoo/dev
livecd ~ # mount --make-rslave /mnt/gentoo/dev
livecd ~ # mount --bind /run /mnt/gentoo/run
livecd ~ # mount --make-slave /mnt/gentoo/run
livecd ~ # chroot /mnt/gentoo /bin/bash
livecd / # source /etc/profile
livecd / # export PS1="(chroot) ${PS1}"
Sync to latest ebuild repo:
(chroot) livecd / # emerge-webrsync
...
(chroot) livecd / # emerge --sync
Read news (or, at least, mark it as read…):
(chroot) livecd / # eselect news read
Select profile (or not, the default one was what I wanted):
(chroot) livecd / # eselect profile list
...
(chroot) livecd / # eselect profile <number>
Updating the @world set. Handbook says this is necessary.
(chroot) livecd / # emerge --ask --verbose --update --deep --newuse @world
I can’t live without VIM…
(chroot) livecd / # emerge --ask --verbose vim
Configure USE variable? I didn’t see a need to change anything.
Set up CPU flags:
(chroot) livecd / # emerge --ask app-portage/cpuid2cpuflags
(chroot) livecd / # cpuid2cpuflags
(chroot) livecd / # echo "*/* $(cpuid2cpuflags)" > /etc/portage/package.use/00cpu-flags
Set up timezone (find zone in e.g. ls /usr/share/zoneinfo/)
(chroot) livecd / # echo "America/Los_Angeles" > /etc/timezone
(chroot) livecd / # emerge --config sys-libs/timezone-data
Configure locales:
(chroot) livecd / # vim /etc/locale.gen
Uncomment:
en_US ISO-8859-1
en_US.UTF-8 UTF-8
(chroot) livecd / # locale-gen
(chroot) livecd / # env-update && source /etc/profile && export PS1="(chroot) ${PS1}"
Kernel configuration:
Get extra firmware (probably not necessary?)
(chroot) livecd / # emerge --ask sys-kernel/linux-firmware
Set up kernel:
(chroot) livecd / # emerge --ask sys-kernel/installkernel-gentoo
(chroot) livecd / # emerge --ask sys-kernel/gentoo-kernel
Set up fstab:
(chroot) livecd / # blkid
(chroot) livecd / # vim /etc/fstab
UUID="41F9-7C9A" /efi vfat defaults 0 2
UUID="95d0078d-00f9-4b14-b03e-64a9742bf609" / xfs defaults,noatime 0 1
Networking:
(chroot) livecd / # echo "gentoo-arm64" > /etc/hostname
(chroot) livecd / # emerge --ask net-misc/dhcpcd
(chroot) livecd / # rc-update add dhcpcd default
(chroot) livecd / # emerge --ask --noreplace net-misc/netifrc
Cron:
(chroot) livecd / # emerge --ask sys-process/cronie
(chroot) livecd / # rc-update add cronie default
Syslog (Note: syslog-ng seems to put logs in /var/log/messages)
(chroot) livecd / # emerge --ask app-admin/syslog-ng
(chroot) livecd / # rc-update add syslog-ng default
(chroot) livecd / # emerge --ask app-admin/logrotate
SSH:
(chroot) livecd / # rc-update add sshd default
Bash completion:
(chroot) livecd / # emerge --ask app-shells/bash-completion
Timekeeping. Skipped this - won’t the virtual host provide good time?
root #emerge --ask net-misc/chrony
root #rc-update add chronyd default
FS tools:
(chroot) livecd / # emerge --ask sys-fs/xfsprogs sys-fs/dosfstools
This not available on arm64:
(chroot) livecd / # emerge --ask sys-block/io-scheduler-udev-rules
Bootloader:
(chroot) livecd / # echo 'GRUB_PLATFORMS="efi-64"' >> /etc/portage/make.conf
(chroot) livecd / # emerge --ask --verbose sys-boot/grub
(chroot) livecd / # grub-install --target=arm64-efi --efi-directory=/efi
(chroot) livecd / # grub-mkconfig -o /boot/grub/grub.cfg
Set up a user:
(chroot) livecd / # useradd -m -G users,wheel,audio -s /bin/bash jamie
(chroot) livecd / # passwd jamie
SET THE ROOT PASSWORD
(chroot) livecd / # passwd
OR, set up sudo:
(chroot) livecd / # emerge --ask --verbose app-admin/sudo
(chroot) livecd / # vi /etc/sudoers
Uncomment line that starts %wheel
Done!
(chroot) livecd / # exit
livecd / # umount -l /mnt/gentoo/dev{/shm,/pts,}
livecd / # umount -R /mnt/gentoo
livecd / # shutdown -h now
Remove the live CD from the virtual machine and boot.
Remove old live-CD key for the IP address from
jamie@Jamies-MacBook-Air ~> vim /Users/jamie/.ssh/known_hosts
SSH in to new virtual machine:
Set up Avahi (zeroconf/bonjour) - this will allow you to e.g. ssh gentoo-arm64.local from the Mac terminal. Note: this installs a load of stuff because nothing has yet used xml2man, which Avahi uses for its docs.
jamie@gentoo-arm64 ~ $ sudo emerge --ask net-dns/avahi
jamie@gentoo-arm64 ~ $ sudo rc-update add avahi-daemon default
Other notes:
The virtual machine seems always to boot form the virtual hard drive, even if the virtual CD is available. To reboot from the CD if you need to rescue something:
In GRUB command line
set root=(hd0,gpt2)
chainloader /efi/bootaa64.efi
boot
Paths may be wrong, but Grub has an ’ls’ that you can look at things in:
ls (hd0,gpt2)/efi
etc.
After boot, device names may be different - check with:
livecd ~ # blkid
Setting up DistCC
One goal for me is to use the arm64 machine as a compile server for x86 machines.
We need the latest distcc, with a fix for https://github.com/distcc/distcc/pull/427, which is in ’testing’ and not yet released to stable gentoo.
jamie@gentoo-arm64 ~ $ echo "sys-devel/distcc ~arm64" > /etc/portage/package.accept_keywords/distcc
Build with crossdev support:
jamie@gentoo-arm64 ~ $ echo "sys-devel/distcc crossdev" > "/etc/portage/package.use/sys-devel:distcc"
jamie@gentoo-arm64 ~ $ sudo emerge --ask distcc
jamie@gentoo-arm64 ~ $ sudo emerge --ask crossdev
As root:
gentoo-arm64 /home/jamie # mkdir -p /var/db/repos/crossdev/{profiles,metadata}
gentoo-arm64 /home/jamie # echo 'crossdev' > /var/db/repos/crossdev/profiles/repo_name
gentoo-arm64 /home/jamie # echo 'masters = gentoo' > /var/db/repos/crossdev/metadata/layout.conf
gentoo-arm64 /home/jamie # chown -R portage:portage /var/db/repos/crossdev
gentoo-arm64 /home/jamie # vim /var/db/repos/crossdev/metadata/layout.conf
add "thin-manifests=true"
gentoo-arm64 /home/jamie # mkdir /etc/portage/repos.conf
gentoo-arm64 /home/jamie # vim /etc/portage/repos.conf/crossdev.conf
[crossdev]
location = /var/db/repos/crossdev
priority = 10
masters = gentoo
auto-sync = no
jamie@gentoo-arm64 ~ $ sudo crossdev --stable -t i486-pc-linux-gnu
jamie@gentoo-arm64 ~ $ sudo vim /etc/conf.d/distccd
DISTCCD_OPTS="${DISTCCD_OPTS} --log-level info --log-file /var/log/distccd.log"
DISTCCD_OPTS="${DISTCCD_OPTS} --allow 192.168.0.0/16"
jamie@gentoo-arm64 ~ $ sudo touch /var/log/distccd.log
jamie@gentoo-arm64 ~ $ sudo chown distcc:root /var/log/distccd.log
jamie@gentoo-arm64 ~ $ sudo rc-update add distccd default
That’s is - despite many web pages detailing insane complexity of using distcc with crossdev, nowadays it ‘just works’ after installation.