Development setup

In order to test your setup, one of the most flexible way is to use a complete distribution exported as nfs, and used as a rootfs by the kernel.

Basically, the setup is:

  • libvirt to launch a vm (and handle all the network part)
  • the test vm will direct boot into the kernel
  • distribution will be installed into a directory on the dev box
  • nfs server with the filesystem exported for the vm

Let’s go step by step:

  • Create a sample vm:
    • use virt-manager to create a vm with no disk, and setup direct kernel boot with your kernel

    • it is simpler to setup the vm the first time with virt-manager, but after that, you should directly edit the vm file with virsh edit $domain

    • command line parameters: console=ttyS0

    • you can launch/restart your vm with virsh (relevant commands are edit, start, destroy and console):

      $ virsh -c qemu:///system start $vmname
  • create the base system with pacstrap or debootstrap:

    $ sudo pacstrap -cd $rootfsdir base base-devel git

  • add an export for the nfs server (you should use /etc/exports.d/ directory for that) for example:

    $ cat /etc/exports.d/$name.exports

    The network specification is here for the default libvirt network. To apply changes, run exportfs -ra or restart the NFS server.

  • Finally, you need to change the kernel command line to use your nfs as a root file system (~linux/Documentation/filesystems/nfs/nfsroot.txt for all the details), basically, you need something like this:

    root=/dev/nfs nfsroot=$rootfsdir,vers=3 rw console=ttyS0 ip=dhcp

Explanation are for debian/ubuntu, for Archlinux, it’ll come later.

creating the base distribution using debootstrap

There is examples at the end of the man page of debootstrap.

$ sudo apt install debootstrap $ sudo debootstrap jessie $ROOTFS $ sudo chroot $ROOTFS passwd -d root

kernel headers & modules installation

You’ll probably need to adjust permissions

$ sudo mkdir -p $ROOTFS/lib/modules $ sudo chown $USER $ROOTFS/usr/include/ $ROOTFS/lib/modules

You need to install your modules and kernel headers into the target root file system.

$ make -C $KSRC INSTALL_HDR_PATH=$ROOTFS/usr headers_install $ make -C $KSRC INSTALL_MOD_PATH=$ROOTFS modules_install

libvirt configuration & installation

$ apt install libvirt-bin virtinst

In order to have virsh working for qemu:///system by default

  • you need to change /etc/libvirt/libvirt.conf and un comment or write uri_default = “qemu:///system”
  • add your user to the libvirt group (you can use gpasswd for that)

You can now create a vm that can use your kernel (YMMV with the version of virt-install)

$ virt-install
–name $vmname –memory 512 –os-variant virtio26 –graphics none –disk none –boot kernel=$KERNEL,kernel_args=”console=ttyS0”

setup NFS

Of course, you’ll need to adapt the network with your network prefix for the nfs export.

$ sudo apt install nfs-kernel-server $ sudo mkdir -p /etc/exports.d $ cat <<EOF | sudo tee /etc/exports.d/$vmname.exports $ROOTFS,no_root_squash,no_subtree_check) EOF $ sudo exportfs -rav $ sudo systemctl restart nfs-kernel-server

You should test that you can mount the nfs export in order to be sure that your server is correctly setup. tcpdump/wireshark can help debugging it.

See also:

  • exports(5)

nfs root

Now you only need to change the kernel command line for your nfs server.

For example:

root=/dev/nfs nfsroot=$nfs_server_ip:$ROOTFS rw ip=dhcp

See also:

  • ~linux/Documentation/filesystems/nfs/nfsroot.txt