notes.neulen.net

bhyve

Mar 16, 20256 min read

Troubleshooting

VM gets stuck in grub during installation

Source: https://unix.stackexchange.com/questions/418401/grub-error-you-need-to-load-kernel-first

When your Linux VM gets stuck in grub during installation it can most probably only be booted using UEFI. Make sure to set loader="uefi" in the configuration file of the VM in “/vm/<vm_name>/<vm_name>.conf`.

VM boot error failed to load Boot

Source:

If an error like failed to load Boot is shown during the boot process it might be needed to select the correct file to boot with manually.

This can be done by switching into the Boot Maintenance Manager from the boot menu. In the Manager select Boot From File and navigate to the grubx64.efi file.

To fix this permanently do the following in the guest as root:

cd /boot/efi
mkdir -p EFI/BOOT
cp <path_to_efi_file>/grubx64.efi EFI/BOOT/bootx64.efi

Now reboot and check if it worked.

Cheatsheet

Automatically start VMs after boot

To automatically start bhyve VMs add the following enties into the /etc/rc.conf file:

vm_list="vm01 vm02"
vm_delay="5"

The entry vm_list defines which VMs should be started and the entry vm_delay defines the time in seconds that the system should wait between each VM start.

Install and configure vm-bhyve

Source:

Setting up a Linux VM on a headless FreeBSD server involves using FreeBSD’s built-in virtualization support, such as bhyve, along with some command-line tools for managing virtual machines. Here’s a high-level guide to get you started:

1. Install bhyve

First, make sure your FreeBSD server is set up for virtualization and that bhyve, the FreeBSD hypervisor, is installed. If it’s not already installed, you can install it via the package manager:

pkg install vm-bhyve grub2-bhyve bhyve-firmware

2. Configure vm-bhyve

After installing vm-bhyve, you’ll need to configure it. Start by adding following lines to the /etc/rc.conf and fell fre to edit the target directory for bhyve:

vm_enable="YES"
vm_dir="zfs:zroot/bhyve"
kld_list="vmm"

Now initialize vm-bhyve:

vm init

Then, configure the default network switch for VMs:

vm switch create public
vm switch add public em0 # Replace em0 with your network interface name

Copy over the default templates:

cp /usr/local/share/examples/vm-bhyve/* /zroot/bhyve/.templates

Setup Linux VM with bhyve

1. Download a Linux Distribution

Download a Linux ISO that you wish to install on your VM. You can do this by downloading the ISO to your FreeBSD server using fetch or curl. For example, to download Ubuntu Server:

vm iso <url_to_iso_file>

Debian iso download link

If you are searching for the latest Debian netinstall iso, take a look here.

2. Create and Configure the VM

Create a new VM using vm-bhyve. You’ll need to specify the size of the disk, the amount of memory, and the number of CPUs:

vm create -s 10G -m 2G -c 2 linuxvm

Or if you have a template or want to use one from the default ones use the following command. This command will create a VM called linuxvm with the debian template.

vm create -t debian linuxvm

Next, you’ll probably need to configure the VM a bit unless you had a predefined template. Edit the VM’s configuration file (located in /vm/linuxvm/linuxvm.conf by default) and adust everything like below.

Also make sure to enable graphics. This will cause the interface to be available via VNC.

loader="grub"  
cpu=1  
memory=3G  
network0_type="virtio-net"  
network0_switch="public"  
disk0_type="nvme"  
disk0_name="disk0.img"  
grub_run_partition="1"  
grub_run_dir="/boot/grub"  
uuid="f6f592e2-2b0d-11ef-89c8-0cc47a0a1218"  
network0_mac="58:9c:fc:0b:8d:7a"  
graphics="yes"

Warning

Keep in mind that not all Linux distributions - for instance Fedora - support the loader grub. In this case select uefi as the loader. Otherwise those Linux distros won’t be able to boot. (See VM gets stuck in grub during installation)

3. Start the VM and Install Linux

Start the VM and connect to its console:

vm install linuxvm <path_to_iso>
vm list

With the vm list you should see that the VM has started and is in locked or running state and has a VNC interface as well. You should be able to connect to that VNC interface using for instance this command on a remote machine:

vncviewer -SendClipboard -AcceptClipboard -LowColorLevel -QualityLevel 6 <ip_of_server>::<vnc_port> &

You can also jump into the console and use a command line installer using this command:

vm console linuxvm

4. Managing the VM

You can start, stop, and manage your VM with vm-bhyve commands:

vm start linuxvm
vm stop linuxvm
vm list

5. Accessing the VM

Since your server is headless, you’ll primarily interact with your Linux VM via SSH. Ensure the Linux VM is configured to start the SSH server automatically. You can then SSH into the VM using its IP address, which you can find from the FreeBSD host or within the VM itself.

This guide provides a basic overview. Depending on the specifics of your setup and the Linux distribution you’re installing, you might need to adjust some steps. Refer to the FreeBSD and vm-bhyve documentation for more detailed information.

Setup FreeBSD VM with bhyve

1. Download a FreeBSD bootonly install image

Download a FreeBSD bootonly install image from their website.

2. Create and Configure the VM

Create a new VM using vm-bhyve. It is abvisable to use the default template freebsd-zvol.conf for a FreeBSD guest.

vm create -t freebsd-zvol freebsdvm

3. Start the VM and Install FreeBSD

Start the VM and connect to its console - the option f will open the VMs console in the current terminal directly:

vm install -f freebsdvm <path_to_iso>

Setup qcow based VM with bhyve

Source:

To set up a qcow image based VM in bhyce first install qemu:

pkg install qemu

If all other vm-bhyve packages are already installed and configured as shown below, everything should be good to go. If the configuration of vm-bhyve needs to be done follow the chapter Install and configure vm-bhyve.

Download a qcow image using this command:

vm img <url_to_image_file>

Homeassistant qcow download link

If you are searching for the latest Homeassitant qcow image, take a look here.

When the configuration is finished the VM can be created.

vm create -t <template_name> -c <core_count> -m <ram_size> -s <disk_size> -i <image_name>.qcow2 <vm_name>

To run a Linux qcow image it is recommended to use the debian template and change the configuration file as follows:

loader="uefi"
graphics="yes"
cpu="4"
memory="4GB"
network0_type="virtio-net"
network0_switch="public"
disk0_type="nvme"
disk0_name="disk0.img"
uuid="<value>"
network0_mac="<value>"

Warning

Setting up a VM with a sparse-zvol based on a qemu image seems to be unsupported and will result in the error ERROR: failed to write img file with qemu-img.

PCIe Passthrough

Source:

Graph View

Backlinks