2026-10-08-ovmf-virtiofs-deadloop

2026-10-08 — QEMU 108 deadloop: virtiofs + OVMF under PVE 9

Post-upgrade boot failure: QEMU 108 wedged in OVMF with zero disk I/O after the PVE 8→9 upgrade. Resolved by deleting virtiofs0 — the service was then rebuilt as LXC 114 (bind-mount pattern) and the VM destroyed.

Symptom

After the upgrade reboot, VM 108 showed qmpstatus: running but was dead at the firmware stage:

• diskread: 0 / diskwrite: 0 — never touched the disk
• netin ~40 B, netout: 0; ~47 MiB of the 16 GiB touched
• qm terminal hung (serial0 socket, nothing emitted); noVNC console pure black; ~50% CPU on one vCPU (a firmware deadloop signature)
• qm guest-agent ping timeouts — the guest OS never loaded

Diagnosis path

Via diag-pipe read-only jobs on the host:

• qm monitor → info registers: vCPU frozen at a single RIP (0xbeb6e058, just below 3 GB — OVMF DXE-phase address space), identical across samples — a parked/deadlooping CPU, not a spin.
• info status/info vcpus: QEMU healthy, running, KVM fine.
• virtiofsd alive: both daemon processes running, vhost-user sockets present, /etc/pve/mapping/directory.cfg intact — the host-side backend was not the problem.
• devvm (112) unaffected on the same machine type (pc-i440fx-11.0+pve0) — but devvm is SeaBIOS while 108 is OVMF (bios: ovmf + efidisk). That asymmetry was the tell: OVMF ships a VirtioFs DXE driver that probes the vhost-user-fs device during firmware init; SeaBIOS has no such driver and never touches it.

Ruled out (each tested by config change + reboot)

• vga: qxl → std: still dead.
• machine: pc-i440fx-9.2 (pin to creation-era type): still dead.
• machine: q35: still dead.
• virtiofsd process health: alive throughout.

Root cause

OVMF’s VirtioFs DXE driver deadloops probing the virtiofs0 vhost-user device under QEMU 11.0.3 (PVE 9). The guest had no pinned machine: version, so it silently floated to the newest machine type on upgrade — but the wedge is the firmware/driver path, not the machine type (q35 failed too).

Fix: qm set 108 --delete virtiofs0 → booted immediately. Re-adding the device re-wedges it; there is no config-level fix, so virtiofs + OVMF is effectively unsupported on this host under PVE 9.

Resolution

Rebuilt the service as unprivileged LXC 114 — same LAN IP (10.0.1.28, caddy/DNS untouched), mp0 bind mount for the share (no virtiofs), /dev/net/tun passthrough + nesting=1 for mullvad-daemon (vpngw precedent), Debian 13. Fresh mullvad CLI setup — forced anti-censorship udp2tcp (the old VM was on auto, burning a doomed raw-WG attempt on every connect). New mullvad-watchdog installed. VM 108 destroyed after the CT proved out.

Lessons

• virtiofs + OVMF = boot deadloop on PVE 9. If any QEMU guest ever moves to OVMF, remove virtiofs devices first. devvm stays on SeaBIOS — don’t switch it without dropping virtiofs0.
• For QEMU guests that need the share under OVMF: alternatives are a guest-side CIFS mount or NFS re-export — not virtiofs.
• Diagnosis shortcut: zero diskread/diskwrite + frozen info registers RIP in the ~3 GB region = firmware-stage wedge; bisect hardware config (vga, machine, virtiofs), not guest state.

Follow-ups

• virtiofs0 deleted → VM booted (interim state)
• LXC 114 rebuilt, verified, backup pool joined (lxc-suspend)
• VM 108 destroyed (qm destroy --purge)
• Watchdog installed and verified on CT 114
• Docs updated — storage (virtiofs hazard warning), decisions