Topic 05: Time-Sliced vCPU Scheduling & Broker Protocol

🎙️ Deep-Dive Spoken Script

Interviewer: “How did you implement scheduling between the two VMs in userspace?”

In the second part of the project, I also implemented scheduling between the two VMs in userspace.

The basic idea was that I had a scheduling trace containing a sequence of VM IDs, where 1 represents the Producer VM and 2 represents the Consumer VM. The hypervisor reads this sequence and uses it to decide which VM should run next.

For each step, the hypervisor selects the corresponding VM and vCPU and calls KVM_RUN on that vCPU:

int next_vm = sched_trace[step];
if (next_vm == 1) {
    ioctl(vm1->vcpu_fd, KVM_RUN, 0);
    handle_exit(vm1);
} else {
    ioctl(vm2->vcpu_fd, KVM_RUN, 0);
    handle_exit(vm2);
}

The guest then starts or continues executing until it produces an exit, such as an I/O exit.

When KVM_RUN returns, the hypervisor handles that exit. For example, if the Producer has generated new data, the hypervisor synchronizes the buffer with the Consumer. It then moves to the next VM specified by the scheduling trace.

Because KVM maintains the register state of each vCPU, I don't have to manually save and restore all the guest registers when switching between the two VMs. I simply run the appropriate vCPU through its vCPU file descriptor, and KVM resumes that vCPU with its saved state.

I also used a delay between scheduling steps to make the execution sequence observable during the experiment.

So overall, the userspace hypervisor is doing two things together: it decides which vCPU runs next, and it acts as the broker that handles VM exits and synchronizes data between the two VMs.

🔬 In-Depth Q&A Database

Q14: How does userspace time-sliced vCPU scheduling work in your hypervisor?

In commercial hypervisors like QEMU/KVM, each vCPU typically runs inside an independent host POSIX thread (pthread_create), leaving time-slicing to the host Linux CFS scheduler.

In our specialized hypervisor, scheduling is implemented deterministically in userspace using a single thread. The hypervisor reads an interleaved scheduling trace (e.g. 1 1 2 1 2 2), issuing KVM_RUN calls sequentially. This allows precise observation of race conditions, buffer full/empty states, and synchronization timing.

Q15: How does this custom hypervisor compare to production hypervisors like QEMU?

• QEMU: Emulates complete motherboard chipsets (PCI bus, IDE/NVMe, ACPI, virtio devices, APIC) across millions of lines of code.
• Our Hypervisor: A minimalist MicroVMM focused purely on core virtualization primitives—opening /dev/kvm, allocating memory via mmap, trapping I/O port exits, and orchestrating vCPU time-slices directly.

Q16: How do your systems projects (Hypervisor, Container Runtime, LKM, RustOS) fit together?

They explore compute abstraction and isolation at every layer of the modern systems hierarchy:

• RustOS (Silicon to Ring 0): Building an OS from scratch on bare-metal x86_64, writing GDT, IDT, 4-level paging, and timer preemptive context switching.
• Linux Kernel Modules (Ring 0 Extensibility): Extending a production monolithic kernel dynamically, walking 5-level hardware MMU page tables, implementing character device drivers, and manipulating kernel process trees.
• Bare-Metal KVM Hypervisor (Hardware-Assisted Virtualization): Utilizing hardware virtualization extensions (VT-x) to run isolated guest kernels in Ring 0 under userspace supervision.
• Conductor Container Runtime (OS-Level Virtualization): Isolating standard userspace processes on a single shared kernel using Linux namespaces, cgroups v2, and OverlayFS without hypervisor overhead.

⚡ Syntax & Whiteboard Quick-Lookup Sheet