6 min read #linux-kernel #kvm-and-virtualization
On this page

KVM Architecture

Coverage: KVM as Linux hypervisor → VMX/SVM hardware virtualization → vCPU lifecycle → VM exit handling → KVM + QEMU collaboration model → /dev/kvm ioctl API Kernel version: 2.6.20 ~ 6.x

Overview

KVM (Kernel-based Virtual Machine) transforms the Linux kernel into a type-1 hypervisor—leveraging Intel VMX or AMD SVM hardware virtualization extensions to allow guests to run directly on physical CPUs in VMX non-root mode. Each VM is a QEMU process in user space, and each vCPU is a thread within that process.

Architecture Layers

KVM Architecture Layers: QEMU User Space · Kernel Space KVM QEMU (User Space) Device Emulation virtio / VGA / Storage / Network VM Management live migration / snapshot KVM ioctl /dev/kvm ioctl system call crosses user space → kernel space Kernel (KVM) kvm.ko Architecture-independent core kvm-intel.ko VMX support kvm-amd.ko SVM support QEMU communicates with the kernel only via /dev/kvm ioctls, without directly touching virtualization registers; kvm.ko is the architecture-independent core, while kvm-intel.ko / kvm-amd.ko interface with VMX / SVM respectively.

VM Entry / VM Exit

VM Entry / VM Exit: The loop of guest execution and kernel exit handling Guest Execution VMLAUNCH (VMX) / VMRUN (SVM) → Enters VMX non-root mode Guest executes directly on physical CPU (until a VM exit is triggered) VM Exit (root mode) Guest executes sensitive instructions (HLT / IO / EPT violation …) KVM reads exit reason and dispatches handling: IO_INSTRUCTION → Emulate IO instruction EPT_VIOLATION → Handle guest page table changes CPUID → Emulate CPUID HLT → Idle injection MSR_READ/WRITE → Emulate MSR access After handling → VMRESUME back to guest Sensitive instruction triggers → Exit to root After handling → VMRESUME back to guest VM Exit is the core loop of KVM: guest attempts sensitive operation → traps to kernel → KVM software emulation → VMRESUME resumes; Reading the exit reason and dispatching by type is the main source of overhead in guest/host switching in virtualization.

vCPU Thread Model

// Each vCPU = one QEMU thread, executing a loop:
while (running) {
    ioctl(vcpu_fd, KVM_RUN, 0);
    // Returns: VM exit
    switch (vcpu->kvm_run->exit_reason) {
    case KVM_EXIT_IO:       handle_io(vcpu);       break;
    case KVM_EXIT_MMIO:     handle_mmio(vcpu);     break;
    case KVM_EXIT_IRQ_WINDOW_OPEN: inject_interrupt(vcpu); break;
    // ...
    }
}

Scheduler Interaction

// KVM knows when a vCPU is preempted via preempt notifier:
// → kvm_sched_out(): vCPU is scheduled out
// → kvm_sched_in():  vCPU is scheduled in
// Used for: steal time accounting, TSC offset calculation

/dev/kvm Interface

// Create VM:
int kvm_fd = open("/dev/kvm", O_RDWR);
int vm_fd = ioctl(kvm_fd, KVM_CREATE_VM, 0);

// Register memory (guest physical address space):
struct kvm_userspace_memory_region region = {
    .slot = 0, .guest_phys_addr = 0, .memory_size = 1ULL << 30,
    .userspace_addr = (__u64)guest_ram_ptr, .flags = 0 };
ioctl(vm_fd, KVM_SET_USER_MEMORY_REGION, &region);

// Create vCPU:
int vcpu_fd = ioctl(vm_fd, KVM_CREATE_VCPU, cpu_id);

// Get KVM version/capabilities:
int version = ioctl(kvm_fd, KVM_GET_API_VERSION, 0);
struct kvm_cpuid2 cpuid = { .nent = N }; 
ioctl(kvm_fd, KVM_GET_SUPPORTED_CPUID, &cpuid);

Interrupt Injection

// KVM injects interrupts to the guest:
//   1. Set interrupt info fields in VMCS (VMX) or VMCB (SVM)
//   2. Set interrupt window → guest responds immediately after VM entry
//   3. If vCPU is not running (scheduled out) → set kick flags

// Interrupt sources:
//   - virtio devices (msix)
//   - Emulated devices (8259A PIC, IOAPIC)
//   - IPI (multiple vCPUs within guest)
//   - Timer (hrtimer → KVM timer)

References

  • Source Code: virt/kvm/kvm_main.c (core), arch/x86/kvm/vmx/vmx.c (Intel VMX), arch/x86/kvm/svm/svm.c (AMD SVM)
  • Kernel Documentation: Documentation/virt/kvm/
  • LWN: "KVM: the Linux virtual machine monitor"

Keywords: KVM, VMX, SVM, vCPU, VM exit, VMLAUNCH, VMRESUME, /dev/kvm, KVM_RUN, interrupt injection

Related pages

© 2026 liz's doc