11 min read #linux-kernel #Memory Management
On this page

Virtual Memory

Coverage: Page table traversal (4-level/5-level) → PTE format → TLB mechanism → HugeTLB/THP → KPTI isolation Kernel versions: 2.6 ~ 6.x, with emphasis on 5-level paging and mTHP evolution

Overview

Virtual memory is the cornerstone of Linux memory management. It isolates each process within its own address space while enabling demand paging, copy-on-write (COW), memory sharing (mmap MAP_SHARED), and swap through page tables.

Understanding virtual memory hinges on understanding page table traversal—how the CPU translates a virtual address into a physical address. This process occurs on every memory access (handled by MMU hardware), making speed critical.

Page Table Structure

x86-64 4-level paging (48-bit)

flowchart LR
    VA["Virtual Address (48-bit)"]

    VA --> PGD["PGD/PML4 Index<br/>[47:39] 9 bit<br/>512 entries"]
    PGD -->|"CR3 →"| PUD["PUD Index<br/>[38:30] 9 bit<br/>512 entries"]
    PUD --> PMD["PMD Index<br/>[29:21] 9 bit<br/>512 entries"]
    PMD --> PTE["PTE Index<br/>[20:12] 9 bit<br/>512 entries"]
    PTE --> PAGE["4KB Physical Page<br/>offset [11:0] 12 bit"]

    classDef va fill:#e3f2fd,stroke:#1565c0
    classDef level fill:#f3e5f5,stroke:#7b1fa2
    classDef phys fill:#e8f5e9,stroke:#2e7d32
    class VA va
    class PGD,PUD,PMD,PTE level
    class PAGE phys

Why exactly 9 bits per level (512 entries)? Because each PTE is 8 bytes, and 512 × 8 = 4KB, which fits exactly into one physical page. This self-similar design allows page tables themselves to be managed by the page allocator.

5-level paging (57-bit, kernel 4.14+)

flowchart LR
    VA5["Virtual Address (57-bit)"]
    VA5 --> P5E["PML5 Index<br/>[56:48] 9 bit<br/>512 entries"]
    P5E --> P4D["P4D Index<br/>[47:39] 9 bit<br/>512 entries"]
    P4D --> PUD5["PUD Index<br/>[38:30] 9 bit"]
    PUD5 --> PMD5["PMD Index<br/>[29:21] 9 bit"]
    PMD5 --> PTE5["PTE Index<br/>[20:12] 9 bit"]
    PTE5 --> PAGE5["4KB Physical Page<br/>offset [11:0]"]

    classDef va fill:#e3f2fd,stroke:#1565c0
    classDef level fill:#f3e5f5,stroke:#7b1fa2
    classDef phys fill:#e8f5e9,stroke:#2e7d32
    class VA5 va
    class P5E,P4D,PUD5,PMD5,PTE5 level
    class PAGE5 phys

Requires CPU support for the LA57 feature (Ice Lake+). Check: grep la57 /proc/cpuinfo

5-level paging expands the virtual address space from 256TB (128T user + 128T kernel) to 128PB. This is not for today's applications—it is for future CXL shared memory pools and PB-scale NVDIMMs.

How Large Pages Reduce Levels

flowchart LR
    subgraph HUGE["Large Page Traversal Levels"]
        direction LR
        A["PGD"] --> B["PUD"]
        B -->|"2MB: PS=1"| C["PMD → 2MB Large Page ✅"]
        B -->|"1GB: PS=1"| D["PUD → 1GB Large Page ✅"]
    end

    subgraph NORMAL["4KB Page Traversal"]
        direction LR
        E["PGD"] --> F["PUD"] --> G["PMD"] --> H["PTE → 4KB"]
    end

    classDef huge fill:#e8f5e9,stroke:#2e7d32
    classDef normal fill:#e3f2fd,stroke:#1565c0
    class HUGE huge
    class NORMAL normal

In PMD/PUD entries: bit 7 (PS bit) = 1 → do not traverse further; this level points directly to the final physical address.

Page Table Entry (PTE) Format

x86-64 PTE (simplified, 64 bit):
  bit 0:   Present       → Page is in physical memory (1) or swapped out/unmapped (0)
  bit 1:   R/W           → Writable (1) or Read-only (0) — Key for COW!
  bit 2:   U/S           → User accessible (1) or Kernel only (0)
  bit 3:   PWT           → Page Write-Through caching
  bit 4:   PCD           → Page Cache Disable
  bit 5:   Accessed      → Set to 1 by hardware (page has been accessed)
                          → Key for reclamation: unaccessed pages are swap candidates
  bit 6:   Dirty         → Set to 1 by hardware (page has been written to)
                          → Dirty pages must be written back during swap; clean pages can be discarded
  bit 7:   PAT / PS      → PTE level: PAT (Page Attribute Table)
                          → PMD/PUD level: PS (Page Size, indicating large page)
  bit 8:   Global        → Not flushed on CR3 switch (used for kernel page tables)
  bit 11:  NX bit 63     → No-Execute (XD), non-executable
  bit 63:  NX            → Alias for bit 11

Physical Address: bits [51:12] → Points to a 4KB-aligned physical page
  Remaining bits [11:0] are the offset and are not involved in address translation

The Accessed and Dirty bits act as hardware accelerators for memory reclamation. Without them, the kernel would need to simulate "has this been used?" via page faults, costing hundreds of times more.

TLB (Translation Lookaside Buffer)

Basic Concepts

TLB = Page table cache inside the MMU (hardware)
  L1 TLB: ~64 entries, 1 cycle latency (separate for instructions and data)
  L2 TLB: ~1500 entries, ~5 cycles

TLB miss:
  1. Hardware traverses page table (x86: hardware page walker, 4 memory accesses)
  2. If Present=0 at any level during traversal → page fault → handled by kernel
  3. Once PTE is found, it is loaded into the TLB

TLB Coverage:
  4KB pages:  L2 TLB 1500 entries × 4KB = 6MB coverage
  2MB pages:  L2 TLB 1500 entries × 2MB = 3GB coverage
  1GB pages:  32 entries (L1 only) × 1GB = 32GB coverage

PCID / ASID

Problem: Writing CR3 during context switch invalidates all non-global entries in the TLB
         → Every memory access for the new process is a TLB miss → severe performance loss

Solution:
  x86 PCID (Process-Context Identifier, 12-bit):
    Attach ASID when writing CR3 → TLB entries with different ASIDs:
      - Are not flushed (remain in TLB)
      - Do not cause false hits (ASID mismatch)
    → When a hot process switches back, the TLB still has high hit rates

  ARM64 ASID (8-bit or 16-bit):
    Similar principle; ASID is specified when writing TTBR0_EL1
    Hardware distinguishes TLB entries for different ASIDs

  Kernel Page Tables (Global bit):
    Set Global bit on kernel page table PTEs → not flushed on CR3 switch
    → Kernel TLB entries are shared across different processes

TLB Shootdown

TLB Shootdown: Multi-core TLB consistency coordination flow CPU 0 · Initiating Core Other CPUs · Target Cores 1. Modify Page Table e.g., munmap() / mprotect() 2. Flush Local TLB Immediately INVLPG instruction or rewrite CR3 Still executing original task Has not yet noticed page table changes (TLB still caches old PTEs) 3. Send IPI Broadcast Inter-Processor Interrupt Receive IPI Interrupt Pause current execution flow IPI Block and Wait flush_tlb_mm_range() has not returned Flush TLB in Interrupt Context Clear stale PTE caches for that mm Reply Complete (ACK) Notify initiating core that processing is done All Target CPUs Have Replied flush_tlb_mm_range() returns; TLB states are consistent across cores Lazy TLB Target is in kernel thread (no user space) → deferred flush Batching Merge multiple flushes → single IPI Hardware Broadcast AMD INVLPGB / Intel TLB broadcast Why IPI is mandatory: Flushing only the local TLB is insufficient—other cores still cache old PTEs, and without broadcast notification, they will continue using old translations, causing inter-core memory view inconsistencies or even security issues.

HugeTLB

Reserves large blocks of contiguous physical memory, providing deterministic large page mappings:

// Configuration
echo 128 > /sys/kernel/mm/hugepages/hugepages-2048kB/nr_hugepages
// Or boot param: hugepagesz=1G hugepages=4 hugepagesz=2M hugepages=1024

// Usage 1: hugetlbfs
mount -t hugetlbfs -o pagesize=2M none /dev/hugepages
mmap(NULL, size, PROT_READ|PROT_WRITE, MAP_PRIVATE|MAP_HUGETLB, fd, 0);

// Usage 2: mmap flag
mmap(NULL, size, PROT_READ|PROT_WRITE,
     MAP_PRIVATE|MAP_ANONYMOUS|MAP_HUGETLB, -1, 0);

// QEMU/libvirt uses HugeTLB to map guest RAM for VMs
// → Reduces TLB misses in nested page tables (stage-2 page table)

Advantages of HugeTLB: Deterministically available (allocated at reservation time), high TLB efficiency. Disadvantages: Requires contiguous physical memory, not swappable (before 5.x), wastes entire pages for small data.

THP (Transparent Huge Pages)

Mechanism

The kernel automatically merges contiguous 4KB pages into 2MB large pages, transparently to the user:

Allocation:
  Process malloc(128KB) → triggers page fault → do_anonymous_page()
    → If VM supports THP and is aligned → attempt to allocate 2MB zero page
    → Success: Map entire 2MB with a single large page (reduces subsequent 512 page faults)
    → Failure: Fall back to 4KB pages

khugepaged:
  Background kernel thread → periodically scans process address spaces
    → Discovers contiguous 4KB pages that can be merged → migrate/merge → upgrade to 2MB THP
    → Discovers most pages within a THP are freed → split back into 4KB pages
    → Controlled via /sys/kernel/mm/transparent_hugepage/khugepaged/

Policy Control

cat /sys/kernel/mm/transparent_hugepage/enabled
# always:  Always attempt to allocate THP (default)
# madvise: Only VMA marked with MADV_HUGEPAGE
# never:   Disable

# Application-level control:
madvise(ptr, size, MADV_HUGEPAGE);   // Suggests using THP

THP Costs

Advantages:
  + Significantly reduced TLB miss rate (512x coverage increase)
  + Reduced page faults (one allocation covers 512 pages)
  + Reduced kernel page table traversal levels

Disadvantages:
  - Allocation latency (requires finding 2MB contiguous physical memory → may need compact/reclaim)
  - Internal fragmentation (3KB of valid data occupies 2MB of physical memory)
  - Slower performance for certain workloads (e.g., some databases manage large pages themselves)

This is why DBAs often recommend: echo never > transparent_hugepage/enabled

mTHP (multi-size THP, 6.12+)

An evolving solution: Allows THP sizes to be non-fixed at 2MB
  → 4KB / 16KB / 32KB / 64KB / 128KB / 256KB / 512KB / 1MB / 2MB
  → Select appropriate size on demand
  → Reduces internal fragmentation (no need for full 2MB)
  → Retains TLB efficiency (much better than 4KB)

This is an actively developed area within the kernel community

KPTI: Page Table Isolation

Meltdown and Fix

Meltdown (2018):
  CPU does not check U/S bit during speculative execution → user space can speculatively read kernel memory
  → Residual data in cache can be read via side channels

KPTI (Kernel Page Table Isolation):
  x86: PTI (Page Table Isolation)
  ARM: KPTI (Kernel Page Table Isolation)

  Each process has **two** page tables:
    Kernel page table: Full mapping (user + kernel addresses)
    User page table: Minimal mapping (user + only trampoline)

  Trampoline (entry trampoline):
    Contains only entry_SYSCALL_64 / iret / Interrupt Descriptor Table (IDT)
    → Sufficient to enter kernel mode → immediately switch to kernel page table upon entry
    → Switch back to user page table before returning from syscall → then iret

Performance Impact

Syscall-intensive workloads: ~5%~30% performance degradation
  Each syscall: 2 CR3 writes + 2 TLB misses (trampoline is too small)
  Mitigations:
    - PCID: Avoids full TLB flush (but CR3 write overhead remains)
    - KPTI is automatically disabled on CPUs not affected by Meltdown (AMD / newer Intel)

Debugging Interfaces

# Full page table traversal (requires root)
cat /proc/<pid>/pagemap  # Binary, 8 bytes per page
# bit 63: page present
# bits [54:0]: PFN (Physical Frame Number)

# Check THP usage
grep -i thp /proc/meminfo
# AnonHugePages: THP used for anonymous memory
# ShmemHugePages: THP used for shmem

# Check if 5-level paging is enabled
grep la57 /proc/cpuinfo

# Large page info
cat /proc/meminfo | grep -i huge
cat /sys/kernel/mm/hugepages/hugepages-*/nr_hugepages

# Page table size (per-process)
cat /proc/<pid>/status | grep VmPTE

References and Further Reading

  • Kernel Documentation: Documentation/x86/x86_64/mm.rst, Documentation/vm/transhuge.rst, Documentation/x86/pti.rst
  • LWN:
    • "5-level paging" (lwn.net/Articles/717293/)
    • "Transparent huge pages in 6.x" (lwn.net/Articles/971579/)
    • "The multi-size THP patch set" (lwn.net/Articles/931701/)
  • Source Files:
    • arch/x86/include/asm/pgtable.h — PTE format definitions
    • arch/x86/mm/tlb.c — TLB flush implementation
    • mm/huge_memory.c — THP core
    • mm/hugetlb.c — HugeTLB
    • arch/x86/mm/pti.c — KPTI

Keywords: Page Table, PGD/PUD/PMD/PTE, PTE flags (A/D/RW/NX), TLB, PCID, ASID, HugeTLB, THP, khugepaged, mTHP, KPTI, Meltdown

Related pages

© 2026 liz's doc