Page Reclaim and Swap: How to Free Memory Without Losing Needed Data

A service’s 100 MiB of hot data, repeatedly used, is evicted by a one-time 2 GiB file scan; reclaim statistics look good, but the application keeps re-reading. The goal of reclaim is not just to free pages, but also to consider how content is recovered, who bears the pause, and whether it will be needed again soon.

First, explain candidates based on content backing, then deduce working set, swapping, and concurrent swap-in, and finally connect compaction, defragmentation, and monitoring into a single line.

1. Ask Where Content is Recovered From First, Then How Cold It Is

For an unmodified regular file page, if the backend can re-provide the content, the memory copy can be discarded after removing relevant references. If the page contains unflushed modifications, discarding it directly would break file semantics, requiring advancement according to the writeback protocol. Anonymous memory usually does not have an original file to re-read; content that must be retained needs backing such as swap, or already has a valid backing copy.

Saying "anonymous pages must be written to disk every time to be released" is too absolute. If there is a valid swap copy, compressed cache support, the application explicitly discards the content, or the entire mapping lifecycle ends, the processing paths differ. Conversely, although tmpfs presents a file interface, its content cannot be casually discarded like regular disk files; its shmem backing relationship is closer to swappable memory.

Hot/cold status, content backing, and references are three independent dimensions:

Therefore, candidate selection cannot always be fixed as "clean file, dirty file, anonymous"; it should simultaneously consider release conditions and future recovery costs.

There is no universal fixed eviction order of clean file, dirty file, then anonymous pages.

2. Understanding Hot/Cold Judgment with a Working Set Example

Assume a service repeatedly accesses 100 MiB of hot data per round, while a one-time scan continuously reads in a 2 GiB file. If all new pages are treated as hotspots worth long-term retention simply because the scan just occurred, the hot working set may be evicted; the next round of business will re-read it back. Every page freed looks like a success, but the system is constantly doing the repetitive work of discarding and re-reading.

The traditional active/inactive mechanism uses access clues and queue positions to approximate usage differentiation, managing types like anon/file separately; it is not a precise global LRU recording every access, nor is it the historical path only available in 5.x. MGLRU organizes age information in multiple generations when enabled; aging discovers recent usage through clues such as page table access bits, eviction advances from older candidates, and selection is adjusted based on feedback like refault.

A generation is not a precise timestamp on every page, nor does it mean all pages collectively expire at fixed second intervals. Scanning page tables and reverse mappings still incurs costs; MGLRU improves information collection and organization, but it does not know how hot every page is without scanning. Accesses via file descriptors and accesses via page tables provide different clues. See MGLRU Design.

Why Refault Explains More Than a Single Successful Reclaim

Comparing two teaching windows reveals what looking solely at reclaim rates misses:

A's scan/steal looks good, but may incur significant repeated recovery costs; B's lower reclaim volume does not directly represent worse performance.

This does not mean every refault is judged as an error: when the working set is larger than available memory, some content must inevitably move in and out. The value of feedback lies in judging whether candidate groups, types, and protection strategies are still appropriate. Treating pgsteal/pgscan merely as an allocator score misses recovery costs, page size differences, and application latency.

3. kswapd and Direct Reclaim: Who Pays the Reclaim Time

Background Work First: Reserving Margin for Subsequent Allocations

Background kswapd attempts to restore available margin in node/zone before and after resource contention, making future allocations easier to complete via the fast path. The common simplified model wakes up under low watermarks and balances toward high watermarks, but the real goal is influenced by order, zone, and other policies. It does not mean "a round starts fixed as soon as free drops below high."

Applicant Does It Themselves: Reclaim Enters This Latency

If the allocator cannot obtain pages from allowed resources, and GFP allows direct reclaim, it can perform reclaim itself. This adds scanning, waiting, and other work to the latency of this allocation; thus, even without obvious disk throughput peaks, significant memory pauses may occur. Direct reclaim does not guarantee waiting until success; it may still fall back, fail, or enter other processing later.

Determine Resource Domain First: Global or cgroup

The reclaim scope is equally important. When a specific memory cgroup reaches its constraint, reclaim can occur within it, even if the whole machine has plenty of free pages. Throttling related to memory.high, hard limits of memory.max, NUMA allowed nodes, and ordinary global memory pressure are not the same issue. First, confirm which resource domain lacks quota, then decide whether global LRU analysis is needed.

Slab objects are not all located in the regular page LRU. The kernel requests cache subsystems with reclaim capabilities to release objects via shrinkers, such as cache items no longer needed; the subsystem is responsible for judging references, locks, and business state. If there are still active users, arbitrary objects cannot be forcibly destroyed just to satisfy page allocations.

4. Completing an Anonymous Page Swap-out: Saving Both Content and Name

Suppose process P's address A points to physical page F, which contains data the application must retain. If F is only written to swap, but PTE continues to allow writes, the application may modify F after writing, invalidating the backing copy; if PTE is only cleared, without recording where the content is, there is no way to recover it later on a page fault.

Swap-out requires maintaining three types of facts simultaneously:

These three types of work are connected by concurrent locks and retry protocols. Writing to disk first and then arbitrarily modifying PTE cannot guarantee no writes occurred during the interval, nor can it guarantee the page is no longer in use.

Reverse mapping answers "which virtual mappings might use this content," without blindly scanning all process page tables. File mappings can find candidate VMAs from the mapping index of address_space; anonymous mappings can use relationships like anon_vma; specific page tables must still be checked, as candidate VMAs do not equal existing valid PTEs. Fork, COW, and large folios change sharing and mapping statistics; reference counts cannot replace precise enumeration of user mappings.

Why Two Threads Swapping In Should Not Get Uncoordinated Copies

When a PTE with swap identity S triggers a page fault, the kernel first needs to determine if related content is already resident or if someone is already recovering it. If another thread has already read S's content into memory, the later arrival can wait or reuse the appropriate result, rather than unconditionally reading storage again. This reduces repeated I/O and maintains consistency of shared content, state, and references.

Swap cache participates in linking swap identity with resident folios. It is a conceptual responsibility and should not fix old address_space/XArray structures as the implementation for all new kernels. Finding a resident object does not mean it can be used immediately: whether I/O is complete, whether references are stable, and whether the page still corresponds to that identity must all be confirmed. After installing the final mapping, when the swap slot is released depends on other references; one cannot reason "one swap-in immediately deletes backing."

This explains why swap-in faults do not necessarily produce disk I/O: content may still be in memory or compressed backing. Large page swap-out does not always need to be split into 4 KiB first; supported paths and continuous backing conditions evolve with the kernel, so current implementation should be the standard.

5. What zswap, zram, and Swappiness Control

zswap is a compressed memory cache on the swap path, usually collaborating with configured swap backing. Successful compression allows retaining more content with less RAM; under cache pressure, it can write to backing; uncompressed data and reaching limits result in different handling. It is not a generic file cache without swap identity. See zswap.

zram exposes a compressed memory block device, which can be configured as swap or used for other block device purposes. Its logical disksize does not equal pre-allocated RAM of the same amount; compressed data and metadata still occupy actual memory, and some configurations support writeback to backing. "8 GiB zram" cannot be directly added to physical memory as definite extra capacity. See zram.

Space Accounting: What is the Net Gain from Compressing 100 MiB?

Ignoring metadata, the net freed is 60 MiB. If it can only be compressed to 95 MiB, the gain is only 5 MiB, yet the compression/decompression CPU cost is still paid. When the working set is repeatedly accessed, these recovery operations may also enter the request tail latency.

Swappiness: Comparing Costs, Not Setting Usage Thresholds

Swappiness expresses the relative preference for swap vs. file page reclaim costs; the current interface range is 0 to 200, and it cannot be treated as "start swapping only when memory reaches X%." In environments with relatively cheap compressed memory backing, values over 100 can have cost justification; 0 is not a switch to turn off swap. When tuning, look at real re-read, swap-in, and CPU costs; do not treat fixed 60 or 0 as the answer for all workloads. See VM sysctl.

6. Reclaim and Memory Defragmentation Solve Two Different Constraints

Continuing with the 16-page model, let's see what happens when quantity is sufficient but shape is not.

Assume 8 pages are free among 16, but free and occupied pages are interleaved. Order 0 allocations have many candidates, but order 3 has no ready-made contiguous block of 8 pages. If movable occupied pages are migrated to the other side, a suitable block may form, and the total free pages remain 8. Migration requires maintaining virtual address semantics, handling page tables and related references; when encountering pins or non-migratable objects, this ideal layout may not be achievable. See Page Migration.

kcompactd and the allocator's direct compaction share the defragmentation work, similar to background/synchronous cost-bearing methods. High-order allocation failures and active compaction do not necessarily mean the machine has no available memory; conversely, large anonymous working sets lacking capacity cannot be solved solely by arranging pages more neatly.

OOM: Failure Paths Also Constrained by Allocation Context

When reclaim within allowed ranges cannot make sufficient progress, some allocations may enter OOM processing, but not every high-order or GFP_ATOMIC failure triggers OOM. After OOM kills a task, pages also become gradually available as references are released; pages fixed externally or with other shared references may still delay resource recovery. Judgment should combine constraint domains, allocation context, and victim information from OOM reports.

7. Observe a Time Window, Not Cumulative Large Numbers Since Boot

Below, we only read statistics, at 0.2-second intervals, without actively allocating large amounts of memory, reclaiming caches, or configuring swap. It displays similar fields for scanning and reclaiming in pairs, and reads the actual enabled bits for MGLRU.

from pathlib import Path
import time

def vmstat():
    return {key: int(value) for key, value in
            (line.split() for line in Path("/proc/vmstat").read_text().splitlines())}

before = vmstat()
start = time.monotonic()
time.sleep(0.2)
after = vmstat()
elapsed = time.monotonic() - start
for suffix in ("kswapd", "direct"):
    scan_key, steal_key = "pgscan_" + suffix, "pgsteal_" + suffix
    if scan_key in before and steal_key in before:
        scan = after[scan_key] - before[scan_key]
        steal = after[steal_key] - before[steal_key]
        print(suffix, "scan=", scan, "steal=", steal,
              "ratio=", steal / scan if scan > 0 else "No scan")
for key in ("pswpin", "pswpout", "compact_stall", "compact_success", "compact_fail"):
    if key in before and key in after:
        print(key, after[key] - before[key])
print("window_seconds", elapsed)
for name in ("/sys/kernel/mm/lru_gen/enabled", "/proc/pressure/memory"):
    try:
        print(name, Path(name).read_text().strip())
    except OSError as error:
        print(name, type(error).__name__)

Counts may be missing due to kernel configuration; zero values for inactive windows only indicate that no corresponding events were observed in this window. MGLRU's enabled is a bitmask; interpret bits according to the interface documentation; directory existence only indicates the interface is exposed, not that the main feature is enabled. See MGLRU Management Interface.

PSI memory some describes the time at which at least one task was paused due to this resource; full describes the time when all non-idle tasks within the relevant scope were blocked; it measures pauses, not RAM usage rate. Cumulative total is in microseconds; avg10/60/300 are averages over corresponding windows; global and cgroup scopes cannot be mixed. See PSI.

After reading counts, continue locating by causal order:

1. Confirm Constraint Domain. Is the pressure global, cgroup, or specific allowed node?
2. Confirm Pause Location. Did direct reclaim push work onto the allocator path?
3. Compare Same Window. Correlate scan, reclaim, refault, swap, compaction, and application latency.

anon/file and kswapd/direct are different grouping dimensions; do not indiscriminately sum all these counts. Only when constraints, recovery costs, and application pauses correspond can we determine whether to improve locality, expand capacity, or reduce unnecessary contiguous allocation requirements.