RCU: Publication, Grace Periods, and Reclamation

The configuration pointer has been switched to a new object, but old readers have not yet finished. RCU separates the update of the entry point from the reclamation of old memory.

RCU separates "removing an object from a shared structure" from "freeing the object's memory." Readers can access published objects with lower synchronization overhead; after an updater revokes the reachable entry points of the old object, it waits for an appropriate grace period before reclaiming the old memory is allowed.

It does not require copying the entire structure for every update, nor does it automatically provide readers with a globally consistent, up-to-date snapshot across multiple objects. RCU is more like a set of protocols regarding publication and lifetime, which must be designed in conjunction with the update rules of the data structure itself.

1. Three Responsibilities Must Be Handled Separately

The __rcu annotation aids static checking; it is not automatic reference counting at runtime. rcu_dereference() must be used according to the corresponding read-side or protected conditions; you cannot assume any object is safe just because this macro is used.

2. Grace Period Waits for More Than Just "All Readers in the World"

After removing the old object, an appropriate grace period ensures that the relevant read-side critical sections that might previously have held the old object have ended. New readers can continue to enter; the system does not need to wait for the instant when "no CPU has any RCU readers at all."

The object must first be revoked from all relevant publication entry points, and readers must only obtain it via agreed-upon entry points. If another linked list still points to the old object, or if a reader takes a raw pointer out of the critical section to continue using it, waiting for a single grace period will not fix the lifetime error.

A grace period does not start a fixed timer from the pointer assignment instruction, nor is it a fixed number of milliseconds. Scheduling, configuration, read-side duration, and implementation all affect the wait time. Synchronous waiting can sleep; asynchronous callbacks avoid blocking the updater synchronously but can accumulate unreclaimed objects.

There is no universal fixed millisecond duration. synchronize_rcu can sleep; call_rcu queues later reclamation, which can accumulate memory if updates outpace reclamation.

3. Preemptible RCU and Quiescent States

You cannot uniformly say that rcu_read_lock() is always preempt_disable(). In preemptible RCU configurations, read-side tasks can be preempted, and the kernel tracks relevant readers; the instruction overhead for read-sides differs across implementations.

Normal RCU read-sides should adhere to API rules that prohibit arbitrary blocking; being preempted by the scheduler is not the same as actively calling operations that may sleep. If you need to sleep normally on the read-side, consider appropriate interfaces like SRCU, rather than inserting schedule yourself.

A quiescent state is evidence that the relevant read-side has passed; it may involve user mode, idle, context switches, and other states; it does not require that every CPU undergo a context switch at least once during each grace period. A preempted RCU reader cannot be considered to have exited simply because the CPU switched away. See What is RCU.

4. A Replaceable Configuration Object

The following is an example of an independent translation unit for a kernel component, excluding registration entry points. Once the object is published, its value is not modified in place; updaters use a mutex for serialization, and readers only copy out scalars.

#include <linux/errno.h>
#include <linux/mutex.h>
#include <linux/rcupdate.h>
#include <linux/slab.h>

struct kb_config {
    int value;
    struct rcu_head rcu;
};

static DEFINE_MUTEX(config_lock);
static struct kb_config __rcu *current_config;

int kb_config_get(int *value);
int kb_config_replace(int value);
void kb_config_shutdown(void);

static void free_config(struct rcu_head *head)
{
    struct kb_config *p = container_of(head, struct kb_config, rcu);
    kfree(p);
}

int kb_config_get(int *value)
{
    struct kb_config *p;
    int ret = -ENOENT;

    rcu_read_lock();
    p = rcu_dereference(current_config);
    if (p) {
        *value = p->value;
        ret = 0;
    }
    rcu_read_unlock();
    return ret;
}

int kb_config_replace(int value)
{
    struct kb_config *next, *old;

    next = kmalloc(sizeof(*next), GFP_KERNEL);
    if (!next)
        return -ENOMEM;
    next->value = value;

    mutex_lock(&config_lock);
    old = rcu_replace_pointer(current_config, next,
                              lockdep_is_held(&config_lock));
    mutex_unlock(&config_lock);
    if (old)
        call_rcu(&old->rcu, free_config);
    return 0;
}

/* Callers must first shut down all entry points and wait for executing API calls to exit. */
void kb_config_shutdown(void)
{
    struct kb_config *old;

    mutex_lock(&config_lock);
    old = rcu_replace_pointer(current_config, NULL,
                              lockdep_is_held(&config_lock));
    mutex_unlock(&config_lock);
    synchronize_rcu();
    kfree(old);
    rcu_barrier(); /* Wait for previously queued callbacks that will execute this component's code. */
}

The value output address is guaranteed to be valid by the caller and should not point to user pointers that require page faults. The shutdown's entry point closure and waiting for call exits are responsibilities of external components: adding config_lock alone cannot prevent a replace operation that hasn't yet reached the lock acquisition point from republishing after shutdown.

Between Writers, RCU Does Not Replace Mutual Exclusion

Suppose two writers both read the old object O, and then publish N1 and N2 respectively. If they each reclaim the O they read, they might double-free the same object. The example serializes the "obtain old value, replace, schedule reclamation" operation using an updater-side mutex.

5. Synchronous Waiting and Asynchronous Callbacks

call_rcu() receives an embedded rcu_head, and the callback parameter is struct rcu_head *; use container_of to retrieve the outer object. kfree_rcu(p, rcu) is a common release interface, but it should not be interpreted as forcibly casting kfree into a callback pointer; their parameters and implementations are different.

Callbacks for call_rcu() must adhere to execution constraints that prohibit arbitrary sleeping. Callbacks become eligible for execution only after the grace period, but there is no guarantee that they will run immediately on a specific CPU the moment the grace period ends.

Module Exit: Reader End and Callback Completion Are Not the Same Condition

When module callbacks reside in the module's own code, you must first resolve new submission entry points, then wait for callbacks to complete, and only then can you unload. See RCU and Unloadable Modules.

When asynchronous update rates are high, you must also control the growth of memory pending reclamation. Expedited grace periods can reduce some waiting times but increase system overhead and cannot serve as a universal fix for all RCU stalls.

6. SRCU and Tasks RCU

SRCU uses an independent srcu_struct domain. The index token obtained by the reader must be passed back to the unlock of the same domain:

int idx = srcu_read_lock(&domain);
/* Obtain and use the object according to the SRCU protocol; sleeping is allowed under permitted conditions. */
srcu_read_unlock(&domain, idx);

Updaters use synchronize_srcu(&domain) or the corresponding callback API. You cannot wait for a grace period containing yourself from within your own same-domain read-side critical section, nor can you destroy the domain before reclamation is complete. Readers and waits from different domains are not interchangeable.

RCU Tasks, Tasks Rude, and Tasks Trace are designed for special task/tracing synchronization needs, with different waiting conditions. They are not a single variant called "trace-specific preempt-disabled RCU." Without corresponding underlying requirements, you should not replace normal RCU just because the name contains "Tasks."

7. What If You Continue Using After the Read-Side Ends?

If you need to hold an object for a long time after exiting the read-side, you can attempt to obtain an independent reference according to the object's protocol within the critical section, and confirm success; the rules for reclamation and zero-initial acquisition must be designed together. Simply storing a raw pointer in a global variable or asynchronous work item, and then exiting the RCU read-side, will lose the original lifetime guarantee.

RCU also does not automatically serialize concurrent updates to the same object field. Immutable object replacement is a more easily reasoned pattern; when in-place updates are needed, you must additionally specify locks, atomic operations, and the consistency acceptable to readers.

8. RCU Stalls Should Be Analyzed Based on Evidence

A stall indicates that RCU did not make necessary progress within the expected time. Causes may include long read-sides, long interrupts/preemptions disabled, CPU starvation, timer or scheduling anomalies, etc. You cannot see an alert and immediately assert that rcu_read_unlock was definitely missed.

Locate the issue by combining CPU/task information, grace periods, repeated stack traces, and running configuration from the report. Increasing the stall timeout only changes the reporting threshold; it does not fix the progress issue. See RCU Stall Warnings and Synchronization Debugging.