Debugging and Testing

A successful build, a warning-free run, and a SKIP test case each represent different outcomes. Kernel verification requires keeping inputs, execution paths, environmental prerequisites, and results in the same record.

printk and Console Log Levels

pr_err, pr_info, pr_debug, etc., provide logging entry points, while the dev_* forms include device context. Logging to the kernel buffer and final output to the console are different stages; console_loglevel filters console output and does not equate to deleting all lower-level records.

Smaller log level numbers indicate higher severity. console_loglevel=4 typically outputs messages with a number less than 4, i.e., error and more severe levels; this setting should not be described as meaning the system only has these logs. High-frequency path logging can also alter timing and performance, requiring rate limiting or switching to appropriate tracepoints.

Dynamic Debug Controls Existing Call Sites

Dynamic debug enables pr_debug/dev_dbg and other messages based on module, function, file, line number, or other conditions, provided the call sites are supported and compiled in. It cannot create logging entry points for arbitrary functions that lack debug statements.

This document provides /proc/dynamic_debug/control; when debugfs is enabled and mounted, /sys/kernel/debug/dynamic_debug/control may also be available. First, read the actual existing control file to confirm entries and original flags, then make limited-range modifications. Dynamic debug

For example, the command string module my_driver +p enables the corresponding module entry; -p disables printing, and flags like f/l/t add function, line number, and thread information. When restoring, revert to the previous state; do not unconditionally clear debug points that others have enabled.

KUnit and kselftest Check Different Boundaries

KUnit verifies internal functions or subsystem logic within the kernel environment and can be executed via UML, QEMU, or a test kernel. kselftest typically checks kernel interfaces and behavior from user space, with some suites cooperating with test modules; the two are complementary. KUnit Introduction, kselftest

KUnit involves more than just a KUNIT_CASE array; it requires registering a suite and integrating it into Kconfig/Kbuild. The following uses a buffer range check as an example: capacity is capacity, access is [offset, offset+length); length zero is allowed if offset exactly equals capacity, but out-of-bounds access is prohibited. If you directly judge offset + length <= capacity, unsigned integer overflow might incorrectly treat an illegal range as valid.

Save as kb_range_test.c:

#include <kunit/test.h>
#include <linux/module.h>
#include <linux/limits.h>

static bool range_fits(size_t capacity, size_t offset, size_t length)
{
    return offset <= capacity && length <= capacity - offset;
}

static void range_boundary_test(struct kunit *test)
{
    KUNIT_EXPECT_TRUE(test, range_fits(4096, 4092, 4));
    KUNIT_EXPECT_FALSE(test, range_fits(4096, 4092, 5));
    KUNIT_EXPECT_TRUE(test, range_fits(4096, 4096, 0));
    KUNIT_EXPECT_FALSE(test, range_fits(4096, 4097, 0));
    KUNIT_EXPECT_TRUE(test, range_fits(0, 0, 0));
}
static void range_overflow_test(struct kunit *test)
{
    KUNIT_EXPECT_FALSE(test, range_fits(4096, SIZE_MAX - 3, 4));
    KUNIT_EXPECT_TRUE(test, range_fits(SIZE_MAX, SIZE_MAX, 0));
    KUNIT_EXPECT_FALSE(test, range_fits(SIZE_MAX, SIZE_MAX, 1));
}
static struct kunit_case range_cases[] = {
    KUNIT_CASE(range_boundary_test),
    KUNIT_CASE(range_overflow_test),
    {}
};
static struct kunit_suite range_suite = {
    .name = "kb-range",
    .test_cases = range_cases,
};
kunit_test_suite(range_suite);
MODULE_LICENSE("GPL");

The first group checks exclusive upper bounds and empty interval contracts; the second group checks addition risks near the maximum integer. offset <= capacity also ensures the subsequent subtraction does not underflow; C's && short-circuit evaluation ensures the second term is only calculated if this premise holds. This helper only verifies numerical ranges; it does not guarantee pointer validity, page accessibility, or device DMA execution rights.

When integrating the test source code into the directory, add the corresponding Kconfig entry and Kbuild rules, and ensure the upper-level configuration and build include this directory:

config KB_RANGE_KUNIT_TEST
    tristate "Range boundary tests" if !KUNIT_ALL_TESTS
    depends on KUNIT
    default KUNIT_ALL_TESTS

obj-$(CONFIG_KB_RANGE_KUNIT_TEST) += kb_range_test.o

For a clean source tree with this test integrated, you can use .kunitconfig in an independent build directory to enable CONFIG_KUNIT=y and CONFIG_KB_RANGE_KUNIT_TEST=y, then filter by suite name. Filters only select compiled tests and cannot replace configuration enabling.

./tools/testing/kunit/kunit.py run --build_dir=.kunit-range --kunitconfig=.kunitconfig kb-range

The default UML path builds and runs a user-space Linux test kernel, rather than loading the test module in the current host. If the source tree already has incompatible build artifacts, use a separate clean source copy; do not execute mrproper on the actively maintained kernel tree just for tutorial commands. KUnit Configuration, Build, and Filtering

Built-in tests and module tests have different execution timings. kunit.py can help configure, build, and execute tests; the first run may require a full test kernel build; merely compiling this C file does not mean the test has been executed.

Result Interpretation: Record SKIP and PASS Separately

TARGETS in selftests can filter suites, but network, BPF, and other tests require corresponding kernel configurations, permissions, and dependencies, and may alter the test environment. Prioritize running suites related to modifications and record SKIP reasons; "all skipped" is not "all passed."

Sanitizers, Lock Checks, and Breakpoints Provide Different Evidence

KASAN focuses on applicable out-of-bounds and use-after-free errors; KCSAN helps discover data races; lockdep checks lock dependency constraints. They are influenced by configuration, execution paths, and sampling mechanisms; do not treat unreported issues as proof of no errors.

KGDB allows debugging the kernel with GDB under appropriate configuration and debug link. SysRq-g may halt the target kernel waiting for the debugger, suitable for test environments with recovery means; it should not be executed directly as a daily read-only diagnostic command.

Breakpoints and logs both alter scheduling timing. Hard-to-reproduce races can be combined with tracepoints, minimal inputs, and repeated triggers to save the order of occurrence, rather than just keeping the final stack trace.

syzkaller and LTP

syzkaller uses system call descriptions, resource relationships, and coverage feedback to generate and mutate programs, looking for crashes, hangs, and other anomalies in a configured target environment, then attempting to minimize and reproduce them. QEMU is a common execution backend, not the only one; automatic bisect also requires corresponding infrastructure and cannot be claimed to be included in any arbitrary run. syzkaller

LTP provides regression test suites for system interfaces, suitable for supplementing verification of kernel and libc interaction behavior. Test case failures may stem from the kernel, environment, permissions, or test prerequisites; specific results and versions should be preserved, rather than just reporting a total failure count. LTP

Fault Injection: Turning Uncontrollable Failures into Verifiable Inputs

Injection points such as memory allocation, I/O, and function return values help check cleanup, retry, and rollback paths. The framework's probability, interval, times, task-filter, and filtering conditions jointly affect triggering; setting failslab/interval to 100 alone does not guarantee that any allocation fails once every 100 times. Fault Injection

Theoretical examples do not need to enable global injection on the current host; a test VM is more convenient for reproduction and recovery.

Fail One by One in Acquisition Order, Check if Only Owned Resources Are Released

Assume initialization acquires context, buffer, registration in sequence, and only after all succeed are the objects published to the caller. Each step's failure should be tested separately, rather than just letting the first allocation fail once and calling it "error path covered."

Below is a deterministic local lifecycle model. The fail_at-th acquisition fails before establishing the resource; ownership of successful objects is transferred to the caller; on exit, publishing is revoked first, then resources are released. It does not enable host failslab, nor does it simulate true device asynchronous callbacks.

class Resources:
    def __init__(self, fail_at=None):
        self.fail_at = fail_at
        self.attempts = 0
        self.live = set()
        self.freed = []
        self.published = None

    def acquire(self, name):
        self.attempts += 1
        if self.attempts == self.fail_at:
            raise MemoryError(name)
        assert name not in self.live
        self.live.add(name)
        return name

    def release(self, name):
        assert self.published is None and name in self.live
        self.live.remove(name)
        self.freed.append(name)

    def create(self):
        acquired = []
        try:
            for name in ("context", "buffer", "registration"):
                acquired.append(self.acquire(name))
        except MemoryError:
            for name in reversed(acquired):
                self.release(name)
            raise
        self.published = tuple(acquired)
        return self.published

    def destroy(self, owned):
        assert self.published is owned
        self.published = None
        for name in reversed(owned):
            self.release(name)

for failure, expected in [(1, []), (2, ["context"]), (3, ["buffer", "context"])]:
    resources = Resources(failure)
    try:
        resources.create()
    except MemoryError:
        pass
    else:
        raise AssertionError("Target failure did not occur")
    assert resources.attempts == failure
    assert not resources.live and resources.published is None
    assert resources.freed == expected

resources = Resources()
owned = resources.create()
assert resources.live == set(owned) and not resources.freed
resources.destroy(owned)
assert not resources.live and resources.published is None
assert resources.freed == ["registration", "buffer", "context"]

Here, we not only check for function return failures, but also verify that the target injection point was indeed reached, subsequent acquisitions did not continue, resources were not leaked or double-freed, and incomplete objects were not visible externally. These are the verifiable invariants of error paths.

Real driver registration operations might start work before returning, or external objects might still hold references; after revoking publishing, new requests must be canceled, in-flight operations and callbacks must be waited for, and then the buffers/contexts they used must be released. Mechanically reversing a few free statements is not enough. The probability and filtering parameters of the injection API are used to trigger paths; resource and visibility checks are used to judge whether that path is correct; the two cannot replace each other.

Move the failure point through acquisition, then compare cleanup policies. Count both retained resources and premature releases while dependents are still live.

Deliverable Reproduction Records

Record the accurate kernel version or commit, .config, boot parameters, module and taint status, as well as minimal input, actual behavior, expected behavior, and original logs. If the fix is only verified under a specific configuration, this boundary should be explicitly stated.

Connecting this evidence to the artifacts of Build and Modules allows confirming that the test ran the target code, not another kernel with the same name.