Signals and ptrace: From Asynchronous Notifications to Controlled Stops

kill() returns success, but the handler hasn't executed yet; several child processes exit simultaneously, but the parent only receives one SIGCHLD. Understanding these phenomena requires separating event generation, pending state, and actual delivery.

ptrace adds another layer of stop protocols: the tracer can observe the tracee at specific points, but must identify the reason for the stop to resume it correctly.

1. Generation, Pending, and Delivery are Three Different Stages

Successful return from the send interface usually means the signal request passed checks and was accepted by the kernel; it is not confirmation that the receiver has executed the handler. The receiving thread might be blocking that signal, not yet scheduled, or in a kernel path where it cannot respond temporarily. If your application needs to confirm that an operation is complete, you should design an explicit acknowledgment instead of treating the return value of kill() as a completion notification.

First, determine who owns the state:

kill(pid, sig) usually sends to the process; pthread_kill() specifies a thread. When the target thread blocks the signal, a thread-directed signal cannot be redirected to someone else.

When reading the diagram above, first look at A's mask, then at B and C's eligibility to receive: A blocks SIGUSR1, so B or C can be selected, but they won't each execute once. If all three threads block it, the signal remains pending until there is a suitable delivery opportunity.

Change the send target and masks, then try delivery on threads. Shared and private pending are shown separately; manually selecting threads is for comparing possible receivers and does not guarantee the kernel's selection order.

To handle signals centrally, you should block the target set before creating worker threads, so new threads inherit the mask, and a dedicated thread consumes them synchronously. This way, the reception location is arranged by the program, without relying on which thread happens to be selected.

SIGKILL and SIGSTOP cannot have handlers installed or be ignored; attempting to modify their disposition with sigaction() will fail; adding them to the mask set also does not truly block them. They are still constrained by execution state: SIGKILL does not mean the CPU arbitrarily discards kernel stacks holding locks; the task may have to wait until it can reach a path that handles fatal signals. Wait designs like TASK_KILLABLE determine whether the corresponding wait can be interrupted by fatal signals. signal(7)

2. Why Standard Signals Cannot Be Used as Event Counters

If you block SIGUSR1 and send it five times consecutively, do not expect the handler to execute five times when you unblock it: when a standard signal of the same number is already pending, subsequent generation usually merges. It retains the state that "this type of event needs attention."

SIGCHLD: One Notification, Recycling May Take Multiple Steps

The correct way to consume SIGCHLD can be derived from this: after one notification arrives, you should repeatedly call the appropriate non-blocking wait to retrieve the status of all currently recyclable child processes until there are no more results. If you only recycle one, the others may have already exited, without independent subsequent SIGCHLDs to remind you. Even if signals merge, the exit status is still saved by the kernel's child process wait mechanism; the notification and the fact record belong to different objects.

Realtime Signals: Ordered Queuing Still Requires Handling Resource Limits

Realtime signals can carry values via sigqueue(), and multiple instances of the same signal are delivered in the order they were sent. The diagram retains the three values 10, 11, and 12, but the queue capacity is limited, and sending may fail with EAGAIN.

Different realtime signal numbers also have priority order; you cannot merge all numbers into a single global FIFO. When using SIGRTMIN + n, you should check SIGRTMAX; glibc may reserve realtime signals for the thread library, and SIGRTMIN cannot be hardcoded as 34. signal(7)

After sending consecutively, retrieve the two types of signals separately. The diagram deliberately uses a small capacity to expose send failures; it is not Linux's fixed queue limit.

Experiment: One Pending and Three Queued Values

Below, we block signals within a single-threaded process to verify the merging of five standard signals and the ordered retrieval of three realtime signals of the same number. It does not install asynchronous handlers, nor does it send signals to other processes. Save as pending.c, compile with cc -std=c11 -O2 -Wall -Wextra -Werror pending.c -o pending, and do not define NDEBUG.

#define _POSIX_C_SOURCE 200809L
#include <assert.h>
#include <errno.h>
#include <signal.h>
#include <stdio.h>
#include <time.h>
#include <unistd.h>

int main(void) {
    const int rt = SIGRTMIN;
    assert(rt <= SIGRTMAX);
    sigset_t all, old, standard, realtime;
    sigemptyset(&all);
    sigaddset(&all, SIGUSR1);
    sigaddset(&all, rt);
    assert(sigprocmask(SIG_BLOCK, &all, &old) == 0);
    struct sigaction sa = { .sa_handler = SIG_DFL };
    sigemptyset(&sa.sa_mask);
    assert(sigaction(SIGUSR1, &sa, NULL) == 0);
    assert(sigaction(rt, &sa, NULL) == 0);
    for (int i = 0; i < 5; ++i)
        assert(kill(getpid(), SIGUSR1) == 0);
    for (int i = 10; i < 13; ++i) {
        union sigval value = { .sival_int = i };
        assert(sigqueue(getpid(), rt, value) == 0);
    }
    sigemptyset(&standard);
    sigaddset(&standard, SIGUSR1);
    sigemptyset(&realtime);
    sigaddset(&realtime, rt);
    struct timespec zero = {0};
    siginfo_t info;
    assert(sigtimedwait(&standard, &info, &zero) == SIGUSR1);
    errno = 0;
    assert(sigtimedwait(&standard, &info, &zero) == -1 && errno == EAGAIN);
    for (int expected = 10; expected < 13; ++expected) {
        assert(sigtimedwait(&realtime, &info, &zero) == rt);
        assert(info.si_code == SI_QUEUE);
        assert(info.si_value.sival_int == expected);
    }
    errno = 0;
    assert(sigtimedwait(&realtime, &info, &zero) == -1 && errno == EAGAIN);
    assert(sigprocmask(SIG_SETMASK, &old, NULL) == 0);
    puts("standard: 1 pending; realtime payloads: 10, 11, 12");
}

Zero timeout means only matching signals that are already pending at this moment are taken; if none, it returns EAGAIN. The program consumes all signals it sent before restoring the original mask. The experiment assumes no external senders and sufficient queue quota; production programs must handle sigqueue resource shortages and cannot rely on assertions for recovery strategies. sigtimedwait(2)

3. How Handlers Enter and How to Return to the Interrupted Position

For deliveries that require calling a user handler, the kernel prepares a signal frame at the appropriate user-mode return boundary: it saves the user registers, mask, and relevant context needed for restoration, selects the main user stack or a suitable alternate signal stack, and changes the user execution entry point to the handler. It runs in the receiving thread's user-mode context, not as a kernel interrupt handler, nor as a new thread temporarily created by the kernel.

After the handler returns, it enters rt_sigreturn via the applicable user-mode return trampoline. The kernel verifies and restores the corresponding state, continuing the interrupted execution. Frame layout and extended register details vary by architecture and ABI; you cannot arbitrarily interpret all architectures using a single assumed fixed-size C struct. Default termination, stop, or ignore actions also do not need to follow this "call user handler" path.

Select the execution position of the main program, then deliver a signal at this point. The TID remains unchanged; observe this entry and return from the PC, temporarily saved frame, and mask.

Why Handlers Cannot Print Logs Normally

Suppose the original thread is executing stdio, and its internal state has not finished updating. If the signal handler inserts execution at this time, and the handler calls printf, it may re-enter the same unfinished state; if it tries to acquire a lock already held by the original thread, it might wait for itself to unlock.

This is the source of asynchronous signal safety restrictions. Handlers should only perform allowed operations, such as setting a volatile sig_atomic_t flag, or writing to a notification channel with non-blocking/capacity policies, and then hand it over to the normal execution flow. signal-safety(7)

Moving Complex Processing Back to the Normal Thread

sig_atomic_t is suitable for signal handling communication within a specified range, but it does not automatically provide the release/acquire protocol required for multi-threaded data publication. If the requirement is indeed to centrally handle asynchronous requests like exit or reload, you can use sigwaitinfo() or Linux signalfd(), arranging the processing in the normal thread execution flow; in multi-threaded environments, ensure relevant threads consistently block these signals to avoid delivery by other threads first. SIGSEGV generated by illegal memory access or other synchronization exceptions cannot mechanically apply this centralized consumption mode to recover the program.

4. EINTR, Restart, and the Race Condition of "Check Then Sleep"

After a wait is interrupted by a signal, first determine what the interface actually returned:

Short returns may have already completed part of the work and cannot be uniformly resubmitted as "did nothing."

Timeout Retry: Retain the Original Deadline

SA_RESTART is not a global "never EINTR" switch. For example, nanosleep and various wait interfaces have their own interruption rules. Loops with timeouts must retain the original deadline: if you wait the full 5 seconds every time you are interrupted, frequent signals may cause the total wait to far exceed 5 seconds. Use a monotonic clock to calculate the remaining budget, or use interfaces that support absolute deadlines, to maintain end-to-end time limits.

Wait Race Condition: Flag is True, But pause() Just Started

Consider this sequence: the main program checks the flag and finds it false; the signal handler then sets it true; the main program finally calls pause(). The notification has passed, but the main program will wait for the next signal.

Usually, you should first block the target signal and check the flag, then use sigsuspend() to atomically replace the mask and wait; re-check the condition after returning. It protects the same window as Condition Wait: from checking the condition to entering sleep. signal(7)

5. ptrace: Identify the Stop Reason First, Then Decide How to Resume

The tracee in ptrace is a thread in Linux. Establishing a tracing relationship with a TID does not mean other threads stop simultaneously; the debugger must also manage thread discovery, creation events, and the stop state of each thread to construct a consistent view of the entire process.

PTRACE_ATTACH establishes the relationship and sends a stop request, but the return of the call does not mean the target has entered a checkable stop; the tracer should wait for the corresponding state. PTRACE_SEIZE does not stop the target itself; you can request a stop later with PTRACE_INTERRUPT. The tracing loop cannot simply be "wait returns, then read registers," because wait may also report final exit or other events.

You can establish the most basic classification using the table below, then handle options, thread groups, and kernel version differences:

Below, compare resume decisions using independent stop snapshots. Only real SIGUSR1 delivery stops provide the option to forward; the target thread's handler result will be retained after resume.

Resume Decision: Syscall Stops and Signal Delivery Are Different

PTRACE_SYSCALL lets the target continue to the next applicable syscall stop. If you simply maintain a boolean value, alternating between "entry/exit" at each stop, you may misalign when encountering signals, exec, seccomp events, or other stops. When supported, you can use PTRACE_GET_SYSCALL_INFO to get entry, exit, or seccomp info; you still must identify the stop type first and cannot blindly read syscall numbers for all wait results.

A common mistake is injecting all SIGTRAPs back on resume. Single-step or syscall events requested by the debugger themselves do not all represent that the target should have received a SIGTRAP; incorrect injection may cause a normally functioning program to execute a default termination action. Conversely, suppressing all real business signals also changes target behavior. The tracer's resume decision is part of program semantics. ptrace(2)

6. Reading Memory and seccomp: Observable Does Not Mean Consistent or Safe

PTRACE_PEEKDATA may legally read the value -1. To judge failure, you should first clear errno, then combine the return value and errno, rather than assuming the address is illegal just because you see -1. Using /proc/PID/mem or process_vm_readv() does not bypass access control; credentials, dumpable, user namespaces, Yama, or other LSM policies all affect allowed operations.

Consistency: Stopping One Thread, Other Writers Are Still Active

Passing permissions does not mean you read a consistent snapshot. Suppose the tracer only stops thread A, while thread B is still modifying a linked list; reading head then next may skip two structural changes. Even if all normal threads are stopped, other processes sharing memory or devices may still change the data. Analysis tools should clarify whether they need best-effort sampling or a consistent snapshot provided by program cooperation.

Interface Boundaries: TRACE and USER_NOTIF

seccomp's SECCOMP_RET_TRACE can generate seccomp trace events under corresponding ptrace options; SECCOMP_RET_USER_NOTIF hands them over to user-mode supervisors via a dedicated listener fd, which is not a ptrace interface.

The two can appear in the same complex system, but notification collection, response, and task lifecycle handling each have their own protocols. seccomp filters

User-mode supervisors reading pointers in system call arguments and then letting the original system call continue face a typical TOCTOU: after the check, other threads may modify the memory pointed to by the pointer. Merely "checking the path string" is not enough to guarantee the target ultimately operates on the same object.

ptrace itself is not a trusted sandbox boundary; responsibilities for observation, proxy execution, object references, and access control should be designed separately, rather than directly deriving "can safely authorize any system call" from "can intercept system calls." seccomp user notification

7. Next-Level Questions to Ask When Locating Issues Quickly

These entry points connect "what happened" to specific states; then look for the location of state changes in the source code.

Reading the kernel implementation can start from the generation/delivery path in kernel/signal.c, the relationship and permission management in kernel/ptrace.c, and then to architecture signal frames and ptrace register operations. The internal signal set and user-space sigset_t layout are not the same ABI, and there is no need to copy a struct definition that is easy to become outdated with version updates to understand the mechanism.

Process and thread resource relationships return to Lifecycle, and CPU competition after signal wake-up returns to the Scheduler.