epoll and Event-Driven Programming

Coverage: epoll_create1/epoll_ctl/epoll_wait → LT vs ET → timerfd/signalfd/eventfd → Comparison with select/poll → Key points of epoll internal implementation Applicable to: Linux 2.6+

Overview

epoll is the standard interface for high-performance event-driven programming on Linux. nginx, HAProxy, and Redis are all built on epoll. The fundamental difference from select/poll is that epoll provides O(1) waiting (returning only active file descriptors), whereas select/poll must scan the entire file descriptor set with O(n) complexity each time.

epoll API

int epfd = epoll_create1(EPOLL_CLOEXEC);  // Create an epoll instance

struct epoll_event ev = {
    .events   = EPOLLIN | EPOLLET,       // Monitor for readability, edge-triggered
    .data.fd  = listen_fd,               // User data (can be a pointer)
};
epoll_ctl(epfd, EPOLL_CTL_ADD, listen_fd, &ev);  // Register the file descriptor

struct epoll_event events[MAX_EVENTS];
int nfds = epoll_wait(epfd, events, MAX_EVENTS, timeout_ms);

for (int i = 0; i < nfds; i++) {
    if (events[i].events & EPOLLIN)
        handle_read(events[i].data.fd);
}

epoll_event.events

EPOLLIN:       Readable (data arrived or connection established)
EPOLLOUT:      Writable (send buffer has space)
EPOLLERR:      Error (automatic, no need to set)
EPOLLHUP:      Hangup (peer closed)
EPOLLRDHUP:    Peer closed the write half of the connection (4.17+)
EPOLLET:       Edge-Triggered (see below)
EPOLLONESHOT:  One-shot (automatically removed after triggering, until re-added via EPOLL_CTL_MOD)
EPOLLEXCLUSIVE: Exclusive wake-up (reduces thundering herd, 4.5+)

LT vs ET: Key Differences

LT (Level-Triggered, default):
  As long as the fd remains in a readable/writable state → epoll_wait returns every time
  → Simple: You can read only part of the data and read the rest next time
  → Risk: If not fully read, epoll_wait will return again next time → loop

ET (Edge-Triggered):
  Notifies only on state transitions (unreadable→readable, unwritable→writable)
  → Efficient: No repeated notifications
  → Requirement: Must read/write until EAGAIN (process everything)
  → Must use O_NONBLOCK (otherwise read may block)

Correct Usage of ET Mode

// In ET mode, you must loop to read:
while (1) {
    ssize_t n = read(fd, buf, sizeof(buf));
    if (n > 0) {
        process(buf, n);
    } else if (n == 0) {
        close(fd);  // EOF
        break;
    } else {
        if (errno == EAGAIN || errno == EWOULDBLOCK)
            break;  // All data read
        // handle error
        break;
    }
}

timerfd / signalfd / eventfd: epoll Partners

// Unified event loop: Convert all asynchronous events into file descriptors → managed uniformly by epoll

int epfd = epoll_create1(0);

// 1. Network socket
epoll_add(epfd, listen_fd, EPOLLIN);

// 2. timerfd: Timer
int tfd = timerfd_create(CLOCK_MONOTONIC, TFD_NONBLOCK);
epoll_add(epfd, tfd, EPOLLIN);

// 3. signalfd: Signals
sigset_t mask; sigaddset(&mask, SIGINT); sigaddset(&mask, SIGTERM);
sigprocmask(SIG_BLOCK, &mask, NULL);
int sfd = signalfd(-1, &mask, SFD_NONBLOCK);
epoll_add(epfd, sfd, EPOLLIN);

// 4. eventfd: Inter-thread notification
int efd = eventfd(0, EFD_NONBLOCK);
epoll_add(epfd, efd, EPOLLIN);

// Single thread: epoll_wait → Unified handling of all events
while (1) {
    int nfds = epoll_wait(epfd, events, MAX, -1);
    for (int i = 0; i < nfds; i++) {
        if      (events[i].data.fd == listen_fd) accept();
        else if (events[i].data.fd == tfd)       handle_timer();
        else if (events[i].data.fd == sfd)       handle_signal();
        else if (events[i].data.fd == efd)       handle_notification();
        else                                      handle_client(events[i].data.fd);
    }
}

Comparison with select/poll

epoll Internal Implementation

// fs/eventpoll.c
// An epoll instance contains:
//   1. Red-black tree (rbr): List of registered fds (epoll_ctl ADD → insert)
//   2. Ready list (rdllist): Active fds (returned by epoll_wait)
//
// When an event arrives → driver/protocol stack calls ep_poll_callback:
//   → Check if event matches? → Add to rdllist
//   → If there are waiters (epoll_wait) → Wake up

References

• man: epoll(7), epoll_create1(2), epoll_ctl(2), epoll_wait(2)
• Source code: fs/eventpoll.c
• LWN: "The epoll API", "Edge triggered epoll"

Keywords: epoll, EPOLLET, EPOLLONESHOT, edge-triggered, timerfd, signalfd, eventfd, event loop