1. The C10K Problem and Blocking I/O

1.1 Thread-per-Connection

In the late 90s, the standard model for a web server (like early Apache) was "thread-per-connection". When a client connected, the server spawned a new OS thread (or process) to handle it. Inside that thread, the code called read() on the network socket to get the HTTP request.

By default, sockets in Linux are blocking. If the client has a slow 56k modem and hasn't sent the full request yet, the read() system call puts the thread to sleep. The OS removes the thread from the CPU scheduler and places it in a wait queue. When data finally arrives, the OS wakes the thread up.

This model is simple to program, but it fails catastrophically at scale. If you have 10,000 concurrent users, you need 10,000 threads. Each thread requires an 8MB stack. That is 80GB of RAM just for sleeping threads doing absolutely nothing. Furthermore, the CPU wastes immense amounts of time context-switching between 10,000 threads trying to figure out which one actually has work to do.

2. The Dark Ages: select() and poll()

To solve the CPU melting problem, Unix introduced the select() system call. You give the kernel an array of File Descriptors (FDs) representing your sockets, and you tell the kernel: "Put my single thread to sleep until at least one of these FDs has data to read."

This was a revolution. One thread could multiplex thousands of connections. But select() and its successor poll() had severe algorithmic flaws:

• O(N) Copying: Every time you call select(), you have to copy the entire array of 10,000 FDs from user-space memory into kernel-space memory.
• O(N) Scanning: When select() returns, it only tells you "Hey, 3 sockets are ready!" It doesn't tell you which ones. Your user-space code must loop through all 10,000 FDs in a massive O(N) loop to figure out which 3 are active.

As web traffic exploded in the 2000s, select() became the primary bottleneck. The CPU was spending all its time copying arrays and iterating over 9,997 idle sockets just to find the 3 active ones.

3. Enter epoll: The Event-Driven Kernel

3.1 The Three Pillars of epoll

In Linux 2.5.44, epoll was introduced. It fundamentally changed the architecture from a "stateless polling" model to a "stateful event-driven" model within the kernel itself. Instead of passing an array of 10,000 FDs on every single system call, you register FDs with the kernel once. The kernel keeps track of them.

epoll exposes three distinct system calls:

• epoll_create(): Creates an epoll instance in the kernel. This allocates a Red-Black Tree to store the FDs you want to monitor, and a double-linked "Ready List" to hold the events that have occurred.
• epoll_ctl(): Used to ADD, MODIFY, or DELETE file descriptors from the Red-Black Tree. Because it's an RB-Tree, lookups and inserts are O(log N). You only call this when a new client connects or disconnects.
• epoll_wait(): The heart of the event loop. This call puts your thread to sleep. It wakes up when events are present in the Ready List. Crucially, it returns only the active FDs (an O(1) operation relative to total connections). No more scanning 10,000 idle sockets!

4. Worked Trace: From NIC Interrupt to epoll_wait

Let's trace a packet arriving at your server to see exactly how it wakes up your Node.js or Python asyncio event loop.

/* A simplified view of a C Event Loop using epoll */
int epoll_fd = epoll_create1(0);

// Add the server listening socket
struct epoll_event event;
event.events = EPOLLIN;
event.data.fd = server_socket;
epoll_ctl(epoll_fd, EPOLL_CTL_ADD, server_socket, &event);

struct epoll_event events[MAX_EVENTS];

while (1) {
    // Sleep until hardware interrupts populate the Ready List
    int num_ready = epoll_wait(epoll_fd, events, MAX_EVENTS, -1);
    
    for (int i = 0; i < num_ready; i++) {
        if (events[i].data.fd == server_socket) {
            // Accept new client connection, add to epoll
            int client_fd = accept(server_socket, NULL, NULL);
            set_nonblocking(client_fd);
            struct epoll_event ev = { .events = EPOLLIN, .data.fd = client_fd };
            epoll_ctl(epoll_fd, EPOLL_CTL_ADD, client_fd, &ev);
        } else {
            // Handle HTTP request on existing client (O(1) lookup!)
            handle_client_data(events[i].data.fd);
        }
    }
}

5. Level-Triggered vs Edge-Triggered (EPOLLET)

epoll supports two modes of operation, and misunderstanding them is the source of endless production bugs in custom C/C++ servers.

• Level-Triggered (Default): As long as there is unread data in the socket's receive buffer, epoll_wait() will keep returning that FD immediately. It is forgiving. If you read 1KB but 2KB arrived, the next loop iteration will hand you the FD again.
• Edge-Triggered (EPOLLET): epoll_wait() only returns the FD once, precisely when the socket state changes from "empty" to "has data". If you read 1KB, and leave 1KB in the buffer, and go back to sleep, you will sleep forever. The kernel assumes you handled it.

6. Frequently Asked Questions