TCP Sockets and epoll Event Loops Under the Hood: From Hardware Interrupts to io_uring

1. The Architecture of Network Concurrency

To process network requests, a server must perform three operations for every client: accept the TCP handshake connection, read incoming bytes from a socket file descriptor (FD), and write back an HTTP response. In traditional multi-threaded servers (like early Apache 1.3), each client connection was assigned to a dedicated operating system thread.

When a thread calls read(fd, buf, len) on a standard blocking socket, the CPU suspends the thread if no network packets have arrived. The OS kernel moves the thread from the CPU runqueue to the socket's wait queue. When 10,000 idle HTTP clients keep connections open (keep-alive), 10,000 threads sit in memory. With a typical 8MB default stack per thread, this consumes 80GB of RAM just for sleeping threads! Context switching across 10,000 threads causes devastating L1/L2 cache thrashing.

2. Non-Blocking I/O & The Evolution of Multiplexing

To eliminate thread overhead, sockets can be set to non-blocking mode using fcntl(fd, F_SETFL, O_NONBLOCK). In non-blocking mode, if no data is available in the socket's receive queue, read() does not sleep; it immediately returns -1 with errno = EAGAIN (or EWOULDBLOCK).

However, repeatedly executing read() in a while(true) loop across 10,000 non-blocking sockets burns 100% CPU on useless user-to-kernel context switches. To solve this, Unix introduced I/O Multiplexing system calls: select(), poll(), and eventually epoll.

2.1 Why select() and poll() Scale Terribly

In both select() and poll(), the application passes an array of monitored FDs to the kernel on every single call. The system call performs two costly steps:

• Memory Copying ($O(N)$): Copies the entire array of $N$ file descriptors from user-space RAM into kernel memory space.
• Linear Scanning ($O(N)$): The kernel loops through all $N$ file descriptors to check their read/write status. When it returns to user space, it only returns the total count of active sockets. The application must then loop through all $N$ sockets in user space to find which ones actually received data!

If a server manages 10,000 connections but only 3 receive network packets during a tick, `select()` wastes compute cycles scanning 9,997 idle sockets on every single iteration!

3. Internal Architecture of epoll

Linux epoll converts the stateless polling model into an internal stateful kernel subsystem. Instead of passing the socket list on every iteration, you register sockets with the kernel once. The kernel maintains these sockets across system calls using two primary data structures within an eventpoll object:

3.1 The Three epoll System Calls

• epoll_create1(int flags): Allocates a new struct eventpoll object in kernel memory, initializing the Red-Black tree root and ready list. Returns a file descriptor representing the epoll instance.
• epoll_ctl(int epfd, int op, int fd, struct epoll_event *event): Modifies the Red-Black tree. Operates with EPOLL_CTL_ADD, EPOLL_CTL_MOD, or EPOLL_CTL_DEL in $O(\log N)$ time. Registers an internal kernel callback ep_poll_callback() for the socket.
• epoll_wait(int epfd, struct epoll_event *events, int maxevents, int timeout): Puts the calling thread to sleep if rdllist is empty. When events arrive, it copies up to maxevents active descriptors from rdllist directly to user space in $O(1)$ time!

4. Step-by-Step Hardware Packet Trace: From Ethernet NIC to epoll_wait

Let's trace a physical TCP network packet as it travels from the physical Ethernet cable to waking up a user-space event loop:

5. Level-Triggered (LT) vs. Edge-Triggered (ET / EPOLLET) Modes

Understanding the distinction between Level-Triggered and Edge-Triggered modes is vital for writing bug-free high-performance network servers:

5.1 Level-Triggered Mode (Default)

In Level-Triggered mode, `epoll_wait()` will continue to report a file descriptor on every subsequent call as long as there is unread data remaining in the socket's kernel receive buffer. It is forgiving. If 4096 bytes arrive and your application only reads 1024 bytes, the next call to `epoll_wait()` will immediately return the socket again.

5.2 Edge-Triggered Mode (EPOLLET)

In Edge-Triggered mode, `epoll_wait()` only notifies the application when a state change occurs (e.g., when the socket receive buffer transitions from empty to non-empty). If 4096 bytes arrive, and you only read 1024 bytes and call `epoll_wait()` again, the application will block indefinitely because no new network edge event occurred!

/* Correct Edge-Triggered (EPOLLET) Read Loop Pattern in C */
void handle_edge_triggered_read(int client_fd) {
    char buffer[4096];
    while (1) {
        ssize_t bytes_read = read(client_fd, buffer, sizeof(buffer));
        if (bytes_read > 0) {
            process_payload(buffer, bytes_read);
        } else if (bytes_read == 0) {
            /* Client disconnected */
            close(client_fd);
            break;
        } else {
            if (errno == EAGAIN || errno == EWOULDBLOCK) {
                /* Fully drained kernel receive buffer! Safe to return to epoll_wait */
                break;
            }
            /* Actual socket error */
            perror("read error");
            close(client_fd);
            break;
        }
    }
}

6. Deep Dive into TCP State Transitions and Socket Buffers

A network socket in Linux is represented by struct sock and struct tcp_sock in <net/sock.h>. During its lifecycle, a socket traverses the standard TCP State Machine:

LISTEN -> SYN_RCVD -> ESTABLISHED -> FIN_WAIT_1 -> FIN_WAIT_2 -> TIME_WAIT -> CLOSED

Every active socket allocates two ring buffers in kernel RAM:

• sk_receive_queue: Stores incoming TCP segments wrapped in sk_buff structures. The memory size is bounded by sysctl_tcp_rmem. When full, the TCP layer drops incoming packets and stops advancing the TCP Window Size (Flow Control).
• sk_write_queue: Holds outbound payload buffers waiting to be transmitted over Ethernet and acknowledged by the client's TCP ACK.

7. Deep Kernel C Struct Annotations: struct eventpoll and struct epitem

To fully grasp epoll's internal implementation, let's examine the actual Linux kernel source code definitions in fs/eventpoll.c:

/* Internal structure representing an epoll instance in fs/eventpoll.c */
struct eventpoll {
    spinlock_t lock;             /* Spinlock protecting rdllist access */
    struct mutex mtx;            /* Mutex protecting Red-Black tree modifications */
    wait_queue_head_t wq;        /* Wait queue for epoll_wait() sleeping threads */
    wait_queue_head_t poll_wait; /* Wait queue for nested epoll polling */
    struct list_head rdllist;    /* Doubly-linked list of ready file descriptors */
    struct rb_root_cached rbr;   /* Red-Black tree root containing all registered FDs */
    struct epitem *ovflist;      /* Overflow list for events arriving during user transfer */
};

/* Structure representing each monitored file descriptor in epoll */
struct epitem {
    struct rb_node rbn;          /* Red-Black tree node links */
    struct list_head rdllink;    /* Ready list doubly-linked list node */
    struct epoll_filefd fffd;    /* Structural pair: struct file* + file descriptor int */
    struct eventpoll *ep;        /* Pointer to parent eventpoll container */
    struct epoll_event event;    /* Registered events mask and user data union */
};

8. Event Loop Design Patterns: Reactor, Proactor, and Leader-Follower

High-performance networking libraries organize their event loops around classic concurrency patterns:

• Reactor Pattern (Nginx, Netty, Node.js): A single event loop thread monitors FDs via `epoll_wait()`. When an FD becomes ready, the reactor thread dispatches the event to a synchronous handler or worker pool.
• Proactor Pattern (Windows IOCP, io_uring): The application initiates an asynchronous I/O operation. A kernel worker executes the I/O in the background and notifies the proactor event loop when the operation completes.
• Leader-Follower Pattern: Multiple worker threads share a single epoll instance. One thread acts as the "Leader" waiting inside `epoll_wait()`. When an event occurs, the Leader promotes a "Follower" to become the new Leader, while the old Leader processes the event.

9. Zero-Copy Networking: sendfile(), splice(), and MSG_ZEROCOPY

In a standard web server sending static files over HTTP, a traditional read/write pipeline causes four memory copies and four user/kernel context switches per chunk:

1. Disk DMA -> Kernel Page Cache
2. Kernel Page Cache -> User Space Buffer (read sys_call)
3. User Space Buffer -> Kernel Socket Buffer (write sys_call)
4. Kernel Socket Buffer -> NIC DMA Ring

By using Zero-Copy Networking system calls in combination with epoll, high-throughput systems bypass user-space memory entirely:

/* Zero-Copy File Transmission using sendfile() */
ssize_t sent = sendfile(client_fd, file_fd, &offset, count);
/* Data transfers directly: Disk DMA -> Kernel Page Cache -> NIC DMA Ring! */

With sendfile() or splice(), data transfers directly from the kernel Page Cache to the NIC DMA buffer without passing through user-space memory, reducing CPU overhead by up to 70%!

10. Advanced Socket Option Tuning: TCP_NODELAY, TCP_CORK, and SO_LINGER

Fine-tuning socket options changes how the kernel packages data packets over the network:

• TCP_NODELAY: Disables Nagle's algorithm. Nagle's algorithm buffers small outgoing packets (e.g. 20-byte TCP payloads) until a full Segment (MSS) is formed or a TCP ACK is received. Setting `TCP_NODELAY` forces the kernel to transmit packets immediately, reducing latency for RPC services (gRPC, Redis).
• TCP_CORK: The exact opposite of `TCP_NODELAY`. `TCP_CORK` instructs the kernel to append outgoing data into a single combined packet until the buffer is full or uncorked, maximizing throughput for static file downloads.
• SO_LINGER: Controls how `close()` handles unsent data in the socket write queue. Setting `l_onoff = 1` and `l_linger = 0` forces an immediate `RST` packet, bypassing the standard `TIME_WAIT` state to prevent socket exhaustion during heavy benchmark bursts.

11. Kernel Slab Allocators: Memory Footprint of 100,000 Connections

When operating a machine with 100,000 idle TCP connections, memory overhead is dominated by kernel slab allocations (`kmem_cache`):

By tuning tcp_rmem and tcp_wmem lower bound parameters, systems engineers can reduce per-connection RAM from 6.2KB down to under 3.5KB, allowing a single 64GB node to maintain over 10 million concurrent WebSocket connections!

12. eBPF XDP (eXpress Data Path) Packet Filtering Integration

For ultra-high-throughput firewalls and load balancers, even NAPI SoftIRQ processing is too slow. eBPF XDP allows developers to attach custom BPF programs directly to the NIC DMA ring driver before kernel `sk_buff` allocation occurs:

/* Simple XDP BPF program dropping malicious SYN floods at driver level */
SEC("xdp")
int xdp_syn_filter(struct xdp_md *ctx) {
    void *data = (void *)(long)ctx->data;
    void *data_end = (void *)(long)ctx->data_end;
    struct ethhdr *eth = data;
    
    if ((void *)(eth + 1) > data_end) return XDP_PASS;
    if (eth->h_proto != __constant_htons(ETH_P_IP)) return XDP_PASS;

    struct iphdr *iph = (void *)(eth + 1);
    if ((void *)(iph + 1) > data_end) return XDP_PASS;

    /* Drop IP packets from specific malicious CIDR instantaneously */
    if (iph->saddr == __constant_htonl(0x0A000005)) {
        return XDP_DROP; /* Zero allocation drop! */
    }
    return XDP_PASS;
}

By dropping malicious traffic at the XDP layer, the CPU processes up to 24 million packets per second per core, preventing SYN flood attacks from overwhelming socket receive queues and epoll ready lists!

13. Architectural Comparison: Nginx vs HAProxy vs Envoy vs Caddy

Every major web proxy handles I/O multiplexing differently based on its architecture:

• Nginx: Uses single-threaded process workers (one per CPU core). Each worker executes an independent Edge-Triggered `epoll` loop. Outstanding performance for static file serving and reverse proxying.
• HAProxy: Uses a single process with multiple threads sharing a single multi-threaded event loop engine. Uses Level-Triggered `epoll` with optimized batching for sub-millisecond HTTP/TCP load balancing.
• Envoy: Uses a Thread-Local Reactor model. Each worker thread runs its own `libevent` (`epoll`) instance. Locks are strictly avoided between threads using thread-local storage and event passing.
• Caddy (Go): Relies on Go's `netpoller` runtime. Automatically scales goroutines across CPU cores, delegating low-level epoll management to Go's internal runtime scheduler.

14. Complete Production C Code: Multi-Threaded epoll Web Server

Below is a production-ready C implementation demonstrating a non-blocking epoll event loop handling HTTP requests:

#include <stdio.h>
#include <stdlib.h>
#include <string.h>
#include <unistd.h>
#include <fcntl.h>
#include <errno.h>
#include <sys/socket.h>
#include <netinet/in.h>
#include <sys/epoll.h>

#define MAX_EVENTS 64
#define PORT 8080

static int set_nonblocking(int fd) {
    int flags = fcntl(fd, F_GETFL, 0);
    if (flags == -1) return -1;
    return fcntl(fd, F_SETFL, flags | O_NONBLOCK);
}

int main() {
    int listen_fd = socket(AF_INET, SOCK_STREAM, 0);
    int opt = 1;
    setsockopt(listen_fd, SOL_SOCKET, SO_REUSEADDR, &opt, sizeof(opt));

    struct sockaddr_in addr = {
        .sin_family = AF_INET,
        .sin_addr.s_addr = INADDR_ANY,
        .sin_port = htons(PORT)
    };
    bind(listen_fd, (struct sockaddr *)&addr, sizeof(addr));
    listen(listen_fd, SOMAXCONN);
    set_nonblocking(listen_fd);

    int epfd = epoll_create1(0);
    struct epoll_event ev = { .events = EPOLLIN | EPOLLEXCLUSIVE, .data.fd = listen_fd };
    epoll_ctl(epfd, EPOLL_CTL_ADD, listen_fd, &ev);

    struct epoll_event events[MAX_EVENTS];
    printf("Server listening on port %d...
", PORT);

    while (1) {
        int nfds = epoll_wait(epfd, events, MAX_EVENTS, -1);
        for (int n = 0; n < nfds; ++n) {
            if (events[n].data.fd == listen_fd) {
                /* Accept new client */
                int conn_sock = accept(listen_fd, NULL, NULL);
                if (conn_sock != -1) {
                    set_nonblocking(conn_sock);
                    struct epoll_event client_ev = { .events = EPOLLIN | EPOLLET, .data.fd = conn_sock };
                    epoll_ctl(epfd, EPOLL_CTL_ADD, conn_sock, &client_ev);
                }
            } else {
                /* Handle data on non-blocking client */
                char buf[512];
                ssize_t count = read(events[n].data.fd, buf, sizeof(buf));
                if (count > 0) {
                    const char *resp = "HTTP/1.1 200 OK

Content-Length: 13



Hello, World!";
                    write(events[n].data.fd, resp, strlen(resp));
                }
                close(events[n].data.fd);
            }
        }
    }
    return 0;
}

15. High-Performance Multithreading: SO_REUSEPORT and EPOLLEXCLUSIVE

In early multi-process servers (such as multi-worker Nginx), multiple worker processes opened individual epoll instances monitoring the exact same listening server socket. When a new client performed a TCP handshake, the kernel woke up ALL sleeping worker processes—a phenomenon known as the Thundering Herd Problem. One worker successfully executed `accept()`, while the remaining $N-1$ workers received `EAGAIN` and went back to sleep, wasting massive CPU cycles.

15.1 Modern Kernel Solutions

• EPOLLEXCLUSIVE (Linux 4.5+): When registering a listening socket with `epoll_ctl()`, setting `EPOLLEXCLUSIVE` ensures that the kernel wakes up only a single sleeping epoll thread instead of all of them.
• SO_REUSEPORT (Linux 3.9+): Allows multiple independent worker processes to bind to the exact same TCP port. The kernel maintains separate listening sockets per worker and automatically load-balances incoming TCP handshakes across processes using a 4-tuple hash (`sip`, `sport`, `dip`, `dport`), eliminating lock contention entirely!

16. Modern Asynchronous I/O with liburing in C

To write high-throughput code using io_uring without dealing with raw memory offsets, developers use the official liburing library:

/* Submitting non-blocking read via liburing */
#include <liburing.h>

struct io_uring ring;
io_uring_queue_init(256, &ring, 0);

struct io_uring_sqe *sqe = io_uring_get_sqe(&ring);
io_uring_prep_read(sqe, client_fd, buffer, sizeof(buffer), 0);
io_uring_submit(&ring);

/* Retrieve completion event without system call */
struct io_uring_cqe *cqe;
io_uring_wait_cqe(&ring, &cqe);
if (cqe->res > 0) {
    process_payload(buffer, cqe->res);
}
io_uring_cqe_seen(&ring, cqe);

17. Modern Network Engine Hardware Offloading (TSO, LRO, RSS)

High-end 100GbE enterprise network cards relieve the CPU of TCP packet processing tasks using hardware offloading features:

• TCP Segmentation Offload (TSO): The CPU passes a single giant 64KB TCP payload to the NIC. The Ethernet card's onboard ASIC splits the payload into individual 1500-byte Ethernet frames in hardware, saving thousands of CPU instructions per megabyte.
• Large Receive Offload (LRO): The NIC ASIC coalesces multiple incoming TCP packets into a single contiguous memory buffer before delivering it to the kernel, reducing the number of `ep_poll_callback()` invocations.
• Receive Side Scaling (RSS): The NIC uses hardware hashing to distribute incoming network flows across multiple RX DMA rings mapped to different CPU cores, enabling parallel SoftIRQ processing.

18. Language Runtime Event Loops: Python Asyncio & Rust Tokio Under the Hood

How do higher-level programming language runtimes bridge their async/await syntax to epoll?

/* Python asyncio selector_events.py snippet abstraction */
class _UnixSelectorEventLoop(base_events.BaseEventLoop):
    def _poll(self, timeout):
        # Internally delegates to select.epoll() in C
        event_list = self._selector.select(timeout)
        for key, mask in event_list:
            fileobj, callback = key.fileobj, key.data
            callback(fileobj, mask)

In Rust, the tokio runtime sits on top of mio (Metal I/O). mio wraps Linux epoll, macOS kqueue, and Windows IOCP into a zero-cost C-speed abstraction layer. When a Tokio task awaits a socket read, Tokio registers the raw socket handle with `mio::Poll` and yields the green thread, resuming execution when `epoll_wait()` fires!

19. Production Kernel Tuning Parameters

High-concurrency servers require sysctl network tuning to prevent dropped connections during TCP handshake spikes:

20. Operational Diagnostics & Diagnostic Tooling Runbook

When debugging high-concurrency servers in production, use native Linux diagnostic commands:

# Inspect socket queues and active connection states
ss -tulpn -o state established

# Count open file descriptors for target process
lsof -p <PID> | wc -l

# Trace epoll system call activity on running process
sudo strace -ff -e trace=epoll_create1,epoll_ctl,epoll_wait -p <PID>

# Capture raw network packets on specific interface and port
sudo tcpdump -i eth0 -n "port 8080 and (tcp-syn or tcp-fin or tcp-rst)"

21. Architectural Decision Matrix & Performance Runbook

When designing high-scale networking software, use the following decision matrix to choose the optimal Linux I/O paradigm:

22. Production Engineering Summary Checklist

Before launching an epoll-based network service into production, ensure your infrastructure meets the following architectural checklist:

• Non-blocking Flags: Verify all accepted sockets have O_NONBLOCK set via fcntl() before registering with `epoll_ctl()`.
• EAGAIN Read Loops: Ensure Edge-Triggered (EPOLLET) sockets drain receive queues in a while loop until EAGAIN is encountered.
• FD Resource Limits: Increase ulimit -n 1048576 in systemd unit files to prevent EMFILE errors under load.
• Thundering Herd Mitigation: Use SO_REUSEPORT or EPOLLEXCLUSIVE when binding multiple worker processes to the same port.
• Backlog Tuning: Increase net.core.somaxconn to 65535 to absorb sudden traffic bursts without dropping TCP SYN packets.

23. Developer FAQ

Written by Professor Pixel · CodingPancake Systems Architecture Series