1. The Async Problem: Why We Need Event Loops

1.1 Thread-Per-Connection Cost

In traditional synchronous systems, network concurrency is achieved by spawning a separate operating system thread for each incoming connection (a thread-per-connection model). While conceptually simple, this model scales poorly. Each OS thread requires substantial memory overhead for its stack (often 8 MB by default in Linux) and introduces high CPU overhead due to context switching. As the number of concurrent connections grows to tens of thousands (the C10K problem), the operating system spends more time switching CPU registers and cache lines between threads than executing application logic.

The event loop resolves this by running a single thread that monitors thousands of network sockets concurrently. Instead of a thread blocking on a read operation, the socket is configured in non-blocking mode. When no data is available, the read operation immediately returns an error. The event loop registers the socket with the OS kernel and continues executing other tasks. When data arrives, the OS notifies the loop, which resumes the corresponding coroutine. This converts thread-context switching into simple function calls.

1.2 Blocking vs Non-Blocking Visualized

In a blocking model, a thread is pinned to a single connection and must wait for network packets. In the non-blocking event loop model, the thread remains active, multiplexing task execution and I/O polling via a central registry.

Common Misconception — Async makes CPU-bound tasks faster: A common developer misconception is that async programming improves CPU-bound computations. Because an event loop runs on a single thread, executing a CPU-intensive operation (like calculating prime numbers) will block the loop entirely. No other task can run, and no I/O events will be polled. Async is designed exclusively to hide latency for I/O-bound operations, not to parallelize CPU calculations.

2. Generators as Coroutines: The Yield Keyword

2.1 The Mechanics of Yield

In Python, a function containing the yield keyword is a **generator**. When called, it does not execute the function body; instead, it returns a generator object. The compiler sets up a frame object on the heap that preserves the local variables and the instruction pointer. When next() is called on the generator, execution proceeds until it hits a yield statement. The generator suspends execution, saves its stack frame state, and returns the yielded value to the caller. The caller can then do other work and later resume the generator by calling send() or __next__().

This pause-and-resume capability is the foundation of coroutines. A coroutine is simply a generator that yields control when it needs to wait for I/O. The event loop acts as the coordinator, resuming the generator when the I/O operation completes.

Mermaid Diagram: The flow of control between the event loop and a generator-based coroutine during execution.

3. Non-Blocking I/O: The Selectors Module

3.1 OS-Level Multiplexing

To monitor multiple socket descriptors without polling them in an active CPU-wasting loop, we use OS-level I/O multiplexing. Operating systems provide system calls like select, poll, epoll (Linux), and kqueue (macOS/BSD). These system calls accept a list of file descriptors and block the thread until at least one descriptor changes state (e.g. data is ready to read, or the write buffer has space).

Python abstracts these systems in the standard library's selectors module. The DefaultSelector automatically selects the most efficient engine available on your operating system (typically epoll on Linux and kqueue on macOS). We register sockets with the selector, specifying the events we care about (read or write) and associating custom callback data with each socket.

4. The Event Loop Architecture: Core Components

4.1 The Loop Execution Cycle

An event loop consists of three core components: (1) A **Task Queue**: holding coroutines that are ready to run. (2) A **Selector Registry**: tracking file descriptors that are waiting for OS network events. (3) A **Timer Queue**: storing tasks scheduled to run after a specific delay. The event loop's main cycle runs continuously:

• It checks the Timer Queue and moves expired timers to the ready Task Queue.
• It executes all ready tasks in the Task Queue until the queue is empty.
• If tasks are waiting for network I/O, it calls selector.select(timeout) to block the thread until I/O events occur or the next timer expires.
• It processes the returned I/O events, executing the associated callback functions to move tasks back to the ready queue.

Pitfall — Running Blocking Sockets: If a developer forgets to call socket.setblocking(False) before registering it with the selector, calling socket.accept() or socket.recv() will block the entire operating system thread. This freezes the event loop, rendering all asynchronous operations synchronous and blocking other connections. Always verify that sockets are configured in non-blocking mode.

5. Future and Task Wrappers: Tracking Async Operations

5.1 Futures and Coroutines

A **Future** represents the eventual result of an asynchronous operation. It contains a state (pending, resolved, or cancelled), a result value, and a list of callback functions. When an operation starts (like reading from a socket), a Future is returned. The caller can attach a callback to this Future. When the event loop receives notification that the socket has data, it reads the data, stores it in the Future, and resolves it, trigger-executing all attached callbacks.

A **Task** is a subclass of Future that wraps a coroutine. It is responsible for driving the coroutine to completion. When initialized, the Task schedules itself on the event loop. When run, it calls send(None) on the wrapped coroutine. If the coroutine yields a Future, the Task registers a callback on that Future, suspending its own execution. When the Future resolves, the callback executes, calling send(result) on the coroutine to advance its execution.

6. Scheduling Timers: Handling Delayed Execution

6.1 Timer Queue Mechanics

Asynchronous applications regularly require scheduling tasks to run in the future (e.g. timeouts, polling loops, debouncing). In our event loop, we implement a Timer Queue using a min-heap. Each entry in the heap is a tuple: (execution_time, callback). The min-heap guarantees that the timer with the earliest execution time is always at the root of the tree, allowing us to find it in $\mathcal{O}(1)$ time and extract it in $\mathcal{O}(\log k)$ time.

During each iteration, the loop looks at the root of the min-heap. If $T_{\text{next}} \le 0$, the timer has expired. The loop pops the timer and appends the callback to the Task Queue. If $T_{\text{next}} > 0$, the loop calculates this value as the timeout argument for the next selector.select(timeout) call, ensuring the thread sleeps no longer than necessary.

7. Advanced: Coroutine Yield Chains and `yield from`

7.1 Delegating Generators

In a realistic asynchronous codebase, coroutines call other sub-coroutines. This forms a execution chain. C++ and Python handle this delegation using call stacks. In Python, the yield from expression (and later await) establishes a direct bidirectional channel between the outer delegating generator and the inner sub-generator. Values yielded by the sub-generator are passed directly to the caller, and calls to send() are passed directly to the sub-generator.

If an exception is thrown inside the event loop, it can be propagated down this delegation chain. Implementing this from scratch requires maintaining a call stack of generator frames inside our Task class, allowing us to catch exceptions and route them correctly.

7.2 The Bidirectional Delegation Channel

When a coroutine executes result = yield from sub_coro(), the compiler does not simply run a loop calling next(). Instead, it sets up a complex delegation machine that manages bidirectional communication. Formally, the behavior of yield from EXPR is equivalent to the following state machine expansion:

While the sub-generator _i is active, any values sent into the delegating generator via send() are passed directly to the sub-generator. If the caller raises an exception inside the delegating generator using the throw() method, the exception is caught and forwarded directly into the sub-generator using _i.throw(). If the sub-generator raises a StopIteration exception, the loop catches it, extracts the value attribute (which represents the sub-generator's return value), and assigns it to the local variable _r.

This delegation protocol allows developers to compose asynchronous tasks hierarchically. Below is a comparison table of coroutine control options:

Pitfall — Swallowing Exceptions in Yield Chains: A common developer bug occurs when a sub-coroutine raises an exception, and a middle delegating coroutine wraps the call in a generic try/except block without re-raising or resolving the future. This leaves the outer Task permanently suspended, waiting for a resolution callback that will never fire. Always ensure that error states propagate to the root Task frame.

8. Advanced: Comparison of Multiplexing Systems

8.1 Scaling Network Events

Different operating systems use different multiplexing engines. Selecting the correct system changes the algorithmic complexity of network polling. The table below compares the performance characteristics of various multiplexing system calls:

Pitfall — Select Complexity Bottleneck: If you construct an event loop using select() and register 1,000 sockets, every iteration requires the kernel to scan all 1,000 descriptors, even if only one has data. This degrades search performance to $\mathcal{O}(k)$ per loop. Always use epoll or kqueue via Python's DefaultSelector to maintain constant-time $\mathcal{O}(1)$ polling complexity.

8.2 Polling Trigger Modes: Level-Triggered vs Edge-Triggered

When configuring multiplexing engines like epoll, developers must choose between two operating modes: **Level-Triggered (LT)** and **Edge-Triggered (ET)**. These modes dictate how notifications are generated when a socket's read/write state changes:

• **Level-Triggered (LT)**: This is the default mode. The system call (e.g. select or epoll_wait) will continuously notify you as long as the socket's buffer has data to read or space to write. If you read only half of the incoming bytes, the next loop iteration will immediately trigger again to notify you about the remaining data. LT is safer and easier to program but generates more kernel interrupts.
• **Edge-Triggered (ET)**: The system call notifies you only when a state transition occurs on the descriptor (e.g. when new data arrives on an empty buffer). If you read only half of the data, the kernel will **not** trigger again. The loop will hang waiting for more data to arrive. ET requires you to read the socket inside a loop until it returns EWOULDBLOCK or EAGAIN, ensuring the buffer is completely empty.

In a custom event loop, managing these trigger modes is critical to avoid busy-waiting CPU spikes. If you register a socket in level-triggered write mode, the selector will continuously return events because the socket's write buffer is almost always empty and ready for data. This drives CPU utilization to 100% in a tight loop. To prevent this write-polling storm, only register sockets for write events when you actually have data buffered in memory that is ready to be sent, and unregister them immediately after the write operation completes.

9. Complete Custom Event Loop Implementation from Scratch

9.1 The Python Code

The following Python code implements our custom Future, Task, and Event Loop classes using the selectors module, simulating async operations:

10. Interactive: Python Event Loop State Visualizer

Click "Step Loop" to process task executions. Watch tasks move from the Task Queue to the active Call Stack, trigger selector operations, and resolve async futures:

Event Loop Task Throughput comparison

The chart below compares execution times (in milliseconds) to run 1,000 tasks using sequential blocking execution vs multiplexed event loop cycles:

12. Frequently Asked Questions