Internal Mechanisms of async/await

An async fn is desugared by the compiler into a state machine implementing the Future trait—each .await point is a state. Pin ensures the Future does not move in memory (self-referential structures), and Waker informs the executor when it can resume polling. Understanding this desugaring process is key to understanding Rust async’s "zero-cost abstraction" and why Futures cannot be dropped arbitrarily.

What an async fn becomes after compilation

async fn read_config() -> Config {
    let file = read_file().await;      // await point 1
    parse(&file).await                 // await point 2
}

The compiler converts the async fn into an anonymous type implementing the Future trait—a state machine:

// Pseudo-code generated by the compiler:
enum ReadConfigFuture {
    State0 { ... },                    // Initial, about to call read_file
    State1 { file_content: String },   // Waiting for read_file to complete, holds file_content
    State2 { file: String },           // Waiting for parse to complete
    Done,
}

impl Future for ReadConfigFuture {
    type Output = Config;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Config> {
        match self {
            State0 => { /* start read_file, transition to State1 */ Poll::Pending },
            State1 { file_content } => { /* check if read_file done */ ... },
            // ...
        }
    }
}

Key point: Each .await point corresponds to a state machine transition. The state machine stores local variables that span across the await—they must remain alive after poll() returns Pending, until the next poll().

The Future trait

pub trait Future {
    type Output;
    fn poll(self: Pin<&mut Self>, cx: &mut Context<'_>) -> Poll<Self::Output>;
}

The executor repeatedly calls poll(). If the Future is not yet complete, it returns Poll::Pending. Before returning Pending, the Future must register a waker via cx.waker()—a callback function that is invoked when the Future might become ready (e.g., I/O completion, timer expiration). Upon receiving a wake signal, the executor re-polls the Future.

The role of Pin

State1 holds file_content: String. If the Future is moved to a new memory address, internal pointers holding that address (if any) would become dangling. For self-referential Futures (common in select! or scenarios involving borrowed locals), Pin<&mut Self> guarantees that the Future will not be moved—the compiler does not generate memcpy operations to move pinned values.

Skeleton of a simple executor

fn block_on<F: Future>(mut future: F) -> F::Output {
    let waker = ...;                   // Create waker
    let mut cx = Context::from_waker(&waker);
    let mut future = unsafe { Pin::new_unchecked(&mut future) };
    loop {
        match future.as_mut().poll(&mut cx) {
            Poll::Ready(v) => return v,
            Poll::Pending => std::thread::park(),  // Wait for wake (production code uses epoll)
        }
    }
}

This is merely conceptual. Real-world executors (tokio, async-std) use epoll/kqueue/IOCP for I/O multiplexing, rather than busy-waiting.

References

• Rust Book: Async chapter (currently in progress for official book)
• Tokio tutorial: tokio.rs/tokio/tutorial
• async-book: rust-lang.github.io/async-book

Keywords: async, await, Future, Pin, poll, state machine, executor, waker