Async Iterators in Rust

Asynchronous iteration in Rust presents a case study in language design evolution. Unlike synchronous iterators, stable since Rust 1.0, async iteration remains under development. The stdlib AsyncIterator trait on nightly Rust provides direct async fn next() syntax. While ecosystem crates like futures::Stream bridge the gap on stable Rust, the nightly AsyncIterator represents the canonical direction for the language.

The Iterator Pattern Revisited

Before diving into async iterators, let’s revisit the synchronous Iterator trait:

pub trait Iterator {
    type Item;
    fn next(&mut self) -> Option<Self::Item>;
}

This elegant abstraction has served Rust exceptionally well. The next method returns Option<Item>, signalling termination with None. The mutable reference ensures exclusive access during iteration. Simple, predictable, efficient.

Async Iterator Interface

The nightly AsyncIterator trait provides direct async fn next() support in traits:

#![feature(async_iterator)]

pub trait AsyncIterator {
    type Item;
    async fn next(&mut self) -> Option<Self::Item>;
}

This interface mirrors the synchronous Iterator trait structure without requiring explicit pinning, polling, or boxing. For stable Rust, the ecosystem provides futures::Stream which exposes the underlying polling mechanism:

use std::future::Future;
use std::pin::Pin;
use std::task::{Context, Poll};

pub trait Stream {
    type Item;

    fn poll_next(
        self: Pin<&mut Self>,
        cx: &mut Context<'_>,
    ) -> Poll<Option<Self::Item>>;
}

This lower-level interface exposes the async function implementation details. The Pin type ensures the stream cannot be moved in memory once polling begins – necessary for self-referential futures. While futures::Stream provides the current stable implementation, both approaches merit examination: the nightly AsyncIterator represents the language’s direction, while manual polling reveals the underlying execution model.

Implementing a Simple Stream

Let’s implement a stream that yields numbers with delays:

use std::time::Duration;
use tokio::time::{sleep, Sleep};

struct DelayedRange {
    current: u32,
    end: u32,
    delay: Pin<Box<Sleep>>,
}

impl DelayedRange {
    fn new(start: u32, end: u32, duration: Duration) -> Self {
        Self {
            current: start,
            end,
            delay: Box::pin(sleep(duration)),
        }
    }
}

impl Stream for DelayedRange {
    type Item = u32;

    fn poll_next(
        mut self: Pin<&mut Self>,
        cx: &mut Context<'_>,
    ) -> Poll<Option<Self::Item>> {
        // Check if we've finished
        if self.current >= self.end {
            return Poll::Ready(None);
        }

        // Poll the delay future
        match self.delay.as_mut().poll(cx) {
            Poll::Ready(_) => {
                let value = self.current;
                self.current += 1;

                // Reset the delay for next iteration
                let duration = Duration::from_secs(1);
                self.delay = Box::pin(sleep(duration));

                Poll::Ready(Some(value))
            }
            Poll::Pending => Poll::Pending,
        }
    }
}

Nightly AsyncIterator Implementation

The nightly AsyncIterator implementation demonstrates reduced complexity:

#![feature(async_iterator)]

use std::time::Duration;
use tokio::time::sleep;

struct DelayedRangeAsync {
    current: u32,
    end: u32,
    duration: Duration,
}

impl DelayedRangeAsync {
    fn new(start: u32, end: u32, duration: Duration) -> Self {
        Self {
            current: start,
            end,
            duration,
        }
    }
}

impl AsyncIterator for DelayedRangeAsync {
    type Item = u32;

    async fn next(&mut self) -> Option<Self::Item> {
        if self.current >= self.end {
            return None;
        }

        sleep(self.duration).await;
        let value = self.current;
        self.current += 1;
        Some(value)
    }
}

This eliminates explicit pinning, manual state management, and polling complexity. The syntax aligns with async iteration patterns in other languages and represents Rust’s planned evolution.

The Pinning Problem

The Pin<&mut Self> parameter addresses a fundamental constraint in async iteration. Rust’s memory model permits moving values by default, but async code often creates self-referential structures where a future holds references to its own fields.

Moving such a structure would invalidate the internal reference. Pin is Rust’s way of saying “I promise this won’t move”. It’s a zero-cost abstraction that the compiler uses to enforce safety at compile time.

Language Evolution

The nightly AsyncIterator trait undergoes active development as the canonical interface for async iteration in Rust. Experimental projects can utilise this interface to preview the planned syntax.

Current stable Rust development relies on ecosystem solutions. The async-stream crate provides macros for stream creation, while the futures crate implements combinators (map, filter, fold) that parallel the iterator API.

Upon stabilisation of async functions in traits, AsyncIterator will transition from nightly to stable Rust. The manual polling interface in futures::Stream will retain utility for understanding execution models and optimising performance-critical code requiring fine-grained async state machine control.

Syntax quick hits

• AsyncIterator is the future of async iteration in Rust.
• futures::Stream is the current stable solution.
• async fn next is the natural async iteration syntax.
• poll_next exposes the underlying polling mechanism.
• Pin<&mut T> enforces immovability during polling.