async and await

Most of what an agent does is wait: for a model to answer, for an API to respond, for a database to reply. While a normal function waits, the whole program stands still. asyncio lets one thread do other useful work during those waits.

The pieces are small. async def makes a coroutine function; calling it gives you a coroutine object, not a result. await runs an awaitable and, if it has to wait, hands control to the event loop so another task can run. asyncio.run(main()) starts the loop. asyncio.gather(a, b) schedules several coroutines as tasks, runs them concurrently, and returns their results as a list in the order you passed them.

There is no magic: a task only gives up control at an await. Two tasks never run at the same instant; they take turns at the points where they would otherwise be waiting. That is enough to overlap network calls, which is what matters.

Press Run to watch asyncio.gather run a 1.0 s lookup and a 0.5 s lookup together. The timestamps are real, measured from the start of the program. Then switch Try it to the version that awaits one after the other.

# Two slow lookups: one after the other, or together with asyncio.gather.
# Run as: python3 fetch.py            (together)
#         python3 fetch.py one_by_one
import asyncio
import sys
import time

START = time.perf_counter()


def stamp():
    return f"{time.perf_counter() - START:.1f}s"


async def fetch(name, seconds):
    print(stamp(), "start", name)
    await asyncio.sleep(seconds)  # stands in for a network call
    print(stamp(), "done ", name)
    return name + " ok"


async def one_by_one():
    weather = await fetch("weather", 1.0)
    trains = await fetch("trains", 0.5)
    return [weather, trains]


async def together():
    return await asyncio.gather(fetch("weather", 1.0), fetch("trains", 0.5))


mode = sys.argv[1] if len(sys.argv) > 1 else "together"
results = asyncio.run(one_by_one() if mode == "one_by_one" else together())
print(stamp(), "results:", results)

What to notice

1. Both tasks start at 0.0 s. gather schedules them and the first await asyncio.sleep in weather hands control to the loop, which immediately starts trains.
2. They finish in time order. trains is done at 0.5 s and weather at 1.0 s. The order of finishing is not the order of starting.
3. gather keeps the order you asked for. The result list is ['weather ok', 'trains ok'] even though trains finished first.
4. Total is the maximum, not the sum. 1.0 s together versus 1.5 s one by one. With ten API calls of one second each, that is 1 s versus 10 s.
5. await on its own is not concurrency. In one_by_one, each await fetch(...) runs to completion before the next starts. You need gather or asyncio.create_task to have two things in flight.
6. asyncio.sleep stands in for I/O. Any library call that supports await (an HTTP client, a database driver, a model SDK) behaves the same way: it suspends the task while the network does the work.

Here is the real output of the two runs, including the wall-clock stamps:

$ python3 fetch.py                $ python3 fetch.py one_by_one
0.0s start weather                0.0s start weather
0.0s start trains                 1.0s done  weather
0.5s done  trains                 1.0s start trains
1.0s done  weather                1.5s done  trains
1.0s results: [...]               1.5s results: [...]

Why this matters for AI work

This is why an agent runner is async. In What an AI agent really is the program loops with async for event in runner.run_async(...). The runner awaits the model, awaits each tool, and yields events in between, so a web server can handle other users while one agent waits on Gemini. When you write a tool that calls an API, use an async def with an async HTTP client so the wait hands control back to the loop; a blocking call makes everyone wait.