Skip to content
QSWEQB
hardConceptCodingDesignMidSeniorStaff

You run ten coroutines concurrently and one of them raises. What happens to the other nine?

With gather they keep running unsupervised and their failures may surface as an unretrieved-exception warning. A TaskGroup, added in Python 3.11, cancels the siblings, waits for them to finish unwinding, and raises an ExceptionGroup you handle with except*.

5 min readUpdated 2026-07-26

What the interviewer is scoring

  • Does the candidate distinguish what gather reports to the caller from what happens to the sibling tasks
  • Whether cancellation is described as an exception delivered at the next await, not as a stop button
  • That CancelledError deriving from BaseException is connected to why broad except clauses break cancellation
  • Can they say why a fire-and-forget task needs a reference held somewhere
  • Whether cleanup is placed in finally with an awareness that awaits in the finally can themselves be cancelled

Answer

gather reports; it does not supervise

asyncio.gather schedules its awaitables and waits for all of them. With the default return_exceptions=False, the first exception propagates to whoever awaited the gather immediately — and that is all it does. The remaining tasks are not cancelled. They carry on running in the background with nobody awaiting them, holding connections and writing to databases while your caller is already handling the error. If one of them later fails too, its exception has no destination, and you find out through the loop's exception handler logging Task exception was never retrieved when the task object is finally garbage collected, at a moment unrelated to when the failure happened.

return_exceptions=True changes only the reporting: failures come back as ordinary result values in position, so you get a complete list, and again nothing is cancelled. Both modes are defensible. What is not defensible is believing that one of them bounds the lifetime of the work you started.

Task groups make the block the boundary

asyncio.TaskGroup, added in Python 3.11, is the structured-concurrency answer. Tasks are created inside an async with block, and the block cannot be left while any of them is still alive. If one raises, the group cancels its siblings, waits for each to finish unwinding, and then raises an ExceptionGroup collecting whatever went wrong. Because errors arrive as a group rather than as a single exception, you match them with the except* syntax from PEP 654, which runs its handler for the subgroup of matching exceptions and re-raises the rest.

import asyncio


async def work(n):
    await asyncio.sleep(n)
    if n == 2:
        raise ValueError("two is unlucky")
    return n


async def with_gather():
    results = await asyncio.gather(work(1), work(2), work(3), return_exceptions=True)
    print("gather:", results)          # all three ran; nothing was cancelled


async def with_taskgroup():
    try:
        async with asyncio.TaskGroup() as tg:      # 3.11+
            for n in (1, 2, 3):
                tg.create_task(work(n))
    except* ValueError as group:                   # PEP 654 syntax, also 3.11+
        print("taskgroup:", group.exceptions)      # work(3) was cancelled first


asyncio.run(with_gather())
asyncio.run(with_taskgroup())

The practical consequence is the one to lead with in an interview: the group turns "we started some work" into a scope with a known exit condition, so a request handler cannot return while a coroutine it spawned is still touching the database. That is a correctness property, not a tidiness preference.

Cancellation is cooperative, and it is an exception

task.cancel() does not stop anything. It arranges for CancelledError to be raised inside the coroutine at its next suspension point, and a coroutine that is in the middle of a long synchronous computation will not notice until it awaits something. So a task can be uncancellable in practice, and a task that catches the exception and carries on is simply refusing.

Since Python 3.8, CancelledError inherits from BaseException rather than Exception, and that change is worth understanding rather than memorising: it means a well-meaning except Exception around your business logic no longer swallows a cancellation and turns a shutdown into a hang. The corollary is that except BaseException, or a bare except:, does swallow it, which is why those two are worse in async code than they already were in synchronous code.

Cleanup therefore belongs in finally, with one subtlety that separates people who have debugged this. A finally block that itself awaits something can be interrupted by a second cancellation, so a graceful close that awaits a flush may not complete during shutdown. asyncio.shield protects an awaitable from cancellation propagating inward, but it does not protect the awaiter: your await on the shield still raises, and the shielded task keeps running, so shielding is how you get an orphan if you use it carelessly.

import asyncio


async def consume(queue):
    try:
        while True:
            item = await queue.get()       # the cancellation lands here
            print("handled", item)
    except asyncio.CancelledError:
        print("draining before exit")      # re-raise, or the caller thinks you refused
        raise
    finally:
        print("cleanup always runs")

Catching CancelledError at all is legitimate only to run cleanup and then re-raise. Returning normally from the handler tells Task that the cancellation was absorbed, and in a task group that leaves the group waiting on something it has already asked to stop.

Timeouts, said precisely

asyncio.timeout — also 3.11 — is a context manager that cancels the body when the deadline passes and converts the resulting cancellation into TimeoutError. Preferring it to asyncio.wait_for matters because it wraps a block rather than a single awaitable, so a sequence of three calls can share one budget instead of three independent ones.

The mechanism is more interesting than the API. The context manager cancels the task and then, on the way out, has to decide whether the CancelledError it sees is the one it caused or one that arrived from outside — because converting somebody else's cancellation into a TimeoutError would silently defeat their shutdown. Python 3.11 added Task.uncancel and a per-task cancellation counter for exactly this, which is what makes nested timeouts and a task group containing timeouts behave sensibly. You do not need to call uncancel yourself; you need to know why re-raising rather than converting is the rule.

The task nobody is holding

The last failure mode is the quietest. The event loop keeps only a weak reference to a running task, so a task you created and dropped can be garbage collected mid-execution. The documentation is explicit about saving a reference, and the idiom is a module-level set plus a done callback to remove it.

import asyncio

_background: set[asyncio.Task] = set()


def spawn(coro) -> asyncio.Task:
    task = asyncio.create_task(coro)
    _background.add(task)                       # the loop holds only a weak ref
    task.add_done_callback(_background.discard) # so drop it once it is finished
    return task

This is worth raising unprompted, because it is the failure that presents as "the audit event is written about ninety-nine percent of the time" and resists every reproduction attempt. If you find yourself needing this pattern often, that is the signal that the work wants a task group or a queue with a long-lived consumer instead: a background task with no owner has no error handling, no back-pressure, and no shutdown story.

Likely follow-ups

  • Why does return_exceptions=True change what the caller sees but not what the tasks do?
  • What is asyncio.shield for, and what does it not protect against?
  • How does asyncio.timeout know to raise TimeoutError rather than propagating CancelledError?
  • What does the "Task exception was never retrieved" warning actually mean?

Related questions

Further reading

asynciotask-groupscancellationexception-groupsstructured-concurrency