Python asyncio.gather()

Summary: in this tutorial, you’ll learn how to use the Python asyncio.gather() function to run multiple asynchronous operations.

Introduction to the Python asyncio.gather() function

Sometimes, you may want to run multiple asynchronous operations and get the results once they are complete. To do that you can use the asyncio.gather() function:

gather(*aws, return_exceptions=False) -> Future[tuple[()]]Code language: Python (python)

The asyncio.gather() function has two parameters:

• aws is a sequence of awaitable objects. If any object in the aws is a coroutine, the asyncio.gather() function will automatically schedule it as a task.
• return_exceptions is False by default. If an exception occurs in an awaitable object, it is immediately propagated to the task that awaits on asyncio.gather(). Other awaitables will continue to run and won’t be canceled.

The asyncio.gather() returns the results of awaitables as a tuple with the same order as you pass the awaitables to the function.

If the return_exceptions is True. The asyncio.gather() will add the exception if any to the result and not propagate the exception to the caller.

Python asyncio.gather() examples

Let’s take some examples of using the asyncio.gather() function.

1) Using asyncio.gather() to run multiple asynchronous operations

The following example uses the asyncio.gather() to run two asynchronous operations and displays the results:

import asyncio


async def call_api(message, result, delay=3):
    print(message)
    await asyncio.sleep(delay)
    return result


async def main():
    a, b = await asyncio.gather(
        call_api('Calling API 1 ...', 1),
        call_api('Calling API 2 ...', 2)
    )
    print(a, b)


asyncio.run(main())Code language: Python (python)

Output:

Calling API 1 ...
Calling API 2 ...
100 200Code language: Python (python)

How it works.

First, define a coroutine call_api() that simulates an asynchronous operation. The call_api() displays a message, delays a number of seconds, and returns a result:

async def call_api(message, result, delay=3):
    print(message)
    await asyncio.sleep(delay)
    return resultCode language: Python (python)

Second, use the asyncio.gather() function to run two call_api():

async def main():
    a, b = await asyncio.gather(
        call_api('Calling API 1 ...', 100, 1),
        call_api('Calling API 2 ...', 200, 2)
    )
    print(a, b)Code language: Python (python)

The first coroutine takes 1 second and returns 100 while the second coroutine takes 2 seconds and returns 100.

After 2 seconds, the gather returns the result as a tuple that contains the result of the first and second coroutines.

Note that a is 100 and b is 200 which are the results of the corresponding coroutine that we pass to the asyncio.gather() function.

2) Using asyncio.gather() to run multiple asynchronous operations with exceptions

The following example shows how to use the asyncio.gather() function to execute multiple asynchronous operations where an operation raises an exception:

import asyncio


class APIError(Exception):
    def __init__(self, message):
        self._message = message

    def __str__(self):
        return self._message


async def call_api_failed():
    await asyncio.sleep(3)
    raise APIError('API failed')


async def call_api(message, result, delay=3):
    print(message)
    await asyncio.sleep(delay)
    return result


async def main():
    a, b, c = await asyncio.gather(
        call_api('Calling API 1 ...', 100, 1),
        call_api('Calling API 2 ...', 200, 2),
        call_api_failed()
    )
    print(a, b, c)


asyncio.run(main())Code language: Python (python)

How it works.

First, define a new APIError exception class that inherits from the Exception class:

class APIError(Exception):
    def __init__(self, message):
        self._message = message

    def __str__(self):
        return self._messageCode language: Python (python)

Second, define the call_api_failed() coroutine that delays 1 second and raises an APIError exception:

async def call_api_failed():
    await asyncio.sleep(3)
    raise APIError('API failed')Code language: Python (python)

Third, pass three coroutines to the asyncio.gather() function. After one second, the call_api_failed() raises an exception that immediately propagates to the asyncio.gather() function:

async def main():
    a, b, c = await asyncio.gather(
        call_api('Calling API 1 ...', 100, 1),
        call_api('Calling API 2 ...', 200, 2),
        call_api_failed()
    )
    print(a, b, c)Code language: Python (python)

If you run the program, you’ll see the APIError exception.

3) Using asyncio.gather() to return an exception in the result

To get the exception in the result, you set the return_exceptions parameter to True as follows:

import asyncio


class APIError(Exception):
    def __init__(self, message):
        self._message = message

    def __str__(self):
        return self._message


async def call_api(message, result, delay=3):
    print(message)
    await asyncio.sleep(delay)
    return result


async def call_api_failed():
    await asyncio.sleep(1)
    raise APIError('API failed')


async def main():
    a, b, c = await asyncio.gather(
        call_api('Calling API 1 ...', 100, 1),
        call_api('Calling API 2 ...', 200, 2),
        call_api_failed(),
        return_exceptions=True
    )
    print(a, b, c)


asyncio.run(main())Code language: Python (python)

Output:

Calling API 1 ...
Calling API 2 ...
100 200 API failedCode language: Python (python)

In this example, because the return_exceptions is set to True, the asyncio.gather() returns the exception as part of the result.

Summary

• The asyncio.gather() runs multiple asynchronous operations, wraps a coroutine as a task, and returns a tuple of results in the same order of awaitables.
• Set return_exceptions to True to allow errors to be returned as results.