Coroutines and Tasks¶
This section outlines high-level asyncio APIs to work with coroutines and Tasks.
Coroutines¶
Coroutines declared with the async/await syntax is the preferred way of writing asyncio applications. For example, the following snippet of code prints “hello”, waits 1 second, and then prints “world”:
Note that simply calling a coroutine will not schedule it to be executed:
To actually run a coroutine, asyncio provides the following mechanisms:
The asyncio.run() function to run the top-level entry point “main()” function (see the above example.)
Awaiting on a coroutine. The following snippet of code will print “hello” after waiting for 1 second, and then print “world” after waiting for another 2 seconds:
The asyncio.create_task() function to run coroutines concurrently as asyncio Tasks .
Let’s modify the above example and run two say_after coroutines concurrently:
Note that expected output now shows that the snippet runs 1 second faster than before:
The asyncio.TaskGroup class provides a more modern alternative to create_task() . Using this API, the last example becomes:
The timing and output should be the same as for the previous version.
Awaitables¶
We say that an object is an awaitable object if it can be used in an await expression. Many asyncio APIs are designed to accept awaitables.
There are three main types of awaitable objects: coroutines, Tasks, and Futures.
Python coroutines are awaitables and therefore can be awaited from other coroutines:
In this documentation the term “coroutine” can be used for two closely related concepts:
a coroutine function: an async def function;
a coroutine object: an object returned by calling a coroutine function.
Tasks are used to schedule coroutines concurrently.
When a coroutine is wrapped into a Task with functions like asyncio.create_task() the coroutine is automatically scheduled to run soon:
A Future is a special low-level awaitable object that represents an eventual result of an asynchronous operation.
When a Future object is awaited it means that the coroutine will wait until the Future is resolved in some other place.
Future objects in asyncio are needed to allow callback-based code to be used with async/await.
Normally there is no need to create Future objects at the application level code.
Future objects, sometimes exposed by libraries and some asyncio APIs, can be awaited:
A good example of a low-level function that returns a Future object is loop.run_in_executor() .
Creating Tasks¶
asyncio. create_task ( coro , * , name = None , context = None ) ¶
Wrap the coro coroutine into a Task and schedule its execution. Return the Task object.
If name is not None , it is set as the name of the task using Task.set_name() .
An optional keyword-only context argument allows specifying a custom contextvars.Context for the coro to run in. The current context copy is created when no context is provided.
The task is executed in the loop returned by get_running_loop() , RuntimeError is raised if there is no running loop in current thread.
asyncio.TaskGroup.create_task() is a newer alternative that allows for convenient waiting for a group of related tasks.
Save a reference to the result of this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done. For reliable “fire-and-forget” background tasks, gather them in a collection:
New in version 3.7.
Changed in version 3.8: Added the name parameter.
Changed in version 3.11: Added the context parameter.
Task Cancellation¶
Tasks can easily and safely be cancelled. When a task is cancelled, asyncio.CancelledError will be raised in the task at the next opportunity.
It is recommended that coroutines use try/finally blocks to robustly perform clean-up logic. In case asyncio.CancelledError is explicitly caught, it should generally be propagated when clean-up is complete. asyncio.CancelledError directly subclasses BaseException so most code will not need to be aware of it.
The asyncio components that enable structured concurrency, like asyncio.TaskGroup and asyncio.timeout() , are implemented using cancellation internally and might misbehave if a coroutine swallows asyncio.CancelledError . Similarly, user code should not generally call uncancel . However, in cases when suppressing asyncio.CancelledError is truly desired, it is necessary to also call uncancel() to completely remove the cancellation state.
Task Groups¶
Task groups combine a task creation API with a convenient and reliable way to wait for all tasks in the group to finish.
class asyncio. TaskGroup ¶
An asynchronous context manager holding a group of tasks. Tasks can be added to the group using create_task() . All tasks are awaited when the context manager exits.
New in version 3.11.
Create a task in this task group. The signature matches that of asyncio.create_task() .
The async with statement will wait for all tasks in the group to finish. While waiting, new tasks may still be added to the group (for example, by passing tg into one of the coroutines and calling tg.create_task() in that coroutine). Once the last task has finished and the async with block is exited, no new tasks may be added to the group.
The first time any of the tasks belonging to the group fails with an exception other than asyncio.CancelledError , the remaining tasks in the group are cancelled. No further tasks can then be added to the group. At this point, if the body of the async with statement is still active (i.e., __aexit__() hasn’t been called yet), the task directly containing the async with statement is also cancelled. The resulting asyncio.CancelledError will interrupt an await , but it will not bubble out of the containing async with statement.
Once all tasks have finished, if any tasks have failed with an exception other than asyncio.CancelledError , those exceptions are combined in an ExceptionGroup or BaseExceptionGroup (as appropriate; see their documentation) which is then raised.
Two base exceptions are treated specially: If any task fails with KeyboardInterrupt or SystemExit , the task group still cancels the remaining tasks and waits for them, but then the initial KeyboardInterrupt or SystemExit is re-raised instead of ExceptionGroup or BaseExceptionGroup .
If the body of the async with statement exits with an exception (so __aexit__() is called with an exception set), this is treated the same as if one of the tasks failed: the remaining tasks are cancelled and then waited for, and non-cancellation exceptions are grouped into an exception group and raised. The exception passed into __aexit__() , unless it is asyncio.CancelledError , is also included in the exception group. The same special case is made for KeyboardInterrupt and SystemExit as in the previous paragraph.
Sleeping¶
Block for delay seconds.
If result is provided, it is returned to the caller when the coroutine completes.
sleep() always suspends the current task, allowing other tasks to run.
Setting the delay to 0 provides an optimized path to allow other tasks to run. This can be used by long-running functions to avoid blocking the event loop for the full duration of the function call.
Example of coroutine displaying the current date every second for 5 seconds:
Changed in version 3.10: Removed the loop parameter.
Running Tasks Concurrently¶
Run awaitable objects in the aws sequence concurrently.
If any awaitable in aws is a coroutine, it is automatically scheduled as a Task.
If all awaitables are completed successfully, the result is an aggregate list of returned values. The order of result values corresponds to the order of awaitables in aws.
If return_exceptions is False (default), the first raised exception is immediately propagated to the task that awaits on gather() . Other awaitables in the aws sequence won’t be cancelled and will continue to run.
If return_exceptions is True , exceptions are treated the same as successful results, and aggregated in the result list.
If gather() is cancelled, all submitted awaitables (that have not completed yet) are also cancelled.
If any Task or Future from the aws sequence is cancelled, it is treated as if it raised CancelledError – the gather() call is not cancelled in this case. This is to prevent the cancellation of one submitted Task/Future to cause other Tasks/Futures to be cancelled.
A more modern way to create and run tasks concurrently and wait for their completion is asyncio.TaskGroup .
If return_exceptions is False, cancelling gather() after it has been marked done won’t cancel any submitted awaitables. For instance, gather can be marked done after propagating an exception to the caller, therefore, calling gather.cancel() after catching an exception (raised by one of the awaitables) from gather won’t cancel any other awaitables.
Changed in version 3.7: If the gather itself is cancelled, the cancellation is propagated regardless of return_exceptions.
Changed in version 3.10: Removed the loop parameter.
Deprecated since version 3.10: Deprecation warning is emitted if no positional arguments are provided or not all positional arguments are Future-like objects and there is no running event loop.
Shielding From Cancellation¶
If aw is a coroutine it is automatically scheduled as a Task.
is equivalent to:
except that if the coroutine containing it is cancelled, the Task running in something() is not cancelled. From the point of view of something() , the cancellation did not happen. Although its caller is still cancelled, so the “await” expression still raises a CancelledError .
If something() is cancelled by other means (i.e. from within itself) that would also cancel shield() .
If it is desired to completely ignore cancellation (not recommended) the shield() function should be combined with a try/except clause, as follows:
Save a reference to tasks passed to this function, to avoid a task disappearing mid-execution. The event loop only keeps weak references to tasks. A task that isn’t referenced elsewhere may get garbage collected at any time, even before it’s done.
Changed in version 3.10: Removed the loop parameter.
Deprecated since version 3.10: Deprecation warning is emitted if aw is not Future-like object and there is no running event loop.
Timeouts¶
An asynchronous context manager that can be used to limit the amount of time spent waiting on something.
delay can either be None , or a float/int number of seconds to wait. If delay is None , no time limit will be applied; this can be useful if the delay is unknown when the context manager is created.
In either case, the context manager can be rescheduled after creation using Timeout.reschedule() .
If long_running_task takes more than 10 seconds to complete, the context manager will cancel the current task and handle the resulting asyncio.CancelledError internally, transforming it into an asyncio.TimeoutError which can be caught and handled.
The asyncio.timeout() context manager is what transforms the asyncio.CancelledError into an asyncio.TimeoutError , which means the asyncio.TimeoutError can only be caught outside of the context manager.
The context manager produced by asyncio.timeout() can be rescheduled to a different deadline and inspected.
class asyncio. Timeout ( when ) ¶
An asynchronous context manager for cancelling overdue coroutines.
when should be an absolute time at which the context should time out, as measured by the event loop’s clock:
If when is None , the timeout will never trigger.
If when < loop.time() , the timeout will trigger on the next iteration of the event loop.
when ( ) → float | None ¶
Return the current deadline, or None if the current deadline is not set.
reschedule ( when : float | None ) ¶
Reschedule the timeout.
expired ( ) → bool ¶
Return whether the context manager has exceeded its deadline (expired).
Timeout context managers can be safely nested.
New in version 3.11.
Similar to asyncio.timeout() , except when is the absolute time to stop waiting, or None .
New in version 3.11.
Wait for the aw awaitable to complete with a timeout.
If aw is a coroutine it is automatically scheduled as a Task.
timeout can either be None or a float or int number of seconds to wait for. If timeout is None , block until the future completes.
If a timeout occurs, it cancels the task and raises TimeoutError .
To avoid the task cancellation , wrap it in shield() .
The function will wait until the future is actually cancelled, so the total wait time may exceed the timeout. If an exception happens during cancellation, it is propagated.
If the wait is cancelled, the future aw is also cancelled.
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.7: When aw is cancelled due to a timeout, wait_for waits for aw to be cancelled. Previously, it raised TimeoutError immediately.
Changed in version 3.10: Removed the loop parameter.
Waiting Primitives¶
Run Future and Task instances in the aws iterable concurrently and block until the condition specified by return_when.
The aws iterable must not be empty and generators yielding tasks are not accepted.
Returns two sets of Tasks/Futures: (done, pending) .
timeout (a float or int), if specified, can be used to control the maximum number of seconds to wait before returning.
Note that this function does not raise TimeoutError . Futures or Tasks that aren’t done when the timeout occurs are simply returned in the second set.
return_when indicates when this function should return. It must be one of the following constants:
The function will return when any future finishes or is cancelled.
The function will return when any future finishes by raising an exception. If no future raises an exception then it is equivalent to ALL_COMPLETED .
The function will return when all futures finish or are cancelled.
Unlike wait_for() , wait() does not cancel the futures when a timeout occurs.
Changed in version 3.10: Removed the loop parameter.
Changed in version 3.11: Passing coroutine objects to wait() directly is forbidden.
Run awaitable objects in the aws iterable concurrently. Generators yielding tasks are not accepted as aws iterable. Return an iterator of coroutines. Each coroutine returned can be awaited to get the earliest next result from the iterable of the remaining awaitables.
Raises TimeoutError if the timeout occurs before all Futures are done.
Changed in version 3.10: Removed the loop parameter.
Deprecated since version 3.10: Deprecation warning is emitted if not all awaitable objects in the aws iterable are Future-like objects and there is no running event loop.
Running in Threads¶
Asynchronously run function func in a separate thread.
Any *args and **kwargs supplied for this function are directly passed to func. Also, the current contextvars.Context is propagated, allowing context variables from the event loop thread to be accessed in the separate thread.
Return a coroutine that can be awaited to get the eventual result of func.
This coroutine function is primarily intended to be used for executing IO-bound functions/methods that would otherwise block the event loop if they were run in the main thread. For example:
Directly calling blocking_io() in any coroutine would block the event loop for its duration, resulting in an additional 1 second of run time. Instead, by using asyncio.to_thread() , we can run it in a separate thread without blocking the event loop.
Due to the GIL , asyncio.to_thread() can typically only be used to make IO-bound functions non-blocking. However, for extension modules that release the GIL or alternative Python implementations that don’t have one, asyncio.to_thread() can also be used for CPU-bound functions.
New in version 3.9.
Scheduling From Other Threads¶
Submit a coroutine to the given event loop. Thread-safe.
Return a concurrent.futures.Future to wait for the result from another OS thread.
This function is meant to be called from a different OS thread than the one where the event loop is running. Example:
If an exception is raised in the coroutine, the returned Future will be notified. It can also be used to cancel the task in the event loop:
See the concurrency and multithreading section of the documentation.
Unlike other asyncio functions this function requires the loop argument to be passed explicitly.
New in version 3.5.1.
Introspection¶
Return the currently running Task instance, or None if no task is running.
If loop is None get_running_loop() is used to get the current loop.
New in version 3.7.
Return a set of not yet finished Task objects run by the loop.
If loop is None , get_running_loop() is used for getting current loop.
New in version 3.7.
Return True if obj is a coroutine object.
New in version 3.4.
Task Object¶
A Future-like object that runs a Python coroutine . Not thread-safe.
Tasks are used to run coroutines in event loops. If a coroutine awaits on a Future, the Task suspends the execution of the coroutine and waits for the completion of the Future. When the Future is done, the execution of the wrapped coroutine resumes.
Event loops use cooperative scheduling: an event loop runs one Task at a time. While a Task awaits for the completion of a Future, the event loop runs other Tasks, callbacks, or performs IO operations.
Use the high-level asyncio.create_task() function to create Tasks, or the low-level loop.create_task() or ensure_future() functions. Manual instantiation of Tasks is discouraged.
To cancel a running Task use the cancel() method. Calling it will cause the Task to throw a CancelledError exception into the wrapped coroutine. If a coroutine is awaiting on a Future object during cancellation, the Future object will be cancelled.
cancelled() can be used to check if the Task was cancelled. The method returns True if the wrapped coroutine did not suppress the CancelledError exception and was actually cancelled.
An optional keyword-only context argument allows specifying a custom contextvars.Context for the coro to run in. If no context is provided, the Task copies the current context and later runs its coroutine in the copied context.
Changed in version 3.7: Added support for the contextvars module.
Changed in version 3.8: Added the name parameter.
Deprecated since version 3.10: Deprecation warning is emitted if loop is not specified and there is no running event loop.
Changed in version 3.11: Added the context parameter.
Return True if the Task is done.
A Task is done when the wrapped coroutine either returned a value, raised an exception, or the Task was cancelled.
Return the result of the Task.
If the Task is done, the result of the wrapped coroutine is returned (or if the coroutine raised an exception, that exception is re-raised.)
If the Task has been cancelled, this method raises a CancelledError exception.
If the Task’s result isn’t yet available, this method raises a InvalidStateError exception.
Return the exception of the Task.
If the wrapped coroutine raised an exception that exception is returned. If the wrapped coroutine returned normally this method returns None .
If the Task has been cancelled, this method raises a CancelledError exception.
If the Task isn’t done yet, this method raises an InvalidStateError exception.
add_done_callback ( callback , * , context = None ) ¶
Add a callback to be run when the Task is done.
This method should only be used in low-level callback-based code.
See the documentation of Future.add_done_callback() for more details.
Remove callback from the callbacks list.
This method should only be used in low-level callback-based code.
See the documentation of Future.remove_done_callback() for more details.
Return the list of stack frames for this Task.
If the wrapped coroutine is not done, this returns the stack where it is suspended. If the coroutine has completed successfully or was cancelled, this returns an empty list. If the coroutine was terminated by an exception, this returns the list of traceback frames.
The frames are always ordered from oldest to newest.
Only one stack frame is returned for a suspended coroutine.
The optional limit argument sets the maximum number of frames to return; by default all available frames are returned. The ordering of the returned list differs depending on whether a stack or a traceback is returned: the newest frames of a stack are returned, but the oldest frames of a traceback are returned. (This matches the behavior of the traceback module.)
print_stack ( * , limit = None , file = None ) ¶
Print the stack or traceback for this Task.
This produces output similar to that of the traceback module for the frames retrieved by get_stack() .
The limit argument is passed to get_stack() directly.
The file argument is an I/O stream to which the output is written; by default output is written to sys.stdout .
Return the coroutine object wrapped by the Task .
New in version 3.8.
Return the name of the Task.
If no name has been explicitly assigned to the Task, the default asyncio Task implementation generates a default name during instantiation.
New in version 3.8.
Set the name of the Task.
The value argument can be any object, which is then converted to a string.
In the default Task implementation, the name will be visible in the repr() output of a task object.
New in version 3.8.
Request the Task to be cancelled.
This arranges for a CancelledError exception to be thrown into the wrapped coroutine on the next cycle of the event loop.
The coroutine then has a chance to clean up or even deny the request by suppressing the exception with a try … … except CancelledError … finally block. Therefore, unlike Future.cancel() , Task.cancel() does not guarantee that the Task will be cancelled, although suppressing cancellation completely is not common and is actively discouraged. Should the coroutine nevertheless decide to suppress the cancellation, it needs to call Task.uncancel() in addition to catching the exception.
Changed in version 3.9: Added the msg parameter.
Changed in version 3.11: The msg parameter is propagated from cancelled task to its awaiter.
The following example illustrates how coroutines can intercept the cancellation request:
Return True if the Task is cancelled.
The Task is cancelled when the cancellation was requested with cancel() and the wrapped coroutine propagated the CancelledError exception thrown into it.
Decrement the count of cancellation requests to this Task.
Returns the remaining number of cancellation requests.
Note that once execution of a cancelled task completed, further calls to uncancel() are ineffective.
New in version 3.11.
This method is used by asyncio’s internals and isn’t expected to be used by end-user code. In particular, if a Task gets successfully uncancelled, this allows for elements of structured concurrency like Task Groups and asyncio.timeout() to continue running, isolating cancellation to the respective structured block. For example:
While the block with make_request() and make_another_request() might get cancelled due to the timeout, unrelated_code() should continue running even in case of the timeout. This is implemented with uncancel() . TaskGroup context managers use uncancel() in a similar fashion.
If end-user code is, for some reason, suppresing cancellation by catching CancelledError , it needs to call this method to remove the cancellation state.
Return the number of pending cancellation requests to this Task, i.e., the number of calls to cancel() less the number of uncancel() calls.
Note that if this number is greater than zero but the Task is still executing, cancelled() will still return False . This is because this number can be lowered by calling uncancel() , which can lead to the task not being cancelled after all if the cancellation requests go down to zero.
This method is used by asyncio’s internals and isn’t expected to be used by end-user code. See uncancel() for more details.
Python Asyncio Part 2 – Awaitables, Tasks, and Futures
Having already covered the basic concepts in Python Asyncio Part 1 – Basic Concepts and Patterns, in this part of the series I will be going into more depth on the actual syntax used when employing this library in Python code. Many of the examples used here are based on code we have actually used as part of BBC R&D’s cloudfit project.
Writing Asynchronous Code
The most basic tool in the tool kit of an asynchronous programmer in Python is the new keyword async def , which is used to declare an asynchronous coroutine function in the same way that def is used to define a normal synchronous function.
TERMINOLOGY: In this article I will refer to async def as a keyword, and in future articles I will refer to async for and async with as keywords. Strictly speaking this isn’t true. In fact async is a keyword and so is def , but since you can’t use async by itself, only in combination with another keyword I think it’s much more convenient and less confusing to think of async def as a single keyword that happens to have a space in the middle of it. It certainly behaves like one in terms of language usage.
In the above example we define a coroutine function example_coroutine_function and an ordinary function example_function . The code block that forms the body of the definition is slightly different in the two cases. The code block for example_function is ordinary synchronous Python, whilst the code-block for example_coroutine_function is asynchronous Python.
- Asynchronous Python code can only be included inside a suitable context that allows it, which almost always means inside a coroutine function defined using async def . There’s one other context where asynchronous code is allowed which we will cover in the next article.
- Asynchronous Python code can use any of the Python keywords, structures, etc… allowed in ordinary Python. Nothing is disallowed (although some things may be discouraged, see later).
- There are several new keywords which can only be used inside asynchronous code: await , async with and async for .
- Note that async def is not one of the keywords reserved for use in asynchronous code. It can be used anywhere were def can be used, though its effect is slightly different.
A declaration of a coroutine function using async def looks deceptively similar to the declaration of an ordinary function using def . Most of the time writing one is pretty similar, however there are some key differences, which are very important for asynchronous programming:
-
The Python def keyword creates a callable object with a name, when the object is called the code block of the function is run. Eg.
means that example_function is now a callable object which takes three parameters. When you invoke it like so:
this causes the function code to be run immediately as a subroutine call, and its return value to be assigned to r .
means that example_coroutine_function is now a callable object which takes three parameters. When you invoke it like so:
this does not cause the function code block to be run. Instead an object of class Coroutine is created, and is assigned to r . To make the code block actually run you need to make use of one of the facilities that asyncio provides for running a coroutine. Most commonly this is the await keyword. The function asyncio.gather is used in an example below. Other examples can be found in the python docs. See for example wait .
- The code block of asynchronous code inside an async def statement.
- The callable object that the async def statement creates.
- The object of class Coroutine that is returned by the callable object when it is called.
TYPING NOTE: If you are using the typing library then the declaration of coroutine functions can be a little confusing at times.
defines example_coroutine_function as a callable that takes two parameters of types A and B and returns an object of type Coroutine[Any, Any, C] . It’s pretty rare that you’ll need to refer to this return type explicitly.
If you’re curious about the two Any type parameters in the above definition they’re related to the way that the event loop works. The first type parameter actually indicates the type of the values that the coroutine will pass to the event loop whenever it yields, whilst the second represents the type of the values that the event loop will pass back to the coroutine whenever the it is reawakened. In practice the actual types of these objects are determined by the internal machinery of the event loop’s implementation, and should never need to be referred to explicitly in client code unless you are writing your own event loop implementation (which is a pretty advanced topic way beyond the scope of these articles).
The await Keyword and Awaitables
One of the new keywords added to the language to support asyncio is await . This keyword is, in many ways, the very core of asynchronous code. It can only be used inside asynchronous code blocks (ie. in the code block of an async def statement defining a coroutine function), and it is used as an expression which takes a single parameter and returns a value.
is a valid Python statement which will perform the await action on the object a and return a value which will be assigned to r . Exactly what will happen when this await statement is executed will depend upon what the object a is.
A coroutine object is “awaitable” (it can be used in an await statement). Recall that when you are executing asynchronous code you are always doing so in the context of a “Task”, which is an object maintained by the Event Loop, and that each Task has its own call stack. The first time a Coroutine object is awaited the code block inside its definition is executed in the current Task, with its new code context added to the top of the call stack for this Task, just like a normal function call. When the code block reaches its end (or otherwise returns) then execution moves back to the await statement that called it. The return value of the await statement is the value returned by the code block. If a Coroutine object is awaited a second time this raises an exception. In this way you can think of awaiting a Coroutine object as being very much like calling a function, with the notable difference that the Coroutine object’s code block can contain asynchronous code, and so can pause the current task during running, which a function’s code block cannot.
In fact there are three types of objects that are awaitable:
- A Coroutine object. When awaited it will execute the code-block of the coroutine in the current Task. The await statement will return the value returned by the code block.
- Any object of class asyncio.Future which when awaited causes the current Task to be paused until a specific condition occurs (see next section).
- An object which implements the magic method __await__ , in which case what happens when it is awaited is defined by that method.
That last one is there so that writers of libraries can create their own new classes of objects which are awaitable and do something special when awaited. It’s usually a good idea to make your custom awaitable objects either behave like a Coroutine object or like a Future object, and document which in the class’s doc strings. Making custom awaitable classes like this is a somewhat more advanced topic, though one that may come up when writing asyncio wrappers for synchronous io libraries, for example.
TYPING NOTE: If you are using typing then there is an abstract class Awaitable which is generic, so that Awaitable[R] for some type R means “anything which is awaitable, and when used in an await statement will return something of type R ”.
One of the most important points to get across is that the currently executing Task cannot be paused by any means other than awaiting a future (or a custom awaitable object that behaves like one). And that is something which can only happen inside asynchronous code. So any await statement might cause your current task to pause, but is not guaranteed to. Conversely any statement which is not an await statement (or an async for or async with under certain circumstances which will be explained in the next post) cannot cause your current Task to be paused.
This means that the traditional multithreaded code problems of data races where different threads of execution both alter the same value are severely reduced in asynchronous code, but not entirely eliminated. In particular for the purposes of data shared between Tasks on the same event loop all synchronous code can be considered “atomic”. To illustrate what this means consider the following code:
then even though both fetcher and monitor access the global variable vals they do so in two tasks that are running in the same event loop. For this reason it is not possible for the print statement in monitor to run unless fetcher is currently asleep waiting for io. This means that it is not possible for the length of vals to be printed whilst the for loop is only part-way through running. So if the get_some_values_from_io always returns 10 values at a time (for example) then the printed length of vals will always be a multiple of ten. It is simply not possible for the print statement to execute at a time when vals has a non-multiple of ten length.
On the other hand if there was an await statement inside the for loop this would no longer be guaranteed.
NOTE: Note that the create_task calls above are redundant. The body of main could be reduced to await asyncio.gather(fetcher(), monitor()) .
Futures
A Future object is a type of awaitable. Unlike a coroutine object when a future is awaited it does not cause a block of code to be executed. Instead a future object can be thought of as representing some process that is ongoing elsewhere and which may or may not yet be finished.
When you await a future the following happens:
- If the process the future represents has finished and returned a value then the await statement immediately returns that value.
- If the process the future represents has finished and raised an exception then the await statement immediately raises that exception.
- If the process the future represents has not yet finished then the current Task is paused until the process has finished. Once it is finished it behaves as described in the first two bullet points here.
All Future objects f have the following synchronous interface in addition to being awaitable:
- f.done() returns True if the process the future represents has finished.
- f.exception() raises an asyncio.InvalidStateError exception if the process has not yet finished. If the process has finished it returns the exception it raised, or None if it terminated without raising.
- f.result() raises an asyncio.InvalidStateError exception if the process has not yet finished. If the process has finished it raises the exception it raised, or returns the value it returned if it finished without raising.
It’s important to note that there is no way for a future that is done to ever change back into one that is not yet done. A future becoming done is a one-time occurrence.
IMPORTANT!: The distinction between a Coroutine and a Future is important. A Coroutine’s code will not be executed until it is awaited. A future represents something that is executing anyway, and simply allows your code to wait for it to finish, check if it has finished, and fetch the result if it has.
IMPORTANT!: Objects which implement the __await__ magic method may do almost anything when awaited. They might behave more like Coroutines, or more like Futures. They may do something else entirely. The documentation for the class in question should usually make it clear what their behaviour is.
You probably won’t create your own futures very often unless you are implementing new libraries that extend asyncio. However you will find that library functions often return futures. If you do need to create your own future directly you can do it with a call to
On the other hand you will probably find that you use a related method, create_task quite often …
TYPING NOTE: If you want to specify that a variable is a Future then you can use the asyncio.Future class as a type annotation. If you want to specify that the Future’s result should be of a specific type, R then you can use the following notation:
(in Python 3.6 you will need to wrap asyncio.Future[R] in quotes for this to work correctly, but in later versions of Python this is no longer needed).
Tasks
As described in the previous article each event loop contains a number of tasks, and every coroutine that is executing is doing so inside a task. So the question of how to create a task seems like an important one.
Creating a task is a simple matter, and can be done entirely in synchronous code:
NOTE: In Python 3.6 the function asyncio.create_task is not available, but you can still create a task using:
this is exactly the same, but a little more verbose.
The method create_task takes a coroutine object as a parameter and returns a Task object, which inherits from asyncio.Future . The call creates the task inside the event loop for the current thread, and starts the task executing at the beginning of the coroutine’s code-block. The returned future will be marked as done() only when the task has finished execution. As you might expect the return value of the coroutine’s code block is the result() which will be stored in the future object when it is finished (and if it raises then the exception will be caught and stored in the future).
Creating a task to wrap a coroutine is a synchronous call, so it can be done anywhere, including inside synchronous or asynchronous code. If you do it in asynchronous code then the event loop is already running (since it is currently executing your asynchronous code), and when it next gets the opportunity (ie. next time your current task pauses) it might make the new task active.
When you do it in synchronous code, however, chances are that the event loop is not yet running. Manualy manipulating event loops is discouranged by the python documentation. Unless you are developing libraries extending asyncio functionality, you should probably avoid trying to create a task from synchronous code.
If you do need to call a single piece of async code in an otherwise synchronous script, you can use asyncio.run() .
Running async programs
With the introduction of asyncio.run() in Python 3.7, and the removal of the loop parameter from many asyncio function in Python 3.10, managing event loops is something that you are unlikely to come across, unless you are developing an async library. The event loop objects are still there and accessible. There is a whole page in the docs discussing them. If you are working in Python 3.7 or greater, rejoice and give thanks for asyncio.run() .
asyncio.run(coro) will run coro , and return the result. It will always start a new event loop, and it cannot be called when the event loop is already running. This leads to a couple of obvious ways to run your async code.
The first is to have everything in async coroutines, and have a very simple entry function:
The second is to wrap each coroutine call in a separate run command. Note that this forgoes all of the benefits of asyncio. Still, there might be the odd script where this is the right thing to do.
Note that these simple examples don’t make use of the ability of async code to work on multiple tasks concurrently. A more sensible example is given at the end. As you work with asyncio in python, you’ll learn about more sophisticated ways to manage your work, but this is enough to get you started.
Manual event loop interaction
If you’re using Python 3.6, and you need to run coroutines from ordinary sync code (which you probably will, if you want to start something.) then you will need to start the event loop. There are two methods for doing this:
will cause the event loop to run forever (or until explicitly killed). This isn’t usually particularly useful. Much more useful is:
which takes a single parameter. If the parameter is a future (such as a task) then the loop will be run until the future is done, returning its result or raising its exception. So putting it together:
will create a new task which executes example_coroutine_function inside the event loop until it finishes, and then return the result.
In fact this can be simplified further since if you pass a coroutine object as the parameter to run_until_complete then it automatically calls create_task for you.
How to yield control
There is no simple command for yielding control to the event loop so that other tasks can run. In most cases in an asyncio program this is not something you will want to do explicitly, preferring to allow control to be yielded automatically when you await a future returned by some underlying library that handles some type of IO.
However occasionally you do need to, and in particular it’s quite useful during testing and debugging. As a result there is a recognised idiom for doing this if you need to. The statement:
will pause the current task and allow other tasks to be executed. The way this works is by using the function asyncio.sleep which is provided by the asyncio library. This function takes a single parameter which is a number of seconds, and returns a future which is not marked done yet but which will be when the specified number of seconds have passed.
Specifying a count of zero seconds works to interrupt the current task if other tasks are pending, but otherwise doesn’t do anything since the sleep time is zero.
The implementation of asyncio.sleep in the standard library has been optimised to make this an efficient operation.
When using asyncio.sleep with a non-zero parameter it’s worth noting that just because the future will become done when the number of seconds has passed does not mean that your task will always wake back up at that time. In fact it may wake back up at any point after that time, since it can only awaken when there’s no other task being run on the event loop.
Summary
- You can only use the keywords await , async with and async for inside asynchronous code.
- Asynchronous code must be contained inside an async def declaration (or one other place we’ll cover in the next article), but the declaration can go anywhere def is allowed.
- When you call await you must call it on one of the following:
- A coroutine object, which is the return value of a coroutine function defined using async def .
- The coroutine’s code will only be executed when it is awaited or wrapped in a task.
- Awaiting a future will not cause code to be executed, but might pause your current task until another process has completed.
- What happens then could be anything, check the documentation for the object in question.
Making an actual program
So that concludes our run down of the basic syntax for writing asynchronous code. With just this you can already create a perfectly good async program which can instantiate multiple tasks and allow them to be swapped in and out. The following example is a fully working Python program using only the things included in this post:
This program will run four tasks which print the numbers from 0 to 99, and after printing each task will yield control to allow other tasks to take over. It neatly demonstrates that asyncio allows multiple things to be done interleaved.
To actually do anything useful you’ll need to make use of one of the libraries that implement io, such as aiohttp, and when you do you might well find that there are a few things in their interfaces which I haven’t covered in this post. Specifically you’ll probably find that the interface makes use of async with and possibly also async for . So those will be the subject of the next post in this series: Python Asyncio Part 3 – Asynchronous Context Managers and Asynchronous Iterators
Python async/await руководство: базовая теория и практика
Что такое Asynchronous I/O? Что лучше, асинхронность или потоки? Конкурентность и параллелизм, кооперативная многозадачность с примерами
Теория параллельных вычислений
Рассмотрим подходы к оптимизации вычислений: истинная параллельная многозадачность (параллелизм) и псевдопараллельная многозадачность (конкурентное вычисление).
- Параллелизм подразумевает выполнение нескольких операций одновременно. Например, распределение задач по ядрам CPU. Порядок выполнения команд всегда один: control flow детерминирован.
- Конкурентное вычисление предполагает переключение контекста процессора и управление задачами через менеджер, но не истинное одновременное выполнение. Потоки или сопрограммы можно “поставить на паузу” — запускать и останавливать в одном общем процессе. Порядок выполнения команд бывает разный: control flow недетерминирован.
Приоритетная и кооперативная многозадачность
- Вытесняющая многозадачность (приоритетная, preemptive multitasking) — когда переключать контекст решает планировщик: в задачах не указано место переключения, зато указаны таймауты.
Например, многопоточность — это разделение задач по потокам. Ограничение Python Global Interpreter Lock (GIL) позволяет CPython выполнять байткод только одного потока за раз. - Невытесняющая многозадачность (совместная, cooperative multitasking) — когда переключать контекст решает сама задача: в коде явно указывается, когда вернуть выполнение планировщику. Например, асинхронный код переключает контекст оператором await в теле функции.
Проблемы конкурентных вычислений
- Взаимная блокировка (deadlock): несколько потребителей запрашивают один ресурс.
- Ресурсное голодание (starvation): поток с низким приоритетом не успевает получить доступ к ресурсу.
- Состояние гонки (race condition): результат меняется, если поменялся порядок выполнения потоков, например, два потока записывают данные в одну переменную.
- Инверсия приоритетов (priority inversion): поток с высоким приоритетом ждет результат потока с низким приоритетом.
Что такое асинхронность?
- Асинхронное программирование — стиль программирования, когда функция возвращает не сам результат, а обещание результата. Переключение контекста явно указано в теле функции. Следовательно, асинхронный код выполняется конкурентнои кооперативно.
- Async I/O, Asynchronous Input and Output — асинхронный ввод/вывод подразумевает выполнение вычислений во время ожидания результата от ввода и вывода данных.
Плюсы асинхронного программирования
- Легче избежать проблем конкурентных вычислений (блокировка, голодание, гонка, инверсия приоритетов), потому что явно указано переключение контекста.
- Сопрограммы легче масштабировать, чем потоки.
- Асинхронный код выполняется в одном потоке.
Основы Python asyncio
В одном потоке запускается цикл событий Event Loop, он выполняет задачи-футуры Task (базовый класс Future). Футуры создаются из сопрограмм Coroutine и обещают отдать результат в будущем, после ожидания с await.
Футура Task или корутина?
- Coroutine всегда нужен await!
Chain of responsibilities & composite, нужно ждать результат в теле функции. Является возвращаемым значением (результатом вызова) асинхронного генератора: type(coro) == function, type(coro()) == coroutine - Task не нужен await!
Producer-Consumer и фоновые задачи, результат не нужно ждать в теле функции. Представляет будущий результат выполнения корутины, наследует базовый класс Future: asyncio.isfuture(Task()) == True
Методы и объекты Python asyncio
- Event loop:
- asyncio.run(main()) — запуск event loop в Python 3.7+.
- asyncio.get_event_loop().run_until_complete(main()) — получение и запуск event loop в Python 3.7-.
- asyncio.get_event_loop().call_later(timeout, coro) — выполняет указанную корутину спустя указанную задержку.
- asyncio.wait_for(coro, timeout), async with asyncio.timeout(timeout) — указывает ограничение по времени для выполнения задания.
- asyncio.wait(tasks_iter, return_when, timeout) — Блокирует выполнение до достижения указанного условия. Низкоуровневый аналог gather, возвращает два контейнера, с завершенными и незавершенными задача. Для return_when доступны такие значения: FIRST_COMPLETED,FIRST_EXCEPTION, ALL_COMPLETED .
- await asyncio.gather(task(), coro()) — ждет завершения переданных футур и сопрограмм, возвращает одну общую футуру. Высокоуровневый аналог wait.
- Future:
- asyncio.create_task(coro()) — создает футуру Task и добавляет ее в конкурентную очередь задач.
- asyncio.ensure_future(coro()) — создание футуры в Python 3.7-.
- asyncio.Task.cancel() — останавливает футуру Task насильно.
- Queue:
- asyncio.Queue — создает очередь first in, first out.
- asyncio.Queue.get() — возвращает следующий элемент очереди.
- asyncio.Queue.put(x) — добавляет новый элемент в очередь.
- asyncio.Queue.join() — останавливает дальнейшее выполнение тела функции, пока очередь не опустеет.
- asyncio.Queue.task_done() — сообщает очереди, что очередной элемент обработан, и можно переходить к следующему в очереди.
Задача ограниченного буфера или Producer-consumer
Состоит из ограниченного буфера и двух типов сопрограмм: производитель и потребитель. Поставщик добавляет элементы в очередь, а потребитель обрабатывает. Задача:
- Не дать производителю добавить больше элементов, чем может вместить очередь.
- Не дать потребителю запросить элемент из пустой очереди.
Ответ: сценарий медленного потребителя для кооперативной многозадачности, семафоры для вытесняющей и параллелизма.
Сценарий медленного потребителя — потребитель оповещает поставщика, когда взял и обработал данные, а поставщик ждет оповещение. Идеально подходят объекты Task и Queue:
Python Aiohttp примеры
Aiohttp — асинхронный веб-фреймворк для связи между сервером и клиентом по стандарту REST.
Пишем асинхронный клиент requests на Aiohttp
Пишем сервер RESTful CRUD API на Aiohttp
Спасибо за чтение!
Буду благодарна, если подпишетесь и поделитесь статьей с остальными Python-разработчиками. Давайте вместе развивать наше сообщество.Async IO in Python: A Complete Walkthrough
Watch Now This tutorial has a related video course created by the Real Python team. Watch it together with the written tutorial to deepen your understanding: Hands-On Python 3 Concurrency With the asyncio Module
Async IO is a concurrent programming design that has received dedicated support in Python, evolving rapidly from Python 3.4 through 3.7, and probably beyond.
You may be thinking with dread, “Concurrency, parallelism, threading, multiprocessing. That’s a lot to grasp already. Where does async IO fit in?”
This tutorial is built to help you answer that question, giving you a firmer grasp of Python’s approach to async IO.
Here’s what you’ll cover:
Asynchronous IO (async IO): a language-agnostic paradigm (model) that has implementations across a host of programming languages
async / await : two new Python keywords that are used to define coroutines
asyncio : the Python package that provides a foundation and API for running and managing coroutines
Coroutines (specialized generator functions) are the heart of async IO in Python, and we’ll dive into them later on.
Note: In this article, I use the term async IO to denote the language-agnostic design of asynchronous IO, while asyncio refers to the Python package.
Before you get started, you’ll need to make sure you’re set up to use asyncio and other libraries found in this tutorial.
Free Bonus: 5 Thoughts On Python Mastery, a free course for Python developers that shows you the roadmap and the mindset you’ll need to take your Python skills to the next level.
Setting Up Your Environment
You’ll need Python 3.7 or above to follow this article in its entirety, as well as the aiohttp and aiofiles packages:
For help with installing Python 3.7 and setting up a virtual environment, check out Python 3 Installation & Setup Guide or Virtual Environments Primer.
With that, let’s jump in.
The 10,000-Foot View of Async IO
Async IO is a bit lesser known than its tried-and-true cousins, multiprocessing and threading. This section will give you a fuller picture of what async IO is and how it fits into its surrounding landscape.
Where Does Async IO Fit In?
Concurrency and parallelism are expansive subjects that are not easy to wade into. While this article focuses on async IO and its implementation in Python, it’s worth taking a minute to compare async IO to its counterparts in order to have context about how async IO fits into the larger, sometimes dizzying puzzle.
Parallelism consists of performing multiple operations at the same time. Multiprocessing is a means to effect parallelism, and it entails spreading tasks over a computer’s central processing units (CPUs, or cores). Multiprocessing is well-suited for CPU-bound tasks: tightly bound for loops and mathematical computations usually fall into this category.
Concurrency is a slightly broader term than parallelism. It suggests that multiple tasks have the ability to run in an overlapping manner. (There’s a saying that concurrency does not imply parallelism.)
Threading is a concurrent execution model whereby multiple threads take turns executing tasks. One process can contain multiple threads. Python has a complicated relationship with threading thanks to its GIL, but that’s beyond the scope of this article.
What’s important to know about threading is that it’s better for IO-bound tasks. While a CPU-bound task is characterized by the computer’s cores continually working hard from start to finish, an IO-bound job is dominated by a lot of waiting on input/output to complete.
To recap the above, concurrency encompasses both multiprocessing (ideal for CPU-bound tasks) and threading (suited for IO-bound tasks). Multiprocessing is a form of parallelism, with parallelism being a specific type (subset) of concurrency. The Python standard library has offered longstanding support for both of these through its multiprocessing , threading , and concurrent.futures packages.
Now it’s time to bring a new member to the mix. Over the last few years, a separate design has been more comprehensively built into CPython: asynchronous IO, enabled through the standard library’s asyncio package and the new async and await language keywords. To be clear, async IO is not a newly invented concept, and it has existed or is being built into other languages and runtime environments, such as Go, C#, or Scala.
The asyncio package is billed by the Python documentation as a library to write concurrent code. However, async IO is not threading, nor is it multiprocessing. It is not built on top of either of these.
In fact, async IO is a single-threaded, single-process design: it uses cooperative multitasking, a term that you’ll flesh out by the end of this tutorial. It has been said in other words that async IO gives a feeling of concurrency despite using a single thread in a single process. Coroutines (a central feature of async IO) can be scheduled concurrently, but they are not inherently concurrent.
To reiterate, async IO is a style of concurrent programming, but it is not parallelism. It’s more closely aligned with threading than with multiprocessing but is very much distinct from both of these and is a standalone member in concurrency’s bag of tricks.
That leaves one more term. What does it mean for something to be asynchronous? This isn’t a rigorous definition, but for our purposes here, I can think of two properties:
- Asynchronous routines are able to “pause” while waiting on their ultimate result and let other routines run in the meantime. , through the mechanism above, facilitates concurrent execution. To put it differently, asynchronous code gives the look and feel of concurrency.
Here’s a diagram to put it all together. The white terms represent concepts, and the green terms represent ways in which they are implemented or effected:
I’ll stop there on the comparisons between concurrent programming models. This tutorial is focused on the subcomponent that is async IO, how to use it, and the APIs that have sprung up around it. For a thorough exploration of threading versus multiprocessing versus async IO, pause here and check out Jim Anderson’s overview of concurrency in Python. Jim is way funnier than me and has sat in more meetings than me, to boot.
Async IO Explained
Async IO may at first seem counterintuitive and paradoxical. How does something that facilitates concurrent code use a single thread and a single CPU core? I’ve never been very good at conjuring up examples, so I’d like to paraphrase one from Miguel Grinberg’s 2017 PyCon talk, which explains everything quite beautifully:
- 24 opponents
- Judit makes each chess move in 5 seconds
- Opponents each take 55 seconds to make a move
- Games average 30 pair-moves (60 moves total)
There is only one Judit Polgár, who has only two hands and makes only one move at a time by herself. But playing asynchronously cuts the exhibition time down from 12 hours to one. So, cooperative multitasking is a fancy way of saying that a program’s event loop (more on that later) communicates with multiple tasks to let each take turns running at the optimal time.
Async IO takes long waiting periods in which functions would otherwise be blocking and allows other functions to run during that downtime. (A function that blocks effectively forbids others from running from the time that it starts until the time that it returns.)
Async IO Is Not Easy
I’ve heard it said, “Use async IO when you can; use threading when you must.” The truth is that building durable multithreaded code can be hard and error-prone. Async IO avoids some of the potential speedbumps that you might otherwise encounter with a threaded design.
But that’s not to say that async IO in Python is easy. Be warned: when you venture a bit below the surface level, async programming can be difficult too! Python’s async model is built around concepts such as callbacks, events, transports, protocols, and futures—just the terminology can be intimidating. The fact that its API has been changing continually makes it no easier.
Luckily, asyncio has matured to a point where most of its features are no longer provisional, while its documentation has received a huge overhaul and some quality resources on the subject are starting to emerge as well.
The asyncio Package and async / await
Now that you have some background on async IO as a design, let’s explore Python’s implementation. Python’s asyncio package (introduced in Python 3.4) and its two keywords, async and await , serve different purposes but come together to help you declare, build, execute, and manage asynchronous code.
The async / await Syntax and Native Coroutines
A Word of Caution: Be careful what you read out there on the Internet. Python’s async IO API has evolved rapidly from Python 3.4 to Python 3.7. Some old patterns are no longer used, and some things that were at first disallowed are now allowed through new introductions.
At the heart of async IO are coroutines. A coroutine is a specialized version of a Python generator function. Let’s start with a baseline definition and then build off of it as you progress here: a coroutine is a function that can suspend its execution before reaching return , and it can indirectly pass control to another coroutine for some time.
Later, you’ll dive a lot deeper into how exactly the traditional generator is repurposed into a coroutine. For now, the easiest way to pick up how coroutines work is to start making some.
Let’s take the immersive approach and write some async IO code. This short program is the Hello World of async IO but goes a long way towards illustrating its core functionality:
When you execute this file, take note of what looks different than if you were to define the functions with just def and time.sleep() :
The order of this output is the heart of async IO. Talking to each of the calls to count() is a single event loop, or coordinator. When each task reaches await asyncio.sleep(1) , the function yells up to the event loop and gives control back to it, saying, “I’m going to be sleeping for 1 second. Go ahead and let something else meaningful be done in the meantime.”
Contrast this to the synchronous version:
When executed, there is a slight but critical change in order and execution time:
While using time.sleep() and asyncio.sleep() may seem banal, they are used as stand-ins for any time-intensive processes that involve wait time. (The most mundane thing you can wait on is a sleep() call that does basically nothing.) That is, time.sleep() can represent any time-consuming blocking function call, while asyncio.sleep() is used to stand in for a non-blocking call (but one that also takes some time to complete).
As you’ll see in the next section, the benefit of awaiting something, including asyncio.sleep() , is that the surrounding function can temporarily cede control to another function that’s more readily able to do something immediately. In contrast, time.sleep() or any other blocking call is incompatible with asynchronous Python code, because it will stop everything in its tracks for the duration of the sleep time.
The Rules of Async IO
At this point, a more formal definition of async , await , and the coroutine functions that they create are in order. This section is a little dense, but getting a hold of async / await is instrumental, so come back to this if you need to:
The syntax async def introduces either a native coroutine or an asynchronous generator. The expressions async with and async for are also valid, and you’ll see them later on.
The keyword await passes function control back to the event loop. (It suspends the execution of the surrounding coroutine.) If Python encounters an await f() expression in the scope of g() , this is how await tells the event loop, “Suspend execution of g() until whatever I’m waiting on—the result of f() —is returned. In the meantime, go let something else run.”
In code, that second bullet point looks roughly like this:
There’s also a strict set of rules around when and how you can and cannot use async / await . These can be handy whether you are still picking up the syntax or already have exposure to using async / await :
A function that you introduce with async def is a coroutine. It may use await , return , or yield , but all of these are optional. Declaring async def noop(): pass is valid:
Using await and/or return creates a coroutine function. To call a coroutine function, you must await it to get its results.
It is less common (and only recently legal in Python) to use yield in an async def block. This creates an asynchronous generator, which you iterate over with async for . Forget about async generators for the time being and focus on getting down the syntax for coroutine functions, which use await and/or return .
Anything defined with async def may not use yield from , which will raise a SyntaxError .
Just like it’s a SyntaxError to use yield outside of a def function, it is a SyntaxError to use await outside of an async def coroutine. You can only use await in the body of coroutines.
Here are some terse examples meant to summarize the above few rules:
Finally, when you use await f() , it’s required that f() be an object that is awaitable. Well, that’s not very helpful, is it? For now, just know that an awaitable object is either (1) another coroutine or (2) an object defining an .__await__() dunder method that returns an iterator. If you’re writing a program, for the large majority of purposes, you should only need to worry about case #1.
That brings us to one more technical distinction that you may see pop up: an older way of marking a function as a coroutine is to decorate a normal def function with @asyncio.coroutine . The result is a generator-based coroutine. This construction has been outdated since the async / await syntax was put in place in Python 3.5.
These two coroutines are essentially equivalent (both are awaitable), but the first is generator-based, while the second is a native coroutine:
If you’re writing any code yourself, prefer native coroutines for the sake of being explicit rather than implicit. Generator-based coroutines will be removed in Python 3.10.
Towards the latter half of this tutorial, we’ll touch on generator-based coroutines for explanation’s sake only. The reason that async / await were introduced is to make coroutines a standalone feature of Python that can be easily differentiated from a normal generator function, thus reducing ambiguity.
Don’t get bogged down in generator-based coroutines, which have been deliberately outdated by async / await . They have their own small set of rules (for instance, await cannot be used in a generator-based coroutine) that are largely irrelevant if you stick to the async / await syntax.
Without further ado, let’s take on a few more involved examples.
Here’s one example of how async IO cuts down on wait time: given a coroutine makerandom() that keeps producing random integers in the range [0, 10], until one of them exceeds a threshold, you want to let multiple calls of this coroutine not need to wait for each other to complete in succession. You can largely follow the patterns from the two scripts above, with slight changes:
The colorized output says a lot more than I can and gives you a sense for how this script is carried out:
This program uses one main coroutine, makerandom() , and runs it concurrently across 3 different inputs. Most programs will contain small, modular coroutines and one wrapper function that serves to chain each of the smaller coroutines together. main() is then used to gather tasks (futures) by mapping the central coroutine across some iterable or pool.
In this miniature example, the pool is range(3) . In a fuller example presented later, it is a set of URLs that need to be requested, parsed, and processed concurrently, and main() encapsulates that entire routine for each URL.
While “making random integers” (which is CPU-bound more than anything) is maybe not the greatest choice as a candidate for asyncio , it’s the presence of asyncio.sleep() in the example that is designed to mimic an IO-bound process where there is uncertain wait time involved. For example, the asyncio.sleep() call might represent sending and receiving not-so-random integers between two clients in a message application.
Async IO Design Patterns
Async IO comes with its own set of possible script designs, which you’ll get introduced to in this section.
Chaining Coroutines
A key feature of coroutines is that they can be chained together. (Remember, a coroutine object is awaitable, so another coroutine can await it.) This allows you to break programs into smaller, manageable, recyclable coroutines:
Pay careful attention to the output, where part1() sleeps for a variable amount of time, and part2() begins working with the results as they become available:
In this setup, the runtime of main() will be equal to the maximum runtime of the tasks that it gathers together and schedules.
Using a Queue
The asyncio package provides queue classes that are designed to be similar to classes of the queue module. In our examples so far, we haven’t really had a need for a queue structure. In chained.py , each task (future) is composed of a set of coroutines that explicitly await each other and pass through a single input per chain.
There is an alternative structure that can also work with async IO: a number of producers, which are not associated with each other, add items to a queue. Each producer may add multiple items to the queue at staggered, random, unannounced times. A group of consumers pull items from the queue as they show up, greedily and without waiting for any other signal.
In this design, there is no chaining of any individual consumer to a producer. The consumers don’t know the number of producers, or even the cumulative number of items that will be added to the queue, in advance.
It takes an individual producer or consumer a variable amount of time to put and extract items from the queue, respectively. The queue serves as a throughput that can communicate with the producers and consumers without them talking to each other directly.
Note: While queues are often used in threaded programs because of the thread-safety of queue.Queue() , you shouldn’t need to concern yourself with thread safety when it comes to async IO. (The exception is when you’re combining the two, but that isn’t done in this tutorial.)
One use-case for queues (as is the case here) is for the queue to act as a transmitter for producers and consumers that aren’t otherwise directly chained or associated with each other.
The synchronous version of this program would look pretty dismal: a group of blocking producers serially add items to the queue, one producer at a time. Only after all producers are done can the queue be processed, by one consumer at a time processing item-by-item. There is a ton of latency in this design. Items may sit idly in the queue rather than be picked up and processed immediately.
An asynchronous version, asyncq.py , is below. The challenging part of this workflow is that there needs to be a signal to the consumers that production is done. Otherwise, await q.get() will hang indefinitely, because the queue will have been fully processed, but consumers won’t have any idea that production is complete.
(Big thanks for some help from a StackOverflow user for helping to straighten out main() : the key is to await q.join() , which blocks until all items in the queue have been received and processed, and then to cancel the consumer tasks, which would otherwise hang up and wait endlessly for additional queue items to appear.)
Here is the full script:
The first few coroutines are helper functions that return a random string, a fractional-second performance counter, and a random integer. A producer puts anywhere from 1 to 5 items into the queue. Each item is a tuple of (i, t) where i is a random string and t is the time at which the producer attempts to put the tuple into the queue.
When a consumer pulls an item out, it simply calculates the elapsed time that the item sat in the queue using the timestamp that the item was put in with.
Keep in mind that asyncio.sleep() is used to mimic some other, more complex coroutine that would eat up time and block all other execution if it were a regular blocking function.
Here is a test run with two producers and five consumers:
In this case, the items process in fractions of a second. A delay can be due to two reasons:
- Standard, largely unavoidable overhead
- Situations where all consumers are sleeping when an item appears in the queue
With regards to the second reason, luckily, it is perfectly normal to scale to hundreds or thousands of consumers. You should have no problem with python3 asyncq.py -p 5 -c 100 . The point here is that, theoretically, you could have different users on different systems controlling the management of producers and consumers, with the queue serving as the central throughput.
So far, you’ve been thrown right into the fire and seen three related examples of asyncio calling coroutines defined with async and await . If you’re not completely following or just want to get deeper into the mechanics of how modern coroutines came to be in Python, you’ll start from square one with the next section.
Async IO’s Roots in Generators
Earlier, you saw an example of the old-style generator-based coroutines, which have been outdated by more explicit native coroutines. The example is worth re-showing with a small tweak:
As an experiment, what happens if you call py34_coro() or py35_coro() on its own, without await , or without any calls to asyncio.run() or other asyncio “porcelain” functions? Calling a coroutine in isolation returns a coroutine object:
This isn’t very interesting on its surface. The result of calling a coroutine on its own is an awaitable coroutine object.
Time for a quiz: what other feature of Python looks like this? (What feature of Python doesn’t actually “do much” when it’s called on its own?)
Hopefully you’re thinking of generators as an answer to this question, because coroutines are enhanced generators under the hood. The behavior is similar in this regard:
Generator functions are, as it so happens, the foundation of async IO (regardless of whether you declare coroutines with async def rather than the older @asyncio.coroutine wrapper). Technically, await is more closely analogous to yield from than it is to yield . (But remember that yield from x() is just syntactic sugar to replace for i in x(): yield i .)
One critical feature of generators as it pertains to async IO is that they can effectively be stopped and restarted at will. For example, you can break out of iterating over a generator object and then resume iteration on the remaining values later. When a generator function reaches yield , it yields that value, but then it sits idle until it is told to yield its subsequent value.
This can be fleshed out through an example:
The await keyword behaves similarly, marking a break point at which the coroutine suspends itself and lets other coroutines work. “Suspended,” in this case, means a coroutine that has temporarily ceded control but not totally exited or finished. Keep in mind that yield , and by extension yield from and await , mark a break point in a generator’s execution.
This is the fundamental difference between functions and generators. A function is all-or-nothing. Once it starts, it won’t stop until it hits a return , then pushes that value to the caller (the function that calls it). A generator, on the other hand, pauses each time it hits a yield and goes no further. Not only can it push this value to calling stack, but it can keep a hold of its local variables when you resume it by calling next() on it.
There’s a second and lesser-known feature of generators that also matters. You can send a value into a generator as well through its .send() method. This allows generators (and coroutines) to call ( await ) each other without blocking. I won’t get any further into the nuts and bolts of this feature, because it matters mainly for the implementation of coroutines behind the scenes, but you shouldn’t ever really need to use it directly yourself.
If you’re interested in exploring more, you can start at PEP 342, where coroutines were formally introduced. Brett Cannon’s How the Heck Does Async-Await Work in Python is also a good read, as is the PYMOTW writeup on asyncio . Lastly, there’s David Beazley’s Curious Course on Coroutines and Concurrency, which dives deep into the mechanism by which coroutines run.
Let’s try to condense all of the above articles into a few sentences: there is a particularly unconventional mechanism by which these coroutines actually get run. Their result is an attribute of the exception object that gets thrown when their .send() method is called. There’s some more wonky detail to all of this, but it probably won’t help you use this part of the language in practice, so let’s move on for now.
To tie things together, here are some key points on the topic of coroutines as generators:
Coroutines are repurposed generators that take advantage of the peculiarities of generator methods.
Old generator-based coroutines use yield from to wait for a coroutine result. Modern Python syntax in native coroutines simply replaces yield from with await as the means of waiting on a coroutine result. The await is analogous to yield from , and it often helps to think of it as such.
The use of await is a signal that marks a break point. It lets a coroutine temporarily suspend execution and permits the program to come back to it later.
Other Features: async for and Async Generators + Comprehensions
Along with plain async / await , Python also enables async for to iterate over an asynchronous iterator. The purpose of an asynchronous iterator is for it to be able to call asynchronous code at each stage when it is iterated over.
A natural extension of this concept is an asynchronous generator. Recall that you can use await , return , or yield in a native coroutine. Using yield within a coroutine became possible in Python 3.6 (via PEP 525), which introduced asynchronous generators with the purpose of allowing await and yield to be used in the same coroutine function body:
Last but not least, Python enables asynchronous comprehension with async for . Like its synchronous cousin, this is largely syntactic sugar:
This is a crucial distinction: neither asynchronous generators nor comprehensions make the iteration concurrent. All that they do is provide the look-and-feel of their synchronous counterparts, but with the ability for the loop in question to give up control to the event loop for some other coroutine to run.
In other words, asynchronous iterators and asynchronous generators are not designed to concurrently map some function over a sequence or iterator. They’re merely designed to let the enclosing coroutine allow other tasks to take their turn. The async for and async with statements are only needed to the extent that using plain for or with would “break” the nature of await in the coroutine. This distinction between asynchronicity and concurrency is a key one to grasp.
The Event Loop and asyncio.run()
You can think of an event loop as something like a while True loop that monitors coroutines, taking feedback on what’s idle, and looking around for things that can be executed in the meantime. It is able to wake up an idle coroutine when whatever that coroutine is waiting on becomes available.
Thus far, the entire management of the event loop has been implicitly handled by one function call:
asyncio.run() , introduced in Python 3.7, is responsible for getting the event loop, running tasks until they are marked as complete, and then closing the event loop.
There’s a more long-winded way of managing the asyncio event loop, with get_event_loop() . The typical pattern looks like this:
You’ll probably see loop.get_event_loop() floating around in older examples, but unless you have a specific need to fine-tune control over the event loop management, asyncio.run() should be sufficient for most programs.
If you do need to interact with the event loop within a Python program, loop is a good-old-fashioned Python object that supports introspection with loop.is_running() and loop.is_closed() . You can manipulate it if you need to get more fine-tuned control, such as in scheduling a callback by passing the loop as an argument.
What is more crucial is understanding a bit beneath the surface about the mechanics of the event loop. Here are a few points worth stressing about the event loop.
#1: Coroutines don’t do much on their own until they are tied to the event loop.
You saw this point before in the explanation on generators, but it’s worth restating. If you have a main coroutine that awaits others, simply calling it in isolation has little effect:
Remember to use asyncio.run() to actually force execution by scheduling the main() coroutine (future object) for execution on the event loop:
(Other coroutines can be executed with await . It is typical to wrap just main() in asyncio.run() , and chained coroutines with await will be called from there.)
#2: By default, an async IO event loop runs in a single thread and on a single CPU core. Usually, running one single-threaded event loop in one CPU core is more than sufficient. It is also possible to run event loops across multiple cores. Check out this talk by John Reese for more, and be warned that your laptop may spontaneously combust.
#3. Event loops are pluggable. That is, you could, if you really wanted, write your own event loop implementation and have it run tasks just the same. This is wonderfully demonstrated in the uvloop package, which is an implementation of the event loop in Cython.
That is what is meant by the term “pluggable event loop”: you can use any working implementation of an event loop, unrelated to the structure of the coroutines themselves. The asyncio package itself ships with two different event loop implementations, with the default being based on the selectors module. (The second implementation is built for Windows only.)
A Full Program: Asynchronous Requests
You’ve made it this far, and now it’s time for the fun and painless part. In this section, you’ll build a web-scraping URL collector, areq.py , using aiohttp , a blazingly fast async HTTP client/server framework. (We just need the client part.) Such a tool could be used to map connections between a cluster of sites, with the links forming a directed graph.
Note: You may be wondering why Python’s requests package isn’t compatible with async IO. requests is built on top of urllib3 , which in turn uses Python’s http and socket modules.
By default, socket operations are blocking. This means that Python won’t like await requests.get(url) because .get() is not awaitable. In contrast, almost everything in aiohttp is an awaitable coroutine, such as session.request() and response.text() . It’s a great package otherwise, but you’re doing yourself a disservice by using requests in asynchronous code.
The high-level program structure will look like this:
Read a sequence of URLs from a local file, urls.txt .
Send GET requests for the URLs and decode the resulting content. If this fails, stop there for a URL.
Search for the URLs within href tags in the HTML of the responses.
Write the results to foundurls.txt .
Do all of the above as asynchronously and concurrently as possible. (Use aiohttp for the requests, and aiofiles for the file-appends. These are two primary examples of IO that are well-suited for the async IO model.)
Here are the contents of urls.txt . It’s not huge, and contains mostly highly trafficked sites:
The second URL in the list should return a 404 response, which you’ll need to handle gracefully. If you’re running an expanded version of this program, you’ll probably need to deal with much hairier problems than this, such a server disconnections and endless redirects.
The requests themselves should be made using a single session, to take advantage of reusage of the session’s internal connection pool.
Let’s take a look at the full program. We’ll walk through things step-by-step after:
This script is longer than our initial toy programs, so let’s break it down.
The constant HREF_RE is a regular expression to extract what we’re ultimately searching for, href tags within HTML:
The coroutine fetch_html() is a wrapper around a GET request to make the request and decode the resulting page HTML. It makes the request, awaits the response, and raises right away in the case of a non-200 status:
If the status is okay, fetch_html() returns the page HTML (a str ). Notably, there is no exception handling done in this function. The logic is to propagate that exception to the caller and let it be handled there:
We await session.request() and resp.text() because they’re awaitable coroutines. The request/response cycle would otherwise be the long-tailed, time-hogging portion of the application, but with async IO, fetch_html() lets the event loop work on other readily available jobs such as parsing and writing URLs that have already been fetched.
Next in the chain of coroutines comes parse() , which waits on fetch_html() for a given URL, and then extracts all of the href tags from that page’s HTML, making sure that each is valid and formatting it as an absolute path.
Admittedly, the second portion of parse() is blocking, but it consists of a quick regex match and ensuring that the links discovered are made into absolute paths.
In this specific case, this synchronous code should be quick and inconspicuous. But just remember that any line within a given coroutine will block other coroutines unless that line uses yield , await , or return . If the parsing was a more intensive process, you might want to consider running this portion in its own process with loop.run_in_executor() .
Next, the coroutine write() takes a file object and a single URL, and waits on parse() to return a set of the parsed URLs, writing each to the file asynchronously along with its source URL through use of aiofiles , a package for async file IO.
Lastly, bulk_crawl_and_write() serves as the main entry point into the script’s chain of coroutines. It uses a single session, and a task is created for each URL that is ultimately read from urls.txt .
Here are a few additional points that deserve mention:
The default ClientSession has an adapter with a maximum of 100 open connections. To change that, pass an instance of asyncio.connector.TCPConnector to ClientSession . You can also specify limits on a per-host basis.
You can specify max timeouts for both the session as a whole and for individual requests.
This script also uses async with , which works with an asynchronous context manager. I haven’t devoted a whole section to this concept because the transition from synchronous to asynchronous context managers is fairly straightforward. The latter has to define .__aenter__() and .__aexit__() rather than .__exit__() and .__enter__() . As you might expect, async with can only be used inside a coroutine function declared with async def .
If you’d like to explore a bit more, the companion files for this tutorial up at GitHub have comments and docstrings attached as well.
Here’s the execution in all of its glory, as areq.py gets, parses, and saves results for 9 URLs in under a second:
That’s not too shabby! As a sanity check, you can check the line-count on the output. In my case, it’s 626, though keep in mind this may fluctuate:
Next Steps: If you’d like to up the ante, make this webcrawler recursive. You can use aio-redis to keep track of which URLs have been crawled within the tree to avoid requesting them twice, and connect links with Python’s networkx library.
Remember to be nice. Sending 1000 concurrent requests to a small, unsuspecting website is bad, bad, bad. There are ways to limit how many concurrent requests you’re making in one batch, such as in using the sempahore objects of asyncio or using a pattern like this one. If you don’t heed this warning, you may get a massive batch of TimeoutError exceptions and only end up hurting your own program.
Async IO in Context
Now that you’ve seen a healthy dose of code, let’s step back for a minute and consider when async IO is an ideal option and how you can make the comparison to arrive at that conclusion or otherwise choose a different model of concurrency.
When and Why Is Async IO the Right Choice?
This tutorial is no place for an extended treatise on async IO versus threading versus multiprocessing. However, it’s useful to have an idea of when async IO is probably the best candidate of the three.
The battle over async IO versus multiprocessing is not really a battle at all. In fact, they can be used in concert. If you have multiple, fairly uniform CPU-bound tasks (a great example is a grid search in libraries such as scikit-learn or keras ), multiprocessing should be an obvious choice.
Simply putting async before every function is a bad idea if all of the functions use blocking calls. (This can actually slow down your code.) But as mentioned previously, there are places where async IO and multiprocessing can live in harmony.
The contest between async IO and threading is a little bit more direct. I mentioned in the introduction that “threading is hard.” The full story is that, even in cases where threading seems easy to implement, it can still lead to infamous impossible-to-trace bugs due to race conditions and memory usage, among other things.
Threading also tends to scale less elegantly than async IO, because threads are a system resource with a finite availability. Creating thousands of threads will fail on many machines, and I don’t recommend trying it in the first place. Creating thousands of async IO tasks is completely feasible.
Async IO shines when you have multiple IO-bound tasks where the tasks would otherwise be dominated by blocking IO-bound wait time, such as:
Network IO, whether your program is the server or the client side
Serverless designs, such as a peer-to-peer, multi-user network like a group chatroom
Read/write operations where you want to mimic a “fire-and-forget” style but worry less about holding a lock on whatever you’re reading and writing to
The biggest reason not to use it is that await only supports a specific set of objects that define a specific set of methods. If you want to do async read operations with a certain DBMS, you’ll need to find not just a Python wrapper for that DBMS, but one that supports the async / await syntax. Coroutines that contain synchronous calls block other coroutines and tasks from running.
For a shortlist of libraries that work with async / await , see the list at the end of this tutorial.
Async IO It Is, but Which One?
This tutorial focuses on async IO, the async / await syntax, and using asyncio for event-loop management and specifying tasks. asyncio certainly isn’t the only async IO library out there. This observation from Nathaniel J. Smith says a lot:
[In] a few years, asyncio might find itself relegated to becoming one of those stdlib libraries that savvy developers avoid, like urllib2 .
…
What I’m arguing, in effect, is that asyncio is a victim of its own success: when it was designed, it used the best approach possible; but since then, work inspired by asyncio – like the addition of async / await – has shifted the landscape so that we can do even better, and now asyncio is hamstrung by its earlier commitments. (Source)
To that end, a few big-name alternatives that do what asyncio does, albeit with different APIs and different approaches, are curio and trio . Personally, I think that if you’re building a moderately sized, straightforward program, just using asyncio is plenty sufficient and understandable, and lets you avoid adding yet another large dependency outside of Python’s standard library.
But by all means, check out curio and trio , and you might find that they get the same thing done in a way that’s more intuitive for you as the user. Many of the package-agnostic concepts presented here should permeate to alternative async IO packages as well.
Odds and Ends
In these next few sections, you’ll cover some miscellaneous parts of asyncio and async / await that haven’t fit neatly into the tutorial thus far, but are still important for building and understanding a full program.
Other Top-Level asyncio Functions
In addition to asyncio.run() , you’ve seen a few other package-level functions such as asyncio.create_task() and asyncio.gather() .
You can use create_task() to schedule the execution of a coroutine object, followed by asyncio.run() :
There’s a subtlety to this pattern: if you don’t await t within main() , it may finish before main() itself signals that it is complete. Because asyncio.run(main()) calls loop.run_until_complete(main()) , the event loop is only concerned (without await t present) that main() is done, not that the tasks that get created within main() are done. Without await t , the loop’s other tasks will be cancelled, possibly before they are completed. If you need to get a list of currently pending tasks, you can use asyncio.Task.all_tasks() .
Note: asyncio.create_task() was introduced in Python 3.7. In Python 3.6 or lower, use asyncio.ensure_future() in place of create_task() .
Separately, there’s asyncio.gather() . While it doesn’t do anything tremendously special, gather() is meant to neatly put a collection of coroutines (futures) into a single future. As a result, it returns a single future object, and, if you await asyncio.gather() and specify multiple tasks or coroutines, you’re waiting for all of them to be completed. (This somewhat parallels queue.join() from our earlier example.) The result of gather() will be a list of the results across the inputs:
You probably noticed that gather() waits on the entire result set of the Futures or coroutines that you pass it. Alternatively, you can loop over asyncio.as_completed() to get tasks as they are completed, in the order of completion. The function returns an iterator that yields tasks as they finish. Below, the result of coro([3, 2, 1]) will be available before coro([10, 5, 0]) is complete, which is not the case with gather() :
Lastly, you may also see asyncio.ensure_future() . You should rarely need it, because it’s a lower-level plumbing API and largely replaced by create_task() , which was introduced later.
The Precedence of await
While they behave somewhat similarly, the await keyword has significantly higher precedence than yield . This means that, because it is more tightly bound, there are a number of instances where you’d need parentheses in a yield from statement that are not required in an analogous await statement. For more information, see examples of await expressions from PEP 492.
Conclusion
You’re now equipped to use async / await and the libraries built off of it. Here’s a recap of what you’ve covered:
Asynchronous IO as a language-agnostic model and a way to effect concurrency by letting coroutines indirectly communicate with each other
The specifics of Python’s new async and await keywords, used to mark and define coroutines
asyncio , the Python package that provides the API to run and manage coroutines
Resources
Python Version Specifics
Async IO in Python has evolved swiftly, and it can be hard to keep track of what came when. Here’s a list of Python minor-version changes and introductions related to asyncio :
3.3: The yield from expression allows for generator delegation.
3.4: asyncio was introduced in the Python standard library with provisional API status.
3.5: async and await became a part of the Python grammar, used to signify and wait on coroutines. They were not yet reserved keywords. (You could still define functions or variables named async and await .)
3.6: Asynchronous generators and asynchronous comprehensions were introduced. The API of asyncio was declared stable rather than provisional.
3.7: async and await became reserved keywords. (They cannot be used as identifiers.) They are intended to replace the asyncio.coroutine() decorator. asyncio.run() was introduced to the asyncio package, among a bunch of other features.
If you want to be safe (and be able to use asyncio.run() ), go with Python 3.7 or above to get the full set of features.
Articles
Here’s a curated list of additional resources:
- Real Python: Speed up your Python Program with Concurrency
- Real Python: What is the Python Global Interpreter Lock?
- CPython: The asyncio package source
- Python docs: Data model > Coroutines
- TalkPython: Async Techniques and Examples in Python
- Brett Cannon: How the Heck Does Async-Await Work in Python 3.5?
- PYMOTW: asyncio
- A. Jesse Jiryu Davis and Guido van Rossum: A Web Crawler With asyncio Coroutines
- Andy Pearce: The State of Python Coroutines: yield from
- Nathaniel J. Smith: Some Thoughts on Asynchronous API Design in a Post- async / await World
- Armin Ronacher: I don’t understand Python’s Asyncio
- Andy Balaam: series on asyncio (4 posts)
- Stack Overflow: Python asyncio.semaphore in async — await function
- Yeray Diaz:
A few Python What’s New sections explain the motivation behind language changes in more detail:
- A coroutine object, which is the return value of a coroutine function defined using async def .