Hướng dẫn python asyncio tutorial

When and Why Is Async IO the Right Choice?

This tutorial is no place for an extended treatise on async IO versus threading versus multiprocessing. However, it’s useful to have an idea of when async IO is probably the best candidate of the three.

The battle over async IO versus multiprocessing is not really a battle at all. In fact, they can be used in concert. If you have multiple, fairly uniform CPU-bound tasks [a great example is a grid search in libraries such as scikit-learn or keras], multiprocessing should be an obvious choice.

Simply putting async before every function is a bad idea if all of the functions use blocking calls. [This can actually slow down your code.] But as mentioned previously, there are places where async IO and multiprocessing can live in harmony.

The contest between async IO and threading is a little bit more direct. I mentioned in the introduction that “threading is hard.” The full story is that, even in cases where threading seems easy to implement, it can still lead to infamous impossible-to-trace bugs due to race conditions and memory usage, among other things.

Threading also tends to scale less elegantly than async IO, because threads are a system resource with a finite availability. Creating thousands of threads will fail on many machines, and I don’t recommend trying it in the first place. Creating thousands of async IO tasks is completely feasible.

Async IO shines when you have multiple IO-bound tasks where the tasks would otherwise be dominated by blocking IO-bound wait time, such as:

• Network IO, whether your program is the server or the client side

• Serverless designs, such as a peer-to-peer, multi-user network like a group chatroom

• Read/write operations where you want to mimic a “fire-and-forget” style but worry less about holding a lock on whatever you’re reading and writing to

The biggest reason not to use it is that await only supports a specific set of objects that define a specific set of methods. If you want to do async read operations with a certain DBMS, you’ll need to find not just a Python wrapper for that DBMS, but one that supports the async/await syntax. Coroutines that contain synchronous calls block other coroutines and tasks from running.

For a shortlist of libraries that work with async/await, see the list at the end of this tutorial.

Async IO It Is, but Which One?

This tutorial focuses on async IO, the async/await syntax, and using asyncio for event-loop management and specifying tasks. asyncio certainly isn’t the only async IO library out there. This observation from Nathaniel J. Smith says a lot:

[In] a few years, asyncio might find itself relegated to becoming one of those stdlib libraries that savvy developers avoid, like urllib2.

…

What I’m arguing, in effect, is that asyncio is a victim of its own success: when it was designed, it used the best approach possible; but since then, work inspired by asyncio – like the addition of async/await – has shifted the landscape so that we can do even better, and now asyncio is hamstrung by its earlier commitments. [Source]

To that end, a few big-name alternatives that do what asyncio does, albeit with different APIs and different approaches, are curio and trio. Personally, I think that if you’re building a moderately sized, straightforward program, just using asyncio is plenty sufficient and understandable, and lets you avoid adding yet another large dependency outside of Python’s standard library.

But by all means, check out curio and trio, and you might find that they get the same thing done in a way that’s more intuitive for you as the user. Many of the package-agnostic concepts presented here should permeate to alternative async IO packages as well.

Python Version Specifics

Async IO in Python has evolved swiftly, and it can be hard to keep track of what came when. Here’s a list of Python minor-version changes and introductions related to asyncio:

• 3.3: The yield from expression allows for generator delegation.

• 3.4: asyncio was introduced in the Python standard library with provisional API status.

• 3.5: async and await became a part of the Python grammar, used to signify and wait on coroutines. They were not yet reserved keywords. [You could still define functions or variables named async and await.]

• 3.6: Asynchronous generators and asynchronous comprehensions were introduced. The API of asyncio was declared stable rather than provisional.

• 3.7: async and await became reserved keywords. [They cannot be used as identifiers.] They are intended to replace the asyncio.coroutine[] decorator. asyncio.run[] was introduced to the asyncio package, among a bunch of other features.

If you want to be safe [and be able to use asyncio.run[]], go with Python 3.7 or above to get the full set of features.

Articles

Here’s a curated list of additional resources:

• Real Python: Speed up your Python Program with Concurrency
• Real Python: What is the Python Global Interpreter Lock?
• CPython: The asyncio package source
• Python docs: Data model > Coroutines
• TalkPython: Async Techniques and Examples in Python
• Brett Cannon: How the Heck Does Async-Await Work in Python 3.5?
• PYMOTW: asyncio
• A. Jesse Jiryu Davis and Guido van Rossum: A Web Crawler With asyncio Coroutines
• Andy Pearce: The State of Python Coroutines: yield from
• Nathaniel J. Smith: Some Thoughts on Asynchronous API Design in a Post-async/await World
• Armin Ronacher: I don’t understand Python’s Asyncio
• Andy Balaam: series on asyncio [4 posts]
• Stack Overflow: Python asyncio.semaphore in async-await function
• Yeray Diaz:
  • AsyncIO for the Working Python Developer
  • Asyncio Coroutine Patterns: Beyond await

A few Python What’s New sections explain the motivation behind language changes in more detail:

• What’s New in Python 3.3 [yield from and PEP 380]
• What’s New in Python 3.6 [PEP 525 & 530]

From David Beazley:

• Generator: Tricks for Systems Programmers
• A Curious Course on Coroutines and Concurrency
• Generators: The Final Frontier

YouTube talks:

• John Reese - Thinking Outside the GIL with AsyncIO and Multiprocessing - PyCon 2018
• Keynote David Beazley - Topics of Interest [Python Asyncio]
• David Beazley - Python Concurrency From the Ground Up: LIVE! - PyCon 2015
• Raymond Hettinger, Keynote on Concurrency, PyBay 2017
• Thinking about Concurrency, Raymond Hettinger, Python core developer
• Miguel Grinberg Asynchronous Python for the Complete Beginner PyCon 2017
• Yury Selivanov asyncawait and asyncio in Python 3 6 and beyond PyCon 2017
• Fear and Awaiting in Async: A Savage Journey to the Heart of the Coroutine Dream
• What Is Async, How Does It Work, and When Should I Use It? [PyCon APAC 2014]

Libraries That Work With async/await

From aio-libs:

• aiohttp: Asynchronous HTTP client/server framework
• aioredis: Async IO Redis support
• aiopg: Async IO PostgreSQL support
• aiomcache: Async IO memcached client
• aiokafka: Async IO Kafka client
• aiozmq: Async IO ZeroMQ support
• aiojobs: Jobs scheduler for managing background tasks
• async_lru: Simple LRU cache for async IO

From magicstack:

• uvloop: Ultra fast async IO event loop
• asyncpg: [Also very fast] async IO PostgreSQL support

From other hosts:

• trio: Friendlier asyncio intended to showcase a radically simpler design
• aiofiles: Async file IO
• asks: Async requests-like http library
• asyncio-redis: Async IO Redis support
• aioprocessing: Integrates multiprocessing module with asyncio
• umongo: Async IO MongoDB client
• unsync: Unsynchronize asyncio
• aiostream: Like itertools, but async