What Does Synchronization With Asyncio Look Like

The most general view of program development approaches claims we have two basic coding options. The first one, synchronous code architecture, works step-by-step, and all the processes run the same way. For instance, each process performs data input and receives output from the server. Consequently, if we have operation one and process two, as well as input/output one and input/output two, input two starts only after output one. 

In asynchronous code, the architecture is not that strict and hierarchic. Our example with two operations looks different, as both operations can be performed simultaneously and remain independent. It is also called concurrent code. 

The main advantage of asynchronous coding is increased application performance. Response rates are higher due to independence in processes. The main drawback is that every request is on its own, and with too many requests, the CPU core won't be able to deal with them.

Python Asynchronous Programming Explained

A few working approaches to discuss in Python asynchronous programming are Multiprocessing, Multitasking, and Concurrency.

Briefly About Multiprocessing and Multitasking 

Multiprocessing is a common processing type when each task is performed independently yet simultaneously. Multiprocessing requires an additional CPU to handle multiple processes. 

Multitasking allows running more than one process simultaneously on the same processor. Multiprocessing supports switching between processes or context switching. 

What Is Concurrency?

Two previous processing types are quite easy to understand. Let's answer the question of what concurrency is. Concurrent code supports the processing of multiple tasks at the same time. They are executed in sequence, yet this sequence is flexible. So, in a way, processes communicate or interrupt each other. These sequences are called threads. Therefore, a single-threaded program executes one sequence/thread. 

On the other hand, the multithreading program executes more than one thread simultaneously. Python asynchronous function such as multithreading means threads regularly interrupt each other and are executed. For a resource-intensive task, the thread might be lowlily prioritized and frozen for some time. Speaking, Python multithreading allows changing the order of performing various tasks or requests. 

Asyncio Applications

Asyncio is a library to write concurrent code using the async/await syntax. In the applications that can benefit from exploiting concurrency, Asyncio is added to simplify the syntax necessary to consume promise-based APIs. It is a foundation for many Python asynchronous frameworks to provide high-performance network and web servers, database connection libraries, distributed task queues, etc. 

Asyncio requires a Python event loop feature to routinely measure the progress of tasks. In case the system sees a task in progress, it can automatically start the next one, avoiding unnecessary waiting. The Python event loop makes it possible to automate the queue, starting the tasks in a different order and not one by one. It is beneficial in projects based on microservice architecture and the ones that deal with tons of data: databases, files, and large amounts of information.  

 When discussing benefits in a microservice architecture, Asyncio allows better communication. Communicating between microservices is usually over REST or gRPC protocol. To talk to many microservices at once, we can run requests concurrently. Another plus for microservice architecture is error handling of the Asyncio APIs, such as wait and gather. They allow handling exceptions from groups of tasks and coroutines.

Asynchronous Python Code Samples

Python asynchronous programming works great with databases and with the use of coroutines. Coroutines are used for cooperative multitasking, where a process voluntarily gives away control periodically to enable multiple applications to be run simultaneously.

Let's look at the example. We included need all the necessary components to show a small case study. We used ArrangoDB —  a free graph database; RabbitMQ —  a free message broker; Asyncio library; and coroutines. First things first, let's connect the database and declare the coroutine:

from aioarango import ArangoClient
async def arango_context(db_name, host, username, password):
client = ArangoClient(hosts=host)
       return await client.db(name=db_name, username=username, password=password)

The "arango_context" function here is a coroutine. Next, we need an "await" keyword when we call it:
Class Integration:

async def db(self):
return await arango_context(
db_name=AS.DATABASE, host=AS.HOST,
username=AS.USER, password=AS.PASSWORD)

Now let’s create a sample data class that retries running a callback function. For instance, we want to test if the data was transformed and saved to the database. Saving data can take time, so we will try to get data after sleeping for three seconds. Only after five failed attempts will an exception arise:

class Retry:

async def run(self, callback_func, *args):
result = None
retry = 0
while True:
if not result and retry >= 5:
raise Exception('Retry exhausted')

try:
result = await callback_func(*args)
if result:
return result

except Exception as exc:
key = f'for {args[0].key}' if args else ''
logger.info('Got an error %s, will retry. Exception: %s', key, exc)

retry += 1
           await asyncio.sleep(3)

Setting up a Python event loop comes next, as it’s the most important aspect of asynchronous programming. The following Python code sample shows how to consume multiple queues concurrently using Python and Asyncio. The first step is to create a Python event loop. The connection to the RabbitMQ will be closed when the service is stopped. Loop will run nonstop, waiting for a RabbitMQ message to the specific processor queue: async def main(loop):

client = Client(settings=rabbit_settings)
await client.build_client()
processors = [
CompanyUpdatedProcessor,
CompanyInvalidatedProcessor,
CompanySyncProcessor,
]
await client.subscribe(processors)
return client.connection


if __name__ == '__main__':
logger.info('Service sync started.')
loop = asyncio.get_event_loop()
connection = loop.run_until_complete(main(loop=loop))
try:
loop.run_forever()
except KeyboardInterrupt:
logger.info('Service sync stopped.')
except Exception as exc:
logger.exception(exc)
finally:
       loop.run_until_complete(connection.close())

 async def consume(self, context, channel, message):
task = asyncio.create_task(self.send_message(context, channel, message))
       done, pending = await asyncio.wait({task}, timeout=None, return_when=ALL_COMPLETED)

Short Sum Up

Python asynchronous programming becomes useful when the project involves microservice architecture, is heavy on processes and needs Multiprocessing, Multitasking, or Concurrency code. In the case of the last one, Asyncio applications are of great use, along with databases, message brokers, and coroutines. Async/await syntax in Python allows multiple promises to be initiated and resolved for values when required.

Let’s summarize what we discussed: 

1. Difference between synchronous and asynchronous programming works
2. Main notions about the asynchronous mechanism. 
3. General overview of Asyncio applications. 
4. Python code samples that show what synchronization with Asyncio looks like. 

You are ready to write web apps that can handle multiple processes.