Asynchronous and non-Blocking I/O of Tornado

Real-time web features require a long-lived mostly-idle connection peruser. In a traditional synchronous web server, this implies devotingone thread to each user, which can be very expensive.

To minimize the cost of concurrent connections, Tornado uses asingle-threaded event loop. This means that all application codeshould aim to be asynchronous and non-blocking because only oneoperation can be active at a time.

The terms asynchronous and non-blocking are closely related and areoften used interchangeably, but they are not quite the same thing.

Blocking

A function blocks when it waits for something to happen beforereturning. A function may block for many reasons: network I/O, diskI/O, mutexes, etc. In fact, every function blocks, at least alittle bit, while it is running and using the CPU (for an extremeexample that demonstrates why CPU blocking must be taken as seriouslyas other kinds of blocking, consider password hashing functions likebcrypt, which by design usehundreds of milliseconds of CPU time, far more than a typical networkor disk access).

A function can be blocking in some respects and non-blocking inothers. For example, tornado.httpclient in the defaultconfiguration blocks on DNS resolution but not on other network access(to mitigate this use ThreadedResolver or atornado.curl_httpclient with a properly-configured build oflibcurl). In the context of Tornado we generally talk aboutblocking in the context of network I/O, although all kinds of blockingare to be minimized.

Asynchronous

An asynchronous function returns before it is finished, andgenerally causes some work to happen in the background beforetriggering some future action in the application (as opposed to normalsynchronous functions, which do everything they are going to dobefore returning). There are many styles of asynchronous interfaces:

• Callback argument
• Return a placeholder (Future, Promise, Deferred)
• Deliver to a queue
• Callback registry (e.g. POSIX signals)

Regardless of which type of interface is used, asynchronous functionsby definition interact differently with their callers; there is nofree way to make a synchronous function asynchronous in a way that istransparent to its callers (systems like gevent use lightweight threads to offer performancecomparable to asynchronous systems, but they do not actually makethings asynchronous).

Examples

Here is a sample synchronous function:

from tornado.httpclient import HTTPClient

def synchronous_fetch(url):
    http_client = HTTPClient()
    response = http_client.fetch(url)
    return response.body

And here is the same function rewritten to be asynchronous with acallback argument:

from tornado.httpclient import AsyncHTTPClient

def asynchronous_fetch(url, callback):
    http_client = AsyncHTTPClient()
    def handle_response(response):
        callback(response.body)
    http_client.fetch(url, callback=handle_response)

And again with a Future instead of a callback:

from tornado.concurrent import Future

def async_fetch_future(url):
    http_client = AsyncHTTPClient()
    my_future = Future()
    fetch_future = http_client.fetch(url)
    fetch_future.add_done_callback(
        lambda f: my_future.set_result(f.result()))
    return my_future

The raw Future version is more complex, but Futures arenonetheless recommended practice in Tornado because they have twomajor advantages. Error handling is more consistent since theFuture.result method can simply raise an exception (as opposed tothe ad-hoc error handling common in callback-oriented interfaces), andFutures lend themselves well to use with coroutines. Coroutineswill be discussed in depth in the next section of this guide. Here isthe coroutine version of our sample function, which is very similar tothe original synchronous version:

from tornado import gen

@gen.coroutine
def fetch_coroutine(url):
    http_client = AsyncHTTPClient()
    response = yield http_client.fetch(url)
    return response.body

http://www.tornadoweb.org/en/stable/guide/async.html