The great mystery on how to make durable, asynchronous functions for asio.

5 min read

Asio is increasingly becoming a core component in both open source and commercial projects.

Asio is a header only library, utilizing clever, black belt C++ template meta-programming. I will not pretend that I comprehend much of it's internal workings.

This article is not an introduction to asio. I assume that you are already familiar with the basics, and know how to implement a highly performant asynchronous echo server or client, using asio. That is actually quite easy, and just require you to comprehend the APIs for handling sockets and so called continuations, in whatever asynch paradigm you prefer.

So, say for example that you like C++20 coroutines. To implement an echo client, you could cook together something like:

void echo(const std::string& message,
          const std::string& server,
          const std::string& port) {

    co_spawn(boost::asio::system_executor(), [&]() ->awaitable<void>  {
        auto executor = get_associated_executor(use_awaitable);

        // Initialize a TCP socket
        tcp::socket s{executor};

        // Resolve the hostname and port
        tcp::resolver resolver{executor};
        tcp::resolver::query q{server, port};
        tcp::resolver::iterator ep = co_await resolver.async_resolve(q, use_awaitable);

        // Connect asynchronously to the echo server
        co_await s.async_connect(*ep, use_awaitable);

        // Asynchronously send the message
        co_await boost::asio::async_write(s, boost::asio::buffer(message), use_awaitable);

        // Prepare a buffer for the reply
        std::array<char, 1024> b;
        boost::asio::mutable_buffer buffer{b.data(), b.size()};

        // Asynchronously wait for and receive the reply
        const auto bytes = co_await s.async_receive(buffer, use_awaitable);

        // Check that we got the same message as we sent
        assert(bytes == message.size());
        assert(message == string_view(b.data(), bytes));

        // Exit the C++ 20 coroutine
        co_return;
    }, detached);
}

Simple, and reasonable efficient. (According to my measurements, C++ coroutines performs a little better than asios stackful coroutines). This function will return immediately, and the coroutine will be executed by one of the threads powering the the provided io_service.

As you probably already know, the asynchronous asio functions above has the same interface no matter if you use a callback functor, stackful coroutine context or a request for a std::future<> as the last argument. As long as you pick one paradigm, and stick to it, asio programming is pretty straight forward. Personally, I have used stackful coroutines for most of my asio code since C++ 11. It allows for simple code with full stack-traces if the app crash. And the performance is fine in most circumstances.

The pain with asio comes when you need to do something more advanced than simple request --> immediate response. What if you need to talk to a database? What if you need to take a request and just post it to the requests' strand or io_service? This is no problem if you use callbacks, but if you are in some kind of coroutine, you may be in for a rough ride. You can't just pretend that you are calling trivial functions - you have to resort to template meta-programming. And if you get anything wrong, you are likely to get an error message as long and comprehendable as a rally-rant from Donald Trump! This is the moment when it's a good idea to understand how these functions are implemented, and how to interface your own code with them.

For example, what if we want to implement a async_echo(), with the logic above, but something that can be used after co_await in another C++20 coroutine, or after yield in a asio stackless coroutine, or directly in a asio stackful coroutine?

Continuations

Over the years, the asio library has iterated over the idea of generic continuations, or composed operations, as it's normally referred to by Chris Kohlhoff, the author of the asio library. The current iteration, as introduced in boost asio 1.70, is quite simple to use. Most of the complexity is - or can be - hidden behind a single new template function.

A continuation in this context is a point in the code, where some async operation has completed, and the result (or just the fact that it has completed) must be returned to the caller. The caller may very well be running in another thread, quite busy doing something else, depending on what paradigm he used when initiating the now completed, async operation. If the caller supplied a callback function, we could call that function directly. If the caller preferred a future, we should set the value so it's value can be consumed, and any thread waiting for it becomes ready. If the caller called from a coroutine of some kind, the coroutine must be instructed to resume at the next expression.

If we refactor echo() above to async_echo(), what happens is:

The caller calls async_echo() and provides a continuation, either a wish for a future, a callback or a coroutine context that allows the coroutine to resume.
async_echo() returns to the caller. Normally it will start the asynchronous operation, but that is no requirement.
At some point in time, async_echo() will have completed (or failed) it's operation. It may have done that in a thread in a asio io_service, or it may have done that in another thread. That's an implementation detail. At this point, it set the value in the future, call the callback or resumes the coroutine (potentially in another thread).

We could of course implement a number of overloads for async_echo(), each suitable for one of the possible continuations. That's quite some work, and a burden on testing, as each would have to be tested separately. And the work would grow enormously if async_echo() was just one of many asynchronous methods we were implementing.

The simple(r) way is to use asio's built in support for this. The code below implements it, using boost asio 1.70's async_initiate template method.

template <typename CompletionToken, typename Executor>
auto async_echo(
        Executor&& executor,
        const std::string& message,
        const std::string& server,
        const std::string& port,
        CompletionToken&& token) {

    return boost::asio::async_compose<CompletionToken, void()>(
                [&message, &server, &port, &executor](auto& self) {

        co_spawn(executor, [message, server, port, &self]() mutable ->awaitable<void>  {
            // ...
            // Same code-block as in echo() above removed
            // ...

            // Do the continuation magic
            self.complete();

            // Exit the C++ 20 coroutine
            co_return;
        }, detached);
    }, token, executor);
}

I have removed most of the repeated code from echo() to make it simpler to read.

As you can see, there is very little additional code required to make your method just as flexible and simple to use as the built in async_* methods in the asio library itself.

You can call your async_echo() function just as you would call any asio async_* methods, using your favorite continuation.

    // Call async_echo() from a stackful coroutine
    spawn(ioCtx, [](boost::asio::yield_context yield) {
        async_echo("async message 1", "127.0.0.1", "55555", yield);
        async_echo("async message 2", "127.0.0.1", "55555", yield);
    });

    // Call async_echo() using a future
    auto f = async_echo("future message", "127.0.0.1", "55555", boost::asio::use_future);

    // Call async_echo() using a completion callback
    async_echo("callback message", "127.0.0.1", "55555", []{
       ; // We could have done something now...
    });

    // Call async_echo() using C++ 20 stackless coroutine
    co_spawn(ioCtx, []() ->awaitable<void> {
        co_await async_echo("C++20 coro message", "127.0.0.1", "55555", use_awaitable);
        co_return;
    }, detached);

So what just happened?

If you look closer at async_compose(), you may notice that it's template arguments are the CompletionToken and a function signature. The CompletionToken is whatever we use as the last argument to async_echo(). It's type depends on how we use the function - if we use a simple callback, or if we use some kind of coroutine. The function signature is the return value for our function. In our case, the async_echo() is not returning a value, not even an error status, - just the fact that it is done.

async_compose() takes three arguments. The first is a functor initiating the async operation (initiator). The second is the CompletionToken, and the third is an executor or an io-object having an executor. In our case, we provide just an executor that is passed to co_spawn(). What async_compose() does is to provide an appropriate completion method, self.complete() that we can call when we are done, and then it calls our initiation functor with self as an argument. We start the async operation and return immediately. async_compose() then either suspends our coroutine (if we call it from a coroutine) or return an appropriate value (for example a future, if we asked for that). That's basically all there is to it.

To give you another example - this time my favorite missing composed operation in the boost library - async_post().

template <typename Fn, typename CompletionToken, typename Executor>
auto async_post(Executor&& executor, Fn&& fn, CompletionToken&& token) {

    return boost::asio::async_compose<CompletionToken, void()>([&executor, fn=std::move(fn)](auto& self) mutable {
        boost::asio::post(executor, [fn=std::move(fn), &self]() mutable {
            fn();
            self.complete();
        });
        ;
    }, token, executor);
}

From asio-async-model-perftest

This is really useful in coroutines when you have to call post() to do something outside the coroutine, potentially using a different IO service or strand, and wait for it to complete.

The code for the other examples are available at github

The Last Viking's Nest

The great mystery on how to make durable, asynchronous functions for asio.

Continuations

Tags