Overview of the project #

Context #

In the realm of asynchronous programming, the Scala ecosystem offers a set of solid and widely adopted monadic constructs and libraries to tackle complex tasks functionally with elegance and efficiency, like Monix Tasks and Cats Effecs, enabling a wide range of interesting and useful features, like composable error handling, cancellation mechanisms and structured concurrency that the standard library lacks. However, they also come with a cost: the pervasiveness of the flatMap operator to compose values makes the code harder to reason about and difficult and awkward to integrate with regular control structures.

In the last years, we have been assisting the increase in adoption of continuations and coroutines in modern runtimes, either exploiting some kind of fiber support, like the project Loom with Virtual Threads, or via code generation, like Kotlin Coroutines. Even Scala is not immune to this trend and a new strawman library, Gears, is currently being developed, aiming to bring direct style support for asynchronous programming. Despite the interest and the potential this new library could bring, it is just a speck that fits into a bigger picture, which is the management of effects: the ongoing research activity led by M. Odersky has indeed the goal to, instead of pushing effect management into external libraries, upgrade the type system to handle effects natively using capabilities, as research-oriented programming languages do with Algebraic Effects (like Koka).

Goals #

The goal of this project is to explore, mainly focusing on Scala, the direct style, developing a few examples (not too complex) leveraging the new strawman library Gears for asynchronous programming, comparing it with Kotlin Coroutines and the current implementation of monadic Futures, seeking to analyze aspects such as:

ergonomics of the two styles;
which of the two approaches has a real advantage in adoption;
pros and cons of the two styles;
any limitations and difficulties encountered in using them.

The contribution #

The project is built around three small examples.

The first aims, through the implementation of the core of a small Web service, to introduce the basics of asynchronous programming in direct style, focusing on structured concurrency and cancellation.

The second introduces the main communication and synchronization primitive of direct style, channels, and how they can be used to exchange data between concurrent tasks (either Futures or coroutines).

In this context, a contribution has been made by proposing the extension of the Scala Gears library with the following two abstractions, currently missing, inspired by those of the Kotlin Coroutines:

terminable channels, i.e. a channel that can be terminated, but whose values can still be read by consumers after its termination until all values are consumed;
Flows, modeling a cold stream of asynchronously computed values which are particularly useful for implementing asynchronous functions that produce, not just a single, but a sequence of values.

The last example investigates how to implement a reactive-like event-based system using direct style, taking as a use case a small sensor control system in an IoT context that needs to react to events coming from different sensors.

To accomplish this, since Scala Gears lacks any kind of transforming operator, some have been introduced on top of channels (taking cues from Reactive frameworks) allowing them to be manipulated functionally.

Conclusions #

The current implementation of monadic Futures present in the standard library is insufficient to handle modern asynchronous programming in a structured and safe way: it lacks structured concurrency and cancellation mechanism, is not referential transparent and requires to be ugly mixed with direct style constructs to be used appropriately.

This leads to the adoption of effectful libraries that offer these and many other powerful abstractions, beautifully wrapped in monadic constructs. This is the main pro and con of the monadic approach: what makes monads remarkable is their capability to turn statements into programmable values and introduce constructs to transform and compose them functionally in a very elegant (and somehow “magical” for ones who are not familiar with them) way. Despite this idyllic beauty, monads and effectful libraries require relevant expertise in functional programming to be fully grasped and effectively composed.

Direct style frameworks are indeed arising as a more natural and intuitive way to handle concurrency, leveraging an imperative-like programming style that is familiar to all developers.

In the JVM ecosystem, the most adopted and known direct-style library is the Kotlin Coroutines, which were introduced in 2018 by modifying the Kotlin language (rather than its runtime, like the project Loom has recently done with Virtual Thread) to support suspending functions. The main advantages of Kotlin Coroutines are that they provide suspension and cancellation mechanisms that are simple to understand and use, as well as a good ecosystem for channels and Flows. Despite this, Coroutines are not still perfect: due to their design, they partially suffer from the colored functions problem and we need to be aware we can not use the same synchronization concurrency abstractions that we would use in Java with threads (like locks, synchronized, …) cause they are not designed to be used in the coroutines context.

Scala Gears is an attempt to bring direct style into the Scala ecosystem. Its API design is inspired by Kotlin Coroutines and, despite the fact it achieves suspension, unlike Kotlin, leveraging Virtual Threads for JVM and delimited continuation for Scala Native, the majority of constructs can be mapped to the Kotlin Coroutines ones. Despite being a very young project, it already offers a good set of abstractions for asynchronous programming, although it cannot yet be considered a mature library ready to be used in a production environment:

some design choices should be addressed:
- closing a channel prevents any further reading, precluding the possibility of processing the remaining values (see second example);
- Task scheduling behaves differently with higher-order functions depending on its signature and wither or not the function passed is suspendable (see third example);
the library is still missing some important abstractions, like the proposed Flow for handling a cold stream of asynchronously computed values, and operators for functionally transforming channels (and in a next future, hopefully, Flows or equivalent abstraction);
performances: the project has been created for experimenting, thus performances have not been considered a priority so far, even though a comparison in overheads of the core primitives has been published.

In conclusion, Scala Gears is a promising project that could bring direct-style async programming into the Scala ecosystem, giving, together with boundary and break Scala 3 support, a nice alternative to the current monadic approach, simplifying the way we handle concurrency, making it more natural and intuitive.

References #

Home Next: boundary & break