Cluster on Manuel Bernhardt

10000 nodes and beyond with Akka Cluster and Rapid

Thu, 30 Apr 2020 09:39:47 +0200

Update: there’s a podcast episode about this

The year is 2021. As the world is still deeply affected by the COVID-19 pandemic, the United Nations have decided that in order to prevent the same scenario of ever happening again, there should be a worldwide, real-time contact-tracing system in place capable of tracking every human being on the planet. Everyone is now equipped with a small, pebble-like device. If a person you have been in contact with is confirmed to carry a new kind of virus for which there is no remedy, the device starts glowing in red. Each device reports the anonymized contact information to one global backend system capable of handling this type of massive scale.

Tour of Akka Typed: Cluster Singleton and Routers

Tue, 03 Dec 2019 14:59:48 +0100

In part 5 of this series, we started to scale the application from a local one to a clustered one by introducing Cluster Sharding. In this article, we will continue our effort to scale the payment processor out and make it resilient through the use of Cluster Singletons and Cluster-Aware Routers.

Before we get started, here’s a quick reminder of what we’ve seen so far: in the first part we had a look at the the raison d’être of Akka Typed and what advantages it has over the classic, untyped API. In the second part we introduced the concepts of message adapters, the ask pattern and actor discovery. In the third part, we covered one of the core concepts of Actor systems: supervision and failure recovery. In the fourth part, we covered one of the most popular use-cases for Akka: event sourcing. Finally, as mentioned before, in the fifth part we started scaling out by making use of cluster sharding.

One step closer: exploiting locality in Akka Cluster based systems

Mon, 28 Oct 2019 12:36:24 +0100

When it comes to the latency of processing a request in a distributed system, everyone knows (or should know) that doing things over the network is the most expensive thing you can do (alongside with disk reads):

The round trip time in a datacenter is about 5000 times longer than a main memory reference

Modern architectures have recognized this and will favor keeping data in memory, avoiding the need to go to disk or to the database, when consistency requirements allow for it. With Akka Cluster Sharding combined with Akka Persistence, it is possible to keep millions of durable entities in memory, reducing latency to a large degree.

Raspberry PI Akka Cluster LED strips

Thu, 02 May 2019 12:00:59 +0200

Ever since Eric Loots from Lightbend first showed me the Raspberry-PI based Akka Cluster with special LED stripes, I’ve wanted (at least) one of them.

The Akka PI Cluster at Scala Days 2018

This is a 5 node cluster with special LED stripes setup in such a way that the LED colour and status (on, off, blinking, etc.) show in which state each node is. It is an excellent way to teach Akka Cluster, and since I do this regularly I was very interested from the beginning to get one of these myself.

Tour of Akka Cluster

Fri, 03 Aug 2018 09:25:34 +0200

Here’s an overview of the Tour of Akka Cluster article series:

Tour of Akka Cluster – Testing with the multi-node-testkit and a handful Raspberry PIs

Fri, 23 Mar 2018 08:19:47 +0100

In the previous parts of this article series, we have looked at various functionality of Akka Cluster: distributed data, persistence, message delivery semantics and cluster sharding. We are far from done with exploring Akka Cluster features, but today it is time to look at one feature that is extremely useful when it comes to building production-quality systems based on Akka Cluster: the test support. As testing itself isn’t the most exciting activity there is, let’s space this article a bit up by throwing a few Raspberry PIs in the mix:

Tour of Akka Cluster – Cluster Sharding

Mon, 26 Feb 2018 16:33:04 +0100

In Tour of Akka Cluster - Eventual consistency, persistent actors and message delivery semantics we started to look into making the reactive payment processor resilient to node crashes by exploring Akka Persistence, a mechansim that provides durability guarantees for actor state. We also briefly mentioned Akka Cluster Sharding which offers a way to automatically distribute actors of the same type over several nodes. In this article we are going to take a closer look at cluster sharding and then make use of it for our reactive payment processor.

Tour of Akka Cluster – Eventual consistency, persistent actors, message delivery semantics

Wed, 31 Jan 2018 14:19:43 +0100

In Tour of Akka Cluster - Akka Distributed Data we looked into building a reactive payment processing system using Akka Distributed Data as a means to provide master-master replication to the order storage component. We did not, however, ensure in any way of form that the system as a whole could withstand individual node crashes. In this article, we will explore options for making the Reactive Payment Processor fault-tolerant.

Evaluating the existing design in the light of redundancy

Let’s make a thought experiment and see if our current design is fit to be “upgraded” in the light of redundancy. Let’s suppose we have made the validation process redundant, so that it would take place on any node (not just the node on which the order was first registered). This way, should the order receiving the node fail, we would still be able to continue validation and processing (so long as it wasn’t the node also handling the particular acquring bank).

Tour of Akka Cluster – Akka Distributed Data

Wed, 03 Jan 2018 15:23:04 +0100

Building distributed systems is hard. Pesky things like the laws of physics get in the way of maintaining state accross geographically (and chronologically) disparate systems, and, if that weren’t already outrageous enough in itself, those systems may be subject to network failures, forcing us to think about annoying tradeoffs regarding consistency, availability and the meaning of life.

One of the more interesting (well, at least from my perspective) tools in the Akka toolbox is Akka Cluster and the modules built on top of it. Through its design it acknowledges the hard reality of life, promoting a “no magic here” approach to building distributed applications and yet at the same time, takes care about many of the harder aspects of doing so.