Akka anti-patterns: using remoting

Senior developer using Akka remoting directly to build a special type of cluster

Whilst I have always successfully discouraged my clients from using Akka Remoting in their application, I often get questions regarding remoting while talking about anti-patterns at conferences and user groups.

Don’t get me wrong. I love Akka remoting. Especially the new Aeron-based Artery remoting version that is based on UDP rather than TCP. Whilst it maintains the same delivery guarantees, it is quite a bit faster than the TCP counterpart.

So why not use Akka remoting? Well:

Never, ever, use remoting directly in your application

— Konrad Malawski, Akka Core Team

Everything is in the wording: just as you don’t use the LTE network directly by receiving and emitting radio waves with your brain (unless you’re Lucy), don’t use Akka Remoting directly in your application. Use your cell phone. Or use Akka Cluster.

“But why?”, you ask, “why should I not be one with the network, connecting my actor systems on a peer-to-peer basis, in total control, with the raw power of the Actor Selection in my hand?”.

“Fool!” is my answer to this, and the stern look on your face scares you a little “what then do you do if the remote system is quarantined? Do you, mere mortal, really wish to reimplement all of Akka Cluster on your own?”

Because, well, yes, that is what it amounts to. Here is the association lifecycle of Akka Remoting for you:

See, when a remote system is quarantined, the only way to recover is to restart its actor system. Akka Cluster provides a membership service for keeping track of nodes that belong to a same application and takes care of many of the subtle and difficult aspects related to running distributed applications. It propagates membership via a gossip protocol and implements the The ? Accrual Failure Detector in order to quickly detect node failures.

So by all means, use Akka remoting — but please, only transitively.

Comments 7

  1. if it is not meant to be used from a user application shouldn’t it be private[akka] ?

    1. Post

      Good question. Franky I would ask this the Akka team. I suppose that the main reason it isn’t private is historical in nature and that remoting was there before cluster, and that therefore there are probably quite a few akka applications out there using it directly. It could also be argued that if you have really specific needs then you’d want to roll your one peer-to-peer based communication between actor systems. So in any case closing it off totally is maybe not an option. But then again the documentation warns you about using it directly, and (at least in the new docs) you need to dig quite a bit to find it.

  2. There are a lot of things in clustering I may NOT need. For example if I am building a distributed master worker pattern oriented application and my workers should know each other, I do not wanna cluster them and then get whispers when one worker goes down – it just makes the whole network so chatty and gossippy. It can affect performance. Clustering has benefits that akka remoting doesn’t – one of them happens to be that you do not need to know what nodes to talk to – you can use Distributed Pub Sub, so its location transparent, also you can have a scenario where you wanna build a distributed cache store where there’s no master …

    1. Post

      The trouble with remoting is that if one node fails, the remote system on that IP address gets permanently quarantined and if you want to restart a system on that node, you ought to also restart the other (partner) system. Taking the example of a master-working deployment, if one worker fails, you would need to restart the failed node but also the master and all other workers. That’s not really viable. If you’re concerned about the overhead of gossip and not so much concerned about getting fast failure detection, you could increase the gossip interval to have less traffic – that being said, the gossip packets sent around are small because unless there’s a change, only the gossip version (a vector clock) is contained in the packet, and not the full gossip with all member information. I’m not sure what you mean with wanting to build a masterless system – akka cluster is by design masterless, the leader is not elected but designed by convention.

  3. about > “what then do you do if the remote system is quarantined”

    I want to test this out myself out of curiosity. Can someone please help me understand how do I get an actor system to quarantine while using akka remote?

  4. Hello.

    Lets imagine that we have all our nodes in a cluster. Could we implement some custom logic on cluster singleton that will communicate with predefined list of other cluster members at point-to-point manner using Akka remoting(provider is cluster of course)? It seems this way we will have our custom functionality and all Akka cluster functionality, is not it?

    1. Post

      If you already use Cluster then you can use the existing remoting channels that cluster establishes – you only need to know the address of another member node (which you can get easily via the Cluster extension) and communicate over that existing channel. The point is that you’re letting Akka Cluster manage the channels for you instead of doing it on your own, including the handling of dropped / unavailable nodes.

Leave a Reply

Your email address will not be published.