History of Aeron and Hydra Platform

This article is first in a series of four blog posts on Aeron and Hydra Platform:

• Part I: History of Aeron and Hydra Platform.
• Part II: 5 trading solutions we delivered on top of Aeron and Hydra Platform. Read here >>
• Part III: Aeron and Hydra Platform in a trading platform. Read here >>
• Part IV: Operating Aeron and Hydra Platform based solutions. Read here >>

In this first blog post, we provide an overview of Aeron, an open-source, low-latency messaging system. We then describe the long term relationship we have with Aeron’s core maintainers. Finally, we discuss the latest addition to Aeron: Aeron cluster.

Messaging is the backbone of any trading system. Key properties such as response time, scalability and resilience are highly dependent on the messaging system we choose. At Adaptive, we decided years ago to adopt an open-source solution called Aeron. We also built a suite of components to accelerate the development of trading systems that we call Hydra Platform.

What is Aeron?

Aeron is an open-source, low-latency messaging system, initially developed for a large US exchange. It enables processes, running on different hosts or the same host, to communicate with minimal and consistent latency. It is particularly well suited for firm real-time systems which have high-throughput and/or require minimal response time. Aeron operates in a peer-to-peer model, it is broker-less, in contrast to other middleware such as RabbitMQ, Kafka, and TIBCO EMS to name a few.

Like most low-latency messaging systems in finance (29west UM, Tibco FTL, MoldUDP64, etc), Aeron is UDP based (unicast or multicast) and provides reliability guarantees on top of UDP.

Efficient persistence of messages sent over Aeron is handled by the Aeron Archive. Aeron Archive can be used to record a message stream to disk and can then be used to replay messages to subscribers. A typical use case is recording market data and allowing downstream systems to replay the market data stream at a later stage. It can essentially be used as a building block for event-sourced architectures.

Aeron Cluster is built on top of Aeron and Aeron Archive. Aeron Cluster is used to build fault-tolerant services such as matching engines, pre-trade risk or credit check systems, real-time workflows (think RFQ, auctions, IOI, etc), smart order routers, etc. I’m mentioning finance-related use cases here but there are very good candidates outside of finance. For example, Aeron Cluster could be used to implement highly-contended retail eCommerce systems, ticketing/reservation systems, electronic advertisement venues/auction systems, to name a few.

In essence, Aeron Cluster’s job is to sequence messages sent by clients to the cluster and replicate that sequence of messages within the cluster. Since each node is guaranteed to receive the same sequence of messages, it is possible to deterministically rebuild the same state across each node of the cluster, to achieve fault tolerance. Aeron Cluster is based on a formally proven consensus algorithm called RAFT. Clustered databases use similar approaches, the difference here is that you can build your own business logic (in Java, C++ or .NET) for stateful, consistent, and highly available services.

In terms of platform support, Aeron runs on Windows, Mac, and Linux, plus you can use Aeron from C, C++, Java (or any JVM language) and .NET.

Aeron was initially designed and developed by two individuals: Martin Thompson and Todd Montgomery.

Martin is well known for his work in the low-latency space: he was the CTO of LMAX, a London based Exchange. He has talked about their architecture and open sourced the Disruptor which you may have heard of. He is frequently speaking at conferences and these days he spends his time hacking on Aeron and consulting with clients.

Todd is a messaging expert and was the CTO of 29West developing Ultra Messaging, a well known low-latency messaging system widely used in finance. He has been researching, designing, and developing network protocols for more than 20 years. These days he also spends most of his time hacking on Aeron and consulting with clients.

A long term relationship

LMAX Architecture and the Disruptor

In 2010, Martin and Mike Barker talked about the architecture of the exchange they were building. I remember watching this talk many times and getting really interested by the architecture, which was later documented by Martin Fowler. A few months later they open-sourced the Disruptor: a novel concurrent data structure which was sitting at the heart of their core architecture. I was eager to understand this stuff inside out and undertook to port the Disruptor to .NET as a pet project. While this exercise was fun and I learned a lot about lock-free concurrency and memory models (my head still hurts), it triggered my first interactions with Martin as I got in touch to ask for help on some tricky areas of the port and suggested some potential improvements. This experience significantly influenced my approach to software architecture and the design of systems I have been working on since then.

Simple Binary Encoding

A few years later, Martin joined forces with Todd and they worked on their first open-source project together: SBE (Simple Binary Encoding). SBE is a standard specified by the FIX committee High Performance Working Group. A large US Exchange asked Martin and Todd to develop an open-source reference implementation. They implemented the Java and C++ version and I ported it to .NET a few months later (disclaimer: I’m no longer maintaining it, all the credit goes to Bill Segal for maintaining the .NET and Go ports since the early versions and keeping them current). The large US exchange liked the work Martin and Todd did for SBE as they hired them again to prototype a new messaging system: Aeron was about to be born.

Aeron

The development of Aeron started in early 2014, it is now a mature and very active project with over 70 contributors, 5,000 followers, and 570 forks.

In 2015 Martin asked Adaptive to develop the .NET client of Aeron. We have been maintaining it since then.

The start of Aeron Cluster

In 2016-2017 Adaptive developed several systems inspired by this style of architecture, leveraging clustered services based on state machine replication. We were able to develop a variety of trading systems ranging from RFQ workflows, matching engines, pre-trade risk/credit check system.; essentially all using the same architecture. We saw some very significant advantages with this style of architecture:

• We were able to establish a very clean separation of concerns between the technical infrastructure stack, dealing with messaging, fault tolerance and other “non-functional” concerns and the actual business logic of the system.

• This separation of concerns offered a simple programming model for developers, allowing them to focus on their business logic – resulting in higher productivity.

While these systems enjoyed the benefits outlined above, they each required significant investment in technical infrastructure, i.e. in the layer below the business logic. Eventually, we became frustrated that we had to build this layer from scratch for each new project. So we started to explore how we might deliver business value to our clients sooner.

In the last quarter of 2017, we were pitching for an internalisation platform (i.e. an internal matching engine), and we thought it was time to solve this problem once and for all. We needed a reusable cluster that we could leverage later for other projects. It was obvious to us that Martin and Todd were best suited for that. Aeron and Aeron Archive were the perfect building blocks for the cluster, and based on their experience, I knew they would do a fantastic job. We partnered with them and delivered the project for our client. Martin and Todd developed the cluster infrastructure (known as Aeron Cluster), and we built the matching engine on top and the rest of the trading platform. The project was a great success. We worked very closely with Martin and Todd during the development of the cluster – iterating together on APIs, then integrating progressively as the cluster was being built. Building the cluster was a significant effort, but it was well worth the pain. The open-source community now has access to a state-of-the-art cluster which drastically reduces the effort required to deliver projects on such an architecture.

During this project, which started in early 2018, Adaptive also decided to invest internally and develop a set of components complementing Aeron and Aeron cluster to further accelerate the development of trading systems for our clients and to simplify the operation of these systems. We call this suite of components Hydra Platform. We will provide an overview of Hydra later in this series.

In part two of this series, we will take a look at several projects we have delivered on top of Aeron and Hydra Platform during the past 3 years: a power and gas marketplace, a collateral trading exchange, a bond trading platform and a spot and futures cryptocurrency exchange.

The series is also available for download as a white paper. In this paper, we provide an overview of Aeron & Aeron Cluster, and look at some of the projects we have developed using Aeron during the past 3 years. We then provide an overview of Hydra Platform which contains some of the building blocks we produced to complement Aeron and further accelerate the development of systems we build. While we do not go into deep technical details, this document is particularly suited for developers, architects and CTOs.

Olivier Deheurles

Co-founder and Chief Technology Officer,
Adaptive Financial Consulting

Let’s talk

Follow @WeAreAdaptive