Posted by Brian McCallion ● May 10, 2018 11:07:25 AM

On AWS GDAX and Gemini foretell the future of financial exchanges


While nobody in finance is 100% certain about crypto currency, we can predict the future of capital markets based on some early characteristics. In traditional trading we find firms often co-locate as close to the exchanges as possible, so as to reduce the latency when placing orders, and thereby the opportunity for their order to be anticipated.

Latency is not your friend when it comes to trades. In practice, network latency is just one part of what constitutes latency. Another source of latency is "tight coupling" between systems. It's counter-intuitive, but asynchronous designs most often reduce latency. Synchronous, tightly coupled operations, or operations that must take place in serial, tend to propagate latency throughout a system. Asynchronous designs enable systems to compartmentalize and restrict latency to certain systems.

When I worked at a brokerage processing all the broker dealers submitted JMS messages with orders. The brokerage processing firm ran a message queue service on an IBM z Series mainframe. Now the mainframe had a lot of CPU and could handle a tremendous amount of trades. IBM even offered an option to "unlock" CPU capacity already provisioned, but not licensed by the customer. Yet another option was that IBM would fly a helicopter to your site and install additional CPU.

And so Cloud architecture benefits because to get additional capacity you don't need to have a helipad on the roof of your building. And one of the benefits of asynchronous processing is that by writing messages as the way to communicate between systems we can scale the capacity of each service. Even if a service is offline, as long as the message queues are available to receive writes, other services are not impacted. We want to scale to keep the queues full, and to have enough messages to keep our compute nodes busy. In the cloud we take this a step further, and we can scale down to zero compute nodes when there is no work, no messages in the queue.

Helicopters landing on the roof are a major distraction. And so once of the qualities we strive for when we design systems for the cloud is for these systems to adapt. And so in the cloud, we find that best practices in the data center often work even better. Asynchronous processing works even better when we can dynamically scale the capacity to process messages. In the data center, you need to provision for peak, or you need that helipad, either literally, or figuratively.

Both GDAX and Gemini, the two largest crypto exchanges both run on AWS us-east-1. At AWS reinvent 2017, AWS introduced a service called Private Link. Private Link enables one customer to offer a white listed low latency endpoint that is only accessible by those white listed customers. And the traffic routes through the AWS region, it does not route through the internet. As a result, latency is very low, and very consistent. Further, bandwidth is no longer a concern, as the AWS internal network will not be the bottleneck. When I think of the future of financial exchanges I think of GDAX and Gemini and how whether you love it or hate it, disruptive technologies often provide a glimpse of what the future holds.

We know about data gravity, and how compute is "attracted" to data. What are the further implications of exchanges moving to the cloud? Let me know your thoughts.



Executive Viewpoint: For Financial Firms the Future is in the Cloud

White paper: Powerful New Technologies

Case Study

Topics: Capital Markets, financial exchanges, AWS Private Link