Having already talked about embracing asynchrony, it seems worthwhile to talk about how all of our asynchronous operations don’t just return a single result. It is actually more usual for them to return a number of values – sometimes an unbounded number. This means we need to be able to reason about, and work with, streams of events, and in particular their behaviour in the frequency domain. For example, we may want to take specific action in response to particularly high or low frequencies of events.
Static Data Isn’t
Large amounts of data in applications like Reactive Trader would usually be loaded during application initialisation. A common pattern for financial applications like ours is to be launched on Monday morning when the trader arrives at his desk, be configured as required for the week ahead, and then left running until the end of the week when the application automatically exits after markets have closed.
This means that we do not have a daily restart to pull down the most recent set of clients, entitlements, or defaults and selections for various reference data. This data – even though it changes infrequently (daily, weekly or monthly) – does still change. That means we need to treat even configuration data as a stream.
In Reactive Trader, the server can update the set of available currencies pairs while the application is loaded. On start up the app retrieves the current set of all available currency pairs, and immediately subscribes to prices for each of them. That asynchronous call, however, doesn’t end after just one result. It remains active, and can yield further results. The type of the stream allows us to represent both the addition of new currencies pairs and the removal of currently active ones. The user interface is updated immediately – tiles are added and removed. A more advanced implementation would check the current state of the currency tile, and only remove it once any outstanding asynchronous operations (such as an executing trade) had completed and been acknowledged by the user.
Streams Big and Small
The high frequency streams in Reactive Trader are quite clearly the price streams. These are front and centre when you load the application, after all. The functionality to enable streaming prices in Reactive Trader is obviously important, but interestingly the concepts are just as important for less obvious functionality, too. One particularly interesting example of this is the resolution of a tenor – that is the date on which a particular trade between two parties must be settled, or the payment made. Most foreign exchange settlements are made two days after the current date (what is referred to as T+2). The date rolls, or changes, for most currencies at 5PM NYT. This is the end of the market day in New York. The NZ dollar rolls at 2pm NYT. This means that an application like ours, which displays the date of when a particular trade needs to settle, needs to be capable of having that date updated in real time. This isn’t particularly complex in the case of an FX SPOT price. We just associated the settlement date with the price itself and include it with every price tick.
But to give you some idea of the subtlety of the issue, when capturing a large, complex trade – the entry of which can stretch over the better part of a day, and can include many different sub-trades that need to settle on different dates – your data entry form is itself required to update the dates for which various tenors have been entered. This requirement turns a simple lookup – for a tenor like 6M, six months – with a single return value, into a subscription to a stream of possible future date resolutions. Usually the first of which will be the only one returned, but the system needs to be prepared to handle further updates to the date. This is a perfect place for Rx to enable us to solve the business problem correctly.
Failure in the Frequency Domain
When working with unbounded streams of data, the frequency of their arrival becomes very important. Too frequent, and the client may not be able to handle events quickly enough and a queue may form. If special care is not taken, this can result in out of date or stale prices being shown to the user, which may ultimately result in failed trade requests. Conversely, streams that go quiet can also be problematics. A price displayed on screen is expected to be fresh and valid. If it is not, we need to tell the user that something has gone wrong and that they are currently unable to trade.
Our Reactive Trader demo application has the ability to control how many messages a second we send to the client. We can go from no updates all the way to more than four thousand a second. This lets us play around with a number of strategies for dealing with bursts of messages on the client. These techniques are generally known as conflation strategies, and they allow us to reduce in frequency the number of updates we process while still accurately representing the current state of the world being modelled.
The standard approach is to simply render every price update to the screen – despite the fact that our screens tend to update only 60 times a second. It also means that we can introduce queuing, most obviously on the WPF dispatcher thread. There are other places that prices can queue, but we will limit our discussion to the queuing problem we have attempted to solve in Reactive Trader. There are multiple implementations of conflation in Reactive Trader, and you can switch between them while the app is running.
Standard simply renders every price update, as already discussed. Drop frame renders only the most recently arrived price update for each currency pair stream. If a price stream is being updated 120 times a second, but the dispatch thread can only render a new price every 1/30 of a second, this implementation will render every 4th price update. The throughput is limited by the performance of the WPF dispatcher, and if subscribed to many price streams, the performance of Reactive Trader can still be unacceptable, as lower priority dispatcher frames (animation, for example) are never rendered.
Conflate is more complex, but allows you to specify the frequency of updates explicitly, rather than it being driven by the throughput of the dispatcher itself. The implementation in Reactive Trader yields the most recent price update every 100 milliseconds. If a price stream is ticking more than once every 100 milliseconds, some updates will be dropped. If a price stream ticks at T=0ms, then again at T=10ms, the second price tick will not be yielded until T=100ms. This introduces 90ms of latency, but reduces load on the UI. Each price stream is handled individually, so subscribing to many price streams can still overload the UI, and mean each individual stream may see quite high latencies for price updates.
Constant rate takes a different approach, and works across all price streams, preventing the UI becoming unresponsive due to a large number of separate subscriptions. Our implementation renders the most recent update for each price stream every 125 milliseconds. This is done no matter how may different price streams are subscribed to. The implementation could be extended to render all pending price updates less frequently depending on load on the system, or some other metric. This allows us to reduce load on the app to maintain responsiveness if required – we can also tell the user we are updating prices less frequently. This approach is very similar to what a computer game does. A physics engine used in a first-person shooter has considerable granularity, but the world is typically only rendered 60 times second. Our implementation of a constant rate scheduler can be found in
PeriodicBatchScheduler, but the concept is more important than the implementation.
At the other end of the scale, we have a requirement to alert the user to a stream not yielding a new update over some time period. Each price stream in Reactive Trader as a service level agreement (SLA) between client and server to yield a new price update every 4 seconds. If the underlying price of the currency pair hasn’t changed, then the server should just send a refresh with the previous price. We have a simple operator in Reactive Trader that transforms a stream of type T to IStale<T>. This IStale<T> is similar to the Option type. Our UI responds to the injected stale update by telling the user the stream has gone stale. As soon as a new update is seen, it is yielded and the UI renders the price as usual.
We also use this stale stream detection when making a trade request. Trade execution has an SLA of responding within two seconds, so if we do not see a trade execution response within this time we update the UI to tell the user something has gone wrong. If we do see a trade execution response outside of the SLA, we update the UI as per usual so the user can see the result of their trade execution.