Investment Strategy Alerts with ScaleOut ComputeServer®

Background

A common challenge faced by hedge funds is the real-time management of their investment strategies during the trading day. For example, a fund that employs pairs trading strategies may have thousands of long and short positions in equities that are judged to be under and over-priced. During the trading day, each equity’s price changes rapidly, resulting in a profit and loss profile that reflects gains and losses every fraction of a second. The challenge in this environment is how to isolate the critical changes from the massive fluctuations occurring every fraction of a second.

One such fund in New York employs hundreds of different equity investment strategies that are constructed around a form of pairs trading. They make use of their domain knowledge in dozens of different industries to create clusters of stocks that they view as a strategy. These strategies typically consist of a set of target equity investments, made up of both long and short positions in groups of stocks, together with cash positions. Each strategy group has a set of business rules that dictate such factors as total amount invested, target allocation among investments, and a set of parameters that define the acceptable value range for the strategy. This particular fund has traders that manage the strategies, with each trader monitoring and managing many strategies each trading day.

Currently this fund uses a relational database to hold their equity positions and they use SQL to analyze the strategies during the trading day. The goal of this analysis is to determine whether a given strategy is "in range" or not. Timeliness of such an analysis is of prime importance. While the traders do not employ high frequency trading or algorithmic methods, they still have a clear need for fast information. If a strategy goes out of range a fast response can have a huge impact on profits. Using SQL, an analysis can be made in about thirty minutes, which is far too slow to enable meaningful responses.

The enclosed sample program uses ScaleOut ComputeServer® to update and analyze a large number of investment strategies and their associated equity positions within a few seconds instead of several minutes. This sample shows how in-memory data storage and computing techniques can automatically distribute these strategies across a cluster of servers and then update and analyze them in parallel. Because ScaleOut ComputeServer handles all of the details of distributing the workload, the application developer only needs to focus on the actual algorithm for updating and analyzing each hedging strategy.

Overview of the Sample

This simulation of a distributed investment strategy alerting system uses ScaleOut ComputeServer as a distributed, in-memory data grid (IMDG) for reliably storing and updating large numbers of objects across a scalable farm of servers. The use of ScaleOut’s Parallel Method Invocation (PMI) feature is key to achieving scalability. PMI provides an easy to use, highly-optimized computational model for this application by automatically pushing computational logic to every node in the grid for parallel execution, thus eliminating the need to explicitly code parallel computations and move data between servers at runtime. As the workload increases, the application’s throughput can be increased dynamically just by adding servers to the IMDG.

This sample uses ScaleOut’s PMI to periodically update a large number of investment strategy objects across a farm in parallel with the latest market prices and to evaluate them for alert conditions. (In this sample application, PMIs are performed at a maximum rate of once every second.) The IMDG holds a large pool of strategy objects, each of which contains the details of the strategy’s core exposure and its long and short positions. When a PMI is started, it distributes a snapshot of the latest market prices to all strategy objects, updates each strategy’s prices, and evaluates each strategy for an alert condition. The set of strategies currently alerted is collected and returned by the PMI for reporting.

The application includes a simple UI to allow the user to examine the strategies and to identify which strategies are currently alerted. The UI also reports the total number of strategies and the overall throughput of the application, as measured by the total number of positions evaluated per second across all strategies.

The components and data flow in the application are illustrated in the following diagram:

Hedging Example

Application Overview

This application consists of three packages in strategyalerts.jar:

marketsimulator: A simplified market simulation application, responsible for generating and evaluating a set of investment strategies and a market feed that simulates incoming security price changes. A population of 3000 securities is used in this sample, and these securities are randomly distributed among the long and short positions of the strategies. This application performs periodic PMIs on the strategies at a maximum rate of once every second.
```
Usage:
com.scaleoutsoftware.demos.marketsimulator.Main strategyCount priceUpdatesPerSec positionsPerStrategy

eg:
java -classpath "C:\Program Files\ScaleOut_Software\StateServer\JavaAPI\lib\*;
 C:\Program Files\ScaleOut_Software\StateServer\JavaAPI\*;.\strategyalerts.jar"
 com.scaleoutsoftware.demos.marketsimulator.Main 2000 1000 10
```
The following ordered command-line arguments are supported:
1. strategyCount: An integer indicating the number of investment strategies to be created at startup. Strategies contain of a number of core and hedge positions that consist of a security and a target allocation for that security within the strategy.
2. priceUpdatesPerSec: An integer indicating the rate at which the simulated exchange should produce price updates for securities used in the simulation.
3. positionsPerStrategy: An integer indicating the number of positions that should be in each strategy. A strategy consists of both core and hedge positions, and this value specifies the number of each type that is held in the strategy. (The total number of positions in each strategy is two times this value.)
alertsdesk: The alertsdesk GUI is a visualization tool based on Swing that displays strategy objects in the IMDG and highlights any alerts that may have been triggered as position prices are updated and evaluated by the MarketSimulator application. This application polls the IMDG for the current set of alerted strategies and for the details of the currently selected strategy being displayed. It also supplies an alerting threshold percentage to the market simulator for use in the PMI operations.
```
Usage:
com.scaleoutsoftware.demos.alertsdesk.AlertForm

eg:
java -classpath "C:\Program Files\ScaleOut_Software\StateServer\JavaAPI\lib\*;
 C:\Program Files\ScaleOut_Software\StateServer\JavaAPI\*;.\strategyalerts.jar"
 com.scaleoutsoftware.demos.alertsdesk.AlertForm
```
common: A package containing common types (Strategy, Position, etc…) that are used by both the marketsimulator and alertsdesk packages.

Design

Classes and Data Storage

The market simulator application generates instances of the following classes and stores these instances directly in the ScaleOut IMDG:

Strategy: A class that consists of a collection of Position objects and an potential "alerted" state if position allocations diverge too far from targeted percentages. An instance of the Strategy class is stored in the IMDG for every strategy that is modeled by the MarketSimulator.
AlertSummary: A result class containing a collection that summarizes the alert status of all the strategies in the IMDG. This summary is the return value of the NamedCache.invoke PMI operation, and it is stored in the grid so that it can be accessed for display by the alertsdesk GUI application. A single instance of this object is stored and periodically updated in the IMDG after each PMI operation completes.
Settings: A class that contains settings that are shared in real time between both the market simulator and the desktop GUI app. Currently, its only setting is the maximum percentage that a strategy’s position is allowed to deviate from the target allocation before an alert is tripped. A single instance of this object is stored in the IMDG.

ScaleOut ComputeServer lets an application organize objects of different types into distinct groups, called named caches. Instances of the above classes are stored in the IMDG in the following named caches, respectively:

"Strategies Cache"
"Strategy Summary Cache"
"Settings Cache"

In addition to the three classes above that are stored directly in the IMDG, the distributed PMI operation used for price updates also uses the following class when NamedCache.invoke() is called:

StrategyUpdateArgs: A PMI parameter object that contains a dictionary of security ticker symbols with their latest respective market prices. An instance of this class is passed as an argument to the NamedCache.Invoke() method, which implicitly uses the IMDG to deploy this market snapshot to all grid hosts performing strategy updates.

Behavior

Upon startup, the market simulator application creates initial strategy objects in the IMDG and spins up a worker thread to simulate an incoming price feed. Note that creating the initial strategies can take several seconds. Once the strategies are created and the price feed has started, the application enters a continuous loop of PMI operations that periodically update position prices within all strategy objects and trigger alerts when significant price changes cause position allocations fall too far outside of targeted strategy allocations. Input from the price feed is processed as follows.

Prices

Prices entering the system via the exchange feed (as implemented by the application’s ExchangeSimulator class) are tracked in an internal dictionary of current security prices. At the start of each PMI operation, the MarketSimulator’s Main method acquires a copy of the latest prices from the exchange and use ScaleOut’s PMI engine to pass these prices to every single Strategy object in the IMDG.

When the PMI operation is initiated through a NamedCache.invoke call, the ScaleOut service deploys the logic in StrategyUpdater.eval() (i.e., the Java jar file containing the required method) to all servers in the farm to so that it can be run in parallel within the IMDG. The snapshot of market prices is passed to the NamedCache.eval() call as an argument, which causes it to be read by all IMDG servers within the PMI operation. With the logic and market prices automatically deployed to every host in the grid by the PMI operation, all servers in the IMDG (using multiple threads on each server) are able to update and evaluate every position in every strategy in parallel.

Other noteworthy aspects of this PMI approach include the following:

If strategies need to be updated more than once per second, then the sleep time between PMI calls can be reduced or eliminated. PMI operations can be performed multiple times per second, if necessary, at the cost of increased CPU usage across the farm.
The ExchangeSimulator.getPriceSnapshot() method always returns a deep copy of current security prices. This prevents concurrency problems with the price collection, since multiple threads in application may be reading and modifying this price data.

Alerts

In addition to updating position values, the StrategyUpdater.eval() method also checks the positions to see if their actual allocations diverge too far from target percentages.

The return value of StrategyUpdater.eval() is an AlertSummary object that indicates which evaluated strategies are in an alerted state. The PMI operation merges all returned summaries from all StrategyUpdater.eval() calls and combines them into a single value that is returned from the originating NamedCache.invoke() caller. This merging operation is effectively the "reduce" step of the PMI feature’s map/reduce style behavior.

Once an AlertSummary is returned from the NamedCache.Invoke call, the market simulator stores it in the IMDG so that the alertsdesk GUI application can display the overall alert status to an end-user.

Reporting

The alertsdesk Swing application is a desktop application that displays overall alert status for all strategies in the IMDG, and it allows users to drill down into individual strategies and view real-time allocations for each position in a strategy.

The alertsdesk application provides the status overview by polling the AlertSummary object in the IMDG that is constantly being updated by PMI operations in the market simulator application. The currently alerted strategies are highlighted in a list on the left-hand side of the display. The end-user can then examine the alerted strategies to make trading decisions on the positions. If a user has selected an individual strategy, the GUI polls that strategy object in the IMDG every second (by default) and displays its positions and their target/actual allocations.

The alertsdesk application lets the user dynamically select the alert threshold. Changing the threshold to a higher percentage deviation from the target holdings generally causes fewer strategies to be highlighted as alerted in the display, and vice versa. The UI shows the user how many strategies are resident in the IMDG, and it reports the throughput of the PMI operations as the total number of positions evaluated per second across all strategies.

Performance Scaling

All performance scaling tests should be evaluated using an IMDG with at least two servers as a baseline. When two or more servers are deployed, the IMDG automatically creates object replicas for high availability, and this introduces overheads not found on a single-server IMDG.

The workload of this application can be increased by creating more strategy objects. This will cause the PMI operations on each server to take longer to complete, and this increases the CPU utilization on these servers. Just adding a server to the IMDG will automatically and dynamically redistribute the workload across more servers without any code changes. Two scaling strategies are available:

For a given workload, adding servers will lower the CPU utilization on each server. This helps maintain sufficient extra CPU capacity in case a server failure occurs.
For a growing workload (i.e., more strategies), adding servers will enable the IMDG to handle a proportionally larger workload for a given CPU utilization. This enables the user to grow the workload on demand and maintain the same, fast response time for any required workload.