The core of our architecture rests on a universal trading system framework. This framework abstracts all of the basic market interfaces, allowing us to write generic strategies that run on any market, including simulation.
As you can see in the above central box, our trading system abstracts several core functionalities.
- Settings Management – the entire trading system is configured via a straightforward xml configuration file. The actual storage and management of this is abstracted by the particular profile. For live running, these settings are version controlled and managed in a central replicated sql database. For simulation, these are stored as a simple file provided to a console based simulator. For optimization purposes, these files serve as the basis for chromosomes in the genetic optimizer (with an optimization file providing the constraints for the search space). At the end of the day, develop a simple generic settings management system that can be abstracted for different targets.
- Contract Manager / Base Contract – The core component of any system is the instrument that you are trading / measuring. The contract manager provides position management and risk management abstractions, as well as contract locating functionalities. Ultimately any object that requires a contract, goes through the contract manager, and is given an abstraction of a base contract. The base contract can be a futures contract, equity, bond etc. This provides for a universal interface to subscribe to market data, and issue / monitor orders.
- Strategy Engine / Base Strategies – The strategy engine is the very heart of any trading system. This basic class subscribes to message pumps and processes the messages to handle orders. It is the most versatile object in the trading system, allowing for nearly any type of strategy.
- Charting – Few systems put enough emphasis on thorough charting, but I find it critical for visualizing the results of a simulation, as well as determining what is happening during live trading. All contracts and strategies implement a simple IChartable interface that allows them to output highly configurable charts, right down to the Graphics handles. This allows the charts to be presented in a live windows forms view, or painted to a Bitmap class for saving to disk.
- Logging – At the end of the day, traceability is critical. Every trade made needs to be serialized to disk / database in order to reconcile with your clearing house. Furthermore, every strategy needs to output useful tracing information to aid in debugging. Beyond the obvious tracing, strategies also need to implement a reporting interface to provide live state information to the user interface in order to determine how it is behaving, and if necessary to modify its parameter set, or to debug the strategy. This again is abstracted, just like settings and charting to go to different destinations based on the target of the trading engine. For simulation it outputs to the simulation results, whereas in live trading we work against easily queried database engines.
There are six major components to an automated trading system.
- Live Trading Engine – Any given system will start with the live trading engine. This is the piece of software which runs in real time and actually places orders and reacts to market data.
- Simulation Engine – When developing strategies, you often need to back test them. In an ideal world back testing would demonstrate profitability, but in reality it is just used to verify that your strategy does what you think it does. The key to a good simulation engine is that you run the exact same code in simulation as you do in production. I can’t understate that last sentence, so I’ll state it again – the key to a good simulation engine is that you run the exact same code in simulation as you do in production.
- Historical Service – this runs hand in hand with the simulation engine. You need a tick database for simulation. This is the backbone of all research applications, from back testing strategies to developing market models, you need a thorough, indexed, tick database. You can also build bar data from ticks, but you better have ticks available for simulation.
- Optimization Engine – All of your automated strategies require parameterization. Generally speaking these are best optimized by hand through selection of sensible variables. Sometimes however, you need to parameterize a simple strategy for a large number of symbols, in which case you want an automated system for optimization. Our system uses a cloud computing service to distribute instances of our simulation engine which run chromosomes from a centralized genetic optimization engine.
- Analytics – You need to ruthlessly track your trading performance. At the core of any solid trading engine is a solid analytics engine which tracks your various strategies.
- Reconciler – This was the biggest surprise coming from retail brokers to institutional brokers, but everyone makes mistakes. Sometimes the exchange will fail to tell your clearing house about trades you made, other times your clearing house will accidentally include another clients trades in your account. At the end of every day you need to reconcile every fill you think you made with the statements you receive from your clearing house and immediately reconcile any errors with your clearing house and the exchange.
Next up, I will cover the major components of the Trading Engine.
So you want to develop an automated trading system. No problem, you’ve noticed every time S&P goes up, the Russell 2000 should not be far behind (strategy used for example, please don’t go trying this). Here again, most people I know get started with seeing the phenomena in TWS, and deciding they want to automate the process. Now you fire up visual studio, create a swiss army knife C# console application and include my C# Interactive Brokers library. Seems easy enough, bind to data update events for S&P and Russell 2000, keep a local state variable of the inside prices of each, and when the calculated spread between them moves up or down, buy or sell the other leg.
Believe it or not – it is this simple, and is how I suggest most developers get started. You really need to see your automated widget trade sooner than later. The longer you live in development hell, the longer it is before you understand what your system need to do. The key to your first automated trader is to make it as simple as possible. Do not try to start off with an interface abstracted, general purpose strategy engine; instead build as simple an engine that trades every day, and recognize the common challenges / design patterns required as you develop.
My next series of articles will cover the architecture of a monolithic strategy engine, but I do not want to preach this as the solution to everyone’s needs. Keep it Simple Stupid.
It has been a long time since I have done a series of blog posts on the various automated trading technologies, but after two years of developing our in house trading system, I’d like to cover some of the basics of automated trading systems and hopefully open a dialog with other automated traders.
I’d like to break these up into three major pieces:
- Vendors and Organization – this includes clearing houses, execution platforms etc, fees etc.
- Technology – These are your basic platform decisions, Windows vs *nix, x86 vs x64 etc.
- Software – the meat of our system – cover the various components required to build a fully functional ATS, as well as a research system.
My next post will be an overview of what is required to do automated trading.
Sorry for the long gap on infrastructure lately – I actually have intentions of discussing my production system again (mostly written against Trading Technologies api), but I still do considerable work with Ib’s api and have run into trouble.
All of my prior interfaces with Ib have been for strictly self contained situations. Essentially where the program is placing orders and monitoring messages for the orders it has placed. I am now writing an interface to listen for any orders placed by any application or directly through the TWS interface. It appears I can use the RequestOpenOrders method with a clientId of -1 and receive all open orders at that moment in time, or I can use the RequestAutoOpenOrders to map TWS order messages to my client with a clientId of 0. But it does not appear there is any way to get all order updates from any target without constantly querying the open orders method.
Does anyone have a better way to do this?