Archive for the 'trading' Category

Trading System Framework

The core of our architecture rests on a universal trading system framework. This framework abstracts all of the basic market interfaces, allowing us to write generic strategies that run on any market, including simulation.

2010-11-22 Trading System Framework

As you can see in the above central box, our trading system abstracts several core functionalities.

  • Settings Management – the entire trading system is configured via a straightforward xml configuration file. The actual storage and management of this is abstracted by the particular profile. For live running, these settings are version controlled and managed in a central replicated sql database. For simulation, these are stored as a simple file provided to a console based simulator. For optimization purposes, these files serve as the basis for chromosomes in the genetic optimizer (with an optimization file providing the constraints for the search space). At the end of the day, develop a simple generic settings management system that can be abstracted for different targets.
  • Contract Manager / Base Contract – The core component of any system is the instrument that you are trading / measuring. The contract manager provides position management and risk management abstractions, as well as contract locating functionalities. Ultimately any object that requires a contract, goes through the contract manager, and is given an abstraction of a base contract. The base contract can be a futures contract, equity, bond etc. This provides for a universal interface to subscribe to market data, and issue / monitor orders.
  • Strategy Engine / Base Strategies – The strategy engine is the very heart of any trading system. This basic class subscribes to message pumps and processes the messages to handle orders. It is the most versatile object in the trading system, allowing for nearly any type of strategy.
  • Charting – Few systems put enough emphasis on thorough charting, but I find it critical for visualizing the results of a simulation, as well as determining what is happening during live trading. All contracts and strategies implement a simple IChartable interface that allows them to output highly configurable charts, right down to the Graphics handles. This allows the charts to be presented in a live windows forms view, or painted to a Bitmap class for saving to disk.
  • Logging – At the end of the day, traceability is critical. Every trade made needs to be serialized to disk / database in order to reconcile with your clearing house. Furthermore, every strategy needs to output useful tracing information to aid in debugging. Beyond the obvious tracing, strategies also need to implement a reporting interface to provide live state information to the user interface in order to determine how it is behaving, and if necessary to modify its parameter set, or to debug the strategy. This again is abstracted, just like settings and charting to go to different destinations based on the target of the trading engine. For simulation it outputs to the simulation results, whereas in live trading we work against easily queried database engines.

Next up I want to cut into application design and multithreading. There is a lot to cover, and I am swamped, so expect the articles to continue to appear as I have time. And if you have any questions feel free to email email hidden; JavaScript is required.

Six Pillars of Automated Trading

There are six major components to an automated trading system.2010-11-04 Automated Trading Overview

  • Live Trading Engine – Any given system will start with the live trading engine. This is the piece of software which runs in real time and actually places orders and reacts to market data.
  • Simulation Engine – When developing strategies, you often need to back test them. In an ideal world back testing would demonstrate profitability, but in reality it is just used to verify that your strategy does what you think it does. The key to a good simulation engine is that you run the exact same code in simulation as you do in production. I can’t understate that last sentence, so I’ll state it again – the key to a good simulation engine is that you run the exact same code in simulation as you do in production.
  • Historical Service – this runs hand in hand with the simulation engine. You need a tick database for simulation. This is the backbone of all research applications, from back testing strategies to developing market models, you need a thorough, indexed, tick database. You can also build bar data from ticks, but you better have ticks available for simulation.
  • Optimization Engine – All of your automated strategies require parameterization. Generally speaking these are best optimized by hand through selection of sensible variables. Sometimes however, you need to parameterize a simple strategy for a large number of symbols, in which case you want an automated system for optimization. Our system uses a cloud computing service to distribute instances of our simulation engine which run chromosomes from a centralized genetic optimization engine.
  • Analytics – You need to ruthlessly track your trading performance. At the core of any solid trading engine is a solid analytics engine which tracks your various strategies.
  • Reconciler – This was the biggest surprise coming from retail brokers to institutional brokers, but everyone makes mistakes. Sometimes the exchange will fail to tell your clearing house about trades you made, other times your clearing house will accidentally include another clients trades in your account. At the end of every day you need to reconcile every fill you think you made with the statements you receive from your clearing house and immediately reconcile any errors with your clearing house and the exchange.

Next up, I will cover the major components of the Trading Engine.

Automated Trading System Development

It has been a long time since I have done a series of blog posts on the various automated trading technologies, but after two years of developing our in house trading system, I’d like to cover some of the basics of automated trading systems and hopefully open a dialog with other automated traders.

I’d like to break these up into three major pieces:

  1. Vendors and Organization – this includes clearing houses, execution platforms etc, fees etc.
  2. Technology – These are your basic platform decisions, Windows vs *nix, x86 vs x64 etc.
  3. Software – the meat of our system – cover the various components required to build a fully functional ATS, as well as a research system.

My next post will be an overview of what is required to do automated trading.

Stay Tuned…

Multithreading made easy

I love when Robotics innovations overlap with Finance. Our trading engine is a discombobulated series of threads interfacing to their respective data sources and execution platforms, with careful locking in between. Making changes to this system has become a nightmare due to all the careful locking concerns. Enter Microsoft CCR. It provides a very clean and thoroughly tested set of primitives for a large scale thread pool with inter task messaging. It even has adapters for using tasks on Windows Forms threads, as well as WPF dispatches. It was first distributed with the Microsoft Robotics Studio, as they suffer from the same problems we do in finance, talking to a host of sensors and actuators with careful locking constraints.

To illustrate a common problem with trading systems. You have a producer consumer model, where your data is coming in from the exchange on a single thread. This data is going to a large number of contracts, and it would be nice to do this in a multithreaded model. Under the ordinary paradigm, you would create a queue, have your producer thread lock on the queue, add the new market ticks, and unlock, letting your consumers take locks on the queue, and beginning to work on this. There are special concerns though, for a given contract you need to make sure all ticks are handled synchronously, and whenever the queue hits zero for a particular contract after data has changed, you want to tell your strategy to process the new data. Here you would need a second producer consumer framework, and things continue expanding from there.

With the CCR, you would break things down into tasks. There would be a task that consumes raw market data and posts it to a port. There would be a task that takes a tick from the market data and posts it to a contract. There would be a task that takes a tick in a contract and updates the contract, checking at the end if the contract’s queue is empty, if so, it would post the entire contract to the strategy saying it has been updated. There would be a task that consumes a contract and updates the strategy. All of these are small work tasks that are setup to run in a common dispatcher pool, and are all invoked every time a work product is posted to a port. No complicated setup, just arbitrated registrations.

I’d love to spend a lot more time talking about this, but here is one short example.

</code>using (var dispatcher = new Dispatcher(0, "Master Dispatcher"))
{
var dispatcherQueue = new DispatcherQueue("Master Queue");
var tickPort = new Port<int>();

Arbiter.Activate(dispatcherQueue, Arbiter.Receive(true, tickPort, tick=>
{
if(tick > 25)
{
Market.PlaceOrder(tick, 1);
}
}));
while(true)
{
tickPort.Post(new Random().Next(50));
}
}

I realize there are many simplifications made in this post – I just wanted to convey how cool the CCR framework is. If it were free, I’d post my Ib wrapper for it! Easily worth the $400, and if you are a student you can get it from the Robotics Studio for free. There is talk of merging it with the parallels framework added in C# 4.0 (another great invention… waiting for VS 2010 to fully integrate it though).

Black Box Development

In late 2008/early 2009 I made the transition from full time engineering to full time Black Box trading software and strategy development. The past several months have certainly been exciting times in the financial markets, and proven to be very good for automated strategies.

I will still be maintaining the IbAPI open source library (just saw IB posted a 9.62 beta), and if anything will be more responsive now.

I am also always interested in discussing interesting opportunities, so please continue to drop me a line at email hidden; JavaScript is required

Good Trading!

-Karl