Monday, November 26, 2012

Algorithmic Trading System Update 11/26/2012

I'm going to being posting my progress on my algo trading system. I know this will likely target a small niche but hopefully, I might be able to attract some people with ideas to share.

In the past couple of weeks, I've set up a connection to my brokerage API. The API lets me access security prices, check my account details, positions, and send orders. I've set up a Sql Server database to house 5 second, 1 minute, and 1 day  price bar data (High, Low, Open Close, Volume, Time). I'm currently feeding data into the database for about 50 tickers. Everything runs on a single machine right now, and it's running very smoothly. CPU utilization is very low and disk space is also very low. With less than 500K rows in the database, the database still remains under 50 megs.

I've had some advice to store price data in intelligently named flat files, but this takes longer to set up than putting everything in a database. I decided to go with a smaller startup time in getting this system running. I'm keeping the price data for all the different securities in the same table, but I'm using different tables for the different time periods (5 seconds, 1 minute, 1 day).

I've created a toy algorithm, which is basically an impractical algorithm so that I can build all of the other infrastructure around it. The algo has 4 parameters. An upper bound, a lower bound, the security, and current position. Basically, if the stock price goes above the upper bound, we open a short position. Then it waits until the price drops to the lower bound and it will close the opened position. This is a short only strategy requires that we don't repeatedly open short positions while the price is above the upper bound, which would be disaster.

This toy algorithm made it easier to create all of the infrastructure around it. The algorithm parameters are stored in the database. Each algorithm can have multiple sets of parameters, so that I can test different parameters with the same algorithm in order to find an optimal set. Each algorithm contains a list of tickers that it will monitor the prices of. I feed any new price data into the appropriate algorithms as they come in, and the "brain" of the algorithm recalculates the bound checks.

This is an algorithmic trading system, not an automated trading system. I am not interested in having an automated system right now. I've been a programmer for a long time and I know that you can't trust code, even if you wrote it yourself. Instead of allowing my system to make any trade it thinks it should do, I've actually made it email me the proposed trade. At that point, I will know to approve or reject the trade.

In order to speed things up, I've spent a lot of time making a lot of the code run in parallel. I'm currently doing this in C# and the listeners of events are called in sequence that they were added. After a little bit of research and time, I am now raising the events asynchronously and simultaneously. Some locking mechanisms had to be added so to prevent race scenarios but it's all working nicely now.

Errors can come up any time and from anywhere, and multi threading makes it that much worse. I've created a table in the database to store any thrown exceptions. As of right now, most exceptions that are thrown are stored in the database.

I don't want to be modifying data tables in a database by hand, so I created a GUI to show me what I'm interested in. It currently shows the quotelist of all of the tickers that I'm watching, all the algorithms that are currently running, and all of the parameters that each algorithm is using. The GUI also shows the recent errors / messages stored in the database.

That's what's been done, and it has taken me approximately about 60 hours of work and research. There's still a ton to do. My system right now cannot make any trades via the API. I need to write an order manager.  It needs to keep track of what orders are made. Some complexities are when I turn off the system and then turn it back on. There's gotta be some reconciliation where a manually opened position can reconcile with an order that was done through the system on a prior day.

I'm still working on storing historical data. The system right now is storing real time data, but I don't have historical data in there yet. The API from my brokerage does not make it easy to retrieve historical data. There are pacing limits in place which prevent you from datamining their servers. If I can write something which will pace my requests, it can become very useful.

This is all infrastructure work on the list. Once this is all good, I need to write some REAL algorithms. As I've mentioned before, I plan to be using a multi-factor model with momentum and moving average factors. I'm sure those pieces will be a real joy to program. The algorithm framework that I have so far is versatile enough to potentially make statistical arbitrage trades. I would love to add those to the list but there's still a lot of learning that I need to do. There are a ton of small opportunities that the big guys don't bother to look for, and I'd like to put myself in the position to be able to exploit those niches.

No comments:

Post a Comment