The main crux of my problem is building a platform that is very flexible but highly efficient. If I was developing a platform that tested less than 2 gig of data...I'd be home free. But, get past the 2 gig barrier and all kinds of these wonderful flexible designs and databases everyone uses as default must be thrown out the window. Can't use a database in the traditional sense for your time series. Not if you want your simulation to finish in under an hour.
The simplest approach is to use flat text files as I've mentioned in the blog before. But, as you ramp up the dynamic nature of your simulation testing...you need a way to index those files. And the kicker is even the index cannot fit into memory. So, you wrestle with the memory barrier and disk io. Even creating binary files in order to reduce the size of your files. But, binary files limit your flexibility. I'm still testing and refining the solution...but slowly getting there. Basically taking a process that used to run 45 minutes long and dropped it down to less than 10 minutes. With the added ability to randomly seek to any date or symbol in the 20,000 symbol database quicker than you can say Jack Robinson.
Oh well, moving on. It's way past portfolio performance time...so better late than never...the portfolio performance as of May 2009:
As you can see...I'm still getting trounced by the market. Largely, because I'm still 50% in cash and the system normally doesn't perform well during these sharp bouncing bottoms the market has recently experienced. That's the one thing that has been difficult the past few weeks with all the recent buy signals. Many of the signals are from stocks that have already surged greater than 50% to 100% in just one month. Even though my position size is very small per stock...it is still at times tough to buy such a strong performing stock.
And that's it from here where I'm looking forward to the 4th of July, a few cold ones, and time with family.
Finally, my sister-in-law turned the big 40 today. Happy Birthday!
Later Trades,
MT
