Five Steps to Backtesting Successfully

May 18, 2008: 11:41 AM CST

We know that backtesting our ideas and strategies in the markets can lead to improvements in trading results – some would consider the process essential – but where do we start and what progression must we take?

I’ve been testing out different ideas and strategies over the last few weeks and have eliminated some ideas I thought might work (but didn’t) and have opened my awareness to ideas I thought were impractical (but worked).

Here’s a quick list of steps to backtest a strategy or trading idea.

1. Clarify the Concept in your Mind
2. Define Rules for the Concept into a Programming Language
3. Run backtests with different parameters across different markets
4. Visually inspect buy and sell signals for logic and clarity (fact-check)
5. Evaluate the System and Continue to Backtest

1. Clarify the Concept

There are different types of systems, including volatility breakout, trend following, mean reversion and you need to know what you’re testing and why. Are you looking to see what happens when a Stochastic gets ‘oversold’ and you buy? Are you looking to buy when two moving averages cross? If so, what does that say about the larger picture of your strategy? Clarify what you want to examine in your mind before you examine it so that it can direct your thinking.

2. Formally Set Rules (program them)

At this stage, you may need assistance, but most software applications already have ‘canned’ strategies or indicators you can use. This step can also be as easy as “insert strategy” when it’s already pre-defined or downloaded from a website. Otherwise, you need to familiarize yourself with the respective language and formalize entry and exit signals.

3. Test Your Strategy Across different Markets

Also, test it across different time intervals and data. For intraday data, it’s best to test 2 years or more; for daily data, expect to test at least 5 or more years, and for weekly data, test over 10 years or more. Also, try to incorporate some sort of defined bull market, bear market, or sideways (consolidation market). Note and compare results in each condition.

4. Inspect the Buy and Sell Signals

Sometimes, a system may look great on ‘paper’ but when you look at the buy and sell signals, there’s something odd that leaps off the page at you. For example, what if your system got you short a few days before the 1987 crash? One signal could show a massive profit which was due to chance which skews that data. Also, make sure your signals are being executed where you expect. If you want to test out a 20 period Bollinger Band, make sure you use that indicator on the chart and see where signals were executed. I had trouble with a Point and Figure strategy I tested and noticed the signals were being executed at unexpected places.

5. Evaluate and Continue

You define what’s most important to you – be it % Win Rate (which is not as important); $ Net Profit; Maximum Drawdown; Annualized % Return; Number of Trades, etc. Only you can define what’s important. Net Profit may not be the best single statistic, even though it may seem logical to maximize this. Look deeper in your data and continue testing until you have a clearer understanding of how your system works and whether the program is capturing what you expect to see.

Don’t expect one test to end all for you. Backtesting is a continual process in which you must guard against over-optimization and strive for robustness at the expense of grandiose profits. I’ll keep sharing some of my experiences with you as I continue to backtest different ideas and strategies.

1 Comment

One Response to “Five Steps to Backtesting Successfully”

  1. Cceng34 Says:

    New Cheap
    Oakley Sport Sunglasses Store White
    Should you be in a prefer to
    come up with a item by some cherished one, sdjfakjsdak bobble head is a great
    Oakley Polarized Sunglasses Store Wholesale
    Bobble head can
    certainly placed the have fun on an indivdual's skin. Many men and women
    discover these types of crazy because the large head that will bobbles on the
    merest engage from the hand.New
    Cheap Oakley Lifestyle Sunglasses Store Discount
    This sort of
    treats are generally extremely distinctive as well as the phone may think
    exclusive along with astonished to obtain