Tuesday, May 29, 2012

"Good" versus "bad" backtests

Suppose you're backtesting two different system you designed. The first system's results show a slightly positive expectacy, 0.05R (5% return per trade). The second system yields a surprisingly negative expectancy of -0.25R (-25% return per trade).

Which backtest yielded better results?

If you're entering the market randomly (e.g. by flipping a coin), your base expectancy is 0.00R. On a long enough time scale, you expect to win as much as you lose. Thus, we should approach all potential systems with an expectancy of 0.00R.

If a system you backtest returns an expectancy close to 0.00R, you can say that the system is "bad", in that you're not much better off than flipping a coin or throwing a dart. If your system returns a significant positive expectancy (I would rate 0.2R or above to be acceptable, from my experience so far), then it is "very good".

But what of a significantly NEGATIVE expectancy? Remember that we approach all systems with a base expectancy of 0.00R. A negative expectancy is telling us what NOT to do. And just as importantly, it's pointing us in the general direction of WHAT to do, i.e. do the "opposite" of the system.

So, we can say that the backtest result of 0.05R to be bad, even though it is slightly positive. We can also say that the backtest result of -0.25R is good, not because that particular system is tradable, but because there's alot more we can learn from. We can use this knowledge to avoid the mistakes highlighted in the system, and go on to create better systems in the future.

No comments:

Post a Comment

Note: Only a member of this blog may post a comment.