I've been spending the last few weeks backtesting. On a typical day, I'll sink at least three hours into manually backtesting various concepts.
Concepts I've backtested in the last fortnight:
1) Volatility breakout (enter when daily price movement > ATR(14) * 2)
This initially showed promise on volatile pairs like the GBPUSD, but it broke down on quiet pairs like the USDCAD. If I focused on the volatile pairs on the daily TF, my sample size would've been too low for my liking, so decided to abort. Off a sample size of ~265 from five currency pairs, I believe my expectancy was around 0.025R after spread. This was using a 0.75 reward-to-risk ratio, which seemed the most optimal from the various R:R ratios I tested. Very marginal.
2. Retracement entry system
I used a 21 EMA to measure the trend. Once the MA showed a clear trend and any old counter-trend candle presented itself, I'd enter on the break of the counter-trend candle. The SL was between the break of the counter-trend candle and the previous swing point. TP was set at 1.68 * SL (a Fibonnaci extension level). After 104 samples, expectancy was 0.04R before spread. Not remarkable so gave this a miss.
3. Price Action on Pullback + Elliot Wave system
This is a similar system to #2. I used a 21 MA to measure the trend. If a clear trend was present, I'd wait to see if an Elliot Wave formation presented itself (higher high, higher lows if bullish, lower highs and lower lows if bearish). In the case of a bullish trend, if a price action signal appeared at a higher low (pullback), you'd enter on the break of this signal. I used pinbars, engulfing bars and two-bar reversals as signals.
So far I gathered 125 samples from the AUDUSD and USDJPY on the daily TF from 1996 to 2011. Using a 2:1 reward-to-risk ratio, expectancy was 0.28R per trade, which is promising (28% return per trade). However, this is a discretionary trading system, which is difficult to backtest accurately. Discretionary systems introduce all sorts of biases.
What I learned so far
One lesson I learned over the last fortnight is how every currency pair has their own personality. For example, I noticed that the USDCAD tends to be more docile, making a volatility-based system problematic. I believe the "holy grail" should be able to trade on most currencies and turn a profit, or at least breakeven during adverse conditions. If a "trading system" only works on a small handful of currency pairs, I think what you have is a meaningless curve-fitting expression. A good system should be profitable globally.