Tweet
<< understand that your methods are not manipulated to fit your model >>
Billyjoe, that's an excellent point. You're referring to "curve-fitting," which is a trap that model-builders frequently fall into. Curve-fitting is optimizing parameters that work perfectly with test data, so that the results show a big winner. The problem is that when those parameters are used in real trading the system usually performs poorly.
The solution is a two-step process: optimize parameters with old data ("back-testing") and then test the parameters against a second set of data ("forward-walking") and see how the system performed. An example would be to optimize parameters using a 1995 - 1999 data set and then applying those parameters (with no further optimization) against a 2000 - 2004 data set. If the results on both the backtest and forward walk are similar, then congratulations; you designed a robust system.
I used those dates because optimizing a system using 1995 - 1999 data was during a huge bull move and it's unlikely the same parameters would would well during the 2000 - 2002 crash. If it does, you designed a robust system.
Billyjoe, that's an excellent point. You're referring to "curve-fitting," which is a trap that model-builders frequently fall into. Curve-fitting is optimizing parameters that work perfectly with test data, so that the results show a big winner. The problem is that when those parameters are used in real trading the system usually performs poorly.
The solution is a two-step process: optimize parameters with old data ("back-testing") and then test the parameters against a second set of data ("forward-walking") and see how the system performed. An example would be to optimize parameters using a 1995 - 1999 data set and then applying those parameters (with no further optimization) against a 2000 - 2004 data set. If the results on both the backtest and forward walk are similar, then congratulations; you designed a robust system.
I used those dates because optimizing a system using 1995 - 1999 data was during a huge bull move and it's unlikely the same parameters would would well during the 2000 - 2002 crash. If it does, you designed a robust system.
Comment