What is forward looking and why it's effective badly to strategy?
-
Hello, I am making a simple strategy as follows:
close = data.sel(field="close")
price_future = qnta.shift(close, -1)
target_price_up = xr.where((price_future < close), 1, -1)As I understand it, this is forward-looking, resulting in a very high Sharpe (>2), however after avoiding forward-looking, Sharpe is < 0. Why does this cause such a serious decrease in sharpness?
Please explain to me. Thank you.
-
The sharp decrease in Sharpe ratio after removing forward-looking bias is because the original strategy artificially inflated performance by using future data, which isn't possible in real trading. Once you eliminate this bias, the strategy's true risk and lower performance are revealed, leading to a more realistic but lower Sharpe ratio.
-
@vyacheslav_b Thank you for responding.
I realize that even now I have a strategy that is not forward-looking
close = data.sel(field="close")
price_yesterday = qnta.shift(close, 1)
target_price_up = xr.where((price_yesterday < close), 1, 0)Why is Sharpe worse after the strategy goes through training-testing by ML/DL models compared to when using a single backtest?
I have tried many different strategies and changed different feature trains, but they all showed that after going through a certain model, the sharpness decreased compared to when using a single backtest. Please answer. Thank you so much.
-
@illustrious-felice Hi, most likely because when you use a "single backtest" you are using all the data at your disposal.
When you use a ML model and split training and validation sets, then you have a set (the validation one) which is blind to the existing data.
In the end, what matters is the live performance.
-
@support Oh, I see. Thank you for your quality feedback. Hopefully, Quantiacs can produce more examples of strategies whose input is not forward-looking (close.shift(time=-1)) so that everyone can more conveniently apply ML models to build strategies. Thank you
-
@illustrious-felice Hello. Sometimes an error can occur at the data preprocessing stage. It's possible to inadvertently use future data.
Quantiacs has an excellent mechanism for quickly checking such errors.
In any strategy, there is a file:
https://github.com/quantiacs/toolbox/blob/main/qnt/precheck.ipynbRun it on 3-5 splits and compare the statistics. If there are discrepancies, then it is likely that the strategy is peeking into the future.
If you set a very large number of splits, it will be an example of how the online check of submitted strategies in the contest works.
Intermediate results can be viewed in HTML format in the folder.
-
@vyacheslav_b Thank you very much.
I would like to ask why when I add a - sign or 1/weight (inverse, reverse) to the strategy's weight, I cannot reverse the PnL chart? On other platforms, I still use this technique to reverse the chart. Thank you
-
@illustrious-felice Hi, that is not automatic as all trades are punished by slippage, and that is a subtraction from profits irrespective on the sign of the weights.
