Q18 testing
-
Hi, I run into quite large differences in Sharpe Ratio's in the development/testing environment and the contest submit fase. Is there a reason for this difference? What is the correct SR?
Best Regards -
Hi, this should not happen. Can you let us know some example? You can also send some details to info@quantiacs.com.
-
@theflyingdutchman
You may have too many parameters in your strategy resulting in over fitting. This often happens when optimizing on asset by asset basis with too many indicators. Try generalizing first. -
I have a different issue with testing and submitting.
I tested my strategy (Ridge Regression, random state = 18) with multi-pass backtesting using qnbt.backtest_ml. The Sharpe Ratio is 1.13, there is no forward looking, because the Sharpe Ratio is the same when predict_each_day is set to True.
Now comes the problem: I submitted this strategy (written in multi-pass backtesting format) and this strategy always gets rejected, because the Sharpe Ratio is drastically smaller 1 (exactly
0.49).Why is there such a big difference in the Sharpe Ratio?
weights = qnbt.backtest_ml(
train=train_model,
predict=predict,
train_period=4x365, # the data length for training in calendar days
retrain_interval=5x365, # how often we have to retrain models (calendar days)
retrain_interval_after_submit=50, # how often retrain models after submission during evaluation (calendar days)
predict_each_day=True, # Is it necessary to call prediction for every day during backtesting?
# Set it to true if you suspect that get_features is looking forward.
competition_type='stocks_nasdaq100', # competition type
lookback_period=365, # how many calendar days are needed by the predict function to generate the output
start_date='2006-01-01', # backtest start date
build_plots=True # do you need the chart?
) -
@eddiee Dear eddiee, sorry for the delay. Did you try to retrain the model every day during the testing? Simulation will be very slow but the result should match then.
-
@eddiee Please use "retrain_interval_after_submit=None". In this way the retraining setting used during development will be used also after submission.
-
@support Thanks a lot!
-
This post is deleted!
