Different Sharpe ratios in backtest and after submission
-
Hi @support,
The SR of the system "Sun73_Q17_1a" is above 1 in JupyterLab (SR = 1.08), but almost half of it (SR=0.50) after submission to the Q17 contest.
Do you have any clue on that? I am using a modified version of the example "Q17 Supervised Learning".
Many thanks!
-
I am using the "in-sample Sharpe" in the comparison above.
-
@sun-73 I am also facing the same issue. I checked the submission weights and it seems the number and type of assets used in the submission are different. Also, the data seems to be different on some days.
-
@sun-73 Thanks for the note, we are checking, sorry for the issue. When did you submit the code? Do you see a difference in the current notebook result and current submission result?
In other words, how long is the lag between submission time and running the notebook?
-
@anshul96go Thank you. When did you submit the code? Do you see a difference in the current notebook result and current submission result?
In other words, how long is the lag between submission time and running the notebook?
-
Hi @support, I backtested the system in JupyterLab two days ago and submitted on the same day. The submission went online yesterday.
Since I am trying to use a multi-pass strategy based on the function "qnbt.backtest_ml", maybe the difference is due to difference parameters of this function considered in Jupyter and the online filters of the quantiacs website.
Are the parameters of the function "qnbt.backtest_ml" exactly the same in those cases?
For example, I run the code using retrain_interval=90 days. When the system is evaluated online is it retrained every day since 2014 (or every 90 days in this example)? In other words, the parameter "retrain_interval" is forced to 1 when evaluated online?
Many thanks again!
-
By the way, I submitted another system yesterday, called "Sun73_Q17_1b", with an in-sample SR=2, but today the system is online in the website with an in-sample SR=0.7
-
@sun-73 Ok, thanks, then if you are using the backtester with the retraining option that is the reason. Can you check your notebook run using retrain_interval = 1 day?
-
@sun-73 Also here, the reason should be the retraining interval. Can you try with retraining_interval=1?
-
@anshul96go Hi, please check answers to Sun-73. The most likely reason is the retraining option in the backtester. Are you using that version of the backtester?
-
Hi @support,
I modified the retraining interval to 1 day and it worked. Thank you for the help.
You guys rock!
