How are models ranked on the leaderboard before the live period?
-
I'm always happy when I see submissions of mine among the upper positions on the leaderboard, allthough I'm aware it doesn't really matter much until after the submission deadline (or actually after the live period). But somehow I think it should give me at least an idea about what to expect in the contest.
However, I really don't understand how they are ranked right now.
At first glance it could be the in-sample sharpe, but there are so many cases where algos with a lower IS sharpe are higher on the board than ones with a higher IS sharpe. And it also wouldn't make much sense because the IS duration isn't the same for every strategy.
Then I thought it might be the total sharpe from the backtest-start up to now. But that doesn't add up either, at least not if the total sharpe is the one displayed when you click on the chart of a strategy.Could you please explain which criteria define the position on the leaderbord right now?
Let's assume we're only talking about one contest, say, the one for Bitcoin.
For the live period it seems to be clear, simply the OOS sharpe starting after the deadline. -
Here's an example with 2 strategies highlighted. The upper one has a sharpe of 1.92, the second one 3.23
-
Hello.
This list is sorted by min(in_sample_sharpe, out_sample_sharpe).
But you can click on headers in this table to change the order.
Regards.
-
@support
Only 15 strategies are selected for the Live competition? -
@sheikh yes, each user can submit at most 15 systems to each contest. If the user does no selection, the top 15 according to Sharpe ratio will be used for the live evaluation.
It is a small attempt to prevent overfitting, otherwise a user could send thousands of submissions and win by luck only.
-
@support
oh I see now what you mean.
15 strategies PER USER are selected.
At first, I thought you were only going to select 15 strategies total for all users.
Thanks.
