Navigation

    Quantiacs Community

    • Register
    • Login
    • Search
    • Categories
    • News
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    How are models ranked on the leaderboard before the live period?

    General Discussion
    3
    6
    494
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • A
      antinomy last edited by

      I'm always happy when I see submissions of mine among the upper positions on the leaderboard, allthough I'm aware it doesn't really matter much until after the submission deadline (or actually after the live period). But somehow I think it should give me at least an idea about what to expect in the contest.

      However, I really don't understand how they are ranked right now.
      At first glance it could be the in-sample sharpe, but there are so many cases where algos with a lower IS sharpe are higher on the board than ones with a higher IS sharpe. And it also wouldn't make much sense because the IS duration isn't the same for every strategy.
      Then I thought it might be the total sharpe from the backtest-start up to now. But that doesn't add up either, at least not if the total sharpe is the one displayed when you click on the chart of a strategy.

      Could you please explain which criteria define the position on the leaderbord right now?
      Let's assume we're only talking about one contest, say, the one for Bitcoin.
      For the live period it seems to be clear, simply the OOS sharpe starting after the deadline.

      1 Reply Last reply Reply Quote 1
      • A
        antinomy last edited by

        Here's an example with 2 strategies highlighted. The upper one has a sharpe of 1.92, the second one 3.23

        Board.jpg ML_BTC.jpg
        Chrono.jpg

        1 Reply Last reply Reply Quote 1
        • support
          support last edited by

          Hello.

          This list is sorted by min(in_sample_sharpe, out_sample_sharpe).

          But you can click on headers in this table to change the order.

          Regards.

          S 1 Reply Last reply Reply Quote 1
          • S
            Sheikh @support last edited by

            @support
            Only 15 strategies are selected for the Live competition?

            support 1 Reply Last reply Reply Quote 0
            • support
              support @Sheikh last edited by

              @sheikh yes, each user can submit at most 15 systems to each contest. If the user does no selection, the top 15 according to Sharpe ratio will be used for the live evaluation.

              It is a small attempt to prevent overfitting, otherwise a user could send thousands of submissions and win by luck only.

              S 1 Reply Last reply Reply Quote 0
              • S
                Sheikh @support last edited by

                @support
                oh I see now what you mean.
                15 strategies PER USER are selected.
                At first, I thought you were only going to select 15 strategies total for all users.
                Thanks.

                1 Reply Last reply Reply Quote 0
                • First post
                  Last post
                Powered by NodeBB | Contributors
                • Documentation
                • About
                • Career
                • My account
                • Privacy policy
                • Terms and Conditions
                • Cookies policy
                Home
                Copyright © 2014 - 2021 Quantiacs LLC.
                Powered by NodeBB | Contributors