Navigation

    Quantiacs Community

    • Register
    • Login
    • Search
    • Categories
    • News
    • Recent
    • Tags
    • Popular
    • Users
    • Groups

    Q18 testing

    Support
    5
    8
    503
    Loading More Posts
    • Oldest to Newest
    • Newest to Oldest
    • Most Votes
    Reply
    • Reply as topic
    Log in to reply
    This topic has been deleted. Only users with topic management privileges can see it.
    • T
      TheFlyingDutchman last edited by

      Hi, I run into quite large differences in Sharpe Ratio's in the development/testing environment and the contest submit fase. Is there a reason for this difference? What is the correct SR?
      Best Regards

      A 1 Reply Last reply Reply Quote 0
      • support
        support last edited by

        Hi, this should not happen. Can you let us know some example? You can also send some details to info@quantiacs.com.

        1 Reply Last reply Reply Quote 0
        • A
          alfredaita @TheFlyingDutchman last edited by

          @theflyingdutchman
          You may have too many parameters in your strategy resulting in over fitting. This often happens when optimizing on asset by asset basis with too many indicators. Try generalizing first.

          1 Reply Last reply Reply Quote 1
          • E
            EDDIEE last edited by

            I have a different issue with testing and submitting.

            I tested my strategy (Ridge Regression, random state = 18) with multi-pass backtesting using qnbt.backtest_ml. The Sharpe Ratio is 1.13, there is no forward looking, because the Sharpe Ratio is the same when predict_each_day is set to True.

            Now comes the problem: I submitted this strategy (written in multi-pass backtesting format) and this strategy always gets rejected, because the Sharpe Ratio is drastically smaller 1 (exactly
            0.49).

            Why is there such a big difference in the Sharpe Ratio?

            weights = qnbt.backtest_ml(
            train=train_model,
            predict=predict,
            train_period=4x365, # the data length for training in calendar days
            retrain_interval=5x365, # how often we have to retrain models (calendar days)
            retrain_interval_after_submit=50, # how often retrain models after submission during evaluation (calendar days)
            predict_each_day=True, # Is it necessary to call prediction for every day during backtesting?
            # Set it to true if you suspect that get_features is looking forward.
            competition_type='stocks_nasdaq100', # competition type
            lookback_period=365, # how many calendar days are needed by the predict function to generate the output
            start_date='2006-01-01', # backtest start date
            build_plots=True # do you need the chart?
            )

            support 2 Replies Last reply Reply Quote 0
            • support
              support @EDDIEE last edited by

              @eddiee Dear eddiee, sorry for the delay. Did you try to retrain the model every day during the testing? Simulation will be very slow but the result should match then.

              1 Reply Last reply Reply Quote 0
              • support
                support @EDDIEE last edited by

                @eddiee Please use "retrain_interval_after_submit=None". In this way the retraining setting used during development will be used also after submission.

                E 1 Reply Last reply Reply Quote 0
                • E
                  EDDIEE @support last edited by

                  @support Thanks a lot!

                  1 Reply Last reply Reply Quote 0
                  • X
                    xfinityauthorize Banned last edited by

                    This post is deleted!
                    1 Reply Last reply Reply Quote 0
                    • First post
                      Last post
                    Powered by NodeBB | Contributors
                    • Documentation
                    • About
                    • Career
                    • My account
                    • Privacy policy
                    • Terms and Conditions
                    • Cookies policy
                    Home
                    Copyright © 2014 - 2021 Quantiacs LLC.
                    Powered by NodeBB | Contributors