Navigation

    Quantiacs Community

    • Register
    • Login
    • Search
    • Categories
    • News
    • Recent
    • Tags
    • Popular
    • Users
    • Groups
    1. Home
    2. antinomy
    3. Best
    A
    • Profile
    • Following 0
    • Followers 1
    • Topics 9
    • Posts 65
    • Best 30
    • Groups 0
    • Blog

    Best posts made by antinomy

    • External Libraries

      Hello @support ,

      I've been using cvxpy in the server environment which I installed by running

      !conda install -y -c conda-forge cvxpy
      

      in init.ipynb. But whenever this environment is newly initialized, the module is gone and I have to run this cell again (which takes awfully long).

      Is this normal or is there something wrong with my environment?
      My current workaround is placing these lines before the import

      try:
          import cvxpy as cp
      except ImportError:
          import subprocess
      
          cmd = 'conda install -y -c conda-forge cvxpy'.split()
          rn = subprocess.run(cmd)
      
          import cvxpy as cp
      

      Is there a better way?

      Best regards.

      posted in Support
      A
      antinomy
    • RE: Bollinger Bands

      @anthony_m
      Bollinger Bands are actually quite easy to calculate.
      The middle band is just the simple moving average, the default period is 20.
      For the other bands you need the standard deviation for the same period.
      The upper band is middle + multiplier * std
      The lower band is middle - multiplier * std
      Where the default for the multiplier is 2.

      There's an article on the formula for Bollinger Bands on Investopedia - they use the 'typical price' (high + low + close) / 3 but I think most people just use the close price.

      For the code it depends if you only need the latest values or the history.
      Using pandas the code for the first alternative could be:

      def strategy(data):
          close = data.sel(field='close').copy().to_pandas().ffill().bfill().fillna(0) 
      
          # let's just use the default 20 period:
          period = 20
          sma = close.iloc[-period:].mean()
          std = close.iloc[-period:].std()
      
          # and the default multiplier of 2:
          multiplier = 2
          upper = sma + multiplier * std
          lower = sma - multiplier * std
      

      If you need more than the last values you can use pandas.rolling:

      def strategy(data):
          close = data.sel(field='close').copy().to_pandas().ffill().bfill().fillna(0)
      
          # let's just use the default 20 period:
          period = 20
          sma = close.rolling(period).mean()
          std = close.rolling(period).std()
      
          # and the default multiplier of 2:
          multiplier = 2
          upper = sma + multiplier * std
          lower = sma - multiplier * std
      
      posted in Strategy help
      A
      antinomy
    • Different Sharpe ratios in backtest and competition filter

      When I run my futures strategy in a notebook on the server starting 2006-01-01 I get this result:

      Check the sharpe ratio...
      Period: 2006-01-01 - 2021-03-01
      Sharpe Ratio = 1.3020322470218595

      However, it gets rejected from the competition because the Sharpe is below 1. When I click on its chart in the "Filtered" tab it shows

      Sharpe Ratio 0.85

      When I copy the code from the html-previev of the rejected algo and paste it into a notebook, I get exactly the same result as above (Sharpe 1.3), so it doesn't seem to be a saving error.

      Why is there such a big difference? I thought the backtest results from the notebook shuld indicate if the strategy is elegible for the competition.
      How can I calculate the Sharpe ratio used for the competition in the notebook so I will know beforehand if the algo gets accepted or not?

      posted in Support
      A
      antinomy
    • RE: Announcing the Winners of the Q14 Contest

      Thank you very much, this is awesome!
      And I also congratulate the other 2 winners!
      I got notified by e-mail 2 days ago and have been totally excited since then 🎉

      posted in News and Feature Releases
      A
      antinomy
    • RE: The Q16 Contest is open!

      @support
      First of all, having looked at various sources for crypto data myself, I know this can be a pain in the neck, so I can appreciate the effort you took to provide it.

      I get the method for avoiding lookahead-bias by including delisted symbols. The key point would be what you mean exactly by disappeared and from where.
      Do you mean they were delisted from the exchange where the winning algos will be traded or did the data source you used just not have data?

      To name 2 examples: DASH and XMR don't have recent prices but I don't know of an exchange they were delisted from. When I look them up on tradingview they do have prices on all the exchanges available there and are still traded with normal volumes.

      Charts for their closing price on quantiacs:

      import qnt.data as qndata
      
      data = qndata.cryptodaily_load_data(min_date='2020')
      close = data.sel(field='close').to_pandas()
      close[['DASH', 'XMR']].plot()
      

      Figure_1.png

      On tradingview:

      UDGfbmYT.png

      There are many reasons why we might need prices for symbols that are currently not among the top 10 in terms of market cap. An obvious one would be that they might be included in the filter again any second and algorithms do need historical data. Also, there are many ways to include symbols in computations without trading them: as indicators, to calculate market averages and so on.

      posted in News and Feature Releases
      A
      antinomy
    • RE: The Q16 Contest is open!

      @support In my posts I was merely thinking about unintentional lookahead bias because when it comes to the intentional kind, there are lots of ways to do that and I believe you never can make all of them impossible.
      But I think that's what the rules are for and the live test is also a good measure to call out intentional or unintentional lookahead bias as well as simple innocent overfitting.

      To clarify the Quantopian example a bit, I don't think what I described was meant to prevent lookahead bias. The 8000 something symbols just was all what they had and the rules for the tradable universe were publicly available (QTradableStocksUS on archive.org). I just thought, providing data for a larger set than what's actually tradable would make the scenarios I mentioned less likely. For that purpose I think both sets could also be openly defined. Let's say the larger one has the top 100 symbols in terms of market cap, dollar volume or whatever and the tradable ones could be the top 10 out of them with the same measurement.

      On the other hand, I still don't know if those scenarios could become a real problem. Because what good does this foreknowledge if you can't trade them yet? And after they're in the top 10 it would be legitimate to use the fact that they just entered, because we would also have known this at that time in real life.

      posted in News and Feature Releases
      A
      antinomy
    • RE: The Q16 Contest is open!

      @support
      I totally agree that the indicator usage is not trivial at all regarding lookahead-bias, still trying to wrap my head around it 😉
      The symbol list alone could already lead to lookahead bias - in theory, I don't have a realistic example.
      Because if the symbol is in the dataset, the algo could know it will be among the top 10 at some point, thus probably go up in price.
      I guess we really need to be extra careful avoiding these pitfalls, but they might also become apparent after the submission...

      From what I understand this contest is kind of a trial run for the stocks contest, so may I make a suggestion?
      On Quantopian there was data for around 8000 assets, including non-stocks like ETFs but for the daily contest for instance, the symbols had to be in the subset of liquid stocks they defined (around 2000 I think).

      The scenarios

      1. there's a price for that stock, it will become large
      2. it's included in the asset list, it must go up some day

      were not really a problem because there was no way to infer from prices or the symbol being present that it will be included in the filter some day.

      Maybe you could do something like that, too?
      It doesn't have to be those kind of numbers, but simply providing data for a larger set of assets containing symbols which will never be included in the filter could avoid this problem (plus of course including delisted stocks).

      For this contest I think your suggestion to retroactively fill the data if a symbol makes it on top again is a good idea.

      posted in News and Feature Releases
      A
      antinomy
    • RE: The Quantiacs Referral Program

      @news-quantiacs
      Hello,
      about that link, the important part is the one that starts with the question mark, with utm_medium being our unique identifier, right?
      So, can we change the link to point to the contest description instead of the login-page, like this?
      https://quantiacs.com/contest?utm_source=reference&utm_medium=19014
      Then interested people could first read more details about the contest before signing up...

      posted in News and Feature Releases
      A
      antinomy
    • RE: Different Sharpe ratios in backtest and competition filter

      I managed to implement it without global variables, now the sharpe ratio matches and it got accepted.
      Thanks again!

      posted in Support
      A
      antinomy
    • RE: Files disappeared from online env

      Bumping in here because something similar happened to me a few weeks ago. I had been training several neural networks and stored the trained models as pickle files online. A few days afterwards they were gone. I don't really need them any more but since it got mentioned now, I'm curious why this happened and how to prevent it in case one actually needs those files.

      posted in Support
      A
      antinomy
    • RE: Submission Issue

      Update:
      The stateless version got accepted now, thanks!

      posted in Support
      A
      antinomy
    • RE: Issues with the Legacy Website

      @jeppe_and Ok, thanks for the quick reply!

      posted in Support
      A
      antinomy
    • RE: Submission Issue

      Just out of curiousity I did some testing and it looks like the class actually was the culprit.
      I submitted a simple strategy in 2 versions, one with a class and the other with a dictionary as state. The class version was rejected (exaclty like the one from my 1st post) and the dictionary version got accepted.

      posted in Support
      A
      antinomy
    • RE: Submission failure

      @support Now they're all there, thank you!

      posted in Support
      A
      antinomy
    • Correlation Check always fails

      Hello,
      whenever I run a backtest on the server I get the message

      WARNING! Can't calculate correlation.

      This has been happening since I started developing for the Q16 contest.
      I don't know if this has any influence on the actual submission check and we'll soon have x times the quickstart template in the contest 😉
      Anyway, it would be good if the correlation check would work before we submitt algos that will eventually fail the correlation filter.

      posted in Support
      A
      antinomy
    • Issues with the Legacy Website

      Hello,

      I'm having 2 issues with legacy.quantiacs.com:

      1. Trouble accessing the site
        Since a few days ago Firefox won't open the legacy website, shwoing this message:

      legacy_issue.png

      1. Accessing the website with Chromium by adding an exception I found that the strategy charts haven't been updated since 2021-10-26.

      Could you please take a look?
      Thanks!

      posted in Support
      A
      antinomy
    • Erroneous Data?

      Hello @support
      Since the live period for the Q16 contest is coming to an end I'm watching my participating algorithms more closely and noticed something odd:
      The closing prices are the same on 2022-02-24 and 2022-02-25 to the last decimal for allmost all cryptos (49 out of 54).

      import qnt.data as qndata
      
      crypto = qndata.cryptodaily.load_data(tail=10)
      c = crypto.sel(field='close').to_pandas().iloc[-3:]
      liquid = crypto.sel(field='is_liquid').fillna(0).values.astype(bool)[-3:]
      # only showing the cryptos which were liquid for the last 3 days:
      c.iloc[:, liquid.all(axis=0)]
      
      asset 	ADA 	AVAX 	BNB 	BTC 	DOGE 	DOT 	ETH 	LINK 	SOL 	XRP
      time 										
      2022-02-23 	0.8664 	73.47 	365.6 	37264.053 	0.1274 	15.97 	2580.9977 	13.34 	84.64 	0.696515
      2022-02-24 	0.8533 	76.39 	361.2 	38348.744 	0.1242 	16.16 	2598.0195 	13.27 	89.41 	0.696359
      2022-02-25 	0.8533 	76.39 	361.2 	38348.744 	0.1242 	16.16 	2598.0195 	13.27 	89.41 	0.696359
      
      (c.values[-1] == c.values[-2]).sum(), c.shape[1]
      
      (49, 54)
      

      Could you please have a look?
      Thanks!

      posted in Support
      A
      antinomy
    • RE: Weights different in testing and submission

      About the slicing error, I had that too a while ago. It took me some time to figure out that it wasn't enough to have the right pandas version in the environment. Because I had another python install with the same version in my PATH, the qntdev-python also looked there and always used the newer pandas. So I placed the -s flag everywhere the qntdev python is supposed to run (PyCharm, Jupyter, terminal) like this

      /path/to/quantiacs/python -s strategy.py
      

      Of course one could simply remove the other python install from PATH but I needed it there.

      posted in Support
      A
      antinomy
    • RE: Saving and recalling a dictionary of trained models

      @alfredaita
      In case you don't want to run init.py every time in order to install external libraries, I came up with a solution for this. You basically install the library in a folder in your home directory and let the strategy create symlinks to the module path at runtime. More details in this post.

      posted in Support
      A
      antinomy
    • RE: External Libraries

      @support
      Yes, pip is way faster. Thanks!
      I might have found an even faster solution but I guess I have to wait a few hours to find out if it really works.

      Here's what I did:

      1. I created a folder in /root/books called "modules" to install cvxpy there to make it persistent:
      !mkdir modules && pip install --target=modules cvxpy
      
      1. Then if the import fails in the strategy, it creates symbolic links in /usr/local/lib/python3.7/site-packages/ that point to the content of /root/books/modules/
      try:
          import cvxpy as cp
      except ImportError:
          import os
          source = '/root/book/modules/'
          target = '/usr/local/lib/python3.7/site-packages/'
          for dirpath, dirnames, filenames in os.walk(source):
              source_path = dirpath.replace(source, '')
              target_path = os.path.join(target, source_path)
              if not os.path.exists(target_path) and not os.path.islink(target_path):
                  os.symlink(dirpath, target_path)
                  continue
              for file in filenames:
                  source_file = os.path.join(dirpath, file)
                  target_file = os.path.join(target, source_path, file)
                  if not os.path.exists(target_file) and not os.path.islink(target_file):
                      os.symlink(source_file, target_file)
          import cvxpy as cp
      

      Creating the symlinks only takes 0.07 seconds, so fingers crossed 🙂

      UPDATE (a few hours later):
      It actually worked. When I just reopened the strategy, the environment was newly initialized. First I tried just importing cvxpy and got the ModuleNotFoundError. Then I ran the strategy including the code above: cvxpy was imported correctly and the strategy ran.

      I'm not sure if that solution works for every module because I don't know if pip might also write something to other directories than site-packages.

      Anyway, I'm happy with this solution.
      Regards

      posted in Support
      A
      antinomy
    • Documentation
    • About
    • Career
    • My account
    • Privacy policy
    • Terms and Conditions
    • Cookies policy
    Home
    Copyright © 2014 - 2021 Quantiacs LLC.
    Powered by NodeBB | Contributors