Posts made by antinomy

And what if a strategy uses the following rules to select assets to trade:

• the primary exchange is NAS
• the sector is not finance
• has price data for at least the previous 3 months
• has an average daily trading volume of at least 200 k based on the previous 3 months
• belongs to the top 100 of the thus far selected assets in terms of market capitalization

Would you say any of these rules violate the contest rules?

Because these are the selection criteria for the N100 constituents. The only difference in my strategy is that I'm using qnt.data.stocks_load_ndx_data.sel(field='is_liquid') instead.
Sure, the first of the rules above manually selects the exchange and the second one manually excludes a sector. But still none of these manually select assets and neither does the filter is_liquid from another dataset.

Also, lets take a look why you prohibited manual asset selection in the first place. Wasn't this to avoid lookahead bias? And isn't this also the reason for the existence of the field is_liquid in any of your datasets? Are you saying that the exact field you introduced to avoid lookahead bias is now the reason you disqualify a strategy because of lookahead bias just because it's from a dataset other than the one for the contest?

@support
Thanks for your reply.

My strategy trades a subset of assets from the S&P 500 universe, selected with certain filter rules. Most are technical and one of them is that they also belong to the Nasdaq 100 index.
Which contest rule would be violated when a strategy selects stocks which are part of the contest universe and also part of another index?

Is selecting assets based on them being part of an index considered as "manual selection"? This would mean every strategy for all the Nasdaq 100 and S&P 500 contests is against the rules and the contest rules would contradict themselves.

Out of curiosity I just cloned the strategy to run it again and see if I get any correlations. And yes:

WARNING! This strategy correlates with other strategies and will be rejected.
Modify the strategy to produce the different output.

The number of systems with a larger Sharpe ratio and correlation larger than 0.9: 2
The max correlation value (with systems with a larger Sharpe ratio): 0.9818594562862993

Then I wanted to know to which strategies it correlates and simply put a print(json.dumps(cr_list, indent=4)) inside qnt.stats.check_correlation and got this:

[
    {
        "id": 18701501,
        "name": null,
        "my": false,
        "template": false,
        "cofactor": 0.9818594562862993,
        "sharpe_ratio": 2.364183164325061
    },
    {
        "id": 18701502,
        "name": null,
        "my": false,
        "template": false,
        "cofactor": 0.9559821139736059,
        "sharpe_ratio": 2.49988419656187
    }
]

The first one is my q23_4 and the other one is my q23_4a (funny that it correlates with itself by 0.98 and not 1.0). However, no correlation with any template is showing up there.

Hi @support

My strategy q23_4 would have made the 3rd place in the Q23 contest but now instead of the allocation there is a red dot whose mouseover reads "The strategy uses manual asset selection or has very high correlation with standard templates; therefore, it has been rejected."

There is definitely no manual selection in any of my strategies and the ones that took part in the contest had to pass the correlation check which also checks against templates.

Could you please tell me what's going on there?

Hi @support ,
sorry to bother you again but I have a new issue with my submissions.
On 2025-07-14 I submitted 3 strategies and they have been stuck at 99 % for 3 days now.
Yesterday I submitted another one and it's also stuck at 99 %
Could you please have a look?
One odd thing: meanwhile the ones from 2025-07-14 have processed the trading day for 2025-07-15 - I hope they don't wait for new data every day

@support All is looking good now, thanks!

Hi @support ,
unfortunately the strategy is still not eligible for the contest, now it says "Competitions for the specified type are not running" and it's not selected to run in the contest.

Just out of curiousity I cloned the strategy and submitted the clone, the result is the same as the original issue:

(the cloned stragegy can be deleted, I will eventually do so before the deadline)

Can you share any insights as to why this happens at all?
Thanks.

Hi @support,
When I run my strategy with qnt.backtester.backtest setting check_correlation=True it passes the correlation test.

When I submitted it to the contest it got rejected and in the candidates tab it has a message about high correlation:

Looking at the short description in the strategy history it passes all tests on every trading day except for the last one (2025-06-24):

I don't quite understand why it passes the correlation check in the backtester but fails it after submission on the last day.
Which correlation test is wrong, the one from the backtester or the one running after submission?
If it helps, the strategy in question is a q 19 submission by another user and when I look at it there are only 87 data points.
My strategy is 18633730 and the one it has or has not a high correlation with is 11938152.

@vyacheslav_b Hi, I agree that diversification is always a good idea for trading. It might be helpful if there was an additional function like check_diversification whith a parameter for the minimum number of assets you want to trade. But this function should only warn you and not fix an undiversified portfolio, because the only way would be to add more assets to trade and asset selection should be done by the strategy itself in my opinion.

@support Hi, I just checked out qnt 0.0.504 and the problem I mentioned seems to be fixed now, thanks!
Would you perhaps consider to add a leverage check to the check function?
Because one might think "qnout.check says everything is OK, so I have a valid portfolio" while actually having vastly overleveraged like in my 2nd example where weights.sum('asset').values.max() is 505.0.
Adding something like this to check would tell us about it:

log_info("Check max portfolio leverage...")
max_leverage = abs(output).sum(ds.ASSET).values.max()
if max_leverage > 1 + 1e-13: # (give some leeway for rounding errors and such)
    log_err("ERROR! The max portfolio leverage is too high.")
    log_err(f"Max leverage: {max_leverage} Limit: 1.0")
    log_err("Use qnt.output.clean() or normalize() to fix.")
else:
    log_info("Ok.")

@support,
Either something about the exposure calculation is wrong or I really need clarification on the rules. About the position limit I only find rule 7. o. in the contest rules which states "The evaluation system limits the maximum position size for a single financial instrument to 10%"

I always assumed this would mean the maximum weight for an asset would be 0.1 meaning 10 % of the portfolio. However the exposure calculation suggests the following:
Either we trade no asset or at least 10 assets per trading day, regardless of the actual weights assigned to each asset.

Consider this example:

import qnt.data as qndata
import qnt.output as qnout
from qnt.filter import filter_sharpe_ratio


data = qndata.stocks.load_spx_data(min_date="2005-01-01")

weights = data.sel(field='is_liquid').fillna(0)
weights *= filter_sharpe_ratio(data, weights, 3) * .01 # assign 1 % to each asset using the top 3 assets by sharpe

qnout.check(weights, data, "stocks_s&p500")

which results in an exposure error:

Check max exposure for index stocks (nasdaq100, s&p500)…
ERROR! The max exposure is too high.
Max exposure: [0.         0.         0.         ... 0.33333333 0.33333333 0.33333333] Hard limit: 0.1
Use qnt.output.cut_big_positions() or normalize_by_max_exposure() to fix.

even though the maximum weight per asset is only 0.01

abs(weights).values.max()
0.01

(By the way, the 4 functions mentioned by @Vyacheslav_B also result in weights which dont't pass the exposure check when used with this example, except drop_bad_days which results in empty weights.)

And if we assign 100 % to every liquid asset, the exposure check passes:

weights = data.sel(field='is_liquid').fillna(0)
qnout.check(weights, data, "stocks_s&p500")

Check max exposure for index stocks (nasdaq100, s&p500)…
Ok.

So, does rule 7. o. mean we have to trade at least 10 assets or none at all each trading day to satisfy the exposure check?

The symbols are all in there, but if they are listed on NYSE you have to prepend NYS: not NAS: to the symbol. Also, I believe by 'BKR.B' you mean 'BRK.B'

[sym for sym in data.asset.values if any(map(lambda x: x in sym, ['JPM', 'LLY', 'BRK.B']))]

['NYS:BRK.B', 'NYS:JPM', 'NYS:LLY']

You can also search for symbols in qndata.stocks_load_spx_list() and get a little more infos like this:

syms = qndata.stocks_load_spx_list()
[sym for sym in syms if sym['symbol'] in ['JPM', 'LLY', 'BRK.B']]

[{'name': 'Berkshire Hathaway Inc',
  'sector': 'Finance',
  'symbol': 'BRK.B',
  'exchange': 'NYS',
  'id': 'NYS:BRK.B',
  'cik': '1067983',
  'FIGI': 'tts-824192'},
 {'name': 'JP Morgan Chase and Co',
  'sector': 'Finance',
  'symbol': 'JPM',
  'exchange': 'NYS',
  'id': 'NYS:JPM',
  'cik': '19617',
  'FIGI': 'tts-825840'},
 {'name': 'Eli Lilly and Co',
  'sector': 'Healthcare',
  'symbol': 'LLY',
  'exchange': 'NYS',
  'id': 'NYS:LLY',
  'cik': '59478',
  'FIGI': 'tts-820450'}]

The value for the key 'id' is what you will find in data.asset

I got the same error after installing qnt locally with pip.
There is indeed a circular import in the current Github repo for the toolbox, introduced by this commit:
https://github.com/quantiacs/toolbox/commit/78beafa93775f33606156169b3e6b8f995804151#diff-89350fe373763b439e4697f9b11cceb811b4a3f0adc7a655707a936ce5646c01R6-R10
when some of the imports in output.py which were inside of fuctions before were moved to the top level.
Now output imports from stats and stats imports from output.

@support Can you please have a look?

@alexeigor @omohyoid
The conda version of qnt doesn't seem to be affected, so if that's an option for you install that one instead.
Otherwise we can use the git version previous to the commit above:

pip uninstall qnt
pip install git+https://github.com/quantiacs/toolbox.git@a1e6351446cd936532af185fb519ef92f5b1ac6d

@steel-camel

!pip install --force-reinstall python_utils

should fix the issue.
But I have no idea what would have caused it, the line in converters.py is totally messed up. The only thing that comes to my mind is a cat on the keyboard

@cyan-gloom
interpolate_na() only eliminates NaNs between 2 valid data points. Take a look at this example:

import qnt.data as qndata
import numpy as np

stocks = qndata.stocks_load_ndx_data()
sample = stocks[:, -5:, -6:] # The latest 5 dates for the last 6 assets

print(sample.sel(field='close').to_pandas())
"""
asset       NYS:NCLH  NYS:ORCL  NYS:PRGO  NYS:QGEN  NYS:RHT  NYS:TEVA
time                                                                 
2023-05-12     13.24     97.85     35.21     45.09      NaN      8.03
2023-05-15     13.71     97.26     34.23     45.36      NaN      8.07
2023-05-16     13.48     98.25     32.84     45.25      NaN      8.13
2023-05-17     14.35     99.77     32.86     44.95      NaN      8.13
2023-05-18     14.53    102.34     33.43     44.92      NaN      8.26
"""

# Let's add some more NaN values:
sample.values[3, (1,3), 0] = np.nan
sample.values[3, 1:4, 1] = np.nan
sample.values[3, :2, 2] = np.nan
sample.values[3, 2:, 3] = np.nan
sample.values[3, :-1, 5] = np.nan
print(sample.sel(field='close').to_pandas())
"""
asset       NYS:NCLH  NYS:ORCL  NYS:PRGO  NYS:QGEN  NYS:RHT  NYS:TEVA
time                                                                 
2023-05-12     13.24     97.85       NaN     45.09      NaN       NaN
2023-05-15       NaN       NaN       NaN     45.36      NaN       NaN
2023-05-16     13.48       NaN     32.84       NaN      NaN       NaN
2023-05-17       NaN       NaN     32.86       NaN      NaN       NaN
2023-05-18     14.53    102.34     33.43       NaN      NaN      8.26
"""

# Interpolate the NaN values:
print(sample.interpolate_na('time').sel(field='close').to_pandas())
"""
asset       NYS:NCLH    NYS:ORCL  NYS:PRGO  NYS:QGEN  NYS:RHT  NYS:TEVA
time                                                                   
2023-05-12    13.240   97.850000       NaN     45.09      NaN       NaN
2023-05-15    13.420  100.095000       NaN     45.36      NaN       NaN
2023-05-16    13.480  100.843333     32.84       NaN      NaN       NaN
2023-05-17    14.005  101.591667     32.86       NaN      NaN       NaN
2023-05-18    14.530  102.340000     33.43       NaN      NaN      8.26
"""

As you can see, only the NaNs in the first 2 columns are being replaced. The others remain untouched and might be dropped when you use dropna().

Another thing you should keep in mind is that you might introduce lookahead bias with interpoloation, e. g. in a single run backtest. In my example for instance (pretend the NaNs I added were already in the data) you would know on 2023-05-15 that ORCL will rise when in reality you would first know that on 2023-05-18.

Asuming whatever train is has a similar structure as the usual stock data, I get the same error as you with:

import itertools
import qnt.data as qndata

stocks = qndata.stocks_load_ndx_data(tail=100)

for comb in itertools.combinations(stocks.asset, 2):
    print(stocks.sel(asset=[comb]))

There are 2 things to consider:

1. comb is a tuple and you can't use tuples as value for the asset argument. You are putting brackets around it, but that gives you a list with one element wich is a tuple, hence the error about setting an array element as a sequence. Using stocks.sel(asset=list(comb)) instead resolves this issue but then you'll get an index error which leads to the second point
2. each element in comb is a DataArray and cannot be used as an index element to select from the data. You want the string values instead, for this you can iterate over asset.values for instance.

My example works when the loop looks like this:

for comb in itertools.combinations(stocks.asset.values, 2):
    print(stocks.sel(asset=list(comb)))

I'm a huge fan of Sentdex, he really tought me a lot about Python in his tutorials.
Have a look at his website and his Youtube channel, for instance there's a tutorial for Python beginners.

It's safe to ignore these notices but if they bother you, you can set the variables together with your API key using the defaults and the messages go away:

import os

os.environ['API_KEY'] = 'YOUR-API-KEY'
os.environ['DATA_BASE_URL'] = 'https://data-api.quantiacs.io/'
os.environ['CACHE_RETENTION'] = '7'
os.environ['CACHE_DIR'] = 'data-cache'

Hello @support
Could you please add CIKs to the NASDAQ100 stock list?
In order to load fundamental data from secgov we need the CIKs for the stocks but they're currently not in the list we get from qnt.data.stocks_load_ndx_list().
Allthough it is still possible to get fundamentals using qnt.data.stocks_load_list(), it takes a little bit acrobatics like this for instance:

import pandas as pd
import qnt.data as qndata


stocks = qndata.stocks_load_ndx_data()
df_ndx = pd.DataFrame(qndata.stocks_load_ndx_list()).set_index('symbol')
df_all = pd.DataFrame(qndata.stocks_load_list()).set_index('symbol')
idx = sorted(set(df_ndx.index) & set(df_all.index))
df = df_ndx.loc[idx]
df['cik'] = df_all.cik[idx]
symbols = list(df.reset_index().T.to_dict().values())
fundamentals = qndata.secgov_load_indicators(symbols, stocks.time)

It would be nice if we could get them with just 2 lines like so:

stocks = qndata.stocks_load_ndx_data()
fundamentals = qndata.secgov_load_indicators(qndata.stocks_load_ndx_list(), stocks.time)

Also, the workaround doesn't work locally because qndata.stocks_load_list() seems to return the same list as qndata.stocks_load_ndx_list().

Thanks in advance!

@eddiee Try step 4 without quotes, this should start jupyter notebook. And if that's your real API-key we see in the image, delete your last post. It's a bad idea to post it in a public forum

Yes, I noticed that too. And after fixing it the backtest takes forever...
Another thing to consider is that it redefines the model with each training but I belive you can retrain already trainded NNs with new Data so they learn based on what they previously learned.