Q20 contest results
-
@dark-pidgeot The correct way to load stocks, free from survivorship bias, is:
data = qndata.stocks.load_ndx_data(min_date="2005-06-01")
Later you can select index members dynamically as:
is_liquid = data.sel(field="is_liquid")
weights = weights * is_liquidassuming that you computed your weights.
Survivorship bias follows from loading a specific hand-picked set of stocks in the load function. Or selecting specific stocks later by indicating explicitely their name.
-
@support Am I disqualified if I do this?
data = qndata.stocks.load_ndx_data(min_date="2005-06-01", assets=["NAS:AAPL", "NAS:AMZN", "NAS:MRNA", "NAS:MSFT"])
I select a few stocks, it's not a dynamic selection
-
@support I don't really understand, if I have a strategy that works on 10 stocks for example, to be eligible for the price, I absolutely have to trade all the stocks?
-
@dark-pidgeot I would think it would be fair if the criteria were that the strategy added value somewhere. In the period from January 2005 to now, additional value should have been created compared to a long-only investment in those four shares.
-
@dark-pidgeot No, you can trade 10 stocks, you do not have to trade all of them. For example you can load all the stocks, then select dynamically the liquid ones, then sort them according to some criterion (example: trade at each point in time only the top 10 in terms of inverse volatility) and that is fine.
But if you choose by hand 10 stocks, then you incur into survivorship bias, as you are selecting 10 stocks which are alive (and most likely successful if you go long only) now, and are using future information (now) to make your selection (hand-picking these 10 stocks at the beginning of the simulation).
-
@theflyingdutchman That is also a good observation. In general, we would like a long-only system with low turnover to beat the Nasdaq index for example. But still, we see a lot of problems with hand-picking stocks and survivorship-bias affected systems.
-
@support I always ask myself the question, when we look at the winners of other competitions, there are some who trade just 5 stocks. can we consider this to be hand-picking?
-
@support said in Q20 contest results:
That is also a good observation. In general, we would like a long-only system with low turnover to beat the Nasdaq index for example. But still, we see a lot of problems with hand-picking stocks and survivorship-bias affected systems.
if I understand correctly
if I initially select 10 stocks, and even if I have a good sharpe ratio, my strategy can make long and short positions. I could be disqualified
-
@theflyingdutchman it seemed to me that strategies should be able to open long and short positions
-
@dark-pidgeot said in Q20 contest results:
@support said in Q20 contest results:
That is also a good observation. In general, we would like a long-only system with low turnover to beat the Nasdaq index for example. But still, we see a lot of problems with hand-picking stocks and survivorship-bias affected systems.
if I understand correctly
if I initially select 10 stocks, and even if I have a good sharpe ratio, my strategy can make long and short positions. I could be disqualified
That's my understanding too. If you initially select 10 stocks, that already implies survivorship-bias, and is in fact "forward looking" because in for instance year 2010 you could not foresee which 10 stocks would still exist in the index in 2024.
-
@buyers_are_back Yes, that is the forward looking associated to survivorship bias. In order to avoid that, the dataset includes also stocks which were part of the index but have been delisted at some point. These stocks should be taken potentially in consideration by any algorithm, which does not happen by hand-picking stocks.
One can reverse the logic and conclude the same for stocks which "should" have been shorted in the pasted because of observing an a posteriori delisting.
-
@dark-pidgeot that is fine
-
@dark-pidgeot yes, ybut ou can start from the full index, and selec dynamically 10 according to some criteria, and then take long and short positions.
-
@dark-pidgeot It is possible unfortunately, and we are trying to improve the way we detect these potential cases.
-
@support Hi,
I would like to know how the classification is done in context. you only look at the shape ratio over the out-o-sample period? -
@dark-pidgeot sorry for the delay, yes (see answer in other thread)
-
@support: I was wondering if the evaluation period for the Q20 contest has been completed? Has my strategy ML RB 1.2 (#14410209) been approved as a prize winner for Q20? I currently lack some insight into the status of the older strategies, most are stil running, some have been suspended.
-
@theflyingdutchman Hi, the Q20 had very few participants, less than 30, and no strategies are being traded at the moment according to the contest rules. Indeed we made no official announcement.
However, the participants had the opportunity to submit the same code (provided it is correct) to the Q21 (in other words no correlations check are performed).
It is very cumbersome to remove strategies from the web page, so they are currently displayed.
If your strategy will continue to perform good and we believe it can be traded, we will contact you.
