@support @magenta-grimer
I am looking at the futures data and I notice for example the VIX. Of course it's only tradable since 2004 but as a feature for modelling I think it is of interest going back in time.

I personally I guess will just add a pasted-together series of old and new VIX, for my own use, but it does bring up the issue of data that is for modelling versus trading - other users might get benefit from inclusion of this series, and while it is a competition, raising the general level of contributions is rational because the pie gets larger, or maybe doesn't get smaller. Adding research-only data series would start to complicate matters operationally but I think this project is inherently complex.

edit - I should clarify too that spot VIX vs the futures analysis requires the spot data, another reasonable analysis that it might be nice to be able to do in quantiacs strategies.

And to be more clear, sorry, I want the test period to be also in the distant past, not pinned to recent calendar time.

@support @magenta-grimer
I am looking at the futures data and I notice for example the VIX. Of course it's only tradable since 2004 but as a feature for modelling I think it is of interest going back in time.

I personally I guess will just add a pasted-together series of old and new VIX, for my own use, but it does bring up the issue of data that is for modelling versus trading - other users might get benefit from inclusion of this series, and while it is a competition, raising the general level of contributions is rational because the pie gets larger, or maybe doesn't get smaller. Adding research-only data series would start to complicate matters operationally but I think this project is inherently complex.

edit - I should clarify too that spot VIX vs the futures analysis requires the spot data, another reasonable analysis that it might be nice to be able to do in quantiacs strategies.

@support These clarifying details are helpful, and relevant. I started with that minimal example and the api has parameters that are confusing to me, and I only realized that the backtest function is itself performing a loop after going into the code.

I don't have any futures data for my own use anymore so I will just focus on porting some models to this framework locally using your data and see if anything good happens. I have worked at a CTA but on daily data the sharpe ratios out of sample over a ten or 15 year period are lower than 1.0 on average. I have been working more with equities, but it is all an uphill battle with one person and limited time and compute. Models that look good in the short term were never the ones that were best long-term, and this is a bit perverse because a greedy strategy of model selection favors the short-term winners. I am not sure I can come up with futures models that place in the competition and are good to trade long-term. I might be able to come up with some models that are good in an ensemble setting, maybe your scoring knows how to judge models on their marginal contribution in a leave-one-out kind of way? Or maybe you could run gradient boosting on all the user models?

Thanks, I might have more questions soon.

And to be more clear, sorry, I want the test period to be also in the distant past, not pinned to recent calendar time.

Hi,
I am starting from the simple ema futures example, and I am trying to get comfortable with the api. I want to move my training window forward incrementally, and also move my test period forward.

I started by trying to set the min_date and max_date in the load_date function, and setting start_date in the backtest function. This leaves a slightly confusing test_period, which I would prefer to calculate myself by simply specifiying the train and test dates, but I notice that in the backtest code pd.Timestamp.today() is used, which means the backtester is bound to somehow use today's calendar date to anchor the test interval.

Can someone help me paramterize these functions in such a way that I can perform iterative, windowed backtesting? I want to start far in the past to do some pre-training without burning too much data.

@support yes coarse heuristics work well as long as you are conservative. For shorter term models I have started using minute bars despite the computational hit, because it helps in a lot of other ways.

I may enter this contest, I am pretty rusty on predictive modelling and I am not sure I can do a good job using just daily prices, there is not a lot of data. I used to work at a CTA and I feel like we wasted a lot of man years using only prices, hoping better models would acheive more alpha. in the end the sharpe is similar to the S&P but uncorrelated, but you have gotten there with some simpler models and enjoyed life.

I have some other questions about the platform and the contest that I will post here.

Best
P.M.

@penrose-moore Ok well I guess I should have read the rules / user agreement first. A percentage of the average True Range over a certain interval seems to be how slippage is incorporated into the scoring, and the penalty is on profitability directly.

Hi,
I just signed up and I notice that there is no mention made of the holding period for user strategies, there are no statistics except for turnover which relate to issues pertaining to scalability or indeed the baseline viability of strategies due to various trading costs. In my previous work we imposed a lot of penalties that were for the most part gleaned from lots of execution data from futures orders we sent to the exchanges over the years.

So while it might be prohibitive in terms of computational cost to micro-simulate each fill for each order, we developed heuristics in an attempt to limit the trading costs by soft penalties. So with daily prices it is really hard to get a realistic simulation of fill prices, you can create a scoring function that penalizes strategies with a lot of turnover per unit of time, etc.

I am curious if holding period ever explicitly enters into the testing procedure? Are the algorithms that place well in competitions subjected to more fine-grained execution testing using intra-day data? I know that if I naively try to just trade in and out every day, my PNL will be very sensitive to the volatility of the instrument, tick size, spread, and entry and exit conditions, and at that time-frame when and how you choose to execute has a non-trivial relationship to PNL.

thanks