@cespadilla Hello.

The reason is in "train_model" function.

def train_model(data): asset_name_all = data.coords['asset'].values features_all = get_features(data) target_all = get_target_classes(data) models = dict() for asset_name in asset_name_all: # drop missing values: target_cur = target_all.sel(asset=asset_name).dropna('time', 'any') features_cur = features_all.sel(asset=asset_name).dropna('time', 'any') target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner') if len(features_cur.time) < 10: continue model = get_model() try: model.fit(feature_for_learn_df.values, target_for_learn_df) models[asset_name] = model except: logging.exception('model training failed') return models

If there are less than 10 features for training the model, then the model is not created (if len(features_cur.time) < 10).

This condition makes sense. I would not remove it.

The second thing that can affect is the retraining interval of the model ("retrain_interval").

weights = qnbt.backtest_ml( train=train_model, predict=predict_weights, train_period=2 *365, # the data length for training in calendar days retrain_interval=10 *365, # how often we have to retrain models (calendar days) retrain_interval_after_submit=1, # how often retrain models after submission during evaluation (calendar days) predict_each_day=False, # Is it necessary to call prediction for every day during backtesting? # Set it to true if you suspect that get_features is looking forward. competition_type='crypto_daily_long_short', # competition type lookback_period=365, # how many calendar days are needed by the predict function to generate the output start_date='2014-01-01', # backtest start date analyze = True, build_plots=True # do you need the chart? )

@xiaolan Yes, allocations are translate to orders internally, it is enough to check the variation in the allocations and transform it into number of contracts bought/sold. When we designed the toolbox the goal was to simplify development as much as possible for the users.

@cross_platform-zebra Hi, there is no other limitation regarding local development. It is already configured to be exactly the same datasets for Nasdaq100 stocks, and returns the same statistics for trading system running locally or online.

@sheikh Hi, when it comes to stocks and historical simulations, the biggest issue is dealing with survivorship bias. The stock universe must include also stocks which have been delisted and we need to define trading rules which allow for trading instruments which make sense at each point in time. This week we are announing a new contest which is preparing the ground for stocks.

@eddiee Hi, Mr. Eddie.

I am new to building strategies using ML/DL on Quantiacs and am very impressed with the OS performance of your ML strategies. I hope you can give me your contact (mail, limkedin,...) so I can learn from your experience in building an ML/DL strategy.

Sincerely thank.

@support thanks!!!

There is a way to use the optimizer with a (stateful) mulit pass algo, but depending on the total number of changed parameters it can take a very long time. However, if it runs on a local computer with many workers this can still be useful.

We could run the backtester with the multi pass algo to get all the weights for the test period and pass these weights to the optimizer.
There's just one problem with this: you can't pass changed parameters to the strategy using the backtester.
In order to solve this I created a nested function where the outer function takes the changed parameters from the optimizer. The inner function is the actual multi pass strategy and doesn't define the params but just uses the ones from the outer function. Still within the outer function we run the backtester with one set of params, get the weights it returns and return them to the optimizer.

The time it takes to run the optimization would roughly be
(time for 1 multi pass backtest) x (total number of parameter changes) / (number of workers that are able to run)
So if one multi pass takes 1 minute, you want to optimize 10 parameter changes and can run 5 workers it would take about 2 minutes.

Here's an example based on the one above with 2 parameter changes and 2 workers:

import qnt.data as qndata import qnt.ta as qnta import qnt.optimizer as qnop import qnt.backtester as qnbt import xarray as xr def load_data(period): """Loads the BTC Futures data for the BTC Futures contest""" return qndata.cryptofutures.load_data(tail=period, dims=("time", "field", "asset")) def multi_pass_strategy(data, ma_slow_param=50, ma_fast_param=10): """The outer function gets called by the optimizer with changed params, the inner function gets passed to the backtester.""" def strategy(data, state): # The state isn't used in this example, this is just to show that it can be used while optimizing. if state is None: state = 0 state += 1 close = data.sel(field="close") ma_slow = qnta.lwma(close, ma_slow_param).isel(time=-1) ma_fast = qnta.lwma(close, ma_fast_param).isel(time=-1) weights = xr.zeros_like(close.isel(time=-1)) weights[:] = 1 if ma_fast > ma_slow else -1 return weights, state """The backtester returns all weights for the test period which will then be returned to the optimizer""" weights, state = qnbt.backtest( strategy=strategy, competition_type="cryptofutures", load_data=load_data, lookback_period=700, start_date='2014-01-01', build_plots=False, ) return weights data = qndata.cryptofutures.load_data(min_date='2014-01-01') result = qnop.optimize_strategy( data, multi_pass_strategy, qnop.full_range_args_generator( ma_slow_param=range(50, 60, 5), # min, max, step # ma_fast_param=range(5, 100, 5) # min, max, step ), workers=2 # you can set more workers on your PC ) print("---") print("Best iteration:") print(result['best_iteration']) qnop.build_plot(result)

There might be more efficient ways to do this, so if anyone has one feel free to post it here.

@support
Cool, thanks very much! 👍

Dear @quani42,

Your submissions are in the queue and will be processed. Also, all submissions that are sent to the contest before the deadline will be eligible to take part in it.

Regards

@support Thank you!

@anthony_m we patched with spot BTC data see answer: https://quantiacs.com/community/topic/6/btc-contest-start-date

@rezhak21 Rules are defined at: https://quantiacs.com/contest and more details for the current contests (submission time till end of May) can be found at: https://quantiacs.com/contest/15

For Futures the in sample period starts on January 1st 2006, for the BTC Futures on January 1st, 2014

@support Thanks for ur help

@theflyingdutchman Hello, before the end of the week the update will be ready, sorry for the delay

@theflyingdutchman Hello,

Another option is to rewrite your strategy for a single-pass version before submitting it. This approach will significantly speed up the calculations. However, it's important to note that the actual statistical values can only be tracked after submitting the strategy to the competition.

For example:
https://github.com/quantiacs/strategy-ml-crypto-long-short/blob/master/strategy.ipynb

To adapt this strategy for a single-pass version, follow these steps:

Comment out or delete the line where qnbt.backtest_ml is used. Insert the following code: import xarray as xr import qnt.ta as qnta import qnt.data as qndata import qnt.output as qnout import qnt.stats as qnstats retrain_interval = 3*365 + 1 data = qndata.stocks.load_ndx_data(tail=retrain_interval) models = train_model(data) weights = predict(models, data) In a new cell, insert code to save the weights: qnout.write(weights)

To view the strategy's statistics, use the following code in a new cell:

# Calculate stats stats = qnstats.calc_stat(data, weights) display(stats.to_pandas().tail()) # Graph performance = stats.to_pandas()["equity"] import qnt.graph as qngraph qngraph.make_plot_filled(performance.index, performance, name="PnL (Equity)", type="log")

The qnbt.backtest_ml function is a unique tool for evaluating machine learning strategies, which stands out from what is offered on other platforms. It allows users to set retraining intervals and analyze statistical metrics of the strategy, as opposed to the traditional evaluation of the machine learning model. This provides a deeper understanding of the strategy's effectiveness under various market conditions.

It's safe to ignore these notices but if they bother you, you can set the variables together with your API key using the defaults and the messages go away:

import os os.environ['API_KEY'] = 'YOUR-API-KEY' os.environ['DATA_BASE_URL'] = 'https://data-api.quantiacs.io/' os.environ['CACHE_RETENTION'] = '7' os.environ['CACHE_DIR'] = 'data-cache'

@vyacheslav_b Apologies for the late response. Thanks for the assistance, all is now well. Cheers

@support Got it, thanks a lot!

@captain-nidoran hi, all your strategies which will take part to the contest should be under the "In Contest" tab in the "Competition" section.

The migration "Candidates" -> "In Contest" was not immediate as we released minor improvements to the front-end side once the submission phase was over.

@captain-nidoran Fixed, sorry for issue