Three questions about the qnbt.backtest_ml used in Machine Learning on a Rolling Basis example.

gjhernandezp

The questions are about the qnbt.backtest_ml used in Machine Learning on a Rolling Basis example from https://quantiacs.com/documentation/en/examples/q18_machine_learning_on_a_rolling_basis.html

weights = qnbt.backtest_ml(
train = train_model,
predict = predict_weights,
train_period = 2 *365, # the data length for training in calendar days
retrain_interval = 10 *365, # how often we have to retrain models (calendar days)
retrain_interval_after_submit = 1, # how often retrain models after submission during evaluation (calendar days)
predict_each_day = False, # Is it necessary to call prediction for every day during backtesting?
# Set it to True if you suspect that get_features is looking forward.
competition_type = "stocks_nasdaq100", # competition type
lookback_period = 365, # how many calendar days are needed by the predict function to generate the output
start_date = "2005-01-01", # backtest start date
analyze = True,
build_plots = True # do you need the chart?
)

Questions

It seems that the models are trained in the train_period of the final data and that trained models are used in all the chunks of data, is that correct ?
Is the re-training after every retrain_interval done with incremental learning of the type that the fitting that partial_fit does (see following link) ?

https://datascience.stackexchange.com/questions/68599/incremental-learning-with-sklearn-warm-start-partial-fit-fit?

How can i provide qnbt.backtest_ml with and already trained model have the model trained with incremental learning every retrain_interval (would train_period = 0 and have train_model retruning the trained models work for this)

support

@gjhernandezp Dear gjhernandezp,

No, models are trained from scratch taking the train_period of the data that is changing depending on the processing day.
No, the retraining is done from scratch, taking the whole available data up until that point in time.
That is not available at the moment, but take a note that when submitting a ML strategy it should be in a single-pass mode. For example instead of using backtest_ml function that is designed for testing purposes, one should use single-pass, for example:

data_train = load_data(train_period)
models = train_model(data_train)

data_predict = load_data(lookback_period)
weights_predict = predict(models, data_predict)

print_stats(data_predict, weights_predict)
qnout.write(weights_predict) # To participate in the competition, save this code in a separate cell.