Calculation time exceeded on submission

TheFlyingDutchman

@support : My Q21 submission gives a “Calculation time exceeded” error. This revers to: “Timeout, An error message stating that the strategy calculation exceeds a given time implies that you need to optimize the code and reduce the execution time. Futures systems should be evaluated in 10 minutes and Bitcoin futures/Crypto long systems in 5 minutes of time.”

Could you please help? Submission number is # 16381634.

Best Regards.

support

@theflyingdutchman Hi, sorry for delay. That is correct, execution takes too long. Can you try to speed up computation on your side? Are you maybe trying to train a ML model on a daily basis?

TheFlyingDutchman

@support The Ml algo does not train on a daily basis, I also tried to decrease the traning days, but that lowered the SR. Could you look into the process? Maybe the execution time does take longer for certain ML algo's.

Vyacheslav_B

@theflyingdutchman Hello,

Another option is to rewrite your strategy for a single-pass version before submitting it. This approach will significantly speed up the calculations. However, it's important to note that the actual statistical values can only be tracked after submitting the strategy to the competition.

For example:
https://github.com/quantiacs/strategy-ml-crypto-long-short/blob/master/strategy.ipynb

To adapt this strategy for a single-pass version, follow these steps:

Comment out or delete the line where qnbt.backtest_ml is used.
Insert the following code:

import xarray as xr
import qnt.ta as qnta
import qnt.data as qndata
import qnt.output as qnout
import qnt.stats as qnstats

retrain_interval = 3*365 + 1
data = qndata.stocks.load_ndx_data(tail=retrain_interval)

models = train_model(data)
weights = predict(models, data)

In a new cell, insert code to save the weights:

qnout.write(weights)

To view the strategy's statistics, use the following code in a new cell:


# Calculate stats
stats = qnstats.calc_stat(data, weights)
display(stats.to_pandas().tail())

# Graph
performance = stats.to_pandas()["equity"]
import qnt.graph as qngraph
qngraph.make_plot_filled(performance.index, performance, name="PnL (Equity)", type="log")

The qnbt.backtest_ml function is a unique tool for evaluating machine learning strategies, which stands out from what is offered on other platforms. It allows users to set retraining intervals and analyze statistical metrics of the strategy, as opposed to the traditional evaluation of the machine learning model. This provides a deeper understanding of the strategy's effectiveness under various market conditions.