Backesting a trading system amounts to perform a simulation of the trading rules on historical data. All trading rules depend to some extent on a set of parameters. These parameters can be the lookback periods used for defining technical indicators or the hyperparameters of a complex machine learning model.
It is very important to study the parameter dependence of the key statistical indicators, for example the Sharpe ratio. A parameter choice which maximizes the value of the Sharpe ratio when the simulation is performed on the past data is a source of backtest overfitting and leads to poor performance on live data.
In this template we provide a tool for studying the parameter dependence of the statistical indicators used for assessing the quality of a trading system.
We recommend optimizing your strategy in a separate notebook because a parametric scan is a time consuming task.
Alternatively it is possible to mark the cells which perform scans using the #DEBUG#
tag. When you submit your notebook, the backtesting engine which performs the evaluation on the Quantiacs server will skip these cells.
You can use the optimizer also in your local environment on your machine. Here you can use more workers and take advantage of parallelization to speed up the grid scan process.
%%javascript
IPython.OutputArea.prototype._should_scroll = function(lines) { return false; }
// disable widget scrolling
import qnt.data as qndata
import qnt.ta as qnta
import qnt.output as qnout
import qnt.stats as qns
import qnt.log as qnlog
import qnt.optimizer as qnop
import qnt.backtester as qnbt
import xarray as xr
For defining the strategy we use a single-pass implementation where all data are accessed at once. This implementation is very fast and will speed up the parametric scan.
You should make sure that your strategy is not implicitly forward looking before submission, see how to prevent forward looking.
The strategy is going long only when the rate of change in the last roc_period
trading days (in this case 10) of the linear-weighted moving average over the last wma_period
trading days (in this case 20) is positive.
def single_pass_strategy(data, wma_period=20, roc_period=10):
wma = qnta.lwma(data.sel(field='close'), wma_period)
sroc = qnta.roc(wma, roc_period)
weights = xr.where(sroc > 0, 1, 0)
weights = weights / len(data.asset) # normalize weights so that sum=1, fully invested
with qnlog.Settings(info=False, err=False): # suppress log messages
weights = qnout.clean(weights, data) # check for problems
return weights
Let us first check the performance of the strategy with the chosen parameters:
#DEBUG#
# evaluator will remove all cells with this tag before evaluation
data = qndata.futures.load_data(min_date='2004-01-01') # indicators need warmup, so prepend data
single_pass_output = single_pass_strategy(data)
single_pass_stat = qns.calc_stat(data, single_pass_output.sel(time=slice('2006-01-01', None)))
display(single_pass_stat.to_pandas().tail())
A parametric scan over pre-defined ranges of wma_period
and roc_period
can be performed with the Quantiacs optimizer function:
#DEBUG#
# evaluator will remove all cells with this tag before evaluation
data = qndata.futures.load_data(min_date='2004-01-01') # indicators need warmup, so prepend data
result = qnop.optimize_strategy(
data,
single_pass_strategy,
qnop.full_range_args_generator(
wma_period=range(10, 150, 5), # min, max, step
roc_period=range(5, 100, 5) # min, max, step
),
workers=1 # you can set more workers when you run this code on your local PC to speed it up
)
qnop.build_plot(result) # interactive chart in the notebook
print("---")
print("Best iteration:")
display(result['best_iteration']) # as a reference, display the iteration with the highest Sharpe ratio
The arguments for the iteration with the highest Sharpe ratio can be later defined manually or calling result['best_iteration']['args']
for the final strategy. Note that cells with the tag #DEBUG#
are disabled.
The final multi-pass call backtest for the optimized strategy is very simple, and it amounts to calling the last iteration of the single-pass implementation with the desired parameters:
best_args = dict(wma_period=20, roc_period=80) # highest Sharpe ratio iteration (not recommended, overfitting!)
def best_strategy(data):
return single_pass_strategy(data, **best_args).isel(time=-1)
weights = qnbt.backtest(
competition_type="futures",
lookback_period=2 * 365,
start_date='2006-01-01',
strategy=best_strategy,
analyze=True,
build_plots=True
)