Additional Data for Bitcoin

[antinomy]

Hello,

in addition to the usual price data there are many informations about Bitcoin (and other cryptos) available. I would find it great if you could add historical data for hashrate, difficulty, market cap and so on.
As an example here is a free dataset on Quandl I'm currently experimenting with.

Could you provide a dataset like that on the platform?

Regards

[support]

@antinomy Hello, thanks for the input, yes, we will add these data to our lists of tasks and notify you as soon as they are implemented

[antinomy]

@support
Cool, thanks!

[antinomy]

@support I see you added data from blockchain.com. That's great, thanks!

For anyone who wants to check it out

import qnt.data as qndata
import pandas as pd

# to get an overview of the datasets:
bchain_datasets = qndata.blockchaincom_load_list()
pd.DataFrame(bchain_datasets).set_index('id')

# access data, e.g. hash rate (use the 'id' field for the request):
hash_rate = qndata.blockchaincom_load_data('hash-rate', tail=365)

(If you develop locally make sure you have the latest qnt library, as of now it would be
conda install quantiacs-source::qnt=0.0.262)

[support]

@antinomy Thanks, they are going to be part of the new contest.

[antinomy]

@support I'm having trouble using blockchain data with the backtester.
An example code:

import qnt.backtester as qnbt
import qnt.data as qndata
import numpy as np
import xarray as xr


def load_data(period):
    crypto = qndata.cryptofutures.load_data(tail=period)
    hr = qndata.blockchaincom_load_data('hash-rate', tail=period)

    return {
        'crypto': crypto,
        'hr': hr
    }, crypto.time.values


def build_data_for_one_step(data, max_date: np.datetime64, lookback_period: int):
    min_date = max_date - np.timedelta64(lookback_period, "D")
    return {
        "crypto": data["crypto"].sel(time=slice(min_date, max_date)),
        "hr": data["hr"].sel(time=slice(min_date, max_date)),
    }

def strategy(data):
    return xr.ones_like(data['crypto'].sel(field='close').isel(time=-1))

weights = qnbt.backtest(
    strategy=strategy,
    competition_type="cryptofutures",
    load_data=load_data,
    lookback_period=500,
    start_date='2014-01-01',
    window=build_data_for_one_step,
)

This raises a HTTP error 500 when the backtester tries to run the first pass:

[...]

Run first pass...
Load data...

100% (17072 of 17072) |##################| Elapsed Time: 0:00:00 Time: 0:00:00
ERROR:root:download error: blockchain.com/data?id=hash-rate
Traceback (most recent call last):
File "/root/book/qnt/data/common.py", line 79, in request_with_retry
with urllib.request.urlopen(req, timeout=TIMEOUT) as response:
File "/usr/local/lib/python3.7/urllib/request.py", line 222, in urlopen
return opener.open(url, data, timeout)
File "/usr/local/lib/python3.7/urllib/request.py", line 531, in open
response = meth(req, response)
File "/usr/local/lib/python3.7/urllib/request.py", line 641, in http_response
'http', request, response, code, msg, hdrs)
File "/usr/local/lib/python3.7/urllib/request.py", line 569, in error
return self._call_chain(*args)
File "/usr/local/lib/python3.7/urllib/request.py", line 503, in _call_chain
result = func(*args)
File "/usr/local/lib/python3.7/urllib/request.py", line 649, in http_error_default
raise HTTPError(req.full_url, code, msg, hdrs, fp)
urllib.error.HTTPError: HTTP Error 500: Internal Server Error
ERROR:root:download error: blockchain.com/data?id=hash-rate

[antinomy]

@support It's working now, thanks!