Fundamental data incomplete?

buyers_are_back

Hi,

could you please check the fundamental data? I want to access shares but for some stocks they are nan or 0. I just list the result of the last day (liquidity is checked). There are probably more in the data before.

Thanks!

import qnt.data.secgov_fundamental as fundamental
indicators_data = fundamental.load_indicators_for(market_data, indicator_names=['shares'])

for symbol in symbols:
    if market_data.sel(asset=symbol, field='is_liquid').values[-1] == 1:
        print(symbol, indicators_data.sel(asset=symbol, field='shares').values[-1])

NAS:META nan
NAS:LULU 56574615254.22
NAS:CDW 33285349512.0
NAS:MDB 25767646010.4
NAS:FANG 34372380817.46
NAS:GFS 28094353231.94
NAS:AVGO 573720125618.37
NAS:AMD 293136048694.22
NAS:CSX 72448787967.15999
NAS:LRCX 122063448150.0
NAS:MSFT 3131260131262.89
NAS:QCOM 183135600000.0
NAS:CDNS 84897732150.0
NAS:BIIB 31780231266.739998
NAS:INTU 177660674450.0
NAS:ASML 374502070437.11
NAS:TTWO 24769146877.199997
NAS:MAR 72319227139.16
NAS:NFLX 268631184172.16
NAS:MDLZ 97282992944.75
NAS:EXC 36793013731.020004
NAS:TSLA 545618293897.8
NAS:KDP 40670292503.1
NAS:WBA 17859797164.170002
NAS:CCEP 32580298037.100002
NAS:MRNA 40159714892.88
NAS:DASH nan
NAS:GEHC 41337329248.62
NAS:INTC 177787400000.0
NAS:ADI 94490338901.0
NAS:AMAT 167292873657.04
NAS:CTAS 63627808606.45
NAS:PEP 236126948757.80002
NAS:SBUX 103698198000.0
NAS:MNST 63000117666.9
NAS:VRTX 107249405203.2
NAS:SNPS 89193820979.90001
NAS:ORLY 66984680438.55
NAS:CPRT 54101457174.100006
NAS:DLTR 27929003604.03
NAS:TXN 151632812349.47998
NAS:AMZN 1827140369086.9001
NAS:BKNG 119802937241.46
NAS:DXCM 51724604035.56999
NAS:ON 31639413394.08
NAS:TMUS 190777094005.5
NAS:WBD 20737341514.5
NAS:VRSK 34046494056.96
NAS:CRWD nan
NAS:KHC 42786029487.490005
NAS:TTD nan
NAS:AEP 43841699622.2
NAS:MU 103767434720.0
NAS:PAYX 43552833819.04
NAS:ROST 49298041330.380005
NAS:ADSK 54802967111.75
NAS:ADP 100286301121.70999
NAS:IDXX 44404626994.02
NAS:AZN nan
NAS:ANSS 29792286582.59
NAS:CSGP 38431317106.1
NAS:CTSH 37163907688.8
NAS:CHTR 0.0
NAS:FTNT 51725867964.920006
NAS:PANW 90655398000.0
NAS:WDAY 0.0
NAS:NXPI 60621839878.799995
NAS:GOOGL nan
NAS:PDD nan
NAS:AMGN 145625242968.73
NAS:KLAC 92102230350.06
NAS:AAPL 2719006406480.0
NAS:EA 35546917294.560005
NAS:XEL 29106817021.1
NAS:ADBE 235577880000.00003
NAS:FAST 43609858258.549995
NAS:CSCO 198977053111.2
NAS:REGN 0.0
NAS:ODFL 46154568504.22
NAS:GILD 91452314560.56
NAS:ILMN 21159124000.0
NAS:ABNB nan
NAS:DDOG nan
NAS:PYPL 67530454850.64
NAS:TEAM nan

support

@buyers_are_back Hi,
Some fields in fundamental data are not available for all companies, and this is how we get the data from official sources. NaN values are set for companies where field is not available at all, but zero values can be found for companies which have had values at some point in history. For example, "NAS:ABNB" doesn't provide this information ('shares') at all, but 'NAS:REGN' have provided, last time at 2012.
By the way, the values you showed are not correct values for this field for all symbols. At least, this field, which refers to Common Shares Outstanding, is a field with integer values.
Thanks for pointing out the potential error in the data.

Kind regards,

buyers_are_back

@support Thank you for the information. Is there a way to get the market cap of each stock in your framework?

Furthermore what is your data source? I see for example Shares Outstanding / Implied Shares Outstanding of "NAS:ABNB" is available on yahoo finance. https://finance.yahoo.com/quote/ABNB/key-statistics

support

@buyers_are_back Hi,
we use fundamental data from The U.S. Securities and Exchange Commission - EDGAR (sec.gov).
We are working on providing market capitalization field, and we will get back with update in coming week.

Regards,

buyers_are_back

@support Thank you, looking forward to your update.

buyers_are_back

@support I have 3 more questions regarding adjusted close.

(1) I have not checked the source code but I assume your backtest / online evaluation is based on split- and divident-adjusted close prices?

(2) In qnt.qndata.stocks only close is provided. I checked it is split-adjusted properly but not dividend-adjusted (see attached screenshot). Should we retroactively compute the adjusted close based on divs? It would be nice to have it in the data so we don't have to compute it every time.

(3) The split_cumprod column is always 1. What does it mean? It's probably not important.

stefanm

This post is deleted!

support

@buyers_are_back Hi,
(1, 2) Yes, that's correct, we provide split adjusted prices in our datasets, but not dividend adjusted. The dividend adjustment is done in evaluation, under statistics calculation, when calculating relative_returns (you can check it in qnt.stats module), you don't need to adjust close price on dividends. But yes, your comment makes sense, it could be useful for strategy development and we will at least provide a method for dividend adjustment in next qnt library update.
(3) split_cumprod is actually split adjustment factor, and if you divide open/high/low/close price with this factor you will get the real price of an asset on that point of time, the price before any split happened.
You are currently using older version of qnt library, just update it and you will see correct split_cumprod values. With the function restore_origin_data() from qnt.data.stocks you can get the "real" historical prices (not split adjusted).
Thank you for your comments.

Vyacheslav_B

@buyers_are_back Hello. Yes, you can define your function as mentioned in the documentation.
https://quantiacs.com/documentation/en/data/fundamental.html

Please pay attention to the section "Potential Issues in Working with Fundamental Data".

The main problem with market capitalization is that it is necessary to adjust the price for splits, and these data are not always available on time. For example, the report with the number of shares is published after the split on the exchange.

The page contains links to where the data comes from (sec.gov).

import numpy as np
import qnt.data as qndata
import qnt.data.secgov_fundamental as fundamental


def build_market_capitalization(fundamental_facts):
    shares = fundamental.build_shares(fundamental_facts)
    prices_no_split = qndata.restore_origin_data(market_data, make_copy=True)
    close_price = prices_no_split.sel(field='close')

    market_capitalization = shares * close_price

    return market_capitalization


custom_builder = {
    'market_capitalization': {
        'facts': fundamental.FACT_GROUPS['shares'],
        'build': build_market_capitalization,
    },
}

market_data = qndata.stocks.load_ndx_data(min_date="2005-01-01")

indicators_data = fundamental.load_indicators_for(market_data,
                                                  indicator_names=['market_capitalization'],
                                                  indicators_builders=custom_builder)

display(indicators_data.sel(field="market_capitalization").to_pandas().tail(2))
display(indicators_data.sel(asset='NAS:AAPL').to_pandas().tail(2))
display(indicators_data.sel(asset=['NAS:AAPL']).sel(field="market_capitalization").to_pandas().tail(2))

buyers_are_back

@vyacheslav_b

Thanks for the reply. I know "market cap = #shares * price". The real issue is that #shares is not available for some stocks (e.g. META) so I have to rely on other data sources like yahoo finance.

@support

Do you have an update? Thanks!

support

@buyers_are_back Hi,
the update was related to market_capitalization field availability and correctness, as vyacheslav_b described. Unfortunately, at the moment we cannot provide missing fundamental data, like number of ordinary shares, for some stocks (e.g. META). We are going to investigate potential new data sources in order to improve our datasets.
Regards,