In the function "train_model()" the for-loop "for asset_name in asset_name_all:" finishes before the model-optimization. So basically the optimized LSTM neural network is always based on the last asset. This can't be on purpose, can it?

best
eduard

def train_model(data):
"""
train the LSTM network
"""

asset_name_all = data.coords['asset'].values

features_all = get_features(data)
target_all = get_target_classes(data)

model = get_model()

for asset_name in asset_name_all:
    
    # drop missing values:
    target_cur = target_all.sel(asset=asset_name).dropna('time', 'any')
    features_cur = features_all.sel(asset=asset_name).dropna('time', 'any')

    # align features and targets:
    target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner')
    
criterion = nn.MSELoss() # define loss function

optimiser = optim.LBFGS(model.parameters(), lr=0.08) # we use an LBFGS solver as optimiser
epochs = 1 #how many epochs 
for i in range(epochs):
        def closure(): # reevaluates the model and returns the loss (forward pass)
            optimiser.zero_grad()
            
            #input tensor
            in_ = torch.zeros(1,len(feature_for_learn_df.values))
            in_[0,:]=torch.tensor(np.array(feature_for_learn_df.values))
            
            #output
            out = model(in_)
            
            #target tensor
            target = torch.zeros(1,len(target_for_learn_df.values))
            target[0,:]=torch.tensor(np.array(target_for_learn_df.values))
            
            #evaluate loss
            loss = criterion(out, target)
            loss.backward()
            
            return loss
        optimiser.step(closure) #updates weights
        
return model

The loop "for asset_name in asset_name_all:" creates a model for each asset, but the individual models are never saved. At the end, the model for the last asset is returned and all the predictions are created based on this last model (asset 'XRP').

def train_model(data):
"""Create and train the models working on an asset-by-asset basis."""

asset_name_all = data.coords['asset'].values

data = data.sel(time=slice('2013-05-01',None)) # cut the noisy data head before 2013-05-01

features_all = get_features(data)
target_all = get_target_classes(data)

model = create_model()

for asset_name in asset_name_all:
    
    # drop missing values:
    target_cur = target_all.sel(asset=asset_name).dropna('time', 'any')
    features_cur = features_all.sel(asset=asset_name).dropna('time', 'any')

    # align features and targets:
    target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner')

    if len(features_cur.time) < 10:
        # not enough points for training
        continue

    
    
    try:
        model.fit(feature_for_learn_df.values, target_for_learn_df)
    except KeyboardInterrupt as e:
        raise e
    except:
        logging.exception('model training failed')

return model

Thanks for your support, I really appreciate, together we are better!

This is how I solved the step four issue:

Kind Regards
Eddie

Hello friends,

how can I update my local development to the Q18 contest? Loading nasdaq data doesn't work.

Best
Eddie

Thanks for your support, I really appreciate, together we are better!

This is how I solved the step four issue:

Kind Regards
Eddie

Sorry, but it still doesn't work.

What should I do after step 4? I guess, I can't just type python commands in the powershell. How can I open the right Jupyter Notebook?

Hi guys, local development doesn't work.
I followed all the steps for installing local development.

I run "conda activate qntdev"
Then I create a Jupyter Notebook in this folder:
"C:\Users\baiti\anaconda3\envs\qntdev"

But when I run this Notebook I get always the error: "No module named 'qnt'"

Can you help me out?

Best
Eddie

Dear quantiacs community and team, the Q17 contest is running sind the 1st of May, but I can't see the live performance chart of my strategy in the contest.
Can you help me please on this issue?
Best
Eduard

@support

This is a possible fix, but no gurantee. You have to adjust also the prediction function.

def train_model(data):
"""Create and train the models working on an asset-by-asset basis."""

models = dict()

asset_name_all = data.coords['asset'].values

data = data.sel(time=slice('2013-05-01',None)) # cut the noisy data head before 2013-05-01

features_all = get_features(data)
target_all = get_target_classes(data)

model = create_model()

for asset_name in asset_name_all:

# drop missing values:
target_cur = target_all.sel(asset=asset_name).dropna('time', 'any')
features_cur = features_all.sel(asset=asset_name).dropna('time', 'any')

# align features and targets:
target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner')

if len(features_cur.time) < 10:
    # not enough points for training
    continue



try:
    model.fit(feature_for_learn_df.values, target_for_learn_df)
    models[asset_name] = model
except KeyboardInterrupt as e:
    raise e
except:
    logging.exception('model training failed')

return models

The loop "for asset_name in asset_name_all:" creates a model for each asset, but the individual models are never saved. At the end, the model for the last asset is returned and all the predictions are created based on this last model (asset 'XRP').

def train_model(data):
"""Create and train the models working on an asset-by-asset basis."""

asset_name_all = data.coords['asset'].values

data = data.sel(time=slice('2013-05-01',None)) # cut the noisy data head before 2013-05-01

features_all = get_features(data)
target_all = get_target_classes(data)

model = create_model()

for asset_name in asset_name_all:
    
    # drop missing values:
    target_cur = target_all.sel(asset=asset_name).dropna('time', 'any')
    features_cur = features_all.sel(asset=asset_name).dropna('time', 'any')

    # align features and targets:
    target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner')

    if len(features_cur.time) < 10:
        # not enough points for training
        continue

    
    
    try:
        model.fit(feature_for_learn_df.values, target_for_learn_df)
    except KeyboardInterrupt as e:
        raise e
    except:
        logging.exception('model training failed')

return model

In the function "train_model()" the for-loop "for asset_name in asset_name_all:" finishes before the model-optimization. So basically the optimized LSTM neural network is always based on the last asset. This can't be on purpose, can it?

best
eduard

def train_model(data):
"""
train the LSTM network
"""

asset_name_all = data.coords['asset'].values

features_all = get_features(data)
target_all = get_target_classes(data)

model = get_model()

for asset_name in asset_name_all:
    
    # drop missing values:
    target_cur = target_all.sel(asset=asset_name).dropna('time', 'any')
    features_cur = features_all.sel(asset=asset_name).dropna('time', 'any')

    # align features and targets:
    target_for_learn_df, feature_for_learn_df = xr.align(target_cur, features_cur, join='inner')
    
criterion = nn.MSELoss() # define loss function

optimiser = optim.LBFGS(model.parameters(), lr=0.08) # we use an LBFGS solver as optimiser
epochs = 1 #how many epochs 
for i in range(epochs):
        def closure(): # reevaluates the model and returns the loss (forward pass)
            optimiser.zero_grad()
            
            #input tensor
            in_ = torch.zeros(1,len(feature_for_learn_df.values))
            in_[0,:]=torch.tensor(np.array(feature_for_learn_df.values))
            
            #output
            out = model(in_)
            
            #target tensor
            target = torch.zeros(1,len(target_for_learn_df.values))
            target[0,:]=torch.tensor(np.array(target_for_learn_df.values))
            
            #evaluate loss
            loss = criterion(out, target)
            loss.backward()
            
            return loss
        optimiser.step(closure) #updates weights
        
return model