Ensemble Methods

class softsensor.ensemble_wrappers.AsyncEnsemble(model, ensemble_weights)

Async ensemble wrapper for point prediction models. Ensemble members predict the entire time series independently.

Parameters:

• model (uncertainty model) – model that is used for prediction

• ensemble_weights (list[torch state_dict]) – weights of the individual base models

Return type:

None.

Examples

>>> import softsensor.ensemble_wrappers as ew
>>> import softsensor.autoreg_models as am
>>> import torch
>>> m = []
>>> for i in range(5):
>>>     m_temp = am.ARNN(2, 1, 10, 10, [16, 8])
>>>     m.append(m_temp.state_dict())
>>> ensemble = ew.AsyncEnsemble(am.ARNN(2, 1, 10, 10, [16, 8]), m)
>>> input = torch.randn(32, 2, 10)
>>> rec_input = torch.randn(32, 1, 10)
>>> out = ensemble.estimate_uncertainty_mean_std(input, rec_input)
>>> print(out[0]) # Mean
>>> print(out[1]) # Std

prediction(dataloader, device='cpu', sens_params=None)

Prediction of a whole Time Series

Parameters:

• dataloader (Dataloader) – Dataloader to predict output

• device (str, optional) – device to compute on. The default is ‘cpu’.

• sens_params (dict, optional) – Dictionary that contains the parameters for the sensitivity analysis. Key ‘method’ defines the method for sensitivity analysis: ‘gradient’ or ‘perturbation’. Key ‘comp’ defines whether gradients are computed for sensitivity analysis. Key ‘plot’ defines whether the results of the sensitivity analysis are visualized. Key ‘sens_length’ defines the number of randomly sampled subset of timesteps for the analysis. (If not a multiple of the model’s forecast, the number will be rounded up to the next multiple.) The default is None, i.e. no sensitivity analysis is computed.

Returns:

• if comp_sens is False

• torch.Tensor (Tensor of same langth as input, containing the predictions.)

• if comp_sens is True – (torch.Tensor, dict) : Tuple of Tensor of same length as input and sensitivity dict. Key is the prediction type, value is the sensitivity tensor.

Examples

Based on the Example from initialisation

>>> import softsensor.meas_handling as ms
>>> import numpy as np
>>> import pandas as pd
>>> t = np.linspace(0, 1.0, 101)
>>> d = {'inp1': np.random.randn(101),
         'inp2': np.random.randn(101),
         'out': np.random.randn(101)}
>>> handler = ms.Meas_handling([pd.DataFrame(d, index=t)], ['train'],
                               ['inp1', 'inp2'], ['out'], fs=100)
>>> loader = handler.give_list(window_size=10, keyword='training',
                               rnn_window=10, batch_size=1)
>>> pred = ensemble.prediction(loader[0]) # Prediction of Mean and Variance
>>> print(pred[0].shape)
torch.Size([1, 101])

class softsensor.ensemble_wrappers.AsyncMCDropout(model, n_samples)

Async MCDO wrapper for probabilistic point prediction models

Parameters:

• model (uncertainty model) – probabilistic model that is used for prediction

• n_samples (int) – amount of samples to draw for monte carlo estimation

Return type:

None.

Examples

>>> import softsensor.autoreg_models as am
>>> import softsensor.ensemble_wrappers as ew
>>> import torch
>>> m = am.ARNN(2, 1, 10, 10, [16, 8], dropout=.5, concrete_dropout=True)
>>> ensemble = ew.AsyncMCDropout(m, n_samples=10)
>>> input = torch.randn(32, 2, 10)
>>> rec_input = torch.randn(32, 1, 10)
>>> out = ensemble.estimate_uncertainty_mean_std(input, rec_input)
>>> print(out[0].shape) # Mean
>>> print(out[1].shape) # Std

estimate_uncertainty_mean_std(inp, x_rec)

Computes the mean prediction and epistemic uncertainty of the ensemble for one step

Parameters:

• inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation, shape=[batch size, external channels, window_size]

• x_rec (torch.tensor, dtype=torch.float) – Recurrent Input for forward Propagation. shape=[batch size, pred_size, rnn_window]

Returns:

mean: torch.tensor dtype=torch.float()

shape=[batch size, pred_size]

std: torch.tensor dtype=torch.float() in [0,1]

shape=[batch size, pred_size]

Return type:

(mean, std)

prediction(dataloader, device='cpu')

Prediction of a whole Time Series

Parameters:

• dataloader (Dataloader) – Dataloader to predict output

• device (str, optional) – device to compute on. The default is ‘cpu’.

Returns:

• if loss_ft=None –

  (torch.Tensor, list[torch.Tensor])

  tuple of Torch Tensor of same length as input and variances

• if loss_ft=torch loss function –

  (torch.Tensor, list[torch.Tensor], loss)

  tuple of Torch Tensor of same length as input, variances and loss

Examples

>>> import softsensor.meas_handling as ms
>>> import numpy as np
>>> import pandas as pd
>>> t = np.linspace(0, 1.0, 101)
>>> d = {'inp1': np.random.randn(101),
         'inp2': np.random.randn(101),
         'out': np.random.randn(101)}
>>> handler = ms.Meas_handling([pd.DataFrame(d, index=t)], ['train'],
                               ['inp1', 'inp2'], ['out'], fs=100)
>>> loader = handler.give_list(window_size=10, keyword='training',
                               rnn_window=10, batch_size=1)
>>> pred = ensemble.prediction(loader[0]) # Prediction of Mean and Variance

class softsensor.ensemble_wrappers.AsyncMCDropoutMVE(model, n_samples)

Async MCDO wrapper for MVE models

Parameters:

• model (uncertainty model) – probabilistic model that is used for prediction

• n_samples (int) – amount of samples to draw for monte carlo estimation

Return type:

None.

Examples

>>> import softsensor.autoreg_models as am
>>> import softsensor.ensemble_wrappers as ew
>>> import torch
>>> params = {'input_channels': 2,
              'pred_size': 1,
              'window_size': 10,
              'rnn_window': 10}
>>> mean_model = am.ARNN(**params, hidden_size=[16, 8],
                         dropout=.5, concrete_dropout=True)
>>> m = am.SeparateMVEARNN(**params,mean_model=mean_model,
                           var_hidden_size=[16, 8])
>>> ensemble = ew.AsyncMCDropoutMVE(m, n_samples=10)
>>> input = torch.randn(32, 2, 10)
>>> rec_input = torch.randn(32, 1, 10)
>>> out = ensemble.estimate_uncertainty_mean_stds(input, rec_input)
>>> print(len(out))

estimate_uncertainty_mean_stds(inp, x_rec)

Computes the mean prediction and uncertainty of the ensemble for one step

Parameters:

• inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation, shape=[batch size, external channels, window_size]

• x_rec (torch.tensor, dtype=torch.float) – Recurrent Input for forward Propagation. shape=[batch size, pred_size, rnn_window]

Returns:

mean: torch.tensor dtype=torch.float()

shape=[batch size, pred_size]

std: torch.tensor dtype=torch.float() in [0,1]

shape=[batch size, pred_size]

Return type:

(mean, aleatoric std, epistemic std)

prediction(dataloader, device='cpu', reduce=False)

Prediction of a whole Time Series

Parameters:

• dataloader (Dataloader) – Dataloader to predict output

• device (str, optional) – device to compute on. The default is ‘cpu’.

• reduce (bool, optional) – Whether the combined uncertainty or both uncertainties should be returned. The default is True.

Returns:

• if loss_ft=None –

  (torch.Tensor, torch.Tensor) if reduce else (torch.Tensor, torch.Tensor, torch.Tensor)

  tuple of Torch Tensor of same length as input (prediction, uncertainties) where uncertainties = aleatoric_var + epistemic_var if reduce else (aleatoric_var, epistemic_var)

• if loss_ft=torch.loss –

  ((torch.Tensor, torch.Tensor), float) if reduce else ((torch.Tensor, torch.Tensor, torch.Tensor), float)

  adds the computed loss as a second output ((prediction, uncertainties), loss)

• >>> import softsensor.meas_handling as ms

• >>> import numpy as np

• >>> import pandas as pd

• >>> t = np.linspace(0, 1.0, 101)

• >>> d = {‘inp1’ (np.random.randn(101),) – ‘inp2’: np.random.randn(101), ‘out’: np.random.randn(101)}

• >>> handler = ms.Meas_handling([pd.DataFrame(d, index=t)], [‘train’], – [‘inp1’, ‘inp2’], [‘out’], fs=100)

• >>> loader = handler.give_list(window_size=10, keyword=’training’, – rnn_window=10, batch_size=1)

• >>> pred = ensemble.prediction(loader[0], reduce=False) # Prediction of Mean and Variance

• >>> print(pred[0].shape) # Mean

• >>> print(pred[1].shape) # aletoric Variance

• >>> print(pred[2].shape) # epistemic Variance

class softsensor.ensemble_wrappers.AsyncMVEEnsemble(model, ensemble_weights)

Async ensemble wrapper for MVE models

Parameters:

• model (uncertainty model) – model that is used for prediction

• ensemble_weights (list[torch state_dict]) – weights of the individual base models

Return type:

None.

Examples

>>> import softsensor.ensemble_wrappers as ew
>>> import softsensor.autoreg_models as am
>>> import torch
>>> params = {'input_channels': 2,
              'pred_size': 1,
              'window_size': 10,
              'rnn_window': 10}
>>> m = []
>>> for i in range(5):
>>>     mean_model = am.ARNN(**params, hidden_size=[16, 8])
>>>     m_temp = am.SeparateMVEARNN(**params,mean_model=mean_model,
                                    var_hidden_size=[16, 8])
>>>     m.append(m_temp.state_dict())
>>> ensemble = ew.AsyncMVEEnsemble(m_temp, m)
>>> input = torch.randn(32, 2, 10)
>>> rec_input = torch.randn(32, 1, 10)
>>> out = ensemble.estimate_uncertainty_mean_stds(input, rec_input)
>>> print(len(out))
3

estimate_uncertainty_mean_stds(inp, x_rec)

Computes the mean prediction and uncertainty of the ensemble for one step

Parameters:

• inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation, shape=[batch size, external channels, window_size]

• x_rec (torch.tensor, dtype=torch.float) – Recurrent Input for forward Propagation. shape=[batch size, pred_size, rnn_window]

Returns:

mean: torch.tensor dtype=torch.float()

shape=[batch size, pred_size]

std: torch.tensor dtype=torch.float() in [0,1]

shape=[batch size, pred_size]

Return type:

(mean, aleatoric std, epistemic std)

prediction(dataloader, device='cpu', reduce=False, sens_params=None)

Prediction of a whole Time Series

Parameters:

• dataloader (Dataloader) – Dataloader to predict output

• device (str, optional) – device to compute on. The default is ‘cpu’.

• reduce (bool, optional) – Whether the combined uncertainty or both uncertainties should be returned. The default is True.

• sens_params (dict, optional) – Dictionary that contains the parameters for the sensitivity analysis. Key ‘method’ defines the method for sensitivity analysis: ‘gradient’ or ‘perturbation’. Key ‘comp’ defines whether gradients are computed for sensitivity analysis. Key ‘plot’ defines whether the results of the sensitivity analysis are visualized. Key ‘sens_length’ defines the number of randomly sampled subset of timesteps for the analysis. (If not a multiple of the model’s forecast, the number will be rounded up to the next multiple.) The default is None, i.e. no sensitivity analysis is computed.

Returns:

• if comp_sens is False

• torch.Tensor (Tensor of same langth as input, containing the predictions.)

• if comp_sens is True – (torch.Tensor, dict) : Tuple of Tensor of same length as input and sensitivity dict. Key of that dict is the prediction type (str), value is the sensitivity tensor.

Examples

Based on the Example from initialisation

>>> import softsensor.meas_handling as ms
>>> import numpy as np
>>> import pandas as pd
>>> t = np.linspace(0, 1.0, 101)
>>> d = {'inp1': np.random.randn(101),
         'inp2': np.random.randn(101),
         'out': np.random.randn(101)}
>>> handler = ms.Meas_handling([pd.DataFrame(d, index=t)], ['train'],
                               ['inp1', 'inp2'], ['out'], fs=100)
>>> loader = handler.give_list(window_size=10, keyword='training',
                               rnn_window=10, batch_size=1)
>>> pred = ensemble.prediction(loader[0], reduce=False) # Prediction of Mean and Variance
>>> print(pred[0].shape) # Mean
>>> print(pred[1].shape) # aletoric Variance
>>> print(pred[2].shape) # epistemic Variance

class softsensor.ensemble_wrappers.SyncEnsemble(model, ensemble_weights)

Sync ensemble wrapper for point prediction models.

All ensemble models have the same recurrent state at each point.

Parameters:

• model (uncertainty model) – model that is used for prediction

• ensemble_weights (list[torch state_dict]) – weights of the individual base models

Return type:

None.

Examples

>>> import softsensor.ensemble_wrappers as ew
>>> import softsensor.autoreg_models as am
>>> import torch
>>> m = []
>>> for i in range(5):
>>>     m_temp = am.ARNN(2, 1, 10, 10, [16, 8])
>>>     m.append(m_temp.state_dict())
>>> ensemble = ew.SyncEnsemble(am.ARNN(2, 1, 10, 10, [16, 8]), m)
>>> input = torch.randn(32, 2, 10)
>>> rec_input = torch.randn(32, 1, 10)
>>> out = ensemble.estimate_uncertainty_mean_std(input, rec_input)
>>> print(out[0]) # Mean
>>> print(out[1]) # Std

estimate_uncertainty_mean_std(inp, x_rec)

Computes the mean prediction and epistemic uncertainty of the ensemble for one step

Parameters:

• inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation, shape=[batch size, external channels, window_size]

• x_rec (torch.tensor, dtype=torch.float) – Recurrent Input for forward Propagation. shape=[batch size, pred_size, rnn_window]

Returns:

mean: torch.tensor dtype=torch.float()

shape=[batch size, pred_size]

std: torch.tensor dtype=torch.float() in [0,1]

shape=[batch size, pred_size]

Return type:

(mean, std)

forward_sens(inp)

Computes the mean prediction and epistemic uncertainty of the ensemble for one step, but only with one input tensor that is already concatenated to allow for gradient-based sensitivity analysis

Parameters:

inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation that is already concatenated, shape=[batch size, external channels*window_size + pred_size*rnn_window]

Returns:

mean: torch.tensor dtype=torch.float()

shape=[batch size, pred_size]

std: torch.tensor dtype=torch.float() in [0,1]

shape=[batch size, pred_size]

Return type:

(mean, std)

get_recurrent_weights()

Function that returns the weight that effect the Recurrent input of the underlying Network (mean network)

Returns:

recurrent_weights – List of the Weights that effect the Recurrent input of the Network.

Return type:

list of weight Tensors

load_and_fire(weights, inputs, x_rec)

Loads the state_dict of a base model and computes forward

Parameters:

• weights (torch state_dict) – Weights of a base model

• inp (torch.tensor dtype=torch.float) – Input tensor for forward propagation, shape=[batch size, external channels, window_size]

• x_rec (torch.tensor, dtype=torch.float) – Recurrent Input for forward Propagation. shape=[batch size, pred_size, rnn_window]

Returns:

pred – shape=[batch size, pred_size]

Return type:

torch.tensor dtype=torch.float()

prediction(dataloader, device='cpu', sens_params=None)

Prediction of a whole Time Series

Parameters:

• dataloader (Dataloader) – Dataloader to predict output

• device (str, optional) – device to compute on. The default is ‘cpu’.

• sens_params (dict, optional) – Dictionary that contains the parameters for the sensitivity analysis. Key ‘method’ defines the method for sensitivity analysis: ‘gradient’ or ‘perturbation’. Key ‘comp’ defines whether gradients are computed for sensitivity analysis. Key ‘plot’ defines whether the results of the sensitivity analysis are visualized. Key ‘sens_length’ defines the number of randomly sampled subset of timesteps for the analysis. (If not a multiple of the model’s forecast, the number will be rounded up to the next multiple.) The default is None, i.e. no sensitivity analysis is computed.

Returns:

• if comp_sens is False

• torch.Tensor (Tensor of same langth as input, containing the predictions.)

• if comp_sens is True – (torch.Tensor, dict) : Tuple of Tensor of same length as input and sensitivity dict. Key is the prediction type, value is the sensitivity tensor.

Examples

Based on the Example from initialisation

>>> import softsensor.meas_handling as ms
>>> import numpy as np
>>> import pandas as pd
>>> t = np.linspace(0, 1.0, 101)
>>> d = {'inp1': np.random.randn(101),
         'inp2': np.random.randn(101),
         'out': np.random.randn(101)}
>>> handler = ms.Meas_handling([pd.DataFrame(d, index=t)], ['train'],
                               ['inp1', 'inp2'], ['out'], fs=100)
>>> loader = handler.give_list(window_size=10, keyword='training',
                               rnn_window=10, batch_size=1)
>>> pred = ensemble.prediction(loader[0]) # Prediction of Mean and Variance
>>> print(pred.shape)
torch.Size([2, 1, 101])