Bayesian Layer Baseclass

This is the parent class to all Bayesian layers. The NormalBayesianLayer is an abstract baseclass and a subclass of torch.nn.Module.

The documentation in the forward() and extra_repr() on this page may be inherited from PyTorch docstrings.

Methods

class NormalBayesianLayer(parameter_shapes, sampling=True, prior_mu=0.0, prior_sigma=0.1, posterior_mu_initial=(0.0, 0.1), posterior_rho_initial=(-3.0, 0.1), device=None, dtype=None)

Initialize the random variables governing the parameters of the layer.

Parameters:

• parameter_shapes (dict) – Dictionary with "name": shape pairs for each parameter. For each key in the dictionary, assign learnable parameters name_mu and name_rho with the given shape.

• sampling (bool) – If True, sample layer parameters from their respective Gaussian distributions. If False, use distribution mean as parameter values. Default: True

• prior_mu (float) – Prior mean, \(\mu_\text{prior}\) of the prior normal distribution. Default: 0.0

• prior_sigma (float) – Prior standard deviation, \(\sigma_\text{prior}\), of the prior normal distribution. Default: 0.1

• posterior_mu_initial (tuple[float, float]) – Mean and standard deviation of the initial posterior distribution for \(\mu\). The initial posterior is \(\mathcal{N}(\mu_\text{posterior}[0], \mu_\text{posterior}[1])\). Default: (0.0, 0.1)

• posterior_rho_initial (tuple[float, float]) – Mean and standard deviation of the initial posterior distribution for \(\rho\). The initial posterior is \(\mathcal{N}(\rho_\text{posterior}[0], \rho_\text{posterior}[1])\). The standard deviation of the posterior is computed as \(\sigma = \ln( 1 + \exp(\rho))\) to ensure it is positive. Default: (-3.0, 0.1)

• device (Union[device, str, None]) – A torch.device representing the device on which tensors are allocated

• dtype (Union[dtype, tuple, None]) – A torch.dtype (or tuple of them) representing the data type of the tensor

reset_parameters()

Populate parameters with samples from posterior Normal distributions.

get_bayesian_weights()

Get the weights for the Bayesian layer.

If sampling is True, then sample weights from their respective distributions. Otherwise, use distribution means for weights.

Return type:

tuple

Returns:

Tuple containing weight tensors

sample(mode=True)

Set sampling mode for this and all child Modules

Note

This method and self.sampling only affects Bayesian layers

Parameters:

mode (bool) – If True, sample from distributions, otherwise use distribution means. Default: True

abstract forward(*args, **kwargs)

Define the computation performed at every call.

Should be overridden by all subclasses.

Note

Although the recipe for forward pass needs to be defined within this function, one should call the Module instance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.

abstract extra_repr()

Return the extra representation of the module.

To print customized extra information, you should re-implement this method in your own modules. Both single-line and multi-line strings are acceptable.

Attributes

NormalBayesianLayer.parameter_shapes: dict

Prefix names and shapes of all learnable parameters

NormalBayesianLayer.sampling: bool

Boolean represents whether this module is in sampling mode or not.

NormalBayesianLayer.prior_mu: float

NormalBayesianLayer.prior_sigma: float

NormalBayesianLayer.posterior_mu_initial: tuple[float, float]

NormalBayesianLayer.posterior_rho_initial: tuple[float, float]