Feed Forward Neural Network

The FeedForwardNeuralNetwork class does not perform any internal computation. It simply passes tensors along to the network used during initialization. It is designed to assemble and control Bayesian and ProbabilisticDropout layers in a large neural network via its sample() and drop() methods.

The following example sets the sampling mode in all the BayesianLinear and dropping mode in all ProbabilisticDropout layers with a single call to model.sample() and model.drop().

 1>>> import torch.nn as nn
 2>>> import UQpy.scientific_machine_learning as sml
 3>>> network = nn.Sequential(
 4>>>     sml.BayesianLinear(1, 4),
 5>>>     nn.ReLU(),
 6>>>     sml.ProbabilisticDropout(),
 7>>>     sml.BayesianLinear(4, 4),
 8>>>     nn.ReLU(),
 9>>>     sml.ProbabilisticDropout(),
10>>>     sml.BayesianLinear(4, 1),
11>>> )
12>>> model = sml.FeedForwardNeuralNetwork(network)
13>>> print("Initially sampling:", model.sampling)
14>>> print("Initially dropping:", model.dropping)
15>>> model.sample(False)
16>>> model.drop(False)
17>>> print("Currently sampling:", model.sampling)
18>>> print("Currently dropping:", model.dropping)
19>>> print(model)
20Initially sampling: True
21Initially dropping: True
22Currently sampling: False
23Currently dropping: False
24FeedForwardNeuralNetwork(
25  (network): Sequential(
26    (0): BayesianLinear(in_features=1, out_features=4, sampling=False)
27    (1): ReLU()
28    (2): ProbabilisticDropout(p=0.5, dropping=False)
29    (3): BayesianLinear(in_features=4, out_features=4, sampling=False)
30    (4): ReLU()
31    (5): ProbabilisticDropout(p=0.5, dropping=False)
32    (6): BayesianLinear(in_features=4, out_features=1, sampling=False)
33  )
34)

The FeedForwardNeuralNetwork class is imported using the following command:

>>> from UQpy.scientific_machine_learning import FeedForwardNeuralNetwork

Methods

class FeedForwardNeuralNetwork(network)[source]

Initialize a typical feed-forward neural network using the architecture provided by network

Parameters:

network (Module) – Network defining the function from \(f(x)=y\)

forward(*args, **kwargs)[source]

Forward call of the neural network

Parameters:

  • args – Input arguments pass to network

  • kwargs – Keyword arguments passed to network

Returns:

Output arguments

summary(**kwargs)

Call torchinfo.summary() on self. See torchinfo documentation for details.

Parameters:

kwargs – Keyword arguments passed to torchinfo.summary.

Returns:

Model statistics

count_parameters()

Get the total number of parameters that require a gradient computation in the model

drop(mode=True)

Set dropping mode.

Note

This method and self.dropping only affects UQpy’s dropout layers

Parameters:

mode (bool) – If True perform dropout, otherwise act as the identity function.

sample(mode=True)

Set sampling mode.

Note

This method and self.sampling only affects UQpy’s Bayesian layers

Parameters:

mode (bool) – If True sample from distributions, otherwise use distribution means.

Returns:

self

is_deterministic()

Check if neural network is behaving deterministically or probabilistically.

Note

This flag may be incorrect if the model has sources of randomness that do not depend on the attributes training, dropping, or sampling.

Return type:

bool

Returns:

True if sampling, dropping, and training are all False. Otherwise, returns False.

set_deterministic(mode=True)

Set training, dropping, and sampling to the opposite of mode.

This is equivalent to

>>> model.train(not mode)
>>> model.drop(not mode)
>>> model.sample(not mode)

If the model has sources of randomness that do not depend on the training, dropping, or sampling attributes, they will not be affected.

Attributes

FeedForwardNeuralNetwork.network: nn.Module

Neural network architecture defined as a torch.nn.Module

FeedForwardNeuralNetwork.dropping: bool

Boolean represents whether this module is in dropping mode or not.

FeedForwardNeuralNetwork.sampling: bool

Boolean represents whether this module is in sampling mode or not.