Neural Network Layers
In this module, a layer in a neural network refers to an operation on tensor \(x\) that maps it to a tensor \(y\).
Many layers have learnable parameters implemented via torch.nn.Parameter.
They may be deterministic or have probabilistic behavior, such as the Dropout layers.
The layers available here are designed for compatability with torch’s layers, and recreate their naming and syntax conventions as much as practical.
Notation
We use notation consistent with PyTorch to denote the input and output tensor shapes in any given layer. These variables are often combined to describe the shape of a tensor with many dimensions. Unless otherwise stated, the variables are:
Symbol |
Description |
|---|---|
\(N\) |
Batch size |
\(C, C_\text{in}, C_\text{out}\) |
Number of channels in a signal |
\(L\) |
Length of a signal. Typically used for 1d signals. |
\(H, W\) |
Height, width of a signal. Typically used for 2d signals. |
\(D, H, W\) |
Depth, height, width of a signal. Typically used for 3d signals. |
Layer Baseclass
The Layer is an abstract baseclass and a subclass of torch.nn.Module, just as all torch layers are.
All of the Dropout layers share their own base class, as do the Bayesian layers.
This is the parent class to all layers.
Like all abstract baseclasses, this cannot be instantiated but can be subclassed to write custom layers.
All layers use the forward() method to define the forward model call.
Some documentation within the Layer class may be inherited from PyTorch docstrings.
Methods
- class Layer(**kwargs)[source]
Initialize internal Module state, shared by both nn.Module and ScriptModule.
- abstract forward(x)[source]
Define the computation performed at every call.
Should be overridden by all subclasses. :rtype:
TensorNote
Although the recipe for forward pass needs to be defined within this function, one should call the
Moduleinstance afterwards instead of this since the former takes care of running the registered hooks while the latter silently ignores them.