Javascript Neural Networks - Affine Function Layer

Overview


An affine function is a function of the form.
{% y = \textbf{W} \vec{x} + \vec{b} %}
where {% \textbf{W} %} is a matrix and {% \vec{b} %} is a column vector.

This type of function forms one of the core forms of layers in a neural network. In the simplest kind of network, affine function layers are the only layers with tunable parameters, that is, the elements in the weight matrix {% W %} or the bias vector {% \vec{b} %}.

Creating an Affine Layer


There are multiple ways to create an affine layer using the backpropagation library. The first way is to specify the number of inputs to the layer, and the number of outputs. 


let ba = await import('/lib/backpropagation/v1.0.0/backpropagation.mjs');

let layers = [];
layers.push(ba.affine({
    inputs:20,
    outputs:5,
}));

The weight matrix and bias will be filled with random numbers.

Alternatively, one can specify the weights and the bias used by providing those matrices. 


let ba = await import('/lib/backpropagation/v1.0.0/backpropagation.mjs');

let layers = [];
layers.push(ba.affine({
    weights:[[1,0],[0,1]],
    bias:[[1],[1]],
}));

Note that inputs to the layer are assumed to be column vectors. The bias therefore is also a column vector. That is, a column vector has the form : 


let vector = [[1],[1],[1],[1],[1]]

That is, each element in the column vector is wrapped in array. See linear algebra api for details.

Providing the actual values of the weight matrix and the bias is useful if you want to initialize the elements of each matrix yourself, or you have trained a model and saved the matrices.