Overview
Each layer of the neural network is implemented as a Javascript object with a set of functions defined on the object.
Methods
The methods of the neural network are implemented using standard linear algebra methods and following the linear algebra library. The inputs to any layer is a column vector, and is implemented as in the linear algebra library. (as an array of arrays, each inner array represents a row)
- evaluate(input) Takes the input as a column vector, and returns a column vector as an output.
- inputGradient(input) - takes the inputs to the function, and returns a Jacobian matrix of the differential of the outputs of the layer to the inputs. (NOte, the Jacobian in this instance should follow the denominator format)
- parameterGradient(input) - this function should only be present if there are parameters in the layer that are to be edited using the backpropagation algorithm. It returns a Jacobian matrix of the differential of the outputs of the layer to the parameters. (NOte, the Jacobian in this instance should follow the denominator format)
Backpropagation Algorithm
The backpropagation algorithm uses the inputGradient and parameterGradient (if available) methods on each layer.
In order to optimize the library, functions that return a matrix or vector (an array of arrays) may reuse the object they returned on the next call to that function. That is, to avoid excessive object creation, a layer may hold onto a matrix that it has returned in one function call, and will just re-populate that matrix in the next call to the same function.
As such, if you need to hold onto the results of a particular function call after the function is called again, you should make a copy of the result.