Loss Functions

Overview

Loss Functions are the functions used to measure the error of a neural network model against an input/output pair or set of pairs.

API Examples

The following shows calculating the total loss or error using pre-defined loss functions from the api. The error function takes four parameters.

layers - this is the array that represents the model
inputs - this is an array of column vectors
outputs - this is an array of column vectors
loss function

meanSquaredError


	let error = ba.error(layers, inputs, outputs, ba.meanSquaredError());

crossEntropyLoss


	let error = ba.error(layers, inputs, outputs, ba.crossEntropyLoss());

Implementation

An error function must implement two functions:

evaluate - evaluates the loss for the two values, one being the evaluated value, and the other being the target (or real) value.
inputGradient - the inputGradient function calculates the gradient of the layer given an input and corresponding output.

The following code demonstrates implementing


function meanSquaredError(){
	let layer = {
	    type:'meanSquaredError',
	    evaluate:function(val1, val2){
	        let diff = la.subtract(val1, val2);
	        let result = la.multiply(la.transpose(diff), diff);
	        result[0][0] = 0.5 * result[0][0];
	        return result;
	    },
	    inputGradient:function(input, output){
	        let result = [];
	        for(let i=0;i<input.length;i++){
	            result.push([input[i][0] - output[i][0]]);
	        }
	        return result;
	    },
	};
	return layer;
}