Stochastic Gradient Descent Implementation

Overview

Error Function

The stochastic gradient descent algorithm will implement an error function will compute the error on a set of randomly selected points from the oringinal dataset.

The following code uses the sampling library to create a dataset of size 1, with the single item chosen at random from the orginal trainging dataset. The algorithm can choose a batch size greater than 1 if necessary.


	function getError(){
	let batchSize = 1;
	let random = rs.randomize(data,batchSize);
	let error = function(arg1, arg2){
	let total = 0;
	for(let item of random){
	let value = method(...item.args);
	total += Math.abs(value-item.value);
	}
	return total;
	}
	return error;
	}

Optimize

Next, the gradient descent library is used to run several interations of the gradient descent method, using a different error function for each run.


	//set args to initial values
	let args = [0.1,0.1]
	for(let i=0;i<1000000; i++){
	//dont hang the processor
	if(i%1000===0)await $wait();
	args = op.iterate(getError(), ()=>0.001, args, 1, 0.00000001);
	}