Fitting Sequential Models

Overview


Once a model has been created, it needs to be fit to a dataset in order to be useful. That is, typically the parameters of the model are initialized to random numbers. Fitting the model to data means running the model on a set of inputs and cross-checking the result to a set of outputs. The model then uses an optimizer to optimize the models parameters in order to match the outputs as closely as possible.

Choosing an Optimizer


TensorFlow.js supports the following optimizers, most of which are variations on the gradient descent algorithm.

  • Stochastic Gradient Descent
  • Momentum
  • AdaGrad
  • AdaDelta
  • Adam
  • Adamax
  • RMSprop


(Please see optimizers for the official API)

The following code demonstrates creating an optimizer. 


const learningRate = 0.2;
const optimizer = tf.train.adam(learningRate);
const optimizer2 = tf.train.sgd(learningRate);

Compiling the Model


Before using a model, it needs to be compiled. The compile method may throw an error if the model is mis-specified somehow. 


model.compile({
  loss:tf.losses.meanSquaredError,
  optimizer:tf.train.adam(0.1)
});


Fitting the Model


Once the model has been specified, it can be fit to a set of data. Typically the data will come as an array of data points. In the following sample data, the data is separated as the features, given in the X array, and the target labels, given in the y array. The data needs to be put into a tensor before being sent to the model. (Note, these tensors will need to be disposed of later.) 


let X = [[1,1],[1,2],[3,1]];
let y= [[1],[1],[2]];

let xs = tf.tensor(X);
let ys = tf.tensor(y);


To fit the model to the data, call the fit method. You can specify the following options to tune the fitting process

  • batchSize - number of data points to include in an epoch. This can usually be omitted
  • epochs - number of iterations to run
  • shuffle - if set to true, it will randomly shuffle the input data before running the training for the epoch. As a general rule, this should be generally be set to true. 



await model.fit(xs,ys,{
  batchSize:2,
  shuffle:true,
  epochs:10
});


T

Once the model is fit, it saves the newly fit weights. The fit method can be called multiple times, and the model will pick up where it left off. 


for(let i=0;i<10;i++){
  await model.fit(tf.tensor(X),tf.tensor(y),{
    shuffle:true,
    epochs:10
  });
}


In this example, one could choose to increase the number of epochs instead of running the above loop. However, being able to call fit multiple times creates flexibility. For example, the model can be recompiled between fits as in the following code. 


await model.fit(xs,ys,{
  shuffle:true,
  epochs:10
});

const optimizer2 = tf.train.sgd(0.5);
model.compile({
  optimizer:optimizer2,
  loss:tf.losses.logLoss,
  shuffle:true,
  metrics:["accuracy"]
});

await model.fit(xs,ys,{
  shuffle:true,
  epochs:10
});
Try it!

Callbacks



const logCallback = {
  onEpochEnd: async (epoch, logs) => {			  
    console.log(`Epoch ${epoch}: loss = ${logs.loss}`);
  },
  onBatchEnd:async (batch, logs)=>{

  }
}

model.fit(trainXs, trainYs, {
  batchSize: BATCH_SIZE,
  validationData: [testXs, testYs],
  epochs: 10,
  shuffle: true,		
  callbacks:[logCallback]
});

Contents