Troubleshooting Sequential Models

Overview


It is sometimes hard to decipher the errors that tensor-flow outputs. Usually, the problem that occurs is that the shapes of the tensors created in code do not match tensor-flows expectations about those shapes.

It is important to understand how tensor-flow thinks about its process. As in any neural network, tensor-flow applies the weights matrix to the inputs to arrive at a set of outputs. However, it does not do this:
{% outputs = Activation(Weights \times Inputs) %}
Rather, it does this
{% outputs = Activation(Inputs \times Weights) %}
That is, it multiplies the inputs from the left. As an example, suppose you had this model 



const model = tf.sequential({
  layers:[
    tf.layers.dense({
      inputShape:[4],
      units:2,
    }),
  ]
});


It expects 4 inputs. However, the inputs should be specified like the following, as a row vector 


let input = tf.tensor([[1, 2, 1, 1]]);


This is a 2-d matrix with a single row.

The weights for the model would be a matrix as follows 


weights = [[0.5,-0.2],[0.5,0.8],[0.7,0.3],0.3,0.1]]


That is, it would be 4x2 matrix. When left multiplied by the inputs, the result is a 1X2 matrix, which is ready to multiply the next set of weights.

Contents