Javascript Neural Networks - Activation Functions

Overview

Activation functions are functions that are applied to the outputs of a layer in a neural network. (see neural networks)

Activation functions are added to a Javascript neural network model as a separate layer. The library provides a set of predefined activation functions, additional functions can also be implemented and used. (see Implementing Activation Functions )

Adding an Activation Function

Activation functions take their inputs as a column vector, and return a new column vector. There is typically no customization that needs to take place. That is, you do not need to set the number of inputs (because this is set to the outputs of the prior layer) and the number of outputs are usually taken to be the same as the number of inputs.

As such, adding an activation function is as simple as the following, which adds a sigmoid function layer.

Functions

The following demonstrates adding pre-defined activation functions.

• Sigmoid
  
  

  
  layers.push(ba.sigmoid());
  					        

  
  
• ReLu
  
  

  
  layers.push(ba.relu());