What is the sigmoid function, and what is its use in machine learning's neural networks?

On the field of Artificial Neural Networks, the sigmoid funcion is a type of activation function for artifical neurons.

The most basic activation funciton is the Heaviside (binary step, 0 or 1, high or low):



The Sigmoid function (a special case of the logistic function) and its formula looks like:



You can have several types of activation functions and they are best suitable for different purposes. In the specific case of the Sigmoid:

  • Real-valued and differentiable (you need this to find gradients);

  • Analytic tractability for the differentiaton operation;

  • It is an acceptable mathematical representation of a biological neuron behaviour. The output shows if the neuron is firing or not.

There is a table with the various types of activation functions:

Activation function

Most of the time you will be concerned about the following points when choosing the activation function:

  • continuity of the function

  • computational power to process all neurons of the network

  • type of the desired output (logistic/continuous variables or classification/categorical data)


Posts Em Destaque
Posts Recentes
Arquivo
Procurar por tags
Siga
  • Facebook Basic Square
  • Twitter Basic Square
  • Google+ Basic Square