What is the sigmoid function, and what is its use in machine learning's neural networks?
On the field of Artificial Neural Networks, the sigmoid funcion is a type of activation function for artifical neurons.
The most basic activation funciton is the Heaviside (binary step, 0 or 1, high or low):
The Sigmoid function (a special case of the logistic function) and its formula looks like:
You can have several types of activation functions and they are best suitable for different purposes. In the specific case of the Sigmoid:
Real-valued and differentiable (you need this to find gradients);
Analytic tractability for the differentiaton operation;
It is an acceptable mathematical representation of a biological neuron behaviour. The output shows if the neuron is firing or not.
There is a table with the various types of activation functions:
Activation function
Most of the time you will be concerned about the following points when choosing the activation function:
continuity of the function
computational power to process all neurons of the network
type of the desired output (logistic/continuous variables or classification/categorical data)
