Neural networks are simulation of biological neurons. Similar to dendrites in biological neurons, channels of data are responsible for carrying data between neurons. The strength of these channels is referred to as the weights associated with them.
A Simple Neuron
A simple biological neuron has 3 major parts:
- Dendrites : Act as the channel that gets signals to cell body with their strength as weights
- Cell Body : Collects input through dendrites and processes to produce output
- Axon : Responsible for transmitting signals to other neurons
Trending AI Articles:
1. Introducing Open Mined: Decentralised AI
2. TensorFlow Object Detection API tutorial
3. TensorFlow Object Detection API: basics of detection (2/2)
4. Part-of-Speech tagging tutorial with the Keras Deep Learning library
Inspired from the structure of a biological neuron a single neuron can be explained though the following diagram:
The input brought by the input channels is summed or accumulated (Σ), further processing an output through the [f(Σ)] .
An American psychologist, William James came up with two important aspects of neural models which later on became the basics of neural networks :
- If two neurons are active in immediate succession or together, then they will excite each other on re-occurrence which will in turn grow the intensity of the connection between them
- The sum of signals a neuron receives accounts for it’s activity, where signals are proportional to the strength of the connection the signal passes through.
Simplest Neural Network
An Artificial Neuron is a simple model developed with approximation of a biological neuron by McCulloch-Pitts in 1940. It uses a threshold function to produce an output of either 0 or 1 and act as a classifier. This model has fixed weights and does not learn. This is the most simplest neural network from a biological perspective.
Consider a threshold neuron with ‘x¹’ and ‘x²’ as input and ‘t’ being the expected target output.
The neuron has weights ‘w¹’ and ‘w²’ respectively for the channels of input data. For calculating the output the net input ‘u’ is calculated as :
The net input for all such sets of inputs is calculated and then an arbitrary value of the threshold is chosen which lies between the lowest and the highest net input calculated. Let’s assume that a threshold of ‘u⁰’ is chosen.
The final output ‘y’ will be done in the following manner :
The classification is done on the basis of the value of threshold into two different categories. The above threshold function that classifies inputs using a simple neuron.
The neural networks we see today for multiple implementations draws inspiration from biological neurons. A threshold neuron is such simplest structure, which doesn’t adjust its weight and hence cannot learn. It can be used as a classifier on the basis of it’s threshold.