It is believed that deep learning was invented at the dawn of 21st-century, but believe it or not, it has originated since the 1940s.
The reason most of us are unaware of deep learning advancements that were developed in the 20th century is due to the fact that the approaches used then were relatively unpopular due to their various shortcomings and the fact that it had a couple of revitalizations since then.
There were THREE Waves:
Cybernetics — During 1940–1960
Connectionism — During 1980–1990
Deep Learning — Since 2006
The first 2 waves were unpopular due to the critics of their shortcomings, however, there is no doubt that it has helped advance the field to where it is today and some of the algorithms developed during those times are used widely till today in various machine learning and deep learning models.
1. Natural Language Generation:
The Commercial State of the Art in 20202. This Entire Article Was Written by Open AI’s GPT2
3. Learning To Classify Images Without Labels
4. Becoming a Data Scientist, Data Analyst, Financial Analyst and Research Analyst
After two dips, the third wave emerged in 2006 with a breakthrough. The advancements by Geoffrey Hinton were used by other researchers to train different types of Deep Networks. This enabled researchers around the world to Train Deeper and Deeper Neural Networks and led to the popularisation of the term Deep Learning.
Credit: BecomingHuman By: Hexaberry Data Science Community
