When Should You Use It?

Deep learning, which is a subset of AI (Artificial Intelligence), has been around since the 1950s. It’s focused on developing systems that mimic the brain’s neural network structure.

Yet it was not until the 1980s that deep learning started to show promise, spurred by the pioneering theories of researchers like Geoffrey Hinton, Yoshua Bengio and Yann Lecun. There was also the benefit of accelerating improvements in computer power.

Despite all this, there remained lots of skepticism. Deep learning approaches still looked more like interesting academic exercises that were not ready for prime time.

But this all changed in a big way in 2012, when Hinton, Ilya Sutskever, and Alex Krizhevsky used sophisticated deep learning to recognize images in an enormous dataset. The results were stunning, as they blew away previous records. So began the deep learning revolution.

Nowadays if you do a cursory search of the news for the phrase “deep learning” you’ll see hundreds of mentions. Many of them will be from mainstream publications.

Yes, it’s a case of a 60-plus-year-old overnight success story. And it is certainly well deserved.

But of course, the enthusiasm can still stretch beyond reality. Keep in mind that deep learning is far from a miracle technology and does not represent the final stages of true AI nirvana. If anything, the use cases are still fairly narrow and there are considerable challenges.

“Deep learning is most effective when there isn’t an obvious structure to the data that you can exploit and build features around,” said Dr. Scott Clark, who is the co-founder and CEO of SigOpt. “Common examples of this are text, video, image, or time series datasets. The great thing about deep learning is that it will automatically build and exploit patterns in the data in order to make better decisions. The downside is that this can sometimes take a lot of data and a lot of compute resources to converge to a good solution. It tends to be the most effective in places where there is a lot of data, a lot of compute power, and there is a need for the best possible solution.”

True, it is getting easier to use deep learning. Part of this is due to the ubiquity of open source platforms like TensorFlow and PyTorch. Then there is the emergence of cloud-based AI Systems, such as Google’s AutoML.

But such things only go so far. “Each neural network model has tens or hundreds of hyperparameters, so turning and optimizing these parameters requires deep knowledge and experiences from human experts,” said Jisheng Wang, who is the head of data science at Mist. “Interpretability is also a big challenge when using deep learning models, especially for enterprise software, which prefers to keep humans in the loop. While deep learning reduces the human effort of feature engineering, it also increases the difficulty for humans to understand and interpret the model. So in certain applications where we require human interaction and feedback for continuous improvement, deep learning may not be the appropriate choice.”

However, there are alternatives that may not be as complex, such as traditional machine learning. “In cases with smaller datasets and simpler correlations, techniques like KNN or random forest may be more appropriate and effective,” said Sheldon Fernandez, who is the CEO of DarwinAI.

Now this is not to somehow imply that you should mostly shun deep learning. The technology is definitely powerful and continues to show great progress (just look at the recent innovation of Generative Adversarial Networks or GANs).  Many companies — from mega operators like Google to early-stage startups — are also focused on developing systems to make the process easier and more robust.

But as with any advanced technology, it needs to be treated with care. Even experts can get things wrong.  “A deep learning model might easily get a problematic or nonsensical correlation,” said Sheldon,. “That is, the network might draw conclusions based on quirks in the dataset that are catastrophic from a practical point of view.”

Tom serves on the advisory boards of tech startups and can be reached at his site.