Thursday, April 15, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Neural Networks

How do pretrained models work? – Becoming Human: Artificial Intelligence Magazine

April 9, 2019
in Neural Networks
How do pretrained models work? – Becoming Human: Artificial Intelligence Magazine
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Credit: BecomingHuman

The first thing we want to understand is how a neural network works. All of us know that a neural network is a collection of neurons and activation functions. The first set of neurons is called the input layer, the last set is called the output layer and the middle ones are called hidden layers.

You might also like

Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021

How to Enter Your First Zindi Competition | by Davis David

Why I Think That Avengers: Age of Ultron is One of the Best Sci-Fi Movies About A.I | by Brighton Nkomo | Apr, 2021

Trending AI Articles:

1. Deep Learning Book Notes, Chapter 1

2. Deep Learning Book Notes, Chapter 2

3. Machines Demonstrate Self-Awareness

4. Visual Music & Machine Learning Workshop for Kids

When we train a neural network, the initial layers of a neural network can identify very simple things. Say a straight line or a slant one. Something really basic.

As we go deeper into our network, we can identify more sophisticated things as shown below.

Layer 2 can identify shapes like squares or circles.

Layer 3 can identify intricate patterns.

And finally, the deepest layers can identify things like dog faces. These things can be identified because the weights of our model are set to a certain value.

Resnet34 is one such model. It is trained in to classify a 1000 images. Now think about this. If you want to make a classifier, any classifier, the initial layers are going to detect slant lines no matter what you classify. It is really the final layers that learn to identify sophisticated stuff that need training.

Hence what we do is, we take Resnet34 and add some more layers to it. Let’s take a look at the corresponding code snippets to understand both things together.

Initially, we only train the added layers. We do so because the weights of these layers are initialized to random values. The layers of Resnet34 are freezed and undergo no training.

Once we’ve trained the last layers a little, we can unfreeze the layers of Resnet34. We then find the learning rate and train the whole model.

Our learning rate plot is as follows:

We choose a value for our learning rate just before when the graph starts to rise (1e-04 here). The other option, and the one I have used is to select a slice.

This means that if we had only 3 layers in our network, the first would train at a learning rate = 1e-6, the second at 1e-5 and the last one at 1e-4. In our model, the layers of Resnet34 don’t require much training and can train at a lower learning rate, while the newly added models need to be trained at a slightly higher learning rate. Hence the slice.

This concept of training different parts of a neural network at different learning rates is called discriminative learning, and is a relatively new concept in deep learning.

We continue this process of unfreezing the layers, finding the learning rate and training some more till our training loss is less than our validation loss and we are sure we are not overfitting.

Improving the performance

One trick to improve the performance of a model is to train a model for lower resolution images (size = 128) and use those weights as initial values for higher resolution images. I’ve done the same in this notebook. And the performance of my model increased by a good 2%.

Now an increase from 92% to 94% may not sound like such a big deal but if we are dealing with medical applications we want to be as accurate as possible. And it’s these small tricks that separate the good models from the competition winning models. If you know any such tricks, mention them in the comments section below.

If you liked this article, give it at least 50 claps :p

~Happy learning.

Don’t forget to give us your 👏 !

Credit: BecomingHuman By: Dipam Vasani

Previous Post

US regulators dash Amazon hopes to stop investor vote on gov't facial recognition tech sales

Next Post

AI, Deep Learning Start to Tackle Common Problems in Healthcare

Related Posts

Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021
Neural Networks

Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021

April 14, 2021
How to Enter Your First Zindi Competition | by Davis David
Neural Networks

How to Enter Your First Zindi Competition | by Davis David

April 14, 2021
Why I Think That Avengers: Age of Ultron is One of the Best Sci-Fi Movies About A.I | by Brighton Nkomo | Apr, 2021
Neural Networks

Why I Think That Avengers: Age of Ultron is One of the Best Sci-Fi Movies About A.I | by Brighton Nkomo | Apr, 2021

April 14, 2021
Music and Artificial Intelligence | by Ryan M. Raiker, MBA | Apr, 2021
Neural Networks

Music and Artificial Intelligence | by Ryan M. Raiker, MBA | Apr, 2021

April 13, 2021
BERT Transformers — How Do They Work? | by James Montantes | Apr, 2021
Neural Networks

BERT Transformers — How Do They Work? | by James Montantes | Apr, 2021

April 13, 2021
Next Post
AI, Deep Learning Start to Tackle Common Problems in Healthcare

AI, Deep Learning Start to Tackle Common Problems in Healthcare

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization
Machine Learning

Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization

April 14, 2021
Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021
Neural Networks

Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021

April 14, 2021
The Search Engine Land Awards are open: Wednesday’s daily brief
Digital Marketing

The Search Engine Land Awards are open: Wednesday’s daily brief

April 14, 2021
Six courses to build your technology skills in 2021 – IBM Developer
Technology Companies

IBM joins Eclipse Adoptium and offers free certified JDKs with Eclipse OpenJ9 – IBM Developer

April 14, 2021
Cyber criminals are installing cryptojacking malware on unpatched Microsoft Exchange servers
Internet Security

Cyber criminals are installing cryptojacking malware on unpatched Microsoft Exchange servers

April 14, 2021
Simplify, then Add Lightness – Consolidating the Technology to Better Defend Ourselves
Internet Privacy

Simplify, then Add Lightness – Consolidating the Technology to Better Defend Ourselves

April 14, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • Sailthru Announces Machine Learning Features for Improved Lifecycle Optimization April 14, 2021
  • Data Labeling Service — How to Get Good Training Data for ML Project? | by ByteBridge | Apr, 2021 April 14, 2021
  • The Search Engine Land Awards are open: Wednesday’s daily brief April 14, 2021
  • IBM joins Eclipse Adoptium and offers free certified JDKs with Eclipse OpenJ9 – IBM Developer April 14, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates