Monday, April 12, 2021
  • Setup menu at Appearance » Menus and assign menu to Top Bar Navigation
Advertisement
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News
No Result
View All Result
NikolaNews
No Result
View All Result
Home Data Science

Importance of Hyper-parameters in Model development

January 17, 2020
in Data Science
Discover how machine learning can solve finance industry challenges by Jannes Klaas
586
SHARES
3.3k
VIEWS
Share on FacebookShare on Twitter

Machine Learning (ML) development is an iterative process in which the accuracy of predictions made by the models is continuously improved by repeating the training and evaluation phases. In each of these iterations, certain parameters are tweaked continuously by developers. Any parameter manually selected based on learning from previous experiments qualify to be called a model hyper-parameter. These parameters represent intuitive decisions whose value cannot be estimated from data or from ML theory. The hyper-parameters are knobs that you tweak during each iteration of training a model to improve the accuracy in the predictions made by the model. The hyper-parameters are variables that govern the training process itself. They are often specified by practitioners experienced in machine learning development. They are often tuned independently for a given predictive modeling problem.

Building an ML model is a long process that requires domain knowledge, experience and intuition. In ML, hyper-parameter optimization or tuning is the problem of choosing a set of optimal hyper-parameters for a learning algorithm. We may not know the best combination of values for hyper-parameters in advance for a given problem. We may use rules of thumb, copy values used on other problems, or search for the best value by trial and error. When a machine learning algorithm is tuned for specific problems by changing the higher level APIs for optimization, we need to tune the hyper-parameters also to discover the parameters that results in a model with higher accuracy in prediction. Hyper-parameter tuning is often referred to as searching the parameter space for optimum values. With Deep Learning models, the search space is usually very large, and a single model might take days to train. The common Hyper-parameters are:

You might also like

Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027

Interpretive Analytics in One Picture

Job Scope For MSBI In 2021

  • Epochs – A full training pass over the entire dataset such that each example has been seen once.
  • Learning rate – A scalar used to train a model via gradient descent. During each iteration, the gradient descent algorithm multiplies learning rate by the gradient. Resulting product is called the gradient step.
  • Momentum in Stochastic Gradient Descent – The coefficient of friction controlling the rate at which the descent happens, when it goes towards the bottom.
  • Regularization method – Regularization is used to prevent overfitting by the model. Different kinds of regularization include L1 regularization (Lasso) and L2 regularization (Ridge)
  • Regularization Rate – The penalty on a model’s complexity. The scalar value ƛ specifies the importance of the regularization function relative to the loss function. Raising the value of ƛ reduces over-fitting at the cost of model accuracy.
  • Early Stopping – Regularization by early stopping callback function tests a training condition for every epoch and if a set number of epochs elapses without showing any improvement, then it automatically stops the training.
  • The patience parameter is the number of epochs to check for improvement.
  • K in k-means clustering – Number of clusters to be discovered
  • C and Sigma – For Support Vector Machines
  • Number of hidden layers – For Neural Networks
  • Number of units per layer – For Neural Networks
  • max_depth – Maximum depth of a tree in Random Forest method
  • n_estimators- Number of trees in the Random Forest. More number of trees gives better performance

Model optimization using hyper-parameter tuning is a search problem to identify the ideal combination of these parameters. The commonly used methods for optimization using hyper-parameters are; Grid search, Random search and Beyesian optimization. In Grid search, a list of all possible values for each hyper-parameter in a specified range is constructed and all possible combinations of these values are tried sequentially. In grid search, the number of experiments to be carried out increases drastically with the number of hyper-parameters. Rather than training on all possible configurations, in Random search method the network is trained only on a subset of the configurations. Choice of the configurations to be trained is randomly picked up and only the best configuration is trained in each iteration. In Beyesian optimization, we are using ML techniques to figure out the hyper-parameters. It predicts regions of the hyper-parameter space that might give better results. Gaussian process is the technique used and it finds out the optimal hyper parameters from the results of the previously conducted experiments with various types of parameter configurations.


Credit: Data Science Central By: Janardhanan PS

Previous Post

AI Innovation in Manufacturing, Robotics, New Apps on Display at CES

Next Post

Equifax direct payments to members to end class action could top $500 million

Related Posts

Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027
Data Science

Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027

April 12, 2021
Interpretive Analytics in One Picture
Data Science

Interpretive Analytics in One Picture

April 12, 2021
Job Scope For MSBI In 2021
Data Science

Job Scope For MSBI In 2021

April 11, 2021
Leveraging SAP’s Enterprise Data Management tools to enable ML/AI success
Data Science

Leveraging SAP’s Enterprise Data Management tools to enable ML/AI success

April 11, 2021
Vue.js vs AngularJS Development in 2021: Side-by-Side Comparison
Data Science

Vue.js vs AngularJS Development in 2021: Side-by-Side Comparison

April 10, 2021
Next Post
Equifax direct payments to members to end class action could top $500 million

Equifax direct payments to members to end class action could top $500 million

Leave a Reply Cancel reply

Your email address will not be published. Required fields are marked *

Recommended

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

Plasticity in Deep Learning: Dynamic Adaptations for AI Self-Driving Cars

January 6, 2019
Microsoft, Google Use Artificial Intelligence to Fight Hackers

Microsoft, Google Use Artificial Intelligence to Fight Hackers

January 6, 2019

Categories

  • Artificial Intelligence
  • Big Data
  • Blockchain
  • Crypto News
  • Data Science
  • Digital Marketing
  • Internet Privacy
  • Internet Security
  • Learn to Code
  • Machine Learning
  • Marketing Technology
  • Neural Networks
  • Technology Companies

Don't miss it

IIT Hyderabad Offers Interdisciplinary PhD in Artificial Intelligence, Machine Learning and Information Theory
Machine Learning

IIT Hyderabad Offers Interdisciplinary PhD in Artificial Intelligence, Machine Learning and Information Theory

April 12, 2021
Ransomware: The internet’s biggest security crisis is getting worse. We need a way out
Internet Security

Ransomware: The internet’s biggest security crisis is getting worse. We need a way out

April 12, 2021
Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027
Data Science

Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027

April 12, 2021
Hawaiʻi’s Keck Observatory Aids in Discovery of Rare “Quadruply Imaged Quasars”
Machine Learning

Hawaiʻi’s Keck Observatory Aids in Discovery of Rare “Quadruply Imaged Quasars”

April 12, 2021
Interpretive Analytics in One Picture
Data Science

Interpretive Analytics in One Picture

April 12, 2021
AI and Machine Learning Driven Contract Lifecycle Management for Government Contractors
Machine Learning

AI and Machine Learning Driven Contract Lifecycle Management for Government Contractors

April 12, 2021
NikolaNews

NikolaNews.com is an online News Portal which aims to share news about blockchain, AI, Big Data, and Data Privacy and more!

What’s New Here?

  • IIT Hyderabad Offers Interdisciplinary PhD in Artificial Intelligence, Machine Learning and Information Theory April 12, 2021
  • Ransomware: The internet’s biggest security crisis is getting worse. We need a way out April 12, 2021
  • Data Center Infrastructure Market is Projected to Reach USD 100 Billion by 2027 April 12, 2021
  • Hawaiʻi’s Keck Observatory Aids in Discovery of Rare “Quadruply Imaged Quasars” April 12, 2021

Subscribe to get more!

© 2019 NikolaNews.com - Global Tech Updates

No Result
View All Result
  • AI Development
    • Artificial Intelligence
    • Machine Learning
    • Neural Networks
    • Learn to Code
  • Data
    • Blockchain
    • Big Data
    • Data Science
  • IT Security
    • Internet Privacy
    • Internet Security
  • Marketing
    • Digital Marketing
    • Marketing Technology
  • Technology Companies
  • Crypto News

© 2019 NikolaNews.com - Global Tech Updates