Machine Learning Algorithms
Basically, there are two ways to categorize Machine Learning algorithms you may come across in the field.
- The first is a grouping of ML algorithms by the learning style.
- The second is a grouping of ML algorithms by a similarity in form or function.
Generally, both approaches are useful. Although, we will focus in on the grouping of ML algorithms by similarity.
Trending AI Articles:
1. Deep Learning Book Notes, Chapter 1
2. Deep Learning Book Notes, Chapter 2
3. Machines Demonstrate Self-Awareness
4. Visual Music & Machine Learning Workshop for Kids
Further, go on a tour of a variety of different algorithm types.
a. Machine Learning Algorithms Grouped by Learning Style
Basically, there are different ways an algorithm can model a problem. Also, as it relates to the interaction with the experience. Although, it doesn’t matter whatever we want to call the input data.
Also, an algorithm is popular in machine learning and artificial intelligence textbooks. That is to first consider the learning styles that an algorithm can adapt.
Generally, there are only a few main learning styles that Machine Learning algorithms can have. And, also we’ll go through them. Also, we have few examples of ML algorithms and problem types that they suit.
Basically, this way of organizing machine learning algorithms is very useful. Because it forces you to think about the roles of the input data and the model preparation process. Also, to select one that is the most appropriate for your problem to get the best result.
Let’s take a look at three different learning styles in machine learning algorithms:
i. Supervised Learning
Basically, in this Supervised MLAlgorithms, input data is called training data and has a known label or result such as spam/not-spam or a stock price at a time.
In this, a model is prepared through a training process. Also, this required to make predictions. And is corrected when those predictions are wrong. The training process continues until the model achieves the desired level.
- Example problems are classification and regression.
- Example algorithms include logistic regression and back propagation Neural Network.
ii. Unsupervised Learning
In this Unsupervised Machine Learning, input data is not labeled and does not have a known result.
We have to prepare a model by deducing structures present in the input data. This may be to extract general rules. It may be through a mathematical process to reduce redundancy.
- Example problems are clustering, dimensionality reduction, and association rule learning.
- Example algorithms include the Apriori algorithm and k-Means.
iii. Semi-Supervised Learning
Input data is a mixture of labeled and unlabeled examples.
There is a desired prediction problem. But the model must learn the structures to organize the data as well as make predictions.
- Example problems are classification and regression.
- Example algorithms are extensions to other flexible methods. That make assumptions about how to model the unlabeled data.
b. Machine Learning Algorithms Grouped By Similarity
ML Algorithms are often grouped by a similarity in terms of their function.
For example, tree-based methods, and the neural network inspired methods.
I think this is the most useful way to group machine learning algorithms and it is the approach we will use here.
This is a useful grouping method, but it is not perfect. There are still algorithms that could just as easily fit into multiple categories. Such as Learning Vector Quantization. That is both a neural network method and an instance-based method. There are also categories that have the same name. That describes the problem and the class of algorithms. Such as Regression and Clustering.
We could handle these cases by listing ML algorithms twice. Either by selecting the group that subjectively is the “best” fit. I like this latter approach of not duplicating algorithms to keep things simple.
i. Regression Algorithms
Regression Algorithms is concerned with modeling the relationship between variables. That we use to refine using a measure of error in the predictions made by the model.
These methods are a workhorse of statistics. Also, have been co-opted into statistical machine learning. This may be confusing because we can use regression to refer to the class of problem and the class of algorithm.
The most popular regression algorithms In Machine Learning are:
- Ordinary Least Squares Regression (OLSR)
- Linear Regression
- Logistic Regression
- Stepwise Regression
- Multivariate Adaptive Regression Splines (MARS)
- Locally Estimated Scatterplot Smoothing (LOESS)
ii. Instance-based Algorithms
This model is a decision problem with instances of training data. That is deemed important or required to the model.
Such methods build up a database of example data. And it needs to compare new data to the database. For comparison, we use a similarity measure to find the best match and make a prediction. For this reason, instance-based methods are also called winner-take-all methods and memory-based learning. The focus is put on the representation of the stored instances. Thus, similarity measures used between instances.
The most popular instance-based algorithms in Machine Learning are:
- k-Nearest Neighbor (kNN)
- Learning Vector Quantization (LVQ)
- Self-Organizing Map (SOM)
- Locally Weighted Learning (LWL)
iii. Regularization Algorithms
An extension made to another method. That is penalizing models which relate to their complexity. Also, favoring simpler models that are also better at generalizing.
I have listed regularization algorithms here because they are popular, powerful. And generally simple modifications made to other methods.
The most popular regularization algorithms in Machine Learning are:
- Ridge Regression
- Least Absolute Shrinkage and Selection Operator (LASSO)
- Elastic Net
- Least-Angle Regression (LARS)
iv. Decision Tree Algorithms
Decision tree methods construct a model of decisions. That is made based on actual values of attributes in the data.
Decisions fork in tree structures until a prediction decision is made for a given record. Decision trees are trained on data for classification and regression problems. Decision trees are often fast and accurate and a big favorite in machine learning.
The most popular decision tree algorithms in Machine Learning are:
- Classification and Regression Tree (CART)
- Iterative Dichotomiser 3 (ID3)
- C4.5 and C5.0 (different versions of a powerful approach)
- Chi-squared Automatic Interaction Detection (CHAID)
- Decision Stump
- Conditional Decision Trees
v. Bayesian Algorithms
These methods are those that apply Bayes’ Theorem for problems. Such as classification and regression.
The most popular Bayesian algorithms in Machine Learning are:
- Naive Bayes
- Gaussian Naive Bayes
- Multinomial Naive Bayes
- Averaged One-Dependence Estimators (AODE)
- Bayesian Belief Network (BBN)
- Bayesian Network (BN)
vi. Clustering Algorithms
Clustering, like regression, describes the class of problem and the class of methods.
The Clustering methods are organized by the modeling approaches such as centroid-based and hierarchal. All methods are concerned with using the inherent structures in the data. That is a need to best organize the data into groups of maximum commonality.
The most popular clustering algorithms in Machine Learning are:
- Expectation Maximisation (EM)
- Hierarchical Clustering
vii. Association Rule Learning Algorithms
Association rule learning methods extract rules. That best explain observed relationships between variables in data.
These rules can discover important and useful associations in large multidimensional datasets. That can be exploited by an organization.
The most popular association rule learning algorithms in Machine Learning are:
- Apriori algorithm
- Eclat algorithm
viii. Artificial Neural Network Algorithms
These are models that are inspired by the structure of biological neural networks.
They are a class of pattern matching. That we use for regression and classification problems. Although, there is an enormous subfield. As it combines hundreds of algorithms and variations.
The most popular artificial neural network algorithms are:
- Hopfield Network
- Radial Basis Function Network (RBFN)
ix. Deep Learning Algorithms
ML — Deep Learning Algorithms
Deep Learning methods are a modern update to Artificial Neural Networks. That is exploiting abundant cheap computation.
They are concerned with building much larger and more complex neural networks.
The most popular deep learning algorithms are:
- Deep Boltzmann Machine (DBM)
- Deep Belief Networks (DBN)
- Convolutional Neural Network (CNN)
- Stacked Auto-Encoders
x. Dimensionality Reduction Algorithms
Dimensionality Reduction Algorithms
Like clustering methods, dimensionality reduction seeks inherent structure in the data. Although, in this case, to order to summarize.
Generally, it can be useful to visualize dimensional data. Also, we can use it in a supervised learning method. Many of these methods we adopt for use in classification and regression.
- Principal Component Analysis (PCA)
- Principal Component Regression (PCR)
- Partial Least Squares Regression (PLSR)
- Sammon Mapping
- Multidimensional Scaling (MDS)
- Projection Pursuit
- Linear Discriminant Analysis (LDA)
- Mixture Discriminant Analysis (MDA)
- Quadratic Discriminant Analysis (QDA)
- Flexible Discriminant Analysis (FDA)
xii. Ensemble Algorithms
Basically, these methods are models composed of weaker models. Also, as they are trained and whose predictions are combined in some way to make the prediction.
Moreover, much effort is put into what types of weak learners to combine and the ways in which to combine them. Hence, this is a very powerful class of techniques and as such is very popular.
- Bootstrapped Aggregation (Bagging)
- Stacked Generalization (blending)
- Gradient Boosting Machines (GBM)
- Gradient Boosted Regression Trees (GBRT)
- Random Forest
List of Common Machine Learning Algorithms
a. Naïve Bayes Classifier Machine Learning Algorithm
Generally, it would be difficult and impossible to classify a web page, a document, an email. Also, other lengthy text notes manually. This is where the Naïve Bayes Classifier machine learning algorithm comes to the rescue. Also, a classifier is a function that allocates a population’s element value. For instance, Spam Filtering is a popular application of the Naïve Bayes algorithm. Thus, the spam filter here is a classifier that assigns a label “Spam” or “Not Spam” to all the emails.
Basically, it is amongst the most popular learning method grouped by similarities. That works on the popular Bayes Theorem of Probability. It is a simple classification of words. Also, is defined for the subjective analysis of content.
b. K Means Clustering Machine Learning Algorithm
Generally, K-means is a used unsupervised machine learning algorithm for cluster analysis. Also, K-Means is a non-deterministic and iterative method. Besides, the algorithm operates on a given data set through a pre-defined number of clusters, k. Thus, the output of K Means algorithm is k clusters with input data that is separated among the clusters.
c. Support Vector Machine Learning Algorithm
Basically, it is a supervised machine learning algorithm for classification or regression problems. As in this, the dataset teaches SVMabout the classes. So that SVM can classify any new data. Also, it works by classifying the data into different classes by finding a line. That we use to separates the training data set into classes. Moreover, there are many such linear hyperplanes. Further, in this, SVM tries to maximize the distance between various classes. As that has to involve and this is referred to as margin maximization. Also, if the line that maximizes the distance between the classes is identified. Then the probability to generalize well to unseen data is increased.
SVM’s are classified into two categories:
- Linear SVM’s — Basically, in linear SVM, ’s the training data i.e. have to separate classifier by a hyperplane.
- Non-Linear SVM’s- Basically, in non-linear SVM’s it is not possible to separate the training data using a hyperplane.
d. Apriori Machine Learning Algorithm
Basically, it is an unsupervised machine learning algorithm. That we use to generate association rules from a given data set. Also, association rule implies that if an item A occurs, then item B also occurs with a certain probability. Moreover, most of the association rules generated are in the IF_THEN format.
For example, IF people buy an iPad THEN they also buy an iPad Case to protect it.
The basic principle on which the Apriori Machine Learning Algorithm works:
If an item set occurs frequently then all the subsets of the item set, also occur frequently.
If an item set occurs infrequently. Then all the supersets of the item set have infrequent occurrence.
e. Linear Regression Machine Learning Algorithm
It shows the relationship between 2 variables. Also, shows how the change in one variable impacts the other. Basically, the algorithm shows the impact on the dependent variable. That depends on changing the independent variable. Thus, the independent variables as explanatory variables. As they explain the factors impact the dependent variable. Moreover, a dependent variable has often resembled the factor of interest or predictor.
f. Decision Tree Machine Learning Algorithm
Basically, a decision tree is a graphical representation. That makes use of branching method to exemplify all possible outcomes of a decision.
Basically, in a decision tree, the internal node represents a test on the attribute. As each branch of the tree represents the outcome of the test. And also the leaf node represents a particular class label. i.e. the decision made after computing all the attributes. Further, we have to represent classification through the path from a root to the leaf node.
g. Random Forest Machine Learning Algorithm
It is the go-to machine learning algorithm. That we use a bagging approach to create a bunch of decision trees with a random subset of the data. Although, we have to train a model several times on a random sample of the dataset. That need to achieve good prediction performance from the random forest algorithm. Also, in this ensemble learning method, we have to combine the output of all the decision tree. That is to make the final prediction. Moreover, we derive the final prediction by polling the results of each decision tree.
h. Logistic Regression Machine Learning Algorithm
Generally, the name of this algorithm could be a little confusing. Logistic Regression algorithm is for classification tasks and not regression problems. Also, the name ‘Regression’ here implies that a linear model is fit into the feature space. Further, this algorithm applies a logistic function to a linear combination of features. That need to predict the outcome of a categorical dependent variable. Moreover, it was based on predictor variables.
The probabilities that describe the outcome of a single trial are modeled as a function. Also, the function of explanatory variables.
So, this was all about Machine Learning Algorithms. Hope you like our explanation.
We have studied Machine Learning Algorithms. Also, learned categorization of ML Algorithms. We have also used images that make easy to understand Machine Learning Algorithm. Furthermore, if you feel any query, feel to ask in a comment section.