A multi layer perceptron (MLP) is a class of feed forward artificial neural network. MLP consists of at least three layers of nodes: an input layer, a hidden layer and an output layer. Except for the input nodes, each node is a neuron that uses a nonlinear activation function. MLP utilizes a supervised learning technique called back propagation for training.(Wikipedia)
Though the concept has been alive since 1980s, a renewed interest in MLP has resurfaced because of deep learning as a methodology which often comes up with better prediction rates on financial services data than some of the other leaning methods like logistic regression and decision trees.I tried creating a practical manifestation of this concept using a real financial services data set to check if at all the MLP models do provide a descent classification of binary problems.
Trending AI Articles:
1. 100 days to Deep Learning: Part 2 the 100 days
2. Bursting the Jargon bubbles — Deep Learning
3. How Can We Improve the Quality of Our Data?
4. Machine Learning using Logistic Regression in Python with Code
The data is related with direct marketing campaigns (phone calls) of a Portuguese banking institution. The classification goal is to predict if the client will subscribe a term deposit. This data can be found here in this link
Basic Data Exploration & Management
We see there is a significant amount of categorical values which needs to be converted into numerical ones.For purpose of this article and data set we will be using Label encoding from scikit learn which is one of the most powerful yet the simplest encoding available in open source libraries. Also need to drop the not so important columns to make the data more meaningful.
import pandas as pd
df = pd.read_csv('bank-additional-full.csv',sep=';')
from sklearn.preprocessing import LabelEncoder
LE = LabelEncoder()
#Work on the Categorical Values
df['job_code'] = LE.fit_transform(df['job'])
df['marital_code'] = LE.fit_transform(df['marital'])
df['education_code'] = LE.fit_transform(df['education'])
df['housing_code'] = LE.fit_transform(df['housing'])
df['loan_code'] = LE.fit_transform(df['loan'])
df['contact_code'] = LE.fit_transform(df['contact'])
df['poutcome_code'] = LE.fit_transform(df['poutcome'])
df['subscribed'] = LE.fit_transform(df['y'])
# Drop columns
MLP Neural Network Model Creation
Now that the data is a descent shape to be put into some kind of a ML model, it is important that data is normalized before being fed into a model.We shall be using StandardScaler from scikit learn library along with MLP classifier for model creation.
from sklearn.model_selection import train_test_split
X_train,X_test,y_train,y_test = train_test_split(X,y)
from sklearn.preprocessing import StandardScaler
scaler = StandardScaler()
# Fit the Training Data
MLP classifier models take at least 3 layers . For simplicity sake we shall be taking 3 layers — ( 13 input, 10 hidden & 2 output) with maximum iterations of 100. This parameters can be fine tuned later on based on domain and data to improve the accuracy.
X_train = scaler.transform(X_train)
X_test = scaler.transform(X_test)
from sklearn.neural_network import MLPClassifier
mlp = MLPClassifier(hidden_layer_sizes=(13,10,2),max_iter=1000)
Now that the training data has been fit into the model, it is important that we validate the model for it’s accuracy.
from sklearn.metrics import classification_report,confusion_matrix
we can also find out the accuracy % of the model using accuracy score from sklearn library
from sklearn.metrics import accuracy_score
Summary & Inference
As we can see the accuracy % of this model is ~ 92% which testifies the robustness of MLP classifier as one of the most preferred models for binary classification challenges on financial services data.I tried this same data set over logistic regression and decision tree models but MLP did provide better accuracy percentages for this data set.
Thanks for the read — if you found this article interesting and would like to stay in touch, you can find me on Twitter here or LinkedIn
- Data Set — https://archive.ics.uci.edu/ml/datasets/bank+marketing
- Details of MLP — https://en.wikipedia.org/wiki/Multilayer_perceptron