Applications of Deep Neural Networks is a free 500 + page book by Jeff Heaton
The contents are as below
The download link is at the bottom of the page
Introduction
Python Preliminaries
- Assignments
- Your Instructor: Jeff Heaton
- Course Resources
- What is Deep Learning
- What is Machine Learning
- Regression
- Classification
- Beyond Classification and Regression
- What are Neural Networks
- Why Deep Learning?
- Python for Deep Learning
- Software Installation
- Python Introduction
- Jupyter Notebooks
- Python Versions
- Module Assignment
- Introduction to Python
- Python Lists, Dictionaries, Sets and JSON
- Lists and Tuples
- Sets
- Maps/Dictionaries/Hash Tables
- More Advanced Lists
- An Introduction to JSON
- File Handling
- Read a CSV File
- Read (stream) a Large CSV File
- Read a Text File
- Read an Image
- Functions, Lambdas, and Map/Reduce
- Map
- Filter
- Lambda
- Reduce
Python for Machine Learning
- Introduction to Pandas
- Missing Values
- Dealing with Outliers
- Dropping Fields
- Concatenating Rows and Columns
- Training and Validation
- Converting a Dataframe to a Matrix
- Saving a Dataframe to CSV
- Saving a Dataframe to Pickle
- Module Assignment
- Categorical and Continuous Values
- Encoding Continuous Values
- Encoding Categorical Values as Dummies
- Target Encoding for Categoricals
- Encoding Categorical Values as Ordinal
- Grouping, Sorting, and Shuffling
- Shuffling a Dataset
- Sorting a Data Set
- Grouping a Data Set
- Apply and Map
- Using Map with Dataframes
- Using Apply with Dataframes
- Feature Engineering with Apply and Map
- Feature Engineering
- Calculated Fields
- Google API Keys
- Other Examples: Dealing with Addresses
Introduction to TensorFlow
- Deep Learning and Neural Network Introduction
- Classification or Regression
- Neurons and Layers
- Types of Neurons
- Input and Output Neurons
- Hidden Neurons
- Bias Neurons
- Context Neurons
- Other Neuron Types
- Why are Bias Neurons Needed?
- Modern Activation Functions
- Linear Activation Function
- Rectified Linear Units (ReLU)
- Softmax Activation Function
- Classic Activation Functions
- Step Activation Function
- Sigmoid Activation Function
- Hyperbolic Tangent Activation Function
- Why ReLU?
- Module Assignment
- Introduction to Tensorflow and Keras
- Why TensorFlow
- Deep Learning Tools
- Using TensorFlow Directly
- TensorFlow Linear Algebra Examples
- TensorFlow Mandelbrot Set Example
- Introduction to Keras
- Simple TensorFlow Regression: MPG
- Introduction to Neural Network Hyperparameters
- Controlling the Amount of Output
- Regression Prediction
- Simple TensorFlow Classification: Iris
- Saving and Loading a Keras Neural Network
- Early Stopping in Keras to Prevent Overfitting
- Early Stopping with Classification
- Early Stopping with Regression
- Extracting Weights and Manual Network Calculation
- Weight Initialization
- Manual Neural Network Calculation
Training for Tabular Data
- Encoding a Feature Vector for Keras Deep Learning
- Generate X and Y for a Classification Neural Network
- Generate X and Y for a Regression Neural Network
- Module Assignment
- Multiclass Classification with ROC and AUC
- Binary Classification and ROC Charts
- ROC Chart Example
- Multiclass Classification Error Metrics
- Calculate Classification Accuracy
- Calculate Classification Log Loss
- Keras Regression for Deep Neural Networks with RMSE
- Mean Square Error
- Root Mean Square Error
- Lift Chart
- Training Neural Networks
- Classic Backpropagation
- Momentum Backpropagation
- Batch and Online Backpropagation
- Stochastic Gradient Descent
- Other Techniques
- ADAM Update
- Methods Compared
- Specifying the Update Rule in Tensorflow
- Error Calculation from Scratch
- Regression
- Classification
Regularization and Dropout
- Introduction to Regularization: Ridge and Lasso
- L and L Regularization
- Linear Regression
- L (Lasso) Regularization
- L (Ridge) Regularization
- ElasticNet Regularization
- Using K-Fold Cross-validation with Keras
- Regression vs Classification K-Fold Cross-Validation
- Out-of-Sample Regression Predictions with K-Fold Cross-Validation
- Classification with Stratified K-Fold Cross-Validation
- Training with both a Cross-Validation and a Holdout Set
- L and L Regularization to Decrease Overfitting
- Drop Out for Keras to Decrease Overfitting
- Benchmarking Regularization Techniques
- Additional Reading on Hyperparameter Tuning
- Bootstrapping for Regression
- Bootstrapping for Classification
- Benchmarking
Convolutional Neural Networks (CNN) for Computer Vision
- Image Processing in Python
- Creating Images (from pixels) in Python
- Transform Images in Python (at the pixel level)
- Standardize Images
- Adding Noise to an Image
- Module Assignment
- Keras Neural Networks for Digits and Fashion MNIST
- Computer Vision
- Computer Vision Data Sets
- MNIST Digits Data Set
- MNIST Fashion Data Set
- CIFAR Data Set
- Other Resources
- Convolutional Neural Networks (CNNs)
- Convolution Layers
- Max Pooling Layers
- TensorFlow with CNNs
- Access to Data Sets – DIGITS
- Display the Digits
- Training/Fitting CNN – DIGITS
- Evaluate Accuracy – DIGITS
- MNIST Fashion
- Display the Apparel
- Training/Fitting CNN – Fashion
- Implementing a ResNet in Keras
- Keras Sequence vs Functional Model API
- CIFAR Dataset
- ResNet V
- CONTENTS ix
- ResNet V
- Using Your Own Images with Keras
- Recognizing Multiple Images with Darknet
- How Does DarkNet/YOLO Work?
- Using YOLO in Python
- Installing YoloV-TF
- Transfering Weights
- Running DarkFlow (YOLO)
- Module Assignment
Generative Adversarial Networks
- Introduction to GANS for Image and Data Generation
- Implementing DCGANs in Keras
- Face Generation with StyleGAN and Python
- Keras Sequence vs Functional Model API
- Generating High Rez GAN Faces with Google CoLab
- Run StyleGan From Command Line
- Run StyleGAN From Python Code
- Examining the Latent Vector
- GANS for Semi-Supervised Training in Keras
- Semi-Supervised Classification Training
- Semi-Supervised Regression Training
- Application of Semi-Supervised Regression
- An Overview of GAN Research
- Select Projects
Kaggle Data Sets
- Introduction to Kaggle
- Kaggle Ranks
- Typical Kaggle Competition
- How Kaggle Competition Scoring
- Preparing a Kaggle Submission
- Select Kaggle Competitions
- Module Assignment
- Building Ensembles with Scikit-Learn and Keras
- Evaluating Feature Importance
- Classification and Input Perturbation Ranking
- Regression and Input Perturbation Ranking
- Biological Response with Neural Network
- What Features/Columns are Important
- Neural Network Ensemble
- Architecting Network: Hyperparameters
- Number of Hidden Layers and Neuron Counts
- Activation Functions
- Advanced Activation Functions
- Regularization: L, L, Dropout
- Batch Normalization
- Training Parameters
- Experimenting with Hyperparameters
- x CONTENTS
- Bayesian Hyperparameter Optimization for Keras
- Current Semester’s Kaggle
- Iris as a Kaggle Competition
- MPG as a Kaggle Competition (Regression)
- Module Assignment
Transfer Learning
- Introduction to Keras Transfer Learning
- Transfer Learning Example
- Module Assignment
- Popular Pretrained Neural Networks for Keras
- DenseNet
- InceptionResNetV and InceptionV
- MobileNet
- MobileNetV
- NASNet
- ResNet, ResNetV, ResNeXt
- VGG and VGG
- Xception
- Transfer Learning for Computer Vision and Keras
- Transfer
- Transfer Learning for Languages and Keras
- Transfer Learning for Keras Feature Engineering
Time Series in Keras
- Time Series Data Encoding
- Module Assignment
- Programming LSTM with Keras and TensorFlow
- Understanding LSTM
- Simple TensorFlow LSTM Example
- Sun Spots Example
- Further Reading for LSTM
- Text Generation with LSTM
- Additional Information
- Character-Level Text Generation
- Image Captioning with Keras and TensorFlow
- Needed Data
- Running Locally
- Clean/Build Dataset From Flickrk
- Choosing a Computer Vision Neural Network to Transfer
- Creating the Training Set
- Using a Data Generator
- Loading Glove Embeddings
- Building the Neural Network
- Train the Neural Network
- Generating Captions
- Evaluate Performance on Test Data from Flickerk
- Evaluate Performance on My Photos
- Module Assignment
- CONTENTS xi
- Temporal CNN in Keras and TensorFlow
- Sun Spots Example – CNN
Natural Language Processing and Speech Recognition
- Getting Started with Spacy in Python
- Installing Spacy
- Tokenization
- Sentence Diagramming
- Stop Words
- WordVec and Text Classification
- Suggested Software for WordVec
- What are Embedding Layers in Keras
- Simple Embedding Layer Example
- Transferring An Embedding
- Training an Embedding
- Natural Language Processing with Spacy and Keras
- Word-Level Text Generation
- Learning English from Scratch with Keras and TensorFlow
- Imports and Utility Functions
- Getting the Data
- Building the Vocabulary
- Building the Training and Test Data
- Compile the Neural Network
- Train the Neural Network
- Evaluate Accuracy
- Adhoc Query
Reinforcement Learning
- Introduction to the OpenAI Gym
- OpenAI Gym Leaderboard
- Looking at Gym Environments
- Render OpenAI Gym Environments from CoLab
- Introduction to Q-Learning
- Introducing the Mountain Car
- Programmed Car
- Reinforcement Learning
- Running and Observing the Agent
- Inspecting the Q-Table
- Keras Q-Learning in the OpenAI Gym
- DQN and the Cart-Pole Problem
- Hyperparameters
- Environment
- Agent
- Policies
- Metrics and Evaluation
- Replay Buffer
- Data Collection
- Training the agent
- Visualization
- KS-Statistic
- Detecting Drift between Training and Testing Datasets by Training
- Using a Keras Deep Neural Network with a Web Application
- Converting Keras to CoreML
- Creating an IOS CoreML Application
- More Reading
- Other Neural Network Techniques
- What is AutoML
- AutoML from your Local Computer
- AutoML from Google Cloud
- A Simple AutoML System
- Running My Sample AutoML Program
- Using Denoising AutoEncoders in Keras
- Function Approximation
- Multi-Output Regression
- Simple Autoencoder
- Autoencode (single image)
- Standardize Images
- Image Autoencoder (multi-image)
- Adding Noise to an Image
- Denoising Autoencoder
- Anomaly Detection in Keras
- Read in KDD Data Set
- Preprocessing
- Training the Autoencoder
- Detecting an Anomaly
- Training an Intrusion Detection System with KDD
- Read in Raw KDD- Dataset
- Analyzing a Dataset
- Encode the feature vector
- Train the Neural Network
- New Technologies
- Neural Structured Learning (NSL)
- Bert, AlBert, and Other NLP Technologies
- Explainability Frameworks
Advanced/Other Topics
- Flask and Deep Learning Web Services
- Flask Hello World
- MPG Flask
- Flask MPG Client
- Images and Web Services
- Part : Deploying a Model to AWS
- Train Model (optionally, outside of AWS)
- Next Step: Convert Model (must use AWS SageMaker Notebook)
- Step Set up
- Step Load the Keras model using the JSON and weights file
- Step Export the Keras model to the TensorFlow ProtoBuf format (must use AWS
- SageMaker Notebook)
- Step Convert TensorFlow model to a SageMaker readable format (must use AWS
- SageMaker Notebook)
- Tar the entire directory and upload to S
- Step Deploy the trained model (must use AWS SageMaker Notebook)
- Test Model Deployment (optionally, outside of AWS)
- Call the end point
- Additional Reading
- Using a Keras Deep Neural Network with a Web Application
- When to Retrain Your Neural Network
- Preprocessing the Sberbank Russian Housing Market Data
- KS-Statistic
- Detecting Drift between Training and Testing Datasets by Training
- Using a Keras Deep Neural Network with a Web Application
- Converting Keras to CoreML
- Creating an IOS CoreML Application
- More Reading
- Other Neural Network Techniques
- What is AutoML
- AutoML from your Local Computer
- AutoML from Google Cloud
- A Simple AutoML System
- Running My Sample AutoML Program
- Using Denoising AutoEncoders in Keras
- Function Approximation
- Multi-Output Regression
- Simple Autoencoder
- Autoencode (single image)
- Standardize Images
- Image Autoencoder (multi-image)
- Adding Noise to an Image
- Denoising Autoencoder
- Anomaly Detection in Keras
- Read in KDD Data Set
- Preprocessing
- Training the Autoencoder
- Detecting an Anomaly
- Training an Intrusion Detection System with KDD
- Read in Raw KDD- Dataset
- Analyzing a Dataset
- Encode the feature vector
- Train the Neural Network
- New Technologies
- Neural Structured Learning (NSL)
- Bert, AlBert, and Other NLP Technologies
- Explainability Frameworks
Book download link HERE
Image source pixabay
Credit: Data Science Central By: ajit jaokar
