Full Stack Deep Learning
Search…
Full Stack Deep Learning
Course Content
Setting up Machine Learning Projects
Infrastructure and Tooling
Data Management
Machine Learning Teams
Training and Debugging
Overview
Start Simple
Debug
Evaluate
Improve
Tune
Conclusion
Testing and Deployment
Research Areas
Labs
Where to go next
Guest Lectures
Xavier Amatriain (Curai)
Chip Huyen (Snorkel)
Lukas Biewald (Weights & Biases)
Jeremy Howard (Fast.ai)
Richard Socher (Salesforce)
Raquel Urtasun (Uber ATG)
Yangqing Jia (Alibaba)
Andrej Karpathy (Tesla)
Jai Ranganathan (KeepTruckin)
Franziska Bell (Toyota Research)
Corporate Training and Certification
Corporate Training
Certification
Powered By
GitBook
Start Simple
How to start simple with deep learning models?
Start Simple - Troubleshooting
Summary
Choose simple architecture
:
LeNet/ResNet for images.
LSTM for sequences.
Fully-connected network with one hidden layer for all other tasks.
Use sensible hyper-parameter defaults
:
Adam optimizer with a “magic” learning rate value of 3e-4.
ReLU activation for fully-connected and convolutional models and TanH activation for LSTM models.
He initialization for ReLU and Glorot initialization for TanH.
No regularization and data normalization.
Normalize data inputs
: subtracting the mean and dividing by the variance.
Simplify the problem
:
Working with a small training set around 10,000 examples.
Using a fixed number of objects, classes, input size, etc.
Creating a simpler synthetic training set like in research labs.
Previous
Overview
Next
Debug
Last modified
1yr ago
Copy link
Contents
Summary