Comment on page

# Start Simple

How to start simple with deep learning models?

Start Simple - Troubleshooting

**Choose simple architecture**:- LeNet/ResNet for images.
- LSTM for sequences.
- Fully-connected network with one hidden layer for all other tasks.

**Use sensible hyper-parameter defaults**:- Adam optimizer with a “magic” learning rate value of 3e-4.
- ReLU activation for fully-connected and convolutional models and TanH activation for LSTM models.
- He initialization for ReLU and Glorot initialization for TanH.
- No regularization and data normalization.

**Normalize data inputs**: subtracting the mean and dividing by the variance.**Simplify the problem**:- Working with a small training set around 10,000 examples.
- Using a fixed number of objects, classes, input size, etc.
- Creating a simpler synthetic training set like in research labs.

Last modified 3yr ago