Start Simple
How to start simple with deep learning models?
Start Simple - Troubleshooting


    Choose simple architecture:
      LeNet/ResNet for images.
      LSTM for sequences.
      Fully-connected network with one hidden layer for all other tasks.
    Use sensible hyper-parameter defaults:
      Adam optimizer with a “magic” learning rate value of 3e-4.
      ReLU activation for fully-connected and convolutional models and TanH activation for LSTM models.
      He initialization for ReLU and Glorot initialization for TanH.
      No regularization and data normalization.
    Normalize data inputs: subtracting the mean and dividing by the variance.
    Simplify the problem:
      Working with a small training set around 10,000 examples.
      Using a fixed number of objects, classes, input size, etc.
      Creating a simpler synthetic training set like in research labs.
Last modified 1yr ago
Copy link