Jeremy Howard (Fast.ai)
Jeremy Howard is the co-founder of fast.ai, a research institute dedicated to making deep learning more accessible. Previously, Jeremy founded a med tech startup Enlitic, and was President of Kaggle.
Tricks To Train Deep Learning Models
- Instead of automating the machine learning process, we should study how to augment it via human-in-the-loop.
- Platform.ai is a unique visual and code-free tool that labels images and trains computer vision models.
- Stick with a sensible learning rate (most of the time, the default is good).
- With Test-Time Augmentation search, you can beat state-of-the-art results even if they use specialized models.
- Progressive resizing is amazing.
- Heatmaps are useful to visualize what's happening.
- 1cycle is a big time-saver.
- For transfer learning, always train later layers more: (1) gradual unfreezing and (2) discriminative learning rates.
- Use AdamW optimizer.
- If you are doing tons of epochs, consider clipping gradients or annealing Adam's episodes.