> For the complete documentation index, see [llms.txt](https://fall2019.fullstackdeeplearning.com/llms.txt). Markdown versions of documentation pages are available by appending `.md` to page URLs; this page is available as [Markdown](https://fall2019.fullstackdeeplearning.com/guest-lectures/jeremy-howard-fast.ai.md).

# Jeremy Howard (Fast.ai)

{% embed url="<https://www.youtube.com/watch?v=hZd3X_nGdew>" %}
Tricks To Train Deep Learning Models&#x20;
{% endembed %}

* Instead of automating the machine learning process, we should study how to augment it via **human-in-the-loop**.
* [Platform.ai](http://platform.ai) is a unique visual and code-free tool that labels images and trains computer vision models.
* Here are lessons learned from optimizing hyper-parameters for image datasets using [fast.ai](http://fast.ai):
  * Stick with a sensible learning rate (most of the time, the default is good).
  * With Test-Time Augmentation search, you can beat state-of-the-art results even if they use specialized models.
  * Progressive resizing is amazing.
  * Heatmaps are useful to visualize what's happening.
  * 1cycle is a big time-saver.
  * For transfer learning, always train later layers more: (1) gradual unfreezing and (2) discriminative learning rates.
  * Use AdamW optimizer.
  * If you are doing tons of epochs, consider clipping gradients or annealing Adam's episodes.