# Yangqing Jia (Alibaba)

{% embed url="<https://youtu.be/s2MWUamDYUI>" %}
What'd Ya Mean by Frameworks?
{% endembed %}

### The Progress of Deep Learning Frameworks

* 2008: Theano
* 2012: Torch7
* 2013: Caffe
* 2015: Keras, TensorFlow
* 2017: Caffe2, PyTorch, ONNX

### What Are The Deciding Factors?

* A framework is intended for model development.
* A framework helps to improve **developer efficiency -** trying out ideas faster (debugging, interactive development, simplicity, intuitiveness).
* A framework helps to improve **infrastructure efficiency -** running computation faster (implementation, scalability, model definition, cross-platform requirements).
* A good framework makes a **balance** between developer efficiency and infrastructure efficiency.

#### Declarative Toolkits

* Examples include Theano, Caffe, MXNet, TensorFlow, and Caffe2.
* In these frameworks, we declare and compile models, then repeatedly execute the models in a Virtual Machine.
* **Advantages**:
  * Easy to optimize.
  * Easy to serialize for production deployment.
* **Disadvantages**:
  * Non-intuitive programming model.
  * Difficult to design and maintain.

#### Imperative Toolkits

* Examples include PyTorch and Chainer.
* In these frameworks, we define and construct the models by running computation. There is no separate execution engine.
* **Advantages**:
  * Intuitive to write programs.
  * Easy to design, debug, and iterate.
* **Disadvantages**:
  * Difficult to optimize - no domain-specific languages.
  * Hard to deploy on multiple platforms.

#### Facebook Example

* Research to Production at Facebook:
  * PyTorch → Caffe2 (2017): Reimplementation took weeks or months.
  * PyTorch → ONNX → Caffe2 (2018): Enabling model or model fragment transfer.
  * PyTorch + Caffe2 (2019-Present): Combining both the advantages of developer efficiency and infrastructure efficiency.
* Many frameworks start adopting such a combination:
  * Keras/TF-Eager + TensorFlow
  * Gluon + MXNet

### How To Choose Frameworks

* Understand your **need**:
  * Developer Efficiency: Algorithm Research? Startup? Proof of Concept?
  * Infrastructure Efficiency: System Research? Cross-Platform? Scale?
* Learn one framework and **focus on your problem**.
* It's fine to switch.
* The goal of frameworks is to improve **productivity.**

### Beyond Frameworks

* Within the tech stack, there is the **libraries** layer on top of frameworks: TF-Serving, CoreML, Clipper, Ray, etc.
* On top of libraries, we have the **applications** layer: Detectron, FairSeq, Magenta, GluonNLP, etc.
* Below the frameworks, we have the layer of **runtime, compilers, and optimizers**: CuDNN, NNPack, TVM, ONXX, etc.
* The lowest layer is **hardware:** CPU, GPU, DSP, FPGA/ASIC

### Thoughts Across The Stack

* Do NOT use AlexNet!
* Unifications help everyone: ONNX bridges the gap between high-level API & framework frontends with hardware vendor libraries & devices.
* Invest in experiment management.
* Use Computer Science conventional wisdom: programming language, compilers, scientific computation, databases, etc.
* Things change across the stack:
  * Applications layer: quantization brings a balance between speed and accuracy.
  * Libraries layer: auto-quantization interfaces increase ease of use.
  * Frameworks layer: quantized training, auto-scaling, etc.
  * Runtime, compilers, optimizers layer: high-performance fixed-point math.
  * Hardware layer: quantized computation primitives.


---

# Agent Instructions: Querying This Documentation

If you need additional information that is not directly available in this page, you can query the documentation dynamically by asking a question.

Perform an HTTP GET request on the current page URL with the `ask` query parameter:

```
GET https://fall2019.fullstackdeeplearning.com/guest-lectures/yangqing-jia-alibaba.md?ask=<question>
```

The question should be specific, self-contained, and written in natural language.
The response will contain a direct answer to the question and relevant excerpts and sources from the documentation.

Use this mechanism when the answer is not explicitly present in the current page, you need clarification or additional context, or you want to retrieve related documentation sections.