Full Stack Deep Learning
  • Full Stack Deep Learning
  • Course Content
    • Setting up Machine Learning Projects
      • Overview
      • Lifecycle
      • Prioritizing
      • Archetypes
      • Metrics
      • Baselines
    • Infrastructure and Tooling
      • Overview
      • Software Engineering
      • Computing and GPUs
      • Resource Management
      • Frameworks and Distributed Training
      • Experiment Management
      • Hyperparameter Tuning
      • All-in-one Solutions
    • Data Management
      • Overview
      • Sources
      • Labeling
      • Storage
      • Versioning
      • Processing
    • Machine Learning Teams
      • Overview
      • Roles
      • Team Structure
      • Managing Projects
      • Hiring
    • Training and Debugging
      • Overview
      • Start Simple
      • Debug
      • Evaluate
      • Improve
      • Tune
      • Conclusion
    • Testing and Deployment
      • Project Structure
      • ML Test Score
      • CI / Testing
      • Docker
      • Web Deployment
      • Monitoring
      • Hardware/Mobile
    • Research Areas
    • Labs
    • Where to go next
  • Guest Lectures
    • Xavier Amatriain (Curai)
    • Chip Huyen (Snorkel)
    • Lukas Biewald (Weights & Biases)
    • Jeremy Howard (Fast.ai)
    • Richard Socher (Salesforce)
    • Raquel Urtasun (Uber ATG)
    • Yangqing Jia (Alibaba)
    • Andrej Karpathy (Tesla)
    • Jai Ranganathan (KeepTruckin)
    • Franziska Bell (Toyota Research)
  • Corporate Training and Certification
    • Corporate Training
    • Certification
Powered by GitBook
On this page
  • Andrej's Landmark Post - "Software 2.0"
  • Programming In The 2.0 Stack
  • 2.0 IDEs

Was this helpful?

  1. Guest Lectures

Andrej Karpathy (Tesla)

Andrej is currently Senior Director of AI at Tesla,
 and was formerly a Research Scientist at OpenAI. His educational materials about deep learning remain among the most popular.

PreviousYangqing Jia (Alibaba)NextJai Ranganathan (KeepTruckin)

Last updated 4 years ago

Was this helpful?

Andrej's Landmark Post - ""

  • Software 1.0 consists of explicit instructions to the computer written by a programmer.

  • Software 2.0 can be written in much more abstract, human unfriendly language, such as the weights of a neural network.

  • In software 2.0, we restrict the search to a continuous subset of the program space where the search process can be made efficient with back-propagation and stochastic gradient descent.

Programming In The 2.0 Stack

  • If optimization is doing most of the coding, what are the humans doing?

    • 2.0 programmers label data

    • 1.0 programmers maintain the surrounding "dataset infrastructure":

      • Visualize data

      • Create and edit labels

      • Bubble up likely mislabeled examples

      • Suggest data to label

      • Flag labeler disagreements

  • Data labeling is highly iterative and non-trivial.

    • Lane lines are different across the world.

    • Cars have different shapes and sizes.

    • Even traffic lights and traffic signs can be ambiguous.

  • Label imbalances are very frequent.

  • Data imbalances are very common.

⇒ Realistic datasets: high label and data imbalances, noisy labels, highly multi-task, semi-supervised, active.

2.0 IDEs

  • Show a full inventory and statistics of the current dataset.

  • Create and edit annotation layers for any data point.

  • Flag, escalate, and resolve discrepancies in multiple labels.

  • Flag and escalate data points that are likely to be mislabeled.

  • Display predictions on an arbitrary set of test data points.

  • Auto-suggest data points that should be labeled.

⇒ Can we build GitHub for Software 2.0?

Software 2.0
Source: https://medium.com/@karpathy/software-2-0-a64152b37c35
Programming The Software 2.0 Stack