Web Deployment

How to deploy your models to the web?
Web Deployment - Testing and Deployment

Summary

  • For web deployment, you need to be familiar with the concept of REST API.
    • You can deploy the code to Virtual Machines, and then scale by adding instances.
    • You can deploy the code as containers, and then scale via orchestration.
    • You can deploy the code as a “server-less function.”
    • You can deploy the code via a model serving solution.
  • If you are making CPU inference, you can get away with scaling by launching more servers (Docker), or going serverless (AWS Lambda).
  • If you are using GPU inference, things like TF Serving and Ray Serve become useful with features such as adaptive batching.
Last modified 3yr ago