Project Structure
What are the different components of a machine learning system?
Project Structure - Testing and Deployment

Summary

  • The prediction system involves code to process input data, to construct networks with trained weights, and to make predictions.
  • The training system processes raw data, runs experiments, and manages results.
  • The goal of any prediction system is to be deployed into the serving system. Its purpose is to serve predictions and to scale to demand.
  • Training and validation data are used in conjunction with the training system to generate the prediction system.
  • At production time, we have production data that has not been seen before and can only be served by the serving system.
  • The prediction system should be tested by functionality to catch code regressions and by validation to catch model regressions.
  • The training system should have its tests to catch upstream regressions (change in data sources, upgrade of dependencies)
  • For production data, we need monitoring that raises alert to downtime, errors, distribution shifts, etc. and catches service and data regressions.
Copy link
Contents
Summary