Project Structure
What are the different components of a machine learning system?
Project Structure - Testing and Deployment
- The prediction system involves code to process input data, to construct networks with trained weights, and to make predictions.
- The training system processes raw data, runs experiments, and manages results.
- The goal of any prediction system is to be deployed into the serving system. Its purpose is to serve predictions and to scale to demand.
- Training and validation data are used in conjunction with the training system to generate the prediction system.
- At production time, we have production data that has not been seen before and can only be served by the serving system.
- The prediction system should be tested by functionality to catch code regressions and by validation to catch model regressions.
- The training system should have its tests to catch upstream regressions (change in data sources, upgrade of dependencies)
- For production data, we need monitoring that raises alert to downtime, errors, distribution shifts, etc. and catches service and data regressions.
Last modified 2yr ago