Project Structure

What are the different components of a machine learning system?

Project Structure - Testing and Deployment

Summary

• The prediction system involves code to process input data, to construct networks with trained weights, and to make predictions.

• The training system processes raw data, runs experiments, and manages results.

• The goal of any prediction system is to be deployed into the serving system. Its purpose is to serve predictions and to scale to demand.

• Training and validation data are used in conjunction with the training system to generate the prediction system.

• At production time, we have production data that has not been seen before and can only be served by the serving system.

• The prediction system should be tested by functionality to catch code regressions and by validation to catch model regressions.

• The training system should have its tests to catch upstream regressions (change in data sources, upgrade of dependencies)

• For production data, we need monitoring that raises alert to downtime, errors, distribution shifts, etc. and catches service and data regressions.