Deploying and Monitoring Models

Introduction

A model in a notebook helps nobody. Deployment puts it into a product where it makes predictions for real users. And a deployed model is never finished: the world changes, and models decay. This lesson covers the engineering around the model, often called MLOps.

The simple explanation

A model is software plus data, so treat it with the same care as software: version it, test it, deploy it safely and monitor it.

Packaging the model

After training, save the whole pipeline, including preprocessing, not just the classifier, so production uses the same steps:

import joblib

joblib.dump(model, "model-1.0.0.joblib")
loaded = joblib.load("model-1.0.0.joblib")

Only load model files from sources you trust. Some formats can run code when loaded. Record the library versions used, because a model saved with one version may not load in another.

Ways to serve predictions

• Batch: run the model on a schedule over many records, for example nightly scoring. Simple and cheap.
• Online (real time): wrap the model in a web service, often a small API using a framework such as FastAPI, in a container. Responses must be fast.
• On device or in the browser: run a small model locally. This reduces latency and keeps data private.
• Managed platforms: cloud services that host and scale models for you.

Choose by how quickly you need answers and how much traffic you expect.

Training-serving skew

A common failure: features are computed one way in training and another way in production, such as different handling of missing values. The model gets inputs unlike those it learned on. Use shared code or a feature store, and test with real examples end to end.

Version everything

For reproducibility, record:

• Code (in Git)
• Data: which dataset version, and how it was split
• Settings: hyperparameters and random seeds
• The model file and its metrics
• The environment, such as library versions and a container image

Tools called experiment trackers and model registries keep these together. You should be able to answer "which model is serving, and how was it made?"

Safe releases

• Test on realistic data, including edge cases.
• Release with a shadow mode (the new model runs silently alongside the old one) or a canary (a small share of traffic), then compare.
• Keep the old model ready for fast rollback.
• Automate the steps in a pipeline.

Monitoring

Watch the system after launch:

• Service health: errors, latency, cost and uptime
• Input data drift: do the inputs now differ from the training data? For example a new product line, a new season or a changed form.
• Prediction drift: has the pattern of outputs changed?
• Performance: compare predictions with real outcomes when they become known, which may take days or months.
• Fairness: do results differ across groups?

When quality falls, retrain on fresh data, after checking that the new model is better.

Responsible operation

• Keep humans in the loop for important decisions, with a way to challenge them.
• Document the model's purpose, data, limits and risks, sometimes in a "model card".
• Protect personal data, and follow the laws that apply.
• Have a plan for incidents, and for switching the model off.

Try it yourself

For a model you have built or imagined, write a one-page deployment plan: batch or online, how it is versioned, three things you would monitor, the trigger for retraining, and the rollback plan.

Summary

• Save the whole pipeline, serve it by batch, API or on device, and avoid training-serving skew.
• Version code, data, settings and models, and release gradually with a rollback ready.
• Monitor health, drift, performance and fairness, and retrain when quality drops.