Model Deployment & MLOps

From a trained model to a system that serves real requests#

A model sitting in a saved file, however accurate, delivers no value until something real calls it with new inputs and does something with the predictions. MLOps is the discipline of operating machine learning models reliably in production — the ML-specific analogue of DevOps — covering how a model gets served, versioned, and monitored once it's no longer just a notebook experiment.

Serving a model#

Serving means exposing a trained model behind an interface real application code can call. Two common patterns:

• Real-time (online) serving — the model runs behind an API endpoint, and a request gets a prediction back synchronously, typically in well under a second. Appropriate when a user or system is actively waiting on the result (a fraud check during checkout, a live recommendation).

from fastapi import FastAPI
import joblib

app = FastAPI()
model = joblib.load("house_price_model.pkl")

@app.post("/predict")
def predict(features: dict):
    prediction = model.predict([list(features.values())])
    return {"predicted_price": float(prediction[0])}

• Batch serving — predictions are generated for a large set of inputs all at once, on a schedule, and stored for later use rather than computed per-request. Appropriate when there's no user actively waiting synchronously — nightly churn-risk scoring for an entire customer base, for example.

Choosing between them is mostly about latency requirements: if a specific request needs an answer right now, that's online serving; if predictions can be pre-computed ahead of when they're needed, batch serving is simpler and often cheaper to run.

Versioning models#

A production model needs the same discipline as production code: every trained model artifact should be versioned, alongside the exact training data snapshot and code that produced it, so a specific prediction from last month can be traced back to exactly which model version generated it — essential when investigating a bad prediction or a reported regression.

import joblib
from datetime import date

model_version = f"house_price_model_v{date.today().isoformat()}"
joblib.dump(model, f"models/{model_version}.pkl")

# A registry entry (in a database, or a simple file) tracks which version is "live"
registry = {"live_version": model_version, "trained_on": "2026-08-01_dataset_snapshot"}

Versioning also makes rollback simple and safe: if a newly deployed model performs worse in production than the evaluation metrics predicted, pointing the serving layer back at the previous version is a fast, low-risk fix while the regression gets investigated.

Monitoring for drift#

A model's accuracy at launch is not a permanent guarantee — real-world data drifts over time, and a model trained on last year's patterns can silently degrade as this year's data stops resembling it.

• Data drift — the distribution of input features shifts over time (customer demographics change, a new product category is introduced that the training data never saw). The model itself hasn't changed, but the inputs it's now seeing look meaningfully different from what it was trained on.
• Concept drift — the actual relationship between inputs and the correct output changes over time, even if the inputs themselves look similar (fraud patterns evolve specifically to evade whatever a model has learned to catch; customer preferences shift after a market event). This is the more insidious case, because the input data can look completely normal while the model's learned mapping from input to output has quietly become wrong.

# A simplified drift check: compare a key feature's distribution
# in a recent window of production traffic against the training distribution
from scipy.stats import ks_2samp

statistic, p_value = ks_2samp(training_feature_values, recent_production_values)
if p_value < 0.05:
    alert("Possible data drift detected in this feature")

Production ML systems typically track prediction accuracy against ground truth as it becomes available (which can lag by days or weeks, depending on the task), track input feature distributions for drift even before ground truth is available, and retrain on a fixed schedule or automatically when monitored performance drops below a defined threshold.

Common mistakes#

• Treating deployment as the finish line — a model shipped and never revisited will silently degrade as real-world data drifts away from its training distribution, often for a long stretch before anyone notices without active monitoring.
• Not versioning training data alongside the model itself — without it, a reported bad prediction from a specific past model version can't be reliably reproduced or debugged, because the exact data that trained it is no longer tied to that version.
• Waiting for ground-truth labels alone to detect a problem — labels can lag by weeks (for example, whether a loan actually defaulted), while input feature drift is often detectable immediately, well before enough labeled outcomes exist to measure accuracy directly.