The ML Workflow

The end-to-end workflow#

Building an ML model that actually works in production is a pipeline, not a single step. Skipping or doing any of these steps carelessly is the single most common reason models perform well in a notebook and badly in the real world.

Collect data → Clean data → Split (train/val/test) → Train → Evaluate → Deploy → Monitor

1. Collecting and cleaning data#

Real-world data is messy: missing values, duplicate rows, inconsistent formatting, outliers caused by data-entry errors, and labels that are simply wrong. Cleaning typically means:

• Handling missing values (drop the row, or fill it with a sensible default like the column's mean or median).
• Removing duplicates and clearly invalid rows (a house listed at $0, an age of 200).
• Making formats consistent (dates, units, categorical spellings like "NY" vs. "New York").

This step is unglamorous but usually consumes more time than the actual model training — a model can only be as good as the data it learns from.

2. Splitting into train / validation / test sets#

You never train and evaluate on the same data — a model that has already seen the answer key will look artificially good. The standard split:

• Training set (typically ~60-80% of the data) — what the model actually learns from.
• Validation set (~10-20%) — used to tune choices you make about the model (which algorithm, which hyperparameters) without touching the test set.
• Test set (~10-20%) — touched exactly once, at the very end, to get an honest estimate of how the model will perform on genuinely new data.

from sklearn.model_selection import train_test_split

# First split off the test set, then split the remainder into train/validation
X_train_full, X_test, y_train_full, y_test = train_test_split(
    X, y, test_size=0.2, random_state=42
)
X_train, X_val, y_train, y_val = train_test_split(
    X_train_full, y_train_full, test_size=0.25, random_state=42
)

3. Training#

Training is the process of the algorithm adjusting its internal parameters to minimize error on the training set.

from sklearn.ensemble import RandomForestRegressor

model = RandomForestRegressor(n_estimators=100, random_state=42)
model.fit(X_train, y_train)  # learn from the training data

4. Evaluating#

Check how the trained model performs on data it did not train on — first the validation set, while you're still iterating on choices, then the test set exactly once at the end.

from sklearn.metrics import mean_absolute_error

val_predictions = model.predict(X_val)
print(mean_absolute_error(y_val, val_predictions))

If validation performance is much worse than training performance, that's a sign of overfitting (see the next tutorial in this series for a deeper look at that).

5. Deploying#

A model that only exists in a notebook delivers no value. Deploying typically means wrapping the trained model behind an API endpoint (or a batch job) so real application code can send it new inputs and get predictions back:

import joblib

joblib.dump(model, "house_price_model.pkl")

# later, in the serving application:
model = joblib.load("house_price_model.pkl")
predicted_price = model.predict(new_house_features)

6. Monitoring#

Deployment isn't the finish line. Real-world data drifts over time — customer behavior changes, prices inflate, new fraud patterns emerge — so a model's accuracy measured at launch will silently decay. Production ML systems track prediction quality over time and retrain on fresh data on a schedule, or when monitored performance drops below a threshold.

Common mistakes#

• Data leakage — accidentally letting information from the test/validation set influence training, for example scaling the entire dataset before splitting, so statistics from the test set leak into the training data. Always fit preprocessing steps (scalers, imputers) on the training set only, then apply them to validation/test.
• Tuning hyperparameters against the test set instead of the validation set — this quietly turns the test set into a second validation set, and your final "test accuracy" is no longer an honest, unbiased estimate.
• Treating deployment as the end of the project — without monitoring, a model's real-world accuracy can degrade for months before anyone notices.