Machine Learning Interview Questions

A curated set of machine learning interview questions covering the concepts that come up in almost every ML screening interview.

Fundamentals#

Q: What causes overfitting, and how would you fix it? Overfitting happens when a model is complex enough (or trained long enough) to memorize noise and quirks specific to the training data, rather than the general pattern — visible as high training accuracy but much lower validation accuracy. Fixes include getting more training data, simplifying the model (fewer features, shallower trees, fewer parameters), adding regularization (which penalizes overly complex models), and stopping training early once validation performance stops improving.

Q: What's the difference between precision and recall, and when would you favor one over the other? Precision is "of everything flagged positive, how much actually was positive" — it penalizes false positives. Recall is "of everything actually positive, how much did we catch" — it penalizes false negatives. You favor precision when false positives are costly, such as flagging a legitimate transaction as fraud and blocking a real customer. You favor recall when false negatives are costly, such as missing an actual case of cancer in a screening test.

Q: What's the difference between supervised and unsupervised learning? Supervised learning trains on labeled data — each example comes with the correct answer, and the model learns to predict that answer for new inputs. Unsupervised learning has no labels; the algorithm finds structure, like clusters or patterns, in the data on its own, and a human interprets what that structure means afterward.

Q: Explain the bias-variance trade-off, conceptually. Bias is error from a model being too simple to capture the real pattern — it makes systematic, consistent mistakes (underfitting). Variance is error from a model being too sensitive to the specific training data it saw — it would give very different predictions if trained on a slightly different sample (overfitting). Reducing one typically increases the other: a simpler model has higher bias but lower variance; a more complex model has lower bias but higher variance. The goal is finding the sweet spot that minimizes total error on new data, not driving either to zero in isolation.

Q: Why do you need a separate validation set instead of just using the test set while tuning a model? If you repeatedly check the test set while making choices — which hyperparameters, which features, which algorithm — you're implicitly fitting your choices to the test set's specific quirks, the same way training on it would. The test set's whole value is being an honest, untouched estimate of real-world performance, checked exactly once at the end. The validation set exists precisely so you have somewhere to iterate without contaminating that final, unbiased check.

Q: Why can't accuracy alone tell you if a classifier is good, on an imbalanced dataset? On an imbalanced dataset (for example, 95% of emails are legitimate and 5% are spam), a model that always predicts the majority class scores 95% accuracy while catching zero cases of the minority class it's actually meant to detect. Precision, recall, and F1 look specifically at how the model handles each class, rather than being dominated by the majority class's sheer size.

Feature engineering and neural networks#

Q: Why is one-hot encoding usually preferred over ordinal encoding for a categorical feature like color or city? Ordinal encoding assigns each category a single integer, which implicitly tells the model there's a meaningful order and distance between categories (that "green" is somehow greater than "red"). For a nominal category with no real order, that's a false relationship the model can end up treating as real signal. One-hot encoding creates a separate binary column per category instead, avoiding any implied ordering, and is the right default whenever the categories genuinely have no natural rank.

Q: Why can't a single perceptron learn a pattern like XOR, and what fixes that? A single perceptron can only learn a linearly separable decision boundary — one straight line (or flat plane) separating two classes. XOR isn't linearly separable; no single straight line correctly separates its four input combinations by output. Stacking perceptrons into a multi-layer network, with a non-linear activation function between layers, lets the network combine several simple linear boundaries into a much more complex, non-linear one, which is enough to learn XOR and far more complicated patterns.

Q: What does backpropagation actually compute? It computes how much each individual weight in the network contributed to the final prediction error, working backward from the output layer to the input layer using the chain rule from calculus. That per-weight "share of the blame" is what tells the training process which direction and how much to nudge every single weight, which is what makes training a many-layered network computationally practical at all.

Deployment, regularization and tuning#

Q: What's the difference between data drift and concept drift? Data drift is a shift in the distribution of the model's input features over time — the inputs themselves start looking different from training data, even though the true relationship between inputs and outputs hasn't changed. Concept drift is a shift in the actual relationship between inputs and the correct output — the inputs can look completely normal while what they should predict has quietly changed (fraud patterns evolving to evade a known model, for instance). Concept drift is generally harder to detect early, because the input data alone doesn't obviously look wrong.

Q: What's the practical difference between L1 and L2 regularization? L1 (Lasso) penalizes the sum of the absolute values of the model's weights, which tends to push some weights all the way to exactly zero — effectively performing automatic feature selection. L2 (Ridge) penalizes the sum of squared weights, which shrinks all weights toward smaller values but rarely to exactly zero, keeping every feature in play with reduced influence. L1 suits situations with many suspected irrelevant features; L2 suits situations where most features carry at least some genuine signal.

Q: Why does random search often outperform grid search for hyperparameter tuning, given that it tries fewer combinations? Grid search's cost grows multiplicatively with every additional hyperparameter and every additional value tried, and it spends a large share of that budget on fine-grained variations of hyperparameters that often barely affect the outcome. In practice, usually only a few hyperparameters matter much for a given problem, and random search's fixed sampling budget covers the space more efficiently as a whole, often finding a comparably good combination in a fraction of the time a full grid search would take.