Common ML Algorithms

Linear regression#

Linear regression predicts a continuous number as a weighted sum of the input features: price = w1·sqft + w2·bedrooms + w3·age + b. Training finds the weights (w1, w2, w3...) that minimize the average squared error between predictions and actual values across the training data.

Use it when: the relationship between inputs and output is roughly linear, you want a fast-to-train and easy-to-explain baseline, or you specifically need to inspect the learned coefficients (for example, "each additional bedroom adds about $15,000 to the predicted price, holding other factors constant").

from sklearn.linear_model import LinearRegression

model = LinearRegression()
model.fit(X_train, y_train)  # learns weights that minimize squared error
predicted_price = model.predict(new_house_features)

Logistic regression#

Despite the name, logistic regression is a classification algorithm, not a regression one. It computes a weighted sum of the inputs just like linear regression, then squashes the result through a sigmoid function into a probability between 0 and 1 — for example, "87% probability this email is spam."

Use it when: you need a binary (or multi-class) classifier that's fast, well-understood, and gives calibrated probabilities rather than just a hard yes/no — a strong default baseline before reaching for anything more complex.

from sklearn.linear_model import LogisticRegression

model = LogisticRegression()
model.fit(X_train, y_train)
spam_probability = model.predict_proba(new_email_features)[:, 1]

Decision trees#

A decision tree learns a sequence of yes/no questions about the features (is income > $50k? then is age > 30? and so on) that splits the data step by step until each final group ("leaf") is mostly one class or has a consistent average value. It's essentially an automatically-learned flowchart.

Use it when: you need a model whose reasoning a non-technical stakeholder can follow directly (you can literally draw the tree and walk through the decisions), or your data has non-linear relationships and feature interactions that linear/logistic regression can't capture. In practice, single decision trees are often replaced by ensembles of many trees (random forests, gradient boosting) which trade a little interpretability for meaningfully better accuracy.

from sklearn.tree import DecisionTreeClassifier

model = DecisionTreeClassifier(max_depth=5)  # limit depth to avoid overfitting
model.fit(X_train, y_train)

k-means clustering#

k-means is an unsupervised algorithm: given unlabeled data and a chosen number of clusters k, it groups data points so that points within a cluster are close to each other (and far from points in other clusters). It works by picking k initial cluster centers, assigning every point to its nearest center, recomputing each center as the average of its assigned points, and repeating until the assignments stop changing.

Use it when: you want to discover natural groupings in data with no predefined labels — customer segments, document topics, anomaly groups — and you can reasonably guess (or test a few values for) the number of groups you expect.

from sklearn.cluster import KMeans

model = KMeans(n_clusters=4, random_state=42)
model.fit(customer_features)
segment_labels = model.predict(customer_features)  # which of the 4 clusters each customer falls into

Choosing between them#

Common mistakes#

• Reaching for a decision tree grown to full depth without limiting it — an unconstrained tree will happily memorize the training set (one leaf per data point) and generalize terribly.
• Forgetting that k-means requires you to choose the number of clusters up front — a common technique is trying several values of k and picking where additional clusters stop meaningfully reducing within-cluster distance (the "elbow method").
• Using linear/logistic regression on features with wildly different scales (for example, "age in years" and "income in dollars") without scaling them first — this can distort which features the model appears to weight as important.