Machine Learning
🏷 Supervised Classification
The main toolkit for predicting categories — from probabilistic classifiers to geometric decision boundaries.
- 1→
Logistic Regression
Modelling the probability of a binary outcome using the sigmoid function — fitting by maximum likelihood or gradient descent.
- 2→
Naive Bayes
A probabilistic classifier that applies Bayes' theorem with the (often unrealistic) assumption that features are conditionally independent given the class.
- 3→
Linear Discriminant Analysis
A classification method that finds the linear combination of features maximising between-class separation relative to within-class scatter.
- 4→
Quadratic Discriminant Analysis
Like LDA but allows each class its own covariance matrix — giving quadratic rather than linear decision boundaries.
- 5→
K-Nearest Neighbors
Classifying or predicting by looking at the k closest training points — the simplest non-parametric method, intuitive yet powerful.
- 6→
Decision Trees
Flowchart-like models that recursively partition the feature space by asking yes/no questions — interpretable but prone to overfitting.
- 7→
Ensemble Methods
Combining many weak models into one strong one — bagging reduces variance, boosting reduces bias, and random forests and stacking blend both ideas.
- 8→
Support Vector Machines
Finding the classification boundary with the widest possible margin to the nearest points of each class — and the kernel trick that lets it draw curved boundaries.
- 9→
Geometry of Decision Boundaries
Comparing classifiers by the shape of the regions they carve out — linear hyperplanes, Voronoi cells, axis-aligned boxes, and curved kernel boundaries.