AID202 · Foundations of Machine Learning

Lectures 9–10 · Linear Classification

Interactive labs for logistic regression, generative vs. discriminative learning, linear discriminants, Fisher projection, multiclass scores, perceptron learning, and nonlinear feature maps.

probability vs. scorecross-entropydecision boundariesprojectiononline learningfeature maps
10 min · Lecture 9

1. Score → probability → decision

Manipulate a logistic model’s bias and weight. See how the same linear score creates a probability curve, how weight magnitude controls confidence, and how the 0.5 decision boundary stays linear.

Open activity
15 min · Lecture 9

2. Train logistic regression by hand

Step through gradient descent on a 2-D classification problem. Watch cross-entropy, the decision boundary, and individual point losses change together.

Open activity
12–15 min · Lecture 9

3. Generative vs. discriminative

Fit Gaussian Naïve Bayes and logistic regression to the same samples. Vary sample size and violate the generative assumptions to see why the two methods can win in different regimes.

Open activity
10 min · Lecture 10

4. Geometry of a linear discriminant

Rotate the weight vector, move the bias, and move a point. See the boundary stay perpendicular to w and verify the signed distance y(x)/||w||.

Open activity
12 min · Lecture 10

5. Fisher projection explorer

Rotate a projection axis and observe projected class distributions. Compare mean separation alone with Fisher’s ratio of between-class to within-class spread.

Open activity
10 min · Lecture 10

6. Multiclass linear discriminants

Three linear score functions partition the plane. Move the query point and edit class biases to see argmax decisions, pairwise boundaries, and convex decision regions.

Open activity
15 min · Lecture 10

7. Perceptron: learn from mistakes

Step one example at a time. Every mistake visibly changes w and b. Switch among easy, small-margin, noisy, and XOR data to see when convergence should or should not happen.

Open activity
10 min · Lecture 10

8. Feature lifting solves what a line cannot

Use XOR and rings. Add nonlinear features such as x₁x₂ or x₁²+x₂² and watch a problem become linearly separable in the transformed feature space.

Open activity