CS229: Machine Learning

A distilled compilation of my notes for Stanford's CS229: Machine Learning.

Notes

Linear Regression

the supervised learning problem; update rule; probabilistic interpretation; likelihood vs. probability

Locally Weighted Linear Regression

weighted least squares; bandwidth parameter; cost function intuition; parametric learning; applications

Netwon's Method

Netwon's method; update rule; quadratic convergence; Newton's method for vectors

Logistic Regression

the classification problem; motivation for logistic regression; logistic regression algorithm; update rule

Perceptron

perceptron algorithm; graphical interpretation; update rule

Generalized Linear Models

exponential family; constructing GLMs; case studies: LMS, logistic regression, softmax regression

Gaussian Discriminant Analysis

generative learning algorithms; Gaussian discriminant analysis (GDA); GDA vs. logistic regression

Naive Bayes

naive bayes; laplace smoothing

Support Vector Machines

support vector machines; kernels

Learning Theory

data splits; bias-variance trade-off; case of infinite/finite \(\mathcal{H}\); deep double descent

Regularization and Model Selection

cross-validation; feature selection; bayesian statistics and regularization

Decision Trees

non-linearity; selecting regions; defining a loss function

Ensemble Methods

bagging; boostrap; boosting; Adaboost; forward stagewise additive modeling; gradient boosting

Neural Networks

basics; backprop; improving neural network accuracy

Debugging and Error Analysis

debugging ML models (overfitting, underfitting); error analysis

The k-Means Clustering Algorithm

k-means; convergence guarantee

Expectation Maximization Algorithms

mixture of Gaussians (non EM); expectation maximization

Factor Analysis

the factor analysis model; expectation maximization for the factor analysis model

Principal Component Analysis

PCA algorithm; eigenvectors; eigenvalues

Independent Component Analysis

ambiguities; densities and linear transformations; ICA algorithm

Intro to Reinforcement Learning and Adaptive Control

MDPs; Bellman equation; value and policy iteration; continuous state MDP; value function approximation

Linear Quadratic Regulation, Differential Dynamic Programming and Linear Quadratic Gaussian

finite-horizon MDPs; LQR; from non-linear dynamics to LQR; LQG; DDP; LQG

Course Info

Course description:

- Stanford's CS229 provides a broad introduction to machine learning and statistical pattern recognition. Topics include: supervised learning (generative/discriminative learning, parametric/non-parametric learning, neural networks, support vector machines); unsupervised learning (clustering, dimensionality reduction, kernel methods); learning theory (bias/variance trade-offs, practical advice); reinforcement learning and adaptive control.
- The course will also discuss recent applications of machine learning, such as to robotic control, data mining, autonomous navigation, bioinformatics, speech recognition, and text and web data processing.

Credits

The in-line diagrams are taken from the CS229 lecture notes, unless specified otherwise. Reproduced with permission.

Citation

If you found our work useful, please cite it as:

@misc{Chadha2020DistilledNotesCS229, author = {Chadha, Aman}, title = {Distilled Notes for Stanford CS229: Machine Learning}, howpublished = {\url{https://www.aman.ai}}, year = {2020}, note = {Accessed: 2020-08-01}, }

A. Chadha, Distilled Notes for Stanford CS229: Machine Learning, https://www.aman.ai, 2020, Accessed: Aug 1 2020.