Image Classification

  • The problem of image classification has a lot of challenges like illumination and viewpoints.
  • An image classification algorithm can be solved with K nearest neighborhood (KNN) but it can poorly solve the problem. The properties of KNN are:
    • Hyperparameters of KNN are: \(k\) and the distance measure.
    • \(k\) is the number of neighbors we are comparing to.
    • Distance measures include:
      • L2 distance (Euclidean distance).
        • Best for non-coordinate points.
      • L1 distance (Manhattan distance).
        • Best for coordinate points.
  • The optimal set of hyperparameters can be obtained using cross-validation per the following algorithm (in our case we are trying to predict \(k\)):
    1. Split your dataset into \(f\) folds.
    2. Given predicted hyperparameters:
      • Hold out a fold. Train your algorithm with the remaining \(f-1\) folds. Repeat this with every fold as the held-out fold.
    3. Choose the hyperparameters that gives the best training values (average over all the folds).
  • Linear SVM classifier is an option for solving the image classification problem, but the curse of dimensions makes it stop improving at some point.
  • Logistic regression is a also a solution for image classification problem, but image classification problem is non linear!
  • Linear classifiers has to run the following equation: \(Y = wX + b\)
    • Note that the shape of \(w\) is the same as \(x\) and shape of \(b\) is 1.
  • We can add 1 to X vector and remove the bias so that: \(Y = wX\)
    • Note that the shape of \(x\) is \(oldX+1\) and \(w\) is the same as \(x\)
  • The end goal is to obtain the optimal set of parameters \(w\)’s and \(b\)’s that render the best performance based on some evaluation metric (say accuracy, precision, recall etc.).

Citation

If you found our work useful, please cite it as:

@article{Chadha2020ImageClassification,
  title   = {Image Classification},
  author  = {Chadha, Aman},
  journal = {Distilled Notes for Stanford CS231n: Convolutional Neural Networks for Visual Recognition},
  year    = {2020},
  note    = {\url{https://aman.ai}}
}