Reading list

  • A curated set of websites/blogs I follow to get my latest scoop in AI/ML.

Some serious reading

Distill.pub

  • A modern medium for presenting research that showcases AI/ML concepts in clear, dynamic and vivid form.

Christopher Olah’s Blog

  • OpenAI machine learning researcher who likes to understand things clearly, and explain them well.
  • Author of the wildly popular “Understanding LSTM Networks” post.

Lilian Weng’s Blog

  • Robotics researcher @ OpenAI documenting her learning notes.

Andrej Karpathy’s Blog

Jay Alammar’s Blog

Kevin Zakka’s Blog

  • First-year Computer Science Master’s student at Stanford University writes on his experiences with AI/ML.

Light refresher

Adit Deshpande’s Blog

  • UCLA CS ‘19 grad writes on AI/ML.

Lavanya’s Blog

Awesome deep learning

Awesome Deep Vision

  • A curated list of deep learning resources for computer vision.

Awesome NLP

  • A curated list of resources dedicated to NLP.

Awesome GPT-3

  • A group of (awesome) demos and articles about the OpenAI GPT-3 API.

100 Must-Read NLP Papers

  • A list of 100 important natural language processing (NLP) papers that students and researchers working in the field should read.

Course Notes

Stanford CS231n Notes

  • Notes that accompany the Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.

Stanford CS224n Notes

  • Notes that accompany the Stanford CS class CS224n: Natural Language Processing with Deep Learning.

Stanford CS230 Section Notes

  • Notes that accompany the Stanford CS class CS230 Deep Learning.

Stanford CS229 Notes

  • Notes that accompany the Stanford CS class CS229 Machine Learning.

Stanford CS131 Notes

  • Notes that accompany the Stanford CS class CS131 Computer Vision: Foundations and Applications. Github with TeX source.

MIT Lecture Notes on Artificial Intelligence

  • Notes that accompany MIT’s 6.034 Artificial Intelligence.

Papers

Attention Is All You Need by Vaswani et al.
  • The paper from NIPS 2017 that introduced Transformers which are prevalent in most NLP and CV tasks today.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Devlin et al.
  • The paper from ACL 2019 that proposed BERT, a Transformer-based model which proposed pre-training bidirectional representations from unlabeled text by jointly conditioning on both left and right context. Fine-tuning for the task at hand involves using an additional output layer, without substantial task-specific architecture modifications.
Big Bird: Transformers for Longer Sequences by Zaheer et al.
  • The primary limitation of Transformer-based models is the quadratic complexity (mainly in terms of memory, but also computation) on the sequence length due to their full attention mechanism. BigBird remedies this by proposing a sparse attention mechanism that reduces this quadratic complexity to linear.
Deep Residual Learning for Image Recognition by He et al.
  • ResNet paper from CVPR 2016. Most cited in several AI fields.
ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al.
  • The original AlexNet paper from NIPS 2012 that started it all. This trail-blazer introduced Deep Learning to the world :)
Photo-Realistic Single Image Super-Resolution using a GAN by Ledig et al.
  • Published in CVPR 2017 that applies GANs for image super-resolution.
GloVe: Global Vectors for Word Representation by Pennington et al.
  • The paper from EMNLP 2014 that proposed the famous GloVe model for learning vector space representations of words.
What Every Computer Scientist Should Know About Floating-Point Arithmetic by Goldberg et al.
  • This gem helps demystify your errors about computer arithmetic and enables you to write more careful code.
What Every Programmer Should Know About Memory by Drepper
  • This must-read offers a detailed treatment on how system memory works.

Newsletters

The Batch by deeplearning.ai

  • The Batch is a weekly newsletter from deeplearning.ai which presents the most important AI events and perspective in a curated, easy-to-read report for engineers and business leaders.
  • Every Wednesday, The Batch highlights a mix of the most practical research papers, industry-shaping applications, and high-impact business news.

True Positive Weekly by Andriy Burkov

  • The most important artificial intelligence and machine learning links of the week.

The Gradient by the Stanford Artificial Intelligence Laboratory (SAIL)

  • The Gradient is a digital magazine that aims to be a place for discussion about research and trends in artificial intelligence and machine learning.

Hugging Face Newsletter

  • Latest updates on NLP readings, research, and more!

Books

Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville

  • Intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.

Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurelien Geron

  • Gain an intuitive understanding of the concepts and tools for building intelligent systems using a range of techniques, starting with simple linear regression and progressing to deep neural networks.

Pattern Recognition and Machine Learning by Christopher Bishop

  • The first textbook on pattern recognition to present approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible.

The Machine Learning Engineering Book by Andriy Burkov

  • For data analysts who lean towards a machine learning engineering role, and machine learning engineers alike who want to bring more structure to their work.

The Hundred-Page Machine Learning Book by Andriy Burkov

  • All you need to know about Machine Learning in a hundred pages.

Interactive visualizations

Deeplearning.ai Notes

  • Notes that supplement the Coursera Deep Learning Specialization. With interactive visualizations, these tutorials will help you build intuition about foundational deep learning concepts.

Seeing Theory

  • A visual introduction to probability and statistics. Also, includes a textbook called “Seeing Theory”.

Miscellaneous

PapersWithCode State-of-the-Art

  • The latest AI/ML papers, with code and leaderboards comparing implementations in several Computer Vision and NLP sub-tasks.

AI Conference Deadlines

  • Countdowns to top CV/NLP/ML/Robotics/AI conference deadlines.

Acceptance rate for major AI conferences

  • Statistics of acceptance rate for the major AI conferences.

A Survival Guide to a PhD

  • Advice from Karpathy on how one can traverse the PhD experience.

Doing well in your courses by Andrej Karpathy

  • Advice from Karpathy for younger students on how to do well in their undergrad/grad courses.

Planning paper writing by Devi Parikh

On time management by Devi Parikh

Managing the organized chaos that is software development by Mohamed El-Geish

Reacting to Corrective Feedback by Mohamed El-Geish

A Glimpse into the Future of AI by Mohamed El-Geish

Learning from Hundreds of Resumes and Tens of Interviews in a Few Weeks by Mohamed El-Geish

Metacademy

  • Metacademy is built around an interconnected web of concepts, each one annotated with a short description, a set of learning goals, a (very rough) time estimate, and pointers to learning resources.
  • The concepts are arranged in a prerequisite graph, which is used to generate a learning plan for a concept.