Aman's AI Journal • Reading List
- Reading list
- Some serious reading
- Light refresher
- Awesome deep learning
- Course Notes
- Papers
- Attention Is All You Need by Vaswani et al.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Devlin et al.
- Big Bird: Transformers for Longer Sequences by Zaheer et al.
- Deep Residual Learning for Image Recognition by He et al.
- ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al.
- Photo-Realistic Single Image Super-Resolution using a GAN by Ledig et al.
- GloVe: Global Vectors for Word Representation by Pennington et al.
- What Every Computer Scientist Should Know About Floating-Point Arithmetic by Goldberg et al.
- What Every Programmer Should Know About Memory by Drepper
- Newsletters
- Books
- Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurelien Geron
- Pattern Recognition and Machine Learning by Christopher Bishop
- The Machine Learning Engineering Book by Andriy Burkov
- The Hundred-Page Machine Learning Book by Andriy Burkov
- Machine Learning Interviews Book by Chip Huyen
- Interactive visualizations
- Miscellaneous
- PapersWithCode State-of-the-Art
- AI Conference Deadlines
- Acceptance rate for major AI conferences
- Reading List for Topics in Multimodal Machine Learning
- What I learned from looking at 200 machine learning tools by Chip Huyen
- SOTAWHAT - A script to keep track of state-of-the-art AI research by Chip Huyen
- A Survival Guide to a PhD by Andrej Karpathy
- Doing well in your courses by Andrej Karpathy
- Planning paper writing by Devi Parikh
- On time management by Devi Parikh
- Managing the organized chaos that is software development by Mohamed El-Geish
- Reacting to Corrective Feedback by Mohamed El-Geish
- A Glimpse into the Future of AI by Mohamed El-Geish
- Learning from Hundreds of Resumes and Tens of Interviews in a Few Weeks by Mohamed El-Geish
- Metacademy
Reading list
- A curated set of websites/blogs I follow to get my latest scoop in AI/ML.
Some serious reading
Distill.pub
- A modern medium for presenting research that showcases AI/ML concepts in clear, dynamic and vivid form.
Christopher Olah’s Blog
- OpenAI machine learning researcher who likes to understand things clearly, and explain them well.
- Author of the wildly popular “Understanding LSTM Networks” post.
Lilian Weng’s Blog
- Robotics researcher @ OpenAI documenting her learning notes.
Andrej Karpathy’s Blog
- Thoughts on AI/ML from the Sr. Director of AI at Tesla.
- Author of the popular The Unreasonable Effectiveness of Recurrent Neural Networks, Deep Reinforcement Learning: Pong from Pixels, A Recipe for Training Neural Networks and The state of Computer Vision and AI: we are really, really far away posts.
Jay Alammar’s Blog
- Blog posts that focus on visualizing machine learning one concept at a time.
- Author of the famous How GPT3 Works - Visualizations and Animations, The Illustrated Transformer, The Illustrated BERT, ELMo, and co. and Visual and Interactive Guide to the Basics of Neural Networks posts.
Chip Huyen’s Blog
- Stanford alum who created and taught the course TensorFlow for Deep Learning Research writing on AI/ML topics.
- Author of the popular A survivor’s guide to Artificial Intelligence courses at Stanford, What I learned from looking at 200 machine learning tools and SOTAWHAT - A script to keep track of state-of-the-art AI research posts.
Kevin Zakka’s Blog
- First-year Computer Science Master’s student at Stanford University writes on his experiences with AI/ML.
Light refresher
Adit Deshpande’s Blog
- UCLA CS ‘19 grad writes on AI/ML.
Lavanya’s Blog
- Author of the well-recommended Best Practices for Picking a Machine Learning Model and A Whirlwind Tour of Machine Learning Models.
Awesome deep learning
Awesome Deep Vision
- A curated list of deep learning resources for computer vision.
Awesome NLP
- A curated list of resources dedicated to NLP.
Awesome GPT-3
- A group of (awesome) demos and articles about the OpenAI GPT-3 API.
100 Must-Read NLP Papers
- A list of 100 important natural language processing (NLP) papers that students and researchers working in the field should read.
Course Notes
Stanford CS231n Notes
- Notes that accompany the Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.
Stanford CS224n Notes
- Notes that accompany the Stanford CS class CS224n: Natural Language Processing with Deep Learning.
Stanford CS230 Section Notes
- Notes that accompany the Stanford CS class CS230 Deep Learning.
Stanford CS229 Notes
- Notes that accompany the Stanford CS class CS229 Machine Learning.
Stanford CS131 Notes
- Notes that accompany the Stanford CS class CS131 Computer Vision: Foundations and Applications. Github with TeX source.
CMU 11-777 Slides
- Slides that accompany the CMU class 11-777 Multimodal Machine Learning.
MIT Lecture Notes on Artificial Intelligence
- Notes that accompany MIT’s 6.034 Artificial Intelligence.
Papers
Attention Is All You Need by Vaswani et al.
- The paper from NIPS 2017 that introduced Transformers which are prevalent in most NLP and CV tasks today.
BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Devlin et al.
- The paper from ACL 2019 that proposed BERT, a Transformer-based model which proposed pre-training bidirectional representations from unlabeled text by jointly conditioning on both left and right context. Fine-tuning for the task at hand involves using an additional output layer, without substantial task-specific architecture modifications.
Big Bird: Transformers for Longer Sequences by Zaheer et al.
- The primary limitation of Transformer-based models is the quadratic complexity (mainly in terms of memory, but also computation) on the sequence length due to their full attention mechanism. BigBird remedies this by proposing a sparse attention mechanism that reduces this quadratic complexity to linear.
Deep Residual Learning for Image Recognition by He et al.
- ResNet paper from CVPR 2016. Most cited in several AI fields.
ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al.
- The original AlexNet paper from NIPS 2012 that started it all. This trail-blazer introduced Deep Learning to the world :)
Photo-Realistic Single Image Super-Resolution using a GAN by Ledig et al.
- Published in CVPR 2017 that applies GANs for image super-resolution.
GloVe: Global Vectors for Word Representation by Pennington et al.
- The paper from EMNLP 2014 that proposed the famous GloVe model for learning vector space representations of words.
What Every Computer Scientist Should Know About Floating-Point Arithmetic by Goldberg et al.
- This gem helps demystify your errors about computer arithmetic and enables you to write more careful code.
What Every Programmer Should Know About Memory by Drepper
- This must-read offers a detailed treatment on how system memory works.
Newsletters
The Batch by deeplearning.ai
- The Batch is a weekly newsletter from deeplearning.ai which presents the most important AI events and perspective in a curated, easy-to-read report for engineers and business leaders.
- Every Wednesday, The Batch highlights a mix of the most practical research papers, industry-shaping applications, and high-impact business news.
True Positive Weekly by Andriy Burkov
- The most important artificial intelligence and machine learning links of the week.
The Gradient by the Stanford Artificial Intelligence Laboratory (SAIL)
- The Gradient is a digital magazine that aims to be a place for discussion about research and trends in artificial intelligence and machine learning.
Hugging Face Newsletter
- Latest updates on NLP readings, research, and more!
Books
Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
- Intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.
Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurelien Geron
- Gain an intuitive understanding of the concepts and tools for building intelligent systems using a range of techniques, starting with simple linear regression and progressing to deep neural networks.
Pattern Recognition and Machine Learning by Christopher Bishop
- The first textbook on pattern recognition to present approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible.
The Machine Learning Engineering Book by Andriy Burkov
- For data analysts who lean towards a machine learning engineering role, and machine learning engineers alike who want to bring more structure to their work.
The Hundred-Page Machine Learning Book by Andriy Burkov
- All you need to know about Machine Learning in a hundred pages.
Machine Learning Interviews Book by Chip Huyen
- A book on machine learning interviews by Chip Huyen, Stanford Lecturer and Snorkel AI.
Interactive visualizations
Deeplearning.ai Notes
- Notes that supplement the Coursera Deep Learning Specialization. With interactive visualizations, these tutorials will help you build intuition about foundational deep learning concepts.
Seeing Theory
- A visual introduction to probability and statistics. Also, includes a textbook called “Seeing Theory”.
Miscellaneous
PapersWithCode State-of-the-Art
- The latest AI/ML papers, with code and leaderboards comparing implementations in several Computer Vision and NLP sub-tasks.
AI Conference Deadlines
- Countdowns to top CV/NLP/ML/Robotics/AI conference deadlines.
Acceptance rate for major AI conferences
- Statistics of acceptance rate for the major AI conferences.
Reading List for Topics in Multimodal Machine Learning
- Reading list for the CMU class 11-777 Multimodal Machine Learning.
What I learned from looking at 200 machine learning tools by Chip Huyen
- A survey of the current state of machine learning tooling by Chip Huyen, Stanford Lecturer and Snorkel AI.
SOTAWHAT - A script to keep track of state-of-the-art AI research by Chip Huyen
- A tool that returns the summary of the latest SOTA research by Chip Huyen, Stanford Lecturer and Snorkel AI.
A Survival Guide to a PhD by Andrej Karpathy
- Advice from Karpathy on how one can traverse the PhD experience.
Doing well in your courses by Andrej Karpathy
- Advice from Karpathy for younger students on how to do well in their undergrad/grad courses.
Planning paper writing by Devi Parikh
- Planning paper writing by Devi Parikh, Associate Professor at Georgia Tech, Research Scientist at FAIR.
On time management by Devi Parikh
- Calendar. Not to-do lists. by Devi Parikh, Associate Professor at Georgia Tech, Research Scientist at FAIR.
Managing the organized chaos that is software development by Mohamed El-Geish
- How to go about planning the software development process by Mohamed El-Geish, Director of AI, Cisco.
Reacting to Corrective Feedback by Mohamed El-Geish
- Internalizing feedback the right way by Mohamed El-Geish, Director of AI, Cisco.
A Glimpse into the Future of AI by Mohamed El-Geish
- How did AI get here? What’s its growth trajectory like? by Mohamed El-Geish, Director of AI, Cisco.
Learning from Hundreds of Resumes and Tens of Interviews in a Few Weeks by Mohamed El-Geish
- Building effective organizations and hiring people with the right skillset and cultural fit by Mohamed El-Geish, Director of AI, Cisco.
Metacademy
- Metacademy is built around an interconnected web of concepts, each one annotated with a short description, a set of learning goals, a (very rough) time estimate, and pointers to learning resources.
- The concepts are arranged in a prerequisite graph, which is used to generate a learning plan for a concept.
