- Reading list
- Some serious reading
- Light refresher
- Awesome deep learning
- Course Notes
- Attention Is All You Need by Vaswani et al.
- BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding by Devlin et al.
- Big Bird: Transformers for Longer Sequences by Zaheer et al.
- Deep Residual Learning for Image Recognition by He et al.
- ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al.
- Photo-Realistic Single Image Super-Resolution using a GAN by Ledig et al.
- GloVe: Global Vectors for Word Representation by Pennington et al.
- What Every Computer Scientist Should Know About Floating-Point Arithmetic by Goldberg et al.
- What Every Programmer Should Know About Memory by Drepper
- Deep Learning by Ian Goodfellow, Yoshua Bengio and Aaron Courville
- Hands-On Machine Learning with Scikit-Learn, Keras, and TensorFlow by Aurelien Geron
- Pattern Recognition and Machine Learning by Christopher Bishop
- The Machine Learning Engineering Book by Andriy Burkov
- The Hundred-Page Machine Learning Book by Andriy Burkov
- Machine Learning Interviews Book by Chip Huyen
- Interactive visualizations
- PapersWithCode State-of-the-Art
- AI Conference Deadlines
- Acceptance rate for major AI conferences
- Reading List for Topics in Multimodal Machine Learning
- What I learned from looking at 200 machine learning tools by Chip Huyen
- SOTAWHAT - A script to keep track of state-of-the-art AI research by Chip Huyen
- A Survival Guide to a PhD by Andrej Karpathy
- Doing well in your courses by Andrej Karpathy
- Planning paper writing by Devi Parikh
- On time management by Devi Parikh
- Managing the organized chaos that is software development by Mohamed El-Geish
- Reacting to Corrective Feedback by Mohamed El-Geish
- A Glimpse into the Future of AI by Mohamed El-Geish
- Learning from Hundreds of Resumes and Tens of Interviews in a Few Weeks by Mohamed El-Geish
- A curated set of websites/blogs I follow to get my latest scoop in AI/ML.
Some serious reading
- A modern medium for presenting research that showcases AI/ML concepts in clear, dynamic and vivid form.
- OpenAI machine learning researcher who likes to understand things clearly, and explain them well.
- Author of the wildly popular “Understanding LSTM Networks” post.
- Robotics researcher @ OpenAI documenting her learning notes.
- Thoughts on AI/ML from the Sr. Director of AI at Tesla.
- Author of the popular The Unreasonable Effectiveness of Recurrent Neural Networks, Deep Reinforcement Learning: Pong from Pixels, A Recipe for Training Neural Networks and The state of Computer Vision and AI: we are really, really far away posts.
- Blog posts that focus on visualizing machine learning one concept at a time.
- Author of the famous How GPT3 Works - Visualizations and Animations, The Illustrated Transformer, The Illustrated BERT, ELMo, and co. and Visual and Interactive Guide to the Basics of Neural Networks posts.
- Stanford alum who created and taught the course TensorFlow for Deep Learning Research writing on AI/ML topics.
- Author of the popular A survivor’s guide to Artificial Intelligence courses at Stanford, What I learned from looking at 200 machine learning tools and SOTAWHAT - A script to keep track of state-of-the-art AI research posts.
- First-year Computer Science Master’s student at Stanford University writes on his experiences with AI/ML.
- UCLA CS ‘19 grad writes on AI/ML.
- Author of the well-recommended Best Practices for Picking a Machine Learning Model and A Whirlwind Tour of Machine Learning Models.
Awesome deep learning
- A curated list of deep learning resources for computer vision.
- A curated list of resources dedicated to NLP.
- A group of (awesome) demos and articles about the OpenAI GPT-3 API.
- A list of 100 important natural language processing (NLP) papers that students and researchers working in the field should read.
- Notes that accompany the Stanford CS class CS231n: Convolutional Neural Networks for Visual Recognition.
- Notes that accompany the Stanford CS class CS224n: Natural Language Processing with Deep Learning.
- Notes that accompany the Stanford CS class CS230 Deep Learning.
- Notes that accompany the Stanford CS class CS229 Machine Learning.
- Notes that accompany the Stanford CS class CS131 Computer Vision: Foundations and Applications. Github with TeX source.
- Slides that accompany the CMU class 11-777 Multimodal Machine Learning.
- Notes that accompany MIT’s 6.034 Artificial Intelligence.
Attention Is All You Need by Vaswani et al.
- The paper from NIPS 2017 that introduced Transformers which are prevalent in most NLP and CV tasks today.
- The paper from ACL 2019 that proposed BERT, a Transformer-based model which proposed pre-training bidirectional representations from unlabeled text by jointly conditioning on both left and right context. Fine-tuning for the task at hand involves using an additional output layer, without substantial task-specific architecture modifications.
Big Bird: Transformers for Longer Sequences by Zaheer et al.
- The primary limitation of Transformer-based models is the quadratic complexity (mainly in terms of memory, but also computation) on the sequence length due to their full attention mechanism. BigBird remedies this by proposing a sparse attention mechanism that reduces this quadratic complexity to linear.
Deep Residual Learning for Image Recognition by He et al.
- ResNet paper from CVPR 2016. Most cited in several AI fields.
ImageNet Classification with Deep Convolutional Neural Networks by Krizhevsky et al.
- The original AlexNet paper from NIPS 2012 that started it all. This trail-blazer introduced Deep Learning to the world :)
Photo-Realistic Single Image Super-Resolution using a GAN by Ledig et al.
- Published in CVPR 2017 that applies GANs for image super-resolution.
GloVe: Global Vectors for Word Representation by Pennington et al.
- The paper from EMNLP 2014 that proposed the famous GloVe model for learning vector space representations of words.
What Every Computer Scientist Should Know About Floating-Point Arithmetic by Goldberg et al.
- This gem helps demystify your errors about computer arithmetic and enables you to write more careful code.
- This must-read offers a detailed treatment on how system memory works.
- The Batch is a weekly newsletter from deeplearning.ai which presents the most important AI events and perspective in a curated, easy-to-read report for engineers and business leaders.
- Every Wednesday, The Batch highlights a mix of the most practical research papers, industry-shaping applications, and high-impact business news.
- The most important artificial intelligence and machine learning links of the week.
- The Gradient is a digital magazine that aims to be a place for discussion about research and trends in artificial intelligence and machine learning.
- Latest updates on NLP readings, research, and more!
- Intended to help students and practitioners enter the field of machine learning in general and deep learning in particular.
- Gain an intuitive understanding of the concepts and tools for building intelligent systems using a range of techniques, starting with simple linear regression and progressing to deep neural networks.
- The first textbook on pattern recognition to present approximate inference algorithms that permit fast approximate answers in situations where exact answers are not feasible.
- For data analysts who lean towards a machine learning engineering role, and machine learning engineers alike who want to bring more structure to their work.
- All you need to know about Machine Learning in a hundred pages.
- A book on machine learning interviews by Chip Huyen, Stanford Lecturer and Snorkel AI.
- Notes that supplement the Coursera Deep Learning Specialization. With interactive visualizations, these tutorials will help you build intuition about foundational deep learning concepts.
- A visual introduction to probability and statistics. Also, includes a textbook called “Seeing Theory”.
- The latest AI/ML papers, with code and leaderboards comparing implementations in several Computer Vision and NLP sub-tasks.
- Countdowns to top CV/NLP/ML/Robotics/AI conference deadlines.
- Statistics of acceptance rate for the major AI conferences.
- Reading list for the CMU class 11-777 Multimodal Machine Learning.
- A survey of the current state of machine learning tooling by Chip Huyen, Stanford Lecturer and Snorkel AI.
- A tool that returns the summary of the latest SOTA research by Chip Huyen, Stanford Lecturer and Snorkel AI.
- Advice from Karpathy on how one can traverse the PhD experience.
- Advice from Karpathy for younger students on how to do well in their undergrad/grad courses.
- Planning paper writing by Devi Parikh, Associate Professor at Georgia Tech, Research Scientist at FAIR.
- Calendar. Not to-do lists. by Devi Parikh, Associate Professor at Georgia Tech, Research Scientist at FAIR.
- How to go about planning the software development process by Mohamed El-Geish, Director of AI, Cisco.
- Internalizing feedback the right way by Mohamed El-Geish, Director of AI, Cisco.
- How did AI get here? What’s its growth trajectory like? by Mohamed El-Geish, Director of AI, Cisco.
- Building effective organizations and hiring people with the right skillset and cultural fit by Mohamed El-Geish, Director of AI, Cisco.
- Metacademy is built around an interconnected web of concepts, each one annotated with a short description, a set of learning goals, a (very rough) time estimate, and pointers to learning resources.
- The concepts are arranged in a prerequisite graph, which is used to generate a learning plan for a concept.