CS224n: Natural Language Processing with Deep Learning
A distilled compilation of my notes for Stanford's CS224n: Natural Language Processing with Deep Learning.
Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modeling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP.
Each section is divided by topic to help you get bite-sized information that is easy to understand!
Natural language processing (NLP) is a crucial part of artificial intelligence (AI), modeling how people share information. In recent years, deep learning approaches have obtained very high performance on many NLP tasks. In this course, students gain a thorough introduction to cutting-edge neural networks for NLP.
Each section is divided by topic to help you get bite-sized information that is easy to understand!
📌 Table of Contents
Word Vectors
NLP overview; historical context; word vectors; Word2Vec (CBOW + Skip-gram)
NLP Tasks
named entity recognition; dependency parsing
Neural Networks
RNNs; LSTMs; CNNs; regularization; dropout
Language Models
n-gram language model; neural language model; ELMO; GPT-3
Machine Translation
seq2seq; neural machine translation
Attention
the bottleneck problem; seq2seq with attention; self attention; multi-headed attention
Knowledge Graphs
knowledge graph overview; entity linking; ERNIE; KGLM
Transformer
encoder; decoder; BERT; T5 language model
Tokenizer
process of tokenization; sub-word tokenization; Byte Pair Encoding (BPE); WordPiece; SentencePiece
Course Info
Course description:
- Natural language processing (NLP) or computational linguistics is one of the most important technologies of the information age. Applications of NLP are everywhere because people communicate almost everything in language: web search, advertising, emails, customer service, language translation, virtual agents, medical reports, etc.
- In recent years, deep learning (or neural network) approaches have obtained very high performance across many different NLP tasks, using single end-to-end neural models that do not require traditional, task-specific feature engineering.
- In this course, students will gain a thorough introduction to cutting-edge research in Deep Learning for NLP. Through lectures, assignments and a final project, students will learn the necessary skills to design, implement, and understand their own neural network models. As piloted last year, CS224n will be taught using PyTorch this year.
📸 Credits
The in-line diagrams are taken from the link's below:
- Stanford's CS224n: Natural Language Processing with Deep Learning. - Jay Alammar's Blog.
- Stanford's CS224n: Natural Language Processing with Deep Learning. - Jay Alammar's Blog.
Citation
If you found our work useful, please cite it as:
@misc{Chadha2020DistilledNotesCS231n,
author = {Jain, Vinija and Chadha, Aman},
title = {Distilled Notes for Stanford CS224n: Natural Language Processing with Deep Learning},
howpublished = {\url{https://www.aman.ai}},
year = {2020},
note = {Accessed: 2020-07-01},
url = {www.aman.ai}
}
A. Chadha, Distilled Notes for Stanford CS224n: Natural Language Processing with Deep Learning, https://www.aman.ai, 2020, Accessed: July 1 2020.