Primers • AI

Overview

• Here’s a hand-picked selection of articles on AI fundamentals/concepts that cover the entire process of building neural nets to training them to evaluating results.

Algorithms/Architecture

• Linear and Logistic Regression
• k-Nearest Neighbors
• Clustering
• Support Vector Machines (SVM)
• Naive Bayes
• Decision Trees and Ensemble Methods
• ML Algorithms Comparative Analysis
• DL Architectures Comparative Analysis
• Prompt Engineering
• Generative Adversarial Networks (GANs)
• Diffusion Models
• Graph Neural Networks
• Attention
• Separable Convolutions
• Inductive Bias
• Convolutional Neural Networks
• Reinforcement Learning
• Mixture-of-Experts
• State Space Models
• Agents
• Quantization
• Model Acceleration
• Cross Validation

Data/Training

• Data Sampling
• Data Imbalance
• Standardization vs. Normalization
• Learning Paradigms
• Xavier Initialization
• Padding and Packing
• Regularization
• Gradient Descent and Backprop
• Activation Functions
• Loss Functions
• Activation Functions
• Fine-tuning Models
• Splitting Datasets
• Batchnorm
• Dropout
• Double Descent
• Fine-Tuning and Evaluating BERT
• Training Loss > Validation Loss?
• SVM Kernel/Polynomial Trick
• Bias Variance Tradeoff
• Gradient Accumulation and Checkpointing
• Parameter Efficient Fine-Tuning
• Hypernetworks
• Distributed Training Parallelism

Speech

• Speech Processing

Vision

• Vision Transformer (ViT)
• Receptive Field
• Residual Networks/Skip Connections

NLP

• Word Vectors/Embeddings
• NLP Tasks
• Preprocessing
• Tokenization
• Data Sampling
• Neural Architectures
• Attention
• Transformers
• Token Sampling Methods
• Encoder vs. Decoder vs. Encoder-Decoder Models
• Overview of Large Language Models (LLMs)
• LLM Alignment
• Machine Translation
• Knowledge Graphs
• Hallucination Mitigation
• AI Text Detection Techniques
• Named Entity Recognition
• Textual Entailment
• Retrieval Augmented Generation (RAG)
• LLM Context Length Extension
• Document Intelligence
• Code Mixing and Switching
• Large Language Model Ops (LLMOps)
• LLM/VLM Benchmarks

Multimodal

• Overview of Vision-Language Models (VLMs)
• VLM Architectures

Models

• BERT
• GPT
• CLIP
• Meena
• ChatGPT
• GPT-4
• LLaMA
• Alpaca
• Gemini
• Toolformer
• Visual ChatGPT
• TaskMatrix.AI
• BigBird
• OpenAI o1
• DeepSeek R1
• DeepSeek Janus-Pro

Offline/Online Evaluation

• Evaluation Metrics
• F-Beta Score
• A/B Testing

MLOps

• Data Drift
• MLOps Tooling
• MLOps Testing

On-Device AI

• Model Compression
• Personally Identifiable Information (PII)
• Federated Learning
• Differential Privacy

Project Planning, Scheduling, Execution

• Objectives and Key Results (OKRs)
• RICE Framework
• Gantt Charts
• Project Management

Miscellaneous

• Ilya Sutskever’s Top 30
• Debugging Model Training
• Chain Rule
• Bayes’ Theorem
• Probability Calibration
• Multiclass vs. Multilabel Classification
• N-Dimensional Tensor Product
• PyTorch vs. TensorFlow
• Approximate Nearest Neighbors – Similarity Search
• Transferability Estimation
• TensorBoard
• Convolutional Neural Networks for Text Classification
• Relationship between Hidden Markov Models and Naive Bayes
• Maximum Entropy Markov Models
• Conditional Random Fields

Hyperparameters

• Hyperparameter Tuning
• Hyperparameter Logging

Practice

• Interview Questions