AI Papers Reading List

A comprehensive collection of foundational AI and LLM papers

Based on the AI Crash Course by Henry Shi

View Original Repository →

🎯 Start Here

Neural Network → LLM Series (3Blue1Brown)

📚 Survey Papers

LLM Survey
Comprehensive overview of large language models
2024
Agent Survey
Survey of LLM-based agents
2023
Prompt Engineering Survey
Comprehensive guide to prompt engineering techniques
2024

🏗️ Foundational Modelling

Attention Is All You Need *
The original transformer architecture paper
2017
Scaling Laws for Neural Language Models
Understanding how model performance scales with size
2020
GPT-3 *
Language models are few-shot learners
2020
LoRA: Low-Rank Adaptation of Large Language Models
Efficient fine-tuning technique using low-rank decomposition
2021
Training Compute-Optimal Large Language Models
Chinchilla paper on optimal model and data scaling
2022
Training Language Models to Follow Instructions with Human Feedback *
InstructGPT: RLHF for instruction following
2022
Direct Preference Optimization
Simpler alternative to RLHF for alignment
2023
Judging LLM-as-a-Judge
Using LLMs to evaluate other LLMs
2023
Mixtral of Experts
Sparse mixture of experts architecture
2024

🧠 Planning & Reasoning

Mastering Chess and Shogi by Self-Play (AlphaZero)
General game-playing through self-play reinforcement learning
2017
Mastering Atari, Go, Chess and Shogi (MuZero) *
Learning without knowing the rules
2019
Chain-of-Thought Prompting *
Eliciting reasoning in large language models
2022
ReAct: Synergizing Reasoning and Acting
Combining reasoning traces with task-specific actions
2022
Tree of Thoughts
Deliberate problem solving with language models
2023
Graph of Thoughts
Advanced reasoning with graph structures
2023
Let's Verify Step by Step
Outcome vs. process supervision for reasoning
2023
Meta Chain-of-Thought
Learning to improve reasoning chains
2024
ARC Prize *
Progress towards general intelligence through reasoning
2024
DeepSeek-R1 *
Incentivizing reasoning capability in LLMs
2025

🚀 Applications

Toolformer
Language models can teach themselves to use tools
2023
GPT-4 Technical Report
Multimodal large-scale language model
2023
The Llama 3 Herd of Models *
Open source multilingual language models
2024
Gemini 1.5
Long-context understanding and reasoning
2024
DeepSeek-V3
Cost-efficient mixture of experts model
2024
SWE-Agent
Agent-computer interface for automated software engineering
2024
OpenHands
Open platform for software development agents
2024

📊 Benchmarks

Beyond the Imitation Game (BIG-Bench)
Diverse evaluation tasks for language models
2022
SWE-bench
Evaluating language models on real-world software issues
2023
Chatbot Arena
Crowdsourced benchmarking platform for LLMs
2024

🎥 Videos & Lectures

3Blue1Brown - Visual mathematics and deep learning explanations
Build a Large Language Model From Scratch - Comprehensive book guide
Andrej Karpathy: Neural Networks: Zero to Hero - Hands-on tutorial series
Yannic Kilcher - Paper reviews and ML news
Noam Brown on Planning - Expert insights on AI planning
Stanford CS324: Advances in Foundation Models - Building LLMs course
Foundations of LLMs - Comprehensive tutorial paper
Why You're Not Too Old to Get Into AI - Motivational perspective

🌐 Helpful Websites

History of Deep Learning - Timeline and key developments
Full Stack Deep Learning - Production ML resources
Prompting Guide - Comprehensive prompt engineering guide
a16z AI Cannon - Curated list of AI resources
2025 AI Engineer Reading List - Latest papers and trends
State of Generative Models 2024 - Year in review

🎨 Beyond LLMs

Vision Transformer (ViT)
An image is worth 16x16 words - Transformers for image classification
2021
High-Resolution Image Synthesis with Latent Diffusion
Stable Diffusion - Efficient image generation in latent space
2021

🌟 Easy Papers for Beginners

Chain-of-Thought Prompting
Great introduction to reasoning in LLMs
2022
SELF-REFINE
Iterative refinement with self-feedback
2023
📝 As I read and implement these papers, I will update these webpages with notes and implementations.