# Further reading

<!-- https://learn-inference.com/chapters/reading -->

The primary sources. This field moves fast enough that the papers are often more current than anything written about them, and most of these are readable in an evening.

## Architecture

The papers the field is built on, from the original transformer through diffusion and multimodal encoders.

-   [“Attention is All You Need”by Ashish Vaswani et al. (Neural Information Processing Systems, 2017)](https://arxiv.org/abs/1706.03762)
-   [“BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding”by Jacob Devlin et al. (North American Chapter of the Association for Computational Linguistics, 2019)](https://arxiv.org/abs/1810.04805)
-   [“BLIP-2: Bootstrapping Language-Image Pre-training with Frozen Image Encoders and Large Language Models”by Junnan Li et al. (International Conference on Machine Learning, 2023)](https://arxiv.org/abs/2301.12597)
-   [*Deep Learning*by Ian Goodfellow, Yoshua Bengio, and Aaron Courville (The MIT Press, 2016)](https://www.deeplearningbook.org/)
-   [*Deep Learning with Python (2nd Edition)*by François Chollet (Manning, 2021)](https://www.manning.com/books/deep-learning-with-python-second-edition)
-   [“Denoising Diffusion Probabilistic Models”by Jonathan Ho et al. (ArXiv abs/2006.11239, 2020)](https://arxiv.org/abs/2006.11239)
-   [“DiT: Scalable Diffusion Models with Transformers”by William Peebles and Saining Xie (International Conference on Computer Vision (ICCV), 2023)](https://arxiv.org/abs/2212.09748)
-   [“FlashAttention: Fast and Memory-Efficient Exact Attention with IO Awareness”by Tri Dao et al. (ArXiv abs/2205.14135, 2022)](https://arxiv.org/abs/2205.14135)
-   [“FlashAttention-2: Faster Attention with Better Parallelism and Work Partitioning”by Tri Dao (ArXiv abs/2307.08691, 2023)](https://arxiv.org/abs/2307.08691)
-   [“FlashAttention-3: Fast and Accurate Attention with Asynchrony and Low-precision”by Jay Shah et al. (ArXiv abs/2407.08608, 2024)](https://arxiv.org/abs/2407.08608)
-   [*Flash-Attention-4*by Tri Dao (Dao AI Research Lab, 2025)](https://github.com/Dao-AILab/flash-attention)
-   [“Imagen Video: High Definition Video Generation with Diffusion Models”by Jonathan Ho et al. (ArXiv abs/2210.02303, 2022)](https://arxiv.org/abs/2210.02303)
-   [“Language Models Are Few-Shot Learners”by Tom Brown et al. (ArXiv abs/2005.14165)](https://arxiv.org/abs/2005.14165)
-   [“Learning Transferable Visual Models from Natural Language Supervision”by Alec Radford et al. (International Conference on Machine Learning, 2021)](https://arxiv.org/abs/2103.00020)
-   [“Longformer: The Long-Document Transformer”by Iz Beltagy et al. (ArXiv abs/2004.05150, 2020)](https://arxiv.org/abs/2004.05150)
-   [“Mamba: Linear-Time Sequence Modeling with Selective State Spaces”by Albert Gu and Tri Dao (ArXiv abs/2312.00752, 2023)](https://arxiv.org/abs/2312.00752)
-   [“Matryoshka Representation Learning”by Aditya Kusupati et al. (Neural Information Processing Systems, 2022)](https://arxiv.org/abs/2205.13147)
-   [“Outrageously Large Neural Networks: The Sparsely-Gated Mixture-of-Experts Layer”by Noam Shazeer et al. (ArXiv abs/1701.06538, 2017)](https://arxiv.org/abs/1701.06538)
-   [“Reformer: The Efficient Transformer”by Nikita Kitaev et al. (ArXiv abs/2001.04451, 2020)](https://arxiv.org/abs/2001.04451)
-   [“Robust Speech Recognition via Large-Scale Weak Supervision”by Alec Radford et al. (International Conference on Machine Learning, 2022)](https://arxiv.org/abs/2212.04356)
-   [“RoFormer: Enhanced Transformer with Rotary Position Embedding”by Jianlin Su et al. (ArXiv abs/2104.09864, 2021)](https://arxiv.org/abs/2104.09864)
-   [“SDXL: Improving Latent Diffusion Models for High-Resolution Image Synthesis”by Dustin Podell et al. (ArXiv abs/2307.01952, 2023)](https://arxiv.org/abs/2307.01952)
-   [“Segment Anything”by Alexander Kirillov et al. (2023 IEEE/CVF International Conference on Computer Vision (ICCV), 2023)](https://arxiv.org/abs/2304.02643)
-   [“Sentence-BERT: Sentence Embeddings Using Siamese BERT-Networks”by Nils Reimers and Iryna Gurevych (ArXiv abs/1908.10084, 2019)](https://arxiv.org/abs/1908.10084)
-   [“The Llama 3 Herd of Models”by Aaron Grattafiori et al. (ArXiv 2407.21783, 2024)](https://arxiv.org/abs/2407.21783)
-   [“Video Diffusion Models”by Jonathan Ho et al. (ArXiv abs/2204.03458, 2022)](https://arxiv.org/abs/2204.03458)
-   [“Visual Instruction Tuning”by Haotian Liu et al. (ArXiv abs/2304.08485, 2023)](https://arxiv.org/abs/2304.08485)

## Developer Tools

Engines, frameworks, and the libraries you will actually import.

-   [*BitsAndBytes by bitsandbytes-foundation*](https://github.com/TimDettmers/bitsandbytes)
-   [*ComfyUI by comfyanonymous*](https://github.com/comfyanonymous/ComfyUI)
-   [“CUDA by Example: An Introduction to General Purpose GPU Programming”by Jason Sanders and Edward Kandrot (NVIDIA developer, 2025)](https://developer.nvidia.com/cuda-example)
-   [“CUDA C++ Programming Guide Release 13.0”by NVIDIA (2025)](https://docs.nvidia.com/cuda/cuda-c-programming-guide/)
-   [“CUDA cuBLAS Release 13.0”by NVIDIA (2025)](https://docs.nvidia.com/cuda/cublas/)
-   [*CUTLASS by NVIDIA*](https://github.com/NVIDIA/cutlass)
-   [*DeepGEMM by DeepSeek-ai*](https://github.com/deepseek-ai/DeepGEMM)
-   [*Hugging Face Diffusers by Hugging Face*](https://huggingface.co/docs/diffusers/index)
-   [*LMCache*by LMCache Project](https://github.com/LMCache/LMCache)
-   [*NVIDIA Dynamo Documentation*by NVIDIA (2025)](https://docs.nvidia.com/dynamo/latest/index.html)
-   [*NVIDIA Nsight Systems by NVIDIA*](https://developer.nvidia.com/nsight-systems)
-   [*NVIDIA Triton Inference Server by NVIDIA*](https://github.com/triton-inference-server/server)
-   [*ONNX Runtime by Microsoft*](https://onnxruntime.ai/)
-   [“PyTorch Performance Tuning Guide”by Szymon Migacz (PyTorch Foundation, 2020)](https://pytorch.org/tutorials/recipes/recipes/tuning_guide.html)
-   [“PyTorch Profiler”by Shivam Raikundalia (PyTorch Foundation, 2021)](https://pytorch.org/tutorials/recipes/recipes/profiler_recipe.html)
-   [*SGLang Project*by LMSYS Org](https://github.com/sgl-project/sglang)
-   [“TensorRT Documentation”by NVIDIA (2025)](https://docs.nvidia.com/deeplearning/tensorrt/)
-   [*TensorRT-LLM*by NVIDIA](https://github.com/NVIDIA/TensorRT-LLM)
-   [*Transformers*by Hugging Face](https://huggingface.co/docs/transformers/index)
-   [*vLLM Project by The Linux Foundation*](https://github.com/vllm-project/vllm)

## Frontier Open Models

Model cards and technical reports for the open weights worth serving.

-   [*DeepSeek*by DeepSeek AI](https://huggingface.co/deepseek-ai)
-   [*FLUX*by Black Forest Labs](https://huggingface.co/black-forest-labs)
-   [*Gemma*by Google](https://huggingface.co/google)
-   [*GLM*by Z.ai](https://huggingface.co/zai-org)
-   [*GPT OSS*by OpenAI](https://huggingface.co/openai)
-   [*Kimi*by Moonshot AI](https://huggingface.co/moonshotai)
-   [*Llama*by Meta Llama](https://huggingface.co/meta-llama)
-   [*MiniMax*by MiniMax AI](https://huggingface.co/MiniMaxAI)
-   [*Mistral*by Mistral AI](https://huggingface.co/mistralai)
-   [*Nemotron*by NVIDIA](https://huggingface.co/nvidia)
-   [*Orpheus*by Canopy Labs](https://huggingface.co/canopylabs)
-   [*Qwen*by Alibaba Qwen](https://huggingface.co/Qwen)
-   [*Wan*by Wan-AI](https://huggingface.co/Wan-AI)
-   [*Whisper*by OpenAI](https://huggingface.co/openai)

## GPU Infrastructure

Hardware documentation, architecture whitepapers, and interconnect.

-   [*Designing Data-Intensive Applications*by Martin Kleppmann (O’Reilly Media, 2017)](https://dataintensive.net/)
-   [*Grace Hopper / Grace Blackwell Systems*by NVIDIA](https://www.nvidia.com/en-us/data-center/grace-cpu/)
-   [*GPU Glossary*by Frye et al. (Modal, 2025)](https://modal.com/gpu-glossary)
-   [*InfiniBand*by NVIDIA](https://www.nvidia.com/en-us/networking/products/infiniband/)
-   [*Kubernetes Documentation*by The Kubernetes Authors (The Linux Foundation, 2025)](https://kubernetes.io/docs/home/)
-   [*NVIDIA Blackwell Architecture Technical Brief: Built for the Age of AI Reasoning*by NVIDIA (2025)](https://resources.nvidia.com/en-us-blackwell-architecture?ncid=no-ncid)
-   [*NVIDIA H100 Tensor Core GPU Architecture: Exceptional Performance, Scalability and Security for the Data Center*by NVIDIA (2023)](https://resources.nvidia.com/en-us-hopper-architecture/nvidia-h100-tensor-c)
-   [“NVIDIA Tesla: A Unified Graphics and Computing Architecture”by E. Lindholm et al. (IEEE Micro, March–April 2008)](https://ieeexplore.ieee.org/document/4523358)
-   [*NVLink / NVSwitch*by NVIDIA](https://www.nvidia.com/en-us/data-center/nvlink/)
-   [*Programming Massively Parallel Processors: A Hands-on Approach*by Wen-mei Hwu, David Kirk, Izzat El Hajj (Morgan Kaufmann, 2022)](https://www.goodreads.com/work/editions/10244675-programming-massively-parallel-processors-a-hands-on-approach)
-   [*SemiAnalysis*by Dylan Patel (SemiAnalysis, 2025)](https://semianalysis.com/)
-   [*Site Reliability Engineering: How Google Runs Production Services*edited by Betsy Beyer et al. (O’Reilly Media, 2017)](https://sre.google/books/)

## Inference Optimization Research

Where the techniques in Chapter 5 come from: FlashAttention, speculation, quantization, paged attention.

-   [“Adversarial Diffusion Distillation”by Axel Sauer et al. (European Conference on Computer Vision, 2023)](https://arxiv.org/abs/2311.17042)
-   [“Train Short, Test Long: Attention with Linear Biases Enables Input Length Extrapolation”by Ofir Press, Noah Smith, and Mike Lewis (ArXiv abs/2108.12409, 2021)](https://arxiv.org/abs/2108.12409)
-   [“AWQ: Activation-aware Weight Quantization for LLM Compression and Acceleration”by Song Han (MIT, 2024)](https://github.com/mit-han-lab/llm-awq)
-   [*Cache-DIT*by Vipshop](https://github.com/vipshop/cache-dit)
-   [“CacheBlend: Fast Large Language Model Serving for RAG with Cached Knowledge Fusion”by Jiayi Yao et al. (Proceedings of the Twentieth European Conference on Computer Systems, 2024)](https://arxiv.org/abs/2405.16444)
-   [“Adding Conditional Control to Text-to-Image Diffusion Models”by Lvmin Zhang et al. (International Conference on Computer Vision, 2023)](https://arxiv.org/abs/2302.05543)
-   [“Beyond the Buzz: A Pragmatic Take on Inference Disaggregation”by Tiyasa Mitra et al. (ArXiv abs/2506.05508, 2025)](https://arxiv.org/abs/2506.05508)
-   [“Break the Sequential Dependency of LLM Inference Using Lookahead Decoding”by Yichao Fu et al. (ArXiv abs/2402.02057, 2024)](https://arxiv.org/abs/2402.02057)
-   [“EAGLE: Speculative Sampling Requires Rethinking Feature Uncertainty”by Yuhui Li et al. (ArXiv abs/2401.15077, 2024)](https://arxiv.org/abs/2401.15077)
-   [“EAGLE-2: Faster Inference of Language Models with Dynamic Draft Trees”by Yuhui Li et al. (Conference on Empirical Methods in Natural Language Processing, 2024)](https://arxiv.org/abs/2406.16858)
-   [“EAGLE-3: Scaling up Inference Acceleration of Large Language Models via Training-Time Test”by Yuhui Li et al. (ArXiv abs/2503.01840, 2025)](https://arxiv.org/abs/2503.01840)
-   [“FlashInfer: Efficient and Customizable Attention Engine for LLM Inference Serving”by Ye et al. (ArXiv abs/2501.01005, 2025)](https://arxiv.org/abs/2501.01005)
-   [“GPTQ: Accurate Post-Training Quantization for Generative Pre-trained Transformers”by Elias Frantar (ArXiv abs/2210.17323, 2022)](https://arxiv.org/abs/2210.17323)
-   [“High-Resolution Image Synthesis with Latent Diffusion Models”by Robin Rombach et al. (Conference on Computer Vision and Pattern Recognition (CVPR), 2021)](https://arxiv.org/abs/2112.10752)
-   [“Latent Consistency Models: Synthesizing High-Resolution Images with Few-Step Inference”by Simian Luo (ArXiv abs/2310.04378, 2023)](https://arxiv.org/abs/2310.04378)
-   [“LLM.int8(): 8-bit Matrix Multiplication for Transformers at Scale”by Tim Dettmers et al. (ArXiv abs/2208.07339, 2022)](https://arxiv.org/abs/2208.07339)
-   [“Medusa: Simple LLM Inference Acceleration Framework with Multiple Decoding Heads”by Tianle Cai et al. (ArXiv abs/2401.10774, 2024)](https://arxiv.org/abs/2401.10774)
-   [“Megatron-LM: Training Multi-Billion Parameter Language Models Using Model Parallelism”by Mohammad Shoeybi et al. (ArXiv abs/1909.08053, 2019)](https://arxiv.org/abs/1909.08053)
-   [“Efficient Memory Management for Large Language Model Serving with PagedAttention”by Woosuk Kwon et al. (Proceedings of the 29th Symposium on Operating Systems Principles, 2023)](https://arxiv.org/abs/2309.06180)
-   [“Fast Inference from Transformers via Speculative Decoding”by Yaniv Leviathan et al. (International Conference on Machine Learning, 2022)](https://arxiv.org/abs/2211.17192)
-   [“Ring Attention with Blockwise Transformers for Near-Infinite Context”by Hao Liu et al. (ArXiv abs/2310.01889, 2023)](https://arxiv.org/abs/2310.01889)
-   [“SageAttention: Accurate 8-Bit Attention for Plug-and-play Inference Acceleration”by Jintao Zhang et al. (ArXiv abs/2410.02367, 2024)](https://arxiv.org/abs/2410.02367)
-   [*Sequence/Context Parallelism*by Megatron-LM for NVIDIA](https://github.com/NVIDIA/Megatron-LM)
-   [*SmoothQuant*by Song Han (MIT)](https://github.com/mit-han-lab/smoothquant)
-   [“SparseGPT: Massive Language Models Can Be Accurately Pruned in One-Shot”by Elias Frantar and Dan Alistarh, (ArXiv abs/2301.00774, 2023)](https://arxiv.org/abs/2301.00774)
-   [“SpecVLM: Fast Speculative Decoding in Vision-Language Models”Haiduo Huang et al. (ArXiv abs/2509.11815, 2025)](https://arxiv.org/abs/2509.11815)
-   [“TeaCache: Timestep Embedding Aware Cache”by Feng Liu et al. (Alibaba TongYi Vision Intelligence Lab, ArXiv abs/2411.19108, 2025)](https://github.com/ali-vilab/TeaCache)

## Intelligence Evaluation

Benchmarks, their construction, and their well-documented limits.

-   [*ARC AGI Prize*by Greg Kamradt (2025)](https://arcprize.org/)
-   [*Evals for AI Engineers: Systematically Measuring and Improving AI Applications*by Shreya Shankar and Hamel Husain (O’Reilly Media, forthcoming 2026)](https://www.oreilly.com/library/view/evals-for-ai/9798341660717/)
-   [*Grade School Math: Training Verifiers to Solve Math Word Problems*by Karl Cobbe and Vineet Kosaraju (ArXiv abs/2110.14168, 2021)](https://github.com/openai/grade-school-math)
-   [“How to Fine-Tune Qwen3 to GPT-4o Level Performance”by Greg Schoeninger (Fine-Tune Fridays, Oxen AI, 2025)](https://ghost.oxen.ai/how-to-fine-tune-qwen3-to-gpt-4o-level-performance/)
-   [“Humanity’s Last Exam”by Long Phan et al. (Center for AI Safety and Scale AI, ArXiv abs/2501.14249, 2025)](https://www.lastexam.ai/)
-   [“HumanEval: Evaluating Large Language Models Trained on Code”by Mark Chen et al. (OpenAI, 2021)](https://github.com/openai/human-eval)
-   [“MMLU: Measuring Massive Multitask Language Understanding”by Dan Hendrycks et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2021)](https://github.com/hendrycks/test)
-   [“MTEB: Massive Text Embedding Benchmark”by Niklas Muennighoff et al. (Conference of the European Chapter of the Association for Computational Linguistics, 2022)](https://arxiv.org/abs/2210.07316)
-   [“SWE-Bench: Can Language Models Resolve Real-World Github Issues?”by Carlos Jimenez et al. (Proceedings of the International Conference on Learning Representations (ICLR), 2024)](https://www.swebench.com/)

109 sources

Reproduced from Appendix B of *Inference Engineering*. Links go to the primary sources; anything worth understanding properly is better read there than summarized here.