Learning LLMs From Scratch

My notes, explanations, and experiments while reading Sebastian Raschka's Build a Large Language Model (From Scratch).

Why?

I learn by reading with full attention, writing things down, and unpacking ideas until I can explain them. This companion follows the book and grows with my understanding. My longer-term goal is to connect model internals to inference engineering.

The structure

• Step Zero — before the book arrives
• Chapter 1 — Understanding Large Language Models
• Chapter 2 — Working with Text Data
• Chapter 3 — Coding Attention Mechanisms
• Chapter 4 — Implementing a GPT Model from Scratch
• Chapter 5 — Pretraining on Unlabeled Data
• Chapter 6 — Finetuning for Text Classification
• Chapter 7 — Finetuning to Follow Instructions
• Appendix A — Introduction to PyTorch

Pages are currently scaffolds, not completed chapter notes.

How these notes grow

Follow each chapter's sections. Explain unfamiliar terms, trace small examples, add useful figures and links, and explore a tangent when it resolves a real question. Record expected and observed results with code. End each chapter by explaining how its pieces fit together.

Sources

• The book and companion resources
• Official code

These are original learning notes. Attribute adapted material and link to the book for the full treatment.