01.04
Context windows and attention
Attention and the KV cache explain why an advertised million-token window is not a million-token memory, and why where you place instructions and how long a session runs both decay quality.
- Making AI Product Decisions
- 14 min
- Free
Who's hiring for this skill
23 companies have live AI-PM roles that ask for skills taught in this lesson.
- Nebius
- Intercom
- Rogo
- Writer
- Anthropic
- Scale AI
- Snowflake
- Mistral
- Okta
- Harvey
- Fireworks
- Asana
- Cohere
- Coursera
- Gusto
- 1Password
- Deepgram
- Klaviyo
- Cresta
- Eve
- LangChain
- Modal
- Plaid
Concepts in this subchapter
- Context Window
- Self-Attention
- KV Cache
- Context Rot
- Prefill
- Time to First Token
- Context Engineering
- Prompt Caching
- Long Context Pricing
Source video: Attention in transformers, step-by-step | Deep Learning Chapter 6, 3Blue1Brown.