The whole zoo — a working checklist
Zoom all the way out. Here is the complete map, and the questions to ask every time.
Step back and the whole talk collapses into one growing diagram and one checklist. There's a rapidly shifting ecosystem of LLM apps; ChatGPT is the feature-rich incumbent, but each rival wins somewhere. Your job in any moment is to run down a short list: which model/tier, do I need thinking, which tools, which modality, and which quality-of-life features — on web or phone.
This is the diagram Karpathy has been building the entire time — the zip file, then thinking, then tools, then modalities, then quality-of-life features. Seeing it whole is the point of the course.
“There is a rapidly growing, changing, shifting, and thriving ecosystem of LLM apps like ChatGPT. ChatGPT is the first and the incumbent and is probably the most feature-rich… but all the other ones are very rapidly growing.”
Andrej Karpathy·2:01:48
1) The zip file: you're talking to a lossy compression of the internet. 2) Tier/model: know which brain you're using. 3) Thinking? hard math/code → switch to a reasoning model. 4) Tools: fresh info → search; deep dive → deep research; numbers/plots → Python/ADA; prototypes/diagrams → Artifacts; real code → Cursor. 5) Modality: speak, listen, show images, point the camera. 6) Quality of life: memory, custom instructions, custom GPTs. 7) Surface: a feature may exist on web but not phone, or vice-versa.
| When you need… | Reach for |
|---|---|
| Fresh, niche, or changing facts | Internet search (Perplexity / ChatGPT search) |
| A deep, cited survey of a topic | Deep research (10-min report — verify it) |
| To understand a paper or book | Upload / paste it, read it together |
| Exact math, or figures & plots | Python interpreter / Advanced Data Analysis |
| A quick app or a concept diagram | Claude Artifacts |
| Serious, multi-file coding | Cursor + Composer (vibe coding) |
| Hands-free input or a natural demo | Voice / Advanced Voice / camera |
| It to remember or match your style | Memory + custom instructions |
| A repeated task done consistently | A few-shot custom GPT |
Native beats tacked-on; tools beat bare recall; and everything is a first draft. The model — even with search, even with deep research — can hallucinate. Keep the citations, read the code, check the box. That habit is what separates using LLMs well from being misled by them.
“All of this is a little bit of a zoo. It's a little bit crazy — but these are the kinds of features that exist.”
Andrej Karpathy·2:05:52
The load-bearing points
- The whole talk = one growing diagram + one checklist.
- Run the list: zip file → tier → thinking → tools → modality → QoL → surface.
- ChatGPT is the feature-rich incumbent; rivals each win somewhere — keep a council.
- Under everything: tools beat recall, native beats tacked-on, verify everything.
Run the checklist for real
Take a task you'll actually do today and walk the 7-step checklist out loud: which model, thinking or not, which tool, which modality, which QoL feature, web or phone.
Show the point
Doing this deliberately a few times turns the whole course into a reflex.
Teach it back
Without looking, redraw the growing diagram from memory: zip file → thinking → tools → modalities → quality-of-life. Fill in one example per branch.
Show the point
If you can reconstruct the diagram, you own the material. Gaps show you which section to revisit.
You're steering a lossy zip file of the internet — pick the right model and tier, let it think when the problem is hard, hand it tools and modalities to break out of the box, personalize it, and verify everything it tells you.