Keep the context window clean
Tokens are a precious resource. Start a new chat the moment you switch topics.
The context window is working memory — and it's expensive in two ways. Too many irrelevant tokens distract the model and lower accuracy; more tokens also cost slightly more and run slightly slower per step. So treat the window as precious: keep it short, keep it relevant, and wipe it (new chat) whenever the old tokens stop helping.
As you chat, the stream only grows. When you change subjects, those old tokens aren't just useless — they're a tax.
Distraction: "the model might be distracted by all the tokens in the past" and lose accuracy. Cost & speed: "the more tokens are in the window, the more expensive it is… to sample the next token," so it slows down. A bloated window is worse and pricier.
“Think of the tokens in the context window as a precious resource… don't overload it with irrelevant information and keep it as short as you can. And you can expect that to work faster and slightly better.”
Andrej Karpathy·10:49
The fix is almost free and habitual: switch topic → start a new chat. You reset the window to zero and hand the model a clean desk. Only keep prior tokens around when they're genuinely relevant to what's next.
New-chat-on-topic-switch costs you one click and buys faster, sharper answers. It also keeps you from the trap of a mega-thread where the model is quietly anchored to something you asked an hour ago.
The load-bearing points
- The window is working memory — and a metered resource.
- Junk tokens cause distraction (lower accuracy) and cost/latency.
- Habit: new chat when the topic changes.
Reset reflex
For the next day, start a brand-new chat every time you switch to an unrelated task, instead of continuing one long thread. Notice the speed and focus.
Show the point
You're minimising irrelevant tokens — the single easiest accuracy win in the whole talk.
Watch the drift
In a long, meandering thread, ask a question unrelated to the last 20 messages and see if the answer gets subtly colored by earlier context.
Show the point
That coloring is the “distraction” cost. A fresh window removes it entirely.
(1) Distraction — irrelevant tokens can pull the model off and reduce accuracy. (2) Compute — every extra token makes sampling the next token a bit slower and more expensive.