Advanced Data Analysis
A junior data analyst that gathers data, writes code, and plots it — check its work.
Bolt the Python interpreter onto a data workflow and ChatGPT becomes "a junior data analyst": it can search for data, build a table, write plotting code, and show you figures. Genuinely useful — and genuinely error-prone. It makes silent assumptions and sometimes states conclusions its own code contradicts. Amazing, but you must read the code.
He has it research OpenAI's valuation over time (explicitly using search so it doesn't fabricate), build a table, then "plot this, use a log scale for the y-axis." It writes the code and renders the figure.
Silent assumption: for a missing 2015 value it quietly plugged in 0.1 (i.e. $100M) in the code without telling him. Contradiction: it claimed a 2030 extrapolation of ~$1.7T while its own printed variable said ~$20T. "You lied to me." — "Yeah, sorry, I messed up."
“It is kind of like a very, very junior data analyst… It's amazing that it can plot figures, but you have to still know what this code is doing and scrutinize it.”
Andrej Karpathy·63:04
The load-bearing points
- ADA = search + code + plotting = a fast “junior analyst.”
- It makes silent assumptions (e.g. filling missing data).
- It can state a conclusion its own code contradicts.
- Powerful, but read and verify the generated code.
Plot something real
Give it a small dataset (or ask it to gather one via search) and request a chart. Then open the code and check every assumption it made.
Show the point
Look specifically for missing-value handling and any hard-coded numbers — that's where his 0.1 assumption hid.
Catch a contradiction
Ask it to compute a value and also print the underlying variable. Compare the prose answer to the printed number.
Show the point
His $1.7T-vs-$20T mismatch surfaced only because he made it print the raw variable. Do the same.
It can do the mechanical work fast — gather, tabulate, code, plot — but it's absent-minded: it makes unstated assumptions and occasional errors, so its output needs a competent reviewer (you) reading the code.