I’m at an AI lunch-and-learn for a tactical gear maker’s team, working through what AI is bad at. Third on my list: it doesn’t know what it doesn’t know.
To explain why, I describe the context window. Think of it as a spreadsheet, or a Word doc, with a size limit. Whatever is on that page is everything the AI knows. Leave something out and it doesn’t know it. Put too much in and it starts to lose track.
A few minutes later, someone comes back to it. The window has a limit, they say. How long before you hit it?
It depends on what you put in. Writing email, you’ll almost never hit it. I hit it after a few hours of heavy work, or after loading a pile of large images. When it fills, the tool compacts: it keeps what still matters, moves it to a fresh window, and keeps going.
Then I tell them the part that surprises people most.
AI has no memory.
None. Every time you send a message, the tool sends the entire conversation back to the model, from the first line, plus your new one. The model reads all of it, answers, and forgets. Next message, it reads the whole thing again.
The room goes quiet on that one. It explains most of the AI behavior that feels random. Why it forgot something from an hour ago. Why it remembers in one chat and not the next. Why a long chat gets slow and expensive.
One idea covers all of it. AI only knows what’s in the window.
Your search term report, measured
For an Amazon seller, this stops being theory the first time a report goes into a chat.
So I measured one. I own a small Amazon brand. Its Sponsored Products search term data for 60 days, August 9 to October 7, 2026: 1,385 rows. 690 different search terms across 49 campaigns. As a CSV, about 456,000 characters.
AI reads text in tokens. The common rule of thumb is about four characters of English per token, and a sheet full of numbers and IDs usually does worse than prose. Call it well over 100,000 tokens. For a small brand. For two months.
The window is a fixed size, and it depends on the model and the plan you’re on. On the biggest models, at the time of writing, it’s around a million tokens. Then picture a bigger account, a full year, or the business report and the inventory file pasted into the same chat.
Depending on the tool, one of three things happens when the report goes in:
- It fits, and the AI has all of it to work with.
- It doesn’t fit, and the tool cuts it off, refuses it, or reads only part of it.
- It fits, but it crowds the window, and the AI starts losing track of what you told it earlier.
The second and third are the dangerous ones, because AI is confidently wrong. It doesn’t say “I only saw the first half of your report.” It names your best converting search term, and it sounds sure.
A bigger window doesn’t fix it
The obvious fix is a bigger window. On the biggest models, 100,000 tokens fits with room to spare.
But fitting isn’t the same as being read well, and the companies that build these models say so themselves. Anthropic’s documentation puts it plainly: as the token count grows, accuracy and recall degrade. They call it context rot. Anthropic describes it as an attention budget: every token you add uses up a little of it.
Researchers have also found that models read the beginning and the end of a long input best, and lose track of the middle. In a 1,385-row report, the middle is hundreds of rows of search terms. The one you need might be in there.
Compacting doesn’t rescue a pasted report either. Compacting summarizes the conversation so far so the chat can keep going, which is useful for a long working session. But a summary of your search term report isn’t your search term report. The rows it dropped are gone.
Why long chats get expensive
No memory makes it worse.
Paste the report in your first message, then ask ten follow-up questions. The report doesn’t go in once. It rides along with every message, ten more times, whether the question needs it or not.
That’s why a long chat burns through usage limits or an API bill so fast. Some tools cache the repeated part to soften the cost, but the window still fills. Most of what you’re resending stopped mattering an hour ago.
The fix is boring. When the topic changes, start a new chat. Carry over the three sentences that matter, not the whole transcript.
Query it, don’t paste it
Here’s how I got those numbers.
I didn’t export anything. I asked an AI to count my 60 days of search terms through the Seller Labs MCP Server, which connects AI tools like Claude to a seller’s own Amazon data.
The AI wrote a query. The database did the counting.
1,385 rows went in.
One row came back.
The row count. The term count. The campaign count. The date range.
The report never entered the window. Only the answer did.
That’s the difference. When you paste, the window holds the data and the AI has to do the reading. When you query, the data stays in the database, and the window holds the question and the answer. For a question like that one, a year of search terms takes the same window space as a week.
And the answer is checkable. A query is a few lines anyone can read: these dates, this account, this filter. A paste is a black box. You can’t see which half it read.
Three rules for sellers and agencies
- Ask for answers, not reports. “Which search terms spent over $20 with no sales in the last 60 days?” is a question for a query, not a paste.
- One chat, one job. New topic, new window. Bring the conclusion, leave the transcript.
- When you have to paste, paste less. Filter to the columns and dates the question needs, then ask the AI how many rows it saw before you trust the answer.
Agencies, this one is sharper for you. Pasting several clients’ reports into one long session puts all of them in the same window, and that’s where one account’s numbers can bleed into another account’s answer. One client per chat, or better, one query per question.
What it actually knows
The room asked how long before you hit the limit. The better question is what’s in the window when you ask, because that’s all the AI knows.
It will treat whatever’s in there as the whole truth. It won’t tell you what’s missing.
So don’t hand it the haystack and hope. Give it the question, and let it go get the answer.
