Context Engineering: Where the Term Came From, What It Denotes, and Which of Its Parts Have Actually Been Measured

"Context engineering" is a real term with a datable public origin, a definition four independent parties agree on, and a body of surrounding claims of which roughly a third have a number attached and the rest do not. This page separates those two piles. It does not argue about whether prompt engineering is dead; that argument has no measurable content and none of the sources below make it.

The short answer

The term denotes the selection of the token set an LLM sees at a given inference step — not the wording of a prompt. It went from two posts on X in June 2025 to a 1400-paper survey in July 2025 to a vendor definition in September 2025, and the definitions agree on the denotation. Of the components that go into that token set, five have a published measurement with a stated base; at least five more are described in detail by the same sources with no measurement at all. Both lists are below, and the second list is the useful one.

The founding sentence, its author, and its exact time

Regime: citational, entirely from the primary. Attested for the timestamp.

Andrej Karpathy, post ID 1937902205765607626:

> +1 for "context engineering" over "prompt engineering". > > People associate prompts with short task descriptions you'd give an LLM in your day-to-day use. > When in every industrial-strength LLM app, context engineering is the delicate art and science > of filling the context window with just the right information for the next step. Science because > doing this right involves task descriptions and explanations, few shot examples, RAG, related > (possibly multimodal) data, tools, state and history, compacting... Too little or of the wrong > form and the LLM doesn't have the right context for optimal performance. Too much or too > irrelevant and the LLM costs might go up and performance might come down.

Where to read it, precisely. Fetching x.com/karpathy/status/1937902205765607626 with a browser user-agent returns HTTP 200 and 203,202 bytes. The complete 1,375-character post is in that document four times over: twice rendered in the body, and twice in the embedded GraphQL payload keyed "NoteTweet",text:"...". What *is* truncated is only the social-card metadata — the og:description field stops at "...filling the context window" and the <title> element gets one word further before eliding. This matters as a method note, not as a limitation: a checker who greps the meta tags will conclude the sentence is unverifiable from the primary and will go hunting for mirrors. It is verifiable from the primary. Grep the body.

The exact time, without trusting any mirror. X post IDs are Snowflake identifiers whose high bits are a millisecond timestamp. Decoding it arithmetically — ((1937902205765607626 >> 22) + 1288834974657) ms since the Unix epoch — gives 2025-06-25T15:54:24.993Z, agreeing to the second with the created_at an API mirror reports. This is an attestation: the method is one line, it uses nothing but the ID printed in the URL, and anyone can rerun it.

Date of the fact: 25 June 2025. Not the date we fetched it.

Karpathy did not coin it, and his post says so in its first four characters

Regime: citational.

The post opens "+1 for". The thing being +1'd is Tobi Lütke's post ID 1935533422589399127, six days earlier — and this is established by the primary itself, which embeds that post as a quote-tweet card carrying data-href="/tobi/status/1935533422589399127". Lütke's post, quoted with the typographic marks the original uses:

> I really like the term “context engineering” over prompt engineering. > > It describes the core skill better: the art of providing all the context for the task to be > plausibly solvable by the LLM.

Snowflake decode: 2025-06-19T03:01:43.074Z.

And the words are older than either post, attached to a different referent. A title query to OpenAlex restricted to 1990–2024 returns 20 records — but that index is tokenised, and only 14 of the 20 actually contain the contiguous phrase; the rest are artefacts like *Engineering in Context: Engineering in Developing Countries*. Of the 14, three were resolved against Crossref and came back matching: *Quality on Context Engineering* (Kirsch Pinheiro & Souveyet, DOI 10.1007/978-3-319-57837-8_36, LNCS / *Modeling and Using Context*, issued 2017); *Context Engineering Hybrid Spaces for Perceptual Augmentation* (Carl H. Smith, DOI 10.14236/ewic/eva2014.58, issued July 2014); *Context Engineering for Learning* (Licínio Roque, DOI 10.4018/978-1-59140-488-0.ch003, IGI Global — Crossref carries no issue date for this record, so no year is claimed for it here). The two earliest hits are undated by DOI and remain unconfirmed: OpenAlex W174611792 (Roque & Almeida, ICEIS, 2002) and W1500247756 (Roque, Almeida & Figueiredo, JAIS, 2004). All of these are the same two words pointing at context-aware information systems, not at the token set of a language model.

What we could not establish, and will not claim

who used "context engineering" first in the LLM sense. 19 June 2025 03:01:43 UTC is the earliest LLM-sense public use we verified, which is a different statement from "the first" — and a title index can only show presence, never absence. If you have an earlier dated instance, this section is wrong and should be corrected — see *Supersession*.

What it denotes: four definitions that agree

Regime: citational.

| Source | Date of the fact | Definition, verbatim | |---|---|---| | Lütke, post 1935533422589399127 | 19 Jun 2025 | "the art of providing all the context for the task to be plausibly solvable by the LLM" | | Karpathy, post 1937902205765607626 | 25 Jun 2025 | "the delicate art and science of filling the context window with just the right information for the next step" | | Mei et al., arXiv:2507.13334v2 | 17 Jul 2025 | "a formal discipline that transcends simple prompt design to encompass the systematic optimization of information payloads for LLMs" | | Anthropic engineering post | 29 Sep 2025 | "the set of strategies for curating and maintaining the optimal set of tokens (information) during LLM inference, including all the other information that may land there outside of the prompts" |

The shared content is precise and worth stating flatly: the object being engineered is the token set at an inference step, and it includes material the developer never typed. Anthropic's post puts the substrate even more plainly one paragraph earlier — "Context refers to the set of tokens included when sampling from a large-language model (LLM)". That is the whole denotation. Everything past it is a technique claim, and technique claims need numbers.

Note also what none of the four says: none says prompt engineering has ended. Anthropic's post calls context engineering "the natural progression of prompt engineering"; Karpathy calls it "just one small piece of an emerging thick layer"; and Anthropic's own multi-agent post, written three months earlier, says "prompt engineering was our primary lever for improving these behaviors".

The inventory: what is in the window

Karpathy's post enumerates, verbatim: "task descriptions and explanations, few shot examples, RAG, related (possibly multimodal) data, tools, state and history, compacting". Anthropic's enumerates, verbatim: "(system instructions, tools, Model Context Protocol (MCP), external data, message history, etc)". Merged and deduplicated, that is seven things: instructions; examples; tool definitions; retrieved documents; tool results; conversation history; and whatever an earlier step compressed or summarised.

The rest of this page splits those seven by whether anyone has published a measurement of their effect.

1. Retrieved chunks — and the famous number is a retrieval number, not an answer number

Regime: citational (vendor primary).

Anthropic, *Introducing Contextual Retrieval*, 19 September 2024:

- Contextual Embeddings: top-20-chunk retrieval failure rate 5.7% → 3.7% (a 35% relative reduction) - \+ Contextual BM25: 5.7% → 2.9% (49%) - \+ Cohere reranking: 5.7% → 1.9% (67%)

The base, and what is missing from it. Metric stated: "1 minus recall@20". Configuration stated: top-20 chunks, reranking over an initial top-150, Cohere reranker. Domains named: "codebases, fiction, ArXiv papers, Science Papers". Query counts, corpus sizes and per-domain totals are not published.

And the headline is a derived number that does not announce itself as one. The post's own sentence is that the graphs "show the average performance across all knowledge domains with the top-performing embedding configuration (Gemini Text 004)". So 5.7% is not a measurement of a system; it is a mean over four unlike domains, under the best of the embedding models tested, with the number of queries per domain unstated and therefore the weighting of that mean unknowable. Per-domain results exist only as charts in the appendix. What a reader can actually hold is the absolute movement — 3.8 percentage points, from 5.7 to 1.9 — and the knowledge that it is an average of four things.

The bridging sentence is asserted, not measured. The post says these improvements "directly translates to better performance in downstream tasks". No downstream task result appears in the post. Recall@20 is an intermediate metric; the claim that it moves answer quality is plausible and unmeasured *here*.

A derived number in the same post that does declare its base, as a contrast: "the one-time cost to generate contextualized chunks is $1.02 per million document tokens", stated together with the four assumptions it rests on ("800 token chunks, 8k token documents, 50 token context instructions, and 100 tokens of context per chunk"). That is how a derived figure should arrive, and it is in the same post as the one that does not.

2. Irrelevant material — one sentence is enough

Regime: citational.

Shi et al., *Large Language Models Can Be Easily Distracted by Irrelevant Context*, arXiv:2302.00093, submitted 31 January 2023, ICML 2023.

Base: 100 base problems drawn from a 1,000-problem sample of the GSM8K training split, filtered to ones at least one prompting technique solves — the paper calls this an "easy" subset and says so; 58,052 generated variants, each adding exactly one irrelevant sentence; 4,000 sampled for evaluation (GSM-IC-4K, covering all 100 base problems); greedy decoding; models code-davinci-002 and text-davinci-003.

With chain-of-thought on code-davinci-002:

| | Clean base (Table 2) | With one irrelevant sentence (Table 3) | |---|---|---| | Micro accuracy | 95.0 | 72.4 | | Macro accuracy (base problems solved under *every* variant) | — | 6.0 |

These two columns have different denominators — 100 problems against 4,000 variants — which is exactly why the paper carries a third column we should quote alongside them: the *normalised* micro accuracy, 72.4 against the fraction of base problems solved, is 76.2. Six of a hundred problems survive every distractor variant. The paper's own summary: "fewer than 30% of the base problems are consistently solved after adding distractors."

A measured effect of an instruction line. Prepending "Solve grade school math problems. Feel free to ignore irrelevant information given in the questions." moves micro from 72.4 to 77.8 and macro from 6.0 to 15.0. This is one of very few published measurements of a *wording* change inside this literature, and it recovers roughly a quarter of the loss, not all of it.

Date of the fact and its cost

January 2023, on two models OpenAI has since retired. The *direction* is established. The *magnitude* is not portable to 2026 models, and we found nobody who has re-run this protocol on current ones.

3. Conversation history, when it arrives in pieces

Regime: citational.

Laban, Hayashi, Zhou, Neville, *LLMs Get Lost In Multi-Turn Conversation*, arXiv:2505.06120v1, 9 May 2025. Base: 15 LLMs, six generation tasks, 600 instructions sharded across them, N = 10 simulations per model-and-setting pair, more than 200,000 simulated conversations, temperature T = 1, ~$5,000 of inference.

- FULL (whole instruction, one turn) → SHARDED (same instruction revealed one piece per turn): average degradation −39%, on every model and every task. - CONCAT control: 95.1% of FULL. Same sharded pieces, concatenated and delivered at once.

The CONCAT row is the load-bearing one and it is what makes this a context-engineering result rather than a conversation result: the information content is not the problem; its arrival schedule is. The paper decomposes the loss into a small aptitude drop and a large reliability loss, and names the mechanism — models commit early to an answer built on an incomplete window and then defend it.

One discrepancy, reported rather than resolved

in the same v1, the abstract and §6.1 say −39%; the Figure 1 caption says −35%.

4. Number of examples — helps, then hurts, but not on the task you might think

Regime: citational.

Agarwal et al., *Many-Shot In-Context Learning*, arXiv:2404.11018v3 (v1 17 Apr 2024, v3 17 Oct 2024), NeurIPS 2024 Spotlight.

Model identifier, with the authors' own caveat: Gemini 1.5 Pro, the February 2024 version from the Gemini 1.5 tech report — footnote 1 states "the Gemini 1.5 Pro API now serves a newer version starting from April 2024", and Figure 5 of the same paper plots "Gemini 1.5 Pro" and "Gemini 1.5 Pro (Apr 2024)" as two distinct lines. The paper is its own best illustration of why a model name is not an identifier.

Across the eleven tasks in Figure 1, moving from typical few-shot counts to the best many-shot count yields gains of between +5.0 and +36.4 points, with many-shot optima ranging from 50 to 8,192 shots.

And it is not monotonic — on one of two summarisation datasets. This distinction is easy to lose and worth holding: shots are drawn from the XSum dev set in both cases, but

- on GEM-XSum (ROUGE-L over 150 test articles), performance improves up to 50 shots and then deteriorates; the authors also report the many-shot prompted model "generating summaries with fabricated dates and times", despite no such data appearing in the in-context examples; - on XLSum, evaluated as transfer from those same XSum shots, performance "monotonically improves with more shots" — and XLSum is the dataset behind the "Summarization" bar in Figure 1.

So the summarisation row that earns a delta in Figure 1 is *not* the row that degrades. Figure 2 does record "slight performance deterioration beyond a certain number of shots" on code verification and planning.

The usable claim is therefore narrow: more examples is a dial with an optimum; the optimum is task-specific and dataset-specific, and it can only be found by sweeping it.

5. Number of tool definitions shown

Regime: citational, with a correction to the source's own abstract.

Repantis, Gawde, Singh, Blackwell II (Meta Platforms), *How Many Tools Should an LLM Agent See? A Chance-Corrected Answer*, arXiv:2605.24660v2 (v1 23 May 2026, v2 7 Jun 2026).

Base for the downstream table: Berkeley Function Calling Leaderboard "simple" category, 400 entries, 370 unique functions pooled into a shared registry that the original benchmark does not have, BM25 scorer over function descriptions and parameter names, 280/120 train/test split, 3 seeds, Claude Sonnet 4.6 forced to pick exactly one tool. The adaptive agents were retrained for this table with a higher step cost, yielding K=2.2 rather than the main table's K=7.4.

| Method | Gold tool presented % | Choice acc. % (given presented) | End-to-end % | Avg list length | |---|---|---|---|---| | BoR (adaptive) | 76.9 ± 0.4 | 93.1 ± 0.5 | 71.7 ± 0.0 | 2.2 | | Fixed K=5 | 84.2 | 87.1 | 73.3 | 5.0 | | Fixed K=1 | 65.0 | 100.0 | 65.0 | 1.0 |

Read the third column, not the second. The paper's abstract reports the 93.1 vs 87.1 pair as evidence that shorter adaptive lists "improve the LLM's ability to select the right tool". They do — but the paper's own table caption defines that column as "the fraction of presented-gold queries where Claude selects the gold tool", i.e. *conditioned on the gold tool being in the list*, and the fixed-K=1 row shows what that conditioning is worth: a one-tool list scores a perfect 100.0 on it while losing end-to-end. On the end-to-end product — which the same caption defines as the product of the first two columns — the fixed 5-tool list wins on this split, 73.3 to 71.7.

What this row supports

list length is a real dial with a real effect, and choosing it per query is defensible. What it does not support: "fewer tools is better". One benchmark category, one 120-query test split, one model.

Position inside the window

Measured, thoroughly, and already covered — the U-shaped curve of *Lost in the Middle*, the claimed-vs-effective distinction from RULER, and why a single effective-context number is incoherent are all in Effective Context vs Advertised Context: Which Number Actually Bounds Your Agent. Not restated here. The geometric limit on what single-vector retrieval can place in the window at all is in The Geometric Ceiling of Single-Vector Retrieval: What arXiv 2508.21038 Proves, and the Four Things It Does Not.

Declared, not measured

These appear in the same primary sources as the numbers above, described at length, with no measurement attached. That is not an accusation — the sources are explicit that they are offering practice, not results. It becomes a problem only downstream, where the two piles get quoted in the same voice.

| Technique | Where it is described | What number is attached | |---|---|---| | System-prompt "altitude" (the "Goldilocks zone" between brittle hardcoded logic and vague guidance) | Anthropic, 29 Sep 2025 | none | | XML tags vs Markdown headers for structuring prompts | Anthropic, 29 Sep 2025 | none, in either direction — the post says formatting "is likely becoming less important as models become more capable" | | Compaction (summarise the window, restart from the summary) | Anthropic, 29 Sep 2025 | none | | Structured note-taking / agentic memory | Anthropic, 29 Sep 2025 | none; the supporting example is a Pokémon playthrough anecdote | | "Just-in-time" retrieval by lightweight identifiers instead of pre-loading | Anthropic, 29 Sep 2025 | none | | Sub-agent architectures | Anthropic multi-agent post, 13 Jun 2025 | 90.2% over single-agent Claude Opus 4 — on "our internal research eval", whose size, contents and rubric are not published, so the figure is not checkable by anyone outside | | "Context rot" | Chroma technical report, 14 Jul 2025, 18 LLMs, aligned GPT-4.1 judge, 194,480 LLM calls | the textual claim is quotable ("Even a single distractor reduces performance relative to the baseline... and adding four distractors compounds this degradation further"); the magnitudes are not — the report body contains ten percentage figures in total and every one of them is a refusal rate, a judge-alignment score, a chart-legend threshold or a property of a borrowed benchmark. Findings live in charts. |

The whole Anthropic context-engineering post, incidentally, contains zero percentage figures of any kind — we checked with a regular expression, not by impression. Five described techniques with no number, one with an unpublished number, one with charts. That is the honest state of the field as of 26 August 2026, and it is more useful to know than a list of ten techniques all asserted with equal confidence.

What does NOT work

"Prompt engineering is dead." No source cited on this page says it, three say the opposite in as many words, and the claim has no measurable content: there is no experiment whose outcome would settle it. Any page opening with this sentence is repeating a headline, not a finding.

Quoting "−49% retrieval failures" as an answer-quality improvement. It is 1−recall@20, an intermediate metric, moving 5.7% → 2.9%. The vendor's own sentence connecting it to downstream performance is an assertion in the same post, unaccompanied by a downstream measurement.

Quoting any of the 35/49/67% figures without their absolute base — or without saying they are an average. The whole span from untreated to fully treated is 3.8 percentage points, and 5.7% is a mean across four unlike domains under the best embedding model tested, with per-domain query counts unpublished. "67% fewer failures", "3.8 points", and "3.8 points averaged over four corpora none of which is yours" are the same fact stated with increasing usefulness.

"Karpathy coined the term." He wrote "+1 for", six days and thirteen hours after Lütke's post, quoting it directly in the same post; and the words themselves carry DOIs going back to at least July 2014 in a different field.

Concluding from the meta tags that the founding post cannot be verified from the primary. We made this mistake in an earlier draft of this page and printed the opposite of the truth. X truncates og:description and <title> on long-form posts; the full text is in the document body and again in the embedded GraphQL payload. Check the body before declaring a primary insufficient and reaching for a mirror. The general lesson is worth more than the instance: *absence in the convenient field is not absence in the document.*

Reporting an aggregator's hit count as a phrase count. OpenAlex returns 20 pre-2025 works for a "context engineering" title query; only 14 contain the phrase. The other six match because the index is tokenised — *Engineering in Context: Engineering in Developing Countries* is not a hit. Any bibliometric claim built on a raw count from a tokenised index inherits its false positives.

Citing arXiv:2503.15509 as pre-2025 academic use of the term. Its v1 PDF — 23 pages, extracted locally — contains the string "context engineering" zero times and is titled *Representing data in words*. The subtitle "A context engineering approach" appears only in v2, dated 13 March 2026. This is retroactive relabelling of a paper to a trending term, and an arXiv ID alone will not catch it: you have to open the version.

Treating the survey's "over 1400 research papers" as 1400 measurements of context engineering. It is a count of papers surveyed under a framing introduced by the survey itself.

Reporting analyst-house pronouncements you cannot reach a document for. Claims of the form "[a named research firm] declared 2026 the year of context" circulate in secondary write-ups. We hold none of them, in either direction: we could not obtain a primary document, and we also cannot show that any such document exists — so this page neither repeats the claim nor asserts that the firm made it. A document ID would move this from an exclusion to an entry.

Carrying the GSM-IC magnitudes into 2026. code-davinci-002 and text-davinci-003 are retired. Cite the direction; do not cite 95.0 → 72.4 as a current fact.

Pairing the 90.2% sub-agent figure with the 80%-of-variance figure. They come from the same Anthropic post and from two different evaluations: 90.2% is on an unpublished internal research eval, while the variance decomposition is stated for BrowseComp. Two numbers in one blog post are not one experiment.

Reading the many-shot summarisation result off Figure 1. Figure 1's summarisation bar is XLSum, which improves monotonically; the dataset that peaks at 50 shots and declines is GEM-XSum. Same paper, same prompt pool, opposite shapes.

Trusting a metric that is conditioned on the thing you care about. Tool-choice accuracy given the gold tool was presented, answer accuracy given retrieval succeeded, citation accuracy given a source was found — each is informative and none is the end-to-end number. Fixed-K=1 scoring 100.0 is the cleanest illustration on record.

How to check a context-engineering claim, in six questions

1. Which of the seven components does it move? Instructions, examples, tool definitions, retrieved documents, tool results, history, or compressed prior state. A claim that does not name one is not about context engineering. 2. Does the metric measure the model's answer, or a step before it? Recall@k, presence, selection-given-presence and judge-score are all steps before it. 3. Is the headline a mean? If it averages over domains, tasks or models, ask how many and how weighted. "5.7%" looks like a measurement and is a mean over four corpora. 4. What is n, what is the split, how many seeds, which judge? A benchmark name without a category, a split size and — where an LLM grades — the judge model is not an identifier. 5. Which model, which version, on what date? "Gemini 1.5 Pro" named two different models four months apart; the paper above says so itself, in a footnote most citations drop. Which *arXiv version*, too: one paper on this page acquired the term in its title a year after v1. 6. Is the relative change reported with its absolute base? If not, you cannot tell a 3.8-point improvement from a 38-point one.

Our interest, stated

Sinapsi sells wikis to be retrieved into agent context windows. That places our product squarely in row 1 of the measured table, and it means two of the five measured rows cut against our commercial interest and are printed anyway: adding material to a window has a measured downside (row 2), and more of a good thing has a measured optimum past which it degrades (row 4). A third, row 1, is the row we would most like to lean on, and it is the row this page spends the most words qualifying.

No measurement of ours appears anywhere on this page. We have not run a with-and-without test of our own corpus on any published benchmark, and our sibling page The Binding Constraint Is Verified Context, Not the Model or the Rail (our prediction) states that the decisive test — same model, same task, with and without verified external memory — has not been run by anyone, us included. Nothing here should be read as evidence that our product works. For the market-side reality of what agents actually pay for context-shaped goods, see Paid MCP Tools: 65 Listings, 12.5 Calls a Day, and One Site Making Seven Times the Whole Market; for why memory vendors' benchmark numbers are not comparable to each other, see Three LoCoMo Scores Side by Side Are Not a Ranking: the denominator, the judge, and the one knob worth nine points.

Supersession

This page changes if any of the following happens, and each is a specific, checkable event:

- Someone documents an LLM-sense public use of "context engineering" earlier than 2025-06-19T03:01:43Z. The origin section is then wrong. We claim "earliest we verified", not "first", precisely because we expect this. - A phrase-exact bibliographic search (not a tokenised title index) is run over pre-2025 literature. The 14-of-20 figure is an artefact of one aggregator's matching behaviour and should be replaced by a number from an index that matches phrases. - Anthropic publishes the internal research eval behind the 90.2% figure, or the per-domain query counts behind the 5.7% baseline. The sub-agent row moves to the measured table; the Contextual Retrieval row stops being an average over an unknown weighting. - Anyone re-runs the GSM-IC protocol on 2026 models. The distractor row gains a current magnitude and the caveat comes off. - Chroma publishes tabulated results or code for the context-rot report. The "directional only" verdict comes off. - Anyone runs the tool-list-depth experiment on more than one benchmark category and more than one model. Row 5 either generalises or does not; right now it does neither. - x.com stops serving the founding posts to unauthenticated clients. The Snowflake decode then becomes the only timestamp evidence that does not depend on a mirror, and the quotations fall back to Willison's contemporaneous reproduction.