GPT-6 Astra bills any request above 272,000 input tokens at 2× input and cache rates and 1.5× output, on the entire request rather than the overflow. A 270,000-token prompt costs about $3.20; a 275,000-token one, about $6.25. That is a long-context tier at work.
The bill moved to the context layer, and the leverage moved with it.
Five parts make the case: the shift from prompt engineering is real, a million-token window is not a million usable tokens, compaction throws something away, the leverage lives outside the model, and a pricing threshold reprices the request.
In This Series
- What context engineering covers that prompt engineering does not: the definition, the rebrand question, the four operations.
- Why a million-token context window is not a million usable tokens: advertised versus usable capacity and context rot.
- What context compaction throws away and how to measure the loss: what summarisation drops and how to detect the loss.
- The context layer you own outside the model: retrieval, sub-agents, the context library and the audit.
- Why crossing a long-context pricing threshold reprices the whole request: the pricing cliff, prompt caching and estimation.
What actually changes when context engineering replaces prompt engineering?
Prompt engineering optimised wording inside a fixed window. Context engineering optimises what enters the window, at what point in the run, at what fidelity and at what price: retrieved documents, tool schemas, memory, sub-agent handoffs, and their order. The unit of work moves from a string written at design time to a pipeline executed on turn 47, so the artefact you maintain is the harness, not the prompt.
Four operations sit under it — eager load, just-in-time fetch, compact, sub-agent — each on a different cost curve in tokens, latency, recall or cash; the context budget forces the choice. The rebrand charge is fair: good teams already curated examples. But three discontinuities remain — the unit of work became a runtime, the failure mode became “the model never saw it”, and the cost model became per-window-tier. What the term actually covers settles the rebrand question.
Why doesn’t a million-token context window remove the constraint?
The advertised window is not the usable window. Models degrade as input length grows even when nothing needs to be found. Chroma‘s context-rot work and a 2026 study (arXiv 2605.12366v1) found frontier models missing a dangerous action buried after 800,000 tokens of benign activity 2 to 30 times more often. Recall is strongest at the start and end of the window and weakest in the middle, so “it’s in there somewhere” is not a retrieval strategy.
Separate the spec-sheet number from the working number and the effective context window shrinks long before the hard limit: the middle gets read least, so position is a design variable. Chroma’s report held complexity constant and varied only length; degradation still appeared. Most long-context benchmarks measure needle-in-a-haystack search; RULER, NoLiMa, LongMemEval and ConstraintRot stress harder things. That detail is the gap between advertised and usable capacity.
Why does compaction make an agent break a policy it followed forty turns earlier?
Compaction is the operation most teams adopt first and audit last. Summarisation is lossy by construction: negations become affirmations, the reason behind a decision disappears before the decision does, and hedges, exceptions and ordering constraints read as filler and vanish first. Because compaction fires on a token budget, it fires hardest when the run is longest and the accumulated constraints are most numerous.
The failure surfaces hours later as a plausible-looking policy breach, with no log line and no crash, and gets misattributed to the model rather than the harness. Constraint loss differs from context poisoning: poisoning corrupts what remains, loss removes it. Constraint survival is what to measure; constraint pinning keeps load-bearing rules outside the summariser. The measurement model lives in what compaction removes. Summarisation also invalidates the cache, so the next call bills full rate.
Where does the leverage actually live — in the model, or in the layer around it?
Mostly outside the model. LangChain moved a coding agent from outside the top 30 to top 5 on Terminal-Bench 2.0, 52.8% to 66.5%, by changing only harness, tools and middleware. NVIDIA’s SoL-Pi cut coding-agent token traffic by up to 49% through compaction. Both are assembly decisions: what enters the window, which tools exist, who hands off to whom. The number moves in the parts of the stack you own.
LangChain’s result and NVIDIA’s SoL-Pi are both first-party, so treat them as directional, not gospel. On large codebases retrieval binds: the model reasons correctly but does not see the relevant context. The context library, tool schemas, compaction policy and audit trail are yours, partly standardised through MCP. Someone has to own assembly: the context layer outside the model.
Why does crossing a long-context pricing threshold reprice the whole request?
Because the vendor prices a capacity tier rather than a marginal token. Cross 272,000 input tokens on GPT-6 Astra and 2× input and cache rates and 1.5× output apply to the entire request, not just the overflow. Crossing adds roughly $3 however few tokens spill, so the last thousand tokens cost most and the tail of the distribution sets your bill.
At 10,000 requests a day with 5% crossing, that is about $1,500 daily and over $500,000 a year; the worked arithmetic rests on the published rate card, with no surcharge_applied field to confirm the crossing and tokenizer inflation drifting estimates 10 to 20%. Prompt caching compounds it: a stable prefix earns a discount, but compaction rewrites it, invalidating the cache and billing the next call at full rate. Both the window behind the tier and the invalidation trace earlier. How a threshold reprices a whole request has the breakdown.
What to Do Next Quarter
Budget: model the threshold crossing explicitly and instrument it. The meter is absent by default, and 10 to 20% drift separates $3.20 from $6.25.
Staffing: name who owns the context library, the tool schemas and the compaction policy. Unstaffed, it goes missing until a failure forces it.
Audit: make the assembled context window reconstructable after a failed run, or every post-mortem is guesswork.
Route by symptom: invoice to cost, quality regression to compaction, window size to capacity, a cold start to the definition.
Start with the parts of the stack you actually own and the constraints a summariser drops, where the leverage concentrates.
Or get the technical guide if the definition needs settling first.
Resource Hub: Context Engineering Deep Dives
The Frame and the Constraint
- What context engineering covers that prompt engineering does not: the definition, the rebrand question, and the four operations.
- Why a million-token context window is not a million usable tokens: advertised versus usable capacity and how to read a benchmark.
What Breaks and What You Own
- What context compaction throws away and how to measure the loss: what summarisation drops and how to measure constraint survival.
- The context layer you own outside the model: retrieval, sub-agents, the context library, the audit.
The Economics
- Why crossing a long-context pricing threshold reprices the whole request: the 272K cliff, prompt caching and estimation.
Frequently Asked Questions
Is context engineering just retrieval-augmented generation with a new name?
No. RAG is one operation inside it. Retrieval is the just-in-time fetch case, and context engineering also decides what gets loaded eagerly, what gets compacted, what goes to a sub-agent and in what order all of it arrives. A RAG pipeline that retrieves the right document and then buries it mid-window still fails, because retrieval was never the whole job.
Do I have to throw away the prompts my team already wrote?
No, keep them. Most survive as components: tool descriptions, summarisation templates and compaction instructions are still prompts and still need the same care. What changes is where they sit. They become steps in a pipeline rather than the entire artefact, and the pipeline is what you version, review and test after a failure.
How do I choose between eager loading, just-in-time fetching, compacting and sub-agents?
Pick by what the task cannot afford to lose. Eager loading buys certainty at a token cost, just-in-time fetching buys budget at a latency cost, compaction buys room by spending recall, and a sub-agent buys isolation by spending evidence. They are alternatives rather than a maturity ladder, and the context budget forces the choice per task.
What does a sub-agent actually cost me?
A sub-agent spends tokens and buys isolation, but the parent never sees the trace, which is a claim in place of evidence. It fails quietly, through inherited credentials or a confident summary of a failed search. Use one where the parent genuinely does not need the detail, and require a strict output format so the handoff is checkable.
Is it true that fine-tuning replaces the need for context engineering?
No. Fine-tuning changes what the model knows how to do, not what the model sees on turn 47. A model trained on your house style still cannot read a document nobody retrieved. Use fine-tuning for behaviour and format, context engineering for facts and live state, and expect the second to move the number faster on retrieval-bound work.
What should I measure to know whether context engineering is working?
Constraint survival first, then cost per successful task. Track whether load-bearing rules still hold at turn forty, log what actually entered the window on each run, and watch the cache-hit rate behind the bill. A paired harness test with the model held fixed separates a harness win from a model upgrade.
Does any of this matter if I am shipping a chatbot rather than an agent?
Yes, though the stakes differ. A chatbot has no turn 47, so compaction rarely fires, but position still matters: the middle of a long prompt gets read least, and retrieved context can crowd out the instructions that were meant to govern it. Single-turn products mostly need selection and ordering, not the full four operations.
How do I stop a retrieved document from hijacking the run?
Treat retrieved text as data, never as instruction. Delimit it, label its provenance, and keep the rules that govern behaviour in a pinned block the retriever and summariser cannot touch. Context poisoning corrupts what remains in the window; loss removes it. Both are assembly failures, and both show up in the log if you record what entered.
Do I need new tooling for this, or does MCP cover it?
MCP standardises how tools are exposed, as JSON-RPC messages between hosts, clients and servers, and that solves one slice: tool definitions and discovery. It does not decide what gets retrieved, what gets compacted, what survives summarisation or who audits the window. Those are policy and version control, and they stay yours.
How do I count tokens accurately when character estimates drift 10 to 20%?
Count with the vendor’s tokenizer for the exact model, not by dividing characters by four. Measure the real distribution of request sizes rather than the mean, because the bill is set by the tail. Then put an alarm just below the threshold, since usage data may not flag the surcharge at all.
Does loading context carefully make the agent slower?
Sometimes, and that is the trade. Just-in-time fetching moves latency into the run itself, and a sub-agent adds a round trip. Eager loading is faster per turn and more expensive per token. The question worth asking is whether the added latency buys a task that succeeds, not whether a failing run is fast.
Is this only practical for large teams with dedicated platform staff?
No, but it needs one named owner. The assets are a context library, tool schemas, a compaction policy and an audit trail, and none of them require a platform team, only change control. What fails is the unowned version, where nobody decides what enters the window until a post-mortem forces the question.
