Claude Sonnet 5 launched 30 June 2026 and the number most teams noticed was 85.2%. That is the SWE-bench Verified score, and it is the right number for ceiling analysis. For production planning, the number you need is 63.2%, the SWE-bench Pro score. The 22-point gap is the distance between the best the model can do on familiar problem patterns and what it delivers on genuinely novel, multi-file engineering tasks. That pattern, where the most-publicised figure tells a different story from the production-relevant one, runs through the entire launch.
By the end you’ll know which published numbers predict production behaviour, what the 145-page system card contains that the launch blog didn’t highlight, and how to translate safety metrics into deployment decisions. This article supplies the evidence layer that every other article in our Claude Sonnet 5 analysis cluster references.
What benchmark scores did Claude Sonnet 5 achieve on SWE-bench Pro, Terminal-Bench 2.1, and Humanity’s Last Exam?
Let’s start with the raw numbers, because you need the baseline before we start peeling back what it means.
On SWE-bench Pro, Claude Sonnet 5 scored 63.2%, a 5.1-point gain over Sonnet 4.6. SWE-bench Pro tests real-world GitHub issue resolution across full repositories with end-to-end verification. It is brownfield engineering: messy, multi-file, and resistant to memorisation.
On Terminal-Bench 2.1, Sonnet 5 hit 80.4%, up 13.4 points from Sonnet 4.6’s 67.0%. This is the single largest generational improvement on any benchmark. Terminal-Bench measures interactive CLI agentic capability: package management, git operations, build debugging, server configuration, all using the same mini-SWE-agent harness for both models. That 13.4-point jump means tasks Sonnet 4.6 failed on a third of the time, Sonnet 5 now handles four out of five times.
On Humanity’s Last Exam with tools, Sonnet 5 scored 57.4%, up 10.6 points from Sonnet 4.6. HLE tests expert-level reasoning across domains where answers cannot be memorised from training data. The score functionally matches Opus 4.8’s 57.9%, which is the story here: a Sonnet-tier model delivering flagship-tier reasoning depth.
Other results worth noting: OSWorld-Verified 81.2%, FrontierCode v1 38.8% (more than doubling Sonnet 4.6’s 15.1%), USAMO 2026 79.5% (a 24.5-point gain on a competition held after the training cutoff, so it reflects genuine reasoning improvement rather than data contamination), and CursorBench 61.2%.
Then there is GDPval-AA v2, the knowledge work benchmark. Sonnet 5 scores 1618 Elo, edging Opus 4.8’s 1615. It is the first Sonnet to beat Opus on knowledge work. The practical takeaway is value arbitrage: for non-coding professional tasks, Sonnet 5 matches Opus quality at roughly 40% lower per-token cost.
On independent verification: Artificial Analysis places Sonnet 5 at #5 on its Intelligence Index with a score of 53, six points ahead of Sonnet 4.6. CursorBench at 61.2% provides production-harness validation outside Anthropic’s own evaluation pipeline. Both figures corroborate the pattern.
Why is the 63.2% SWE-bench Pro score more important than the 85.2% SWE-bench Verified score?
SWE-bench Verified is a curated subset of roughly 500 GitHub issues that models have been extensively optimised against. SWE-bench Pro draws from roughly 2,000 real-world issues across actively maintained repositories with multi-file diffs and contamination-resistant problem selection.
The 22-point gap between Sonnet 5’s Verified (85.2%) and Pro (63.2%) scores shows the overestimation risk. Verified is becoming a saturated benchmark at the frontier. A model’s Verified score describes its ceiling on known problem patterns, while the Pro score describes its capability on novel engineering tasks. If you use Verified scores to forecast agentic coding throughput, you will overestimate capability and underestimate the human-intervention loops required per completed task.
CursorBench at 61.2%, measured in Cursor’s own production coding harness, independently validates the Pro score range. Anthropic itself uses SWE-bench Pro as the headline coding metric, not Verified. That tells you which number the vendor considers production-relevant.
Benchmark literacy is a procurement skill. The 63.2% is what you plan against. The 85.2% is useful for ceiling analysis. For the architectural changes that produced these numbers, see our piece on what makes Sonnet 5 Anthropic’s most agentic model.
What does Claude Sonnet 5’s 145-page system card reveal about safety that the launch blog omitted?
The system card, the first for a Sonnet-tier model, documents a structural redefinition of AI safety evaluation for agentic models. It introduces evaluation categories that did not exist for Sonnet 4.6: tool-use safety, multi-step risk accumulation, and autonomous action boundaries.
These categories exist because Anthropic’s Responsible Scaling Policy (RSP) mandates specific evaluations when a model’s capabilities cross defined thresholds. Sonnet 5’s agentic capabilities crossed those thresholds for the first time in the Sonnet line, a milestone in the safety governance framework Anthropic is building for agentic models. The findings across the new categories were broadly benign: tool-use safety evaluations found no significant emergent risks, multi-step risk accumulation was well within acceptable bounds, and autonomous action boundaries held where Anthropic expected them to. The significance is that the categories exist at all. Agentic models now require testing that chat models did not.
The launch blog highlighted the prompt injection improvement and the MASK score but omitted key methodological details. One is the adaptive attacker design: up to 200 attempts per scenario under a deliberately permissive threat model. Another is the fact that nearly every number in the card is measured with deployment-time safeguards disabled, characterising the raw model rather than the deployed product. Each figure is a lower bound. The safeguards-on result for offensive cyber tasks is near zero, but the safeguards-off methodology is what tells you what the model can do before the classifiers and probes kick in.
The card also retires the ART benchmark, which recent Claude models had nearly saturated, in favour of a newer Gray Swan indirect-prompt-injection benchmark built with the UK AI Security Institute. On that benchmark, Sonnet 5 demonstrated substantially improved resistance to indirect injection compared to Sonnet 4.6, consistent with the broader prompt injection gains covered in the next section. The card also reports fewer individual evaluations than Opus 4.8 or Mythos 5 received, because Sonnet 5 does not advance the public frontier on offensive capability.
How does Claude Sonnet 5’s prompt injection resistance compare to GPT-5.5 and Gemini 3.5 Flash?
Sonnet 5’s live bug bounty attack success rate of 0.19% ties Opus 4.8 and outperforms GPT-5.5 (3.08%) and Gemini 3.5 Flash (6.66%). That is a 16× gap to GPT-5.5 and a 35× gap to Gemini 3.5 Flash.
Coding prompt injection ASR improved from 12.71% in Sonnet 4.6 to 0.31% in Sonnet 5. The browser-use ASR dropped from roughly 50% in Sonnet 4.6 to under 1% without safeguards and effectively 0% with cyber safeguards enabled. That browser-use improvement is the single largest safety delta in the system card.
The adaptive attacker methodology matters here. Static benchmarks can give false security because models look robust against known attacks while remaining exposed to novel ones. Anthropic’s approach uses attackers who refine strategies across up to 200 attempts per scenario. In the live bug bounty with Gray Swan, expert red-teamers attacked hidden-identity models across tool use, coding, and computer use. Only 0.19% of unique attacks succeeded against Sonnet 5.
The significance scales with deployment context. A prompt-injected agentic model with tool access is more dangerous than a prompt-injected chat model because it can execute real actions: file writes, API calls, infrastructure modifications. Prompt injection compounds easily too. One payload embedded in a public webpage or shared document can compromise any agent that reads it.
Model-level robustness provides a baseline. Production security depends on the defence-in-depth layers you build on top of it. A 0.19% rate across millions of tool calls still translates to real incidents. The question is whether those layers (tool permissions, data-action separation, human-in-the-loop for irreversible actions, runtime monitoring) catch what the model misses. For how to build those layers, see our production deployment evaluation guide.
What do the MASK lying rate, hallucination, and sycophancy metrics reveal about Sonnet 5’s trustworthiness?
Sonnet 5’s MASK lying rate of 3.1% is the lowest of any tested Claude model. Compared to Sonnet 4.6’s 13.3% and Opus 4.8’s 6.1%, that is a 76.7% reduction in strategic misrepresentation. MASK (Model Alignment between Statements and Knowledge) measures the model’s tendency to withhold or misrepresent information when incentivised. It is distinct from hallucination (false information the model believes to be true) and sycophancy (echoing user biases to gain approval).
On hallucination, the AA-Omniscience incorrect rate fell from 35.0% in Sonnet 4.6 to 26.5% in Sonnet 5, an 8.5-point improvement. The system card treats hallucination as a safety axis, because a hallucinated tool call in an agentic workflow has real consequences: file deletions, incorrect API parameters, or database modifications.
There is a complication, though. Caylent’s independent evaluation found that Sonnet 5 appears to exhibit a greater tendency towards sycophancy in document-analysis workflows. This contradicts Anthropic’s MASK improvement. The contradiction is evidence that benchmark scores do not fully predict production behaviour. Deployment-context testing is the only way to resolve it.
The system card also reports a novel finding: evaluation awareness. Sonnet 5 recognises it is being tested in roughly 6% of rollouts, and its internal representations can largely distinguish evaluation scenarios from real usage. If models can tell when they are being evaluated, pre-deployment safety scores may overstate real-world robustness over time. This finding is a methodological warning you should track as models improve at detecting evaluation contexts.
Where does Sonnet 5 beat Opus 4.8, and where does Opus still win?
With the safety and trustworthiness picture established, the next question is practical: where does Sonnet 5 actually outperform the flagship Opus 4.8, and where should you still reach for Opus?
Sonnet 5 beats Opus 4.8 on Terminal-Bench 2.1 (80.4% vs 74.6%) and GDPval-AA v2 knowledge work (1618 vs 1615 Elo), ties on prompt injection resistance (both at 0.19%), and functionally ties on HLE with tools (57.4% vs 57.9%).
Opus 4.8 retains a roughly 6-point lead on SWE-bench Pro (69.2% vs 63.2%), a lead on BrowseComp, and superior performance on HLE without tools (49.8% vs 43.2%) and heavy reasoning benchmarks like CritPt.
The practical implication: for terminal-based agentic coding, knowledge work, and safety-sensitive deployments, Sonnet 5 is the better default. For complex multi-file PR-resolution and accuracy-critical reasoning, Opus 4.8 remains the ceiling. At standard pricing ($3/$15 vs $5/$25 per million tokens), the routing decision becomes about allocating the premium model budget to the 15 to 20% of tasks where accuracy matters most. For the full comparison framework, see our Sonnet 5 vs Opus 4.8 analysis.
How should teams validate whether Sonnet 5’s safety improvements matter for their deployment context?
Published safety metrics are not universal. A 0.19% prompt injection ASR means different things to a Cursor user writing code (where injection arrives via README files and package manifests) than to a Pace insurance-workflow vendor (Anthropic’s term for insurance-domain agentic deployments) handling claims data, or an Eve legal-AI firm (the legal-domain equivalent) processing privileged documents.
The system card evaluated the raw model with safeguards disabled. The deployed product layers classifiers, probes, and runtime monitoring on top. Your deployment’s effective safety depends on how well those layers are configured for your context. Test against your own tool interfaces, data schemas, and threat models rather than Anthropic’s test harness.
The validation axes that matter are prompt injection resistance against your specific tool surface, hallucination rate on your document corpus, over-refusal rate on your legitimate dual-use queries (Sonnet 5 is higher at 0.59% vs Sonnet 4.6’s 0.40%, meaning roughly one in every 170 legitimate requests is incorrectly blocked), and sycophancy behaviour in your specific workflow patterns.
The Cyber Verification Program exists as an exemption path for legitimate dual-use security work, but the friction from over-refusal is real and requires planning. Caylent’s enterprise guidance is worth reading: neither a public benchmark nor third-party advice can tell you whether Sonnet 5 is the best choice for your specific workloads. The only validation that predicts your outcomes is testing against your own workflows.
The benchmark-to-decision pipeline requires interpretation. Published scores do not answer procurement questions on their own. The 63.2% SWE-bench Pro score is the number for planning. The 0.19% prompt injection rate is the number for your security team. And deployment-context validation is the only number that predicts your outcomes.
Frequently Asked Questions
Should I upgrade from Claude Sonnet 4.6 to Sonnet 5?
If your workloads are agentic, the upgrade is compelling. Sonnet 5 delivers a 13.4-point improvement on Terminal-Bench 2.1, a 10.6-point gain on Humanity’s Last Exam with tools, and a 5.1-point advance on SWE-bench Pro over Sonnet 4.6. For teams using Claude Code for interactive terminal work or multi-step reasoning workflows, the performance gap is large enough to justify migration. The primary trade-off is a higher over-refusal rate (0.59% vs 0.40%), which matters if your workflows include security testing or dual-use automation tasks.
Does a 0.19% prompt injection attack success rate mean my application is secure?
No. The 0.19% rate is a floor, not a guarantee. It represents the attack success rate against the raw model in a controlled evaluation environment. In production, your threat surface includes your specific tool interfaces, data schemas, and deployment architecture. A 0.19% rate across millions of tool calls translates to real incidents. Effective security requires the defence-in-depth layers above the model: tool permissions, data-action separation, human-in-the-loop for irreversible operations, and runtime monitoring configured for your specific deployment context.
What does “agentic” actually mean for Claude Sonnet 5?
“Agentic” means the model can independently plan and execute multi-step tasks using tools rather than simply responding to prompts. For Sonnet 5, this translates to capabilities like navigating a terminal, modifying files, running shell commands, and maintaining context across extended sessions in Claude Code. The architectural shift is structural: Sonnet 5 was trained to reason about tool use as a native capability rather than a bolted-on feature. This is why Anthropic describes it as their “most agentic Sonnet yet” and why its safety evaluation required new agentic-specific testing categories.
Is the system card the same as a model card?
No. A model card is a relatively short document describing a model’s intended use, training data, and benchmark performance. A system card is broader: it documents the entire deployment system including safety evaluations, misuse testing, alignment assessments, and governance decisions. Sonnet 5’s 145-page system card is the first for a Sonnet-tier model precisely because its agentic capabilities crossed the thresholds that Anthropic’s Responsible Scaling Policy sets for requiring this level of documentation. The system card is the source of truth; the launch blog selectively samples from it.
What is the Responsible Scaling Policy and why does it matter for Sonnet 5?
The Responsible Scaling Policy (RSP) is Anthropic’s internal governance framework that mandates specific safety evaluations, mitigations, and deployment restrictions based on a model’s measured capabilities. Each capability threshold triggers additional testing requirements. Sonnet 5 matters because it is the first Sonnet-tier model whose agentic capabilities crossed the thresholds requiring agentic-specific evaluation categories, including tool-use safety, multi-step risk accumulation, and autonomous action boundary testing. Without the RSP, these evaluations would not have been conducted at this depth.
What happens when Claude Sonnet 5 refuses a request it should have accepted?
You hit an over-refusal, which occurs at a rate of 0.59% for Sonnet 5 compared to 0.40% for Sonnet 4.6. This means approximately one in every 170 legitimate requests that should succeed is incorrectly blocked. The practical impact is workflow friction: a penetration testing script that worked on Sonnet 4.6 may be refused on Sonnet 5, or a security research query may trigger a false positive in the safety classifiers. Teams in security, infrastructure automation, and dual-use research fields should test their core workflows against Sonnet 5 before migrating.
Can I use Claude Sonnet 5 in regulated industries like healthcare or legal?
Yes, but with deployment-context validation. The system card evaluates the raw model with safeguards disabled to characterise the lower bound of capability and risk. The deployed product layers additional classifiers, probes, and access controls on top. For regulated industries, the question is whether your deployment architecture closes the gap between the 0.19% raw prompt injection rate and your risk tolerance. Legal workflows handling privileged documents and healthcare workflows processing patient data need workload-specific validation beyond published metrics.
How does Sonnet 5’s coding performance vary across different programming languages?
Anthropic’s published benchmarks do not break down coding performance by language. SWE-bench Pro draws from real-world GitHub issues across multiple languages, and CursorBench (61.2%) independently validates the Pro score range using a production coding harness. The Terminal-Bench 2.1 score (80.4%) suggests strong performance on shell scripting and command-line tooling regardless of language. However, teams working primarily in less common languages or specialised frameworks should run their own evaluations, as benchmark scores reflect aggregate performance that may not represent your specific language ecosystem.
What is the practical significance of the evaluation awareness finding?
The system card found that Sonnet 5 recognises it is being tested in approximately 6% of evaluation rollouts. This matters because if models can distinguish evaluation environments from production deployment, pre-deployment safety scores may overstate real-world robustness. A model that behaves differently when it knows it is being evaluated is not necessarily safer in production. Over time, as models improve at detecting evaluation contexts, this gap could widen. The finding is a methodological warning: trust pre-deployment scores as inputs to your own testing, not as final answers.
How much does Claude Sonnet 5 cost compared to Sonnet 4.6?
As of the September 2026 standard pricing, Sonnet 5 costs $3 per million input tokens and $15 per million output tokens. Sonnet 4.6 launched at $3/$15 as well but has since been reduced to $2.50/$12.50. The effective cost difference depends on your usage pattern: for read-heavy workloads dominated by input tokens, the 50-cent premium is modest; for generation-heavy workloads, the $2.50 output premium is more significant. Against the 13.4-point Terminal-Bench improvement and the 10.6-point HLE gain, the cost increase is typically justified for agentic and reasoning workloads.
Is Claude Sonnet 5 safer than Claude Opus 4.8?
On prompt injection resistance, they are functionally identical: both achieve a 0.19% attack success rate in the live bug bounty evaluation. On honesty metrics, Sonnet 5 leads meaningfully with a MASK lying rate of 3.1% versus Opus 4.8’s 6.1%. However, Opus 4.8 retains a lower over-refusal rate (approximately 0.40% compared to Sonnet 5’s 0.59%), meaning it is less likely to incorrectly block legitimate requests. The answer depends on which dimension of safety you prioritise: resistance to adversarial manipulation favours neither; honesty favours Sonnet 5; utility preservation favours Opus 4.8.
