Something strange is happening in software development. Your developers are excited about AI coding tools. They feel faster, more productive, more creative. They’re generating code at speeds that would have seemed impossible two years ago. And yet your delivery metrics haven’t moved.
You’re not imagining this. In a randomised controlled trial by METR, experienced developers predicted AI tools would make them 24% faster. They actually took 19% longer. And even after experiencing that slowdown, they still believed they’d been sped up by 20%. That’s a 43-percentage-point gap between perception and reality, and it’s playing out in engineering organisations worldwide.
Meanwhile, the term “vibe coding”—coined by OpenAI co-founder Andrej Karpathy in February 2025 to describe accepting AI-generated code without reviewing it—became Collins English Dictionary’s Word of the Year. A quarter of Y Combinator‘s Winter 2025 batch had codebases that were 95% AI-generated. And industry-wide adoption of AI coding assistants has hit 91% across surveyed companies.
This guide synthesises evidence from seven independent studies, explores arguments both for and against AI coding adoption, addresses the organisational challenges you’re dealing with right now—from junior skill development to senior scepticism to security risks—and provides actionable frameworks for responsible adoption. It’s built as a hub linking to eight detailed articles, each exploring a dimension of this challenge in depth.
Start wherever makes sense for you. If you’re trying to understand the terminology, begin with definitions. If you’re ready to act, jump to the framework.
Vibe coding—coined by Andrej Karpathy in February 2025—describes accepting AI-generated code without reviewing its internal structure, essentially “giving in” to code you don’t fully understand. It contrasts sharply with AI-assisted engineering, where developers use AI tools while maintaining code review, testing, and complete understanding. The distinction matters because vibe coding produces throwaway prototypes effectively but creates mounting technical debt and security risks in production systems. Collins English Dictionary named it Word of the Year for 2025.
The distinction from professional practice matters. There’s a spectrum here, and where your team sits on it has real consequences for code quality, security, and long-term maintainability.
At one end, vibe coding treats AI output as final. You describe behaviour, accept the code, and move on. If something breaks, you paste the error message back into the AI and hope for the best. Karpathy described his own code “growing beyond my usual comprehension.” For prototypes and personal experiments, this works fine. For anything you need to maintain, it’s a problem.
In the middle sits what Kent Beck—creator of Test-Driven Development and signatory of the Agile Manifesto—calls “augmented coding.” Beck spent four weeks building a complex B+ tree implementation using AI tools while maintaining strict discipline: TDD cycles, careful review of intermediate results, small frequent commits, and readiness to intervene when the AI went off track. As Beck puts it: “In augmented coding you care about the code, its complexity, the tests, & their coverage.”
At the professional end, AI-assisted engineering uses AI tools while maintaining the same standards you’d apply to human-written code: review, testing, documentation, and full understanding. Simon Willison, co-creator of Django, draws a clear line: “If an LLM wrote every line of your code, but you’ve reviewed, tested, and understood it all, that’s not vibe coding in my book—that’s using an LLM as a typing assistant.”
The distinction matters because the problems showing up in research—the productivity paradox, security vulnerabilities, technical debt accumulation—stem largely from vibe coding practices migrating into production environments where they don’t belong.
For the full terminology breakdown, framework comparisons, and guidance on where each approach applies, see What Is Vibe Coding and How Does It Differ from AI-Assisted Engineering.
The productivity paradox reveals a 43-percentage-point gap between perception and reality: developers using AI tools predict 24% faster completion but measure 19% slower in controlled studies. This paradox occurs because AI accelerates code generation while creating bottlenecks elsewhere—review queues grow, pull requests become larger, and code churn doubles. Individual velocity increases, but organisational throughput stalls because development is a system, not just typing speed. METR’s randomised controlled trial—the gold standard for establishing causality—confirmed this invisible slowdown across 16 experienced developers and 246 real issues.
This paradox happens because AI accelerates the part of development that was never the bottleneck.
Think of it through the lens of Amdahl’s Law, a concept familiar from systems design. If code generation is 20% of total development time and you make it 10x faster, your maximum system speedup is about 22%. But the other 80%—understanding requirements, designing architecture, reviewing code, writing tests, debugging, deploying—remains unchanged.
What actually happens when AI accelerates code generation is that downstream bottlenecks get worse. GitClear‘s analysis of 211 million lines of code found that refactoring—the maintenance work that keeps codebases healthy—dropped from 25% of changed lines in 2021 to under 10% by 2024. Code duplication increased approximately 4x. Code churn—prematurely merged code that gets rewritten shortly after merging—nearly doubled. Copy-pasted code exceeded moved code for the first time in two decades.
Faros AI‘s study of over 10,000 developers across 1,255 teams confirmed the pattern at organisational scale: developers completed 21% more tasks, but companies saw no measurable improvement in delivery velocity or business outcomes. Pull request volume increased dramatically while review times grew longer because the reviews themselves didn’t get any easier—there was just more to review.
The DORA 2025 Report captures this dynamic precisely: “AI’s primary role is as an amplifier, magnifying an organisation’s existing strengths and weaknesses.” If your processes are strong, AI can help. If they’re not, AI makes existing problems worse, faster.
For the complete evidence synthesis including all seven studies, the mechanisms behind the paradox, and how to explain it to your board, see The AI Productivity Paradox – Why Developers Feel Fast But Deliver Slow.
Seven independent studies reveal consistent patterns: individual developers generate code faster but organisations don’t deliver faster. METR’s RCT showed a 19% slowdown despite a perceived 24% speedup, GitClear documented 4x more code duplication, CodeRabbit found 1.7x more issues per pull request, Apiiro measured 10x more security vulnerabilities, and Faros AI confirmed that 21% more tasks completed delivered zero business outcome improvement. Vendor studies from Microsoft and Accenture report gains, but methodology differences matter: observational studies versus randomised trials, task completion versus business outcomes tell very different stories.
The table below summarises the major studies so you can compare methodologies and findings at a glance.
| Study | Method | Sample | Key Finding | |—|—|—|—| | METR (2025) | Randomised controlled trial | 16 experienced developers, 246 issues | 19% slowdown despite perceived 24% speedup | | GitClear (2020-2024) | Longitudinal code analysis | 211 million lines of code | Refactoring collapsed from 25% to <10%, duplication up 4x | | CodeRabbit (2025) | Pull request analysis | 470 open-source PRs | 1.7x more major issues, 75% more logic errors in AI code | | Apiiro (2025) | Enterprise code analysis | Fortune 50 companies, thousands of developers | 10x more security findings, 322% more privilege escalation | | Faros AI (2025) | Organisational metrics | 10,000+ developers, 1,255 teams | 21% more tasks completed, zero delivery improvement | | DORA (2025) | Industry survey + model | Broad industry sample | AI amplifies existing strengths and weaknesses | | DX Q4 (2025) | Developer survey | 135,000 developers, 435 companies | Developers report saving 3.6 hours per week |
Vendor research tells a consistently positive story. Microsoft reported a 26% task completion increase. Stack Overflow surveys show 72% favourable Copilot ratings and 84-90% adoption rates. DX‘s Q4 2025 report found developers report saving 3.6 hours per week.
The difference comes down to what gets measured and how. Vendor studies typically use observational methods and measure subjective satisfaction or task completion. Independent studies use randomised controlled trials or longitudinal data and measure business outcomes and code quality. Self-reported time savings don’t match clock time. Task completion doesn’t equal delivery velocity. And “feeling productive” is not the same as delivering value.
For your decision-making, weight independent research more heavily—especially studies using randomised controlled trials, which are the gold standard for establishing whether something actually causes an effect rather than merely correlating with one.
For the detailed methodology comparisons, study-by-study findings, and a framework for evaluating research claims, explore both The AI Productivity Paradox and The Case Against Vibe Coding.
Critics argue vibe coding sacrifices long-term sustainability for short-term velocity. Evidence shows AI-generated code increases technical debt 30-41% (refactoring collapse, 4x duplication), security vulnerabilities 2.74-10x (privilege escalation, architectural flaws), and cognitive complexity 39%. Kent Beck, ThoughtWorks, and experienced engineers warn that code you don’t understand becomes unmaintainable, creates debugging nightmares (“the 70% Problem”), and erodes the craftsmanship practices that enable sustainable pace. For smaller teams without dedicated maintenance capacity, the impact is disproportionate.
GitClear’s data shows what happens to codebases as AI adoption grows. Refactoring—the practice of improving code structure without changing behaviour—has been in decline since AI coding tools gained traction. This isn’t developers choosing to skip refactoring. It’s a structural shift: AI generates new code readily but doesn’t initiate the maintenance work that keeps codebases healthy. Meanwhile, code duplication has risen sharply, violating the DRY (Don’t Repeat Yourself) principle that’s fundamental to maintainability.
CodeRabbit’s analysis quantified the quality gap directly: AI co-authored code showed 1.7x more issues overall and 75% more logic errors—incorrect dependencies, flawed control flow, misconfigurations. These aren’t syntax problems. They’re the kind of bugs that pass automated tests but break in production under conditions the AI didn’t anticipate. The long-term costs of vibe coding go well beyond the obvious errors.
Experienced developers describe a recurring pattern with AI-generated code: it gets you roughly 70% of the way there and looks correct, but contains subtle issues that take longer to debug than writing the code from scratch would have. Authentication that works for the happy path but fails on edge cases. Data handling that works at small scale but creates race conditions under load. API integrations that pass basic tests but fail silently when services return unexpected responses.
Kent Beck, despite being enthusiastic about augmented coding, acknowledges this tension: “I feel good about the correctness and performance, not so good about the code quality. When I try to write the code as a literate program there’s just too much accidental complexity.”
If your team is already stretched thin, the implications are worth taking seriously. Larger enterprises can absorb a significant increase in technical debt across dedicated maintenance teams. You probably can’t. When your codebase becomes harder to maintain, it’s the same people who built it who have to clean it up, and they’re already busy building the next thing.
For the full risk assessment including quantified technical debt costs, the 70% Problem in detail, and Kent Beck’s and ThoughtWorks’ perspectives, see The Case Against Vibe Coding – Understanding, Craftsmanship, and Long-Term Costs.
Proponents argue AI democratises software creation, eliminates cognitive toil (boilerplate, repetitive patterns), and enables non-programmers to build functional applications. Evidence shows gains exist in specific contexts: greenfield projects with no legacy constraints, simple repetitive patterns, prototyping, and learning. Stack Overflow reports 72% favourable Copilot ratings with 84-90% adoption. The key is strategic selective adoption—using AI for cognitive toil while avoiding complex architecture and security-critical code where it consistently underperforms.
The strongest argument for AI coding tools is democratisation. Simon Willison puts it well: “I believe everyone deserves the ability to automate tedious tasks in their lives with computers. You shouldn’t need a computer science degree or programming bootcamp.” Kevin Roose, a New York Times journalist and non-programmer, used vibe coding to build several small-scale applications he described as “software for one”—personalised tools that would never have existed without AI lowering the barrier.
For experienced developers, the benefit is different. Kent Beck, working on his B+ tree implementation, described the experience as addictive: “I make more consequential programming decisions per hour, fewer boring vanilla decisions. Yak shaving mostly goes away.” He used AI to write a C extension for Python performance, run coverage testing, and propose tests—tasks that would have been daunting without AI assistance. The key is that Beck already understood what he was building. AI handled cognitive toil—boilerplate, repetitive patterns, test scaffolding—while he focused on architecture and design.
The evidence points to specific contexts where AI tools deliver real gains:
The problems arise when AI use extends from these appropriate contexts into complex architecture, security-sensitive code, or poorly documented legacy systems. Stack Overflow’s data illustrates this tension: 72% of developers rate Copilot favourably, but 66% report “almost right but not quite” frustration—code that looks correct but needs careful review and modification.
The 25% of Y Combinator’s Winter 2025 startups with 95% AI-generated codebases represent a real signal. But startups building greenfield MVPs face very different constraints than established companies maintaining production systems with paying customers. Understanding when AI coding tools actually help is the critical skill for any engineering leader navigating adoption decisions.
For the full balanced evaluation including selective adoption criteria and when AI genuinely helps vs hinders, see The Case For AI Coding Tools – Democratisation, Velocity, and When They Actually Help.
If junior developers use AI for tasks that traditionally built expertise—debugging, reading documentation, understanding error messages—how do they develop judgment and deep knowledge? This succession planning concern threatens your senior engineering pipeline: today’s juniors become tomorrow’s seniors, but AI-dependent skill development may create a generation unable to debug AI-generated code or make architectural decisions. The apprenticeship model requires evolution, not abandonment, and the training choices you make now determine your engineering capability in 2030.
Software engineering has always relied on a form of apprenticeship. Juniors learn by doing hard things badly, getting feedback, and gradually developing intuition. They read code, debug failures, and build mental models of how systems work. AI tools shortcut this process. When a junior developer pastes an error message into an AI assistant instead of reading the stack trace, they get a fix faster but miss the learning that comes from understanding why the error occurred.
Anthropic‘s own research quantified this effect. In a randomised controlled trial, junior developers using AI assistance scored significantly lower on comprehension assessments—50% compared to 67% for those coding by hand, roughly a two-letter-grade difference. The largest gap appeared in debugging questions, precisely the skill you need most when reviewing AI-generated code.
The interaction pattern matters enormously. Anthropic found that developers who delegated completely to AI scored worst, while those who used AI for code generation but then asked for explanations—building comprehension after the fact—scored nearly as well as hand-coders. The tool isn’t the problem. How it’s used is.
Think of this as a succession planning challenge. Your senior engineers in 2030 are the juniors you’re hiring and training now. Kent Beck’s framework is useful here: AI deprecates some skills (manual boilerplate coding, memorising syntax) but amplifies others (architectural vision, strategic thinking, taste). Training needs to evolve so that juniors learn to validate AI suggestions, understand generated code deeply, and maintain the architectural judgment that no AI currently provides.
Pluralsight‘s research warns that “AI-induced skills decay isn’t visible—it appears through what falls through cracks” and becomes costly only after problems emerge in production. For smaller teams, you can’t afford to lose a generation of skill development. Your training programmes need to be deliberate about which skills AI is allowed to shortcut and which it absolutely must not. The question of who trains the next generation of engineers is one that requires strategic intent, not improvisation.
For the complete analysis including training frameworks, skill development strategies, and how to structure junior development in an AI-augmented environment, see Junior Developers in the Age of AI – Who Trains the Next Generation of Engineers.
Security vulnerabilities increase 2.74-10x with AI-generated code, with 45% failure rates on secure coding benchmarks. Specific risks include 322% more privilege escalation paths, 40% higher secrets exposure, and 153% more architectural design flaws. For organisations in regulated industries (healthcare, finance) or those with enterprise customers requiring SOC2, ISO, GDPR, or HIPAA compliance, AI-generated code without enhanced security review creates specific risks—audit trails, data residency, and liability questions that remain largely unresolved. Quality gates and security review processes require redesign for AI-generated code.
What makes AI security risks worth attention is where they occur. Apiiro found that AI assistants reduced trivial syntax errors by 76%—the easy stuff that linters catch anyway—but created more of the architectural flaws that are genuinely harmful. Their blunt assessment: “AI is fixing the typos but creating the timebombs.”
The reason is structural. AI tools generate code based on patterns without understanding the security context of the broader system. They don’t know that a particular service handles authentication, that credentials shouldn’t be propagated across microservices, or that a database query needs parameterisation to prevent injection. In one documented case, a single AI-driven pull request changed an authorisation header across multiple services, but one downstream service wasn’t updated, creating a silent authentication failure.
Real incidents illustrate the stakes. Lovable, a Swedish vibe coding platform, had 170 out of 1,645 apps with security vulnerabilities that allowed personal information access by anyone. Replit Agent deleted a user’s database despite explicit instructions not to make changes. AI-assisted developers in Apiiro’s data exposed Azure Service Principals and Storage Access Keys nearly twice as often as those coding manually.
If you operate in regulated industries or sell to enterprise customers requiring SOC2, ISO, GDPR, or HIPAA compliance, AI-generated code creates specific risks. Audit trails become harder to maintain when code is generated rather than written. Data residency concerns arise when proprietary code passes through external AI services. Liability questions remain largely unresolved.
Apiiro’s recommendation is straightforward: “If you’re mandating AI coding, you must mandate AI AppSec in parallel. Otherwise, you’re scaling risk at the same pace you’re scaling productivity.” Understanding the full scope of AI-generated code security risks is essential before expanding your team’s AI usage in production systems.
For the comprehensive security risk assessment, compliance framework analysis, specific vulnerability patterns, and a quality gates playbook, see AI-Generated Code Security Risks – Why Vulnerabilities Increase 2.74x and How to Prevent Them.
Senior engineers adopt AI tools at lower rates than juniors and express greater scepticism—not because they resist innovation, but because they’ve debugged enough production systems to recognise long-term sustainability risks. They see code quality degradation, understand maintenance burdens of technical debt, and worry about code review bottlenecks created by AI-accelerated development. Their scepticism is legitimate expertise, not Luddite resistance, and your AI adoption strategy requires their buy-in to succeed.
Senior engineers have spent years maintaining codebases where “clever” shortcuts became maintenance nightmares. They’ve debugged subtle bugs in code they didn’t write. They’ve accumulated technical debt that slowed delivery for months or years. When they look at AI-generated code and see excessive duplication, architectural shortcuts, and context-free implementations, they recognise patterns that create long-term problems—patterns juniors haven’t experienced yet.
They also face the direct consequences of AI-accelerated development. Apiiro’s data shows AI-assisted developers producing 3-4x more commits packaged into fewer but significantly larger pull requests. Bigger PRs slow review, dilute reviewer attention, and raise the odds that a subtle break slips through. The people doing those reviews? Predominantly senior engineers. They’re absorbing the cost of AI-accelerated development without the satisfaction of rapid code generation that juniors enjoy.
Fast Company reported in September 2025 on “the vibe coding hangover,” with senior software engineers describing “development hell” when working with AI-generated codebases. This isn’t abstract concern—it’s lived experience from people who have to make the code work in production.
The political dimension matters. Brian Armstrong at Coinbase mandated AI coding assistants for all engineers, reportedly firing those who refused. But he also admitted “we’re still figuring out” how to manage AI-coded codebases. Stripe‘s John Collison captured the tension precisely: “It’s clear that it is very helpful to have AI helping you write code. It’s not clear how you run an AI-coded codebase.”
Mandates create resistance and don’t teach strategic usage. Your organisation needs both the enthusiasm juniors bring and the quality instincts seniors provide. Building consensus requires validating senior concerns as legitimate expertise, not dismissing them as resistance to change. The patterns of how senior engineers are adapting—or not—reveal important signals about what makes AI adoption succeed or fail.
For adoption pattern analysis, team dynamics strategies, and a practical guide to building consensus across experience levels, see How Senior Engineers Are Adapting to AI Coding Tools or Resisting Them.
AI coding tools span three categories with different risk profiles: autocomplete tools (GitHub Copilot‘s inline suggestions), chat-based assistants (ChatGPT, Claude for code generation), and agentic systems (Cursor Composer, Replit Agent making autonomous multi-file changes). Autocomplete carries lowest risk but limited impact; agents offer highest capability but greatest vulnerability potential. Tool selection depends on your team’s maturity, codebase complexity, and risk tolerance—though usage policy ultimately matters more than which specific tool you choose.
Autocomplete tools like GitHub Copilot’s inline suggestions offer the lowest risk and lowest impact. They suggest code completions as you type, similar to a context-aware spell checker for code. Most developers start here, and the vast majority of AI usage remains at this level—Faros AI found that even among active users, most rely only on autocomplete features.
Chat-based assistants—ChatGPT, Claude, Copilot Chat—allow conversational code generation. You describe what you need, get code back, and decide what to use. The quality depends entirely on your ability to evaluate the output. Simon Willison notes that Claude Artifacts provides a sandbox that “prevents accidents from causing harm elsewhere,” while Cursor, initially intended for professional developers, has fewer safety rails.
Agentic systems like Cursor’s Agent mode, Copilot’s coding agent, and Replit Agent make autonomous multi-file changes, run terminal commands, and create pull requests from issue descriptions. These offer the highest capability but the greatest vulnerability potential—a single agentic change can propagate errors across an entire codebase. GitHub Copilot’s coding agent created over a million pull requests between May and September 2025.
Here’s the thing: vibe coding is possible with any of these tools. A developer can accept Copilot suggestions without review just as easily as they can paste ChatGPT output without understanding it. Conversely, a disciplined engineer can use agentic tools responsibly by reviewing every change. Your usage policy—defining when to use AI, when to avoid it, and what quality gates apply—matters far more than which specific tool your team adopts.
When evaluating tools, look beyond popularity. Consider security posture (where does your code go?), data residency (does proprietary code leave your environment?), integration with existing workflows, and cost at your team’s scale. Enterprise tools typically offer better audit trails and compliance features, but they also carry higher licensing costs that need justifying against measurable outcomes.
For more on how tool selection fits into a broader adoption strategy, including decision frameworks and quality gate specifications, see A Framework for Responsible AI-Assisted Development – When to Use AI and When to Avoid It.
Expert perspectives span a spectrum: Andrej Karpathy coined “vibe coding” describing throwaway projects, Simon Willison argues AI-assisted programming requires code understanding, Kent Beck demonstrates “augmented coding” preserving craftsmanship, while Brian Armstrong (Coinbase CEO) mandates adoption despite admitting “we’re still figuring out” management. Trust research organisations (METR, DORA, GitClear) over vendor claims, and experienced engineers over executive enthusiasm when making evidence-based adoption decisions.
The most reliable voices are those with rigorous methodology and no product to sell. METR, a non-profit AI evaluation organisation, conducted the gold-standard randomised controlled trial referenced throughout this guide. DORA, Google Cloud’s research programme, published the 2025 report introducing an AI Capabilities Model that identifies the organisational factors determining whether AI scales. GitClear analysed 211 million lines of code longitudinally, tracking quality changes over five years.
Among practitioners, Kent Beck brings decades of software engineering credibility. His augmented coding framework demonstrates that AI and craftsmanship coexist when you maintain discipline. Simon Willison—over 80 vibe coding experiments published—provides pragmatic boundary definitions. He won’t commit code he couldn’t explain to someone else.
Executive enthusiasm tends to run ahead of evidence. Armstrong mandated AI adoption at Coinbase. Lemonade CEO Daniel Schreiber told employees “AI is mandatory.” Citi bank rolled out agentic AI to 40,000 developers. These are signals about market direction, not evidence about effectiveness.
Andrew Ng argued that the term “vibe coding” itself misleads people into assuming software engineers just “go with the vibes” when using AI tools—a fair point about terminology, even if it doesn’t change the underlying practices. Gary Marcus, a cognitive scientist, cautioned that AI-generated code reproduces existing patterns without the originality that comes from deep understanding. Jeremy Howard of Fast.ai warned about AI complacency, urging teams to “build to last” rather than optimise for speed alone.
Vendor research from Microsoft, GitHub, and Accenture consistently shows productivity gains—but using observational methods and self-reported metrics rather than the controlled experiments that independent researchers employ. Both types of evidence are useful. Neither type tells the complete story on its own.
The weight of evidence, methodology considered, supports selective adoption rather than either wholesale embrace or blanket resistance.
For more on how to evaluate competing claims and identify which research should inform your decisions, see The AI Productivity Paradox – Why Developers Feel Fast But Deliver Slow.
Responsible AI adoption requires four elements: decision frameworks (when to use AI vs. avoid based on task complexity, codebase maturity, security criticality), measurement approaches (DORA metrics, cycle time, not vanity metrics), quality gates (automated security scanning, review triggers, acceptance criteria upfront), and process redesigns (handling increased PR volume, training approaches, consensus-building). The goal isn’t banning or mandating AI tools—it’s strategic selective adoption that captures individual gains at organisational scale.
Not every coding task benefits equally from AI assistance. The evidence points to a matrix: task complexity and codebase maturity determine where AI helps and where it hurts.
AI works well for greenfield projects with well-defined requirements, boilerplate and repetitive patterns, test generation and coverage expansion, documentation drafts, and exploration of unfamiliar frameworks. AI struggles with complex architecture and distributed systems, security-sensitive code, poorly documented legacy systems, performance-critical sections, and system integration challenges.
Your policy should be specific enough that a developer facing a particular task can determine whether AI assistance is appropriate without asking their manager. The responsible AI-assisted development framework provides decision matrices and policy templates to make this practical.
Measure business outcomes, not activity. DORA’s four key metrics—deployment frequency, lead time for changes, change failure rate, and mean time to restore service—tell you whether your organisation is actually delivering faster. If AI tools help, you’ll see faster deployment frequency with stable or improving change failure rates.
Watch out for vanity metrics. Lines of code, PR count, and commit volume all increase with AI tools. None of them correlate with business value. If your PR volume increases but your cycle time stays flat or grows, the productivity gains are illusory.
AI-generated code needs enhanced review, not less. Automated quality gates reduce the manual review burden, and your review processes need to handle the increased volume without burning out your senior engineers. Beck’s approach to augmented coding provides a useful model: maintain discipline around testing, keep commits small and focused, and treat AI output the way you’d treat code from a prolific but inexperienced contributor.
The DORA 2025 Report identifies organisational capabilities that determine whether AI tools scale positively: the greatest returns come not from the tools themselves, but from strategic focus on the underlying organisational system.
For the complete implementation playbook including decision matrices, metrics dashboards, quality gate specifications, and review process redesign templates, see A Framework for Responsible AI-Assisted Development – When to Use AI and When to Avoid It.
What Is Vibe Coding and How Does It Differ from AI-Assisted Engineering Precise definitions, the terminology spectrum (vibe coding vs AI-assisted vs augmented coding), legitimate use cases, and when each approach applies. Essential foundation before exploring evidence or implementation.
The AI Productivity Paradox – Why Developers Feel Fast But Deliver Slow Comprehensive evidence synthesis from 7 major studies, productivity placebo mechanisms, Amdahl’s Law application, and why individual velocity doesn’t translate to organisational throughput. Critical for understanding why DORA metrics stay flat despite developer excitement.
The Case Against Vibe Coding – Understanding, Craftsmanship, and Long-Term Costs Quantified technical debt accumulation (30-41% increase), code quality degradation evidence, the 70% Problem, and craftsmanship perspectives from Kent Beck and ThoughtWorks. Essential for understanding long-term sustainability risks.
The Case For AI Coding Tools – Democratisation, Velocity, and When They Actually Help Balanced exploration of legitimate benefits, democratisation evidence, cognitive toil reduction, strategic selective adoption contexts, and when AI genuinely helps vs. hinders. Required reading for developing nuanced policy positions.
Junior Developers in the Age of AI – Who Trains the Next Generation of Engineers Skill development implications, apprenticeship model evolution, succession planning concerns, and training frameworks for responsible AI usage. Critical for managing long-term engineering capability.
How Senior Engineers Are Adapting to AI Coding Tools or Resisting Them Adoption pattern differences, review bottleneck realities, team dynamics navigation, and consensus-building strategies. Essential for managing cultural tensions and process breakdowns.
AI-Generated Code Security Risks – Why Vulnerabilities Increase 2.74x and How to Prevent Them Comprehensive security risk assessment (2.74-10x vulnerability increases), compliance implications (SOC2/ISO/GDPR/HIPAA), quality gates implementation, and mitigation playbook. Required for organisations in regulated industries or with enterprise customers.
A Framework for Responsible AI-Assisted Development – When to Use AI and When to Avoid It Complete decision frameworks, measurement dashboards (DORA metrics + cycle time), quality gates implementation guides, code review redesign approaches, and training curricula. Your primary implementation resource translating understanding to operational execution.
Not necessarily. The distinction is about context-appropriate tool usage. Vibe coding works well for throwaway prototypes, learning experiments, and personal projects where long-term maintainability doesn’t matter. Production systems require AI-assisted engineering with code review, testing, and full understanding—what Kent Beck calls “augmented coding.” Our guide to what vibe coding is and how it differs from AI-assisted engineering maps this spectrum precisely. The risk is vibe coding practices migrating from prototypes to production without appropriate quality gates. Simon Willison is direct about it: “Vibe coding your way to a production codebase is clearly risky.”
Measure business outcomes using DORA metrics: deployment frequency, lead time for changes, change failure rate, and mean time to restore service. If AI tools are helping, you’ll see faster deployment frequency with stable or improving change failure rates. If you see increased PR volume but longer cycle times, growing technical debt indicators, or more security vulnerabilities, the productivity gains are illusory. Our productivity paradox article details what to measure and how.
Neither extreme works well. Mandates create resistance and don’t teach strategic usage. Bans prevent learning and alienate developers who’ve already adopted tools independently. Instead, establish clear policies: when to use AI (cognitive toil, boilerplate, greenfield), when to avoid it (complex architecture, security-sensitive code, poorly documented legacy), and what quality gates apply. Our framework article provides policy templates and consensus-building approaches.
The data is concerning for production systems. Research shows 2.74-10x increases in security vulnerabilities, with 322% more privilege escalation paths and 10x more security findings overall. AI tools excel at syntax but struggle with architectural security decisions—authentication, authorisation, secrets management. For regulated industries or enterprise customers requiring SOC2/ISO certification, AI-generated code without enhanced security review creates compliance risks. Our security risks article details specific vulnerability patterns and mitigation strategies.
Only if training evolves appropriately. Anthropic’s research found that junior developers using AI assistance scored significantly lower on comprehension assessments—but the interaction pattern mattered enormously. Developers who used AI for generation and then asked for explanations scored nearly as well as hand-coders. AI doesn’t inherently prevent learning; it shifts what must be taught. Train juniors to validate AI suggestions, understand generated code, and maintain architectural judgment. Our junior developer skill development guide provides training frameworks.
Senior resistance reflects legitimate expertise, not technological conservatism. Experienced engineers have maintained codebases where shortcuts became nightmares, debugged subtle bugs in code they didn’t write, and accumulated technical debt that slowed delivery for years. They recognise patterns in AI-generated code that create long-term problems juniors haven’t experienced yet. Additionally, seniors bear the review burden of AI-accelerated development. Our senior engineers article explores adoption patterns and consensus-building strategies.
Yes, through what Kent Beck calls “augmented coding”—using AI as a skilled assistant while maintaining code quality standards, testing discipline, and full understanding. The key is treating AI like a prolific but junior developer whose output requires review, not an oracle producing production-ready code. Beck used AI extensively on a complex project while maintaining strict TDD discipline, careful review of intermediate results, and readiness to intervene. Our case against article and framework article explore this approach in depth.
Vendor research (Microsoft, GitHub, DX) typically shows 20-26% productivity gains using observational studies measuring task completion and developer satisfaction. Independent research (METR, GitClear, Apiiro) uses randomised controlled trials or longitudinal data measuring business outcomes and code quality, often showing slowdowns or no improvement despite increased activity. The gap reflects methodology: controlled experiments vs observations, objective metrics vs self-reporting, business outcomes vs activity metrics. Both are useful. For decision-making, weight independent research more heavily. Our productivity paradox article provides the complete methodology comparison.
Code review capacity becomes the bottleneck when AI accelerates code generation. The approaches that work include automated quality gates that reduce the manual review surface, AI code review checklists highlighting common AI-generated issues, junior upskilling to handle routine reviews, strategic batching of related changes, and acceptance criteria defined before code generation rather than after. The goal isn’t reviewing everything AI produces with equal rigour—it’s risk-appropriate review calibrated to code complexity and how sensitive the area of the codebase is. Our framework article provides complete review redesign approaches.
Three key messages. First, developers are adopting AI tools enthusiastically and report feeling faster. Second, independent research shows organisations aren’t delivering faster—individual velocity gains don’t translate to business outcomes due to review bottlenecks, technical debt, and code quality issues. Third, selective adoption with quality gates can capture gains while mitigating risks, but requires investment in measurement, training, and process redesign. Avoid binary “AI will 10x productivity” or “AI destroys quality” narratives—a nuanced, evidence-based position is both more accurate and more credible. Our productivity paradox and framework articles provide board communication frameworks.
The vibe coding debate is fundamentally about whether speed and understanding can coexist. The evidence says they can—but only with deliberate effort.
AI coding tools aren’t going away. Adoption is at 91% and climbing. The question for your organisation isn’t whether to use them but how to use them without sacrificing the code quality, security, and engineering capability that your business depends on.
Start with honest measurement. Baseline your DORA metrics before making adoption decisions. Define clear policies that distinguish between appropriate and inappropriate AI usage contexts. Implement quality gates that catch AI-specific problems before they reach production. Redesign your review processes to handle increased volume. And invest in your junior engineers’ development—they’re your senior engineering pipeline, and the skills they build now determine your organisation’s capability in five years. If you operate in regulated industries, understanding the compliance implications of AI-generated code should be an early priority.
The path forward isn’t vibe coding and it isn’t banning AI. It’s the harder middle ground: context-specific adoption with the discipline to use these tools where they help and avoid them where they don’t.
If you’re ready to start building that capability, our framework for responsible AI-assisted development is the place to begin. If you’re still assessing the evidence, start with the productivity paradox. If the technical debt and craftsmanship concerns resonate with you, the case against vibe coding provides the quantified evidence you need. And if you’re navigating team dynamics around adoption, the senior engineers article addresses that directly.
AI coding tools promise faster delivery. A lot of teams are adopting them and getting the opposite – more technical debt, more security vulnerabilities, and review bottlenecks instead.
The productivity paradox is real. Individual developers feel faster. At the same time, team-level delivery stability is falling apart. Research from over 10,000 developers confirms it – teams with high AI adoption complete 21% more tasks and merge 98% more pull requests. But they also see a 9% increase in bugs and zero improvement in overall delivery metrics.
So the solution isn’t to ban AI tools. It’s not to embrace them uncritically either. You need structure. You need a framework that defines when AI genuinely helps, when it harms, and how to measure the difference. This article gives you an actionable implementation playbook – a decision framework for task-level AI suitability, DORA metrics instrumentation, 10 specific quality gates, code review redesign strategies, and a 90-day rollout plan. This guide is part of our comprehensive vibe coding complete overview for engineering leaders, where we explore every dimension of the vibe coding phenomenon from definitions to organisational impact.
The framework starts with a simple question – which tasks should use AI, and which shouldn’t?
Strategic selective adoption means evaluating each task against four things – complexity, context, risk, and pattern familiarity.
Simple, well-defined, low-risk tasks with common patterns – those are strong candidates. Think boilerplate CRUD operations, REST API scaffolding, unit test generation, repetitive utility scripts. These are where AI tools work best. The strategic selective adoption case explores the full evidence for when AI genuinely helps, from democratisation benefits to vendor research findings.
Complex architecture decisions, security-critical code, poorly documented legacy systems – keep those human-led. Authentication, authorisation, cryptography, payment processing – these need experienced developers who understand the implications of every design choice. The technical debt and security concerns involved when this boundary is crossed are substantial – code quality degrades measurably and security vulnerabilities increase.
The decision flowchart looks like this. Is the task security-critical? If yes, manual implementation. Is the codebase complex or legacy? If yes, manual with AI reference only – it can’t intuit unwritten rules. Is the pattern well-defined? If no, manual with AI assistance. Is it boilerplate or repetitive? If yes, AI with review. Is it prototyping or throwaway? If yes, AI generation acceptable.
Concrete examples help. Generating data validation schemas – suitable. The patterns are well-known, the risk is low, and tests can verify correctness. Implementing OAuth2 flows – avoid. Security implications are high, the attack surface is large, and subtle mistakes create vulnerabilities. Scaffolding REST endpoints – suitable. The structure is repetitive, frameworks provide guard rails, and automated testing catches most issues. Designing distributed caching strategy – avoid. The system interactions are complex, performance characteristics depend on specific infrastructure, and the AI lacks the context to make informed trade-offs.
The distinction between suitable and unsuitable comes down to risk tolerance and context depth. In greenfield projects with full test coverage, you can hit that 80% threshold. In mature codebases with complex invariants, the calculus inverts.
Prototyping and stakeholder demos are special cases. When the code is throwaway, vibe coding is acceptable. Speed and exploration matter more than correctness and maintainability. Build a quick proof-of-concept to validate an idea. Show stakeholders what the feature might look like. Just don’t let throwaway code become production code without proper review and refactoring.
Once you know which tasks suit AI, you need to measure whether it’s actually helping.
DORA metrics are four research-backed performance indicators from Google’s DevOps Research and Assessment programme that correlate with business outcomes.
Deployment Frequency – how often code ships to production. High performance is daily or more.
Lead Time for Changes – time from commit to running in production. High performance is less than 24 hours.
Mean Time to Recover – time to restore service after incident. High performance is less than 4 hours.
Change Failure Rate – percentage of deployments causing failures. High performance is less than 15%.
Add Cycle Time as a fifth metric. It measures task start to working production code. This reveals true end-to-end productivity.
These metrics expose the productivity paradox mechanisms that undermine AI adoption. Teams report 98% more PRs and 154% larger PRs after adopting AI tools. But change failure rates rise. Lead times increase. Review becomes a bottleneck.
Lines of code and PR count are vanity metrics. They reward output volume rather than delivery outcomes.
SMB benchmarks for 50-500 employee companies look different from enterprise targets. Deployment frequency of 1-5 per day – that’s high performance for teams without dedicated DevOps engineers. Lead time under 24 hours – achievable with streamlined processes and automated CI/CD. MTTR under 4 hours – assumes reasonable on-call rotation and incident response procedures. Change failure rate under 20% – realistic quality bar when you’re moving quickly. Cycle time under 5 days – accounts for the full software development lifecycle from planning to production.
These aren’t aspirational targets. They’re practical benchmarks that teams of your size actually achieve when they optimise for delivery outcomes rather than activity metrics.
Why these metrics correlate with business outcomes – they measure actual delivery capability, not busyness. A team that deploys daily can respond to customer feedback quickly. A team with low change failure rates wastes less time on firefighting and rework. A team with short MTTR recovers from incidents before customers notice. A team with fast cycle time delivers features when they’re still relevant.
Elite performers who excel in these metrics are twice as likely to meet organisational performance targets. The metrics aren’t just engineering curiosities. They predict revenue growth, market share, and customer satisfaction.
Measuring the right things requires connecting data from across your development pipeline.
Cycle time measures the elapsed duration from when you start a task to when the resulting code is running in production.
Instrumentation requires connecting three data sources. Task tracking systems – Jira, Linear, GitHub Issues – for task start timestamps. CI/CD pipeline telemetry – GitHub Actions, Jenkins, GitLab CI – for build and deployment tracking. Production monitoring – feature flag activation, deployment verification – for completion timestamps.
The Faros AI approach demonstrates instrumentation at scale. They correlate data across 10,000+ developers by integrating metrics from version control, CI/CD, and project management.
Telemetry-based measurement beats self-reported time tracking. It captures actual workflow bottlenecks – review wait time, CI queue delays, deployment failures – without developer overhead.
Compare cycle time for AI-assisted tasks versus manually-implemented tasks. That’s how you quantify whether AI is genuinely accelerating delivery or merely shifting effort from coding to review and debugging.
Track trends over time. A rising cycle time after AI adoption signals process problems. This happens even if individual developers report feeling faster.
Tooling options – Jira + GitHub Actions + Datadog as a minimum viable stack. DX Platform and Swarmia offer dedicated engineering intelligence platforms.
Ten automated quality gates catch common failures before they reach production.
Gate 1: Automated secrets scanning prevents leaked credentials. AI models train on public repositories. Many of those contain accidentally committed API keys and passwords. The model learns this anti-pattern and reproduces it. Tools – git-secrets (free), GitHub secret scanning (free), GitGuardian (paid). Triggers – pre-commit hook and PR creation. Action – block commit, alert security team. Setup takes an hour.
Gate 2: Static application security testing catches security vulnerabilities in source code before deployment. AI doesn’t understand security context. It pattern-matches on code it’s seen, which often includes insecure implementations. SQL injection, cross-site scripting, path traversal – SAST tools find these automatically. Tools – CodeQL (free for open source), SonarQube (community edition free), Semgrep (open source). Triggers – every PR. Action – block merge on high/critical findings, require security review for medium findings. For a deep-dive into the compliance risk mitigation context behind these controls – including why AI-generated code sees 2.74x more security vulnerabilities – see our dedicated security risk assessment.
Gate 3: Dependency vulnerability checks detect known vulnerabilities in third-party packages. AI loves pulling in dependencies. It’ll import entire libraries to use one function. It doesn’t check if those libraries have known security issues. Tools – Dependabot (free on GitHub), Snyk (free tier), npm audit (free). Triggers – PR creation and weekly scheduled scan. Action – block merge on critical vulnerabilities, create tickets for high/medium findings.
Gate 4: Automated linting and formatting ensures code style consistency. AI-generated code often violates project style conventions. The model learned from diverse codebases, so it doesn’t match yours. Tools – ESLint, Prettier, Black, Ruff. Triggers – pre-commit hook and CI. Action – auto-fix where possible (formatting), block on remaining violations (linting rules).
Gate 5: Test coverage requirements enforce minimum quality standards. Set the bar at 80% coverage for new code. AI generates code fast but often skips edge cases in tests. Tools – Jest, pytest-cov, JaCoCo, Istanbul. Triggers – every PR. Action – block merge if coverage drops below threshold.
Gate 6: Manual security review triggers ensure human eyes on security-critical code. Automated tools catch common vulnerabilities. Humans catch business logic flaws and architectural issues. Triggers – file path patterns matching auth/security directories (anything under auth/, security/, crypto/). Action – automatically request security-focused reviewer, require explicit approval before merge.
Gate 7: Naming convention enforcement catches AI-generated names that violate project standards. AI uses generic names like data, result, temp, handler that make code harder to maintain. Tools – custom ESLint rules, Checkstyle, CI checks. Triggers – every PR. Action – block merge on violations.
Gate 8: Cognitive complexity limits prevent AI from generating overly complex functions. AI loves nested conditions and long functions. SonarQube’s cognitive complexity metric measures how hard code is to understand. Tools – SonarQube, CodeClimate. Triggers – every PR. Action – flag functions exceeding threshold (typically 15).
Gate 9: Code duplication detection identifies copy-paste patterns common in AI output. AI reuses patterns across the codebase instead of extracting shared utilities. Tools – PMD CPD, SonarQube, jscpd. Triggers – every PR. Action – warn on duplication above 3%, block above 5%.
Gate 10: Acceptance criteria validation is the most important gate. It ensures the right thing was built. Acceptance criteria get documented in the ticket before coding starts. Reviewer validates implementation against criteria during review. Tools – PR templates with checklists. Action – block merge until criteria confirmed. This prevents the 70% Problem where AI builds something that’s “almost right” but misses the actual requirements.
Implementation priority – start with gates 1, 2, and 4. Secrets scanning, SAST, and linting. These give highest impact with lowest effort. You can implement all three in a day. Then add gate 5 (test coverage) and gate 3 (dependency scanning) in week two. Save gates 6-10 for when your team is comfortable with the first five.
AI coding tools create a review capacity crisis. Teams report reviews taking 91% longer, overwhelming senior engineers. The team dynamics navigation challenge behind this – senior skepticism, consensus-building across divided teams – deserves dedicated attention alongside process redesign.
Solution 1: Pair review for AI-generated code – assign two reviewers with divided focus. One checks functional correctness and business logic. The second checks for AI-specific issues – hallucinated dependencies, inconsistent error handling, security blind spots.
Solution 2: AI code review checklists give reviewers specific things to look for. Are all imported dependencies actually used and necessary? AI often imports entire libraries for one function. Does error handling follow project conventions? Are there hardcoded values that should be configuration? Does the code handle edge cases the AI may have overlooked? Is the approach consistent with existing architecture patterns?
Solution 3: Automated gates reduce manual burden so reviewers focus on the stuff that requires human judgment. Quality gates catch mechanical issues – leaked secrets, security vulnerabilities, style violations, missing tests – before human reviewers see the code. Manual reviewers focus on business logic, architecture, and design decisions.
Solution 4: Junior developer upskilling expands review capacity without hiring. Train mid-level developers to handle reviews of straightforward AI-generated code. Boilerplate, CRUD operations, utilities – this doesn’t need senior attention if the code passed all quality gates. Create a review training programme. Level 1 reviews simple AI code with senior oversight. Level 2 reviews moderate AI code independently. Level 3 reviews complex AI code and mentors Level 1 reviewers.
Solution 5: Batching strategies reduce context-switching overhead. Group similar AI-generated PRs for batch review. Review all REST endpoint scaffolding together. Review all data model updates together. Reviewer sees the same type of code five times in a row and gets faster at spotting issues. Schedule dedicated review blocks for batches rather than ad-hoc reviews throughout the day.
Solution 6: Acceptance criteria upfront is the highest-leverage intervention. It prevents problems rather than catching them. Define what “done” looks like before AI generation. Write acceptance criteria in the ticket – functional requirements (what it does), non-functional requirements (performance, security), test coverage expectations, and definition of done. AI generates code to meet the criteria. Review validates against the criteria.
Working in small batches is a complementary strategy. Constraining AI to smaller scopes – one function, one endpoint, one feature at a time – reduces per-PR review burden. Set a team working agreement – no more than 400 lines changed per PR.
Test-Driven Development with AI follows a three-step cycle.
Write a failing test first. You define expected behaviour. The test is your specification.
Let AI generate code to pass the test. AI is constrained by the specification.
Review and refactor. You validate the approach and improve the design.
TDD works as a quality control mechanism because tests act as a formal specification. Kent Beck‘s augmented coding framework describes this – human expertise defines what correct looks like through tests, AI handles the mechanical work of generating implementations, and the human reviews with full understanding of what “correct” means.
Here’s an example. For an authentication feature, write tests specifying bcrypt for password hashing, 30-minute session timeout, rate limiting after 5 failed attempts with 15-minute lockout, and email verification for password reset. AI generates an implementation that must pass all these tests. You review for correctness and architectural fit.
TDD inverts the usual AI risk. Instead of reviewing AI output hoping to catch everything it got wrong, you define correctness first and verify the AI met the specification.
This addresses the 70% Problem – when AI gets code “almost right” but the last 30% of completion and debugging consumes disproportionate effort. With TDD, incomplete or incorrect code is immediately surfaced by failing tests.
The DORA 2025 Report identifies seven organisational capabilities that determine whether AI coding tools deliver lasting benefits or create problems at scale.
Capability 1: Clear AI stance – explicit policy on acceptable AI usage, prohibited tasks, and quality expectations.
Capability 2: Healthy data ecosystems – clean, well-structured data practices. If your documentation is outdated and your code is messy, AI will generate more of the same.
Capability 3: AI-accessible internal data – internal documentation, architecture decision records, and coding standards accessible to AI tools so they generate contextually appropriate code.
Capability 4: Strong version control practices – rigorous tracking of what code was AI-generated versus human-written. This enables retrospective quality analysis.
Capability 5: Working in small batches – the discipline to constrain AI output to small increments rather than large code blocks. AI-generated PRs averaging 154% larger directly undermines this.
Capability 6: User-centric focus – measuring outcomes (user satisfaction, business impact) rather than activity (lines of code, PRs merged).
Capability 7: Quality internal platforms – robust CI/CD, testing infrastructure, and developer tooling that can absorb increased code volume without becoming bottlenecks.
Self-assessment – for each capability, rate your organisation on a 1-5 scale. Scores below 3 represent risks that should be addressed before scaling AI adoption.
Build these capabilities before scaling AI adoption. Research shows that AI amplifies an organisation’s existing strengths and weaknesses.
An AI coding policy translates the decision framework, quality gates, and process redesigns into a formal document. This ensures consistent adoption.
Section 1 – Acceptable usage contexts – boilerplate code (CRUD operations, REST APIs), repetitive patterns, prototyping and demos, unit test generation, code documentation, simple scripts and utilities.
Section 2 – Prohibited tasks – authentication and authorisation logic, cryptographic implementations, payment processing, security-critical code, complex architectural decisions, poorly documented legacy systems.
Section 3 – Quality standards – Kent Beck augmented coding approach (TDD, code review, test coverage above 80%), security review required for auth/authz/crypto, acceptance criteria defined before generation, DORA metrics tracked.
Section 4 – Review requirements – enhanced scrutiny for all AI-generated code, pair review for authentication/authorisation/security-critical code, automated gates mandatory pre-review, manual security review triggered by file path patterns.
Section 5 – Training expectations – all developers complete three-level curriculum covering awareness (AI limitations, 70% Problem), strategic selection (decision framework, task suitability), and quality validation (debugging, testing, reviewing AI code).
Section 6 – Measurement – monthly reporting on DORA metrics, cycle time, code quality indicators (defect rates, cognitive complexity, duplication), and review metrics (time, volume, bottlenecks).
Template policy:
AI Coding Tools Policy: [Company Name]
Purpose: Enable strategic use of AI coding tools while maintaining code quality, security, and maintainability.
Acceptable Usage Contexts: Boilerplate code, repetitive patterns, prototyping and demos, unit test generation, code documentation, simple scripts.
Prohibited Tasks: Authentication and authorisation logic, cryptographic implementations, payment processing, security-critical code, complex architectural decisions, poorly documented legacy systems.
Quality Standards: Kent Beck Augmented Coding (TDD, code review, test coverage above 80%), security review required for auth/authz/crypto, acceptance criteria defined before generation, DORA metrics tracked.
Review Requirements: Enhanced scrutiny for all AI-generated code, pair review for security-critical code, automated gates (SAST, secrets scanning, linting) mandatory, manual triggers for security-focused engineers.
Training Expectations: Level 1 (Awareness) – AI limitations, 70% Problem. Level 2 (Strategic Selection) – decision framework, task suitability. Level 3 (Quality Validation) – debugging, testing, reviewing AI code.
Measurement: Track monthly – DORA metrics (deployment frequency, lead time, MTTR, change failure rate), cycle time (task start to production), code quality (defect rates, cognitive complexity, duplication), review metrics (time, volume, bottlenecks).
Policy Owner: [CTO Name], effective [Date], review quarterly.
A phased 90-day plan translates the framework into a week-by-week execution roadmap.
Weeks 1-2 (Baseline) – Audit current AI usage via developer survey. How many people are using AI tools? Which tools? For what tasks? What problems are they experiencing? Measure baseline DORA metrics – deployment frequency, lead time, MTTR, change failure rate. Pull the last 90 days of data from your CI/CD pipeline and incident tracking system. Identify quality gate gaps. Document current code review process and capacity constraints.
Weeks 3-4 (Quality Gates) – Implement secrets scanning with git-secrets and GitHub secret scanning. Set up SAST using CodeQL (if GitHub) or SonarQube community edition (if self-hosted). Configure automated linting and formatting CI checks with ESLint, Prettier, or Black. Enable dependency vulnerability scanning with Dependabot (GitHub) or Snyk free tier. These four gates are your foundation – they catch the most expensive failures with minimal effort.
Weeks 5-6 (Training) – Deliver Level 1 workshop on AI limitations and the 70% Problem. Two-hour session covering how AI generates code (pattern matching, not understanding), common failure modes, and why “almost right” code is expensive. Deliver Level 2 workshop on the decision framework and task suitability assessment. Two-hour session with the decision flowchart, concrete examples, and hands-on practice categorising real tasks from your backlog. Assign hands-on exercises using real codebase examples.
Weeks 7-8 (Pilot) – Recruit volunteer adopters – mix of enthusiastic and sceptical engineers. You want 4-6 people representing different experience levels. Define pilot scope with specific projects. Choose greenfield features or well-isolated refactoring work. Avoid security-critical or legacy systems for the pilot. Implement acceptance criteria process for AI-assisted tasks. Track pilot metrics – cycle time, defect rate, review time.
Weeks 9-10 (Measure) – Collect pilot data across all DORA metrics. Compare pilot group versus control group outcomes. Did deployment frequency improve? Did change failure rate increase? Document specific successes and failures with examples. Feature X went smoothly – AI generated boilerplate, tests caught issues, review was fast. Feature Y was a disaster – AI made wrong assumptions, rework took longer than manual implementation.
Weeks 11-12 (Adjust) – Refine policy based on pilot learnings. Update the acceptable/prohibited task lists. Expand to full team via phased rollout. Add one squad per week until everyone’s onboarded. Update quality gates based on observed failure patterns. If AI keeps generating a specific type of bug, add a gate to catch it. Communicate results and rationale for adjustments to the team.
Week 13 (Retrospective) – Full team retrospective on the adoption process. What went well? What was frustrating? What should we change? Measure final DORA metrics versus week 1-2 baseline. Calculate the delta. Present findings to leadership. Plan ongoing iteration cadence. Schedule quarterly policy review. Calendar it now so it doesn’t slip.
The key is the pilot-then-expand approach to manage risk and generate internal evidence. Your team needs to see it work in your codebase. For the complete strategic synthesis that ties together definitions, productivity evidence, security risks, and team dynamics into a unified perspective, see our comprehensive vibe coding strategic synthesis for engineering leaders.
Here are answers to common questions about implementing this framework.
The 70% Problem describes where AI-generated code appears nearly complete but requires disproportionate effort to finish. The nature of the problem evolved from syntax bugs to conceptual failures. Modern AI makes architectural mistakes and wrong assumptions about requirements that are harder to detect and more expensive to fix. TDD and acceptance criteria mitigate this by defining correctness upfront.
Junior developers can use AI tools safely for tasks identified as suitable – boilerplate, repetitive patterns, test generation – provided quality gates are in place and code undergoes standard review. However, they must complete at least Level 1 and Level 2 training to understand AI limitations. Security-critical or architecturally complex tasks should remain with senior engineers regardless of AI assistance.
Most AI coding tools integrate with version control to tag AI-assisted commits or PRs. Teams can use PR templates requiring developers to indicate AI assistance level – fully generated, AI-assisted, human-written. Some engineering metrics platforms like DX Platform and Swarmia can correlate AI tool usage data with repository activity for automated tracking.
Vibe coding is uncritical acceptance of AI-generated code without understanding its logic, architecture, or implications. AI-assisted engineering applies AI tools strategically within a framework of quality gates, measurement, acceptance criteria, and human review. The distinction is governance – AI-assisted engineering has explicit boundaries, measurement, and quality enforcement. Vibe coding has none.
Most teams see meaningful data within 4-6 weeks of instrumentation. Baseline measurements in weeks 1-2 provide the starting point, and trends become visible by weeks 9-10 of the 90-day plan. Significant improvements typically emerge over 2-3 quarters as teams internalise the practices.
Quality gates add time to the merge process – typically 5-15 minutes for automated checks. But they save significantly more time by catching issues before they reach production. Teams that implement quality gates consistently report lower change failure rates and shorter MTTR. The net effect is faster delivery, not slower.
Frame the framework as enabling better AI usage rather than restricting it. Developers who understand the decision framework, quality gates, and measurement approach typically appreciate that the structure helps them avoid the frustrating 70% Problem. Include resistant developers in the pilot group so they experience the benefits firsthand.
For teams under 20 developers, start with three gates – automated secrets scanning (git-secrets, free), SAST via CodeQL or SonarQube community edition (free), and automated linting/formatting (ESLint, Prettier, Black, free). These three catch the most critical failures – leaked credentials, security vulnerabilities, and style inconsistencies – with minimal setup effort. You can implement all three in a day.
Compare DORA metrics (deployment frequency, lead time, MTTR, change failure rate) and cycle time before and after structured AI adoption. Avoid measuring ROI by lines of code or number of PRs, which are vanity metrics. Track defect rates, review time per PR, and developer satisfaction alongside DORA metrics for a comprehensive view.
The framework recommends prohibiting AI for security-critical code generation – authentication, authorisation, cryptography, payment processing. But it allows AI for security-adjacent tasks like generating test cases for security features or drafting documentation. All security-related code should trigger manual security review regardless of how it was written.
AI tools naturally generate larger code blocks. AI-generated PRs average 154% more lines changed. This violates the DORA principle that small batch sizes correlate with high performance. The solution is to constrain AI to small scopes. Generate one function or one endpoint at a time, review and merge, then generate the next. Set a team working agreement of no more than 400 lines changed per PR.
Stop tracking or de-emphasise lines of code, number of commits, number of PRs merged, and time spent coding as productivity indicators. These are vanity metrics that reward output volume. They will inflate dramatically with AI usage without reflecting actual delivery quality. Replace them with DORA metrics (deployment frequency, lead time, MTTR, change failure rate) and cycle time, which measure outcomes rather than activity.
This framework is part of a broader strategic synthesis of the vibe coding phenomenon that covers every dimension engineering leaders need to navigate – from what vibe coding is and why developers feel faster while delivering slower, to the security risks and workforce implications for your organisation.
How Senior Engineers Are Adapting to AI Coding Tools or Resisting ThemThere’s a weird paradox happening in engineering teams. Senior engineers are the group most sceptical about AI coding tools. But research shows that when seniors do adopt, they ship 2.5x more AI-generated code than juniors and deliver big returns through architectural prototyping and AI-assisted debugging.
Meanwhile, juniors who are excited about AI are producing 98% more pull requests, and sceptical seniors are drowning in review work that takes 91% longer. Teams are splitting along experience lines over code quality standards.
This article is part of our comprehensive guide to vibe coding and the death of craftsmanship, where we explore the cultural and technical implications of AI coding tools for engineering leaders.
This is a political and cultural problem with real organisational risk. For small to medium businesses that can’t afford to lose institutional knowledge, this tension needs your attention.
The good news is there’s a practical path: data-backed frameworks for building consensus, redesigning review processes, and finding middle ground without forcing adoption or banning tools.
Faros AI analysed telemetry from over 10,000 developers across 1,255 teams and found that less experienced engineers lean harder on AI tools. Senior engineers with deep system knowledge resist.
It’s not that they’re stubborn or set in their ways. It’s rational caution that comes from experience.
Seniors have lived through hype cycles before. They’ve seen short-term speed create long-term technical debt. They’re on the hook for system health that spans years, not sprints.
Four related barriers drive senior resistance. First, trust and reliability concerns from years of watching cascading errors play out. Second, the complexity gap where AI excels at common patterns but struggles with unique, context-heavy architectural problems. Third, professional identity and craftsmanship values under threat from delegating work they actually enjoy. Fourth, time pressure from juggling technical leadership with hands-on coding that leaves little room for tool experimentation.
Junior enthusiasm comes from different conditions. They work on well-defined tasks with clear requirements. AI helps fill knowledge gaps. They’ve had less exposure to the downstream consequences of subtle code errors. For more on junior developers in the age of AI and skill development concerns, see our dedicated analysis.
Senior developers work on fundamentally different problems. Designing distributed systems. Debugging performance bottlenecks. Making architectural trade-offs. Navigating decades of piled-up technical debt.
The institutional knowledge about why certain architectural decisions were made, where performance bottlenecks exist, and how different systems interact—this context is invisible to AI tools.
As seniority increases, engineers spend less time coding and more time on high-value tasks like collaborating with stakeholders, designing APIs for dozens of teams, optimising queries handling millions of requests, and making architectural decisions with years-long implications.
Trust erosion is real and measurable. Favourable views of AI tools declined from 70% to 60% despite 84% adoption. Only 3% of developers report “highly trust” in outputs, while 46% actively distrust AI tool accuracy.
66% of developers report frustration with “AI solutions that are almost right, but not quite.”
What matters here is that senior scepticism isn’t Luddite resistance. It’s legitimate stewardship and it needs to be validated rather than dismissed.
The review bottleneck is where this tension creates concrete organisational pain. Faros AI research shows teams with high AI adoption merge 98% more pull requests, but PR review time increases 91%, and average PR size grows 154%.
The burden falls hardest on senior engineers. They perform code review, they’ve got the architectural context to catch subtle errors, and they understand system-wide implications of changes.
AI-generated code is “almost right but not quite.” CodeRabbit’s analysis of 470 pull requests found AI-authored code contains 1.7x more issues overall.
Logic and correctness errors are 75% more common. Security issues are up to 2.74x higher. Error handling and exception-path gaps are nearly 2x more common. Readability issues spike more than 3x.
PRs per author increased 20% year-over-year thanks to AI, but incidents per pull request increased 23.5%.
Here’s the review paradox – the group sceptical of AI tools is now drowning in AI-generated output they didn’t create and didn’t ask for. They’re forced to spend more time verifying code from a tool they don’t trust.
AI lacks local business logic and infers code patterns statistically, not semantically. Without strict constraints, models miss the rules of your system that senior engineers have internalised.
AI generates surface-level correctness. Code that looks right but may skip control-flow protections or misuse dependency ordering.
Just under 30% of senior developers reported editing AI output enough to wipe out most of the time savings, compared to 17% of junior developers.
Amdahl’s Law applies here – AI accelerates code generation but review, testing, and release remain sequential bottlenecks. The system moves only as fast as its slowest link.
The team dynamics challenge stems from politics and culture. Engineering teams are dividing into camps along experience lines. Enthusiastic juniors in one corner. Cautious seniors in the other. Code quality debates becoming personal.
The adoption spectrum shows four groups. Enthusiastic Adopters around 25%, mostly juniors. Strategic Users around 35%, mixed experience. Cautious Sceptics around 30%, mostly seniors. Resisters around 10%, predominantly senior.
For SMBs, losing a senior engineer means losing institutional knowledge. Why architectural decisions were made. Where performance bottlenecks exist. How systems interact. Knowledge that no AI tool can replace and no junior can immediately backfill.
You’re caught in the middle. Mandate AI adoption and risk alienating seniors who may leave. Ban AI tools and risk losing enthusiastic juniors who see the organisation as backward. Do nothing and let the tension fester into team dysfunction.
Code quality debates become proxy wars for deeper tensions. Professional identity. Organisational direction. Who gets to define “good engineering.”
Senior developers are essential to your engineering organisation. Win over your senior developers, and scattered AI wins become compounding, org-level impact.
Senior concerns are backed by independent research, not personal bias. METR’s randomised controlled trial found experienced developers using AI took 19% longer despite believing they were 24% faster.
That’s a perception-reality gap.
The study involved 16 experienced developers working on 246 real open-source issues. Each issue was randomly assigned to allow or disallow AI use.
Code quality data supports senior instincts. GitClear found a 4x increase in code duplication and refactoring collapsed from 25% of changed lines in 2021 to under 10% in 2024.
CodeRabbit found 2.74x more security vulnerabilities in AI-generated code. Logic and correctness errors were 75% more common.
The 70% problem explains why perceived speed gains disappear. After AI generates initial code (the fast 30%), developers face the remaining 70% of work. Integration. Authentication. Security. Edge cases. Debugging. Understanding generated code.
Seniors have seen this pattern before. Short-term velocity gains creating long-term technical debt. AI accelerates this cycle by producing code that looks correct but embeds subtle mistakes at scale.
Kent Beck’s augmented coding framework resonates with seniors because it maintains the values they hold. Caring about code complexity, tests, coverage, and quality while still using AI assistance. In contrast, vibe coding prioritises system behaviour over code quality.
Augmented coding is Kent Beck’s framework for using AI tools while maintaining code quality. “In augmented coding you care about the code, its complexity, the tests, and their coverage. The value system is similar to hand coding—tidy code that works. It’s just that I don’t type much of that code.”
In vibe coding you don’t care about the code, just the behaviour of the system. If there’s an error, you feed it back into the genie in hopes of a good enough fix. For a detailed exploration of augmented coding vs vibe coding terminology, see our comprehensive definitions guide.
The framework provides common ground because it satisfies both camps. Juniors can use AI tools for code generation. Seniors see their quality standards maintained through TDD, code review, and test coverage requirements.
Tidy First methodology separates structural changes from behavioural changes. Never mix them in the same commit. Always make structural changes first. Validate with tests before and after. Creating disciplined guardrails around AI output.
Kent Beck’s B+ Tree project demonstrates the approach. Write the simplest failing test. Implement minimum code to pass. Refactor after tests pass. Always following the TDD cycle Red → Green → Refactor.
Warning signs that AI is going off track: Loops. Functionality that wasn’t requested. Any indication that the genie was cheating, for example by disabling or deleting tests.
This represents disciplined AI usage that avoids the extremes of blanket adoption or total bans. Seniors maintain authority over architectural decisions and quality standards while embracing AI for eliminating cognitive toil, writing tests, generating boilerplate, and handling routine patterns.
How augmented coding preserves professional identity – it keeps consequential decisions with humans while delegating routine work to AI.
Understanding the augmented coding framework gives you the foundation for building team consensus. Here’s how to implement it across your organisation.
Start by acknowledging concerns with data in Week 1. Present METR data showing 19% slower despite 20% faster belief. GitClear findings of 4x duplication and refactoring collapse. CodeRabbit results showing 1.7x more issues. Validate that senior scepticism is backed by independent research, not personal bias.
Week 2: Establish quality standards. Adopt Kent Beck’s augmented coding framework as the organisational standard. Define requirements for TDD, code review, and test coverage. Implement DORA metrics as objective measurement baseline.
Weeks 3-6: Pilot with volunteers. Start with enthusiastic adopters who volunteer. Measure cycle time, defect rate, and review time. Document what works versus what fails. Keep sceptics informed without pressuring participation.
Week 7: Measure outcomes objectively. Compare pilot group against control group using DORA metrics and cycle time. Present findings to the full team with transparency about both gains and problems discovered.
Week 8 and beyond: Adjust based on evidence. Expand successful patterns. Restrict problematic ones. Iterate with continuous improvement. Avoid both blanket adoption mandates and total bans.
Peer-to-peer learning proved 22% more effective than top-down mandates.
Visible leadership advocacy made developers 7x more likely to become daily users.
Start by identifying respected technical leads inside your company. Empower them to be early champions who openly share their own experiments and lessons with AI.
Give engineers time on the calendar, budget for tool exploration, and a clear message that experimentation is encouraged—even if not every attempt succeeds.
Encourage senior engineers to share their success stories in simple, bite-sized formats. Short demos. Quick team videos. Informal knowledge swaps.
DORA metrics serve as objective, non-partisan measurement framework for evaluating AI tool impact. Deployment frequency. Lead time for changes. Change failure rate. Mean time to recovery.
The problem is structural. 98% more PRs, 91% longer review times, 154% larger PRs, and the same senior capacity. Solutions must address process, not just headcount. For comprehensive review process redesign strategies, see our implementation framework.
Pair review for AI-generated code works like this – assign two reviewers for AI-generated code touching authentication, authorisation, or security-sensitive paths. One focuses on logic. One on security. Reduces individual burden while maintaining quality.
AI code review checklists should cover: project pattern adherence (not generic boilerplate), hard-coded secrets, authorisation checks, comprehensive error handling (not just happy path), edge case test coverage, and project naming conventions.
Automated quality gates include SAST scanning like CodeQL and SonarQube. Secrets detection using git-secrets and TruffleHog. Linting and formatting enforcement with ESLint, Prettier, Black. Test coverage requirements like 80% for new code.
The PR Contract framework requires every PR to include four elements. What and why in 1-2 sentences. Proof it works via tests or manual verification. Risk tier and AI role disclosure. Specific review focus areas for human input.
Break AI-generated work into small, reviewable pieces with clear commit messages. This counters AI’s tendency to generate massive changes that overwhelm reviewers.
AI review tools like CodeRabbit can serve as effective first-pass filters, catching common issues before human review. But they can’t replace senior engineers for architectural alignment, system-wide impact assessment, and context-dependent quality decisions.
The recommended approach is layered review. Automated tools catch formatting, security, and pattern violations. Seniors focus on architectural oversight and business logic validation.
To enthusiastic juniors: “I’m excited you’re exploring AI coding tools. They can eliminate cognitive toil and accelerate certain tasks. To maintain code quality and avoid the productivity paradox we’re seeing industry-wide, we’re adopting Kent Beck’s augmented coding framework: use AI, but with TDD, code review, and test coverage. This ensures we capture velocity gains without piling up technical debt.”
To sceptical seniors: “Your concerns about AI-generated code quality are backed by independent research. GitClear found 4x code duplication and refactoring collapsed by 60%. CodeRabbit found 1.7x more issues in AI code. We’re not doing blanket adoption. We’re implementing Kent Beck’s augmented coding framework with enhanced quality gates. You’ll have authority to require manual implementation for complex or security-sensitive code. Your expertise is indispensable for reviewing AI-generated code and maintaining our standards.”
To the board: “We’re taking a strategic selective adoption approach to AI coding tools. Industry research shows a productivity paradox: developers feel 24% faster but measure 19% slower due to review bottlenecks and code quality issues. We’re piloting AI usage with quality gates (automated testing, security review, enhanced code review) to capture genuine gains while avoiding technical debt accumulation that would slow delivery long-term. We’ll measure outcomes with DORA metrics and adjust based on evidence.”
For the broader strategic context on how AI is reshaping engineering culture and what this means across your organisation, see our vibe coding complete guide for engineering leaders.
Yes. Fastly’s survey of 791 developers found that about a third of senior developers say over half their shipped code is AI-generated—nearly 2.5x the rate reported by junior developers at 13%. Their architectural knowledge and system understanding mean they can direct AI tools more effectively, catching errors juniors miss and applying AI to higher-impact tasks. The challenge is getting them to adopt in the first place.
METR’s randomised controlled trial found a perception-reality gap – developers expected AI to speed them up by 24%, and even after experiencing a 19% slowdown, they still believed AI had sped them up by 20%. The likely explanation is that AI eliminates tedious typing and boilerplate work, which feels faster, but the time spent prompting, reviewing, correcting, and integrating AI output exceeds the time saved.
Banning AI tools risks losing enthusiastic team members and falling behind industry adoption. Instead, adopt a “strategic selective” approach using frameworks like Kent Beck’s augmented coding – allow AI usage with quality gates including TDD, code review requirements, and test coverage thresholds. This maintains quality standards while capturing genuine productivity gains.
Measure organisational outcomes rather than individual activity. Track DORA metrics (deployment frequency, lead time, change failure rate, mean time to recovery). Compare pilot groups against control groups. Look at cycle time and defect rates rather than lines of code or PR counts. Faros AI’s research shows individual gains often fail to translate to company-level improvements.
Vibe coding prioritises speed over correctness – developers accept AI-generated code based on whether it “feels right” without thorough verification. Augmented coding, Kent Beck’s framework, maintains traditional engineering values: caring about code complexity, tests, coverage, and quality while using AI to generate code. The distinction is whether you care about the code itself or only the behaviour of the system.
Faros AI’s analysis of 10,000+ developers found AI adoption increases PR review time by 91%, with PRs growing 154% larger and teams merging 98% more of them. CodeRabbit’s study found AI-authored PRs contain 1.7x more issues, meaning each review requires more thorough scrutiny. The review bottleneck is now the primary constraint on delivery velocity in AI-augmented teams.
AI review tools like CodeRabbit can serve as effective first-pass filters, catching common issues before human review. However, they can’t replace senior engineers for architectural alignment, system-wide impact assessment, and context-dependent quality decisions. The recommended approach is layered review – automated tools catch formatting, security, and pattern violations; seniors focus on architectural oversight and business logic validation.
Peer-to-peer learning is 22% more effective than top-down mandates. Identify respected senior engineers who are willing to experiment. Give them dedicated time and budget for exploration. Have them share results with colleagues. Faros AI’s case studies show that visible leadership advocacy makes developers 7x more likely to become daily users, but the advocacy must come from trusted technical voices, not management directives.
Track four categories: velocity (cycle time, deployment frequency), quality (defect rate, change failure rate, code review feedback), volume (PRs merged, PR size, review time), and team health (developer satisfaction, context switching, review burden distribution). Compare these metrics between the pilot group and a control group over at least 4-6 weeks to get meaningful signal.
AI excels for seniors at: documentation drafts, test data generation, boilerplate code, exploring unfamiliar frameworks, writing performance benchmarks, and routine bug fixes. Seniors should avoid AI for: core architectural decisions, performance-sensitive code sections, complex debugging of production systems, and security-sensitive authentication and authorisation logic. Kent Beck’s augmented coding framework provides specific guidance on this boundary.
For more strategic guidance on navigating the vibe coding phenomenon, explore our complete strategic framework for engineering leaders, which synthesizes team dynamics, consensus-building approaches, and actionable implementation strategies.
AI-Generated Code Security Risks – Why Vulnerabilities Increase 2.74x and How to Prevent ThemAI coding assistants are making development faster. But there’s a cost that most engineering leaders haven’t properly measured yet. Veracode’s 2025 GenAI Code Security Report tested more than 100 LLMs across 4 languages and found that AI-generated code contains 2.74x more vulnerabilities than human-written code. The failure rate on secure coding benchmarks? 45%. Apiiro’s research across Fortune 50 enterprises backs this up: 322% more privilege escalation paths, 153% more design flaws, and a 40% jump in secrets exposure.
If you’re chasing enterprise deals or working in regulated industries, these numbers aren’t just interesting—they’re deal-breakers. This article is part of our comprehensive guide to vibe coding and the death of craftsmanship, where we explore the full spectrum of AI coding tool impacts on software development. Here, we break down the actual security risks, look at real incidents, and give you 8 specific quality gates that stop AI-generated vulnerabilities from getting into production.
Veracode’s 2025 GenAI Code Security Report tested more than 100 LLMs across Java, JavaScript, Python, and C#. The result? AI-generated code has 2.74x more vulnerabilities than code written by humans. This wasn’t a small study—it’s a consistent pattern across multiple LLMs and languages.
The methodology here matters. Veracode compared AI output against human baselines in controlled conditions. The results held across all four languages tested.
Here’s what they found: 45% of AI-generated code samples brought in OWASP Top 10 vulnerabilities. Cross-Site Scripting (CWE-80) had an 86% failure rate. Java was the worst performer with a 72% security failure rate for AI-generated code.
Apiiro’s independent research looked at Fortune 50 enterprises and found that CVSS 7.0+ vulnerabilities showed up 2.5x more often in AI-generated code. By June 2025, AI-generated code was adding over 10,000 new security findings per month across the repositories they studied—that’s a 10× increase from December 2024.
So what does that 2.74x number mean in practical terms? For every 1,000 lines of code, you’re adding nearly three times as many security vulnerabilities compared to human-written code. In production systems, these compound. A 10,000-line feature that would normally introduce 10 vulnerabilities now introduces 27.
These numbers mean compliance exposure, erosion of customer trust, and incident response costs that multiply exponentially. CodeRabbit’s December 2025 analysis of 470 open-source GitHub pull requests found AI co-authored code had approximately 1.7 times more major issues compared to human-written code—which validates the broader pattern and connects to the broader quality degradation context we explore in depth elsewhere.
The research is straightforward: AI coding assistants are productivity tools, not security tools. Treat them as anything else and you create risk.
Privilege escalation is when attackers get unauthorised access to levels beyond their permissions. Apiiro found AI-generated code creates 322% more escalation paths than human-written code. That’s not a small bump—it’s a multiplication of your attack surface.
The root cause is simple. AI models pattern-match from training data that includes millions of public repositories with permissive access defaults. They reproduce those patterns without understanding trust boundaries. They don’t reason about who should have access to what.
Here’s a concrete example. An AI generates an admin route handler that checks authentication but skips authorisation validation. The code checks that you’re logged in. It doesn’t check that you should have admin access. Now any logged-in user can access admin functions.
While syntax errors in AI-written code dropped by 76% and logic bugs fell by 60%, privilege escalation paths jumped 322%. This is a trade-off, and it’s not one most organisations knowingly made.
Detection needs a layered approach. Semgrep and SonarQube with custom rules flag authorisation code for mandatory security review. Penetration testing checks that role-based boundaries actually hold under attack. Manual review examines all authentication and authorisation code paths. Automated tools catch roughly 70% of escalation paths. The remaining 30% need human architectural analysis.
There’s another problem. AI-generated changes concentrated into fewer but significantly larger pull requests, each touching multiple files and services. This dilutes reviewer attention and makes it more likely that subtle security issues slip through.
AI-generated projects show a 40% increase in secrets exposure—hardcoded API keys, passwords, tokens, and certificates embedded directly in source code. AI-assisted developers exposed Azure Service Principals and Storage Access Keys nearly twice as often compared to non-AI developers, which creates immediate production infrastructure vulnerabilities.
The mechanism is straightforward. AI training data includes millions of public repositories where developers committed credentials. The models learn to reproduce these patterns as “normal” code. Multiple studies have indicated that LLMs can reproduce email addresses, SSH keys, and API tokens that were in their training corpus.
When developers paste code into AI assistants like ChatGPT or Claude for debugging, secrets embedded in that code may get processed by external LLM providers. Google can index ChatGPT conversations, which means sensitive information discussed in what should be a private chat could become publicly searchable.
Here’s a real example. ChatGPT-5 responded to a Twilio API request with code suggesting hardcoded secrets. An inexperienced developer, or one under pressure, might just swap in their real credentials without routing the secret through a vault.
Prevention needs pre-commit hooks using git-secrets, Gitleaks, or TruffleHog that block commits containing API keys, passwords, or tokens before they hit the repository. GitHub secret scanning catches exposed credentials in public repositories. Secrets management platforms like Doppler get rid of hardcoded credentials entirely by injecting them at runtime.
Integrate these with your CI/CD pipeline. Pre-commit hooks look for embedded credentials in new code. Continuous monitoring scans code, logs, and shared collaboration tools for leaked credentials. When detection systems find exposed keys, they automatically revoke them and alert the team—speed matters here, because the faster you revoke, the smaller the damage window.
Apiiro documented a 153% increase in design-level security flaws in AI-generated code—architectural weaknesses you can’t fix with simple patches. These include authentication bypass patterns, insecure direct object references, missing input validation at trust boundaries, and improper session management.
Design flaws are different from implementation bugs. An implementation bug is a hardcoded secret or a missing input sanitisation check—you fix it with a line-level change. A design flaw is an authentication bypass or an entire authorisation model built on flawed assumptions. You fix it by restructuring flows across multiple services.
Design flaws are significantly more expensive to remediate than implementation bugs—typically 10-100x more expensive. A hardcoded secret takes minutes to fix with environment variable substitution. An authentication bypass pattern may need you to restructure entire authorisation flows across multiple components.
AI generates code that satisfies functional requirements but lacks system-level security reasoning. It doesn’t understand how individual components interact within a broader security architecture.
The numbers tell the story. AI-assisted developers produced 3-4× more commits than non-AI peers, yet generated 10× more security findings. Paradoxically, overall pull request volume dropped by nearly one-third, meaning larger PRs with more issues concentrated in each review.
Context rot makes this worse. As AI-assisted codebases grow, the model loses track of security decisions made in earlier components. It can’t maintain a mental model of the entire system’s security posture across thousands of lines of code. This creates inconsistencies—authentication handled one way in one service, a different way in another.
AI coding assistants create compliance challenges because code, credentials, and data leave the organisation’s environment when sent to LLM providers for processing. 73% of AI coding tool implementations are terminated by enterprise security reviews because vendors treat security as an afterthought.
Here’s the core problem: when you paste code into ChatGPT or send it to GitHub Copilot, that code gets transmitted to external services. If that code contains customer data, business logic, or credentials, you’ve just created a data handling control violation.
SOC2 data handling controls are violated when source code containing customer data or business logic is transmitted to external AI services without proper classification.
GDPR data residency requirements may be breached when code is processed in unknown geographic locations by LLM providers. If you’re handling EU customer data and your developers are using US-hosted AI assistants, you’ve potentially violated data processing agreements.
HIPAA PHI exposure risk shows up when healthcare organisations use AI assistants on codebases containing protected health information. Healthcare organisations processing electronic Protected Health Information require AI coding tools with documented HIPAA Technical Safeguards compliance. Business Associate Agreements remain required for any AI tool processing ePHI.
ISO 27001 information security management requires documented controls for all data processing. AI tool usage creates audit trail gaps where code decisions can’t be attributed to a specific human. When an auditor asks “who made this security decision,” the answer “an AI model” doesn’t satisfy the requirement.
Here’s a real case. A financial services organisation experienced a $2.3 million regulatory response after an API key committed to an AI training endpoint appeared in code suggestions for other developers months later. The key had leaked through the training process. The model memorised it. The model suggested it to another developer. That’s a compliance nightmare.
When you’re pursuing enterprise sales, customers demand compliance attestation. Failing to demonstrate controls around AI coding tool usage can block deals with regulated buyers. Enterprise procurement teams are asking: “Do you use AI coding assistants? What controls do you have in place? Can you prove code doesn’t leak through these tools?”
Mitigation approaches include self-hosted AI tools, air-gapped deployments like Tabnine Enterprise, vendor risk assessments, and data classification policies. The straightforward answer for most organisations: establish annual recertification processes, maintain AI governance documentation, and make sure Business Associate Agreements cover AI tool vendors.
Replit Agent deleted a production database and fabricated replacement data, which demonstrates the risk of AI agents executing destructive operations without validation safeguards. The agent deleted over 1,200 records of company executives despite explicit instructions not to make any changes. It then misled the user by stating the data was unrecoverable.
170 of 1,645 Swedish vibe-coded applications built with Lovable contained exploitable vulnerabilities including SQL injection and XSS—a 10.3% vulnerability rate in production apps. These were live applications serving real users. The vulnerabilities were there because basic SAST scanning wasn’t part of the deployment process.
A Stack Overflow hackathon experiment using Bolt introduced SQL injection vulnerabilities that would have been caught by basic SAST scanning. The code shipped without security review. The pattern is consistent: functional code that works but contains fundamental security flaws—exactly the kind of technical debt costs that accumulate when velocity takes priority over craftsmanship.
Gemini CLI had a remote code execution vulnerability discovered in its AI coding interface. Amazon Q’s VS Code extension contained a vulnerability. These incidents show that even AI tool infrastructure itself carries security risks.
Each of these incidents could have been prevented by specific security controls detailed in our framework for responsible AI-assisted development.
Eight specific security controls prevent AI-generated vulnerabilities from reaching production. Each addresses known vulnerability patterns identified in the research above.
Gate 1 — Automated secrets scanning: Prevents the 40% increase in secrets exposure by detecting hardcoded API keys, passwords, and tokens before they reach the repository. Tools like git-secrets, TruffleHog, and Gitleaks integrate at the pre-commit level, while GitHub secret scanning provides ongoing repository monitoring.
Gate 2 — Privilege escalation analysis: Addresses the 322% increase in privilege escalation paths by flagging all authorisation code for mandatory security review. Semgrep and SonarQube with custom rules detect authentication and authorisation patterns that require architectural validation beyond automated scanning.
Gate 3 — Dependency vulnerability checks: Catches vulnerable packages that AI models sometimes suggest from outdated training data. Dependabot, Snyk, and npm audit prevent merges when dependencies contain known CVEs, protecting against supply chain risks.
Gate 4 — Manual security review triggers: Makes sure all authentication, authorisation, cryptography, and payment processing code receives review from security-focused engineers who understand architectural security context, not just code correctness.
Gate 5 — Compliance audit trails: Satisfies SOC2 and ISO audit requirements by logging AI assistance usage in commit messages and maintaining records that attribute code decisions to specific humans, addressing the audit trail gaps created by AI-generated code.
Gate 6 — Automated testing requirements: Forces developers to write tests for AI-generated code, which often reveals security issues during test development. SonarQube’s “Sonar way for AI Code” quality gate requires no new issues, all new security hotspots reviewed, new code test coverage ≥80%, and duplication ≤3%.
Gate 7 — SAST/DAST integration: Catches injection attacks, authentication bypasses, and session management flaws that show up differently in static analysis versus runtime testing. CodeQL analyses source code patterns while OWASP ZAP simulates attacks against running applications.
Gate 8 — Security-specific acceptance criteria: Prevents architectural security gaps by defining security requirements before AI code generation begins, forcing developers to articulate security constraints rather than relying on AI to infer them.
Many of these tools are free. GitHub secret scanning is built-in for public repositories. Dependabot provides automated dependency vulnerability alerts. SonarQube Community Edition handles SAST analysis. Gitleaks handles pre-commit secrets detection. Semgrep open-source enables custom security rule scanning.
Integration approach matters. These gates layer into existing CI/CD pipelines rather than creating parallel processes. The automated gates run first—secrets scanning, SAST, dependency checks—filtering out known vulnerability patterns before human reviewers spend time. Manual review focuses on areas where automated tools are weakest: authentication logic, authorisation flows, business logic alignment, and architectural security decisions. For detailed implementation guidance, see our quality gates implementation framework.
For teams starting from zero, prioritise secrets scanning and SAST as highest-impact first steps. These catch the majority of common vulnerabilities with minimal configuration.
Treat all AI-generated code as untrusted input requiring verification—the same standard applied to third-party library code or contributions from external contractors. This is the mindset shift that changes how teams review AI output.
Automated tooling runs first: SAST scans, secrets scanning, and dependency checks filter out known vulnerability patterns before human reviewers spend time. This handles volume efficiently. Then manual review focuses on areas where automated tools are weakest: authentication logic, authorisation flows, business logic alignment, and architectural security decisions.
The review focuses on input validation at every boundary, authorisation at every access point, parameterised queries instead of string concatenation, error handling that doesn’t leak information, and secrets managed through environment variables or a secrets manager.
Automated gates handle 80% of detection while manual review concentrates senior developer attention on the highest-risk 20%. This is resource-efficient security. For comprehensive security review processes that integrate with your existing workflows, our framework provides specific implementation steps.
Integration with existing code review process is necessary—adding a “security considerations” section to PR templates rather than creating a separate review workflow. Separate workflows create friction. Developers skip steps. Embedding security into existing review makes it automatic.
Simon Willison stated: “If an LLM wrote every line of your code, but you’ve reviewed, tested, and understood it all, that’s not vibe coding in my book—that’s using an LLM as a typing assistant.” That’s the standard. Review, test, understand. Don’t blindly trust.
Establish clear boundaries: prohibit AI-generated code for authentication, authorisation, cryptography, and payment processing without mandatory security review. These are high-risk domains where architectural security reasoning matters most and where AI models are weakest.
Define permitted-with-review categories: CRUD operations, business logic, UI components, and utility functions can use AI assistance with standard enhanced review. These have lower risk profiles. The functional correctness AI provides is valuable. The security risks are more manageable with standard review processes.
Require security-specific acceptance criteria before any AI code generation: explicitly state security constraints. Example: “All database queries must use parameterised statements. Session tokens must expire after 30 minutes of inactivity. Authentication attempts must be rate-limited to 5 per minute per IP address.” These criteria become test cases that validate the AI output.
Document the policy in engineering handbooks and enforce through automated gates—policy without enforcement is wishful thinking.
Regularly review and update the policy as AI capabilities change and new vulnerability patterns emerge from research. The Veracode and Apiiro research we’ve discussed is from 2025. By the time you’re reading this, there may be newer findings. Policies need to adapt.
The policy structure becomes:
Prohibited tasks (requiring human implementation): Authentication systems, authorisation frameworks, cryptographic implementations, payment processing, secrets management systems.
Permitted with enhanced review: CRUD operations, business logic, UI components, API integrations, data transformations, utility functions.
Acceptance criteria requirements: Security specifications must be defined before generation. Test coverage requirements for generated code. Review requirements based on code type.
Enforcement mechanisms: Automated gates in CI/CD. Pre-commit hooks. SAST integration. Manual review triggers.
Xage Security believes that even the most advanced AI agents should operate within a Zero Trust architecture, where access is explicit—not implicit—and actions require explicit approval. That’s the philosophy. Never trust, always verify. For AI-generated code, verify means review, test, and validate against security criteria.
For a complete strategic overview of how security risks fit into the broader AI coding landscape, return to our comprehensive analysis of vibe coding and the death of craftsmanship, which synthesizes the evidence across productivity, quality, security, and organizational dimensions.
Cross-Site Scripting (CWE-80) with an 86% failure rate according to Veracode research. AI models frequently output user input without proper sanitisation, creating injection points that allow attackers to execute malicious scripts in other users’ browsers.
GitHub Copilot Enterprise offers SOC 2 Type II certification, code referencing filters, and organisation-level policy controls. However, the underlying LLMs still generate code with similar vulnerability patterns. Enterprise features improve compliance posture and audit trails but don’t eliminate the need for security quality gates on generated output.
Design flaws are typically 10-100x more expensive to remediate than implementation-level bugs. A hardcoded secret (implementation bug) takes minutes to fix with environment variable substitution, while an authentication bypass pattern (design flaw) may require restructuring entire authorisation flows across multiple services.
AI coding assistants can be used in HIPAA-regulated environments with proper controls: prohibit AI assistance on code handling Protected Health Information (PHI), use air-gapped deployment options like Tabnine Enterprise, implement audit logging of all AI tool usage, and make sure Business Associate Agreements (BAAs) cover AI tool vendors.
SAST (Static Application Security Testing) analyses source code without executing it, catching vulnerabilities like SQL injection patterns and hardcoded secrets during development. DAST (Dynamic Application Security Testing) tests running applications by simulating attacks, catching runtime issues like authentication bypasses and session management flaws that only show up during execution.
AI coding tools create SOC 2 challenges in three areas: data handling controls (source code sent to external LLM providers), change management documentation (AI-generated code decisions difficult to attribute to humans), and access controls (developers accessing AI tools may bypass established code review workflows). Organisations must document AI tool usage policies and maintain audit trails.
Several effective tools are free: GitHub secret scanning (built-in for public repositories), Dependabot (automated dependency vulnerability alerts), SonarQube Community Edition (SAST analysis), Gitleaks (pre-commit secrets detection), and Semgrep open-source (custom security rule scanning). These provide substantial coverage without requiring dedicated security team budgets.
Java’s higher failure rate likely reflects the language’s verbose security APIs, complex authentication frameworks (Spring Security, Jakarta EE), and the extensive configuration required for secure defaults. AI models generate functionally correct Java code but frequently misconfigure security frameworks, omit required annotations, or use deprecated security patterns from outdated training data.
Security-specific acceptance criteria define explicit security requirements before AI code generation begins. Example: “User registration must hash passwords with bcrypt (cost factor 12+), enforce minimum 12-character passwords, implement account lockout after 5 failed attempts, and log all authentication events.” Writing these criteria forces developers to articulate security constraints rather than relying on AI to infer them.
Detection requires a layered approach: Semgrep custom rules flag authorisation patterns for review, SonarQube identifies missing access control checks, penetration testing validates that role-based boundaries hold under attack, and mandatory manual review examines all authentication and authorisation code paths. Automated tools catch approximately 70% of escalation paths; the remaining 30% require human architectural analysis.
Secure prompt engineering reduces but doesn’t eliminate vulnerabilities. Specifying “use parameterised queries” or “validate all inputs” in prompts improves output quality, but LLMs still produce vulnerable patterns, especially in complex security logic. Prompt engineering is one layer in a defence-in-depth strategy, not a replacement for SAST, code review, and quality gates.
First, implement automated secrets scanning with pre-commit hooks (Gitleaks or git-secrets)—this addresses the highest-risk, easiest-to-detect vulnerability class. Second, enable SAST scanning (CodeQL or SonarQube) in CI/CD pipelines to catch injection and authentication flaws before merge. Third, establish a mandatory security review trigger for all authentication, authorisation, and data-handling code regardless of whether it was AI-generated or human-written.
Junior Developers in the Age of AI – Who Trains the Next Generation of EngineersJunior developers love AI tools. 78% of them trust AI specificity compared to just 39% for seniors. But here’s the problem—Anthropic research shows a 17-point comprehension gap when juniors learn with AI assistance. That’s 50% code understanding versus 67%, with statistical significance at Cohen’s d=0.738.
The traditional learning pathway gets short-circuited. Debugging builds deep knowledge. Reading documentation teaches problem-solving. Understanding error messages develops system thinking. AI just hands you answers without the struggle that creates expertise.
So here’s the succession planning problem. Who becomes your senior engineers in 5 years if today’s juniors never develop foundational debugging skills and architectural judgement?
For companies with 50 to 500 employees, the risk is real. Smaller teams can’t afford a skill gap generation.
This article is part of our vibe coding comprehensive guide, exploring the broader implications of AI-assisted development for engineering teams. Here we give you an evidence-based training framework to maintain your senior engineer pipeline while still leveraging AI’s efficiency gains.
They gain speed—onboarding compressed from 24 months to 9 months per Kent Beck—but they show comprehension gaps. Particularly in debugging tasks that previously built deep system knowledge.
The traditional pathway worked like this: debug your own code, understand error messages, read documentation, build mental models, develop troubleshooting instincts, gain architectural judgement. It took time. It involved struggle.
The AI-assisted pathway is different. Accept AI-generated code, struggle to debug code you didn’t write, lack context for why solutions work, miss foundational knowledge.
The Anthropic study measured comprehension with coding quizzes. The largest gap appeared in debugging questions. Low-scoring patterns—averaging less than 40%—included AI delegation and progressive reliance. High-scoring patterns—65% or better—included generation-then-comprehension. How you use AI influences what you retain.
Kent Beck’s “Valley of Regret” shows the problem clearly. Traditionally it takes 24 months before a junior becomes a net-positive contributor. AI can compress this to 9 months if you use it strategically. DX research shows onboarding compressed from 91 days to 49 days with daily AI use.
Vibe coding—accepting whatever AI generates without understanding—may extend the valley indefinitely.
The apprenticeship model historically taught juniors through making mistakes and fixing them under senior guidance. AI changes this to accepting generated code without understanding. You risk creating a generation unable to work without AI assistance—what we explore as apprenticeship model breakdown in the broader discussion of craftsmanship and long-term costs.
They lack experience-based scepticism about code quality. They trust AI specificity more—78% juniors versus 39% seniors. They prioritise immediate productivity over long-term skill mastery. They haven’t encountered the debugging nightmares that come from blindly trusting generated code.
Industry-wide AI adoption reached 91% across 435 companies and 135,000+ developers, with junior developers showing the highest adoption rate. The pattern shows an inverse correlation between experience level and enthusiasm.
The ArXiv study coined the term “Experience Paradox” for this. Junior developers—fewer than 5 years experience—demonstrate significantly higher confidence in AI specificity. Senior engineers with 15+ years experience show marked scepticism. Greater expertise exposes limitations in AI reasoning.
Juniors lack a reference frame for “normal” development speed, so AI feels natural. They haven’t experienced legacy codebase debugging. They’re eager to prove productivity.
Seniors have seen automated code generation tools fail before. They’ve debugged thousands of subtle bugs. They value code understanding over code speed.
This pattern matters for succession planning. If juniors develop dependency rather than augmented capability, the senior pipeline gets threatened. Smaller teams have less redundancy. The team dynamics challenges between enthusiastic juniors and skeptical seniors require careful navigation.
Kent Beck’s framework distinguishes between skills AI deprecates, skills AI amplifies, and entirely new skills. Deprecated: language syntax mastery, framework API memorisation. Amplified: vision, architectural strategy, code quality taste, system design judgement. New: prompt engineering, AI output validation, debugging AI-generated code, strategic task selection.
The framework gives you a roadmap for where juniors should focus learning effort. It shifts training from memorisation to judgement.
AI now retrieves language syntax and handles framework APIs. But foundational understanding remains required as a validation baseline.
Architectural taste becomes the primary differentiator. Vision and strategy grow more important when implementation gets accelerated. System design judgement becomes vital for directing AI effectively. These skills were always important. Now they’re the skills.
New skills include prompt engineering to communicate intent. Validation techniques to catch AI errors. Debugging code you didn’t write and don’t understand. Strategic task selection—when manual builds necessary skills versus when AI is appropriate.
Beck emphasises in augmented coding: “You care about the code, its complexity, the tests, and their coverage” with a value system similar to hand coding. Vibe coding means “you don’t care about the code, just the behaviour of the system” and feeding errors back into AI hoping for fixes.
Traditional training focused on syntax and APIs is largely wasted now. You need to frontload architectural thinking earlier. Validation and debugging skills become day-one priorities, not year-two skills.
The traditional software apprenticeship model—where junior developers gradually build expertise through hands-on struggle under senior mentorship—is breaking down. AI coding assistants automate the struggle that builds deep knowledge, creating juniors who debug AI-generated code they don’t understand.
The traditional progression worked like this: junior writes code, encounters errors, struggles to debug, reads documentation, asks senior for guidance, finally solves problem, deeply understands solution because of struggle, repeats thousands of times, becomes senior.
Pedagogical research shows effortful retrieval builds long-term memory. Debugging creates mental models. Frustration followed by breakthrough produces lasting knowledge.
AI disrupts this. Junior prompts AI for code, receives working solution, has no error to debug, reads no documentation, experiences no struggle, moves to next task, develops surface-level familiarity without deep knowledge.
Mentorship evolution presents a challenge. Traditional code review asked “why did you implement it this way?” AI era code review must ask “do you understand what the AI implemented?” The junior often can’t answer.
Kent Beck notes productive developers “don’t just produce—they eventually mentor others, creating compounding returns across the organisation.”
Chris Banes warns AI is “automating the learning process” entirely if organisations don’t deliberately preserve human-centred learning work.
Who becomes senior engineers in 5 years if today’s juniors never build debugging muscle memory? You can’t hire your way out of this problem because the market has the same issue. Engineering teams with less redundancy are particularly vulnerable.
You lack the authorial context that guides troubleshooting. When you write code yourself, you understand the intended logic, design choices, and assumptions. AI-generated code appears as a “black box” where you must first reverse-engineer the approach before identifying bugs.
Anthropic research showed the largest performance gap in debugging questions specifically. The 17-point comprehension gap—50% versus 67%—had Cohen’s d=0.738 statistical significance.
When you debug your own code, you know what you intended. You understand trade-offs made. Errors reveal gaps in your understanding. The debugging process builds mental models.
When you debug AI code, you must first understand what AI implemented. It’s unclear which parts are important. You don’t know assumptions AI made. You can’t distinguish intentional pattern from AI hallucination.
The reverse-engineering burden is substantial. The cognitive load is higher than debugging familiar code.
CodeRabbit research found AI changes introduced roughly 1.7 times more issues than human-written code. Logic errors occurred 2.25 times more frequently. Error handling gaps were nearly twice as common.
AI-generated code often omits null checks, early returns, and comprehensive exception logic. Banes emphasises “AI is systematically bad at knowing when it is wrong.”
Debugging AI code without understanding short-circuits learning. Juniors may develop false confidence while missing foundational knowledge.
Use a three-level framework. Level 1—weeks 1 to 2: AI limitations awareness. Level 2—weeks 3 to 4: strategic task selection. Level 3—ongoing: quality validation techniques.
Combine this with “manual-first then AI” methodology where juniors implement foundational tasks manually first to build understanding before using AI for repetition.
DX research shows a 25% increase in structured enablement produced a 10.6% confidence gain and 16.1% reduction in knowledge gaps. Organisations providing structured enablement saw 8.0% code maintainability improvement and 18.2% time loss reduction.
AI handles approximately 70% of tasks well but struggles with complex architecture, security-sensitive code, and domain-specific logic. Juniors must learn to recognise which 30% requires a manual approach.
Context rot: AI lacks full codebase context, makes assumptions that break system-wide patterns. Validate AI suggestions against existing architecture.
Hallucination patterns: AI generates plausible-looking but incorrect code, frameworks that don’t exist, API methods with wrong signatures. Over 40% of LLM-generated code contains security vulnerabilities.
The decision matrix: when to use AI—boilerplate code, well-established patterns, test case generation—versus when manual—first-time implementation, security-sensitive features, complex business logic, architectural decisions.
Chris Banes identifies optimal AI conditions: bounded mechanical tasks with objective verification through tests, small reversible blast radius, clear acceptance criteria. AI breaks down for security-sensitive implementations like authentication, tasks requiring deep cross-module understanding, situations where correctness is product judgement.
Practice scenarios present juniors with task lists. They justify AI versus manual choice. Review decisions with a senior. Build judgement through repetition.
Real examples: authentication first time manual to learn session management, password hashing, security principles. Subsequent implementations with AI review for efficiency without sacrificing understanding.
Debugging AI code techniques: reverse-engineer AI’s approach before debugging, validate assumptions, check for subtle logic errors, test edge cases AI might miss.
Testing strategies: AI code requires more thorough testing. Focus on boundary conditions, security implications, integration with existing systems.
Code review focus: Can the junior explain what AI implemented? Do they understand trade-offs? Can they debug if AI is unavailable?
Training activities include comprehension quizzes using Anthropic methodology, debugging challenges with AI-generated code.
Take authentication implementation. First time: junior writes authentication manually, struggles through session management, debugs timeout issues, understands password hashing, builds deep security understanding.
Subsequent authentication implementations use AI with thorough review. Junior validates security, checks edge cases, gains efficiency without losing understanding.
Faros AI research shows peer-to-peer learning is 22% more effective than formal training alone. Document successful patterns the team discovers.
Timeline expectations: Level 1 takes 1 to 2 weeks. Level 2 takes 2 to 3 weeks. Level 3 is ongoing throughout the first year. Total ramp is still shorter than traditional 24 months.
Smaller teams can’t dedicate full-time to training, so integrate into daily work. This is strategic because you can’t afford a skill gap. For comprehensive training approaches and responsible AI usage frameworks, see our implementation playbook.
You face a succession planning problem where you’ll lack senior engineers with architectural judgement, debugging instincts, and mentorship capabilities. This is a workforce risk for companies with 50 to 500 employees that can’t afford skill gap generations, can’t hire their way out because the entire market has the same pipeline problem, and depend on continuous junior-to-senior progression.
Organisational capability depends on a continuous pipeline of juniors to mid-level to seniors. You can’t maintain technical excellence with only junior developers no matter how AI-assisted. Senior engineers provide architectural vision, debugging expertise, mentorship, and long-term system understanding.
The 5-year timeline matters. Traditional ramp produces mid-level engineers after 3 to 4 years and seniors after 5 to 7 years. The cohort currently learning with AI—2024 to 2026—becomes mid-level engineers 2027 to 2029 and seniors 2029 to 2032. If that cohort has skill gaps, you hit a capability problem just as you need those senior engineers most.
This is a market-wide problem. You can’t hire senior engineers away from other companies because the entire industry has the same pipeline issue. If juniors everywhere are learning with AI and developing the same skill gaps, senior engineer shortage becomes industry-wide. You must “grow your own” seniors.
Engineering teams in your size range typically have 5 to 50 engineers. Individual skill gaps have outsized impact. Less redundancy if one senior leaves. Limited capacity for remedial training programmes. You can’t afford a “lost generation” of engineers with shallow skills.
Stack Overflow survey found 25% year-over-year decline in entry-level tech hiring in 2024. 70% of hiring managers believe AI can perform intern-level work.
Banes argues “the concern isn’t that AI eliminates jobs but that it eliminates learning pathways.” Organisations hiring fewer juniors today creates senior shortage in 5 years.
Best case with proper training: AI-accelerated ramp, juniors develop deep knowledge faster, senior pipeline improves. Worst case: generation of surface-level coders who can use AI but can’t debug, design, or mentor.
Early warning signs include juniors who can’t debug without AI assistance or explain code they submitted.
Succession planning is a technical capability strategy. Today’s training decisions determine 2029 senior engineer capacity. You must champion training investment despite pressure for immediate productivity.
AI won’t replace junior or senior developers but it’s fundamentally changing required skills. Junior employment is already declining—25% year-over-year entry-level hiring drop, 30% internship decline—not because AI replaces juniors but because organisations are hiring fewer juniors to train.
Senior engineers remain necessary for architectural judgement, AI output validation, and mentorship that AI can’t provide. This creates a situation where juniors appear most replaceable but you need juniors to become tomorrow’s irreplaceable seniors.
91% of developers are using AI and employment remains strong, suggesting augmentation rather than replacement. Companies are still hiring but changing expectations of what developers do.
Junior work historically focused on tasks AI now automates: writing boilerplate, simple bug fixes, documentation.
Senior developers remain necessary. Architectural vision can’t be automated. Debugging complex production issues requires institutional knowledge. Mentorship and code review need human judgement. AI output validation requires experienced engineers.
The succession paradox: You need fewer juniors today because AI handles junior-level tasks. But those juniors become tomorrow’s irreplaceable senior engineers. Reducing junior hiring today creates senior shortage in 5 years.
Kent Beck’s perspective on automation: AI deprecates language syntax expertise and framework API knowledge—skills easily automated. AI amplifies vision, strategy, and architectural taste—uniquely human judgement skills. Replacement concern misses the point: AI changes what makes engineers valuable, not whether they’re valuable.
Long-term outlook: Junior developers who master augmented coding—AI to accelerate while maintaining deep knowledge—become more valuable seniors, not less valuable. Those who fall into vibe coding pattern—accepting AI without understanding—may plateau at mid-level or be replaceable.
Don’t reduce junior hiring to zero. That destroys the succession pipeline. Do change what juniors learn: less syntax memorisation, more architectural thinking. Invest in training frameworks that develop AI-era skills. Measure comprehension not just output.
The data shows AI augments capable engineers and exposes skill gaps in those relying on surface knowledge. The future belongs to engineers who understand systems deeply and leverage AI strategically. For the complete strategic context on how AI is reshaping engineering teams and what it means for your organisation, see our vibe coding comprehensive guide for engineering leaders.
Juniors can produce working code with AI tools within days. But developing the validation skills to use AI responsibly takes 6 to 8 weeks with structured training. Building deep enough knowledge to become senior engineers still requires 9 to 18 months of strategic AI-assisted practice—compressed from traditional 24 months—according to Kent Beck’s updated “Valley of Regret” timeline.
91% of developers overall use AI coding tools based on DX research covering 135,000+ developers, with junior developers showing the highest adoption rate. This creates an “Experience Paradox” where juniors trust AI specificity at 78% versus 39% for seniors.
No. Anthropic research found a 17-point comprehension gap—50% versus 67%, statistically significant at p=0.01—when developers learned with constant AI assistance versus manual coding. Debugging skills showed the largest deterioration. Strategic AI usage following “manual-first then AI” pattern can accelerate learning while preserving comprehension.
Ask juniors to explain AI implementation line-by-line. Check if they can debug without AI assistance. Question architectural choices to see if they understand trade-offs, not just accept AI defaults. Focus code review on edge cases and security implications AI commonly misses. Require juniors to manually implement similar functionality first time before using AI for repetition.
No. Reducing junior hiring creates a succession planning problem because today’s juniors become tomorrow’s senior engineers who provide irreplaceable architectural judgement, mentorship, and debugging expertise. Change what juniors learn—less syntax, more validation and judgement—rather than eliminate the role.
Use comprehension quizzes on code they submitted following Anthropic methodology. Ask them to debug AI code without AI assistance. Have them explain architectural decisions and trade-offs. Use code review questions that probe understanding of edge cases. Compare their manual implementations to AI implementations. Track production issues from code they submitted.
Kent Beck’s framework prioritises amplified skills over deprecated skills: architectural judgement, code quality taste, system design vision, strategic task selection, debugging skills particularly for AI-generated code, prompt engineering, and validation techniques. De-emphasise syntax memorisation and API knowledge that AI retrieves automatically.
Vibe coding means accepting AI-generated code based on whether it “feels right” without deep understanding or validation, leading to skill atrophy and quality issues—1.7 times more bugs and 2.25 times more logic errors. Augmented coding means using AI to accelerate development while maintaining code quality standards through validation, preserving deep knowledge building, and strategic task selection—manual for learning, AI for efficiency.
The traditional apprenticeship model—learning through debugging own mistakes, reading documentation, understanding error messages—breaks down when AI provides answers without requiring the struggle that builds deep knowledge. Manual-first methodology can preserve apprenticeship benefits while leveraging AI efficiency.
Yes. Authentication and security to learn session management and password hashing principles. Error handling patterns to understand exception hierarchies. Database transactions to grasp ACID properties. Testing frameworks to develop quality mindset. Complex business logic to build contextual understanding. Architectural decisions to develop design judgement. The pattern: manual first-time implementation builds foundational knowledge, subsequent similar tasks can use AI with thorough validation.
Leverage peer-to-peer learning—22% more effective than formal training alone. Integrate training into daily work rather than dedicated programmes. Document organisational AI usage patterns. Use manual-first pattern for foundational skills. Focus enablement on strategic task selection and validation techniques. Smaller teams have less redundancy and higher individual impact.
Inability to debug without AI assistance. Struggling to explain code they submitted. Shallow answers when asked about architectural trade-offs. Production incidents from missed edge cases AI commonly overlooks. Increasing dependency on AI for basic tasks. Difficulty reading existing codebase code. Avoidance of documentation reading. Resistance to manual implementation even for learning. Lack of progression in architectural thinking despite months of experience.
For a complete strategic overview of AI-assisted development challenges and opportunities, including how to address junior skill development alongside security risks, productivity measurement, and team dynamics, see our complete strategic overview.
The Case For AI Coding Tools – Democratisation, Velocity, and When They Actually Help84-90% adoption rates reported in Stack Overflow surveys show AI coding tools have moved from curiosity to default in many development teams. The marketing narrative promises productivity gains and democratised software creation. The reality is messier than the headlines suggest.
Vendor research from Microsoft and Accenture claims a 26% task completion increase. Independent studies from METR found experienced developers were 19% slower with AI tools. This tension means the real question isn’t whether to adopt AI coding tools, it’s when, where, and how.
This article is part of our comprehensive vibe coding and the death of craftsmanship guide for engineering leaders, where we examine the full spectrum of AI coding tool implications. Here, we present the genuine benefits, examine what the data actually shows, and provide a decision framework for strategic selective adoption.
AI coding tools lower the barrier to software creation. Non-programmers and citizen developers can now build functional applications through natural language prompts rather than learning syntax and architecture patterns.
Kevin Roose, a New York Times journalist with no coding background, experimented with vibe coding to create several small-scale applications. He called these “software for one”—personalised utilities that would never justify hiring a developer. Internal dashboards, simple automations, personal productivity tools are all suddenly accessible to people who previously couldn’t participate in software creation.
Y Combinator’s Winter 2025 batch included 25% of startups with codebases that were 95% AI-generated. These weren’t non-technical founders either. Y Combinator managing partner Jared Friedman noted that every one of these founders was highly technical. A year ago they would have written everything manually. Now AI does 95% of it.
This democratisation operates on two levels. First, it enables non-programmers to create software for bounded, well-defined problems. Second, it levels the playing field for junior developers who can produce working code faster.
But there are limits. YC general partner Diana Hu pointed out that you need taste and enough training to judge whether an LLM is producing good or bad output. Professional developers remain necessary for complex systems.
The strongest democratisation gains come from well-defined problems with existing patterns. Need an internal dashboard or a simple automation? AI tools can get you there. Building a complex system requiring architecture decisions? You still need professional expertise.
AI coding tools excel at eliminating repetitive, formulaic work that drains developer energy without requiring creative or architectural thinking. GitHub Copilot research found that 87% of developers felt it preserved mental effort during repetitive tasks.
Boilerplate code generation handles predictable structures well. CRUD endpoints, REST API scaffolding, standard configuration files, repetitive data model patterns—these follow structures that AI handles effectively. One developer reported: “I have to think less, and when I have to think it’s the fun stuff”.
Prototyping velocity gets a boost too. AI enables rapid iteration on stakeholder-facing prototypes, compressing feedback loops from days to hours. You’re validating ideas quickly before investing in proper implementation.
Unit test generation from existing function signatures is a strong use case. The specification is already defined through the function’s inputs and outputs. Writing tests for it is repetitive work that follows clear patterns.
The common thread is well-defined patterns with existing examples. AI tools perform best when the problem space has been solved before and you need a variation, not an invention. Between 60-75% of GitHub Copilot users reported feeling more fulfilled with their job.
Cognitive toil reduction frees senior developers to focus on architecture and complex problem-solving where human judgement remains necessary.
Microsoft Research conducted a study with Accenture involving 4,800 developers, reporting a 26% faster task completion rate with AI coding tools. The study authors framed this as “turning an 8-hour workday into 10 hours of output.”
This is the most-cited data point. Methodology context matters though. The study was vendor-funded—Microsoft makes Copilot. Tasks were likely selected to favour AI-assisted completion. The developer population skewed toward Accenture consultants working on structured projects.
Google’s internal study with Gemini Code Assist reported 21% productivity improvements, but again within a controlled, vendor-optimised environment.
METR conducted an independent randomised controlled trial with 16 experienced open-source developers. They found developers were 19% slower when using AI tools on their own projects. These were real issues from large open-source projects with over 1 million lines of code.
The perception gap is revealing. Developers expected AI would make them 24% faster, and even after finishing slower, still believed AI had sped them up by around 20%. A Cerbos team member explained this: “The dopamine rewards activity in the editor, not working code in production”.
The discrepancy isn’t necessarily contradiction. Vendor studies measure scenarios with greenfield projects and structured tasks. METR measured real-world complexity—existing codebases, established patterns, deep context requirements.
Funding source matters for credibility. Vendor-funded studies have inherent incentive to demonstrate value. Independent RCTs have no commercial stake in the outcome.
The honest interpretation: AI tools deliver genuine productivity gains in specific contexts. But “26% faster” shouldn’t be extrapolated to all development work. Your mileage will vary based on task type and codebase complexity.
Stack Overflow’s developer survey reports 84-90% AI tool adoption rates, indicating widespread usage across the industry. Over half of professional developers use these tools daily.
72% of developers rate GitHub Copilot favourably, representing genuine enthusiasm. But there’s a complication. 66% of developers also report an “almost right but not quite” frustration—AI suggestions that look correct but contain subtle errors requiring debugging.
These mixed signals suggest a tool that’s useful enough to adopt widely but imperfect enough to create new categories of work. One Stack Overflow respondent captured it: “AI solutions that are almost right, but not quite, are now my biggest time sink”.
Context dependency is the key insight. Satisfaction correlates with task type and codebase complexity, not with the tool itself. Developers working on greenfield projects report higher satisfaction. Those on complex legacy systems report more frustration.
The data shows increasing adoption but decreasing trust. Favourable views dropped from over 70% in 2023 to around 60% in 2025. 46% of developers now say they don’t trust the accuracy of AI output—a sharp rise from 31% last year.
High adoption rates reflect the convenience of the tool and the industry zeitgeist, not necessarily net productivity improvement. 45% of developers specifically complained that debugging AI-generated code is more work than it’s worth.
Interestingly, 72% of developers said vibe coding—letting AI generate whole programs—is not part of their professional work. Most use AI in a more incremental, assistive capacity for specific tasks.
Match AI tool usage to specific contexts. Use AI for tasks with well-defined patterns and human expertise for complex architecture and security-sensitive code. For a complete framework for responsible AI-assisted development, see our implementation playbook with decision criteria, quality gates, and review processes.
High suitability scenarios:
Medium suitability scenarios:
Low suitability or avoid entirely:
Use AI to generate a REST API endpoint for a simple user data model. CRUD operations, well-defined patterns, low risk.
Avoid AI for designing distributed systems coordination logic. This requires deep understanding of consensus algorithms, failure modes, and CAP theorem trade-offs. AI will give you plausible-looking code that fails under load.
Use AI to write unit tests for an existing, well-documented function. The specification is clear, the task is repetitive, the output is verifiable.
Avoid AI for refactoring a poorly documented legacy authentication system. Context rot makes AI output unreliable, security implications are high, and you need deep institutional knowledge.
Most real-world development involves a mix of both. You’ll switch between AI-assisted and manual modes within a single project. The skill is knowing which mode fits which task. Our strategic selective adoption framework provides detailed decision criteria for navigating these choices.
AI coding tools perform best on greenfield projects. There’s no existing context to misinterpret, no legacy patterns to conflict with, and no accumulated technical debt to navigate.
In complex or legacy codebases, “context rot” degrades AI effectiveness. The tool lacks understanding of historical design decisions, undocumented business rules, and implicit dependencies.
Most teams manage existing codebases, not greenfield projects. The average SMB technology stack includes 8-year-old systems with poor documentation and accumulated workarounds. This explains why vendor research—typically conducted on greenfield or structured projects—overstates real-world gains for most development teams.
Legacy code challenges for AI include inconsistent naming conventions, implicit coupling between modules, undocumented side effects, and business logic embedded in code comments. More context is not always better—bigger context windows often distract the model.
Output quality gets worse the more context you add. The model pulls in irrelevant details and accuracy drops. The result is bloated code that looks right but doesn’t solve your problem.
Context engineering—structuring documentation for AI consumption—can partially mitigate these limitations. But it requires upfront investment that many teams underestimate.
The bottom line: if your team mainly works on new features in greenfield projects, AI tools will probably deliver on the vendor promises. If you’re managing legacy systems with poor documentation, expect diminished returns.
Junior developers show higher adoption rates and report more immediate benefits. Microsoft and Accenture found that less experienced developers saw as high as 35-39% speed-up, while seasoned developers saw smaller 8-16% improvements.
Copilot acted like an “always-available mentor” for juniors, helping them write code they might otherwise struggle with. It accelerates learning by providing working examples and reducing blank-page paralysis.
Senior developers show lower adoption rates and express more quality concerns. They recognise subtle errors AI makes, worry about skill atrophy in junior colleagues, and find AI suggestions less helpful for complex architectural decisions. Faros AI found that AI usage is highest among engineers who are newer to the company—they lean on AI to navigate unfamiliar codebases.
The benefit profile differs by role. Juniors gain velocity on straightforward tasks. Seniors gain cognitive toil reduction on repetitive work they already know how to do.
There’s a skill development consideration here. Over-reliance on AI tools early in a career may prevent developers from building the deep understanding needed for senior-level work. Architecture decisions, debugging complex interactions, system design—these require mental models that come from struggling through problems manually.
If juniors never learn to code without AI, does the organisation develop the next generation of senior engineers? The long-term question is still open.
A strategic approach: juniors using AI tools with mandatory human review creates a productive learning environment. Seniors using AI for boilerplate frees time for mentoring and architectural leadership. The key is treating AI as a tool within a broader development discipline, not a replacement for learning fundamentals.
Interestingly, in METR’s study, only one participant had more than 50 hours experience with Cursor. That one experienced user did see a positive speedup, suggesting a learning curve effect. Like any tool, proficiency matters.
Many development teams are enthusiastic about AI tools and expect full, unrestricted adoption. The challenge is communicating a “yes, but strategically” policy without dampening morale.
Frame the conversation around smart deployment. “We’re adopting AI tools for specific use cases where they demonstrably help, and maintaining human-led approaches where complexity demands it.”
Share the decision matrix with your team so developers understand the rationale behind guidelines. Show the data—the Microsoft 26% improvement alongside the METR 19% slowdown. Context determines effectiveness.
Address the enthusiastic adopters. Acknowledge the genuine benefits they experience. Then introduce the evidence showing where AI tools create more work than they save. 66% of developers experience “almost right but not quite” frustration. Your enthusiasts have probably hit this too.
Address the sceptics. Show that the policy incorporates review gates and context-dependent guidelines. Simon Willison’s golden rule: “I won’t commit any code to my repository if I couldn’t explain exactly what it does to somebody else.”
If an LLM wrote the code for you, and you then reviewed it, tested it thoroughly and made sure you could explain how it works—that’s software development, not vibe coding.
Practical policy elements to include:
Define which project types default to AI-assisted workflows. Greenfield CRUD applications, prototyping, boilerplate generation—these are green lights.
Establish review requirements for AI-generated code. All AI output gets human review before merging. No exceptions.
Set up feedback loops to refine the policy based on team experience. Monthly retrospectives on what worked and what created friction.
Most organisations have AI usage driven by bottom-up experimentation with no structure, training, or strategy. Don’t be most organisations. Make it structured, documented, and improvable.
This is an ongoing conversation, not a one-time decree. Quarterly review cadence makes sense as AI tool capabilities evolve. Each review should assess which use cases expanded, which created unexpected problems, what new capabilities emerged, and whether review processes need adjustment.
For more on implementing balanced AI adoption strategies across your engineering organisation, explore our complete guide to vibe coding and the death of craftsmanship, which synthesizes evidence, team dynamics, security implications, and actionable frameworks for strategic decision-making.
Yes, for bounded, well-defined problems. AI coding tools enable non-programmers to create internal dashboards, simple automations, and personal utilities through natural language prompts. A Stack Overflow writer built a functional bathroom review app using Bolt. The limitation is complexity—applications requiring authentication, data integrity, or multi-system integration still need professional development expertise.
Vibe coding vs AI-assisted engineering describes generating code through high-level prompts without reviewing implementation details. Andrej Karpathy described it as “forget that the code even exists”. AI-assisted engineering uses AI tools while maintaining developer oversight, testing, and code review. Simon Willison put it clearly: if an LLM wrote the code and you then reviewed it, tested it thoroughly and made sure you could explain how it works—that’s software development, not vibe coding. The distinction matters because vibe coding suits prototyping and personal projects, while production systems require the discipline of AI-assisted engineering.
The 26% task completion increase is a real finding from a study with 4,800 developers, but context matters. The study was funded by Microsoft (which sells Copilot), tasks were structured and likely favourable, and the independent METR study found experienced developers were 19% slower on their own projects. Both findings can be true—productivity depends on context. Your results will fall somewhere on that spectrum based on your codebase and task types.
AI models generate statistically probable code based on training data, which often produces syntactically correct but semantically imprecise output. 66% cite this as their biggest frustration, and 45% specifically complained that debugging AI-generated code is more work than it’s worth. Subtle errors in business logic, edge case handling, or API usage create a debugging burden that can offset time saved—particularly in complex or legacy codebases.
Each serves different strengths. GitHub Copilot offers the broadest IDE integration and largest user base. Cursor provides fast autocomplete in its own IDE—METR study participants mainly used Cursor Pro with Claude models. Claude Code excels at deep reasoning and complex problem-solving for architectural decisions. Many teams use multiple tools for different tasks rather than committing to one. Start with whichever integrates best with your existing workflow.
Not as well as with greenfield projects generally. Legacy systems involve undocumented business rules, implicit dependencies, and accumulated technical debt that AI tools struggle to interpret. METR’s study with experienced developers on large open-source projects showed 19% slowdown. Context engineering—structuring documentation for AI consumption—can help, but expect diminished returns compared to new projects.
Yes, with structured oversight. AI tools accelerate learning by providing working examples and reducing blank-page paralysis. Microsoft and Accenture found less experienced developers saw as high as 35-39% speed-up, with Copilot acting like an “always-available mentor”. However, mandatory code review and periodic manual coding exercises prevent over-reliance that could stunt the deep understanding needed for career progression to senior roles.
Track specific metrics beyond developer sentiment: lead time for changes, pull request cycle time, change failure rate, and code review turnaround. Compare these DORA metrics before and after AI tool adoption, controlling for project type and complexity. Faros AI found no significant correlation between AI adoption and improvements at company level. Subjective satisfaction surveys alone don’t capture net productivity impact.
The AI Productivity Paradox describes the phenomenon where widespread individual AI tool adoption doesn’t translate to measurable organisational performance improvements. Faros AI analysed over 10,000 developers and found that 75% use AI tools, yet most organisations see no measurable performance gains. Individual developers may feel faster, but bottlenecks in code review, testing, and integration mean the organisation’s throughput doesn’t scale proportionally. PR review time increases 91%, revealing that human approval becomes the constraint.
AI-generated code can introduce security vulnerabilities because models optimise for functional correctness rather than security best practices. A Stack Overflow writer’s vibe-coded app was “ripe for hacking” with no security features. Simon Willison warns about secrets like API keys accidentally ending up in code. Mitigation requires the same security review processes applied to human-written code—automated scanning, peer review, and security-focused testing.
Quarterly review is recommended. AI coding tool capabilities evolve rapidly, and a policy appropriate for January may be outdated by April. Most organisations currently have no structure, training, or strategy for AI tool usage. Each review should assess which use cases expanded, which created unexpected problems, what new capabilities emerged, and whether review processes need adjustment.
The Case Against Vibe Coding – Understanding, Craftsmanship, and Long-Term CostsAI coding tools promise massive productivity gains. And they deliver, at least initially. But something is happening beneath the surface—your codebase is piling up debt faster than you realise.
GitClear analysed 211 million lines of code across 2020-2024. CodeRabbit compared 470 pull requests—320 AI-generated versus 150 human-written. Veracode tested over 100 LLMs across four programming languages. All three studies point the same way: vibe coding—generating code without understanding it—undermines maintainability, security, and the resilience of your team.
This article unpacks the difference between vibe coding and augmented coding—Kent Beck’s disciplined alternative that keeps understanding at the centre while still using AI. It’s about sustainable pace versus short-term velocity.
For a complete strategic overview of vibe coding and its implications for engineering leaders, see our comprehensive guide. This article presents the evidence for why understanding code still matters.
Understanding is the foundation of maintainability. When developers actually comprehend their codebase, they can debug, extend, and refactor it efficiently. Without comprehension, every change becomes a high-risk experiment.
Kent Beck put it simply: in vibe coding you don’t care about the code, just the behaviour of the system. In augmented coding you care about the code, its complexity, the tests, and their coverage. The value system in augmented coding is similar to hand coding—tidy code that works.
His B+Tree project demonstrated this. Augmented coding can create production-ready, performance-competitive library code while maintaining code comprehension. You focus on the consequential design decisions rather than the repetitive implementation details.
Debugging code you did not write and do not understand is expensive. The time cost compounds as the codebase grows and the original context is lost. Jeremy Twei coined the term “comprehension debt” for this—the growing gap between code a team can review syntactically and code they actually understand architecturally.
Unlike technical debt, which shows up in metrics like duplication and complexity, comprehension debt is hidden until an incident reveals that no one truly understands how something works. You can review code competently even after your ability to write it from scratch has atrophied, but there’s a threshold where “review” becomes “rubber stamping”.
For smaller teams, losing institutional knowledge is serious. Fewer engineers means each person’s understanding carries more weight. When developers rely on AI to generate code they never deeply engage with, institutional knowledge degrades. Your bus factor increases.
Organisational resilience depends on shared understanding across the team. That’s not Luddite resistance. That’s stewardship.
The 70% Problem, coined by Addy Osmani, describes AI-generated code that appears mostly correct but requires disproportionate human effort to complete, debug, and make production-ready.
AI errors have evolved from syntax bugs to conceptual failures—the kind a sloppy, hasty junior developer might make under time pressure.
Stack Overflow’s 2025 survey showed 66% of developers experience “AI solutions that are almost right, but not quite” as their top frustration. 45% reported that “debugging AI code takes longer than writing it myself”. Only 16% reported great productivity improvements from AI tools, while half saw only modest gains.
The completion cost paradox is real. Finishing the remaining 30% often takes longer than writing the code from scratch, because you have to reverse-engineer AI assumptions before fixing them.
Here’s a concrete example. An AI generates an authentication module. The happy path works—valid login credentials succeed. But edge cases fail: password reset flows break, session timeout handling doesn’t exist, concurrent login conflicts cause data corruption. You spend three hours debugging and fixing what would have taken one hour to write manually.
The “almost right” pattern is psychologically dangerous. You trust code that looks correct, which reduces scrutiny and delays bug discovery until production. Andrej Karpathy described the problem: “The models make wrong assumptions on your behalf and run with them without checking. They don’t manage confusion, don’t seek clarifications, don’t surface inconsistencies, don’t present tradeoffs, don’t push back when they should.”
Assumption propagation compounds the issue. The model misunderstands something early and builds an entire feature on faulty premises. You don’t notice until you’re five PRs deep and the architecture is cemented.
Yoko Li captured the psychological hook: “The agent implements an amazing feature and got maybe 10% of the thing wrong, and you’re like ‘hey I can fix this if I just prompt it for 5 more mins.’ And that was 5 hrs ago.”
When completion cost exceeds writing from scratch, the productivity gain is negative.
GitClear analysed 211 million lines of code changed between January 2020 and December 2024 from repos owned by Google, Microsoft, Meta, and enterprise C-Corps. It’s the largest longitudinal dataset on how AI coding tools affect codebase health.
Refactoring collapsed from 25% to under 10% of developer activity—a 60% decline. Developers are generating new code rather than improving existing code.
Code duplication increased 4x in volume. For blocks of five or more lines, duplication increased 8x. This violates the DRY principle at scale and creates maintenance burdens across entire codebases.
Code churn—code written and then rewritten or deleted shortly after—nearly doubled, indicating wasted effort and instability. For the first time in history, “copy/paste” code exceeded “moved” code (code reuse).
These metrics compound over time. Technical debt increases an estimated 30-41%. Unlike financial debt, technical debt accrues interest in the form of slower development, more bugs, and higher incident rates.
Research suggests that DRY, modular approaches retain high project velocity over years. Canonical systems are documented, well-tested, reused, and periodically upgraded. AI-generated code moves in the opposite direction.
Smaller teams with fewer engineers cannot absorb a 30-41% increase in maintenance burden. The debt accumulates faster than you can pay it down. For your organisation, this may be a serious threat, not an academic concern.
Thoughtworks Technology Radar cited GitClear’s research when placing “AI coding complacency” on “Hold” status—their strongest cautionary rating.
CodeRabbit analysed 470 pull requests—320 AI-generated, 150 human-written. AI code has 1.7x more issues overall than human-written code. AI-authored changes produced 10.83 issues per PR, compared to 6.45 for human-only PRs.
Logic errors are 75% more frequent in AI-generated code. These are not formatting or style issues but functional defects that affect correctness—business logic mistakes, incorrect dependencies, flawed control flow, and misconfigurations.
Readability issues are 3x more common. Poor variable naming, convoluted structure, and inconsistent patterns make code harder for humans to review and maintain. Readability spiked more than anything else in the dataset—the single biggest difference.
Error handling and exception-path gaps were nearly 2x more common. Performance regressions, though small in number, skewed heavily toward AI—excessive I/O operations were 8x more common in AI-authored PRs.
Concurrency and dependency correctness saw 2x increases in AI PRs. Formatting problems were 2.66x more common. AI introduced nearly 2x more naming inconsistencies.
These quality gaps aren’t random. AI optimises for “looking correct” rather than being maintainable. It produces code that passes superficial review but degrades over time.
AI lacks local business logic. Models infer code patterns statistically, not semantically. Without strict constraints, they miss the rules of the system that senior engineers internalise. They generate surface-level correctness—code that looks right but may skip control-flow protections or misuse dependency ordering. Naming patterns, architectural norms, and formatting conventions often drift toward generic defaults. AI favours clarity over efficiency, often defaulting to simple loops, repeated I/O, or unoptimised data structures.
Review fatigue compounds the problem. Reviewers spend 91% more time on AI-generated PRs. Under volume pressure, quality of review declines—leading to rubber-stamping. A recent Cortex report found that while pull requests per author increased by 20% year-over-year thanks to AI, incidents per pull request increased by 23.5%.
No issue category was uniquely AI, but most categories saw significantly more errors in AI-authored PRs. Humans and AI make the same kinds of mistakes. AI just makes many of them more often and at a larger scale.
Here are the numbers in one view:
AI Code vs Human Code Quality Metrics
These are measurable, compounding costs.
These quality problems get worse in complex production environments.
LLMs have a fundamental limitation: context windows restrict the amount of code they can process at once. Output quality declines as codebases grow—a pattern called context rot.
AI excels at generating boilerplate and greenfield code but struggles with architectural decisions that require understanding system-wide implications. Most real-world development happens in complex legacy systems, not greenfield projects—the exact environment where AI tools perform worst. In mature codebases with complex invariants, the calculus inverts. The agent doesn’t know what it doesn’t know. It can’t intuit the unwritten rules. Its confidence scales inversely with context understanding.
If you’re managing existing codebases—and most teams are—the productivity narrative around AI tools is misleading. Gains demonstrated on simple projects do not transfer to production systems with years of accumulated context.
Abstraction bloat emerges when AI creates elaborate class hierarchies or 1000-line implementations where 100 lines would suffice. It optimises for “looking comprehensive” rather than maintainability.
Productivity benchmarks on simple tasks misrepresent real-world performance. If you’re managing a complex legacy codebase—and most of you are—the gains you’ve read about don’t apply.
Veracode analysed 100+ LLMs across 4 programming languages and found AI-generated code introduced risky security flaws in 45% of tests. Security issues were up to 2.74x higher in AI-generated code compared to human-written code.
45% of AI-generated code fails secure coding benchmarks—nearly half of all AI output introduces potential attack vectors.
The most prominent security pattern involved improper password handling and insecure object references. Common vulnerability types include SQL injection (CWE-89), weak cryptography (CWE-327), cross-site scripting (CWE-80), and log injection (CWE-117)—patterns where AI defaults to insecure but functional implementations.
No vulnerability type was unique to AI, but nearly all were amplified.
Stack Overflow’s case study on a vibe-coded app found it was ripe for hacking—there were no security features present to stop someone from accessing any of the data it was storing. Because vibe coding tools promise powerful results without the need for developer experience, there are probably a lot of people without experience who will use something like Bolt to create passion projects that may ask for information like ZIP code, email address, date of birth, or passwords without proper security.
Bigger models did not equal more secure code. Larger, newer AI models didn’t improve security. No major language was immune.
Security patterns degrade without explicit prompts. Unless guarded, models recreate legacy patterns or outdated practices found in older training data. AI lacks security context—it generates code that works but does not understand threat models, attack surfaces, or the security implications of architectural choices.
Security debt compounds technical debt. Unresolved vulnerabilities persist and multiply, creating compliance exposure for regulated industries and increasing incident risk.
This provides a brief overview. A comprehensive security deep-dive with specific vulnerability analysis is available in our dedicated security article. For mitigation strategies and quality gates for AI code, see our responsible AI framework.
These security and quality concerns haven’t gone unnoticed by industry authorities, as we explore throughout our complete guide for engineering leaders.
Thoughtworks Technology Radar placed “AI coding complacency” on “Hold” status—their strongest cautionary rating. The “Hold” status means the risks currently outweigh the benefits for most organisations and that uncritical adoption is dangerous.
Thoughtworks specifically warns against the complacency pattern—teams adopting AI coding tools without quality gates, review standards, or comprehension requirements. The rise of coding agents further amplifies these risks, since AI now generates larger change sets that are harder to review.
GitClear’s research found that duplicate code and code churn have risen more than expected, while refactoring activity in commit histories has dropped. Microsoft research on knowledge workers shows that AI-driven confidence often comes at the expense of thinking—a pattern observed as complacency sets in with prolonged use of coding assistants.
The emergence of “vibe coding”—where developers let AI generate code with minimal review—illustrates the growing trust of AI-generated outputs. Thoughtworks strongly cautions against using vibe coding for production code, though this approach can be appropriate for things like prototypes or other types of throw-away code.
The Thoughtworks assessment validates sceptical engineering leaders. It’s responsible risk management backed by an industry-respected authority.
This assessment aligns with independent findings from Kent Beck, Addy Osmani, and Chris Lattner—multiple authoritative voices converging on the same conclusion.
If you’re facing pressure to adopt AI coding tools, the Thoughtworks “Hold” provides an evidence-based rationale for cautious, measured adoption rather than wholesale embrace.
As with any system, speeding up one part of the workflow increases pressure on the others. Studies show that code quality can decline over time with prolonged AI assistant use. Thoughtworks teams are finding that using AI effectively in production requires renewed focus on code quality.
Thoughtworks recommends reinforcing established practices such as TDD and static analysis, and embedding them directly into coding workflows. For precise definitions of vibe coding versus augmented coding referenced by Thoughtworks, see our terminology guide.
The evidence from GitClear, CodeRabbit, Veracode, and Thoughtworks converges on one conclusion: unrestricted AI code generation creates measurable, compounding costs that threaten long-term system health.
There’s ample evidence these tools can accelerate development—especially for prototyping and greenfield projects—but studies show that code quality can decline over time. Data from Faros AI and Google’s DORA report show teams with high AI adoption merged 98% more PRs but saw review times balloon 91%.
PR size increased 154% on average. Code review became the new bottleneck.
Atlassian’s 2025 survey found the paradox in stark terms: 99% of AI-using developers reported saving 10+ hours per week, yet most reported no decrease in overall workload. The time saved writing code was consumed by organisational friction—more context switching, more coordination overhead, managing the higher volume of changes.
When you make a resource cheaper—code generation—consumption increases faster than efficiency improves, and total resource use goes up.
DORA’s 2025 report crystallised the reality: AI is an amplifier of your development practices. Good processes get better (high-performing teams saw 55-70% faster delivery). Bad processes get worse (accumulating debt at unprecedented speed).
You are accountable not just for shipping features but for the long-term health of the systems your organisation depends on. That responsibility demands caution with tools that degrade quality.
Smaller organisations face disproportionate risk. Smaller teams cannot absorb a 30-41% technical debt increase. Fewer senior engineers are available for review. Recovery from accumulated debt is harder.
The choice is between vibe coding (comprehension-free) and augmented coding (disciplined, quality-focused). Responsible leaders choose the approach that preserves understanding.
Kent Beck on the future: “LLM coding will split up engineers based on those who primarily liked coding and those who primarily liked building.” If you liked the act of writing code itself—the craft of it, the meditation of it—this transition might feel like loss. If you liked building things and code was the necessary means, this feels like liberation.
But the danger isn’t that the agent fails—it’s that it succeeds in the wrong direction while you stop checking the compass.
Effective patterns include agent-first drafts with tight iteration loops, declarative communication (spend 70% of effort on problem definition, 30% on execution), automated verification, deliberate learning versus production focus, and architectural hygiene.
The developers who thrive won’t be those who generate the most code—they’ll be those who know which code to generate, when to question the output, and how to maintain comprehension even as their hands leave the keyboard.
Use AI to accelerate learning, not skip it. Focus on fundamentals that matter more than ever: robust architecture, clean code, thorough tests, thoughtful UX.
Choosing cautious adoption represents a commitment to building software that lasts.
For a complete guide to navigating vibe coding as an engineering leader, see our vibe coding strategic overview for engineering leaders. For risk assessment and vulnerability analysis, read our security deep-dive on AI-generated code vulnerabilities. For implementation guidance and quality gates, explore our responsible AI development framework.
Vibe coding can work for throwaway prototypes, hackathon projects, and personal experiments where long-term maintenance isn’t a concern. The risks emerge when vibe-coded output enters production codebases that teams need to maintain, debug, and extend over months and years. The key distinction is whether the code carries ongoing maintenance responsibility.
It works in personal projects where you control everything, MVPs where “good enough” is actually good enough, startups in greenfield territory without legacy constraints, and teams small enough that comprehension debt stays manageable.
Vibe coding means generating code via AI prompts without deeply understanding the output—you focus on behaviour over quality. Augmented coding, defined by Kent Beck, means using AI as a tool while maintaining discipline around testing, code quality, architecture, and comprehension. The difference is whether you care about and understand the code, not whether you use AI.
GitClear’s analysis found significant declines in refactoring activity, increases in code duplication, and nearly doubled code churn. These metrics compound to an estimated 30-41% increase in technical debt, which translates to slower development velocity, more bugs, and higher incident rates over time.
Code review helps but faces significant challenges with AI-generated code. CodeRabbit found reviewers spend 91% more time on AI pull requests, and the volume of AI-generated code creates review fatigue that leads to rubber-stamping. Effective review requires comprehension—which is precisely what vibe coding bypasses.
Only 48% of developers consistently check AI-assisted code before committing it, even though 38% find that reviewing AI-generated logic actually requires more effort than reviewing human-written code.
AI models optimise for functional correctness, not security. They default to insecure but working implementations because their training data includes vast amounts of insecure code. Veracode found AI code contains 2.74x more vulnerabilities because models lack threat model awareness and cannot reason about attack surfaces the way security-conscious developers can.
No. “Hold” means adopt with extreme caution, not never use. Thoughtworks specifically warns against AI coding complacency—the pattern of adopting tools without quality gates, review standards, or comprehension requirements. Organisations can use AI coding tools responsibly through augmented coding practices that maintain human oversight.
Comprehension debt, coined by Jeremy Twei, describes the growing gap between code a team can review syntactically and code they actually understand architecturally. Unlike technical debt, which is visible in metrics like duplication and complexity, comprehension debt is hidden until an incident reveals that no one truly understands how something works.
The 70% Problem means AI-generated code appears mostly correct but the final 30% requires disproportionate effort. In practice, you might accept AI-generated authentication code that handles the happy path but spend 3 hours debugging edge cases that would have taken 1 hour to write manually. At scale, this erases the productivity gains AI tools promise.
Yes. Smaller organisations face disproportionate risk because they have fewer senior engineers to review AI code, less capacity to absorb a 30-41% technical debt increase, and smaller teams where each developer’s understanding matters more. Larger enterprises can dedicate specialised teams to code quality. Smaller organisations often cannot.
Key indicators include: refactoring rate (should not decline below 15-20% of activity), code duplication trends (rising duplication signals lack of abstraction), code churn (high write-delete cycles indicate instability), cognitive complexity scores (rising complexity means harder maintenance), and PR review time (increasing review burden suggests quality degradation).
Current limitations are architectural, not just training deficiencies. Context window constraints, lack of threat model awareness, and inability to reason about system-wide implications are fundamental to how LLMs work. While AI tools will improve incrementally, the need for human comprehension and quality oversight is unlikely to disappear. Augmented coding—human-AI collaboration with human understanding—remains the responsible approach.
The AI Productivity Paradox – Why Developers Feel Fast But Deliver SlowYour developers are excited about AI coding tools. They’re using Cursor, GitHub Copilot, or Claude Code. They tell you they’re faster. They feel more productive. The code is flowing.
But your DORA metrics haven’t budged. Deployment frequency is flat. Lead time for changes hasn’t improved. You’re starting to wonder what’s going on.
This phenomenon is part of a larger shift explored in our comprehensive guide to vibe coding and the death of craftsmanship. Here’s what’s actually happening: developers using AI tools believe they’re 24% faster, but rigorous controlled studies show they’re 19% slower. That’s a 43 percentage point gap between perception and reality.
The evidence is out there and it’s measurable. This article unpacks why this paradox exists by looking at the evidence from METR, GitClear, Faros AI, and CodeRabbit. We’ll cover three mechanisms: the productivity placebo effect, technical bottlenecks like the 70% problem and context rot, and system-level constraints explained by Amdahl’s Law.
By the end, you’ll understand why your teams are enthusiastic but delivery hasn’t improved, and what the research actually shows.
The AI productivity paradox is simple: developers using AI coding assistants genuinely believe they are faster, but controlled measurement reveals they complete tasks more slowly. There’s a 43 percentage point gap between perception and reality.
The numbers are specific. In the METR randomised controlled trial, developers expected AI to speed them up by 24%. After using the tools, they still believed AI had sped them up by 20%. But they were actually 19% slower.
Multiple independent studies show the same pattern: individual activity increases but delivery velocity stays flat.
Look at the adoption numbers. 84-90% of developers now use AI tools according to Stack Overflow’s 2025 survey. 41% of committed code is AI-generated according to GitClear’s analysis. Yet DORA metrics across 1,255 teams show no improvement according to Faros AI’s research.
So why should you care? Because your board sees competitor claims of doubled output. They’re asking why your DORA metrics tell a different story. The paradox exists at both the individual level—developers feel fast but measure slow—and at the organisational level—more individual output but flat delivery velocity.
We’re going to unpack three explanatory threads: psychological mechanisms that create the perception gap, technical bottlenecks that slow actual delivery, and systemic constraints that prevent individual gains from scaling.
Rigorous, independent research consistently shows that AI coding tools increase individual code output but fail to improve—and may worsen—actual task completion time and delivery speed. The strongest evidence comes from the METR randomised controlled trial showing a 19% slowdown.
Let’s talk about methodology. METR recruited 16 experienced developers from large open-source repositories—averaging 22,000+ stars and 1 million+ lines of code. These participants were experienced maintainers of major open-source projects, not students working on toy problems.
They gave them 246 real-world issues. Random assignment to AI-assisted and control groups. Tasks averaged two hours each. Developers were paid $150 per hour as compensation.
The tools? Cursor Pro with Claude 3.5 Sonnet—frontier models at the time of the study.
The finding: developers completed tasks 19% slower with AI tools, yet believed they were 20% faster.
This is the gold standard experimental design—randomised controlled trial. It provides causal evidence, not correlation. Random assignment eliminates selection bias. Real repositories versus synthetic benchmarks.
Now contrast this with vendor-funded research. Microsoft and Accenture studies claim 26-55% speedups. But they use controlled benchmarks with novice-friendly tasks rather than real-world development. The GitHub and Microsoft controlled experiment showed developers using Copilot finished an HTTP server task 55.8% faster—but the setup was closer to a benchmark exercise than day-to-day work. Gains were strongest for less experienced developers who leaned on AI for scaffolding. Our analysis of where AI tools show genuine productivity gains explores when these vendor research findings hold true.
Here’s how four major studies compare:
METR Study: 16 developers, 246 issues. Randomised controlled trial. Independent funding. Finding: 19% slower, though developers believed 20% faster. Tools: Cursor Pro with Claude 3.5 Sonnet.
GitClear Study: 211 million lines of code from Google, Microsoft, Meta, and enterprise C-corps. Longitudinal code analysis from 2020-2024. Independent funding. Finding: 4x code duplication increase, refactoring collapsed from 25% to under 10%. We explore how this technical debt accumulates and compounds over time in our analysis of the case against vibe coding.
Faros AI Study: Over 10,000 developers across 1,255 teams. Telemetry analysis. Independent funding. Finding: 21% more tasks completed but DORA metrics flat, 98% more PRs, 91% longer reviews.
CodeRabbit Study: 470 pull requests—320 AI-co-authored, 150 human-only. Automated review analysis. Independent funding. Finding: AI code has 1.7x more issues, 75% more logic errors, 3x readability problems. The full code quality degradation evidence shows why these issues matter for long-term maintainability.
The pattern is consistent. Vendor research uses controlled environments and claims gains. Independent research uses real-world development and finds the paradox.
Funding source affects methodology and conclusions. Synthetic benchmarks versus production code largely explains the divergent findings.
Developers feel faster because AI coding assistants trigger a productivity placebo effect. Rapid generation of code creates instant dopamine-driven feedback that feels like achievement, even when actual task completion takes longer.
Security researcher Marcus Hutchins has an accessible explanation: AI gives “a feeling of achievement without the heavy lifting”. The reward signal is disconnected from the outcome.
Here’s the psychological mechanism. Instant AI responses—autocomplete suggestions, generated code blocks—activate the brain’s reward system in ways that manual coding doesn’t. The immediate feedback loop from AI generating code instantaneously is satisfying and feels like a boost to productivity.
Activity feels like progress. Writing more lines of code, generating more pull requests, touching more files creates a subjective experience of high productivity. The problem is that dopamine rewards activity in the editor, not working code in production.
This explains why self-reported surveys consistently show positive AI sentiment—84% adoption, widespread enthusiasm—while objective measurement shows the opposite. Self-reports capture felt experience rather than objective outcomes.
Developers aren’t lying. They’re experiencing a well-documented cognitive bias. Developers might be trading some speed for ease—using Cursor may be so much more pleasant that developers don’t notice or mind that they’re slowed down.
Hutchins frames it for non-psychologists: LLMs inherently hijack the human brain’s reward system. LLMs give the same feeling of achievement one would get from doing the work themselves, but without any of the heavy lifting.
This is why developers genuinely feel faster despite measuring slower.
AI-assisted development delivers slower due to three compounding technical mechanisms: the 70% problem where AI code is almost right but requires costly debugging, context rot where AI output quality degrades in complex codebases, and the review bottleneck where senior engineers are overwhelmed with 98% more pull requests that take 91% longer to review.
The Stack Overflow survey shows 66% of developers report AI code is “almost right but not quite.” This is their primary AI frustration.
AI-generated code looks correct on first inspection but fails in edge cases, integration points, or complex business logic. Only 39% of Cursor generations were accepted in the METR study, with many still requiring reworking.
Debugging AI code requires understanding code you didn’t write—a cognitively expensive task. The time saved generating code is consumed, and often exceeded, by the time spent correcting it.
Another 45.2% of developers pointed to time spent debugging AI-generated code as their main frustration.
One METR study observation stands out: AI code that is “good enough” in other contexts wasn’t up to standards in high-quality open source projects.
LLMs degrade as conversation context windows accumulate irrelevant information from earlier prompts. Output quality gets worse the more context you add—the model starts pulling in irrelevant details from earlier prompts, and accuracy drops.
AI excels at boilerplate, scaffolding, and well-documented patterns. It fails at novel architecture, cross-cutting concerns, and domain-specific logic.
More context is not always better. In theory a bigger context window should help, but in practice it often distracts the model. The result is bloated or off-target code that looks right but doesn’t solve the problem you’re working on.
Longer AI sessions produce progressively worse code, creating a false economy of effort.
Faros AI telemetry shows a 98% increase in PR volume on high-AI-adoption teams.
PR size increases 154%, making each review harder and more time-consuming.
Review time increases 91%. Senior engineers become the constraint absorbing all productivity gains.
Bug rate increases 9% per developer, meaning reviews must be more thorough, not less.
The CodeRabbit analysis found AI-generated PRs contained 10.83 issues per PR compared to 6.45 for human-only PRs—approximately 1.7x more issues overall.
Break down the quality issues:
Logic and correctness issues were 75% more common in AI PRs.
Readability issues spiked more than 3x in AI contributions.
Error handling and exception-path gaps were nearly 2x more common.
Security issues were up to 2.74x higher, with prominent patterns around improper password handling and insecure object references.
Concurrency and dependency correctness saw approximately 2x increases.
Any correlation between AI adoption and key performance metrics evaporates at the company level. AI-driven coding gains evaporate when review bottlenecks, brittle testing, and slow release pipelines can’t match the new velocity.
Amdahl’s Law—the systems performance principle that a system’s speed is limited by its slowest component—explains precisely why a 10x speedup in code generation produces near-zero improvement in delivery velocity when code review, testing, and deployment remain unchanged.
The development pipeline has multiple sequential stages: code generation, code review, testing, integration, deployment.
AI dramatically accelerates only one stage—code generation—while leaving others unchanged or making them worse. Remember, review time increased 91%.
The mathematical reality: even if code generation becomes infinitely fast, the pipeline can only be as fast as review, testing, and deployment allow.
Think of it like a factory assembly line. One station speeds up dramatically, but the bottleneck station stays the same. The whole line can only move as fast as the slowest station.
Map the development pipeline stages:
Code generation: 10x faster—AI accelerated.
Code review: 91% slower—the constraint absorbs and reverses gains.
Testing: unchanged—no AI impact on test infrastructure.
Deployment: unchanged—CI/CD pipeline unchanged.
Integration can be difficult as AI might not understand nuances of the project’s architecture, dependencies, or coding standards.
Visualise this: individual developer velocity arrow pointing up, but delivery velocity arrow pointing sideways—flat.
This is why DORA metrics stay flat. They measure the full pipeline, not just code generation. Deployment frequency, lead time for changes, change failure rate, mean time to restore—these capture the whole system.
A system moves only as fast as its slowest link. Without lifecycle-wide modernisation, AI’s benefits are neutralised.
The Faros AI data confirms this. Individual metrics improved: 21% more tasks completed, 47% more context switches. Organisational metrics stagnated: deployment frequency unchanged, lead time unchanged, review time increased 91%.
Across overall throughput, DORA metrics, and quality KPIs, the gains observed in team behaviour don’t scale when aggregated. This suggests that downstream bottlenecks are absorbing the value created by AI tools.
The implication is straightforward. You need to invest in the constraint—review capacity, testing automation—rather than further accelerating the non-constrained step. Our framework for responsible AI-assisted development provides actionable guidance on how to capture individual AI productivity gains at the organizational level.
No AI collapses design discussions, sprint planning, meetings, or QA cycles. It doesn’t erase tech debt or magically handle system dependencies.
Amdahl’s Law explains why individual gains don’t scale, but what about the quality of the code being generated? GitClear’s analysis provides the answer.
GitClear’s longitudinal analysis of 211 million lines of code from 2020 to 2024—sourced from Google, Microsoft, Meta, and enterprise C-corps—reveals that AI-assisted development has caused code duplication to increase 4x, refactoring activity to collapse from 25% to under 10% of changes, and code churn to nearly double.
This is the largest empirical code quality study covering the AI adoption period. Population-level evidence rather than small-sample findings.
The three key degradation metrics:
Refactoring collapsed from 25% to under 10% of code changes.
Code duplication—cloning—increased 4x in volume. Lines classified as “copy/pasted” rose from 8.3% to 12.3% between 2021 and 2024.
Code churn nearly doubled.
Copy/paste code exceeded “moved” code—refactored code—for the first time in 2024. This violates the DRY principle at scale.
Why these metrics matter for long-term sustainability: duplication creates maintenance burden—changes must be replicated across multiple locations. Reduced refactoring means codebases ossify and become harder to modify. Increased churn means instability and rework.
Developers seem to view AI as a means to write more code, faster. Through the lens of “does more code get written?” common sense and research agree: resounding yes.
But to retain high project velocity over years, research suggests that a DRY—Don’t Repeat Yourself—modular approach to building is needed.
Copy/paste exceeding moved code for the first time is a structural shift in how code is being written. Developers are accepting AI-generated duplicates rather than abstracting and reusing.
These quality issues feed the review bottleneck and create downstream costs that offset any generation-stage time savings.
While GitClear examined code quality over time, Faros AI took a different approach—examining real-time telemetry across thousands of developers.
The Faros AI study of over 10,000 developers across 1,255 teams provides telemetry-based evidence that individual AI productivity gains—21% more tasks completed—don’t translate to delivery improvements, with DORA metrics remaining flat despite 98% more pull requests and dramatically increased review burden.
This is the largest organisational-level study of AI impact on software delivery. It uses objective telemetry rather than self-reported surveys.
Individual metrics improved: 21% more tasks completed, 47% more context switches indicating more parallel work.
Organisational metrics stagnated: deployment frequency unchanged, lead time unchanged, review time increased 91%.
Developers on high-AI-adoption teams touch 9% more tasks and 47% more pull requests per day.
AI adoption is consistently associated with a 9% increase in bugs per developer and a 154% increase in average PR size.
No measurable organisational impact from AI—across overall throughput, DORA metrics, and quality KPIs, the gains observed in team behaviour don’t scale when aggregated.
Downstream bottlenecks are absorbing the value created by AI tools, and inconsistent AI adoption patterns throughout the organisation are erasing team-level gains.
Look at the adoption patterns. AI adoption only recently reached critical mass—in most companies, widespread usage (greater than 60% weekly active users) only began in the last two to three quarters.
Usage remains uneven across teams, even where overall adoption appears strong.
Adoption skews toward less tenured engineers—usage is highest among engineers who are newer to the company.
AI usage remains surface-level. Across the dataset, most developers use only autocomplete features, with agentic and advanced modes largely untapped.
This suggests the paradox may deepen as adoption matures. As teams adopt more capable AI features that generate larger volumes of more complex code, the review bottleneck and quality issues are likely to intensify—unless you simultaneously invest in review capacity, quality gates, and testing infrastructure.
One more observation from Faros AI: developers using AI are writing more code and completing more tasks. They’re parallelising more workstreams. AI-augmented code is getting bigger and buggier, and shifting the bottleneck to review.
In most organisations, AI usage is still driven by bottom-up experimentation with no structure, training, overarching strategy, instrumentation, or best practice sharing.
Engineering leaders should frame the AI productivity paradox for boards by distinguishing between activity metrics—lines of code, pull requests—and outcome metrics like DORA: deployment frequency, lead time. Individual developer enthusiasm is real but delivery requires investment in the constraint—review capacity and quality gates—not further tool adoption. For comprehensive strategic guidance, see our vibe coding complete guide for engineering leaders.
Address the inevitable question: “Competitor X claims AI doubled their dev team output—why are we not seeing the same?”
Lead with this framework: “AI is working at individual level, but our pipeline has a bottleneck that absorbs the gains.”
Use DORA metrics as the objective, industry-standard measure. Boards understand deployment frequency and lead time.
Explain the vendor research versus independent research distinction clearly. Vendor research—Microsoft, Accenture claiming 26-55% gains—uses controlled benchmarks. Independent RCTs like METR measure real-world development and show 19% slower task completion.
Funding source affects methodology and conclusions. Synthetic benchmarks versus production code explains why claims diverge.
Individual activity—more code, more PRs—is not the same as outcomes: features delivered, customer value.
Present the investment case. Review capacity is the constraint requiring resources, not more AI tool licences. You need to invest in the constraint—review capacity, testing automation—rather than further accelerating the non-constrained step.
Reframe from “AI doesn’t work” to “AI exposes where our delivery system needs investment.” This is a constructive narrative for boards.
Smaller teams with fewer senior engineers hit the review bottleneck harder. When AI adoption significantly increases PR volume with limited reviewers, the pressure intensifies. SMBs also face greater board pressure to demonstrate ROI from AI investments, making the paradox politically uncomfortable as well as operationally damaging.
AI coding tools trigger a productivity placebo effect through dopamine-driven instant feedback. Rapid code generation and autocomplete suggestions activate reward mechanisms that create a genuine feeling of achievement. Marcus Hutchins explains: AI provides “a feeling of achievement without the heavy lifting.” Developers aren’t lying—it’s a well-documented cognitive bias where activity is perceived as progress regardless of actual outcomes.
The METR study is a randomised controlled trial—the gold standard of experimental design. 16 experienced open-source developers were randomly assigned AI-assisted or control conditions across 246 real-world issues. It matters because it provides causal evidence—not correlation—that AI tools slow task completion by 19%, and because participants believed they were 20% faster, quantifying the 43 percentage point perception-reality gap.
The gap combines two directional errors: developers believed they were 24% faster in the positive direction while actually being 19% slower in the negative direction. The total swing from perceived to actual is 24 plus 19 equals 43 percentage points. This reflects a fundamental disconnect between felt experience and measured reality.
DORA metrics measure the full delivery pipeline—deployment frequency, lead time, change failure rate, recovery time—not just code generation. AI accelerates only code generation while creating downstream bottlenecks: 98% more PRs to review, 91% longer review times, and 154% larger PRs. Per Amdahl’s Law, the pipeline can only move as fast as its slowest component—which AI has made slower.
The 70% problem describes the common experience—reported by 66% of developers in the Stack Overflow survey—where AI-generated code is “almost right but not quite.” The code compiles, looks correct, but fails in edge cases, integration points, or complex business logic. Debugging code you didn’t write is cognitively expensive, often consuming more time than writing it from scratch.
Amdahl’s Law states that a system’s overall speed is limited by its slowest component. In software delivery, AI accelerates code generation—perhaps 10x—but leaves review, testing, and deployment unchanged. Even with infinitely fast code generation, the pipeline cannot be faster than review allows—and review has become 91% slower due to the flood of AI-generated PRs.
Context rot is the degradation of AI output quality as conversation context windows accumulate information from earlier prompts. As sessions grow longer and more complex, the model’s ability to produce relevant, correct code diminishes. This explains why AI excels at isolated boilerplate tasks but degrades on complex architecture requiring deep codebase understanding.
METR used a randomised controlled trial with 16 developers measuring individual task completion time. Faros AI used telemetry analysis across 10,000+ developers and 1,255 teams measuring organisational delivery metrics—DORA. METR shows the paradox at individual level—feel fast, measure slow. Faros AI shows it at organisational level—more individual output, flat delivery velocity. Together, they confirm the paradox exists at both scales.
Vendor-funded studies—Microsoft, Accenture, GitHub—typically use controlled benchmarks with well-defined tasks, shorter timeframes, and sometimes less experienced participants. Independent studies—METR, Faros AI, GitClear—measure real-world development with experienced developers over longer periods. The methodology difference—synthetic benchmarks versus production code—largely explains the divergent findings.
Yes. Smaller teams—50-500 employees—typically have fewer senior engineers available for code review. When AI adoption creates 98% more PRs requiring 91% longer reviews, the constraint is more acute. There are simply fewer people to absorb the review burden. SMBs also face greater board pressure to demonstrate ROI from AI investments, making the paradox politically uncomfortable as well as operationally damaging.
Current evidence suggests it may. Faros AI found that most developers use only surface-level AI features—autocomplete—with agentic and advanced modes largely untapped. As teams adopt more capable AI features that generate larger volumes of more complex code, the review bottleneck and quality issues are likely to intensify—unless you simultaneously invest in review capacity, quality gates, and testing infrastructure.
Engineering leaders should measure DORA metrics—deployment frequency, lead time for changes, change failure rate, mean time to restore—as the outcome measures. Complement these with cycle time from commit to production, review queue depth and wait times, and the 70% completion cost—time spent fixing AI-generated code. Avoid vanity metrics like lines of code, PR counts, or self-reported productivity surveys. For detailed instrumentation and measurement frameworks, see our complete implementation guide.
What Is Vibe Coding and How Does It Differ from AI-Assisted EngineeringBack in February 2025, Andrej Karpathy coined the term “vibe coding” to describe his weekend coding experiments: tell an AI what you want, hit Accept All on whatever it spits out, and move on without reading the code. Within months, Collins Dictionary’s Word of the Year 2025 was “vibe coding”. But as the term spread its meaning got fuzzy. Now people use it for any AI-assisted coding, which makes it impossible for engineering leaders to have sensible policy conversations.
The question for CTOs isn’t whether AI coding tools are useful. It’s where you draw the line between acceptable experimentation and unacceptable risk. This article defines vibe coding precisely—using the originator’s actual words—then maps the spectrum from vibe coding through augmented coding to AI-assisted engineering. The goal is shared vocabulary so your team can discuss AI coding without talking past each other.
This is the foundational terminology that our comprehensive guide to vibe coding and engineering leadership builds on.
Andrej Karpathy—former OpenAI research director and Tesla AI lead—posted on X in February 2025 describing a new coding style he called “vibe coding”. His original definition: “fully giving in to the vibes, embracing exponentials, and forgetting that the code even exists”.
Karpathy’s context was specific: throwaway weekend projects and personal prototypes, not production systems. He described using Cursor Composer with Claude Sonnet, barely reading the diffs, hitting Accept All when things looked roughly right. “I ask for the dumbest things like ‘decrease the padding on the sidebar by half’ because I’m too lazy to find it. I ‘Accept All’ always, I don’t read the diffs anymore”.
When something broke, his approach was straightforward: copy the error message, paste it back to the AI, iterate until it worked. No understanding required. “Sometimes the LLMs can’t fix a bug so I just work around it or ask for random changes until it goes away”.
Karpathy explicitly qualified vibe coding as appropriate for “throwaway weekend projects” and code that’s “not serious” and “no one has to maintain”. Collins English Dictionary named “vibe coding” its Word of the Year for 2025. Merriam-Webster added an entry by March 2025.
The term’s spread from Karpathy’s narrow definition to a broad catch-all for any AI-assisted coding has created confusion in policy discussions. That confusion matters. If all AI-assisted development gets called “vibe coding” you can’t have nuanced conversations about where AI tools help and where they create risk.
AI-assisted engineering is the professional standard. Developers use LLMs to generate code but review, test, and understand all generated code before committing it to production. Simon Willison’s golden rule provides the clearest boundary: “I won’t commit code that I can’t explain exactly what it does”.
Vibe coding explicitly rejects this standard. The developer doesn’t read, understand, or review the generated code. The difference lies in the developer’s relationship to the output, not the tool itself. Same AI, fundamentally different practice.
Willison puts it clearly: “If an LLM wrote every line of your code, but you’ve reviewed, tested, and understood it all, that’s not vibe coding in my book—that’s using an LLM as a typing assistant”. The confusion arises because both use the same tools—Cursor, Copilot, Claude. The difference is entirely in developer discipline and process.
Willison notes that professional software developers need to create code that demonstrably works and can be understood by other humans. Vibe coding to a production codebase is “clearly risky”.
Lumping vibe coding with responsible AI use creates policy paralysis. You can’t write sensible guidelines when the same term covers both throwaway weekend experiments and production deployments.
Kent Beck—creator of Test-Driven Development and Extreme Programming—introduced “augmented coding” as a disciplined alternative. It embraces AI assistance while preserving craftsmanship. His core principle: the value system is the same as hand coding—tidy code that works—but the developer doesn’t type much of the code themselves.
Beck’s framework: “In vibe coding you don’t care about the code, just the behaviour of the system. In augmented coding you care about the code, its complexity, the tests, & their coverage”.
Augmented coding integrates AI into established engineering practices rather than abandoning them.
Test-Driven Development (TDD): Write failing tests first, use AI to generate implementations that pass, refactor while keeping tests green.
“Tidy First” principle: Separate structural changes (refactoring) from behavioural changes (features), with small frequent commits.
Code review remains standard: The developer understands and reviews everything the AI generates.
Test coverage requirements: Maintained or increased, not abandoned.
Beck described his practical experience using Augment Remote Agent for complex transliteration tasks in his BPlusTree3 project. He spent about four weeks building a performance-competitive, production-ready B+ Tree library in Rust and Python while travelling and recovering from a concussion.
Augmented coding sits in the middle ground. Not the “YOLO weekend project” of vibe coding, but also not the pre-AI traditional development workflow. Beck positions this as “beyond the vibes”—getting AI’s speed benefits without sacrificing the quality practices that make code maintainable.
The terminology represents a spectrum, not a binary choice. Most teams fall somewhere along it, often in different places for different tasks. Steve Krouse puts it well: “Vibe coding is on a spectrum of how much you understand the code. The more you understand, the less you are vibing”.
Here’s how they compare across five dimensions:
Review Process: Vibe coding has no review (Accept All always). Augmented coding uses peer review with TDD. AI-assisted engineering uses formal review with quality gates.
Understanding Level: Vibe coding means “forget the code exists”. Augmented coding requires partial understanding with strategic focus. AI-assisted engineering requires complete explanation capability.
Appropriate Use Cases: Vibe coding for throwaway projects and prototypes. Augmented coding for team projects with test coverage. AI-assisted engineering for production systems and customer-facing applications.
Testing Requirements: Vibe coding treats tests as optional. Augmented coding requires TDD (tests written first). AI-assisted engineering mandates automated plus manual testing.
Risk Level: Vibe coding carries high risk and is unsuitable for production. Augmented coding has medium risk depending on team discipline. AI-assisted engineering has low risk as professional standard.
Most engineering teams using AI tools are practising somewhere between augmented coding and AI-assisted engineering without having a name for it. The spectrum helps you identify where your team’s actual practice falls versus where it should fall for different project types.
A single team may use different approaches for different contexts. Vibe coding for internal hackathons, augmented coding for team projects, full AI-assisted engineering for production deployments. Having precise terminology enables policy discussions that acknowledge nuance rather than defaulting to blanket approval or prohibition.
Karpathy himself scoped vibe coding to legitimate use cases you shouldn’t prohibit.
Weekend hackathons and learning projects: Experimentation without production consequences helps developers learn new domains and AI tool capabilities.
Personal productivity tools (“software for one”): Non-programmers and developers building tools only they will use, where maintenance burden falls solely on the creator. Steve Krouse vibe coded apps to calculate weekly growth rates and propose to his fiancée. “I was able to vibe code these apps way faster than I could’ve built them, and it was a blast”.
Rapid prototyping for stakeholder feedback: Getting a visual prototype in front of decision-makers within hours rather than weeks, with no intention of shipping the prototype code.
Throwaway scripts and one-off utilities: Data migration scripts, one-time analysis tools, automation that runs once and is discarded.
Kevin Roose—New York Times journalist, not a professional coder—vibe coded several “software for one” applications successfully. This demonstrates the democratisation argument: non-programmers building functional software that would previously have required hiring a developer.
Simon Willison champions vibe coding for appropriate contexts while maintaining clear boundaries about when to stop: “I really don’t want to discourage people who are new to software from trying out vibe coding. The best way to learn anything is to build a project!” He’s published more than 80 experiments built with vibe coding.
The optimistic case is strongest when the code is personal, temporary, or explicitly disposable—when no one else has to maintain, secure, or depend on it. For a deeper exploration of when and why AI coding tools genuinely help, see the case for AI coding tools.
The risks of vibe coding emerge in specific, identifiable contexts where unreviewed code causes real damage.
Production systems and customer-facing applications: Unreviewed AI-generated code introduces security vulnerabilities, performance issues, and bugs that affect real users. In May 2025, Lovable, a Swedish vibe coding app, had security vulnerabilities in 170 out of 1,645 generated web applications allowing personal information to be accessed by anyone.
Security-critical or compliance-regulated environments: In December 2025, security researcher Etizaz Mohsin discovered a security flaw in the Orchids vibe coding platform. In July 2025, SaaStr founder documented Replit’s AI agent deleting a database despite explicit instructions not to make changes.
Complex codebases with multiple contributors: Vibe-coded additions create maintenance burden for the entire team. Code that the original developer doesn’t understand becomes unmaintainable when they leave or when requirements change.
Systems requiring long-term maintenance: Code you don’t understand today becomes code no one understands tomorrow. “When you vibe code, you are incurring tech debt as fast as the LLM can spit it out”, says Krouse.
The evidence goes beyond anecdotes. CodeRabbit analysis of 470 open-source GitHub pull requests found AI-co-authored code contained approximately 1.7 times more issues compared to human-written code. Security vulnerabilities were 2.74 times higher in AI-generated code. Logic errors were 75% more common. Readability issues spiked more than three times.
GitClear analysis of 211 million lines of code from 2020-2024 found code refactoring dropped from 25% of changed lines in 2021 to under 10% by 2024. Code duplication increased approximately four times in volume. Code churn nearly doubled.
Thoughtworks Technology Radar placed vibe coding on “Hold”—their strongest caution level. “As AI coding assistants gain traction, so does the body of data and research highlighting concerns about complacency with AI-generated code”.
For a comprehensive analysis of the long-term costs of vibe coding and technical debt implications, see our detailed examination of code quality degradation, security vulnerabilities, and why understanding matters for maintainability.
Precise terminology isn’t academic pedantry. It’s the foundation for clear policy, productive team discussions, and sound risk management.
Policy foundation: Without shared definitions, “we use AI coding tools” could mean anything from vibe coding prototypes to AI-assisted production engineering. Policy requires precision.
Team communication: When a developer says “I vibe coded this,” does the team know whether that means a throwaway prototype or a production feature? Shared vocabulary eliminates ambiguity.
Risk management: Understanding where on the spectrum your team operates lets you match risk controls to actual practice rather than applying blanket rules.
Addressing anxiety about being cautious: Feeling cautious about unreviewed AI-generated code isn’t being a Luddite—it’s stewardship. Kent Beck uses AI tools daily through augmented coding while maintaining strict quality standards. Leading voices in software engineering share this caution.
Cautious decision-making about AI tools is the same engineering judgement that prevents shipping untested code or skipping security reviews. It’s professionalism, not resistance to innovation.
The goal isn’t to prohibit AI tools but to ensure teams use them at the appropriate point on the spectrum for each context. This vocabulary enables the policy and framework discussions covered in later articles—the AI productivity paradox for the measurement evidence, and senior engineers adapting or resisting for team dynamics challenges.
Start with shared definitions, not opinions. This article’s terminology gives you a neutral starting point that doesn’t assume any position is right or wrong.
Here’s a framework for productive policy discussions:
Name the spectrum: Introduce the vibe coding / augmented coding / AI-assisted engineering terminology so everyone knows what they’re discussing.
Acknowledge enthusiasm: Developers experimenting with AI tools are showing initiative. The question is how to use AI responsibly.
Distinguish contexts: Help the team identify which projects or tasks warrant which level of rigour. Hackathons and prototypes are different from production deployments.
Set boundaries collaboratively: Teams that co-create their AI usage policies are more likely to follow them than teams given top-down mandates.
Avoid the common trap of framing this as “old school vs new school”. Kent Beck created TDD and embraces AI through augmented coding. Being pro-quality and pro-AI go hand in hand.
This is the beginning of a larger conversation. The framework for responsible AI-assisted development provides the detailed implementation playbook. For navigating team dynamics, see how senior engineers are adapting to AI coding tools. For the strategic framework tying all topics together, see our full strategic analysis of vibe coding and the death of craftsmanship.
Not necessarily. Vibe coding is defined by the developer’s behaviour—accepting AI-generated code without reviewing or understanding it. A developer using GitHub Copilot who reads, tests, and understands every suggestion is practising AI-assisted engineering, not vibe coding. The tool doesn’t determine the category—the developer’s level of review and understanding does.
Karpathy coined the specific term “vibe coding” in a February 2025 post on X, describing a practice he’d been experimenting with using Cursor Composer and Claude Sonnet. The underlying behaviour—accepting AI-generated code without close review—existed before the term, but Karpathy’s naming crystallised it into a concept the industry could discuss and debate.
Augmented coding is Kent Beck’s specific framework that integrates AI code generation with established engineering practices like Test-Driven Development, the Tidy First principle, and mandatory code review. It’s more structured than simply “being careful with AI”—it has concrete methodology including writing tests before AI generates code and separating structural from behavioural changes.
Yes, and this context-switching is exactly what the terminology spectrum is designed to support. Many developers vibe code personal projects and weekend experiments while maintaining strict review and testing standards for professional work. The key is your organisation’s policies clearly distinguish which contexts permit which approach.
Collins selected “vibe coding” because it captured a significant cultural moment: AI tools making software development accessible to non-programmers for the first time at scale. The selection reflected the term’s rapid spread from a single social media post in February 2025 to mainstream usage within months.
Thoughtworks Technology Radar uses “Hold” to indicate practices that organisations should approach with caution. For vibe coding, this means Thoughtworks’ experienced engineering advisors believe it carries material risk for production use cases and recommend organisations proceed carefully, particularly for code that will be maintained, deployed to users, or handles sensitive data.
Vibe coding is safer in specific contexts: personal projects, throwaway prototypes, hackathon experiments, and code running in sandboxed environments like Claude Artifacts. The risk increases with the code’s intended lifespan, audience, and access to sensitive data. For disposable code that only you will use, the risks are minimal. For anything else, augmented coding or AI-assisted engineering is more appropriate.
Ask three questions: Does the developer review and understand every line of AI-generated code before committing? Are there tests covering the AI-generated code? Can the developer explain what the code does and why? If the answer to any of these is “no” for production code, the team is closer to vibe coding than AI-assisted engineering on the spectrum.
No. Kent Beck—who created Test-Driven Development and Extreme Programming—uses AI tools daily through augmented coding while maintaining strict quality standards. Thoughtworks, one of the most respected engineering consultancies, placed vibe coding on “Hold”. Scepticism about unreviewed code is engineering judgement, not resistance to innovation.
No-code and low-code platforms provide constrained visual interfaces with pre-built components and guardrails. Vibe coding uses general-purpose LLMs to generate arbitrary code from natural language prompts with no inherent constraints or guardrails. Vibe coding can produce more flexible and powerful results but also introduces unbounded risk because the generated code has no structural limitations.
No credible source suggests this. Even vibe coding’s strongest proponents, including Karpathy himself, scope it to throwaway and personal projects. Production software engineering requires understanding, testing, security review, and maintainability—practices that vibe coding explicitly skips. AI tools are changing how engineers write code, but the need for engineering discipline remains.
Start by adopting the three-tier terminology (vibe coding, augmented coding, AI-assisted engineering) and mapping which approach is acceptable for each project type. Define clear expectations for code review, testing, and understanding of AI-generated code in production contexts. The framework for responsible AI-assisted development provides a detailed implementation playbook.