In the new era of agentic software development, vanity metrics don’t cut it anymore.
Counting tokens, prompts, or “AI commits” might feel like progress, but none of them measure what truly matters: shipped, functional code.
That’s why a growing number of AI engineering teams are standardizing on one north-star metric: merged pull requests per agent-hour.
Why This Metric Matters
This simple ratio — the number of successfully merged PRs divided by total AI agent runtime hours — captures the only output that counts: code that survives review and CI to land in main.
It aligns everyone on the same question:
“How much working code are our AI systems actually delivering per unit of compute time?”
By focusing on outcomes instead of activity, the metric keeps teams grounded in real productivity rather than synthetic gains.
Mastering GitHub Copilot, Vol. 2: Advanced Workflows, Enterprise Integration & The Future of AI Development
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
The Formula
Merged PRs per agent-hour = Valid merged PRs / Total agent runtime hours
Key Definitions
- Valid merged PRs: Non-draft pull requests that merged into
main, passed CI, earned at least one human approval, and weren’t reverted within 7 days. - Agent runtime hours: The wall-clock duration across all AI agent runs (planning, coding, testing, reviewing) contributing to those PRs.
Optional filters tighten quality:
- Exclude dependency bumps or chores unless they include test updates.
- Require ≥15 lines of code changed or ≥2 files touched.
AI-Powered Testing and Code Review Automation for Developers: Build Faster CI/CD Pipelines, Crush Flaky Tests, and Ship High-Quality Code with Confidence
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
Why It’s Hard to Game
Unlike prompt counts or LOC, this metric resists inflation:
- No credit for broken code: PRs reverted within a week don’t count.
- No credit for trivial tasks: Small or scripted edits are excluded.
- No hiding inefficiency: Long-running agents with few merges lower the score directly.
It’s the closest you can get to a truth serum for AI-assisted engineering.
Weekly Productivity Planner – 8.5" x 11" Dashboard Desk Notepad Has 6 Focus Areas to List Tasks for Goals, Projects, Clients, Academic or Meal-Organize Your Daily Work Efficiently, 54 Weeks, Green
BOOST YOUR PRODUCTIVITY – This undated weekly productivity planner notepad focus on the important work and get organized….
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
What It Reveals
- Efficiency: How effectively your AI stack converts runtime into merged work.
- Model performance: Comparing variants (e.g., GPT-5 vs Claude vs local SLMs) on equal footing.
- Prompt pack quality: Whether new task flows actually ship more PRs, not just produce more code.
- Human-AI synergy: Whether developer-in-the-loop patterns accelerate or slow the merge rate.
A consistently rising merged-per-hour metric means the system is learning — both human and machine sides.
Securing the CI/CD Pipeline: Best Practices for DevSecOps
As an affiliate, we earn on qualifying purchases.
As an affiliate, we earn on qualifying purchases.
How to Implement It in Practice
- Tag every agent run with start/stop timestamps, model ID, repo, and PR number.
- Pull PR data from GitHub/GitLab APIs:
merged_at, CI status, labels, approvals, and reverts. - Filter valid PRs using the quality gates above.
- Aggregate runtime per PR (sum if multiple runs contributed).
- Compute and visualize:
- Top-line metric (daily/weekly)
- 7-day revert rate
- Time-to-merge percentiles (p50/p90)
- Slice results by model, repo, and prompt pack to surface what’s driving success.
- Add guardrails: If revert rate exceeds 5% or merges drop >20% week-over-week, investigate before scaling experiments.
A Minimal Dashboard View
| Metric | Definition | Goal |
|---|---|---|
| Merged PRs / Agent-Hour | Primary efficiency signal | ↑ Over Time |
| 7-Day Revert Rate | Stability and code quality | < 5% |
| CI Pass-on-First-Try | Reliability | > 90% |
| Median Time-to-Merge | Flow speed | ↓ Over Time |
The Power of a Single Ratio
By grounding progress in shipped code, merged PRs per agent-hour becomes a universal benchmark across teams, tools, and models. It’s transparent, portable, and brutally honest — everything a scaling metric should be.
In the coming wave of agentic AI development, this ratio may become what “click-through rate” was to early web advertising: the one number that reveals whether your automation actually works.
Bottom line:
If your AI engineering system can raise its merged PRs per agent-hour without sacrificing quality, you’re not just building faster — you’re building smarter.
