In the new era of agentic software development, vanity metrics don’t cut it anymore.
Counting tokens, prompts, or “AI commits” might feel like progress, but none of them measure what truly matters: shipped, functional code.
That’s why a growing number of AI engineering teams are standardizing on one north-star metric: merged pull requests per agent-hour.
Why This Metric Matters
This simple ratio — the number of successfully merged PRs divided by total AI agent runtime hours — captures the only output that counts: code that survives review and CI to land in main.
It aligns everyone on the same question:
“How much working code are our AI systems actually delivering per unit of compute time?”
By focusing on outcomes instead of activity, the metric keeps teams grounded in real productivity rather than synthetic gains.
Top picks for "single metric defin"
Open Amazon search results for this keyword.
As an affiliate, we earn on qualifying purchases.
The Formula
Merged PRs per agent-hour = Valid merged PRs / Total agent runtime hours
Key Definitions
- Valid merged PRs: Non-draft pull requests that merged into
main, passed CI, earned at least one human approval, and weren’t reverted within 7 days. - Agent runtime hours: The wall-clock duration across all AI agent runs (planning, coding, testing, reviewing) contributing to those PRs.
Optional filters tighten quality:
- Exclude dependency bumps or chores unless they include test updates.
- Require ≥15 lines of code changed or ≥2 files touched.
Why It’s Hard to Game
Unlike prompt counts or LOC, this metric resists inflation:
- No credit for broken code: PRs reverted within a week don’t count.
- No credit for trivial tasks: Small or scripted edits are excluded.
- No hiding inefficiency: Long-running agents with few merges lower the score directly.
It’s the closest you can get to a truth serum for AI-assisted engineering.
What It Reveals
- Efficiency: How effectively your AI stack converts runtime into merged work.
- Model performance: Comparing variants (e.g., GPT-5 vs Claude vs local SLMs) on equal footing.
- Prompt pack quality: Whether new task flows actually ship more PRs, not just produce more code.
- Human-AI synergy: Whether developer-in-the-loop patterns accelerate or slow the merge rate.
A consistently rising merged-per-hour metric means the system is learning — both human and machine sides.
How to Implement It in Practice
- Tag every agent run with start/stop timestamps, model ID, repo, and PR number.
- Pull PR data from GitHub/GitLab APIs:
merged_at, CI status, labels, approvals, and reverts. - Filter valid PRs using the quality gates above.
- Aggregate runtime per PR (sum if multiple runs contributed).
- Compute and visualize:
- Top-line metric (daily/weekly)
- 7-day revert rate
- Time-to-merge percentiles (p50/p90)
- Slice results by model, repo, and prompt pack to surface what’s driving success.
- Add guardrails: If revert rate exceeds 5% or merges drop >20% week-over-week, investigate before scaling experiments.
A Minimal Dashboard View
| Metric | Definition | Goal |
|---|---|---|
| Merged PRs / Agent-Hour | Primary efficiency signal | ↑ Over Time |
| 7-Day Revert Rate | Stability and code quality | < 5% |
| CI Pass-on-First-Try | Reliability | > 90% |
| Median Time-to-Merge | Flow speed | ↓ Over Time |
The Power of a Single Ratio
By grounding progress in shipped code, merged PRs per agent-hour becomes a universal benchmark across teams, tools, and models. It’s transparent, portable, and brutally honest — everything a scaling metric should be.
In the coming wave of agentic AI development, this ratio may become what “click-through rate” was to early web advertising: the one number that reveals whether your automation actually works.
Bottom line:
If your AI engineering system can raise its merged PRs per agent-hour without sacrificing quality, you’re not just building faster — you’re building smarter.
