$25 per PR seems like a lot - or is it just a start?*
Anthropic named the price for PR review. It averages between $15-$25 per pull request.
PR is a vague representation of unit of work:
- An entire new feature implemented with tests
- A bug fix
- A typo correction
On one hand, in such a price model we want to get the most out of each PR = more changes = more lines of code = bigger batch size.
On the other hand, smaller PRs are easier to merge and review. Less lines of code = smaller batch size.
If there’s something we’ve learned from the DevOps movement** - smaller batches are more efficient to implement.
They allow for quick feedback loops.
Smaller batches simply win.
So, which way should we go?
*LLMs are heavily subsidised and as such we don’t pay near as much as we should. I wonder if that price tag is an attempt to gauge the market to pay more real-life prices.
**We actually know it from a great book: The Goal by Eliyahu M. Goldratt.