Real-SWE 1.1
Benchmarking frontier AI models on private, real-world, enterprise codebases.
Read the blog and analysisLeaderboard
October 2026- Tied for rank 8Resolution rate: 39.20%, one standard error either side: 35.9% to 42.5%
- Tied for rank 10Resolution rate: 38.40%, one standard error either side: 35.8% to 41.0%
| Rank | Model | Resolution rate |
|---|---|---|
| 1 | 56.80%, one standard error either side: 53.5% to 60.1% | |
| 2 | 56.00%, one standard error either side: 54.4% to 57.6% | |
| 3 | 55.20%, one standard error either side: 52.5% to 57.9% | |
| 4 | 51.20%, one standard error either side: 49.0% to 53.4% | |
| 5 | 47.20%, one standard error either side: 44.8% to 49.6% | |
| 6 | 45.60%, one standard error either side: 42.5% to 48.7% | |
| 7 | 42.40%, one standard error either side: 39.9% to 44.9% | |
| Tied for 8 | 39.20%, one standard error either side: 35.9% to 42.5% | |
| Tied for 8 | 39.20%, one standard error either side: 35.8% to 42.6% | |
| Tied for 10 | 38.40%, one standard error either side: 35.9% to 40.9% | |
| Tied for 10 | 38.40%, one standard error either side: 35.8% to 41.0% | |
| 12 | 33.60%, one standard error either side: 30.9% to 36.3% | |
| 13 | 32.00%, one standard error either side: 28.9% to 35.1% | |
| 14 | 28.00%, one standard error either side: 25.3% to 30.7% | |
| 15 | 26.40%, one standard error either side: 23.9% to 28.9% | |
| 16 | 24.00%, one standard error either side: 21.2% to 26.8% | |
| 17 | 10.40%, one standard error either side: 8.5% to 12.3% | |
| 18 | 8.00%, one standard error either side: 6.5% to 9.5% | |
| 19 | 5.60%, one standard error either side: 4.1% to 7.1% | |
| 20 | 4.80%, one standard error either side: 3.4% to 6.2% |
We use native harnesses to reflect how enterprise engineers work in practice, evaluating model-and-harness combinations rather than models in isolation.
Pareto curve
Resolution rate by estimated cost per rollout
| Model | Resolution rate | Estimated cost per rollout | |
|---|---|---|---|
| 1 | 56.80% | $13.95 | |
| 2 | 56.00% | $3.78 | |
| 3 | 55.20% | $7.16 | |
| 4 | 51.20% | $10.09 | |
| 5 | 47.20% | $0.79 | |
| 6 | 45.60% | $14.24 | |
| 7 | 42.40% | $3.76 | |
| 8 | 39.20% | $4.66 | |
| 9 | 39.20% | $2.03 | |
| 10 | 38.40% | $0.56 | |
| 11 | 38.40% | $0.19 | |
| 12 | 33.60% | $1.13 | |
| 13 | 32.00% | $4.70 | |
| 14 | 28.00% | $2.66 | |
| 15 | 26.40% | $3.35 | |
| 16 | 24.00% | $7.06 | |
| 17 | 10.40% | $0.08 | |
| 18 | 8.00% | $5.13 | |
| 19 | 5.60% | $3.03 | |
| 20 | 4.80% | $1.39 | |
About the benchmark
Private codebases
Agents must navigate proprietary systems whose code and solutions aren’t available on the public internet.
Work with business consequences
Getting billing right, calculating taxes, migrating customers. Changes that affect how a business runs, often across multiple services.
Company-specific complexity
Every company has its own rules and ways of writing code. Agents have to understand those conventions and make changes that work with what’s already there.