I wanted to see how much the coding tool matters when the model stays the same.
I ran gpt-5.6-sol at medium reasoning on 225 Aider Polyglot tasks using Codex, Claude Code, Cursor, Pi, and OpenCode.
Results
| Tool | Passed | Rate |
|---|---|---|
| Codex | 183 / 219 | 83.6% |
| Pi | 168 / 219 | 76.7% |
| Cursor | 167 / 219 | 76.3% |
| Claude Code | 163 / 219 | 74.4% |
| OpenCode | 163 / 219 | 74.4% |
Codex did best. The gap between Codex and each of the other tools was statistically significant. The other four were close enough that I would not rank them from this run.
The denominator is 219 because six tasks already passed before the model changed anything.
Setup
- One attempt per task
- 15-minute limit
- The same model and reasoning level
- The tests included with each task
Claude Code used an OpenAI-compatible proxy, so this does not say anything about Claude models. Cursor used its SDK with a community bridge, not Cursor's hosted agent.
Network access
This was not a closed-book test.
I turned off the built-in web tools, but the agents could still use the shell to reach the public internet. I found 198 retrieval commands in 52 trials.
| Tool | Trials with retrievals |
|---|---|
| Codex | 24 |
| Pi | 10 |
| Claude Code | 9 |
| Cursor | 7 |
| OpenCode | 2 |
I also removed every task where any tool tried to fetch something. That left 187 tasks. Codex still came first at 82.9%, followed by Cursor at 79.7%, Pi at 79.1%, OpenCode at 78.6%, and Claude Code at 77.0%.
That second result is only a check. The full open-network result above is the main one.
Next
The tool around the model clearly changes the result. The next run should block outbound traffic completely.