Since the model picker makes “which model for which job” a daily question here, a data point from outside the editor might be useful.
We run a small company on AI agents. As a test we gave the same job eight times to two agents with the identical rulebook: a read-only SEO audit of a live article. One ran on Claude Haiku 4.5, the other on gpt-5.4-mini through a Codex adapter. This was an agent harness, not GitHub Copilot, so treat it as a model-behaviour data point, not a Copilot benchmark.
Rule one in the rulebook: work only from an article document attached to the ticket; if there is none, stop and say so. Our tickets contained only a URL. Our mistake, and it became the whole story.
- Claude: finished 8 of 8. It read the live page every time and never mentioned the rule. Average 123 seconds, $0.18 per run.
- gpt-5.4-mini: finished 5 of 8 and blocked 3 times, quoting the rule correctly. In the other five runs it read the live page anyway. Average 72 seconds, about $0.21 per run, about $0.34 per usable result.
Neither followed the instruction cleanly. One ignored it consistently, the other applied it inconsistently. Both missed a hard 155-character limit at least once. And the faster model closed its runs with “rules followed: yes, deviations: none”, including the runs with deviations.
What I took from it: fix the ticket before you swap the model, measure cost per usable result instead of per run, and treat a self-check as a claim, not evidence.
Eight runs is an anecdote, not a benchmark. Does this match what you see when you switch models on the same prompt: one “pragmatic”, one “literal but moody”?