OpenAI Says Benchmark Used to Measure AI Coding Skill Is 'Contaminated'—Here's Why

In brief

• OpenAI argues that SWE-bench Verified no longer reflects real coding ability because the benchmark is allegedly contaminated.
• It is now pushing SWE-bench Pro as tougher replacement.
• Scores plunged from ~70% to ~23% on the newer benchmark,