#38 The harness matters

GPT-6 Astra was released last week. Every time a new model comes out, I find myself looking for the pelicans on a bike (bigger comparison here). There are more tests of course, but the ARC-AGI-3, shows that the harness matters greatly, check out this chart:

Two harnesses compared. I'm baffled by the accuracy difference, but the price difference makes me wonder even more.

It is clearly the combination of het model and the harness.

OpenAI will sell you Astra, but not the system that scored 98.6% on ARC-AGI-3 OpenAI's GPT-6 Astra, released on Thursday, scored higher and cost less with a different harness. It's the latest example of why the software around an AI model deserves as much attention as the model. thenewstack.io
2026-09-06
expand_less brightness_auto