#38 The harness matters

GPT-6 Astra was released last week. Every time a new model comes out, I find myself looking for the pelicans on a bike (bigger comparison for ChatGPT here). The ARC-AGI-3 test sparked my interest, as it shows that the harness matters greatly!

Two harnesses compared. I'm baffled by the accuracy difference, but the price difference makes me wonder even more.

This chart makes one thing clear: results depend on the combination of the model and the harness. Check the cost difference 🤯.

Next question: how can we get it?

OpenAI will sell you Astra, but not the system that scored 98.6% on ARC-AGI-3 OpenAI's GPT-6 Astra, released on Thursday, scored higher and cost less with a different harness. It's the latest example of why the software around an AI model deserves as much attention as the model. thenewstack.io

2026-09-06 · #llm