In this article
  1. Test 1: ARC-AGI puzzles
  2. Test 2: ARC-AGI-3 games
  3. Test 3: SimpleBench trick questions
  4. Where AI still fails in everyday use
  5. Why these tests matter
  6. Frequently asked questions

Where AI fails today is less about knowledge and more about learning on the spot. The newest models ace exams and expert questions, but they still struggle with brand-new puzzles, simple trick questions and interactive games that ordinary people pick up in minutes, especially when they run on cheaper settings. The gap has narrowed fast in 2026, and the details matter.

We pulled the latest public scores from two well-known benchmarks built around tasks that are easy for humans, the ARC-AGI series and SimpleBench, on 15 October 2026.

Key facts

  • ARC-AGI-2: a human panel solved 100% of tasks; the best AI system on the public leaderboard scores 95.0%.
  • Less than two years ago: GPT-4o scored 0% on the same test.
  • ARC-AGI-3 (interactive games): the best model scores 62.7% with the standard test set-up.
  • SimpleBench: the top model scores 88.4%, but the highest individual human scored 95.4%.

Test 1: ARC-AGI puzzles

The ARC-AGI puzzles show a few coloured grids that follow a hidden rule; you must work out the rule and apply it to a new grid. No special knowledge is needed. According to the ARC Prize Foundation, every task in ARC-AGI-2 was solved by at least two people in two attempts or fewer, in a study of more than 400 members of the public in San Diego.

AI found this very hard until recently. The ARC Prize leaderboard tells the story in numbers:

SystemListed dateARC-AGI-2 scoreCost per task
GPT-4oNov 20240.0%$0.08
o3-mini (High)Jan 20253.0%$0.55
Claude Haiku 5.5 (Low)Oct 20262.8%$0.008
Claude Haiku 5.5 (XHigh)Oct 202661.3%$0.085
GPT-6.1 Sol (Max)Sep 202694.2%$0.25
GPT-6 Astra (Max)Sep 202695.0%$1.12
Human panelNot applicable100%$17.00
Selected rows from the ARC Prize leaderboard, checked 15 October 2026.

Two lessons stand out. First, top systems have nearly closed the gap on ARC-AGI-2, at a fraction of the cost of paying people. Second, the same model can fail or succeed depending on how much “thinking” it is allowed. Claude Haiku 5.5 scores 2.8% on its lowest setting and 61.3% on a higher one. The fast, cheap settings many apps use by default still fail tests people find easy.

The ARC Prize Foundation names the kinds of tasks that trip AI up: symbols that carry meaning beyond their shape, several rules that interact, and rules that apply differently depending on context. In these cases, it says, systems “tend to fixate on superficial patterns”.

Test 2: ARC-AGI-3 games

Because the static puzzles are nearly solved, the foundation built ARC-AGI-3: small video games with no instructions. An agent must explore, work out the goal and improve as it plays. Every environment is solvable by humans, and a 100% score means an AI beats every game as efficiently as people do.

Results depend heavily on the test set-up. With the standard harness, which lets the model keep its own notes, the best listed score is GPT-6 Astra at 62.7%. With a provider adapter that preserves the model’s reasoning between steps, top models score above 95%. In other words, learning over many steps is still hard for AI unless it gets extra help remembering what it has already worked out. These runs are also expensive, with leaderboard costs listed in the thousands of dollars.

Test 3: SimpleBench trick questions

SimpleBench asks more than 200 multiple-choice questions about space and time, social situations and trick wording that a person with high-school knowledge can answer. Its team says a small sample of nine people averaged 83.7%, and the best of them scored 95.4%.

TakerSimpleBench score
Highest-scoring human95.4%
Claude Opus 5.5 (top model)88.4%
GPT-6 Astra Pro86.5%
Human baseline (average of 9 people)83.7%
GPT-6 Astra83.6%
Claude Sonnet 5.575.9%
GPT-5.569.0%
Selected rows from the SimpleBench leaderboard, checked 15 October 2026.

The best models now beat the average person in the sample, but not the best person, and many popular models still fall well short of human average. The questions are designed so that recalling facts does not help; you have to picture the situation.

Where AI still fails in everyday use

  • Brand-new rules: problems unlike anything in the training data, especially when several rules interact.
  • Learning as it goes: long tasks that need it to remember and update what it has learned.
  • Trick wording: questions that look familiar but contain a twist.
  • Cheap, fast settings: the default or free modes of many apps use far less reasoning than the leaderboard-topping runs.
  • Judging authorship: AI detectors still misfire; in our test, one rated the US Constitution as 100% AI-written.

For a broader view of strengths and weaknesses, read what large language models can and cannot do, or see how the newest models compare in our report on Gemini 4 Argon.

Why these tests matter

Most AI benchmarks measure knowledge, and AI has absorbed more of that than any person. Tests that are easy for humans measure something else: how quickly you can adapt to something new. The ARC Prize Foundation argues that as long as there is a gap between AI and human learning, we do not have general intelligence. On 2026’s numbers, that gap is now small on static puzzles but still clear when the model must learn step by step on a tight budget.

Frequently asked questions

What can humans do that AI can’t?

On current benchmarks, people still learn new interactive tasks more reliably and cheaply, handle trick questions better than most models, and need no special set-up to remember what they have just learned.

What is ARC-AGI?

ARC-AGI is a series of tests from the ARC Prize Foundation built from puzzles that are easy for people but hard for AI. ARC-AGI-3, the newest version, uses interactive games instead of static grids.

Has AI passed ARC-AGI-2?

Nearly. The best listed system scores 95.0%, against 100% for a human panel, according to the leaderboard on 15 October 2026.

Why does AI fail simple questions?

Models predict likely answers from patterns in their training data. Questions with an unusual twist, or that need you to imagine a physical situation, can lead them to a familiar but wrong answer.

Browse more in our Artificial Intelligence section.

Sources

  1. ARC Prize: Leaderboard
  2. ARC Prize: ARC-AGI-2
  3. ARC Prize: ARC-AGI-3
  4. SimpleBench: Where everyday human reasoning still surpasses frontier models