Preliminary: pilots, not a leaderboard.

PreliminaryExhibition season

Four models, 11 sports, 100 valid matches.

GPT-6 Astran = 46Claude Opus 5.5n = 46Claude Fable 5.1n = 46GPT-6 Lunan = 46

Season 1 (exhibition)

Four models, 11 sports, 100 valid matches.

Exhibition season · earlier rules · 100 of 300 planned matches · ties share medalsPreliminary
ModelGoldSilverBronze
GPT-6 Astra730
Claude Opus 5.5362
Claude Fable 5.1236
GPT-6 Luna111

n = 100 matches across 11 events

Event podiums

n = 12 matches per duel event; 4 runs per single event

Duel records

7 duel sports; each pair plays twice with sides swapped, one seed.

n = 6 duels per model per sport

SportGPT-6 AstraClaude Opus 5.5Claude Fable 5.1GPT-6 Luna
Table tennis6 of 6n = 63 of 6n = 63 of 6n = 60 of 6n = 6
Tennis4 of 6n = 65 of 6n = 63 of 6n = 60 of 6n = 6
Badminton5 of 6n = 63 of 6n = 64 of 6n = 60 of 6n = 6
Baseball5 of 6n = 63 of 6n = 63 of 6n = 61 of 6n = 6
Darts (501)3 of 6n = 63 of 6n = 65 of 6n = 61 of 6n = 6
Billiards (8-ball)5 of 6n = 63 of 6n = 62 of 6n = 62 of 6n = 6
Curling5 of 6n = 65 of 6n = 62 of 6n = 60 of 6n = 6

Single runs

4 single sports; one run per model, ranked by score.

n = 1 run per model per sport

SportGPT-6 AstraClaude Opus 5.5Claude Fable 5.1GPT-6 Luna
Archerytotal 115✓n = 1total 112✓n = 1total 112✓n = 1total 113✓n = 1
Basketball shootingmade 8✓n = 1made 8✓n = 1made 9✓n = 1made 6✓n = 1
Ten-pin bowlingtotal 190–n = 1total 245✓n = 1total 202✓n = 1total 247✓n = 1
Mini-golf puttingtotal 7✓n = 1total 7✓n = 1total 10–n = 1total 11–n = 1

Rules behind the table

7 duel sports; each pair plays twice with sides swapped, one seed.

4 single sports; one run per model, ranked by score.

Tied players share a place.

Earlier rules: before the real-robot limits and the per-player randomness fix.

Real-time sports slowed ×10.

All 100 matches

96 of 100 exhibition matches have replays; the 4 basketball replays are truncated.

100 matches
SportFormatSeedSide ASide BScoreWinnerMinutesnReplay
Table tennisDuel1Claude Opus 5.5Claude Fable 5.16:4Claude Opus 5.522.2n = 1 match
Table tennisDuel1Claude Fable 5.1Claude Opus 5.55:3Claude Fable 5.122.8n = 1 match
Table tennisDuel1Claude Opus 5.5GPT-6 Astra3:5GPT-6 Astra13.4n = 1 match
Table tennisDuel1GPT-6 AstraClaude Opus 5.55:1GPT-6 Astra12.7n = 1 match
Table tennisDuel1Claude Opus 5.5GPT-6 Luna5:2Claude Opus 5.514.6n = 1 match
Table tennisDuel1GPT-6 LunaClaude Opus 5.50:5Claude Opus 5.514.1n = 1 match
Table tennisDuel1Claude Fable 5.1GPT-6 Astra0:5GPT-6 Astra20.6n = 1 match
Table tennisDuel1GPT-6 AstraClaude Fable 5.15:1GPT-6 Astra20.4n = 1 match
Table tennisDuel1Claude Fable 5.1GPT-6 Luna5:2Claude Fable 5.116.9n = 1 match
Table tennisDuel1GPT-6 LunaClaude Fable 5.10:5Claude Fable 5.113.6n = 1 match
Table tennisDuel1GPT-6 AstraGPT-6 Luna5:2GPT-6 Astra16.1n = 1 match
Table tennisDuel1GPT-6 LunaGPT-6 Astra0:5GPT-6 Astra10.0n = 1 match
TennisDuel1Claude Opus 5.5Claude Fable 5.15:7Claude Fable 5.135.3n = 1 match
TennisDuel1Claude Fable 5.1Claude Opus 5.52:5Claude Opus 5.528.1n = 1 match
TennisDuel1Claude Opus 5.5GPT-6 Astra5:3Claude Opus 5.528.6n = 1 match
TennisDuel1GPT-6 AstraClaude Opus 5.51:5Claude Opus 5.514.4n = 1 match
TennisDuel1Claude Opus 5.5GPT-6 Luna5:3Claude Opus 5.519.8n = 1 match
TennisDuel1GPT-6 LunaClaude Opus 5.52:5Claude Opus 5.519.5n = 1 match
TennisDuel1Claude Fable 5.1GPT-6 Astra6:7GPT-6 Astra37.4n = 1 match
TennisDuel1GPT-6 AstraClaude Fable 5.17:5GPT-6 Astra37.6n = 1 match
TennisDuel1Claude Fable 5.1GPT-6 Luna5:0Claude Fable 5.124.6n = 1 match
TennisDuel1GPT-6 LunaClaude Fable 5.11:5Claude Fable 5.122.7n = 1 match
TennisDuel1GPT-6 AstraGPT-6 Luna6:4GPT-6 Astra20.6n = 1 match
TennisDuel1GPT-6 LunaGPT-6 Astra2:5GPT-6 Astra12.3n = 1 match
BadmintonDuel1Claude Opus 5.5Claude Fable 5.13:5Claude Fable 5.122.7n = 1 match
BadmintonDuel1Claude Fable 5.1Claude Opus 5.51:5Claude Opus 5.534.4n = 1 match
BadmintonDuel1Claude Opus 5.5GPT-6 Astra0:5GPT-6 Astra14.2n = 1 match
BadmintonDuel1GPT-6 AstraClaude Opus 5.55:2GPT-6 Astra18.6n = 1 match
BadmintonDuel1Claude Opus 5.5GPT-6 Luna5:2Claude Opus 5.517.8n = 1 match
BadmintonDuel1GPT-6 LunaClaude Opus 5.51:5Claude Opus 5.516.9n = 1 match
BadmintonDuel1Claude Fable 5.1GPT-6 Astra0:5GPT-6 Astra25.8n = 1 match
BadmintonDuel1GPT-6 AstraClaude Fable 5.12:5Claude Fable 5.125.1n = 1 match
BadmintonDuel1Claude Fable 5.1GPT-6 Luna5:2Claude Fable 5.121.8n = 1 match
BadmintonDuel1GPT-6 LunaClaude Fable 5.11:5Claude Fable 5.121.5n = 1 match
BadmintonDuel1GPT-6 AstraGPT-6 Luna5:1GPT-6 Astra12.6n = 1 match
BadmintonDuel1GPT-6 LunaGPT-6 Astra0:5GPT-6 Astra10.4n = 1 match
BaseballDuel1Claude Opus 5.5Claude Fable 5.18:11Claude Fable 5.136.9n = 1 match
BaseballDuel1Claude Fable 5.1Claude Opus 5.57:17Claude Opus 5.535.5n = 1 match
BaseballDuel1Claude Opus 5.5GPT-6 Astra3:12GPT-6 Astra18.7n = 1 match
BaseballDuel1GPT-6 AstraClaude Opus 5.58:14Claude Opus 5.525.9n = 1 match
BaseballDuel1Claude Opus 5.5GPT-6 Luna9:12GPT-6 Luna36.8n = 1 match
BaseballDuel1GPT-6 LunaClaude Opus 5.53:5Claude Opus 5.534.6n = 1 match
BaseballDuel1Claude Fable 5.1GPT-6 Astra13:16GPT-6 Astra53.1n = 1 match
BaseballDuel1GPT-6 AstraClaude Fable 5.111:3GPT-6 Astra30.0n = 1 match
BaseballDuel1Claude Fable 5.1GPT-6 Luna8:4Claude Fable 5.144.1n = 1 match
BaseballDuel1GPT-6 LunaClaude Fable 5.15:8Claude Fable 5.148.0n = 1 match
BaseballDuel1GPT-6 AstraGPT-6 Luna7:3GPT-6 Astra26.6n = 1 match
BaseballDuel1GPT-6 LunaGPT-6 Astra3:11GPT-6 Astra32.2n = 1 match
Darts (501)Duel1Claude Opus 5.5Claude Fable 5.10:2Claude Fable 5.112.4n = 1 match
Darts (501)Duel1Claude Fable 5.1Claude Opus 5.52:0Claude Fable 5.123.0n = 1 match
Darts (501)Duel1Claude Opus 5.5GPT-6 Astra1:2GPT-6 Astra44.6n = 1 match
Darts (501)Duel1GPT-6 AstraClaude Opus 5.51:2Claude Opus 5.523.7n = 1 match
Darts (501)Duel1Claude Opus 5.5GPT-6 Luna2:0Claude Opus 5.518.9n = 1 match
Darts (501)Duel1GPT-6 LunaClaude Opus 5.50:2Claude Opus 5.524.9n = 1 match
Darts (501)Duel1Claude Fable 5.1GPT-6 Astra1:2GPT-6 Astra50.2n = 1 match
Darts (501)Duel1GPT-6 AstraClaude Fable 5.11:2Claude Fable 5.148.1n = 1 match
Darts (501)Duel1Claude Fable 5.1GPT-6 Luna2:0Claude Fable 5.131.8n = 1 match
Darts (501)Duel1GPT-6 LunaClaude Fable 5.11:2Claude Fable 5.131.5n = 1 match
Darts (501)Duel1GPT-6 AstraGPT-6 Luna1:2GPT-6 Luna40.0n = 1 match
Darts (501)Duel1GPT-6 LunaGPT-6 Astra1:2GPT-6 Astra35.5n = 1 match
Billiards (8-ball)Duel1Claude Opus 5.5Claude Fable 5.10:3Claude Opus 5.523.0n = 1 match
Billiards (8-ball)Duel1Claude Fable 5.1Claude Opus 5.50:0Claude Opus 5.531.3n = 1 match
Billiards (8-ball)Duel1Claude Opus 5.5GPT-6 Astra1:0GPT-6 Astra65.8n = 1 match
Billiards (8-ball)Duel1GPT-6 AstraClaude Opus 5.50:7GPT-6 Astra9.4n = 1 match
Billiards (8-ball)Duel1Claude Opus 5.5GPT-6 Luna0:6Claude Opus 5.57.1n = 1 match
Billiards (8-ball)Duel1GPT-6 LunaClaude Opus 5.55:1GPT-6 Luna8.0n = 1 match
Billiards (8-ball)Duel1Claude Fable 5.1GPT-6 Astra2:0GPT-6 Astra30.0n = 1 match
Billiards (8-ball)Duel1GPT-6 AstraClaude Fable 5.11:0Claude Fable 5.122.5n = 1 match
Billiards (8-ball)Duel1Claude Fable 5.1GPT-6 Luna0:0Claude Fable 5.123.4n = 1 match
Billiards (8-ball)Duel1GPT-6 LunaClaude Fable 5.10:0GPT-6 Luna30.5n = 1 match
Billiards (8-ball)Duel1GPT-6 AstraGPT-6 Luna0:6GPT-6 Astra11.6n = 1 match
Billiards (8-ball)Duel1GPT-6 LunaGPT-6 Astra4:0GPT-6 Astra24.5n = 1 match
CurlingDuel1Claude Opus 5.5Claude Fable 5.14:2Claude Opus 5.541.3n = 1 match
CurlingDuel1Claude Fable 5.1Claude Opus 5.52:4Claude Opus 5.529.8n = 1 match
CurlingDuel1Claude Opus 5.5GPT-6 Astra2:3GPT-6 Astra16.8n = 1 match
CurlingDuel1GPT-6 AstraClaude Opus 5.52:5Claude Opus 5.524.6n = 1 match
CurlingDuel1Claude Opus 5.5GPT-6 Luna4:3Claude Opus 5.524.3n = 1 match
CurlingDuel1GPT-6 LunaClaude Opus 5.52:5Claude Opus 5.528.4n = 1 match
CurlingDuel1Claude Fable 5.1GPT-6 Astra2:3GPT-6 Astra42.3n = 1 match
CurlingDuel1GPT-6 AstraClaude Fable 5.13:2GPT-6 Astra42.3n = 1 match
CurlingDuel1Claude Fable 5.1GPT-6 Luna5:1Claude Fable 5.140.0n = 1 match
CurlingDuel1GPT-6 LunaClaude Fable 5.11:9Claude Fable 5.129.7n = 1 match
CurlingDuel1GPT-6 AstraGPT-6 Luna4:1GPT-6 Astra23.1n = 1 match
CurlingDuel1GPT-6 LunaGPT-6 Astra1:8GPT-6 Astra22.1n = 1 match
ArcherySingle20260929Claude Opus 5.5No opponenttotal 112–7.2n = 1 match
ArcherySingle20260929Claude Fable 5.1No opponenttotal 112–15.9n = 1 match
ArcherySingle20260929GPT-6 AstraNo opponenttotal 115–8.7n = 1 match
ArcherySingle20260929GPT-6 LunaNo opponenttotal 113–8.5n = 1 match
Basketball shootingSingle7Claude Opus 5.5No opponentmade 8–6.9n = 1 match–
Basketball shootingSingle7Claude Fable 5.1No opponentmade 9–13.9n = 1 match–
Basketball shootingSingle7GPT-6 AstraNo opponentmade 8–7.4n = 1 match–
Basketball shootingSingle7GPT-6 LunaNo opponentmade 6–6.6n = 1 match–
Ten-pin bowlingSingle20260930Claude Opus 5.5No opponenttotal 245–15.3n = 1 match
Ten-pin bowlingSingle20260930Claude Fable 5.1No opponenttotal 202–13.9n = 1 match
Ten-pin bowlingSingle20260930GPT-6 AstraNo opponenttotal 190–8.4n = 1 match
Ten-pin bowlingSingle20260930GPT-6 LunaNo opponenttotal 247–7.5n = 1 match
Mini-golf puttingSingle20260928Claude Opus 5.5No opponenttotal 7–8.9n = 1 match
Mini-golf puttingSingle20260928Claude Fable 5.1No opponenttotal 10–23.3n = 1 match
Mini-golf puttingSingle20260928GPT-6 AstraNo opponenttotal 7–8.5n = 1 match
Mini-golf puttingSingle20260928GPT-6 LunaNo opponenttotal 11–7.5n = 1 match

Match replay

PreliminaryExhibition season · earlier rules · 100 of 300 planned matches · ties share medals

Reading the game

DiagnosticPreliminary

Seeing is not winning

Checked against the simulator's ground truth, the models located the ball to within 0.7–0.8 cm (median) and predicted the contact point to within 3.6–4.6 cm, yet those errors did not separate returned from missed balls (135 vs 31 balls, p = 0.92), and the model that saw most accurately won 1 of 6 matches. Points are lost in execution and placement, not in seeing.

n = 3 matches with logging + 3 without per model (18 model matches); 3 models; the reference player logged 3 matches

n = 3 models × 3 matches with logging · n = 135 returned balls, 31 missed balls (all players)

Setting

Table tennis, single player against the scripted opponent (L1 seeds 101 and 111, L2 seed 202), Athlete mode, cameras only, slowed ×10.

This measures perception and physical prediction, not reading the opponent.

Asking for the log changes the program; only 3 matches per model per condition.

Two-mode pilots

Both modes are defined and have been validated in pilots.

Sample sizes are shown per pilot; fewer than 5 runs are shown as numbers only.

PrototypePreliminary

Models wrote a better hit skill than ours (Unitree G1, table tennis, Robotics Engineer mode)

On a humanoid, Opus 5.5 and Astra implemented a table-tennis hit that beat our own platform skill on the hidden API test (mean 22.7 and 21.7 vs 8 of 29); the matches themselves were all lost because the body caps the outcome.

3 runs per model (9 model runs); each run = one match + post-match evaluation

Hidden requests passed out of 29

3 runs per model (9 model runs); each run = one match + post-match evaluation

GPT-6 Astra21.7
Claude Opus 5.522.7
GPT-6.1 SOL8.7

Per run

3 runs per model (9 model runs); each run = one match + post-match evaluation

GPT-6 Astra28, 25, 12
Claude Opus 5.524, 21, 23
GPT-6.1 SOL3, 13, 10

Games won out of 12

3 runs per model (9 model runs); each run = one match + post-match evaluation

GPT-6 Astra4
Claude Opus 5.52
GPT-6.1 SOL0
platform skill (ours)0

Matches won 0 of 9 (best: Astra 5:7)

Reference players and baselines

3 runs per model (9 model runs); each run = one match + post-match evaluation

platform skill (ours)8
tools-only reference (ours)12
boundary baseline: walk + IK block, no swing0
Setting

Unitree G1 table tennis pilot. Robotics Engineer mode: the agent gets the robot model, an IK library, a joint-target interface with real limits and Unitree's walking policy, and must implement hit(t, pos, normal, vel) itself. After the match the final implementation is scored in a separate container on 29 hidden, feasible hit requests (success = position ≤ 8 cm, face ≤ 10 deg, speed error ≤ 0.5 m/s + 30%, time ±20 ms, no rejected command, no fall). Observation: true state (no cameras) in this pilot. Slowed ×10, L1 seed 101.

  • Tolerances are calibrated for G1 and are much wider than for Franka.
  • 29 hidden requests only; Astra's first run (28/29) is near the ceiling.
  • Astra's third run scored 12 of 29 hidden requests.
PrototypePreliminary

A perfect API test does not mean a good skill in play (fencing, both modes)

In fencing an implementation can pass all 40 hidden API requests and still win only 5 of 24 games when our reference strategy plays with it, so both numbers must be reported.

2 matches per model per mode (12 model matches)

Hidden requests passed out of 40

2 matches per model per mode (12 model matches)

GPT-6 Astra40, 31
Claude Opus 5.539, 39
GPT-6.1 SOL40, 28

Games won out of 24

2 matches per model per mode (12 model matches)

GPT-6 Astra13, 12
Claude Opus 5.522, 15
GPT-6.1 SOL5, 2

Athlete 6 of 6 (5:0 to 5:2)

Robotics Engineer 5 of 6

Reference players and baselines

2 matches per model per mode (12 model matches)

platform implementationAPI test: 40 · Competition test: 24
tools-only referenceAPI test: 40 · Competition test: 24
boundary baseline (IK only)API test: 0 · Competition test: 4
Setting

Epee fencing, Franka Panda + 1-axis rail, against scripted L1. Skill API strike(t, point, dir, vel). Robotics Engineer mode: the agent implements strike from joint interface + IK library; post-match API test on 40 hidden requests and a competition test (our fixed reference strategy drives the agent's strike, 24 games).

  • L1 is too weak to separate models in the Athlete mode (all won).
DiagnosticPreliminary

Seeing accurately is not winning (table tennis, checked against simulator ground truth)

All three models located the ball to under 1 cm (median 0.7–0.8 cm), yet the error did not separate returned from missed balls (p = 0.92) and the most accurate model won 1 of 6 matches: points are lost in execution and placement, not in seeing.

3 matches with logging + 3 without, per model (18 model matches); the reference player logged 3 matches

Ball position error (median, cm)

3 matches with logging + 3 without, per model (18 model matches); the reference player logged 3 matches

GPT-6 Astra0.7
Claude Opus 5.50.7
GPT-6.1 SOL0.8
reference player0.4

Contact-point error (median, cm)

3 matches with logging + 3 without, per model (18 model matches); the reference player logged 3 matches

GPT-6 Astra3.6
Claude Opus 5.53.8
GPT-6.1 SOL4.6
reference player3.1

Opponent paddle error (median, cm)

3 matches with logging + 3 without, per model (18 model matches); the reference player logged 3 matches

GPT-6 Astra0.3
Claude Opus 5.50.4
GPT-6.1 SOL0.3
reference player29.5

Matches won

3 matches with logging + 3 without, per model (18 model matches); the reference player logged 3 matches

GPT-6 Astra5
Claude Opus 5.51
GPT-6.1 SOL6

Contact-point error (median, cm)

n = 135 returned balls, 31 missed balls (all players)

Returned2.4
Missed2.1
Setting

Table tennis, single player against the scripted opponent (L1 seeds 101 and 111, L2 seed 202), Athlete mode, cameras only, slowed ×10. In the logging condition the player's program writes its own estimate (ball position and velocity, predicted bounce and contact point, opponent paddle position) before every command; after the match the recording is replayed bit-exactly and each line is compared with the simulator state.

  • This measures perception and physical prediction, not reading the opponent.
  • Asking for the log changes the program (SOL and Astra started aiming away from the opponent once asked to report its position); run both conditions.
  • Only 3 matches per model per condition.
DiagnosticPreliminary

Models can read the opponent, but do not by default (table tennis duels)

Claims about the opponent were checked against landing data across 192 reviews; 0 were wrong after the fact-label fix.

n = 192 reviews across 14 valid matches and 5 conditions.

Setting

Table tennis duels GPT-6.1 SOL vs GPT-6 Astra; the between-point reviews are scored by an LLM judge for opponent-directed understanding or prediction; conditions: no prompt / prompt asking to read the game / scouting report, 5- and 11-point games.

  • LLM judge (GPT-6 Astra after the fact-label fix); the judge is lenient.
  • Prompting is itself an intervention; this is a diagnostic, not a measure of spontaneous game reading.
PrototypePreliminary

The body caps the outcome (the same table-tennis match on five bodies)

The legged bodies won 0 of 20 model matches in this table-tennis pilot (n = 20 model matches).

1 match per cell; per body: Franka 12, RB-Y1 12, R1 Pro 12, G1 8, H1-2 12 model matches

Matches won

1 match per cell; per body: Franka 12, RB-Y1 12, R1 Pro 12, G1 8, H1-2 12 model matches

Franka Panda (fixed base)Athlete: 4/6 · Robotics Engineer: 3/6
Rainbow RB-Y1 (wheeled)Athlete: 5/6 · Robotics Engineer: 0/6
Galaxea R1 Pro (wheeled)Athlete: 3/6 · Robotics Engineer: 0/6
Unitree G1 (legged)Athlete: 0/5 · Robotics Engineer: 0/3
Unitree H1-2 (legged)Athlete: 0/6 · Robotics Engineer: 0/6
Setting

Table tennis single player against the scripted opponent (L1 seed 101, L2 seed 202), slowed ×20, cameras only, both modes; each body evaluated as a prototype. One match per body × mode × level × model.

  • Research variant: separate from the standard results table (one fixed body per sport in the Athlete mode).
  • One match per cell; differences between models are smaller than differences between bodies.

Changelog