Artificial Analysis has launched AA-Robotics v0.1, a research-preview benchmark that gives frontier language models direct control of a real 6-axis robot arm. The early result is a useful reality check for physical AI: the models can now complete simple manipulation tasks, but they remain far slower and more expensive than a human doing the same work.
In the first eight tabletop tasks, GPT-6 Astra scored 69.4 out of 100, followed by GPT-6.1 Sol at 64.6, Gemini 4 Argon at 55.3, Claude Opus 5.5 at 48.8 and Claude Fable 5.1 at 46.3. The tasks include pressing a button, placing a toy inside a marked area, pouring a drink, tidying blocks, matching shapes, writing “hello world,” building a tower and measuring a box.
The setup is deliberately simple but revealing. Each model sees the work area through four cameras and writes code against an HTTP API to move a Hiwonder NexArm. Attempts are zero-shot, and a human judge scores the final state of each task. MIT CSAIL’s survey of frontier models in robotics lists the benchmark as a real-arm evaluation and reports the same first-release scores and timing data.
The speed gap is the larger story. Artificial Analysis reports that a person averages about 20 seconds per task, while four of the five tested models cluster around 30 minutes. Gemini 4 Argon averages about 52 minutes. That puts today’s frontier models roughly two orders of magnitude behind the human baseline on elapsed time.
Cost also varies sharply. GPT-6.1 Sol averaged $1.93 per task, while GPT-6 Astra averaged $16.36. Artificial Analysis priced the human baseline at about $0.21 per task using average U.S. hourly earnings. The benchmark therefore shows a real capability milestone without implying that general-purpose language models are ready to replace dedicated industrial robot controllers.
That distinction matters as robotics software moves toward more general models. BitcoinVersus.Tech has recently covered Standard Bots adding AI skills to factory robots, DexForce training humanoids with synthetic data, and Intel’s physical-AI push. AA-Robotics adds a measurement layer to that trend by testing how far general frontier models can get when they are placed directly in the control loop.
The main limitation is scope. Version 0.1 covers only five models, eight tasks and one robot arm, and the results come from Artificial Analysis rather than an independent replication. The benchmark is best treated as an early snapshot of embodied-AI progress, not a universal ranking of robotics capability.
BitcoinVersus.Tech Editor’s Note:
We volunteer daily to ensure the credibility of the information on this platform is Verifiably True. If you would like to support to help further secure the integrity of our research initiatives, please donate using the QR code provided
For best QR-code scanning results, please scan this donation code from a desktop or larger screen. Mobile screens may prevent camera recognition.
Leave a Reply