Best AI agent now finishes 41.3% of paid freelance projects to client standard, up from 2.5%
People grade each result against a paid professional's. The test setup was strengthened in 2026, and most projects still fail.
Remote Labor Index: best AI agent on paid freelance projects
Highest share of projects judged acceptable to a client (Scale Labs leaderboard), by date listed.
What this measures: Real freelance projects that professionals were paid to do. Human evaluators judge whether an AI agent's work would be accepted by a client. At the October 2025 launch the best agent passed 2.5%.
Dates are when each model was listed on the leaderboard, not its release. The test setup was strengthened in July 2026 (stronger agent software, bigger budget), so later points mix model and setup gains.
Show data table and sources
| Date | Label | Value |
|---|---|---|
| 27 Oct 2025 | Manus 1.0 | 2.5% |
| 9 Jan 2026 | Claude Opus 4.5 | 3.8% |
| 5 Mar 2026 | Claude Opus 4.6 (Cowork) | 4.2% |
| 1 Jul 2026 | Claude Fable 5 | 16% |
| 17 Sept 2026 | GPT-6 Astra | 21% |
| 7 Oct 2026 | Claude Opus 5.5 | 21% |
| 9 Nov 2026 | Claude Fable 5.2 | 41% |
- Remote Labor Index (leaderboard) (Scale Labs)
The Remote Labor Index (RLI) tests AI agents on real freelance jobs such as 3D models, architecture plans, animation and data analysis. The Center for AI Safety (CAIS) and Scale AI built it from 240 projects that 358 Upwork freelancers were paid to do, worth $143,991 in all. 230 are used for scoring. The median project took a person 11.5 hours and paid $200. Human evaluators compare each AI deliverable with the professional’s and pass it if a reasonable client would accept it.
At launch in October 2025 the best agent passed 2.5% of projects. On 9 November 2026 Scale’s leaderboard added Claude Fable 5.2 at 41.3%. Claude Opus 5.5 is second at 21.25%, followed by GPT-6 Astra (20.83%), Claude Fable 5.1 (17.92%) and Gemini 4 Argon (17.08%). In July 2026 CAIS had reported 15.8% for Claude Fable 5, against 4.17% for the leader before it. Anthropic’s Fable 5.2 launch post lists the same 41.3%.
The July post also changed how agents are tested. Each model now runs in its family’s coding agent (Claude Code or Codex CLI) with a computer-use tool, gets up to 24 hours per project, and has a second “critic” agent review the work before it is submitted. The cost cap per project is $50, or $150 for Fable 5.
What it means
On small, self-contained digital jobs, a paying client would now accept the best agent’s work about two times in five, up from one in forty. The other 59% of projects still fail, so the result is far from agents replacing freelancers.
Caveats
- Part of the rise comes from the setup. The launch tests used a different agent framework and a $30 cap per task, and the leaderboard gives one score per model without separating model from setup.
- The leaderboard states there is no confidence interval for the automation rate, so close scores (Opus 5.5 and GPT-6 Astra) can’t be ranked with confidence.
- The index covers only some remote work: it excludes projects that need physical labour, long-term evaluation or direct client interaction, and its projects are small (median $200).
- Grading rests on each evaluator’s overall judgement. Scale reports 94.4% agreement between evaluators on pass or fail. Scale and CAIS run the test, not the model makers.