This page can't load inside an embedded panel.

The panel blocks the storage and requests this page needs. Open it in its own tab instead.

Open in a new tab
  • Benchmarks
    Agent ArenaHow easy is it for AI agents to get started with devtools?AX Tier ListCan an agent sign up on its own?
  • CLI
  • Docs
  • Blog
  • Careers
  • Log inRequest access
Benchmarks

Agent Arena

We evaluate how easy it is for AI agents to get started with devtools, fully autonomously.

Want to improve your rank? Get a detailed report with your insights.

Get report
Browser AutomationSandboxesSearchAI FrameworksDatabases & BaaSAuth & InfraEmail & MessagingLLM ObservabilityImage GenerationVideo Generation
xAI Grok (image)33s
Tavily43s
Perplexity47s
Browserbase48s
Brave Search52s
Resend56s
Exa1m 6s
Browserless1m 11s
Mailgun1m 11s
E2B1m 18s
Postmark1m 21s
OpenAI (DALL-E)1m 22s
Hyperbrowser1m 30s
Langfuse1m 31s
Black Forest Labs (FLUX.2)1m 36s
Google Imagen1m 36s
Raindrop1m 36s
LangGraph1m 42s
Steel1m 47s
Keywords AI1m 49s
Together (FLUX.2)1m 50s
Browser Use1m 50s
AgentMail1m 53s
Anchor Browser1m 54s
Vercel AI SDK1m 56s
Parallel1m 58s
Daytona2m 21s
Kernel2m 30s
Notte2m 32s
Fireworks (FLUX.1)2m 39s
Together (Veo 3)2m 41s
LangSmith2m 42s
Modal2m 46s
Helicone2m 50s
Auth02m 55s
Clerk3m 1s
Portkey3m 1s
Vercel3m 5s
Braintrust3m 14s
Vercel Sandbox3m 18s
Supabase3m 21s
Neon3m 28s
Descope3m 33s
Freestyle3m 39s
Netlify3m 46s
CrewAI3m 47s
MongoDB Atlas3m 50s
Turso4m 4s
Fal.ai (Veo 3)4m 13s
Airtop4m 13s
Convex4m 27s
Replicate (Video-01)4m 28s
WorkOS4m 39s
Cloudflare Pages4m 50s
PlanetScale4m 53s
Arize5m 18s
Upstash5m 35s
OpenAI (Sora)5m 56s
ClickHouse Postgres6m 5s
Mastra6m 7s
Fly.io6m 41s
Railway8m 27s
Blaxel8m 44s
Render10m 35s
xAI Grok (video)11m 28s
Cloudflare Sandbox13m 11s
ImagineArt (Kling 1.5)16m 1s
Google (Veo 2)32m 24s
1Goated: #1 in this categoryLangGraph1m 42s$0.2100
2Vercel AI SDK1m 56s$0.2900
3Tavily43s$0.4001
4Perplexity47s$0.4301
5Black Forest Labs (FLUX.2)1m 36s$0.8200
6Browserbase48s$0.4201
7Exa1m 6s$0.4401
8Resend56s$0.5401
9CrewAI3m 47s$0.270.20
10OpenAI (DALL-E)1m 22s$0.4600
11Mailgun1m 11s$0.4101
12Browserless1m 11s$0.6101
13Together (FLUX.2)1m 50s$0.7620
14Together (Veo 3)2m 41s$0.4610
15Fal.ai (Veo 3)4m 13s$1.6910
16xAI Grok (image)33s$0.3500
17Postmark1m 21s$0.5601
18Mastra6m 7s$0.440.10
19Fireworks (FLUX.1)2m 39s$0.5100
20Google Imagen1m 36s$0.4800
21Steel1m 47s$0.830.11
22Browser Use1m 50s$0.830.11
23Anchor Browser1m 54s$0.9801
24Supabase3m 21s$0.4501
25Raindrop1m 36s$1.0301
26Neon3m 28s$0.5101
27Daytona2m 21s$0.810.81
28MongoDB Atlas3m 50s$0.5501
29Hyperbrowser1m 30s$0.872.81
30Langfuse1m 31s$1.1502
31Convex4m 27s$0.6001
32Turso4m 4s$0.610.331
33Clerk3m 1s$1.4101
34Kernel2m 30s$1.390.71
35Notte2m 32s$1.690.21
36Auth02m 55s$1.5901
37E2B1m 18s$0.841.62
38Replicate (Video-01)4m 28s$1.4700
39Modal2m 46s$1.161.31
40Brave Search52s$0.4111
41LangSmith2m 42s$1.2701
42PlanetScale4m 53s$0.5811
43Airtop4m 13s$1.360.11
44Freestyle3m 39s$1.2011
45Descope3m 33s$1.8601
46Keywords AI1m 49s$1.0001
47Upstash5m 35s$0.800.671
48ClickHouse Postgres6m 5s$0.5911
49Parallel1m 58s$0.4501
50WorkOS4m 39s$2.5601
51Vercel Sandbox3m 18s$2.2021
52Portkey3m 1s$1.2711
53Braintrust3m 14s$1.6011
54Vercel3m 5s$1.3321
55AgentMail1m 53s$0.8441
56Blaxel8m 44s$3.5011
57Netlify3m 46s$1.3521
58Cloudflare Sandbox13m 11s$4.732.70
59Heliconemaintenance mode2m 50s$1.1803
60OpenAI (Sora)5m 56s$0.8650
61Cloudflare Pages4m 50s$1.8321
62xAI Grok (video)11m 28s$0.5610
63Arize5m 18s$2.7412
64Fly.io6m 41s$2.6021
65Google (Veo 2)32m 24s$3.6750
66Render10m 35s$5.7412
67Railway8m 27s$4.6281
68ImagineArt (Kling 1.5)16m 1s$5.99120
69SendGridblocks browser automationNo dataNo dataNo dataNo data
70CodeSandboxblocks automated signupNo dataNo dataNo dataNo data
Methodology

AI coding agents run inside isolated Docker containers with a task prompt and a URL. Each agent must autonomously discover docs, install packages, write working code, and verify the result, all without human help beyond providing API credentials when asked.

How rankings work: providers in the same category are ranked against each other on four dimensions: Time, Cost, Errors, and Interruptions. Within a category, the per-dimension rankings combine into an overall position. Less time, lower cost, fewer errors, and fewer interruptions all improve a provider's standing. No composite score, no letter grades.

All results were run with Claude Code on Claude Opus 4.6. Results across more agents and models are coming soon.

All tool calls, errors, timing, and token usage are recorded. Rankings are deterministic from session logs. Multiple independent runs per provider are aggregated.

Don't see your tool?Request an evaluation
Build products agents understand, trust, and choose.
Request access

Benchmarks

  • Agent Arena
  • AX Tier List

Product

  • CLI
  • Docs
  • Log in
  • Book a demo

Company

  • Careers
  • Blog
  • Manifesto

Follow

  • X / Twitter
  • LinkedIn
© 2026 2027 Inc. San Francisco, California. Firegoat AI
PrivacyTermsBack to top