Which AI is actually best at finance?

Every AI company says its model is the best at finance. That is marketing, not evidence. So I put them through a real benchmark, an independent one built with Stanford, then reproduced the whole run myself with Claude sitting in the middle as a neutral judge. Here is what the numbers actually say.

Read this first: the ranking on this page is a snapshot from July 4, 2026. These AI leaderboards change constantly, new models are added and every model is re-scored every few weeks, so by the time you are reading this the order may be completely different, or even reversed. Treat the exact numbers below as an illustration of the method, not as today's truth. For the current standings, always check the live Finance Agent leaderboard: vals.ai/benchmarks/fabv2.


Anthropic says Claude is the best. OpenAI says GPT. Google says Gemini. Microsoft says Copilot. For finance, that is just noise. Passing an equation is nothing like reading a 10-K and catching the footnote that breaks a valuation. You do not need the model that is best on a slide, you need the one that is best at your job.

That is exactly what the vals.ai Finance Agent benchmark measures. It is an independent benchmark, developed with Stanford researchers, that scores the top models on real, domain-specific work. It hands each model a set of tools, points it at SEC filings, and makes it research the answer the way a junior analyst would. The full harness is open source at github.com/vals-ai/finance-agent-v2.

  1. The results — the leaderboard, the specialities, and the cost
  2. How it is graded — the paper, the rubric, and why you can trust it
  3. Running it yourself — the repo, the gotchas, and Claude as judge

Part 01

The results: three charts, one surprise

The leaderboard: not the model you would guess

The number one finance AI in this snapshot is not Claude, and not GPT. It is Gemini 3.5 Flash at about 58%, with Claude Fable 5 right behind at 56% and Claude Opus 4.8 at 54%. The scores are close at the top, which is exactly why the order keeps moving.

vals.ai Finance Agent v2 leaderboard: Gemini 3.5 Flash first at 57.86%, Claude Fable 5 second, Claude Opus 4.8 third, with cost per test and latency columns
vals.ai Finance Agent v2, overall accuracy (snapshot, July 4 2026). Gemini 3.5 Flash leads at 57.86%, but watch the two columns most rankings ignore: cost per test and latency.

By task: no single model wins everything

Break the score down by task and the headline dissolves. There is no single best AI for finance, it depends on the job. Gemini Flash wins earnings analysis at 79% and market analysis at 74%. But a Claude model takes the quantitative work at roughly 81% and disclosure analysis at 69%, the deep-reading tasks where you have to hold a whole filing in your head.

Category leaders bar chart: general quantitative led by Claude Fable 5 at 80.8%, earnings and market analysis by Gemini, financial modeling lowest for everyone at 29.4%
Category leaders by task. Note the bottom bar: financial modeling is the weakest category for every model, topping out near 29%.

That bottom bar is the most important number on the page. Financial modeling is where every model is weakest. So the rule writes itself: let the AI read, analyze and explain, and keep your own hands on the model.

By cost: accuracy is only half the story

Plot every model by accuracy and price per task and the punchline lands. Gemini Flash is not just the most accurate, it is one of the cheapest of the leaders, around $2.51 per task, sitting in the top-left corner where you want to be. The premium models sit off to the right at three to four times the price, for a point or two less accuracy.

Scatter plot of accuracy versus cost per test; Gemini 3.5 Flash sits top-left at $2.51 and 57.86% accuracy while premium models sit far right at higher cost
Accuracy versus cost per test. Gemini 3.5 Flash (top-left, $2.51) is the efficient frontier of this chart; the expensive models to the right do not buy you more accuracy.

The best finance model right now is not Claude and not GPT, and it is also one of the cheapest.


Part 02

How it is graded, and why you can trust it

The benchmark is documented in a 2025 paper by Antoine Bigeard, Langston Nashold, Rayan Krishnan and Shirley Wu, titled Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks. It is built from 537 questions written by experts, with input from banking, hedge-fund and private-equity professionals, spread across nine categories of analyst work: quantitative, earnings analysis, market analysis, disclosure analysis, adjustments, comparables, precedents, financial modeling and general qualitative reasoning. To put the difficulty in perspective, in the paper even the strongest model cleared only 46.8% of the questions, at an average cost of $3.79 per query.

Full citation: Bigeard, A., Nashold, L., Krishnan, R., & Wu, S. (2025). Finance Agent Benchmark: Benchmarking LLMs on Real-world Financial Research Tasks. arXiv:2508.00828.

The grading is the honest part, and it comes down to four steps:

  1. Experts write the question and the gold answer. These are real analyst tasks on real tickers, not multiple choice.
  2. The gold answer is broken into a rubric, an explicit checklist of the individual facts and calculations a correct response must contain. A hard question might have seven or more separate rubric points.
  3. The model researches and answers using real tools, pointed at live SEC filings and market data, exactly like a junior analyst would.
  4. A grader checks the answer against each rubric point, one by one, marking each hit or miss, and then runs a contradiction check that fails an answer stating something false.

The score for a question is the share of its rubric points that were hit, and the overall score averages that across all questions. There is no partial credit for sounding confident, only for matching the checklist, which is precisely why a model that writes a fluent but wrong answer scores badly. Because the standard is a fixed checklist rather than a vibe, the same scorecard applies to every model and the result is reproducible.


Part 03

Running it yourself

The best part is that the harness is open source, so you can see the actual questions, the real tickers, and exactly what the agent does.

GitHub repository vals-ai/finance-agent-v2, MIT license, showing the README that lists the agent tools: web_search via Tavily, edgar_search, parse_html_page, retrieve_information and price_history
The open-source benchmark: github.com/vals-ai/finance-agent-v2.

What is in the repo

The structure is small and readable. data/ holds public.csv, the public set of questions together with their rubrics. finance_agent/ is the Python package that does the work: run_agent.py is the entry point that hands a chosen model the tools and runs it on a question, and tools.py defines the five tools the agent can call, EDGAR filing search, web search, HTML-page parsing, stored-information retrieval, and price history. The Makefile installs everything, and pyproject.toml with uv.lock pin the dependencies. On top of that I added my own run_bench.py, a checkpoint wrapper that runs the harness one question at a time and saves each answer, so a crash or an exhausted key never loses or re-charges finished work.

Wiring up the tools

To run it you connect a few APIs: Tiingo for market and pricing data, EDGAR filing search through the SEC API, and Tavily for web search. All have free tiers, but the SEC one is tiny and gets exhausted fast, so in practice you make a couple of accounts and rotate several keys (a simple semicolon-separated list, cycled on each call) to keep a long run alive.

Pick your contestants, but mind the cost and time

You choose the models: any AIs you want to compare, just add their keys, so you are not locked into my two. But be warned, this is neither free nor fast. Every question makes the model behave like a real analyst, a dozen or more tool calls, live filings, real web searches, so each question costs real API money and can take several minutes of wall-clock time. Running the full 537-question set across two models can add up in API bills and take the better part of an entire day, so start with a small pilot before you commit to a full sweep.

The time-budget gotcha

There is a per-question time budget you set with --max-time, and getting it wrong quietly ruins a run. I first tried a tight budget of around 400 seconds, and it simply was not enough for Gemini: as an agent it makes many tool calls per question and kept hitting the cap mid-research and timing out, so the answer came back empty. Raising the budget to roughly 1000 seconds and up fixed it. The cheap GPT is fast and non-reasoning, so it finishes well inside 400 seconds, but the strong agentic models are slow and deliberate, so plan for long per-question budgets and long total runs.

Before you run anything: verify and add your own guards

One habit worth keeping before you run anything you pull from GitHub, this repo included: verify the source and add your own safety guards first. You see news all the time about malicious code hidden inside repositories and packages, so check who the author actually is, confirm you are on the real project and not a lookalike fork, and read the entry-point scripts to see what they execute and what they send over the network before you run them. Never run a signup or "unlock" step that asks for your name or email, and only wire in your API keys once you are satisfied it is safe. That is exactly why the first thing the prompt below does is make Claude read the entry points and prove they are safe before it runs a single line.

The one catch, and using Claude as the judge

There is one piece that is not open source: the vals.ai grader, the thing that actually scores the answers. So I built my own. I use Claude Opus as the orchestrator. It runs each model through the questions, then grades every answer against vals.ai's published rubric, an exact checklist from the paper. It is not a contestant and it never answers a finance question itself, its only job is to run the models and then judge them line by line. Same scorecard, same standard, no favoritism. You do not have to touch the raw repo or juggle keys by hand to do this: you can point your Claude subscription at the public repo and have it wire up the run, add the safeguards, and grade the results. This is the exact prompt I used:

Prompt · Claude as orchestrator + judge
I want to reproduce the vals.ai Finance Agent benchmark and compare two models on it: Gemini 3.5 Flash (currently #1 on the fabv2 leaderboard) vs a cheap GPT, gpt-4.1-mini, graded by you (Claude Opus) against vals.ai's own published rubric. Do everything in a new vals_benchmark/ folder.1. GitHub safety first. The harness is the open-source repo vals-ai/finance-agent-v2 (MIT license, paper arXiv 2508.00828). Clone it, but before running any install script or agent, open and read the entrypoints (the Makefile, the finance_agent package entry / run_agent.py, and tools.py) and summarize exactly what they execute and what network calls they make. Do not run any signup/"unlock" step and never enter my name or email anywhere. Only proceed once you've shown me the entrypoints are safe.2. Environment. The package needs Python ≥3.12 (the Makefile's 3.11 default is stale). Make a venv with py -3.14 (or newest ≥3.12), then pip install -e ., and confirm the finance-agent CLI imports.3. Keys → one separate .env in vals_benchmark/, and tell me exactly where to paste each. Model keys: OPENAI_API_KEY, GEMINI_API_KEY / GOOGLE_API_KEY, ANTHROPIC_API_KEY. Tool keys: TAVILY_API_KEY (web), PRICING_DATA_API_KEY (Tiingo), SEC_EDGAR_API_KEY (sec-api.io). .env must be gitignored. Never print or commit a key. Before the full run, hit each API with a tiny test call and confirm 200.4. EDGAR key rotation. sec-api.io's free tier is tiny and gets exhausted fast, so support multiple keys in SEC_EDGAR_API_KEY as a semicolon-separated list, and round-robin them (itertools.cycle) on each EDGAR call so one key running out never kills the run.5. Register Gemini 3.5 Flash. google/gemini-3.5-flash isn't in the registry. Add it to the per-provider YAML (model_library/config/google_models.yaml) under the gemini-3 group; editing the compiled all_models.json does nothing. Verify it shows up as FOUND.6. Checkpoint wrapper (never lose or re-charge work). Write run_bench.py that runs the harness one question at a time, saves each answer to results/<model>/qNNN.json, and skips any question already saved. Args: --model, --questions, --max-time.7. Run the pilot. From the public questions in data/public.csv, run a few on both google/gemini-3.5-flash and openai/gpt-4.1-mini, with identical tools (sec-api EDGAR, Tavily, Tiingo, sec.gov fetch, calculator), each answering independently.8. Grade like the paper, as a neutral judge. vals.ai's grader isn't open source, so replicate its published method: for each question take the expert gold answer's rubric and check each model's answer against every rubric point (hit/miss) plus a contradiction check. You are the judge, not a contestant, you never answer a finance question yourself. Write grading.json, a REPORT.md, and a clean PDF scoreboard.Hard rules: read the entrypoints before running anything; keys live only in .env, never printed or committed; .env and results/ are gitignored; if a run crashes, the checkpoint means we resume, not restart.

What the head-to-head showed

I ran the current number one, Gemini 3.5 Flash, against a deliberately cheap and outdated model, GPT-4.1 mini, to keep API costs sane and to show the raw difference the model makes. Every answer was graded by Opus against the vals.ai rubric, point by point, exactly as described above: each rubric item marked hit or miss, plus a contradiction check, and the score is the share of points hit. On the pilot Gemini landed 95% of the rubric points, GPT-4.1 mini landed 25%. That is not a gap, it is a canyon.

And the reason was not raw intelligence. On a disclosure question, Gemini read both 10-Ks and got it right. The cheap model pulled the wrong year's filing and grabbed data off a stock-screener site, and concluded the exact opposite. When you look at the answers, Gemini cites the real SEC filing URLs, while the cheap model cites press releases and MarketBeat. That is the whole story. When you do not force a model through the primary source, it grabs the nearest convenient one, and in finance the nearest convenient one is how you end up wrong and confident at the same time.

The cheap model was not dumb. It just trusted the nearest convenient source instead of the filing.

The real lesson is not "use Gemini." The models change, the questions do not. Wire up the benchmark, force any model through primary sources, and let a neutral judge grade it against a fixed rubric. That is how you find out which AI is actually best at your job, instead of best on someone's slide.


Benchmark source: vals.ai Finance Agent v2. Open-source harness: vals-ai/finance-agent-v2. Paper: Bigeard, Nashold, Krishnan & Wu (2025), arXiv:2508.00828. All leaderboard figures are a snapshot from July 4, 2026 and will have moved since; reproduce them before quoting.