Tesora Actuarial Benchmark (TAB): how well language models do actuarial work
- Work, step Task, step
Each task runs 100 times at each of three levels of difficulty. Scores follow once the first runs are reviewed.
Open-Sourcing the Tesora Actuarial Benchmark
A set of actuarial tasks, answer keys and scorers for testing how well language models do the work.
by Philo Bishay, Eric Lam, and Federico Reyes Gómez November 2026
The Tesora Actuarial Benchmark (TAB) is a set of tasks that tests language models on the checks reserving and pricing actuaries run. A model gets the data, the company’s guideline and a question with one right answer, and it is marked the way a reviewer would mark a junior’s exhibit.
The goal is to show where a model can already be trusted with actuarial work and where it still needs an actuary behind it.
Why we built it
We wanted to know which model is better at which kind of actuarial work, and which stack to recommend. We build our own evaluations, so we wanted that answer task by task rather than as one average score.
From our comparative rater case study: a personal auto rate plan rebuilt from its filings.
That case study is where we saw it. Several of the strongest models failed at the step of reading the filings in, including Claude Fable 5.1 and GPT-6 Astra. A benchmark lets us catch failures like that on synthetic data, before they reach a carrier’s own.
Most published insurance benchmarks test exam knowledge. INS-ActBench found that frontier models beat its human baseline there but trailed human participants by about 24 points on case reasoning (Chen, Lin, and Xu 2026). This benchmark tests work done on data instead, scored by a script against an answer planted in synthetic data, so no model grades another.
How a task works
A task follows the same path a check takes at a carrier, from the request to the review, with the judgment calls replaced by an answer key fixed in advance.
Twenty tasks ask the model to find what was planted and say where. The other 72 present a figure said to follow a stated method. The model says whether it is right, names the error from a closed list if not, and gives the correct figure.
Take RSV-03, which asks for ultimate losses on a commercial auto liability book. Case reserves were strengthened over the last two years, and the claims memo describes the review without ever saying reserves went up. The model has to notice, adjust for it the way the guideline says, and correct the ultimates it was shown.
How it is scored
A script compares each answer with a key fixed when the data were generated. On a computation task the verdict, the named error and the corrected figure all have to be right, which is roughly how a peer review treats a reserve exhibit. Between 30 and 50 percent of presented figures are correct. A model that always says “correct” scores at most half, and one that guesses an error without doing the arithmetic scores nothing.
Each task is generated at three tiers of 100 instances. The hardest tier is built so the most direct approach gets the wrong answer, with the deciding evidence in a second source such as claim counts or a footnote.
The benchmark is narrow by design. It checks whether a model can find what is wrong in the data and correct a stated calculation. It does not check whether a model can choose a method from scratch or defend its selection to a reserve committee.
What it covers
The first release has 23 of the 92 tasks and covers every area but claims triage, whose four tasks come in a later release.
The models
The first runs cover 18 models from nine labs, and six of them have open weights. Every model runs through the same open agent with extended reasoning and tool use on, and each is pinned to one serving provider so that results can be repeated.
Building with the community
We are publishing the method before the scores so that it can be checked first. Results will appear here once the first runs have been reviewed, each with the model version and suite version that produced it. A human baseline will sit beside them, from credentialed actuaries answering 180 instances under the same conditions. The tasks, generators and scorers will follow under an open license. We would like help with all of it.
If your results differ from ours, please reach out. We are always evaluating new tasks for the benchmark.
We want to thank Philo Bishay, who co-wrote the proposal this benchmark comes from. Thank you to the peers who read drafts of this page and worked through the tasks with us. Thank you also to Tesora’s investors and board, many of whom took real time to help us get this right. We are grateful as well to our peers at other frontier AI labs, to the people building open benchmarks whose work we learned from, and to the members of the CAS AI working group.
First-release task: Reserving method review for a liability book
First-release task: Development pattern diagnosis
First-release task: Ultimate losses for commercial auto liability
First-release task: Cape Cod ultimate
First-release task: Annual loss triangles from a claim table
First-release task: Territory relativities from experience
First-release task: On-level earned premium
First-release task: Loss trend selection review
First-release task: Overall rate indication by the loss ratio method
First-release task: Permissible loss ratio and expense provision
First-release task: Loss elimination ratio and deductible relativity
First-release task: Frequency segmentation review for a personal auto plan
First-release task: Pricing adequacy of a quoted account
First-release task: Limited average severities by layer
First-release task: Severity curve for a commercial property book
First-release task: SIU referral review for a personal auto book
First-release task: Litigation rate and severity by accident year
First-release task: Rating variable review for a multistate filing
First-release task: Expected loss to a catastrophe layer
First-release task: VaR and TVaR of retained aggregate loss
First-release task: Catastrophe reinsurance limit to buy
First-release task: Expected cost of a quota share
First-release task: Credibility-weighted loss costs for a group of accounts
Laboratory: Anthropic
Model: Claude Fable 5.1
Model: Claude Opus 5.5
Model: Claude Sonnet 5.5
Model: Claude Haiku 5.5
Laboratory: OpenAI
Model: GPT-6 Astra
Model: GPT-6.1 Sol
Model: GPT-6 Luna
Model: GPT OSS 120B
Laboratory: Google
Model: Gemini 4 Argon
Model: Gemini 3.8 Flash
Laboratory: Alibaba
Model: Qwen3.8-Max
Model: Qwen3.8-27B
Laboratory: Meta
Model: Muse Spark 1.3
Model: Muse Glimmer 30B
Laboratory: DeepSeek
Model: DeepSeek V4 Pro
Laboratory: Moonshot
Model: Kimi K3
Laboratory: Zhipu
Model: GLM-5.3
Laboratory: xAI
Model: Grok 4.7
Guideline
Contents: A reported chain ladder on volume-weighted averages of the latest five years, with no development past the last age. Where case adequacy has changed, a Berquist-Sherman adjustment first, de-trended at a stated severity trend.
What is asked: Apply it as written, including the adjustment only when the data call for it.
Triangles
Contents: Paid and reported losses, with reported, closed and open claim counts, for ten accident years.
What is asked: Follow average case reserve per open claim along each diagonal and find where it steps.
Claims by size band
Contents: Closed counts and paid losses by claim size band.
What is asked: Tell a real strengthening from faster settlement of small claims, which raises the same average.
Claims memo
Contents: An undated reserve review that never says reserves went up.
What is asked: Treat it as evidence to weigh, since it does not state the answer.
Presented ultimates
Contents: Ultimate losses by accident year, said to follow the guideline.
What is asked: Say whether they are right, name the error if they are not, and give the correct figures.
Actuaries
Review tasks and answer keys, and tell us where a planted condition is unfair.
Carriers and consultancies
Tell us which work you would hand to a model first, so the next tasks look like it.
Researchers and AI labs
Run the suite and report any shortcut a model finds through it.