Independent benchmarks for biological AI

The knowledge graph of biological AI benchmarks

Published benchmark results for DNA, RNA, protein and cell models, linked to their sources and to the models, datasets and protocols they tested. Where papers disagree, the graph says so.

linked facts
2.76M
models
373
benchmarks
125
published results
51,833
disagreements flagged
197

Release 2026-11-20-72316db0764b

What is in the graph

One graph, every link between a model and its evidence

Each record is a node and each relationship is an edge: which checkpoint was evaluated, on which dataset subset, under which protocol, with which metric, reported in which table of which paper. You can follow any number back to its source in a few hops.

373modelsand 618 methods, 5,564 configurations
125benchmarks552 tasks, 1,562 protocols
818datasets652 defined subsets and splits
51,833resultsfrom 24,717 evaluations
1,494sourcespapers, preprints, leaderboards and repositories
31use caseslinked to their relevant protocols

Built on open standards

RDF with OWL 2 RL reasoning and SHACL validation, using DCAT, schema.org, OBI, ECO, SEPIO and the ML Schema vocabularies. 2.45 million asserted facts, 300,000 inferred, and 15,000 derived facts about disagreements.

RDFOWL 2 RLSHACLSPARQLDCATschema.orgOBISEPIO

Why a graph

Linked evidence shows where papers disagree

When two studies rank the same pair of models in opposite orders, or one result sits far from all the others, the graph flags it as a lead. The current release flags 197, and 191 are still open. Each is a question to investigate, not a verdict.

An AlphaGenome splicing result that could not be right

In October 2026 an AlphaGenome developer told us a splicing result in a recent preprint could not be right. We traced the preprint's scripts and found an off-by-one error. Read alone, that comparison would have misled anyone choosing a splicing model. Against the rest of the graph, it stood out.

Read the investigation
Explained ESM-1b against EVE on ClinVar missense variants
SourceMetricESM-1bEVE
Brandes 2023, variants scored by all methodsROC-AUC0.8920.883
Brandes 2023, ClinVar head to headROC-AUC0.9060.885
ProteinGym clinical benchmarkranks EVE above ESM-1b
Livesey and Marsh 2025ranks EVE above ESM-1b

Brandes 2023 ranks ESM-1b above EVE in 6 of 8 comparisons on ClinVar and HGMD variants, with 2 ties. Two other sources rank EVE above ESM-1b. Our review found a difference in metric, not an error: Brandes 2023 pools all variants into one ROC-AUC, and ProteinGym averages ROC-AUC per gene. See the lead

How it fits together

From published papers to an agent that can rerun them

A scheduled agent scans every day for new models, benchmarks and datasets. Reviewed records become a versioned release of the graph. Coding agents query that graph, and can rerun a benchmark recipe on your machine and check the result against the published value. So far that is one recipe: Rewire's own MFASS v2 baseline, on CPU.

Each recipe run writes a receipt. Nothing in the plugin exports or submits a result.

What you can use

The parts, and what each one does today

Live

Benchmark database

Browse models, benchmarks and use cases. Each result links to its source, and to the checkpoint when the paper names one, with its review status.

benchmarks.rewirebio.io
Live

Disagreements

Conflicts between papers, found by rules over the graph on every release, with the studies on each side.

See the disagreements
Live

Database MCP server

The graph for coding agents. Entity lookup, provenance, ready-made disagreement queries, and SPARQL for anything else. Full-text search was added in v0.3.0.

rdf-kg-mcp on GitHub
New tools

genomics-mcp

Find and read genomic data from EGA, ENA, ENCODE, GEO, NCBI, ClinVar, gnomAD and your own files, one region at a time. A catalogue of published synthetic-data protocols and GIAB and HPRC truth sets is merged. The Claude Code plugin installs a commit that has it. The 0.1.0 container does not.

genomics-mcp on GitHub
CLI live

rewirebench

Runs benchmark protocols and model adapters with pinned data and splits. Recipes, their receipts and an MCP server are on main. The latest release, v0.5.0, has none of them. A recipe run writes a receipt: what ran, on what hardware, what it scored and what it cost. The MCP server lets a coding agent plan and run recipes on CPU. GPU runs are not built: the code plans and prices a RunPod or Spark run and starts neither.

rewire-benchmarks on GitHub
Live

Blog

Plain-language write-ups of new models and benchmarks, and of what our reruns and investigations found.

Read the blog

For coding agents

Ask the graph from your agent

Install the database server and your agent can answer questions like these from the graph, with sources:

  • "Which DNA language models have been tested on splice variant interpretation, and where do the results disagree?"
  • "What evidence is there that GPN-MSA beats CADD on rare-disease variants, and who ran it?"
  • "List every result for Evo 2 with its dataset, split and source table."

Each server runs on your machine. The database server downloads the current public release; your questions and your data stay local.

Claude Code plugin
/plugin marketplace add rewire-bio/rewire-benchmarks
/plugin install bio-model-benchmarks@rewire-benchmarks
Database server, any MCP client
claude mcp add rewire-benchmark-graph -- uvx --from git+https://github.com/rewire-bio/rdf-kg-mcp@v0.4.0 rdf-kg-mcp serve https://benchmarks.rewirebio.io/agents/kg.tar.gz
genomics-mcp, container
docker run --rm -i \
  --mount type=bind,source=/path/to/your/data,target=/data,readonly \
  --mount type=volume,source=genomics-mcp-work,target=/work \
  ghcr.io/rewire-bio/genomics-mcp:0.1.0

Other install routes: genomics-mcp install guide ยท agent guide. The release is also served as JSON with no key: API docs.

Coverage

Broad coverage, deepest where it matters clinically

Deepest

DNA and RNA models

Full records, use-case rankings, run recipes, one of which we have executed ourselves so far (the MFASS v2 baseline), and Rewire's own benchmarks as they are built: variant interpretation, splicing, regulatory variants, pathogen detection.

Deep

Protein models

Full records and published results, with disagreements tracked: variant effect, structure and design.

Broad

Everything else

Single-cell, spatial, chemistry, antibody and design models are listed with their sources and covered on the blog.

For model developers

If your model is in the graph

We contact you first

The rules list disagreements automatically, as open questions. When we investigate one involving your model and plan to write about it, we contact you before we publish.

Corrections are welcome

If a record is wrong, open an issue on rewire-benchmark-data. Corrections are versioned and the previous value stays visible.

Exact identity

Results are tied to the checkpoint that was tested. When a paper does not say which weights it used, the result stays at the model-family level.

No single score

Incompatible benchmarks are never averaged into one leaderboard. Rewire does not build or sell biological models.