Independent benchmarks for biological AI

The knowledge graph of biological AI benchmarks

Published benchmark results for DNA, RNA, protein and cell models from 1,480 sources, each linked to the model checkpoint, the dataset, the protocol and the table it came from. Where papers disagree, the graph says so.

linked facts
2.76M
models
380
benchmarks
125
published results
51,833
disagreements flagged
197

Release 2026-11-17-793d9cca4ad1

What is in the graph

One graph, every link between a model and its evidence

Each record is a node and each relationship is an edge: which checkpoint was evaluated, on which dataset subset, under which protocol, with which metric, reported in which table of which paper. You can follow any number back to its source in a few hops.

380modelsand 618 methods, 5,564 configurations
125benchmarks552 tasks, 1,562 protocols
817datasets652 defined subsets and splits
51,833resultsfrom 24,717 evaluations
1,480sourceseach archived and hashed
31use caseslinked to their relevant protocols

Built on open standards

RDF with OWL 2 RL reasoning and SHACL validation, using DCAT, schema.org, OBI, ECO, SEPIO and the ML Schema vocabularies. 2.44 million asserted facts, 300,000 inferred, and 15,000 derived facts about disagreements.

RDFOWL 2 RLSHACLSPARQLDCATschema.orgOBISEPIO

Why a graph

Linked evidence shows where papers disagree

When two studies rank the same pair of models in opposite orders, or one result sits far from all the others, the graph flags it as a lead. The current release flags 197, and most are still open. Each is a question to investigate, not a verdict.

An AlphaGenome splicing result that could not be right

In October 2026 an AlphaGenome developer told us a splicing result in a recent preprint could not be right. We traced the preprint's scripts and found an off-by-one error. Read alone, that comparison would have misled anyone choosing a splicing model. Against the rest of the graph, it stood out.

Read the investigation
Under investigation ESM-1b against EVE on ClinVar missense variants
SourceMetricESM-1bEVE
Brandes 2023, variants scored by all methodsROC-AUC0.8920.883
Brandes 2023, ClinVar head to headROC-AUC0.9060.885
ProteinGym clinical benchmarkranks EVE above ESM-1b
Livesey and Marsh 2025ranks EVE above ESM-1b

The model developers' own paper ranks ESM-1b first in 6 of 6 comparisons. Two independent sources rank it the other way. See the lead

How it fits together

From published papers to an agent that can rerun them

Agents read new papers, preprints and leaderboards every day. Reviewed records become a versioned release of the graph. Coding agents query that graph, fetch the data they need, and run benchmarks themselves.

Receipted results from a run can be submitted back into the evidence store, so the next release of the graph includes them.

What you can use

The parts, and what each one does today

Live

Benchmark database

Browse models, benchmarks and use cases. Every number links to the paper, table and checkpoint it came from, with its review status.

benchmarks.rewirebio.io
Live

Disagreements

Conflicts between papers, found by rules over the graph on every release, with the studies on each side.

See the disagreements
Live

Database MCP server

The graph for coding agents. Entity lookup, provenance, ready-made disagreement queries, and SPARQL for anything else. Full-text search is merged and ships in the next release.

rdf-kg-mcp on GitHub
New tools

genomics-mcp

Find and read genomic data from EGA, ENA, ENCODE, GEO, NCBI, ClinVar, gnomAD and your own files, one region at a time. Published synthetic-data protocols and GIAB and HPRC truth sets are merged and ship in the next release.

genomics-mcp on GitHub
CLI live

rewirebench

Runs benchmark protocols and model adapters with pinned data, splits and environments, and reports what it scored and what it cost. An MCP server and GPU backend are being built.

rewire-benchmarks on GitHub
Live

Blog

Plain-language write-ups of new models and benchmarks, and of what our reruns and investigations found.

Read the blog

For coding agents

Ask the graph from your agent

Install the database server and your agent can answer questions like these from the graph, with sources:

  • "Which DNA language models have been tested on splice variant interpretation, and where do the results disagree?"
  • "What evidence is there that GPN-MSA beats CADD on rare-disease variants, and who ran it?"
  • "List every result for Evo 2 with its dataset, split and source table."

Each server runs on your machine. The database server downloads the current public release; your questions and your data stay local.

Claude Code plugin
/plugin marketplace add rewire-bio/rewire-benchmarks
/plugin install bio-model-benchmarks@rewire-benchmarks
Database server, any MCP client
claude mcp add rewire-benchmark-graph -- uvx --from git+https://github.com/rewire-bio/rdf-kg-mcp@v0.2.0 rdf-kg-mcp serve https://benchmarks.rewirebio.io/agents/kg.tar.gz
genomics-mcp, container
docker run --rm -i \
  --mount type=bind,source=/path/to/your/data,target=/data,readonly \
  --mount type=volume,source=genomics-mcp-work,target=/work \
  ghcr.io/rewire-bio/genomics-mcp:0.1.0

Other install routes: genomics-mcp install guide ยท agent guide

Coverage

Broad coverage, deepest where it matters clinically

Deepest

DNA and RNA models

Full records, use-case rankings, run recipes we execute ourselves, and Rewire's own benchmarks as they are built: variant interpretation, splicing, regulatory variants, pathogen detection.

Deep

Protein models

Full records and published results, with disagreements tracked: variant effect, structure and design.

Broad

Everything else

Single-cell, spatial, chemistry, antibody and design models are listed with their sources and covered on the blog.

For model developers

If your model is in the graph

We contact you first

The rules list disagreements automatically, as open questions. When we investigate one involving your model and plan to write about it, we contact you before we publish.

Corrections are welcome

If a record is wrong, open an issue on rewire-benchmark-data. Corrections are versioned and the previous value stays visible.

Exact identity

Results are tied to the checkpoint that was tested. When a paper does not say which weights it used, the result stays at the model-family level.

No single score

Incompatible benchmarks are never averaged into one leaderboard. Rewire does not build or sell biological models.