Benchmark database
Browse models, benchmarks and use cases. Each result links to its source, and to the checkpoint when the paper names one, with its review status.
benchmarks.rewirebio.ioIndependent benchmarks for biological AI
Published benchmark results for DNA, RNA, protein and cell models, linked to their sources and to the models, datasets and protocols they tested. Where papers disagree, the graph says so.
Release 2026-11-20-72316db0764b
What is in the graph
Each record is a node and each relationship is an edge: which checkpoint was evaluated, on which dataset subset, under which protocol, with which metric, reported in which table of which paper. You can follow any number back to its source in a few hops.
RDF with OWL 2 RL reasoning and SHACL validation, using DCAT, schema.org, OBI, ECO, SEPIO and the ML Schema vocabularies. 2.45 million asserted facts, 300,000 inferred, and 15,000 derived facts about disagreements.
Why a graph
When two studies rank the same pair of models in opposite orders, or one result sits far from all the others, the graph flags it as a lead. The current release flags 197, and 191 are still open. Each is a question to investigate, not a verdict.
In October 2026 an AlphaGenome developer told us a splicing result in a recent preprint could not be right. We traced the preprint's scripts and found an off-by-one error. Read alone, that comparison would have misled anyone choosing a splicing model. Against the rest of the graph, it stood out.
Read the investigation| Source | Metric | ESM-1b | EVE |
|---|---|---|---|
| Brandes 2023, variants scored by all methods | ROC-AUC | 0.892 | 0.883 |
| Brandes 2023, ClinVar head to head | ROC-AUC | 0.906 | 0.885 |
| ProteinGym clinical benchmark | ranks EVE above ESM-1b | ||
| Livesey and Marsh 2025 | ranks EVE above ESM-1b | ||
Brandes 2023 ranks ESM-1b above EVE in 6 of 8 comparisons on ClinVar and HGMD variants, with 2 ties. Two other sources rank EVE above ESM-1b. Our review found a difference in metric, not an error: Brandes 2023 pools all variants into one ROC-AUC, and ProteinGym averages ROC-AUC per gene. See the lead
How it fits together
A scheduled agent scans every day for new models, benchmarks and datasets. Reviewed records become a versioned release of the graph. Coding agents query that graph, and can rerun a benchmark recipe on your machine and check the result against the published value. So far that is one recipe: Rewire's own MFASS v2 baseline, on CPU.
Each recipe run writes a receipt. Nothing in the plugin exports or submits a result.
What you can use
Browse models, benchmarks and use cases. Each result links to its source, and to the checkpoint when the paper names one, with its review status.
benchmarks.rewirebio.ioConflicts between papers, found by rules over the graph on every release, with the studies on each side.
See the disagreementsThe graph for coding agents. Entity lookup, provenance, ready-made disagreement queries, and SPARQL for anything else. Full-text search was added in v0.3.0.
rdf-kg-mcp on GitHubFind and read genomic data from EGA, ENA, ENCODE, GEO, NCBI, ClinVar, gnomAD and your own files, one region at a time. A catalogue of published synthetic-data protocols and GIAB and HPRC truth sets is merged. The Claude Code plugin installs a commit that has it. The 0.1.0 container does not.
genomics-mcp on GitHubRuns benchmark protocols and model adapters with pinned data and splits. Recipes, their receipts and an MCP server are on main. The latest release, v0.5.0, has none of them. A recipe run writes a receipt: what ran, on what hardware, what it scored and what it cost. The MCP server lets a coding agent plan and run recipes on CPU. GPU runs are not built: the code plans and prices a RunPod or Spark run and starts neither.
rewire-benchmarks on GitHubPlain-language write-ups of new models and benchmarks, and of what our reruns and investigations found.
Read the blogFor coding agents
Install the database server and your agent can answer questions like these from the graph, with sources:
Each server runs on your machine. The database server downloads the current public release; your questions and your data stay local.
/plugin marketplace add rewire-bio/rewire-benchmarks /plugin install bio-model-benchmarks@rewire-benchmarks
claude mcp add rewire-benchmark-graph -- uvx --from git+https://github.com/rewire-bio/rdf-kg-mcp@v0.4.0 rdf-kg-mcp serve https://benchmarks.rewirebio.io/agents/kg.tar.gz
docker run --rm -i \ --mount type=bind,source=/path/to/your/data,target=/data,readonly \ --mount type=volume,source=genomics-mcp-work,target=/work \ ghcr.io/rewire-bio/genomics-mcp:0.1.0
Other install routes: genomics-mcp install guide ยท agent guide. The release is also served as JSON with no key: API docs.
Coverage
Full records, use-case rankings, run recipes, one of which we have executed ourselves so far (the MFASS v2 baseline), and Rewire's own benchmarks as they are built: variant interpretation, splicing, regulatory variants, pathogen detection.
Full records and published results, with disagreements tracked: variant effect, structure and design.
Single-cell, spatial, chemistry, antibody and design models are listed with their sources and covered on the blog.
For model developers
The rules list disagreements automatically, as open questions. When we investigate one involving your model and plan to write about it, we contact you before we publish.
If a record is wrong, open an issue on rewire-benchmark-data. Corrections are versioned and the previous value stays visible.
Results are tied to the checkpoint that was tested. When a paper does not say which weights it used, the result stays at the model-family level.
Incompatible benchmarks are never averaged into one leaderboard. Rewire does not build or sell biological models.
From the blog