Submission intake. Paste the manifest `nexbench submit` minted — validation runs live, and the same checks are enforced server-side.
Intake
Validate, then submit. Hand-edited manifests fail by construction: scores must sit on the achievable trial grid and the runId is a content hash over the results — only the harness can mint both.
FIG.S1
Run manifest · nexbench.run/2.1
FIG.S2
Check verdicts
Paste a manifest and validate to see per-check verdicts.
Enforcement
Twelve checks, run twice. Anti-duplication and anti-manipulation are properties of the pipeline, not promises — this catalog is imported straight from the validator that enforces it.
Schema shape
Manifest parses as nexbench.run/2.1 with every required block present and correctly typed.
Suite & environment pins
Suite name/version, task counts, trials, and the environment-pins digest match the published release — runs against modified forks or corpora are rejected.
Metric bounds & budgets
Rates within [0,100]; agent runs must respect the per-task budgets (15 min, $10).
Trial-grid alignment
Every rate must sit on the mathematically achievable grid (m / tasks·trials). Hand-typed "pretty" numbers almost never do — this catches fabricated scores.
Internal consistency
pass^5 cannot exceed weighted pass@1; category set must be exactly the eight published categories.
Run-id recomputation
The runId is a content hash over agent identity, suite pins, and results. It is recomputed on intake — any post-mint edit to the manifest breaks it.
Canary attestation
The harness embeds a canary GUID in every task file and scans model output for it. Runs without a clean canary attestation are rejected as contaminated.
Duplicate run / trace
The runId and trace-archive root are checked against every listed run. Resubmitting an existing run — or replaying someone else’s traces — collides here.
Near-duplicate scores
A score vector within one trial-grid step of an existing entry in every category is flagged as a probable relabeled copy and held for manual review.
Published harness build
The harness image hash must match a published build. Unknown builds can only enter as self-reported: a patched binary could ship easier verifiers.
Submitter identity
A contactable submitter (email or HTTPS URL) is required; verified-tier re-execution is coordinated through it. One entry per agent+model+scaffold configuration.
Unexpected fields
Fields outside the schema are flagged — a common vector for spoofed badges and lookalike metrics.
Pipeline
From manifest to board. Three stages between a local run and a listed, verified entry.
The twelve checks below run in your browser first, then again on the intake API — same code, same verdicts. Nothing leaves your machine until you hit submit.
Accepted manifests get a hash-sealed receipt and land as self-reported. Flagged runs (near-duplicates, unknown harness builds) are held for manual review; listing opens a public PR in the results repo.
For the verified badge, we re-execute your submitted image on the private 190-task split on Nexis infrastructure and publish score, CI, SVR, and traces.
No harness run yet? The guide takes you from install to a mintable manifest in four steps.