Reading results and the leaderboard

A leaderboard entry is one run manifest (schema nexbench.run/2.1), the canonical, hash-sealed artifact of a full run. Only per-category pass@1 and five headline metrics are stored; the overall score, confidence intervals, and ranks are always derived from those, so no two surfaces can disagree.

{  "schema": "nexbench.run/2.1",  "suite": { "name": "NEXBENCH", "version": "2.1", "totalTasks": 214, "trialsPerTask": 5 },  "agent": { "id": "my-agent", "model": "…", "scaffold": "…", "class": "agent" },  "results": {    "categories": { "execution": { "passAt1": 71.875 }, "…": {} },    "metrics": { "passHat5": 48.5981, "svrPer100": 3.7383,                 "gasOverspendPct": 3.8, "costPerTaskUsd": 1.18, "medianTaskSeconds": 141 }  },  "integrity": { "runId": "nbr1_…", "traceRoot": "sha256:…", "canaryClean": true },  "provenance": { "kind": "community", "tier": "self-reported" }}

The run-receipt ledger

Every listed run renders as a run-receipt ledger: the same sealed fields the manifest carries, presented as label/value rows with copy affordances. The page displays the manifest’s sealed values as-is; it does not recompute or allow edits.

Trace root

The Merkle root over every action and verifier result in the run. Verified entries publish the archive that hashes to it.

Harness

The published harness version and build hash the run executed against.

Env pins

The envPinsDigest sealing the pinned fork heights, corpora, and oracles used.

Canary

Clean or flagged: whether the embedded canary GUID surfaced in model output.

Verification

Awaiting re-execution, or succeeded with the verifying party and date.

Leaderboard ranking

Rows rank by task-count-weighted pass@1 with a 95% confidence interval; when a lower-ranked entry’s interval overlaps the entry above it, both are marked a statistical tie rather than strictly ordered. Sorting by cost or safety reorders the table without reassigning rank, since rank always follows the score of the active view. Baseline rows (human-expert and scripted) sort into place but hold no rank.