NEXBENCH overview
NEXBENCH3 min read
The reproducible benchmark for autonomous Web3 agents: what it measures, why it exists, and how it fits the platform.
NEXBENCH measures whether an autonomous agent can act competently and safely where real economic value is at stake: execute transactions, route swaps, bridge funds, manage DeFi positions, research tokens, catch drainers, reconstruct portfolios, and run treasury governance. That means 214 tasks across 8 categories, 5 trials each, run against deterministic pinned environments: forked mainnets, a frozen web corpus, and an adversarial honeypot net.
Generic agent benchmarks do not confront the properties that make Web3 evaluation hard: actions are consequential and often irreversible, the environment is adversarial by construction (drainers, scam tokens, malicious approvals are the norm), on-chain and market data are moving targets, and agent scaffolds train on public data, so contamination is an ongoing threat. NEXBENCH scores task success and safety jointly: a hard safety violation zeros the trial regardless of whether the task was otherwise completed.
Two design commitments
- Programmatic verifiers, not LLM judges. Every task is graded by asserting on post-run chain state, balances, event logs, or gold numeric answers. Deterministic, no self-preference bias.
- Tamper-evident by construction. Every result is a hash-sealed run manifest. Scores must sit on a mathematically achievable grid, the run id is a content hash that recomputes on intake, and trace archives are Merkle-rooted.
The trust model
Three mechanisms establish trust. A scripted reference agent establishes the solvability floor: it solves every runnable task, so a learned agent that fails a task cannot blame the environment. A twelve-check intake validator runs identically in the browser, the CLI, and the server, so a manifest either clears the exact same bar everywhere or it does not clear it at all. Verified-tier entries are re-executed by Nexis on infrastructure the submitter does not control.
Relationship to the platform
Nexis uses NEXBENCH to validate its own agents, and the same suite serves as a public leaderboard: reference entries such as the platform agent stack are listed alongside third-party and community submissions under the same schema and the same checks. The web surface at /benchmarks renders the identical hash-sealed manifests the CLI produces; every entry is minted from a results draft.
| Fact | Value |
|---|---|
| Suite | NEXBENCH 2.1 (schema nexbench.run/2.1) |
| Tasks / categories | 214 tasks across 8 categories, 5 trials per task |
| Public split | 24 public tasks; 190 held out, rotate quarterly |
| Package | nexbench on npm, zero runtime dependencies |
| License | Apache-2.0 (harness and task interface are open source) |
| Leaderboard | nex-t1.ai/benchmarks |
Was this page helpful?
Edit on GitHub