25

NEXBENCH overview

NEXBENCH3 min read

The reproducible benchmark for autonomous Web3 agents: what it measures, why it exists, and how it fits the platform.

NEXBENCH measures whether an autonomous agent can act competently and safely where real economic value is at stake: execute transactions, route swaps, bridge funds, manage DeFi positions, research tokens, catch drainers, reconstruct portfolios, and run treasury governance. That means 214 tasks across 8 categories, 5 trials each, run against deterministic pinned environments: forked mainnets, a frozen web corpus, and an adversarial honeypot net.

Generic agent benchmarks do not confront the properties that make Web3 evaluation hard: actions are consequential and often irreversible, the environment is adversarial by construction (drainers, scam tokens, malicious approvals are the norm), on-chain and market data are moving targets, and agent scaffolds train on public data, so contamination is an ongoing threat. NEXBENCH scores task success and safety jointly: a hard safety violation zeros the trial regardless of whether the task was otherwise completed.

Two design commitments

  • Programmatic verifiers, not LLM judges. Every task is graded by asserting on post-run chain state, balances, event logs, or gold numeric answers. Deterministic, no self-preference bias.
  • Tamper-evident by construction. Every result is a hash-sealed run manifest. Scores must sit on a mathematically achievable grid, the run id is a content hash that recomputes on intake, and trace archives are Merkle-rooted.

The trust model

Three mechanisms establish trust. A scripted reference agent establishes the solvability floor: it solves every runnable task, so a learned agent that fails a task cannot blame the environment. A twelve-check intake validator runs identically in the browser, the CLI, and the server, so a manifest either clears the exact same bar everywhere or it does not clear it at all. Verified-tier entries are re-executed by Nexis on infrastructure the submitter does not control.

Relationship to the platform

Nexis uses NEXBENCH to validate its own agents, and the same suite serves as a public leaderboard: reference entries such as the platform agent stack are listed alongside third-party and community submissions under the same schema and the same checks. The web surface at /benchmarks renders the identical hash-sealed manifests the CLI produces; every entry is minted from a results draft.

FactValue
SuiteNEXBENCH 2.1 (schema nexbench.run/2.1)
Tasks / categories214 tasks across 8 categories, 5 trials per task
Public split24 public tasks; 190 held out, rotate quarterly
Packagenexbench on npm, zero runtime dependencies
LicenseApache-2.0 (harness and task interface are open source)
Leaderboardnex-t1.ai/benchmarks

Was this page helpful?

Edit on GitHub