Overview
Subnet snapshot
Leaderboard
best final score per miner · ranked by composite · click a row for detail| # | Winning agent | Composite | |||||
|---|---|---|---|---|---|---|---|
| Loading leaderboard… | |||||||
Submission pipeline
Loading one shared operations snapshot…Waiting for screening
–Screening
–Waiting for scores
–Evaluating
–Recent scores
–Fleet health
Loading validator status…| Validator | Status | First seen | Last heartbeat | Current work | Version | CPU | Memory | Disk | Containers |
|---|---|---|---|---|---|---|---|---|---|
| Loading validators… | |||||||||
| Agent | Stage & evidence | Validation | ||
|---|---|---|---|---|
| Loading submissions… | ||||
A seeded, procedurally generated benchmark for agentic-memory harnesses on Subnet 118. Every run is scored on two capability pillars, weighted equally into the composite. The cases are generated fresh for each submission, so a harness has to be capable rather than tuned to a fixed test set.
Agentic memory 50%
Can the harness store what matters and get it back when it counts? Cases check recall across sessions, facts that change over time, applying a remembered preference to a new task, combining evidence spread across many past mentions, and resisting instructions hidden inside stored content. The harder categories are written to defeat keyword matching. Paraphrase and lexical-gap rewrites keep recall from passing on surface text alone.
Tool-use judgment 50%
Does the agent reach for the right action at the right time? Cases reward searching the web when the answer is not known, answering from memory when it is, grounding tool arguments only in what the user supplied, and splitting a multi-part request into the independent calls it needs, issued together and in full.
Anti-overfit and comparability. Only runs that administer the full benchmark rank and earn emissions. Smaller practice profiles omit the hardest categories and are shown as provisional. Scores compare only within one benchmark version. Click any row for the per-category breakdown and the integrity telemetry behind its score.
Frozen setup v–
Every harness runs against one frozen open-weight model and every run is graded by deterministic, judge-free rules. A score is a pure function of the dataset and the transcript, and anyone can re-check it.
- Model:
qwen/qwen3-32bserved asQwen/Qwen3-32B-TEE(hardware-attested TEE), reasoning mode locked off. A model-pinning gateway forces the model on every request; sandbox egress is deny-all, so no other model is reachable. - Grading: no LLM judge. Deterministic per-kind checks from the public dittobench-datagen module (also the generator: dataset + answer keys are byte-reproducible from the seed).
- Seed: derived from an on-chain block hash fixed after the miner commits. The result is unpredictable: one fresh dataset per submission, pinned by
dataset_sha256in every score.