MaintainAll / Evaluation
Benchmark index

Evaluated skills

Each report is separate; the evidence level is labelled per card, and scores from different suites are never combined.

DSH skillruntime / sandboxfrozen

DSH sandbox attribution

Does a skill make the agent name the sandbox layer that hid a GPU or /dev object, and cite verifiable evidence?

6/6 (100%)B evidence
0/5 (0%)A0 evidence
6/6 (100%)B semantic
11 sealed records118 dataset casescase11 / case09 / case08
Open detailed report →
Legacy suiteart creationpublished

Art creation

Artifact quality of art-creation skill arms, with previous/candidate comparisons and original case evidence.

92.7%previous quality
80.5%candidate quality
38trials
3 reports12 case contextslegacy atom scale
Open detailed report →
Legacy DSH skillcase-specific signal

DSH observability

Exact percentile/tie wording in the skill plus event-log analysis. Strong on large generated traces, but one task family.

22/22variants pass
1/22variants without
6/6 x2holdout with
documented legacy evidence
Open legacy report →
Legacy DSH skilldisabledcontrols only

DSH plugin validation

Large plugin audit cases are useful controls; no stable correctness separation from the without arm.

8/8 bothplugin audit
7/7 bothsupplychain
7/7 bothroute-authz
documented legacy evidence
Open legacy report →
Legacy DSH skilldisablednull / unstable

DSH runtime guide

Sandbox-awareness and true-resume are controls; runtime-upgrade behavior is candidate-unstable, not a guarantee.

6/6 bothsandbox awareness
6/6 bothtrue-resume
6/6 → 5/6runtime-upgrade
documented legacy evidence
Open legacy report →
Scales: DSH = semantic + evidence; art/legacy DSH = native legacy scales. Not comparable.

All reports

SuiteReportRecords / trialsPrimary resultLink
dsh-skillsSandbox attribution11 sealed recordsB evidence 6/6 (100%); A0 0/5 (0%)dsh-sandbox-attribution/
art-creationArt creation38 legacy trials / 3 reportscandidate 80.5% vs previous 92.7% qualityart-creation/
dsh-skillsDSH observabilitylegacy documentedcase-specific signaldsh-observability/
dsh-skillsDSH plugin validationlegacy documenteddisableddsh-plugin-validation/
dsh-skillsDSH runtime guidelegacy documenteddisableddsh-runtime-guide/

Generated from sealed records and public legacy bundles; no model calls. Source: benchmarks/evaluation/.