MaintainAll / Evaluation
Legacy DSH skill report

DSH observability

case-specific signal. Exact percentile/tie wording in the skill plus event-log analysis. Strong on large generated traces, but one task family.

22/22variants pass
1/22variants without
6/6 x2holdout with
5/6 x2holdout without
One task family, partly generated variants. Not a general observability capability claim.

Documented comparisons

legacy evidence, not sealed records
0255075100variants pass: 100variants pass: 4.5variants passholdout: 100holdout: 83.3holdoutcompaction: 100compaction: 68.8compactioncodex JSONL: 100codex JSONL: 77.8codex JSONLwith skillwithout skill
Case / batchWith skillWithout skillReading
22 large variants (budget 30)22/22 pass1/22 passlarge case-family signal
observability-large-holdout (2618 events)6/6 x25/6 x2, p50 wrongreplicated holdout
observability-compaction-large (2221 events)8/8 x25/8, 6/8; p50/p95/compaction wrongsame task family
observability-codex-large (5000-line JSONL)9/9 x27/9 x2; cache_rate/uncached wrongsame family, different input shape

Source and limitations

These numbers come from benchmarks/evaluation/TASK_LEVEL.md and benchmarks/evaluation/FINAL_REPORT.md. Raw trial records were not retained as sealed records in this rebuild; counts here are documented legacy results.

No cross-skill or cross-model aggregate is computed. The attribution report is the only page in this site backed by the current sealed evidence set.