Legacy DSH skill report
DSH observability
case-specific signal. Exact percentile/tie wording in the skill plus event-log analysis. Strong on large generated traces, but one task family.
22/22variants pass
1/22variants without
6/6 x2holdout with
5/6 x2holdout without
One task family, partly generated variants. Not a general observability capability claim.
Documented comparisons
legacy evidence, not sealed records| Case / batch | With skill | Without skill | Reading |
|---|---|---|---|
| 22 large variants (budget 30) | 22/22 pass | 1/22 pass | large case-family signal |
| observability-large-holdout (2618 events) | 6/6 x2 | 5/6 x2, p50 wrong | replicated holdout |
| observability-compaction-large (2221 events) | 8/8 x2 | 5/8, 6/8; p50/p95/compaction wrong | same task family |
| observability-codex-large (5000-line JSONL) | 9/9 x2 | 7/9 x2; cache_rate/uncached wrong | same family, different input shape |
Source and limitations
These numbers come from benchmarks/evaluation/TASK_LEVEL.md and benchmarks/evaluation/FINAL_REPORT.md. Raw trial records were not retained as sealed records in this rebuild; counts here are documented legacy results.
No cross-skill or cross-model aggregate is computed. The attribution report is the only page in this site backed by the current sealed evidence set.