report
Part of the ACED plugin — see that page for install instructions.
Trigger: the user wants a project-wide health summary of all ACED eval suites — pass rates, trends, and which agent configurations need attention
Generates a project-wide health dashboard across all eval suites.
What it does
Section titled “What it does”- Scans
artifacts/specs/for subdirectories containingeval.md. If none are found, reports that no eval suites are initialized and suggestssdd:start-mission. - For each suite, reads
eval.md(target, threshold), the most recentresults/file, and the second-most-recent for trend. - Computes pass rate, mean score, pass-rate delta vs. previous run, and the worst-scoring failing case.
- Classifies each suite’s health:
| Status | Condition |
|---|---|
healthy |
Pass rate ≥ 90% |
degraded |
Pass rate 70–89% |
critical |
Pass rate < 70% |
trending-down |
Pass rate dropped ≥ 10% vs. previous run |
no-data |
No results yet |
- Reports a table of all suites plus a “needs attention” list, and can print full failing-case detail for a specific suite on request.
Next actions suggested
Section titled “Next actions suggested”critical/trending-down→ runimprovedegraded→ runrunfor details, thenimproveno-data→ runrunhealthywith no trend data → runadd-scenarioto expand coverage