Skip to content

Commit bb95342

Browse files
authored
Merge pull request #219 from mojoatomic/docs/health-gating-215
docs: health gating enablement and calibration (#215)
2 parents ef9bf32 + 0ae7df4 commit bb95342

File tree

3 files changed

+63
-21
lines changed

3 files changed

+63
-21
lines changed

README.md

Lines changed: 19 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -33,7 +33,7 @@ Running `init` creates:
3333

3434
### CI-safe setup (recommended)
3535

36-
Add scripts to your package.json:
36+
Add scripts to your package.json (init will add these automatically):
3737

3838
```json
3939
{
@@ -47,15 +47,28 @@ Add scripts to your package.json:
4747
}
4848
```
4949

50-
GitHub Actions example:
50+
GitHub Actions options:
5151

52+
A) Scaffold with the CLI
53+
```bash
54+
npx eslint-plugin-ai-code-snifftest init --ci
55+
```
56+
57+
B) Reusable workflow
5258
```yaml
53-
- name: Lint (non-blocking)
54-
run: npm run lint:ci
55-
- name: Ratchet (blocks on regression)
56-
run: npm run ci:ratchet
59+
name: ci-ratchet
60+
on: [push, pull_request]
61+
jobs:
62+
ratchet-and-tests:
63+
uses: mojoatomic/eslint-plugin-ai-code-snifftest/.github/workflows/ratchet-reusable.yml@main
64+
with:
65+
node-version: '20'
5766
```
5867
68+
Branch protection
69+
- Require status check: ratchet-and-tests
70+
- Optional: also require your primary test job (e.g., Test on Node.js 20.x)
71+
5972
### Requirements
6073
- Node.js 18+
6174
- ESLint 9+

docs/CONTEXT-AWARE-LINTING.md

Lines changed: 3 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -30,9 +30,9 @@ Configuration (optional)
3030
}
3131

3232
Promoting telemetry to a required PR check
33-
1) Burn‑in: keep telemetry job non‑blocking for a few PRs; review reported signals and health deltas in CI.
34-
2) Branch protection: mark the telemetry job (ratchet-context) as “required” in repository settings; optionally remove continue-on-error.
35-
3) (Optional) Health gating: after calibration, enable policy to block when overall health decreases. This can be added as a future enhancement (tracked in issue) and controlled via config (e.g., ratchet.health.gate with minDelta).
33+
- Prefer enabling health gating (see HEALTH-GATING.md) via the `ratchet-and-tests` job.
34+
- Burn‑in: keep `health.enabled=false`, review telemetry; then flip to true and require the job in branch protection.
35+
- Use `init --ci` or the reusable workflow to wire CI quickly.
3636

3737
Notes
3838
- Support scripts under scripts/**/*.js are excluded from complexity/architecture and plugin-complexity rules, keeping product metrics accurate.

docs/HEALTH-GATING.md

Lines changed: 41 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -17,16 +17,16 @@ Configuration (.ai-coding-guide.json)
1717
{
1818
"ratchet": {
1919
"health": {
20-
"enabled": false,
21-
"gateOn": "overall",
22-
"minOverall": 70,
23-
"intentOverrides": {
20+
"enabled": false,
21+
"gateOn": "overall",
22+
"minOverall": 70,
23+
"intentOverrides": {
2424
"refactoring": { "minOverall": 65 }
2525
},
26-
"bypass": {
27-
"label": "health-bypass",
26+
"bypass": {
27+
"label": "health-bypass",
2828
"commitToken": "[health-bypass]",
29-
"maxBypassPerPR": 1
29+
"maxBypassPerPR": 1
3030
},
3131
"failureMessage": "Code health decreased below threshold. See telemetry output (intent, density, health)."
3232
}
@@ -38,14 +38,43 @@ Notes
3838
- intent detection: CLI accepts --refactoring or --intent=refactoring (best-effort; extend later).
3939
- bypass: commit messages containing the commitToken or env HEALTH_BYPASS=true will bypass; label reserved for future CI integration.
4040

41-
CI recommendation
42-
- Keep the context-aware step non-blocking during burn-in; monitor health scores in PRs.
43-
- Once calibrated, flip health.enabled to true and remove continue-on-error; mark the job required in branch protection.
41+
## Enable in CI
42+
43+
Option A: Scaffold workflow with the CLI
44+
```bash
45+
npx eslint-plugin-ai-code-snifftest init --ci
46+
```
47+
This creates `.github/workflows/ci-ratchet.yml` with a `ratchet-and-tests` job.
48+
49+
Option B: Reusable workflow (no files needed)
50+
```yaml
51+
name: ci-ratchet
52+
on: [push, pull_request]
53+
jobs:
54+
ratchet-and-tests:
55+
uses: mojoatomic/eslint-plugin-ai-code-snifftest/.github/workflows/ratchet-reusable.yml@main
56+
with:
57+
node-version: '20'
58+
```
59+
60+
Branch protection (GitHub → Settings → Branches → main)
61+
- Require status checks: `ratchet-and-tests`
62+
- Optional: also require your test matrix job (e.g., `Test on Node.js 20.x`)
63+
64+
## Calibration (burn‑in → enforce)
65+
1) Burn‑in: keep `health.enabled=false` and watch the reported scores in PRs.
66+
2) Choose thresholds: set `gateOn` and `minOverall` based on observed ranges.
67+
3) Flip the switch: set `health.enabled=true` to enforce. The CI job will fail if score < threshold.
4468

4569
Troubleshooting
4670
- No lines.executable in analysis: health falls back to physical lines; check analyzer wiring.
4771
- Unexpected gate failures: confirm thresholds, gateOn, and any intent override are set as intended; use a one-time bypass with the commit token for urgent fixes.
4872

73+
Failure message anatomy
74+
- HEALTH-GATE FAIL: <message>
75+
- gateOn, threshold, actual, intent printed for quick diagnosis
76+
- Example suggestions printed by the ratchet script for common rules
77+
4978
## Health Score Calculation
5079

5180
### Formula
@@ -69,12 +98,12 @@ overall: 3 // Balanced (all categories)
6998
// Good code
7099
executableLOC = 5000
71100
violations.structural = 10
72-
// density = 10 / 5 = 2.0; score (structural) = 100 - (2.0 * 5) = 90 ✅
101+
// density = 10 / 5 = 2.0; score (structural) = 90 ✅
73102
74103
// Poor code
75104
executableLOC = 5000
76105
violations.structural = 50
77-
// density = 50 / 5 = 10.0; score (structural) = 100 - (10.0 * 5) = 50 ❌
106+
// density = 50 / 5 = 10.0; score (structural) = 50 ❌
78107
79108
// Refactoring intent (relaxed overall)
80109
executableLOC = 5000

0 commit comments

Comments
 (0)