fix: doc drift audits should load the ADR rules they use #108

jercik · 2026-10-03T21:40:31Z

jercik commented

2026-10-03 21:40:31 +00:00

Addresses the audit dependency and copied ADR policy findings from #98 by loading project-docs before applying its classification and maintenance rules. This independently targets main and retains the audit's user decision gate and proposed-ADR protections.

Addresses the audit dependency and copied ADR policy findings from [#98](https://code.j4k.dev/j4k-oss/agent-skills/pulls/98) by loading `project-docs` before applying its classification and maintenance rules. This independently targets `main` and retains the audit's user decision gate and proposed-ADR protections.

jercik added 1 commit

2026-10-03 21:40:31 +00:00

fix: doc drift audits should load the ADR rules they use

commit-msg / commitlint (pull_request) Successful in 28s

Details

Review / Review (pull_request_target) Successful in 3m51s

Details

Node tests / node:test (pull_request) Successful in 4m25s

Details

42bd8e021f

forgejo-actions commented

2026-10-03 21:44:21 +00:00

Review 01M41X6KT55JM31ND1K7BJTQ2J — head 38f4c063e9d39b85386d4ed992068c5959cc0505

Review — j4k-oss/agent-skills @ `487267f5f2`

Scope: diff against base tree 2034125557e6
Status: dispatched — coverage complete (4/4 slots terminal)
Facts: current review-wide projection

Computed under:

{
  "abandonment": "abandonment-v1",
  "anchor_recipe": 1,
  "batch_policy": "batch-v1",
  "coverage": "coverage-v3",
  "dispatch_policy": "dispatch-v2",
  "grounder_version": 1,
  "grounding_read_rule": "grounding-read-v1",
  "promotion_policy": "promotion-v1",
  "report": "report-v3",
  "tally": "tally-v1",
  "triage_settle": "triage-settle-v2"
}

Findings (1)

high — The routing description copies the finding-category set

claim: 01M41XD1ASCVCBH3M2SCJNSGRX
anchor: skills/verify-doc-drift/SKILL.md (snippet)

Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth).

lens: writing-quality · arm: default
verdicts: 1 valid / 0 invalid / 0 uncertain
disposition: none

I read the complete verify-doc-drift skill, its new project-docs dependency, and the writing standard's skill-packaging guidance. The Finding categories section in skills/verify-doc-drift/SKILL.md defines the set with entries incorrect, code-drift, obvious, and duplicate; the frontmatter description repeats those members here. A new or renamed category would require a second edit to the discovery description, so an agent can receive a stale list before loading the body. The packaging guidance says a capability description is a routing rule rather than a summary, and the writing standard says to keep one fact in one home. Remove the category enumeration from the description and leave category definitions in Finding categories; retain the user-intent triggers so discovery still works. The decisive check is whether any machine-readable schema or other source defines this set instead. I found no such source in the scoped skill; its own category section is the definition.

Other claims

grounding-pending (0)
ungrounded (0)
rejected (0)
duplicate-of (0)
unadjudicated (0)

Coverage

Coverage pass: 01M41X6KXAHEZCR637KHX11XPV
Accounting: complete
Slot health: healthy

lens	part	arm	unit status	runs	loss
general-bug	whole	default	no-claims	1	no
writing-quality	whole	default	claims-emitted	1	no
test-trimming	whole	default	no-claims	1	no
restated-sets	whole	default	no-claims	1	no

**Review** `01M41X6KT55JM31ND1K7BJTQ2J` — head `38f4c063e9d39b85386d4ed992068c5959cc0505` # Review — j4k-oss/agent-skills @ 487267f5f268 Scope: diff against base tree `2034125557e6` Status: dispatched — coverage complete (4/4 slots terminal) Facts: current review-wide projection Computed under: ```json { "abandonment": "abandonment-v1", "anchor_recipe": 1, "batch_policy": "batch-v1", "coverage": "coverage-v3", "dispatch_policy": "dispatch-v2", "grounder_version": 1, "grounding_read_rule": "grounding-read-v1", "promotion_policy": "promotion-v1", "report": "report-v3", "tally": "tally-v1", "triage_settle": "triage-settle-v2" } ``` ## Findings (1) ### high — The routing description copies the finding-category set - claim: `01M41XD1ASCVCBH3M2SCJNSGRX` - anchor: `skills/verify-doc-drift/SKILL.md` (snippet) ``` Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth). ``` - lens: writing-quality · arm: default - verdicts: 1 valid / 0 invalid / 0 uncertain - disposition: none > I read the complete `verify-doc-drift` skill, its new `project-docs` dependency, and the writing standard's skill-packaging guidance. The `Finding categories` section in `skills/verify-doc-drift/SKILL.md` defines the set with entries `incorrect`, `code-drift`, `obvious`, and `duplicate`; the frontmatter description repeats those members here. A new or renamed category would require a second edit to the discovery description, so an agent can receive a stale list before loading the body. The packaging guidance says a capability description is a routing rule rather than a summary, and the writing standard says to keep one fact in one home. Remove the category enumeration from the description and leave category definitions in `Finding categories`; retain the user-intent triggers so discovery still works. The decisive check is whether any machine-readable schema or other source defines this set instead. I found no such source in the scoped skill; its own category section is the definition. ## Other claims - grounding-pending (0) - ungrounded (0) - rejected (0) - duplicate-of (0) - unadjudicated (0) ## Coverage Coverage pass: 01M41X6KXAHEZCR637KHX11XPV Accounting: complete Slot health: healthy | lens | part | arm | unit status | runs | loss | | --- | --- | --- | --- | --- | --- | | general-bug | whole | default | no-claims | 1 | no | | writing-quality | whole | default | claims-emitted | 1 | no | | test-trimming | whole | default | no-claims | 1 | no | | restated-sets | whole | default | no-claims | 1 | no |

🚀 1

forgejo-actions reviewed

2026-10-03 21:44:21 +00:00

skills/verify-doc-drift/SKILL.md

					
				@ -1,6 +1,8 @@

				---

				name: verify-doc-drift

				description: Audit a repository's documentation against its actual source code and fix the factual drift — wrong ports/flags/scope names/env vars, stale or non-compiling examples, phantom routes, superseded claims, plus duplicate and obsolete docs. Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth). Use when the user wants to verify documentation matches the code, audit docs for accuracy, hunt doc-vs-code drift, or check that READMEs / CONTEXT.md / ADRs / standards are still true. Triggers on "verify docs", "audit documentation", "doc drift", "do the docs match the code", "check the docs against the code", "are the docs still accurate".

high — Description restates the finding-category set

I read the complete verify-doc-drift skill and its routing description. The authoritative category definitions are in skills/verify-doc-drift/SKILL.md under Finding categories, with entries incorrect, code-drift, obvious, and duplicate. The description copies every member. A future category edit can leave the discovery description silently wrong, giving an agent conflicting instructions before it loads the skill body. The writing standard says to keep one fact in one place and make a capability description a routing rule rather than a summary. Remove the category enumeration and audit-method summary from the description; keep a short Use when clause naming documentation-versus-code accuracy requests. This preserves discoverability while leaving the category definitions at their source. The set and the duplicate are observable in this file; no runtime reproduction was needed.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M41VF2WCHY13RBJP8V2B78TG of review 01M41VCDVX5GGH2GC4T5SES658

**high** — Description restates the finding-category set > I read the complete verify-doc-drift skill and its routing description. The authoritative category definitions are in skills/verify-doc-drift/SKILL.md under Finding categories, with entries incorrect, code-drift, obvious, and duplicate. The description copies every member. A future category edit can leave the discovery description silently wrong, giving an agent conflicting instructions before it loads the skill body. The writing standard says to keep one fact in one place and make a capability description a routing rule rather than a summary. Remove the category enumeration and audit-method summary from the description; keep a short Use when clause naming documentation-versus-code accuracy requests. This preserves discoverability while leaving the category definitions at their source. The set and the duplicate are observable in this file; no runtime reproduction was needed. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M41VF2WCHY13RBJP8V2B78TG` of review `01M41VCDVX5GGH2GC4T5SES658`

👍 1

The description is byte-for-byte unchanged from this PR’s main base e7d02b520e. This task authorizes only the ADR status/action deduplication from #98 comments 111106/111107 and the project-docs dependency/invocation from comment 111109. The proposed description routing rewrite is recorded as deferred outside that scope; its validity is not disputed. Can you acknowledge this bounded deferral? No description change or additional PR is authorized in this task.

The description is byte-for-byte unchanged from this PR’s main base e7d02b520e58f0f41803193e983012270fec38ec. This task authorizes only the ADR status/action deduplication from #98 comments 111106/111107 and the project-docs dependency/invocation from comment 111109. The proposed description routing rewrite is recorded as deferred outside that scope; its validity is not disputed. Can you acknowledge this bounded deferral? No description change or additional PR is authorized in this task.

superseded by review 01M41X6KT55JM31ND1K7BJTQ2J for head 38f4c063e9d39b85386d4ed992068c5959cc0505

superseded by review `01M41X6KT55JM31ND1K7BJTQ2J` for head `38f4c063e9d39b85386d4ed992068c5959cc0505`

🚀 1

Valid. Addressed in follow-up PR #119, which removes the category list from the description; this PR keeps its scope.

Valid. Addressed in follow-up PR https://code.j4k.dev/j4k-oss/agent-skills/pulls/119, which removes the category list from the description; this PR keeps its scope.

jercik marked this conversation as resolved

jercik referenced this pull request

2026-10-03 21:45:52 +00:00

fix(skills): record settled ADR decisions #98

jercik referenced this pull request

2026-10-03 21:45:57 +00:00

fix(skills): record settled ADR decisions #98

jercik referenced this pull request

2026-10-03 21:46:01 +00:00

fix(skills): record settled ADR decisions #98

jercik added 1 commit

2026-10-03 22:12:17 +00:00

fix: integrate merged canonical ADR rules

commit-msg / commitlint (pull_request) Successful in 30s

Details

Node tests / node:test (pull_request) Successful in 4m26s

Details

Review / Review (pull_request_target) Successful in 5m37s

Details

38f4c063e9

forgejo-actions reviewed

2026-10-03 22:17:54 +00:00

skills/verify-doc-drift/SKILL.md

					
				@ -1,6 +1,8 @@

				---

				name: verify-doc-drift

				description: Audit a repository's documentation against its actual source code and fix the factual drift — wrong ports/flags/scope names/env vars, stale or non-compiling examples, phantom routes, superseded claims, plus duplicate and obsolete docs. Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth). Use when the user wants to verify documentation matches the code, audit docs for accuracy, hunt doc-vs-code drift, or check that READMEs / CONTEXT.md / ADRs / standards are still true. Triggers on "verify docs", "audit documentation", "doc drift", "do the docs match the code", "check the docs against the code", "are the docs still accurate".

high — The routing description copies the finding-category set

I read the complete verify-doc-drift skill, its new project-docs dependency, and the writing standard's skill-packaging guidance. The Finding categories section in skills/verify-doc-drift/SKILL.md defines the set with entries incorrect, code-drift, obvious, and duplicate; the frontmatter description repeats those members here. A new or renamed category would require a second edit to the discovery description, so an agent can receive a stale list before loading the body. The packaging guidance says a capability description is a routing rule rather than a summary, and the writing standard says to keep one fact in one home. Remove the category enumeration from the description and leave category definitions in Finding categories; retain the user-intent triggers so discovery still works. The decisive check is whether any machine-readable schema or other source defines this set instead. I found no such source in the scoped skill; its own category section is the definition.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M41XD1ASCVCBH3M2SCJNSGRX of review 01M41X6KT55JM31ND1K7BJTQ2J

**high** — The routing description copies the finding-category set > I read the complete `verify-doc-drift` skill, its new `project-docs` dependency, and the writing standard's skill-packaging guidance. The `Finding categories` section in `skills/verify-doc-drift/SKILL.md` defines the set with entries `incorrect`, `code-drift`, `obvious`, and `duplicate`; the frontmatter description repeats those members here. A new or renamed category would require a second edit to the discovery description, so an agent can receive a stale list before loading the body. The packaging guidance says a capability description is a routing rule rather than a summary, and the writing standard says to keep one fact in one home. Remove the category enumeration from the description and leave category definitions in `Finding categories`; retain the user-intent triggers so discovery still works. The decisive check is whether any machine-readable schema or other source defines this set instead. I found no such source in the scoped skill; its own category section is the definition. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M41XD1ASCVCBH3M2SCJNSGRX` of review `01M41X6KT55JM31ND1K7BJTQ2J`

👍 1

This repeats the description-routing proposal from comment 111299. The description and finding-category definitions are unchanged from merged main b93e79a3. The authorized scope is only canonical project-docs delivery/invocation and deduplication of ADR status and maintenance actions, preserving unique audit rules. A description rewrite is outside that bounded integration. Can this proposal remain deferred for separate user scope rather than expand this PR? No description change is made, and this conversation remains open.

This repeats the description-routing proposal from comment 111299. The description and finding-category definitions are unchanged from merged main b93e79a3. The authorized scope is only canonical project-docs delivery/invocation and deduplication of ADR status and maintenance actions, preserving unique audit rules. A description rewrite is outside that bounded integration. Can this proposal remain deferred for separate user scope rather than expand this PR? No description change is made, and this conversation remains open.

Valid. Addressed in follow-up PR #119, which removes the category list from the description; this PR keeps its scope.

Valid. Addressed in follow-up PR https://code.j4k.dev/j4k-oss/agent-skills/pulls/119, which removes the category list from the description; this PR keeps its scope.

jercik marked this conversation as resolved

jercik referenced this pull request

2026-10-06 07:43:32 +00:00

fix: drop finding categories from doc drift description #119

jercik merged commit 63bbd90b14 into main

2026-10-06 07:44:12 +00:00

jercik deleted branch fix/verify-doc-drift-canonical-adr

2026-10-06 07:44:12 +00:00

jercik referenced this pull request from a commit

2026-10-06 07:44:14 +00:00

fix: doc drift audits should load the ADR rules they use (#108)

jercik referenced this pull request