j4k-oss/agent-skills
Watch
1
0
Fork
You've already forked agent-skills
 0
Code Issues 6 Pull requests Releases Packages Activity Actions

fix: drop finding categories from doc drift description #119

Merged
jercik merged 2 commits from fix/verify-doc-drift-description into main 2026-10-08 08:44:44 +00:00
Conversation 3 Commits 2 Files changed 1 +1 -1
jercik commented 2026-10-06 07:43:32 +00:00
Owner
Copy link

Removes the incorrect / code-drift / obvious / duplicate list from the verify-doc-drift description. The Finding categories section already defines that set, so a renamed category no longer needs a second edit to keep discovery text correct.

Follow-up to review feedback on #108, which kept this out of its scope.

🤖 Generated with Claude Code

Removes the `incorrect / code-drift / obvious / duplicate` list from the `verify-doc-drift` description. The `Finding categories` section already defines that set, so a renamed category no longer needs a second edit to keep discovery text correct. Follow-up to review feedback on [#108](https://code.j4k.dev/j4k-oss/agent-skills/pulls/108), which kept this out of its scope. 🤖 Generated with [Claude Code](https://claude.com/claude-code)
fix: drop finding categories from doc drift description
All checks were successful
commit-msg / commitlint (pull_request) Successful in 22s
Details
Node tests / node:test (pull_request) Successful in 3m6s
Details
Review / Review (pull_request_target) Successful in 10m25s
Details
ffa1502b10
 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
forgejo-actions commented 2026-10-06 07:53:56 +00:00
Copy link

Review 01M483TJS9X3BSCJ8Q4J76E14D — head aafbee014b70b0e2a8cc675ead61b1450a95cc47

Review — j4k-oss/agent-skills @ 3ad00f4038

Scope: diff against base tree 160c43773584
Status: dispatched — coverage complete (5/5 slots terminal)
Facts: current review-wide projection

Computed under:

{
  "abandonment": "abandonment-v1",
  "anchor_recipe": 1,
  "batch_policy": "batch-v1",
  "coverage": "coverage-v3",
  "dispatch_policy": "dispatch-v2",
  "grounder_version": 1,
  "grounding_read_rule": "grounding-read-v1",
  "promotion_policy": "promotion-v1",
  "report": "report-v4",
  "tally": "tally-v1",
  "triage_settle": "triage-settle-v2"
}

Findings (1)

medium — The verification gate rejects findings whose evidence is not a code contradiction

  • claim: 01M483Z2WPVW554W2V8DPRQYEV
  • anchor: skills/verify-doc-drift/SKILL.md (snippet)
Every candidate finding gets a second, independent pass that tries to **refute** it and defaults to REJECTED unless it can point at the contradicting line itself — re-derive the evidence from the code rather than trusting the first pass. This is what separates real drift from a plausible misread; skipping it ships false corrections. Only survivors reach the report.
  • lens: writing-quality · arm: default
  • verdicts: 1 valid / 0 invalid / 0 uncertain
    • pass 01M48413Z3VTR7ST16FKS34KS9 · valid: The exact-grounded gate covers every candidate, requires a contradicting line, and directs evidence to be re-derived from code. The reviewer supplies a coherent category-specific trace: the per-unit loop permits a canonical documentary location for a duplicate and the passage itself for an obvious finding. The separate exact-grounded output paragraph also explicitly permits those documentary citations. An accurate duplicated passage can satisfy that documentary evidence contract without contradicting code, so the universal gate excludes an admitted category. The reported absence of an exception is consistent with the batch; an imagined exception cannot defeat this trace. Referring to the existing per-unit evidence contract preserves independent refutation and default rejection without copying its category set. Medium is appropriate; no model reproduction is needed to establish the written conflict.
  • disposition: none

An agent following this gate literally must reject a supported duplicate or obvious finding, even though the skill explicitly requires those findings. A duplicated instruction can have a canonical documentary home without contradicting any source line; a generic engineering instruction can lack project-specific value while remaining factually true. Neither can satisfy the gate's requirement to point at “the contradicting line itself”.

“The per-unit audit loop” assigns evidence according to the finding category: a duplicate needs its canonical documentation location, and an obvious finding needs the documentary passage itself. “Adversarial verification” instead applies the contradiction requirement to every candidate and tells the verifier to re-derive evidence from code. This changes the acceptance criterion after the initial audit and excludes cases the earlier rules admit.

Replace the first sentence with: “Every candidate finding gets a second, independent pass that tries to refute it and defaults to REJECTED unless it independently establishes the evidence required by ‘The per-unit audit loop’.” This preserves independent scrutiny and rejection by default while giving each finding its specified proof standard. The writing skill's “Specify the Discipline” guidance requires checkable completion criteria, and “One Idea, One Place” favors referring to the existing evidence contract rather than introducing a competing one.

I traced “Finding categories”, “The per-unit audit loop”, “Adversarial verification”, the task steps, and “Output” in the full skill. This is a static instruction conflict; no model trial was run. The decisive check is whether a verifier can accept a demonstrably duplicated passage using the canonical document as evidence without finding a code contradiction. An explicit category-specific exception elsewhere in the loaded instructions could refute this reading; this skill supplies none.

Other claims

  • grounding-pending (0)
  • ungrounded (0)
  • rejected (1)
    • 01M483Y1FEG22VQ5VY9Z168TB7 low — The catalog description embeds the audit procedure before the skill is selected
  • duplicate-of (0)
  • unadjudicated (1)
    • 01M483ZRXHQP27KTX87WYJCAR7 medium — The output contract recopies the audit loop’s category-specific citation rules

Coverage

Coverage pass: 01M483TJWTFPMYQRSADKK99607
Accounting: complete
Slot health: healthy

lens part arm unit status runs loss
general-bug whole default no-claims 1 no
writing-quality whole default claims-emitted 1 no
test-trimming whole default no-claims 1 no
restated-sets whole default no-claims 1 no
project-docs whole default no-claims 1 no
<!-- review:summary --> **Review** `01M483TJS9X3BSCJ8Q4J76E14D` — head `aafbee014b70b0e2a8cc675ead61b1450a95cc47` # Review — j4k-oss/agent-skills @ 3ad00f4038b6 Scope: diff against base tree `160c43773584` Status: dispatched — coverage complete (5/5 slots terminal) Facts: current review-wide projection Computed under: ```json { "abandonment": "abandonment-v1", "anchor_recipe": 1, "batch_policy": "batch-v1", "coverage": "coverage-v3", "dispatch_policy": "dispatch-v2", "grounder_version": 1, "grounding_read_rule": "grounding-read-v1", "promotion_policy": "promotion-v1", "report": "report-v4", "tally": "tally-v1", "triage_settle": "triage-settle-v2" } ``` ## Findings (1) ### medium — The verification gate rejects findings whose evidence is not a code contradiction - claim: `01M483Z2WPVW554W2V8DPRQYEV` - anchor: `skills/verify-doc-drift/SKILL.md` (snippet) ``` Every candidate finding gets a second, independent pass that tries to **refute** it and defaults to REJECTED unless it can point at the contradicting line itself — re-derive the evidence from the code rather than trusting the first pass. This is what separates real drift from a plausible misread; skipping it ships false corrections. Only survivors reach the report. ``` - lens: writing-quality · arm: default - verdicts: 1 valid / 0 invalid / 0 uncertain - pass `01M48413Z3VTR7ST16FKS34KS9` · valid: The exact-grounded gate covers every candidate, requires a contradicting line, and directs evidence to be re-derived from code. The reviewer supplies a coherent category-specific trace: the per-unit loop permits a canonical documentary location for a duplicate and the passage itself for an obvious finding. The separate exact-grounded output paragraph also explicitly permits those documentary citations. An accurate duplicated passage can satisfy that documentary evidence contract without contradicting code, so the universal gate excludes an admitted category. The reported absence of an exception is consistent with the batch; an imagined exception cannot defeat this trace. Referring to the existing per-unit evidence contract preserves independent refutation and default rejection without copying its category set. Medium is appropriate; no model reproduction is needed to establish the written conflict. - disposition: none > An agent following this gate literally must reject a supported duplicate or obvious finding, even though the skill explicitly requires those findings. A duplicated instruction can have a canonical documentary home without contradicting any source line; a generic engineering instruction can lack project-specific value while remaining factually true. Neither can satisfy the gate's requirement to point at “the contradicting line itself”. > > “The per-unit audit loop” assigns evidence according to the finding category: a duplicate needs its canonical documentation location, and an obvious finding needs the documentary passage itself. “Adversarial verification” instead applies the contradiction requirement to every candidate and tells the verifier to re-derive evidence from code. This changes the acceptance criterion after the initial audit and excludes cases the earlier rules admit. > > Replace the first sentence with: “Every candidate finding gets a second, independent pass that tries to refute it and defaults to REJECTED unless it independently establishes the evidence required by ‘The per-unit audit loop’.” This preserves independent scrutiny and rejection by default while giving each finding its specified proof standard. The writing skill's “Specify the Discipline” guidance requires checkable completion criteria, and “One Idea, One Place” favors referring to the existing evidence contract rather than introducing a competing one. > > I traced “Finding categories”, “The per-unit audit loop”, “Adversarial verification”, the task steps, and “Output” in the full skill. This is a static instruction conflict; no model trial was run. The decisive check is whether a verifier can accept a demonstrably duplicated passage using the canonical document as evidence without finding a code contradiction. An explicit category-specific exception elsewhere in the loaded instructions could refute this reading; this skill supplies none. ## Other claims - grounding-pending (0) - ungrounded (0) - rejected (1) - `01M483Y1FEG22VQ5VY9Z168TB7` low — The catalog description embeds the audit procedure before the skill is selected - duplicate-of (0) - unadjudicated (1) - `01M483ZRXHQP27KTX87WYJCAR7` medium — The output contract recopies the audit loop’s category-specific citation rules ## Coverage Coverage pass: 01M483TJWTFPMYQRSADKK99607 Accounting: complete Slot health: healthy | lens | part | arm | unit status | runs | loss | | --- | --- | --- | --- | --- | --- | | general-bug | whole | default | no-claims | 1 | no | | writing-quality | whole | default | claims-emitted | 1 | no | | test-trimming | whole | default | no-claims | 1 | no | | restated-sets | whole | default | no-claims | 1 | no | | project-docs | whole | default | no-claims | 1 | no |
🚀 1
chore: merge main into fix/verify-doc-drift-description
All checks were successful
commit-msg / commitlint (pull_request) Successful in 16s
Details
Node tests / node:test (pull_request) Successful in 2m27s
Details
Review / Review (pull_request_target) Successful in 6m23s
Details
aafbee014b
 
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
jercik commented 2026-10-07 11:57:08 +00:00
Author
Owner
Copy link

Replying to review summary comment #133482 (head aafbee0)

Both claims describe text this PR does not touch, so they do not make this PR harmful to merge. Both are tracked in #121.

  • 01M483Z2WPVW554W2V8DPRQYEV (medium, verification gate): confirmed. "Adversarial verification" requires a contradicting line, which a duplicate or obvious finding cannot supply. Fixed in #121 by pointing the gate at the evidence "The per-unit audit loop" requires.
  • 01M483ZRXHQP27KTX87WYJCAR7 (medium, unadjudicated): confirmed. "Output" restates the per-category citation rules from the audit loop. #121 replaces the restatement with a pointer.
  • 01M483Y1FEG22VQ5VY9Z168TB7 (low, rejected by the service): no action; this PR already removes the category list from the description.
> Replying to review summary comment #133482 (head aafbee0) Both claims describe text this PR does not touch, so they do not make this PR harmful to merge. Both are tracked in https://code.j4k.dev/j4k-oss/agent-skills/pulls/121. - `01M483Z2WPVW554W2V8DPRQYEV` (medium, verification gate): confirmed. "Adversarial verification" requires a contradicting line, which a `duplicate` or `obvious` finding cannot supply. Fixed in #121 by pointing the gate at the evidence "The per-unit audit loop" requires. - `01M483ZRXHQP27KTX87WYJCAR7` (medium, unadjudicated): confirmed. "Output" restates the per-category citation rules from the audit loop. #121 replaces the restatement with a pointer. - `01M483Y1FEG22VQ5VY9Z168TB7` (low, rejected by the service): no action; this PR already removes the category list from the description.
jercik commented 2026-10-07 22:00:42 +00:00
Author
Owner
Copy link

Replying to review 01M483TJS9X3BSCJ8Q4J76E14D on head aafbee0

Conclusions on the report's claims:

  • medium, verification gate rejects findings whose evidence is not a code contradiction (01M483Z2WPVW554W2V8DPRQYEV): already fixed on main in 7669945 (#124), which makes the gate require the citation the finding's category needs. This PR does not touch that sentence, and a trial merge of this head with main is clean.
  • unadjudicated, medium, the output contract recopies the audit loop's category-specific citation rules (01M483ZRXHQP27KTX87WYJCAR7): confirmed, the Output section restates the same per-category citation list. This PR does not introduce it, so it is tracked in #127.
  • rejected, low, the catalog description embeds the audit procedure (01M483Y1FEG22VQ5VY9Z168TB7): the review service already rejected it; no action.
> Replying to review `01M483TJS9X3BSCJ8Q4J76E14D` on head `aafbee0` Conclusions on the report's claims: - **medium, verification gate rejects findings whose evidence is not a code contradiction** (`01M483Z2WPVW554W2V8DPRQYEV`): already fixed on `main` in 7669945 (#124), which makes the gate require the citation the finding's category needs. This PR does not touch that sentence, and a trial merge of this head with `main` is clean. - **unadjudicated, medium, the output contract recopies the audit loop's category-specific citation rules** (`01M483ZRXHQP27KTX87WYJCAR7`): confirmed, the `Output` section restates the same per-category citation list. This PR does not introduce it, so it is tracked in https://code.j4k.dev/j4k-oss/agent-skills/pulls/127. - **rejected, low, the catalog description embeds the audit procedure** (`01M483Y1FEG22VQ5VY9Z168TB7`): the review service already rejected it; no action.
jercik merged commit 664392a48b into main 2026-10-08 08:44:44 +00:00
jercik deleted branch fix/verify-doc-drift-description 2026-10-08 08:44:44 +00:00
Sign in to join this conversation.
Reviewers
No reviewers
Labels
Clear labels
No items
No labels
Milestone
Clear milestone
No items
No milestone
Assignees
Clear assignees
No assignees
2 participants
Notifications
Due date
The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set

Reference
j4k-oss/agent-skills!119
No description provided.