fix: refutation pass should re-derive evidence of any category #122
Loading…
Reference in a new issue
No description provided.
Delete branch "fix/verify-doc-drift-verification-gate"
Deleting a branch is permanent. Although the deleted branch may continue to exist for a short time before it actually gets removed, it CANNOT be undone in most cases. Continue?
The refutation pass in
verify-doc-driftis told to re-derive the evidence "from the code", butduplicateandobviousfindings are cited from documentation. The sentence now says "re-derive that evidence", which defers to the category's citation requirement that main already names.Follow-up to a review finding on #110, which did not introduce the line. #124 fixed the first half of the sentence; this PR fixes the rest. I did not run an audit to measure how often a model resolves the conflict on its own.
🤖 Generated with Claude Code
Review
01M4DR47JSGD8XFTJTFAMH7S7J— head3dd961929db6c66329103c0cf20afafc3e372780Review — j4k-oss/agent-skills @
8bece749d7Scope: diff against base tree
8e004ccf90c2Status: dispatched — coverage complete (3/3 slots terminal)
Facts: current review-wide projection
Computed under:
Findings (1)
low — The catalog description repeats the audit intent and includes workflow detail
01M4DR8EQ4172TVEDX6JA2F51Hskills/verify-doc-drift/SKILL.md(snippet)01M4DR9H2MR6TCC5S4H4XD4W95· valid: Grounded exact snippet shows the description opening with a task summary ('Audit a repository's documentation against its source code and fix the factual drift') before 'Use when', then restating the same intent under 'Use when' and again as a 'Triggers on' phrase list. The installed skill-packaging reference states a capability description is a routing rule, begins with 'Use when …', and does not describe the skill's method or contents; it also notes the description consumes catalog context on every load. The reviewer reports no user-only invocation policy, so the capability contract applies. The proposed replacement keeps the routing intent (verify docs match source, audit for factual drift); 'repository documentation' covers READMEs/CONTEXT.md/ADRs/standards, and each dropped literal phrase ('verify docs', 'doc drift', 'do the docs match the code', etc.) is a paraphrase a model would route to the same intent, so no match is lost. The fix-vs-report direction lives in the body, which the reviewer says handles the report-only override. The packaging guidance permits but does not require literal phrases, so dropping them breaks no constraint. Low severity is appropriate.Other claims
Coverage
Coverage pass: 01M4DR4M5EFWFQ23XN1HSN0N60
Accounting: complete
Slot health: healthy
Closing as obsolete. Main already contains the equivalent change: #124 (
7669945, "fix: doc drift verification should accept each category's citation") rewrote the same sentence inskills/verify-doc-drift/SKILL.md's "Adversarial verification" section so the refutation pass defaults to REJECTED unless it reproduces the citation the finding's category requires. This PR's rewording of that line now conflicts with main and would add nothing.Reopening: the close was a mistake. #124 rewrote the sentence in the "Adversarial verification" section, but main still lacks this PR's other change:
skills/verify-doc-drift/SKILL.mdline 37 still tells the refutation pass to "re-derive the evidence from the code rather than trusting the first pass", which this PR replaces with fresh evidence that satisfies the citation requirement in "The per-unit audit loop". The next sweep pass will trim this PR to the hunks main still needs.fix: verification gate should accept documentary evidenceto fix: refutation pass should re-derive evidence of any categoryPushed
d9f1187: merged main and trimmed this PR to the one phrase main still lacks. #124 already made the refutation pass cite the category's required evidence; the remaining "re-derive the evidence from the code" is now "re-derive that evidence", since duplicate and obvious findings are cited from docs. The earlier review of9e60e4creported no findings, so there was nothing to settle.No push for this pass: the change is still needed (main line 37 of
skills/verify-doc-drift/SKILL.mdstill says "re-derive the evidence from the code") and the trial merge withmainis clean.Finding
01M4DBC94GDDGY33EQ7HRGWSKN(description says every finding is "verified against the code"): valid, but not introduced here. #124 already madeobviousandduplicatefindings cite documentation, so the description was already inconsistent on main. Tracked in #128, which removes the method summary from the description.Unadjudicated claim
01M4DBCNZT1W72QA03FBWWCB7T(the adversarial pass "describes survivors as real drift"): not a defect. The sentence before it already makes the refutation pass default to REJECTED unless the category's citation reproduces, so the "real drift" phrasing is a gloss and does not change what a pass does. No change.Pushed
3dd9619: merged main (which moved the sentence again in #126) and re-applied the one phrase main still lacks. Line 37 ofskills/verify-doc-drift/SKILL.mdnow says "re-derive that evidence" instead of "re-derive the evidence from the code", sinceobviousandduplicatefindings are cited from docs. The earlier trial merge was clean; #126 made it conflict.Finding
01M4DBC94GDDGY33EQ7HRGWSKN(description says findings are "verified against the code"): already fixed on main by #128, which removed the method summary from the description. No change in this PR.Unadjudicated claim
01M4DBCNZT1W72QA03FBWWCB7T: not a defect, as recorded earlier; the refutation pass defaults to REJECTED unless the category's citation reproduces.No push for this pass: the review of
3dd9619is complete with no unadjudicated claims, and the trial merge withmainis clean.Finding
01M4DR8EQ4172TVEDX6JA2F51H(theverify-doc-driftdescription repeats the audit intent and opens with a task summary): valid, but this PR does not touch the description. Tracked in #129, which replaces it with a single "Use when" sentence.