fix(skills): source pointers should match the documented release #109

jercik · 2026-10-03T23:39:07Z

jercik commented

2026-10-03 23:39:07 +00:00

Source pointers in verify-doc-drift and verify-readme now match the documented release: audits read the source for that release and record a proof gap when it is unavailable, and pointers and npx commands use the release or pin to it.

Addresses the release-pointer concession in #97. Stacked children #112 and #113 target this branch and must merge after it.

🤖 Generated with Claude Code

Source pointers in `verify-doc-drift` and `verify-readme` now match the documented release: audits read the source for that release and record a proof gap when it is unavailable, and pointers and `npx` commands use the release or pin to it. Addresses the release-pointer concession in [#97](https://code.j4k.dev/j4k-oss/agent-skills/pulls/97#issuecomment-104038). Stacked children #112 and #113 target this branch and must merge after it. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

jercik added 1 commit

2026-10-03 23:39:07 +00:00

fix(skills): source pointers should match the documented release

commit-msg / commitlint (pull_request) Successful in 28s

Details

Review / Review (pull_request_target) Successful in 3m6s

Details

Node tests / node:test (pull_request) Has been cancelled

Details

7f3e350a7e

jercik referenced this pull request

2026-10-03 23:39:45 +00:00

fix(skills): point CLI docs and drift fixes at sources instead of restating sets #97

forgejo-actions commented

2026-10-03 23:42:13 +00:00

Review 01M482RJE4FVN83JWAJBTNJJHX — head b9c64c5a516403d1551a9e84c532a86d9bd125e3

Review — j4k-oss/agent-skills @ `457b2484ee`

Scope: diff against base tree bc96a7769926
Status: dispatched — coverage complete (5/5 slots terminal)
Facts: current review-wide projection

Computed under:

{
  "abandonment": "abandonment-v1",
  "anchor_recipe": 1,
  "batch_policy": "batch-v1",
  "coverage": "coverage-v3",
  "dispatch_policy": "dispatch-v2",
  "grounder_version": 1,
  "grounding_read_rule": "grounding-read-v1",
  "promotion_policy": "promotion-v1",
  "report": "report-v4",
  "tally": "tally-v1",
  "triage_settle": "triage-settle-v2"
}

Findings (3)

medium — The catalog description duplicates the finding categories and audit procedure

claim: 01M48322GSZKFX3930YSW01KZ4
anchor: skills/verify-doc-drift/SKILL.md (snippet)

description: Audit a repository's documentation against its actual source code and fix the factual drift — wrong ports/flags/scope names/env vars, stale or non-compiling examples, phantom routes, superseded claims, plus duplicate and obsolete docs. Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth). Use when the user wants to verify documentation matches the code, audit docs for accuracy, hunt doc-vs-code drift, or check that READMEs / CONTEXT.md / ADRs / standards are still true. Triggers on "verify docs", "audit documentation", "doc drift", "do the docs match the code", "check the docs against the code", "are the docs still accurate".

lens: writing-quality · arm: default
verdicts: 1 valid / 0 invalid / 0 uncertain
- pass 01M4D6FPZ209XKBHC71AKHQEJS · valid: The exact-grounded description lists finding categories and explains verification and repair before its routing condition. The reviewer identifies Finding categories in this same anchored file, skills/verify-doc-drift/SKILL.md, as the source and supplies the identical members: incorrect, code-drift, obvious, and duplicate. No set exception applies, so the matching copy is a medium defect. Independently, the installed packaging standard requires capability descriptions to begin with Use when and excludes method summaries. The replacement preserves the documentation-audit selection intent and explicitly leaves the category definitions and audit procedure in the body. Reassessment of the earlier claim reveals no concrete refutation in its verdict or disposition.
disposition: none

The skill's persistent catalog entry repeats definitions that already have a home in its loaded body. Changing a finding category now requires updating this description as well as the category definition, and every catalog reader receives the audit procedure before selecting the skill.

The source is the Finding categories section in this file. It defines incorrect, code-drift, obvious, and duplicate. The description copies incorrect, code-drift, obvious, and duplicate; neither list has a member absent from the other. It also describes verification and repair mechanics that the body explains. The restated-sets guideline requires a pointer rather than a matching copy. The installed writing-for-agents packaging reference says a capability description is a routing rule beginning with 'Use when', and must not describe its method or contents.

Replace the description with: 'Use when the user asks to audit documentation against source code, check whether docs are still accurate, or find and fix documentation drift.' Keep the category definitions and audit procedure in the body. This preserves the matching intent while removing the category copy and material that loads before it is needed.

I read the complete skill, its declared project-docs dependency, and the repository README's catalog-delivery contract. This is a static comparison; I did not run a catalog client. The identical category lists establish the duplication at this revision. A delivery contract that withheld descriptions until after selection would refute the catalog-loading cost, but not the independently maintained category copy.

medium — The verification gate rejects findings that have no code contradiction

claim: 01M4832KS2Q3EG6P3YPEKCYJH4
anchor: skills/verify-doc-drift/SKILL.md (snippet)

Every candidate finding gets a second, independent pass that tries to **refute** it and defaults to REJECTED unless it can point at the contradicting line itself — re-derive the evidence from the code rather than trusting the first pass. This is what separates real drift from a plausible misread; skipping it ships false corrections. Only survivors reach the report.

lens: writing-quality · arm: default
verdicts: 1 valid / 0 invalid / 0 uncertain
- pass 01M4D6FPZ209XKBHC71AKHQEJS · valid: The exact-grounded gate applies to every candidate, demands a contradicting line, and requires evidence re-derived from code. The reviewer supplies a coherent instruction trace: duplicate findings can concern accurate facts repeated across documents, obvious findings can concern unnecessary prose, and the per-unit loop accepts documentary evidence for those categories. Such findings need not contradict any code line, so the universal gate conflicts with the reported category-specific evidence contract. The proposed replacement retains independent refutation and requires independently established evidence through the existing per-unit loop; it does not weaken verification into accepting unsupported assertions. Medium is appropriate. The earlier adjudication offers no contrary evidence, and its reasoning remains supported on reassessment.
disposition: none

Valid findings about redundant or unnecessary documentation cannot pass this gate unless the verifier invents a source-code contradiction. An accurate fact repeated in another document has no contradicting code line, so following this instruction rejects the very duplication the skill is meant to remove.

Finding categories defines a duplicate as the same fact repeated in another doc, and an obvious finding as a generic fact without project-specific value. The per-unit audit loop explicitly accepts the canonical document location as evidence for duplication and the passage itself for unnecessary prose. This paragraph instead says every candidate defaults to REJECTED unless the verifier can point at 'the contradicting line itself' and must re-derive evidence 'from the code'. Those requirements fit factual drift but not all permitted findings. The writing-for-agents standard requires precise, operational constraints and removal of conflicting rules.

Replace the first sentence with: 'Give every candidate an independent pass that tries to refute it. Reject it unless that pass independently establishes the evidence required by the per-unit audit loop.' Keep the requirement that only survivors reach the report. This preserves adversarial checking without imposing the wrong kind of evidence.

I compared the complete Finding categories, per-unit audit loop, Adversarial verification, Task, and Output sections. This is a static instruction trace, not an executed agent audit. A verifier faced with accurate duplicate wording would establish the contradiction between the two gates; an explicit instruction exempting such findings from the contradicting-code requirement would refute it. No such exemption appears in this skill.

low — The README skill description puts audit mechanics in persistent catalog context

claim: 01M4833TVNV1CNR4Q35ABVV693
anchor: skills/verify-readme/SKILL.md (snippet)

description: Audit, rewrite, or draft a repository's README.md — CLI tool, library, application, config/dotfiles repo, or monorepo. Applies a universal section order and cross-checks the one-liner against `package.json#description`. Use when the user wants to review, improve, rewrite, or write a README, or mentions "verify readme", "review readme", "rewrite readme", "write readme", "README quality", or "check my README".

lens: writing-quality · arm: default
verdicts: 1 valid / 0 invalid / 0 uncertain
- pass 01M4D6FPZ209XKBHC71AKHQEJS · valid: The exact-grounded description leads with an audit summary and includes the universal section order and package.json cross-check before Use when. The installed packaging standard requires a capability description to start with Use when and describe matching requests rather than its method. The reviewer reports a capability delivery contract and a calling skill, consistent with the grounded trigger-based description. The replacement retains audit, improvement, rewriting, drafting, and quality-checking intents. Removing the method summary and synonymous literal triggers loses no necessary routing distinction; the correction explicitly retains the operational instructions in the body. Low severity fits. The prior adjudication supplies no concrete refutation; its earlier suggested type enumeration is not needed in the current correction.
disposition: none

Every initial skill-catalog reader receives implementation details that are useful only after choosing this skill. 'Applies a universal section order and cross-checks the one-liner' does not distinguish which request should select it; those instructions already live in the body. This adds recurring catalog context and gives the routing condition a later position.

The repository README says capability descriptions are persistent 'Use when' routing instructions, while bodies load after selection. This skill is also a declared dependency of verify-unixy-cli, which calls it for README auditing. The installed writing-for-agents packaging reference requires capability descriptions to begin with 'Use when' and describe matching intents rather than methods or contents.

Use: 'Use when the user asks to audit, improve, rewrite, or draft a repository README, including checking its quality.' Keep the section-order and description-parity instructions in their body sections. The replacement preserves the selection intent without loading the audit method into every catalog.

I read the complete skill, the repository README's delivery explanation, and verify-unixy-cli's dependency declaration and ARG5 reference. This is a static packaging review. Their capability delivery and invocation contracts establish the applicable rule; evidence that this were an exclusively human-invoked workflow would require a different description standard.

Other claims

grounding-pending (0)
ungrounded (0)
rejected (0)
duplicate-of (0)
unadjudicated (2)
- 01M48338ASS1Q5BAYCK31H9QSJ medium — Quick Start and Usage repeat the project-type set defined in Inputs
- 01M4835B3J6CYXS7FMAGZHYY7Q medium — The catalog description copies the project-type set defined in Inputs

Coverage

Coverage pass: 01M482RJFP4ACC17AA0XND7S4W
Accounting: complete
Slot health: healthy

lens	part	arm	unit status	runs	loss
general-bug	whole	default	no-claims	1	no
writing-quality	whole	default	claims-emitted	1	no
test-trimming	whole	default	no-claims	1	no
restated-sets	whole	default	no-claims	1	no
project-docs	whole	default	no-claims	1	no

**Review** `01M482RJE4FVN83JWAJBTNJJHX` — head `b9c64c5a516403d1551a9e84c532a86d9bd125e3` # Review — j4k-oss/agent-skills @ 457b2484ee2a Scope: diff against base tree `bc96a7769926` Status: dispatched — coverage complete (5/5 slots terminal) Facts: current review-wide projection Computed under: ```json { "abandonment": "abandonment-v1", "anchor_recipe": 1, "batch_policy": "batch-v1", "coverage": "coverage-v3", "dispatch_policy": "dispatch-v2", "grounder_version": 1, "grounding_read_rule": "grounding-read-v1", "promotion_policy": "promotion-v1", "report": "report-v4", "tally": "tally-v1", "triage_settle": "triage-settle-v2" } ``` ## Findings (3) ### medium — The catalog description duplicates the finding categories and audit procedure - claim: `01M48322GSZKFX3930YSW01KZ4` - anchor: `skills/verify-doc-drift/SKILL.md` (snippet) ``` description: Audit a repository's documentation against its actual source code and fix the factual drift — wrong ports/flags/scope names/env vars, stale or non-compiling examples, phantom routes, superseded claims, plus duplicate and obsolete docs. Each finding is categorized (incorrect / code-drift / obvious / duplicate) and adversarially verified against the code before it is reported, then docs are fixed to match the code (the reverse only when the doc is the intended source of truth). Use when the user wants to verify documentation matches the code, audit docs for accuracy, hunt doc-vs-code drift, or check that READMEs / CONTEXT.md / ADRs / standards are still true. Triggers on "verify docs", "audit documentation", "doc drift", "do the docs match the code", "check the docs against the code", "are the docs still accurate". ``` - lens: writing-quality · arm: default - verdicts: 1 valid / 0 invalid / 0 uncertain - pass `01M4D6FPZ209XKBHC71AKHQEJS` · valid: The exact-grounded description lists finding categories and explains verification and repair before its routing condition. The reviewer identifies Finding categories in this same anchored file, skills/verify-doc-drift/SKILL.md, as the source and supplies the identical members: incorrect, code-drift, obvious, and duplicate. No set exception applies, so the matching copy is a medium defect. Independently, the installed packaging standard requires capability descriptions to begin with Use when and excludes method summaries. The replacement preserves the documentation-audit selection intent and explicitly leaves the category definitions and audit procedure in the body. Reassessment of the earlier claim reveals no concrete refutation in its verdict or disposition. - disposition: none > The skill's persistent catalog entry repeats definitions that already have a home in its loaded body. Changing a finding category now requires updating this description as well as the category definition, and every catalog reader receives the audit procedure before selecting the skill. > > The source is the Finding categories section in this file. It defines incorrect, code-drift, obvious, and duplicate. The description copies incorrect, code-drift, obvious, and duplicate; neither list has a member absent from the other. It also describes verification and repair mechanics that the body explains. The restated-sets guideline requires a pointer rather than a matching copy. The installed writing-for-agents packaging reference says a capability description is a routing rule beginning with 'Use when', and must not describe its method or contents. > > Replace the description with: 'Use when the user asks to audit documentation against source code, check whether docs are still accurate, or find and fix documentation drift.' Keep the category definitions and audit procedure in the body. This preserves the matching intent while removing the category copy and material that loads before it is needed. > > I read the complete skill, its declared project-docs dependency, and the repository README's catalog-delivery contract. This is a static comparison; I did not run a catalog client. The identical category lists establish the duplication at this revision. A delivery contract that withheld descriptions until after selection would refute the catalog-loading cost, but not the independently maintained category copy. ### medium — The verification gate rejects findings that have no code contradiction - claim: `01M4832KS2Q3EG6P3YPEKCYJH4` - anchor: `skills/verify-doc-drift/SKILL.md` (snippet) ``` Every candidate finding gets a second, independent pass that tries to **refute** it and defaults to REJECTED unless it can point at the contradicting line itself — re-derive the evidence from the code rather than trusting the first pass. This is what separates real drift from a plausible misread; skipping it ships false corrections. Only survivors reach the report. ``` - lens: writing-quality · arm: default - verdicts: 1 valid / 0 invalid / 0 uncertain - pass `01M4D6FPZ209XKBHC71AKHQEJS` · valid: The exact-grounded gate applies to every candidate, demands a contradicting line, and requires evidence re-derived from code. The reviewer supplies a coherent instruction trace: duplicate findings can concern accurate facts repeated across documents, obvious findings can concern unnecessary prose, and the per-unit loop accepts documentary evidence for those categories. Such findings need not contradict any code line, so the universal gate conflicts with the reported category-specific evidence contract. The proposed replacement retains independent refutation and requires independently established evidence through the existing per-unit loop; it does not weaken verification into accepting unsupported assertions. Medium is appropriate. The earlier adjudication offers no contrary evidence, and its reasoning remains supported on reassessment. - disposition: none > Valid findings about redundant or unnecessary documentation cannot pass this gate unless the verifier invents a source-code contradiction. An accurate fact repeated in another document has no contradicting code line, so following this instruction rejects the very duplication the skill is meant to remove. > > Finding categories defines a duplicate as the same fact repeated in another doc, and an obvious finding as a generic fact without project-specific value. The per-unit audit loop explicitly accepts the canonical document location as evidence for duplication and the passage itself for unnecessary prose. This paragraph instead says every candidate defaults to REJECTED unless the verifier can point at 'the contradicting line itself' and must re-derive evidence 'from the code'. Those requirements fit factual drift but not all permitted findings. The writing-for-agents standard requires precise, operational constraints and removal of conflicting rules. > > Replace the first sentence with: 'Give every candidate an independent pass that tries to refute it. Reject it unless that pass independently establishes the evidence required by the per-unit audit loop.' Keep the requirement that only survivors reach the report. This preserves adversarial checking without imposing the wrong kind of evidence. > > I compared the complete Finding categories, per-unit audit loop, Adversarial verification, Task, and Output sections. This is a static instruction trace, not an executed agent audit. A verifier faced with accurate duplicate wording would establish the contradiction between the two gates; an explicit instruction exempting such findings from the contradicting-code requirement would refute it. No such exemption appears in this skill. ### low — The README skill description puts audit mechanics in persistent catalog context - claim: `01M4833TVNV1CNR4Q35ABVV693` - anchor: `skills/verify-readme/SKILL.md` (snippet) ``` description: Audit, rewrite, or draft a repository's README.md — CLI tool, library, application, config/dotfiles repo, or monorepo. Applies a universal section order and cross-checks the one-liner against `package.json#description`. Use when the user wants to review, improve, rewrite, or write a README, or mentions "verify readme", "review readme", "rewrite readme", "write readme", "README quality", or "check my README". ``` - lens: writing-quality · arm: default - verdicts: 1 valid / 0 invalid / 0 uncertain - pass `01M4D6FPZ209XKBHC71AKHQEJS` · valid: The exact-grounded description leads with an audit summary and includes the universal section order and package.json cross-check before Use when. The installed packaging standard requires a capability description to start with Use when and describe matching requests rather than its method. The reviewer reports a capability delivery contract and a calling skill, consistent with the grounded trigger-based description. The replacement retains audit, improvement, rewriting, drafting, and quality-checking intents. Removing the method summary and synonymous literal triggers loses no necessary routing distinction; the correction explicitly retains the operational instructions in the body. Low severity fits. The prior adjudication supplies no concrete refutation; its earlier suggested type enumeration is not needed in the current correction. - disposition: none > Every initial skill-catalog reader receives implementation details that are useful only after choosing this skill. 'Applies a universal section order and cross-checks the one-liner' does not distinguish which request should select it; those instructions already live in the body. This adds recurring catalog context and gives the routing condition a later position. > > The repository README says capability descriptions are persistent 'Use when' routing instructions, while bodies load after selection. This skill is also a declared dependency of verify-unixy-cli, which calls it for README auditing. The installed writing-for-agents packaging reference requires capability descriptions to begin with 'Use when' and describe matching intents rather than methods or contents. > > Use: 'Use when the user asks to audit, improve, rewrite, or draft a repository README, including checking its quality.' Keep the section-order and description-parity instructions in their body sections. The replacement preserves the selection intent without loading the audit method into every catalog. > > I read the complete skill, the repository README's delivery explanation, and verify-unixy-cli's dependency declaration and ARG5 reference. This is a static packaging review. Their capability delivery and invocation contracts establish the applicable rule; evidence that this were an exclusively human-invoked workflow would require a different description standard. ## Other claims - grounding-pending (0) - ungrounded (0) - rejected (0) - duplicate-of (0) - unadjudicated (2) - `01M48338ASS1Q5BAYCK31H9QSJ` medium — Quick Start and Usage repeat the project-type set defined in Inputs - `01M4835B3J6CYXS7FMAGZHYY7Q` medium — The catalog description copies the project-type set defined in Inputs ## Coverage Coverage pass: 01M482RJFP4ACC17AA0XND7S4W Accounting: complete Slot health: healthy | lens | part | arm | unit status | runs | loss | | --- | --- | --- | --- | --- | --- | | general-bug | whole | default | no-claims | 1 | no | | writing-quality | whole | default | claims-emitted | 1 | no | | test-trimming | whole | default | no-claims | 1 | no | | restated-sets | whole | default | no-claims | 1 | no | | project-docs | whole | default | no-claims | 1 | no |

🚀 1

forgejo-actions reviewed

2026-10-03 23:42:13 +00:00

skills/verify-doc-drift/SKILL.md Outdated

					
				@ -41,3 +41,3 @@

				Replace drifted text that restates a set the code defines with a pointer to the source that defines it; correcting the copy only restarts the clock on the next drift. Text restates a set when it lists every member, or states how many there are, of a set that a file, directory, schema, or command defines, such as the flags `--help` prints or the keys a config schema declares.

				Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path.

				Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release. Living documentation can point to a living source.

low — "Living documentation can point to a living source" fails the no-op test and adds two undefined terms

Examined: the 'Restated sets: point instead of recopying' section of skills/verify-doc-drift/SKILL.md, specifically the paragraph this change extends. It now reads: 'Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release. Living documentation can point to a living source.'

Problem: the last sentence is permissive ('can'), and what it permits is already the default. The first sentence names unpinned sources, and the second sentence limits pinning to versioned documentation. An agent fixing drift in documentation that isn't versioned would already point at the current source, so deleting the sentence changes nothing it does. That fails the writing skill's no-op test ('would the models this content serves already behave correctly without this line? Delete it if yes'). The sentence also adds two terms the skill never defines, 'living documentation' and 'living source'. The skill says to 'use one term for one concept, and define project-local terms on first use'. A reader may then wonder whether 'living' names a third category alongside 'versioned' and 'unpublished', which the sentence doesn't settle.

Correction: delete 'Living documentation can point to a living source.' Keep the versioned-documentation sentence, which carries the only new instruction (pin to the documented release). If the author wants the boundary stated, add a short definition to that sentence instead, for example 'For documentation that describes a specific release, such as a versioned docs site, ...'. Deleting the sentence loses nothing, because the preceding sentence and its scope already imply the unpinned default.

Evidence: static reading of the paragraph. A grep of the tree for 'living' finds no other definition of the term in the skills. The same sentence appears in skills/verify-readme/SKILL.md, which is reported separately.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M4227SSC8TVZNGKZ90RNFJFE of review 01M4225KGGVJNP0XQCW31H2RVG

**low** — "Living documentation can point to a living source" fails the no-op test and adds two undefined terms > Examined: the 'Restated sets: point instead of recopying' section of skills/verify-doc-drift/SKILL.md, specifically the paragraph this change extends. It now reads: 'Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release. Living documentation can point to a living source.' > > Problem: the last sentence is permissive ('can'), and what it permits is already the default. The first sentence names unpinned sources, and the second sentence limits pinning to versioned documentation. An agent fixing drift in documentation that isn't versioned would already point at the current source, so deleting the sentence changes nothing it does. That fails the writing skill's no-op test ('would the models this content serves already behave correctly without this line? Delete it if yes'). The sentence also adds two terms the skill never defines, 'living documentation' and 'living source'. The skill says to 'use one term for one concept, and define project-local terms on first use'. A reader may then wonder whether 'living' names a third category alongside 'versioned' and 'unpublished', which the sentence doesn't settle. > > Correction: delete 'Living documentation can point to a living source.' Keep the versioned-documentation sentence, which carries the only new instruction (pin to the documented release). If the author wants the boundary stated, add a short definition to that sentence instead, for example 'For documentation that describes a specific release, such as a versioned docs site, ...'. Deleting the sentence loses nothing, because the preceding sentence and its scope already imply the unpinned default. > > Evidence: static reading of the paragraph. A grep of the tree for 'living' finds no other definition of the term in the skills. The same sentence appears in skills/verify-readme/SKILL.md, which is reported separately. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M4227SSC8TVZNGKZ90RNFJFE` of review `01M4225KGGVJNP0XQCW31H2RVG`

👍 1

low — verify-doc-drift's new versioned-docs sentence says to pin a "hosted docs URL", but the file's list of allowed sources never includes hosted docs URLs

What I examined: the changed paragraph in skills/verify-doc-drift/SKILL.md ("Restated sets: point instead of recopying") and the same paragraph in skills/verify-readme/SKILL.md ("Sets defined elsewhere"), which this change edits identically. I also grepped the tree for other statements of the rule; these two files are the only ones.

What the subject says: in verify-readme the list of reachable sources already includes hosted docs: "a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path." In verify-doc-drift that list never had it: "a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path." The change appends the same new sentence to both files, including "or pin the repository or hosted docs URL to that release." The definite article treats a hosted docs URL as one of the sources the previous sentence already allowed. In verify-doc-drift, it isn't one.

What goes wrong: an agent running verify-doc-drift gets two different answers about whether a hosted docs URL is an acceptable pointer. By the first sentence, it isn't, so a restated set in living documentation can't point at hosted docs. By the second sentence, it is, as long as the doc is versioned. That condition makes no sense, and the two skills now disagree about the same rule. Depending on which sentence the agent follows, it either rejects a valid hosted-docs pointer or rewrites a doc to point at one the skill never allowed. This is a contradiction in the instructions, not a runtime failure, so the cost is limited to inconsistent fixes.

Evidence: compare the two source lists quoted above. That is static reading of the text; no tooling runs these files. Fix: add "or hosted docs" to verify-doc-drift's list ("an absolute repository or hosted docs URL"), or remove "or hosted docs" from the new sentence in that file.

lens general-bug · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M4227Z74X4BP5H6VH1P91EN1 of review 01M4225KGGVJNP0XQCW31H2RVG

**low** — verify-doc-drift's new versioned-docs sentence says to pin a "hosted docs URL", but the file's list of allowed sources never includes hosted docs URLs > What I examined: the changed paragraph in skills/verify-doc-drift/SKILL.md ("Restated sets: point instead of recopying") and the same paragraph in skills/verify-readme/SKILL.md ("Sets defined elsewhere"), which this change edits identically. I also grepped the tree for other statements of the rule; these two files are the only ones. > > What the subject says: in verify-readme the list of reachable sources already includes hosted docs: "a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path." In verify-doc-drift that list never had it: "a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path." The change appends the same new sentence to both files, including "or pin the repository or hosted docs URL to that release." The definite article treats a hosted docs URL as one of the sources the previous sentence already allowed. In verify-doc-drift, it isn't one. > > What goes wrong: an agent running verify-doc-drift gets two different answers about whether a hosted docs URL is an acceptable pointer. By the first sentence, it isn't, so a restated set in living documentation can't point at hosted docs. By the second sentence, it is, as long as the doc is versioned. That condition makes no sense, and the two skills now disagree about the same rule. Depending on which sentence the agent follows, it either rejects a valid hosted-docs pointer or rewrites a doc to point at one the skill never allowed. This is a contradiction in the instructions, not a runtime failure, so the cost is limited to inconsistent fixes. > > Evidence: compare the two source lists quoted above. That is static reading of the text; no tooling runs these files. Fix: add "or hosted docs" to verify-doc-drift's list ("an absolute repository or hosted docs URL"), or remove "or hosted docs" from the new sentence in that file. lens `general-bug` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M4227Z74X4BP5H6VH1P91EN1` of review `01M4225KGGVJNP0XQCW31H2RVG`

👍 1

Fixed in 22825f0: removed the redundant undefined living-documentation sentence from both skills; the release-scoped qualification itself defines the boundary.

Fixed in 22825f0: removed the redundant undefined living-documentation sentence from both skills; the release-scoped qualification itself defines the boundary.

Fixed in 22825f0: verify-doc-drift now qualifies repository URLs only, preserving its existing reachable-source list; verify-readme retains its existing hosted-docs source option.

Fixed in 22825f0: verify-doc-drift now qualifies repository URLs only, preserving its existing reachable-source list; verify-readme retains its existing hosted-docs source option.

jercik marked this conversation as resolved

jercik added 1 commit

2026-10-03 23:43:31 +00:00

fix(skills): clarify release pointer scope

commit-msg / commitlint (pull_request) Successful in 18s

Details

Node tests / node:test (pull_request) Successful in 2m29s

Details

Review / Review (pull_request_target) Successful in 4m23s

Details

22825f0c2a

forgejo-actions reviewed

2026-10-03 23:47:53 +00:00

skills/verify-doc-drift/SKILL.md Outdated

					
				@ -41,3 +41,3 @@

				Replace drifted text that restates a set the code defines with a pointer to the source that defines it; correcting the copy only restarts the clock on the next drift. Text restates a set when it lists every member, or states how many there are, of a set that a file, directory, schema, or command defines, such as the flags `--help` prints or the keys a config schema declares.

				Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path.

				Name a source the doc's reader can reach: a command, a file the published package ships, an absolute repository URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository URL to that release.

medium — Versioned docs are still audited against the current source tree

I read the full verify-doc-drift skill, including its per-unit audit loop and fix direction. The new sentence preserves release-specific links when replacing a restated set, but the audit still tells the agent to read each unit's corresponding source and treat any claim contradicted by 'the code' as incorrect; its opening says to fix docs to match code by default. For a repository that keeps documentation for an older release alongside current code, a correct older flag or API name can therefore be labeled incorrect and rewritten to the current interface before the new pointer rule is reached. This is static reasoning, not a run on a versioned project. The writing-for-agents standard asks for version-scoped, verifiable claims and constraints attached to the action they govern. State in the audit loop that versioned documentation is checked against source for its documented release, and that unavailable release source is a proof gap rather than evidence of drift. Keep the new release-pinned pointer guidance. A version-aware source-selection rule elsewhere in this skill would refute the concern; I found none in the complete file.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M422HQ1JWXC4T3TWTP6Q4Y6Z of review 01M422DKR8F3C4EZ94VMP5R63N

**medium** — Versioned docs are still audited against the current source tree > I read the full verify-doc-drift skill, including its per-unit audit loop and fix direction. The new sentence preserves release-specific links when replacing a restated set, but the audit still tells the agent to read each unit's corresponding source and treat any claim contradicted by 'the code' as incorrect; its opening says to fix docs to match code by default. For a repository that keeps documentation for an older release alongside current code, a correct older flag or API name can therefore be labeled incorrect and rewritten to the current interface before the new pointer rule is reached. This is static reasoning, not a run on a versioned project. The writing-for-agents standard asks for version-scoped, verifiable claims and constraints attached to the action they govern. State in the audit loop that versioned documentation is checked against source for its documented release, and that unavailable release source is a proof gap rather than evidence of drift. Keep the new release-pinned pointer guidance. A version-aware source-selection rule elsewhere in this skill would refute the concern; I found none in the complete file. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M422HQ1JWXC4T3TWTP6Q4Y6Z` of review `01M422DKR8F3C4EZ94VMP5R63N`

👍 1

Fixed in 921c1f9: the audit loop uses source for the documented release and records a proof gap if it is unavailable, retaining the existing no-citation/no-finding discipline.

Fixed in 921c1f9: the audit loop uses source for the documented release and records a proof gap if it is unavailable, retaining the existing no-citation/no-finding discipline.

jercik marked this conversation as resolved

skills/verify-readme/SKILL.md Outdated

					
				@ -107,3 +107,3 @@

				## Sets defined elsewhere

				When a file, directory, schema, or command already defines a set — `--help`, `package.json#scripts`, the env var schema or config parser, the workspace packages, a directory's children — the README names that source instead of listing the members or stating how many there are. A copy must change with every change to the set, and nothing flags it when it doesn't. Name a source the README's reader can reach: a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path.

				When a file, directory, schema, or command already defines a set — `--help`, `package.json#scripts`, the env var schema or config parser, the workspace packages, a directory's children — the README names that source instead of listing the members or stating how many there are. A copy must change with every change to the set, and nothing flags it when it doesn't. Name a source the README's reader can reach: a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release.

medium — Versioned README guidance conflicts with the required unpinned Agent Rule command

I read the full verify-readme skill and its CLI-specific Agent Rule template. The new sentence requires a command matching the documented release, but the same skill requires every CLI README's Agent Rule to start with npx -y <tool> --help and supplies Run npx -y tool-name --help to learn available options. It says this works on fresh machines. On a fresh machine that unqualified package request can install the current release, so an agent reading a README for an older release can receive help for a newer interface and choose unsupported flags. The writing-for-agents standard calls for one clear home for each instruction and precise, version-scoped claims. Make the template and its mandatory first-instruction rule version-aware, using the documented package release for versioned READMEs; keep the live-help behavior for unversioned READMEs. This preserves a reachable flag reference without contradicting the new release constraint. This is static reasoning from the two instructions; I did not run a package installation. A versioned CLI README generated under these rules with a release-qualified help command would refute the practical risk, but the mandatory template currently forbids that qualification.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M422H3DVCYSV4D4239C9Q28M of review 01M422DKR8F3C4EZ94VMP5R63N

**medium** — Versioned README guidance conflicts with the required unpinned Agent Rule command > I read the full verify-readme skill and its CLI-specific Agent Rule template. The new sentence requires a command matching the documented release, but the same skill requires every CLI README's Agent Rule to start with <code>npx -y <tool> --help</code> and supplies `Run `npx -y tool-name --help` to learn available options.` It says this works on fresh machines. On a fresh machine that unqualified package request can install the current release, so an agent reading a README for an older release can receive help for a newer interface and choose unsupported flags. The writing-for-agents standard calls for one clear home for each instruction and precise, version-scoped claims. Make the template and its mandatory first-instruction rule version-aware, using the documented package release for versioned READMEs; keep the live-help behavior for unversioned READMEs. This preserves a reachable flag reference without contradicting the new release constraint. This is static reasoning from the two instructions; I did not run a package installation. A versioned CLI README generated under these rules with a release-qualified help command would refute the practical risk, but the mandatory template currently forbids that qualification. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M422H3DVCYSV4D4239C9Q28M` of review `01M422DKR8F3C4EZ94VMP5R63N`

👍 1

Fixed in 921c1f9: the Agent Rule template and mandatory help instruction qualify the package with the documented release for versioned READMEs. npm11.19.1 installed libnpmexec source confirms bare names may resolve local/global binaries or registry manifest while exact @version is a version spec; no registry installation reproduction is claimed.

Fixed in 921c1f9: the Agent Rule template and mandatory help instruction qualify the package with the documented release for versioned READMEs. npm11.19.1 installed libnpmexec source confirms bare names may resolve local/global binaries or registry manifest while exact @version is a version spec; no registry installation reproduction is claimed.

jercik marked this conversation as resolved

jercik added 1 commit

2026-10-03 23:52:50 +00:00

fix(skills): audit and help sources should match the documented release

commit-msg / commitlint (pull_request) Successful in 24s

Details

Node tests / node:test (pull_request) Successful in 3m12s

Details

Review / Review (pull_request_target) Successful in 6m7s

Details

921c1f9caa

forgejo-actions reviewed

2026-10-03 23:58:56 +00:00

skills/verify-readme/SKILL.md Outdated

					
				@ -163,3 +165,3 @@

				- Start with `# Rule:` followed by the tool name in backticks.

				- Make `npx -y <tool> --help` the first instruction.

				- Make `npx -y <tool> --help` the first instruction, qualifying the package with `@<documented-release>` for a README describing a specific release.

low — Agent Rule release qualifier is repeated on both sides of the template

I read the Agent Rule template and its surrounding instructions in verify-readme. The sentence immediately before the template already says to use tool-name@<documented-release> in the help command for a README describing a specific release. This bullet repeats the same qualification after the template. The installed writing standard's 'One Idea, One Place' guidance calls for one clear home for an instruction, and every extra sentence costs the agent reader; here, an editor could also update one version rule while leaving the other inconsistent. Remove the release-qualification clause from this bullet and retain its distinct requirement that the help command be first. The sentence before the template continues to preserve the release-specific behavior. This is a text-level duplication, not an observed failure in a generated README; the adjacent sentence would refute it if it expressed a different condition, but it states the same one.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M4235CYEDGQPMVK3D61T0YCH of review 01M422YNE5XADPMHJAZHPM3036

**low** — Agent Rule release qualifier is repeated on both sides of the template > I read the Agent Rule template and its surrounding instructions in verify-readme. The sentence immediately before the template already says to use tool-name@<documented-release> in the help command for a README describing a specific release. This bullet repeats the same qualification after the template. The installed writing standard's 'One Idea, One Place' guidance calls for one clear home for an instruction, and every extra sentence costs the agent reader; here, an editor could also update one version rule while leaving the other inconsistent. Remove the release-qualification clause from this bullet and retain its distinct requirement that the help command be first. The sentence before the template continues to preserve the release-specific behavior. This is a text-level duplication, not an observed failure in a generated README; the adjacent sentence would refute it if it expressed a different condition, but it states the same one. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M4235CYEDGQPMVK3D61T0YCH` of review `01M422YNE5XADPMHJAZHPM3036`

👍 1

Fixed in 0ded8c636b: removed duplicate preceding qualifier; mandatory Agent Rule release qualification remains.

Fixed in 0ded8c636b8b1cb676b8d10b1e0f601427145772: removed duplicate preceding qualifier; mandatory Agent Rule release qualification remains.

jercik marked this conversation as resolved

jercik commented

2026-10-04 00:03:35 +00:00

Replying to review comment #111896

For report-only claim 01M4234C95J4M0A5Q8H5K4Y524 of review 01M422YNE5XADPMHJAZHPM3036: no manifest-parity policy change is adopted. #109 implements the release-matching source-pointer concession from #97, plus contradictions directly affecting those pointers and help commands. The existing package.json#description parity rule and Task steps apply to the target repository's README and manifest; defining a cross-release manifest-selection contract would expand that separate check. The proposed mismatched-manifest path has no served-agent or repository reproduction. The user explicitly excludes new policy from this task.

A stable three-surface sweep found this claim only in the stamped summary, with no corresponding inline finding or native anchor. This records the bounded scope rejection without inventing a tracked item or claiming an inline status transition.

> Replying to review comment #111896 For report-only claim `01M4234C95J4M0A5Q8H5K4Y524` of review `01M422YNE5XADPMHJAZHPM3036`: no manifest-parity policy change is adopted. #109 implements the release-matching source-pointer concession from #97, plus contradictions directly affecting those pointers and help commands. The existing `package.json#description` parity rule and Task steps apply to the target repository's README and manifest; defining a cross-release manifest-selection contract would expand that separate check. The proposed mismatched-manifest path has no served-agent or repository reproduction. The user explicitly excludes new policy from this task. A stable three-surface sweep found this claim only in the stamped summary, with no corresponding inline finding or native anchor. This records the bounded scope rejection without inventing a tracked item or claiming an inline status transition.

jercik added 1 commit

2026-10-04 04:14:10 +00:00

fix: keep one mandatory release source qualifier

commit-msg / commitlint (pull_request) Successful in 19s

Details

Node tests / node:test (pull_request) Successful in 3m10s

Details

Review / Review (pull_request_target) Successful in 3m51s

Details

0ded8c636b

forgejo-actions reviewed

2026-10-04 04:18:01 +00:00

skills/verify-doc-drift/SKILL.md Outdated

					
				@ -27,3 +27,3 @@

				## The per-unit audit loop

				Split the docs into units — root docs (README, standards, glossary, ADRs) and one unit per package/app (its README + CONTEXT + ADRs + source doc-comments). For each unit: read the docs in full, then read the corresponding source, **following imports to the definition**, and check each claim. Require a concrete citation — `file:line` plus a short quoted snippet — for every finding: the contradicting source line for `incorrect`/`code-drift`, the canonical doc location for a `duplicate`, the doc line itself for an `obvious`. No citation, no finding. Returning zero findings for accurate docs is the correct outcome; do not pad.

				Split the docs into units — root docs (README, standards, glossary, ADRs) and one unit per package/app (its README + CONTEXT + ADRs + source doc-comments). For each unit: read the docs in full, then read the corresponding source, **following imports to the definition**, and check each claim. For versioned documentation, read the source for its documented release; if that source is unavailable, record a proof gap rather than inferring drift from current code. Require a concrete citation — `file:line` plus a short quoted snippet — for every finding: the contradicting source line for `incorrect`/`code-drift`, the canonical doc location for a `duplicate`, the doc line itself for an `obvious`. No citation, no finding. Returning zero findings for accurate docs is the correct outcome; do not pad.

low — verify-doc-drift tells the agent to "record a proof gap" but neither defines the term nor gives it a place in the Output report

Examined: skills/verify-doc-drift/SKILL.md, the sentence the diff added to "The per-unit audit loop". Also examined the Task steps and the Output section that it feeds.

The added sentence tells the agent to "record a proof gap". The skill does not define "proof gap" anywhere (a grep of the tree for "proof gap" finds only this line), and it does not say where one goes. Everywhere else, the skill's report and workflow have two outcomes. One is a finding: "No citation, no finding", then "Adversarially verify every candidate finding; keep only CONFIRMED survivors", then a report of confirmed findings. The other is the "clean bill of health" in Output: "which claims verified". A claim whose release source is unavailable is neither, so an agent that follows the Output section literally has nowhere to put it. The likely results are dropping the gap silently, which over-states coverage in the clean bill of health, or turning it into a finding, which "No citation, no finding" forbids.

Skill guidance: writing-for-agents, "Use Precise Language": "define project-local terms on first use". Under "Specify the Discipline", the Evidence item requires the artifacts that prove completion to be stated.

Proposed correction: replace the coined term with an action that has a place in the existing Output. "...if that source is unavailable, list the claim as unverified in the report rather than inferring drift from current code." Then add "and which claims couldn't be checked" to the Output sentence "State the clean bill of health too — which claims verified". This keeps the rule (no drift inferred from current code) and tells the reader where the gap lands.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M42J019MA22Z7VSDDMJ1PZ5Q of review 01M42HX67C17WYHYAPWCNJEXDT

**low** — verify-doc-drift tells the agent to "record a proof gap" but neither defines the term nor gives it a place in the Output report > Examined: skills/verify-doc-drift/SKILL.md, the sentence the diff added to "The per-unit audit loop". Also examined the Task steps and the Output section that it feeds. > > The added sentence tells the agent to "record a proof gap". The skill does not define "proof gap" anywhere (a grep of the tree for "proof gap" finds only this line), and it does not say where one goes. Everywhere else, the skill's report and workflow have two outcomes. One is a finding: "No citation, no finding", then "Adversarially verify every candidate finding; keep only CONFIRMED survivors", then a report of confirmed findings. The other is the "clean bill of health" in Output: "which claims verified". A claim whose release source is unavailable is neither, so an agent that follows the Output section literally has nowhere to put it. The likely results are dropping the gap silently, which over-states coverage in the clean bill of health, or turning it into a finding, which "No citation, no finding" forbids. > > Skill guidance: writing-for-agents, "Use Precise Language": "define project-local terms on first use". Under "Specify the Discipline", the Evidence item requires the artifacts that prove completion to be stated. > > Proposed correction: replace the coined term with an action that has a place in the existing Output. "...if that source is unavailable, list the claim as unverified in the report rather than inferring drift from current code." Then add "and which claims couldn't be checked" to the Output sentence "State the clean bill of health too — which claims verified". This keeps the rule (no drift inferred from current code) and tells the reader where the gap lands. > lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M42J019MA22Z7VSDDMJ1PZ5Q` of review `01M42HX67C17WYHYAPWCNJEXDT`

🚀 1

jercik marked this conversation as resolved

skills/verify-readme/SKILL.md Outdated

					
				@ -107,3 +107,3 @@

				## Sets defined elsewhere

				When a file, directory, schema, or command already defines a set — `--help`, `package.json#scripts`, the env var schema or config parser, the workspace packages, a directory's children — the README names that source instead of listing the members or stating how many there are. A copy must change with every change to the set, and nothing flags it when it doesn't. Name a source the README's reader can reach: a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path.

				When a file, directory, schema, or command already defines a set — `--help`, `package.json#scripts`, the env var schema or config parser, the workspace packages, a directory's children — the README names that source instead of listing the members or stating how many there are. A copy must change with every change to the set, and nothing flags it when it doesn't. Name a source the README's reader can reach: a command, a file the published package ships, an absolute repository or hosted docs URL, or, when the project isn't published, a repository-relative path. For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release.

low — verify-readme uses two terms ("versioned documentation" and "a README describing a specific release") for the same condition

Examined: skills/verify-readme/SKILL.md, the two sentences the diff added about release-specific READMEs. In "Sets defined elsewhere" it says: "For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release." In the Agent Rule template's "The block must" list it says: "qualifying the package with @<documented-release> for a README describing a specific release."

Both conditions name one concept, a README that documents a particular release, with two different terms. "Versioned documentation" is also the looser term. Almost every package that verify-readme audits has a version in package.json, and the skill's Inputs section has the agent read package.json. A literal reader can therefore count any versioned package's README as "versioned documentation" and pin every source URL to the current tag. That freezes links the README should keep pointing at the moving default branch. A reader of the same skill can also pin URLs in the Sets section while leaving npx -y <tool> --help unpinned in the Agent Rule (or the reverse), because nothing says the two conditions are the same.

Skill guidance: writing-for-agents, "Use Precise Language": "use one term for one concept, and define project-local terms on first use."

Proposed correction: use the Agent Rule's clearer wording in both places. Change the Sets sentence to "For a README that describes a specific release, use a command or shipped file matching that release, or pin the repository or hosted docs URL to it." Keep the Agent Rule bullet as it is. This keeps both instructions and removes the reading in which every versioned package's README counts.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M42HZRDWPCX18S3KPG2D3HWP of review 01M42HX67C17WYHYAPWCNJEXDT

**low** — verify-readme uses two terms ("versioned documentation" and "a README describing a specific release") for the same condition > Examined: skills/verify-readme/SKILL.md, the two sentences the diff added about release-specific READMEs. In "Sets defined elsewhere" it says: "For versioned documentation, use a command or shipped file matching the documented release, or pin the repository or hosted docs URL to that release." In the Agent Rule template's "The block must" list it says: "qualifying the package with <code>@<documented-release></code> for a README describing a specific release." > > Both conditions name one concept, a README that documents a particular release, with two different terms. "Versioned documentation" is also the looser term. Almost every package that verify-readme audits has a `version` in `package.json`, and the skill's Inputs section has the agent read `package.json`. A literal reader can therefore count any versioned package's README as "versioned documentation" and pin every source URL to the current tag. That freezes links the README should keep pointing at the moving default branch. A reader of the same skill can also pin URLs in the Sets section while leaving <code>npx -y <tool> --help</code> unpinned in the Agent Rule (or the reverse), because nothing says the two conditions are the same. > > Skill guidance: writing-for-agents, "Use Precise Language": "use one term for one concept, and define project-local terms on first use." > > Proposed correction: use the Agent Rule's clearer wording in both places. Change the Sets sentence to "For a README that describes a specific release, use a command or shipped file matching that release, or pin the repository or hosted docs URL to it." Keep the Agent Rule bullet as it is. This keeps both instructions and removes the reading in which every versioned package's README counts. > lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M42HZRDWPCX18S3KPG2D3HWP` of review `01M42HX67C17WYHYAPWCNJEXDT`

🚀 1

jercik marked this conversation as resolved

skills/verify-readme/SKILL.md Outdated

					
				@ -163,3 +163,3 @@

				- Start with `# Rule:` followed by the tool name in backticks.

				- Make `npx -y <tool> --help` the first instruction.

				- Make `npx -y <tool> --help` the first instruction, qualifying the package with `@<documented-release>` for a README describing a specific release.

medium — Agent Rule guidance still has generated rules run untagged npx -y <tool>, which can serve a stale installed copy and contradicts the repo's npx rule

What I examined: the Agent Rule template and its "The block must" list in skills/verify-readme/SKILL.md, the repo rule rules/general/package-manager-execution.md, and the repoq entry in AGENTS.md.

What the skill says: the changed bullet tells the agent to add a tag only for a README that describes a specific release (qualifying the package with @<documented-release> for a README describing a specific release). The template it fills in, which this change left alone, still reads Run \npx -y tool-name --help` to learn available options.So for every other README, which is most of them, the skill tells the agent to write an untaggednpx -y <tool> --help` into the reader's CLAUDE.md/AGENTS.md. The change also deleted the old "never stale" claim and replaced the rationale with "the agent reads the selected package's help text".

Why that's wrong: the repo's own rule says, in rules/general/package-manager-execution.md: "For a one-off npm package, in any repository, use npx -y <pkg>@<tag>; without the tag, npx runs any older copy already installed in the project or globally." AGENTS.md follows that rule for its own tool: "Run npx -y repoq@latest --help ... the explicit tag prevents npx from reusing a stale cached release." With the untagged form, the help text an agent reads can come from whatever older copy happens to be installed, not from the package the README documents. That stale-flag-reference problem is exactly what the Agent Rule exists to avoid. Now that the bullet explicitly limits tagging to release-specific READMEs, it reads as approval to leave the tag off in the default case.

How I checked: I compared the documents to each other; I did not run npx. The claim about how untagged npx resolves comes from the repo's own rule. If npx -y <tool> always fetched the latest registry release no matter what is installed locally, this finding would not hold. The repo's rule says it does not.

Fix: require a tag every time, @latest by default and @<documented-release> for a release-specific README, and update the template line to npx -y tool-name@latest --help.

lens general-bug · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M42HZ3G0EMRR7HYBM050RRYQ of review 01M42HX67C17WYHYAPWCNJEXDT

**medium** — Agent Rule guidance still has generated rules run untagged <code>npx -y <tool></code>, which can serve a stale installed copy and contradicts the repo's npx rule > What I examined: the Agent Rule template and its "The block must" list in `skills/verify-readme/SKILL.md`, the repo rule `rules/general/package-manager-execution.md`, and the `repoq` entry in `AGENTS.md`. > > What the skill says: the changed bullet tells the agent to add a tag only for a README that describes a specific release (<code>qualifying the package with @<documented-release> for a README describing a specific release</code>). The template it fills in, which this change left alone, still reads `Run \`npx -y tool-name --help\` to learn available options.` So for every other README, which is most of them, the skill tells the agent to write an untagged `npx -y <tool> --help` into the reader's CLAUDE.md/AGENTS.md. The change also deleted the old "never stale" claim and replaced the rationale with "the agent reads the selected package's help text". > > Why that's wrong: the repo's own rule says, in `rules/general/package-manager-execution.md`: "For a one-off npm package, in any repository, use <code>npx -y <pkg>@<tag></code>; without the tag, `npx` runs any older copy already installed in the project or globally." `AGENTS.md` follows that rule for its own tool: "Run `npx -y repoq@latest --help` ... the explicit tag prevents `npx` from reusing a stale cached release." With the untagged form, the help text an agent reads can come from whatever older copy happens to be installed, not from the package the README documents. That stale-flag-reference problem is exactly what the Agent Rule exists to avoid. Now that the bullet explicitly limits tagging to release-specific READMEs, it reads as approval to leave the tag off in the default case. > > How I checked: I compared the documents to each other; I did not run `npx`. The claim about how untagged `npx` resolves comes from the repo's own rule. If <code>npx -y <tool></code> always fetched the latest registry release no matter what is installed locally, this finding would not hold. The repo's rule says it does not. > > Fix: require a tag every time, `@latest` by default and <code>@<documented-release></code> for a release-specific README, and update the template line to `npx -y tool-name@latest --help`. lens `general-bug` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M42HZ3G0EMRR7HYBM050RRYQ` of review `01M42HX67C17WYHYAPWCNJEXDT`

🚀 1

jercik marked this conversation as resolved

skills/verify-readme/SKILL.md Outdated

					
				@ -166,3 +166,3 @@

				- State when and why to use the tool.

				Why `npx -y <tool> --help` instead of an embedded flag reference: the agent reads live help text from the installed version — never stale, no rule edits when the CLI interface changes, and it works on fresh machines, CI, and containers.

				Why `npx -y <tool> --help` instead of an embedded flag reference: the agent reads the selected package's help text rather than an embedded flag copy, including on fresh machines, CI, and containers.

low — Agent Rule rationale restates its own contrast and drops the reason live help beats an embedded flag copy

Examined: the "Agent Rule template (CLI only)" section of skills/verify-readme/SKILL.md, including the changed bullet above it ("Make npx -y <tool> --help the first instruction, qualifying the package with @<documented-release> for a README describing a specific release.").

The rewritten rationale reads: "Why npx -y <tool> --help instead of an embedded flag reference: the agent reads the selected package's help text rather than an embedded flag copy, including on fresh machines, CI, and containers." The clause after the colon repeats the question's contrast (help text vs. an embedded flag copy) instead of answering it. It no longer says why that matters. The diff removed the stakes the old line stated ("never stale, no rule edits when the CLI interface changes"). The trailing "including on fresh machines, CI, and containers" also has no clear antecedent: it is unclear what holds "including" there.

Why it matters: the writing-for-agents skill says to "Give rationale only when it helps the agent adapt the rule or understand its stakes", and its duplicate test cuts "a consequence or complement the first sentence already implies." As written, the line fails both tests. It is a rationale with no reason. An agent drafting an Agent Rule that is asked whether to embed a short flag list loses the one fact that decides the question: the help output always matches the package the agent runs, while a copy drifts.

Proposed correction: "Why npx -y <tool> --help instead of an embedded flag reference: the help text comes from the package the agent runs, so it matches that release's flags without rule edits, on fresh machines, CI, and containers alike." This keeps the version-pinning nuance the change introduced, because the help matches the selected release rather than being "never stale". It restores the stakes and removes the circular restatement.

lens writing-quality · arm default · tally 1 valid / 0 invalid / 0 uncertain
claim 01M42HZD16B7Y7RY5QCSX40RJS of review 01M42HX67C17WYHYAPWCNJEXDT

**low** — Agent Rule rationale restates its own contrast and drops the reason live help beats an embedded flag copy > Examined: the "Agent Rule template (CLI only)" section of skills/verify-readme/SKILL.md, including the changed bullet above it ("Make <code>npx -y <tool> --help</code> the first instruction, qualifying the package with <code>@<documented-release></code> for a README describing a specific release."). > > The rewritten rationale reads: "Why <code>npx -y <tool> --help</code> instead of an embedded flag reference: the agent reads the selected package's help text rather than an embedded flag copy, including on fresh machines, CI, and containers." The clause after the colon repeats the question's contrast (help text vs. an embedded flag copy) instead of answering it. It no longer says why that matters. The diff removed the stakes the old line stated ("never stale, no rule edits when the CLI interface changes"). The trailing "including on fresh machines, CI, and containers" also has no clear antecedent: it is unclear what holds "including" there. > > Why it matters: the writing-for-agents skill says to "Give rationale only when it helps the agent adapt the rule or understand its stakes", and its duplicate test cuts "a consequence or complement the first sentence already implies." As written, the line fails both tests. It is a rationale with no reason. An agent drafting an Agent Rule that is asked whether to embed a short flag list loses the one fact that decides the question: the help output always matches the package the agent runs, while a copy drifts. > > Proposed correction: "Why <code>npx -y <tool> --help</code> instead of an embedded flag reference: the help text comes from the package the agent runs, so it matches that release's flags without rule edits, on fresh machines, CI, and containers alike." This keeps the version-pinning nuance the change introduced, because the help matches the selected release rather than being "never stale". It restores the stakes and removes the circular restatement. lens `writing-quality` · arm `default` · tally 1 valid / 0 invalid / 0 uncertain claim `01M42HZD16B7Y7RY5QCSX40RJS` of review `01M42HX67C17WYHYAPWCNJEXDT`

🚀 1

jercik marked this conversation as resolved

jercik referenced this pull request

2026-10-04 04:21:28 +00:00

fix(skills): select help from an explicitly tagged release #112

jercik referenced this pull request

2026-10-04 04:21:29 +00:00

fix(skills): report unavailable release claims as unverified #113

jercik commented

2026-10-04 04:23:13 +00:00

Replying to review comment #112057

Tracked in #112: explicitly tagged help selection follows the existing rules/general/package-manager-execution.md rule. Installed npm 11.19.1 libnpmexec/lib/index.js lines 47–85 and 167–176 confirm that bare-name execution can choose installed binaries while tags select a manifest. This is a bounded follow-up to #109, not a new package policy. #112 also owns the rationale and consistent release wording from #112059 and #112060.

Replying to review comment #112058

Tracked in #113: claims with unavailable documented-release source are reported as unverified in the existing report. This clarifies the existing release-evidence limitation, without declaring drift or adding a finding category.

Both follow-ups stack on #109 and remain unmerged. #109 itself is not claimed fixed by either child.

> Replying to review comment #112057 Tracked in [#112](https://code.j4k.dev/j4k-oss/agent-skills/pulls/112): explicitly tagged help selection follows the existing `rules/general/package-manager-execution.md` rule. Installed npm 11.19.1 `libnpmexec/lib/index.js` lines 47–85 and 167–176 confirm that bare-name execution can choose installed binaries while tags select a manifest. This is a bounded follow-up to #109, not a new package policy. #112 also owns the rationale and consistent release wording from #112059 and #112060. > Replying to review comment #112058 Tracked in [#113](https://code.j4k.dev/j4k-oss/agent-skills/pulls/113): claims with unavailable documented-release source are reported as unverified in the existing report. This clarifies the existing release-evidence limitation, without declaring drift or adding a finding category. Both follow-ups stack on #109 and remain unmerged. #109 itself is not claimed fixed by either child.

jercik force-pushed fix/version-matched-source-pointers from 0ded8c636b

commit-msg / commitlint (pull_request) Successful in 19s

Details

Node tests / node:test (pull_request) Successful in 3m10s

Details

Review / Review (pull_request_target) Successful in 3m51s

Details

to b9c64c5a51

commit-msg / commitlint (pull_request) Successful in 22s

Details

Node tests / node:test (pull_request) Successful in 2m58s

Details

Review / Review (pull_request_target) Successful in 4m8s

Details

2026-10-06 07:44:56 +00:00

Compare

jercik changed target branch from fix/skills-point-to-sources to main

2026-10-06 07:44:56 +00:00

jercik commented

2026-10-06 07:45:00 +00:00

Rebased onto main and retargeted the base from fix/skills-point-to-sources, because #97 has merged. No content changes beyond the rebase. Remaining review items are tracked in #112 and #113.

Rebased onto `main` and retargeted the base from `fix/skills-point-to-sources`, because #97 has merged. No content changes beyond the rebase. Remaining review items are tracked in #112 and #113.

🚀 1

jercik referenced this pull request

2026-10-07 11:57:38 +00:00

fix: doc drift verification should accept each category's citation #124

jercik referenced this pull request

2026-10-07 11:57:41 +00:00

fix: shorten the verify-readme catalog description #125

jercik commented

2026-10-07 11:57:54 +00:00

Conclusions on the five unadjudicated claims in review 01M482RJE4FVN83JWAJBTNJJHX for head b9c64c5. The triage failed, so the claims have titles only. I checked each title against the text on main; none of them concerns a line this PR changes.

01M48322GSZKFX3930YSW01KZ4 (the verify-doc-drift description duplicates the finding categories): valid, pre-existing. Already owned by #119.
01M4832KS2Q3EG6P3YPEKCYJH4 (the verification gate rejects findings without a code contradiction): valid, pre-existing. duplicate and obvious findings cite a doc line, not code. Tracked in #124.
01M48338ASS1Q5BAYCK31H9QSJ (Quick Start and Usage repeat the project-type set): not a defect. The per-type bullets carry the detail each type needs, which Inputs cannot, so they have to name the type.
01M4833TVNV1CNR4Q35ABVV693 and 01M4835B3J6CYXS7FMAGZHYY7Q (the verify-readme description carries audit mechanics and the project-type list): valid, pre-existing. Tracked in #125.

Conclusions on the five unadjudicated claims in review `01M482RJE4FVN83JWAJBTNJJHX` for head `b9c64c5`. The triage failed, so the claims have titles only. I checked each title against the text on `main`; none of them concerns a line this PR changes. - `01M48322GSZKFX3930YSW01KZ4` (the `verify-doc-drift` description duplicates the finding categories): valid, pre-existing. Already owned by #119. - `01M4832KS2Q3EG6P3YPEKCYJH4` (the verification gate rejects findings without a code contradiction): valid, pre-existing. `duplicate` and `obvious` findings cite a doc line, not code. Tracked in #124. - `01M48338ASS1Q5BAYCK31H9QSJ` (Quick Start and Usage repeat the project-type set): not a defect. The per-type bullets carry the detail each type needs, which `Inputs` cannot, so they have to name the type. - `01M4833TVNV1CNR4Q35ABVV693` and `01M4835B3J6CYXS7FMAGZHYY7Q` (the `verify-readme` description carries audit mechanics and the project-type list): valid, pre-existing. Tracked in #125.

🚀 1

jercik referenced this pull request

2026-10-07 12:45:18 +00:00

fix(skills): report unavailable release claims as unverified #113

jercik merged commit 75ac389375 into main

2026-10-08 08:20:51 +00:00

jercik deleted branch fix/version-matched-source-pointers

2026-10-08 08:20:52 +00:00

jercik referenced this pull request from a commit

2026-10-08 08:20:53 +00:00

fix(skills): source pointers should match the documented release (#109)

jercik referenced this pull request