docs: add documentation-audit design (claim-by-claim accuracy + completeness)
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,101 @@
|
||||
# Documentation Audit Design
|
||||
|
||||
**Date:** 2026-06-03
|
||||
|
||||
**Goal:** Audit all prose documentation in the repository for accuracy and
|
||||
completeness against the current code/contracts, then apply fixes — producing
|
||||
both an evidence-backed findings report and corrected docs.
|
||||
|
||||
## Decisions
|
||||
|
||||
- **Deliverable:** Findings report *and* applied fixes (reviewable commit).
|
||||
- **Scope:** All prose docs — top-level (`gateway.md`, `glauth.md`,
|
||||
`StyleGuide.md`, `REVIEW-PROCESS.md`), `docs/**`, the 6 style guides, and all
|
||||
10 client README + design docs (~55 files).
|
||||
- **Verification depth:** Deep, claim-by-claim, against **this repo's code and
|
||||
contracts only** (not the external `mxaccess` / `lmxopcua/gr` reference
|
||||
projects).
|
||||
- **Approach:** A + C dedup — per-subsystem verifier fan-out (A), with findings
|
||||
additionally keyed by code area so renamed terms / moved paths are fixed
|
||||
consistently everywhere at once (C).
|
||||
|
||||
## Architecture
|
||||
|
||||
Two-phase pipeline:
|
||||
|
||||
- **Phase 1 — Verify (fan-out).** ~13 verifier subagents, one per subsystem
|
||||
cluster. Each reads its docs *and* the relevant source/contracts and returns a
|
||||
structured findings list.
|
||||
- **Phase 2 — Fix.** Aggregate findings → dedup by code area into a global
|
||||
substitutions table → apply mechanical substitutions repo-wide, then
|
||||
per-cluster judgment fixes. Every fix cites the finding that justifies it.
|
||||
|
||||
No code changes; prose only. Source of truth is the repo's own code/contracts.
|
||||
|
||||
## Cluster map (Phase 1 fan-out)
|
||||
|
||||
| # | Cluster | Docs | Verified against |
|
||||
|---|---|---|---|
|
||||
| 1 | Architecture | `gateway.md`, `DesignDecisions.md`, `GatewayProcessDesign.md` | top-level src layout, two-process model |
|
||||
| 2 | Worker | `Worker{Bootstrap,Conversion,FrameProtocol,ProcessLauncher,Sta}.md`, `MxAccessWorkerInstanceDesign.md` | `…MxGateway.Worker` |
|
||||
| 3 | Sessions/runtime | `Sessions.md` | Server sessions/workers |
|
||||
| 4 | Auth (high drift) | `Authentication.md`, `Authorization.md`, `glauth.md` | `Security/Authentication`, ZB.MOM.WW.Auth migration |
|
||||
| 5 | Dashboard (high drift) | `DashboardInterfaceDesign.md`, `GatewayDashboardDesign.md` | dashboard + ZB.MOM.WW.Theme migration |
|
||||
| 6 | Config | `GatewayConfiguration.md`, `Diagnostics.md`, `Metrics.md` | `GatewayOptions(Validator)`, appsettings |
|
||||
| 7 | Contracts/gRPC | `Contracts.md`, `Grpc.md`, `ClientProtoGeneration.md` | `.proto` + generated |
|
||||
| 8 | Galaxy repo | `GalaxyRepository.md` | GR SQL browse RPCs |
|
||||
| 9 | Alarms | `AlarmClientDiscovery.md` | alarm worker/server code |
|
||||
| 10 | Testing | `GatewayTesting.md`, `ClientBehaviorFixtures.md`, `ParityFixtureMatrix.md`, `CrossLanguageSmokeMatrix.md`, `ToolchainLinks.md` | test projects, harness, fixtures |
|
||||
| 11 | Clients ×5 | each `clients/<lang>/README.md` + `<Lang>ClientDesign.md`, `ClientLibrariesDesign.md`, `ClientPackaging.md` | each client's source |
|
||||
| 12 | Style guides (normative) | `StyleGuide.md`, `REVIEW-PROCESS.md`, `docs/style-guides/*` | spot-check vs observed conventions; flag-only |
|
||||
| 13 | Design-history/plans | `ImplementationPlan*.md`, `docs/plans/*` | point-in-time records — verify internal consistency + that no *living* doc cites them as current truth; do **not** rewrite to match code |
|
||||
|
||||
## Findings schema
|
||||
|
||||
Each verifier returns, per claim:
|
||||
|
||||
- `doc_file`, `doc_lines`
|
||||
- `claim` — the concrete assertion (verbatim or tight paraphrase)
|
||||
- `claim_type` — `path` | `config-key` | `rpc/proto` | `port` | `term` |
|
||||
`behavior-rule` | `command` | `cross-ref` | `version`
|
||||
- `verdict` — `accurate` | `stale` | `wrong` | `unverifiable` | `gap`
|
||||
(missing doc for an existing feature)
|
||||
- `code_evidence` — `file:line` proving the verdict
|
||||
- `code_area` — dedup tag (e.g. `auth.roles`, `dashboard.theme`)
|
||||
- `severity` — `high` (misleads integrator/operator) | `medium` | `low`
|
||||
- `proposed_fix` — replacement text, or "flag only"
|
||||
|
||||
## Dedup layer (the C ingredient)
|
||||
|
||||
Aggregate all findings; group by `code_area` + normalized claim. Renamed terms
|
||||
and moved paths collapse into one **global substitutions table** (old → new)
|
||||
applied once repo-wide. Doc-specific judgment fixes stay per-doc.
|
||||
|
||||
## Report
|
||||
|
||||
`MxAccessGateway-doc-audit.md` (matching the existing `*-docs-*.md` report
|
||||
convention): summary counts by verdict/severity/cluster, the global
|
||||
substitutions table, then per-doc findings with evidence and proposed fixes.
|
||||
|
||||
## Fix pass & verification
|
||||
|
||||
1. Apply global substitutions first (mechanical, repo-wide).
|
||||
2. Apply per-cluster judgment fixes — each citing a finding ID, honoring
|
||||
`StyleGuide.md` (PascalCase filenames, present tense, explain *why* not
|
||||
*what*, no marketing language).
|
||||
3. Re-verify changed claims (lighter spot pass) to confirm resolution and no new
|
||||
drift.
|
||||
4. Sanity-check any fenced shell/build commands in docs against current
|
||||
invocations.
|
||||
|
||||
## Out of scope (YAGNI)
|
||||
|
||||
- Rewriting design-history docs to match current code (they are records).
|
||||
- XML doc-comments (handled in a prior pass; the `*-docs-*.md` analyzer is clean).
|
||||
- External `mxaccess` / `lmxopcua/gr` reference projects.
|
||||
|
||||
## Branch note
|
||||
|
||||
Current branch is `docs/xml-doc-comments` (a separate PR in flight). The audit
|
||||
fixes are best kept on their own branch (`docs/prose-audit` off `main`) so the
|
||||
two reviews stay independent.
|
||||
Reference in New Issue
Block a user