Add Go-to-.NET gap inventory docs to track porting parity

2026-02-25 15:12:52 -05:00
parent 339c60bac6
commit 79c1ee8776
21 changed files with 7997 additions and 0 deletions
--- a/gaps/instructions.md
+++ b/gaps/instructions.md
@@ -0,0 +1,341 @@
+# Gap Analysis — Orchestrator Instructions
+
+> Instructions for an LLM to launch parallel subagents that analyze and maintain the gap inventory across all 19 categories.
+
+## Overview
+
+The `gaps/` directory tracks what has been ported from the Go NATS server to the .NET port. Each category has its own `<category>.md` file with:
+- Go reference file paths
+- .NET implementation file paths
+- A Gap Inventory table (populated or empty)
+- Instructions for how to analyze the gap
+
+The orchestrator's job is to launch one subagent per category, collect results, and update `stillmissing.md` with fresh LOC numbers.
+
+---
+
+## Mode Detection
+
+Before launching subagents, determine the mode:
+
+```
+IF the Gap Inventory table in a category file has only the empty template row → INITIAL mode
+IF the Gap Inventory table has populated rows → UPDATE mode
+```
+
+You can check all files at once:
+
+```bash
+for f in gaps/*.md; do
+  [ "$f" = "gaps/stillmissing.md" ] && continue
+  [ "$f" = "gaps/instructions.md" ] && continue
+  rows=$(grep -c '^|[^|]*|[^|]*|' "$f" 2>/dev/null)
+  name=$(basename "$f" .md)
+  if [ "$rows" -le 3 ]; then
+    echo "INITIAL: $name"
+  else
+    echo "UPDATE:  $name"
+  fi
+done
+```
+
+---
+
+## Category Registry
+
+Each category file and its complexity tier (determines model selection and context budget):
+
+| Category File | Tier | Model | Go Source LOC | Notes |
+|---------------|------|-------|-------------:|-------|
+| `core-server.md` | LARGE | opus | 21,223 | client.go alone is ~6,700 lines |
+| `protocol.md` | SMALL | sonnet | 1,829 | 3 files, well-scoped |
+| `subscriptions.md` | SMALL | sonnet | 2,416 | 2 files, well-scoped |
+| `auth-and-accounts.md` | LARGE | opus | 7,260 | accounts.go is ~4,100 lines |
+| `configuration.md` | SMALL | sonnet | 1,871 | 3 files |
+| `routes.md` | MEDIUM | sonnet | 3,314 | 1 file |
+| `gateways.md` | MEDIUM | sonnet | 3,426 | 1 file |
+| `leaf-nodes.md` | MEDIUM | sonnet | 3,470 | 1 file |
+| `jetstream.md` | X-LARGE | opus | 55,228 | Must be split into sub-passes |
+| `raft.md` | MEDIUM | opus | 5,037 | 1 file, complex consensus logic |
+| `mqtt.md` | MEDIUM | opus | 5,882 | 1 file |
+| `websocket.md` | SMALL | sonnet | 1,550 | 1 file, near-complete (99%) |
+| `monitoring.md` | MEDIUM | sonnet | 4,396 | 2 files |
+| `events.md` | MEDIUM | sonnet | 4,133 | 2 files |
+| `tls-security.md` | MEDIUM | opus | 4,207 | Multiple files + subdirs |
+| `internal-ds.md` | MEDIUM | sonnet | 4,020 | Already at 105% parity |
+| `logging.md` | TRIVIAL | haiku | 936 | Likely all NOT_APPLICABLE |
+| `utilities-and-other.md` | MEDIUM | sonnet | 3,282 | Mixed bag of small files |
+| `misc-uncategorized.md` | SMALL | haiku | 660 | Remainder items |
+
+---
+
+## Launching Subagents
+
+### Parallelism Strategy
+
+Launch all subagents in a **single message** using multiple `Task` tool calls. All 19 categories are independent — they read different Go files and write to different output files.
+
+**Exception: JetStream (X-LARGE)**. JetStream has 55,228 Go source LOC across 15 files. A single subagent cannot read all files in one context window. Launch it as a **foreground** task and handle it specially (see JetStream section below).
+
+### Subagent Tool Configuration
+
+For each category, use the `Task` tool with:
+
+```
+subagent_type: "general-purpose"
+model: <from Category Registry table>
+description: "Analyze <category> gaps"
+```
+
+### Prompt Templates
+
+#### INITIAL Mode Prompt
+
+Use this prompt when the Gap Inventory table is empty:
+
+```
+You are analyzing the Go-to-.NET porting gap for the {CATEGORY_NAME} module of the NATS server.
+
+## Your Task
+
+Read the category file at `gaps/{CATEGORY_FILE}` — it contains:
+- Detailed analysis instructions (Steps 1-5)
+- Go reference file paths
+- .NET implementation file paths
+- Key porting notes specific to this category
+
+Follow the instructions in that file exactly. Specifically:
+
+1. Read each Go source file listed under "Go Reference Files (Source)"
+2. For each file, extract all exported types, methods, and standalone functions
+3. Read the .NET files listed under ".NET Implementation Files (Source)"
+4. For each Go symbol, search for its .NET equivalent
+5. Classify each item as PORTED, PARTIAL, MISSING, NOT_APPLICABLE, or DEFERRED
+6. Write the results into the Gap Inventory table in `gaps/{CATEGORY_FILE}`
+
+## Output Format
+
+Edit `gaps/{CATEGORY_FILE}` to populate the Gap Inventory table. Each row must have:
+- **Go Symbol**: The type or function name (e.g., `Server.Start`, `processMsg`, `RouteInfo`)
+- **Go File:Line**: Path relative to repo root with line number (e.g., `golang/nats-server/server/route.go:142`)
+- **Status**: One of PORTED, PARTIAL, MISSING, NOT_APPLICABLE, DEFERRED
+- **NET Equivalent**: Path to .NET file:line if it exists (e.g., `src/NATS.Server/Routes/RouteManager.cs:55`)
+- **Notes**: Brief explanation, especially for PARTIAL (what's missing) and NOT_APPLICABLE (why)
+
+Group rows by Go source file. Use markdown sub-headers within the table comments if helpful.
+
+## Important Rules
+
+- Do NOT modify any source code (.go or .cs files) — this is a read-only analysis
+- Do NOT modify `gaps/stillmissing.md` — the orchestrator handles LOC updates
+- DO update the Change Log at the bottom of the category file
+- Focus on **exported/public API surface** first, then important unexported helpers
+- For large files (>2,000 lines), work through them methodically — top to bottom, section by section
+- Skip trivial getters/setters unless they contain logic
+```
+
+#### UPDATE Mode Prompt
+
+Use this prompt when the Gap Inventory table already has data:
+
+```
+You are updating the gap analysis for the {CATEGORY_NAME} module of the NATS server.
+
+## Your Task
+
+The file `gaps/{CATEGORY_FILE}` already has a populated Gap Inventory table from a previous analysis. Your job is to:
+
+1. Read the current Gap Inventory in `gaps/{CATEGORY_FILE}`
+2. For each item marked MISSING or PARTIAL:
+   - Search the .NET codebase to check if it has been ported since the last analysis
+   - If now ported, update the status to PORTED and fill in the .NET Equivalent path
+   - If partially ported, update notes with current state
+3. Check for NEW Go symbols that were added since the last analysis:
+   - Read the Go source files listed in the category file
+   - Compare against the existing inventory
+   - Add rows for any new symbols not yet tracked
+4. Check for NEW .NET files that may cover previously MISSING items:
+   - Glob for new .cs files in the .NET directories listed in the category file
+   - Cross-reference against MISSING items
+5. Update the Change Log at the bottom
+
+## Output Format
+
+Edit `gaps/{CATEGORY_FILE}` in place:
+- Update existing rows (change status, add .NET paths)
+- Add new rows for newly discovered symbols
+- Do NOT remove existing rows — even if the Go symbol was removed, mark it as NOT_APPLICABLE with a note
+- Update the Change Log with today's date and a summary
+
+## After Updating the Inventory
+
+Run the LOC re-count commands from the "Keeping This File Updated" section of the category file and report the new numbers in your output so the orchestrator can update stillmissing.md.
+
+## Important Rules
+
+- Do NOT modify any source code — read-only analysis
+- Do NOT modify `gaps/stillmissing.md` — report numbers, the orchestrator updates it
+- Preserve all existing inventory rows
+```
+
+---
+
+## JetStream Special Handling
+
+JetStream is too large for a single subagent pass. Split it into 5 sub-passes, launched sequentially or with careful scoping:
+
+| Sub-pass | Go Files | Focus |
+|----------|----------|-------|
+| JS-1: Core | `jetstream.go`, `jetstream_api.go`, `jetstream_events.go`, `jetstream_errors.go`, `jetstream_versioning.go`, `jetstream_batching.go` | Orchestration, API handlers |
+| JS-2: Stream | `stream.go`, `store.go` | Stream lifecycle, retention policies |
+| JS-3: Consumer | `consumer.go` | Consumer state machine, delivery modes |
+| JS-4: Storage | `filestore.go`, `memstore.go`, `dirstore.go`, `disk_avail.go` | Persistent storage, compression, encryption |
+| JS-5: Cluster | `jetstream_cluster.go` | Clustered JetStream coordination |
+
+Launch JS-1 through JS-5 as separate subagents (can run in parallel). Each writes to `gaps/jetstream.md` — but since they write different sections of the Gap Inventory table, use **section headers** within the table to avoid conflicts:
+
+```
+<!-- === JS-1: Core === -->
+| JetStream.enableJetStream | jetstream.go:245 | PORTED | ... | ... |
+...
+<!-- === JS-2: Stream === -->
+| stream.processInboundJetStreamMsg | stream.go:112 | MISSING | | ... |
+...
+```
+
+Alternatively, launch them sequentially (JS-1 first, then resume the agent for JS-2, etc.) to avoid write conflicts.
+
+---
+
+## After All Subagents Complete
+
+### Step 1: Collect LOC Numbers
+
+Each subagent in UPDATE mode should report its re-counted LOC. If running INITIAL mode, the orchestrator runs LOC counts itself:
+
+```bash
+cd /Users/dohertj2/Desktop/natsdotnet
+
+echo "=== .NET Source LOC by Category ==="
+echo "core-server|$(find src/NATS.Server/ -maxdepth 1 -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "protocol|$(find src/NATS.Server/Protocol/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "subscriptions|$(find src/NATS.Server/Subscriptions/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "auth-and-accounts|$(find src/NATS.Server/Auth/ src/NATS.Server/Imports/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "configuration|$(find src/NATS.Server/Configuration/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "routes|$(find src/NATS.Server/Routes/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "gateways|$(find src/NATS.Server/Gateways/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "leaf-nodes|$(find src/NATS.Server/LeafNodes/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "jetstream|$(find src/NATS.Server/JetStream/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "raft|$(find src/NATS.Server/Raft/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "mqtt|$(find src/NATS.Server/Mqtt/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "websocket|$(find src/NATS.Server/WebSocket/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "monitoring|$(find src/NATS.Server/Monitoring/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "events|$(find src/NATS.Server/Events/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "tls-security|$(find src/NATS.Server/Tls/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "internal-ds|$(find src/NATS.Server/Internal/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+echo "logging|0"
+echo "utilities-and-other|$(find src/NATS.Server/IO/ src/NATS.Server/Server/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l)"
+
+echo ""
+echo "=== .NET Source TOTAL ==="
+find src/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l
+
+echo ""
+echo "=== .NET Test TOTAL ==="
+find tests/ -name '*.cs' -type f -exec cat {} + 2>/dev/null | wc -l
+
+echo ""
+echo "=== .NET File Counts ==="
+echo "source|$(find src/ -name '*.cs' -type f | wc -l)"
+echo "test|$(find tests/ -name '*.cs' -type f | wc -l)"
+```
+
+### Step 2: Update stillmissing.md
+
+Compare the new LOC numbers against the values in `gaps/stillmissing.md`. For each category where the .NET LOC changed:
+
+1. Edit the row in the "Source Code by Functional Area" table
+2. Recalculate the `.NET/Go` percentage: `round(.NET LOC / Go LOC * 100)`
+3. Update the `.NET Files` count if new files were added
+4. Update the Summary table totals
+5. Update the "Generated" date in the header
+
+**Do NOT change Go LOC numbers** unless the Go reference has been updated (rare).
+
+### Step 3: Generate Summary Report
+
+After all subagents complete, produce a summary for the user:
+
+```
+## Gap Analysis Summary — {DATE}
+
+### Status Counts (across all categories)
+- PORTED: {n}
+- PARTIAL: {n}
+- MISSING: {n}
+- NOT_APPLICABLE: {n}
+- DEFERRED: {n}
+
+### Categories with Most MISSING Items
+1. {category}: {n} missing
+2. {category}: {n} missing
+3. {category}: {n} missing
+
+### LOC Changes Since Last Run
+| Category | Previous .NET LOC | Current .NET LOC | Delta |
+|----------|------------------:|-----------------:|------:|
+| ... | ... | ... | ... |
+
+### Categories Needing Attention
+- {category}: {reason}
+```
+
+Count statuses by grepping the inventory tables:
+
+```bash
+for f in gaps/*.md; do
+  [ "$f" = "gaps/stillmissing.md" ] && continue
+  [ "$f" = "gaps/instructions.md" ] && continue
+  name=$(basename "$f" .md)
+  ported=$(grep -c '| PORTED |' "$f" 2>/dev/null || echo 0)
+  partial=$(grep -c '| PARTIAL |' "$f" 2>/dev/null || echo 0)
+  missing=$(grep -c '| MISSING |' "$f" 2>/dev/null || echo 0)
+  na=$(grep -c '| NOT_APPLICABLE |' "$f" 2>/dev/null || echo 0)
+  deferred=$(grep -c '| DEFERRED |' "$f" 2>/dev/null || echo 0)
+  echo "$name|$ported|$partial|$missing|$na|$deferred"
+done
+```
+
+---
+
+## Quick-Start: Full Initial Run
+
+Copy-paste orchestration sequence:
+
+1. **Check mode** — run the mode detection script above
+2. **Launch all subagents** — use the Task tool 19 times in a single message (see parallelism strategy)
+3. **Handle JetStream** — if INITIAL mode, launch 5 JS sub-passes
+4. **Wait for completion** — all subagents run in background; you're notified on completion
+5. **Collect LOC** — run the LOC counting script
+6. **Update stillmissing.md** — edit the table with new numbers
+7. **Generate summary** — run the status counting script and format the report
+
+## Quick-Start: Incremental Update
+
+After porting work has been done:
+
+1. **Identify changed categories** — check git diff for which `src/` directories were modified:
+   ```bash
+   git diff --name-only HEAD~10 -- src/ | sed 's|src/NATS.Server/||' | cut -d/ -f1 | sort -u
+   ```
+2. **Launch subagents only for changed categories** — no need to re-analyze unchanged modules
+3. **Collect LOC and update stillmissing.md** — same as full run but only for changed categories
+
+---
+
+## Error Handling
+
+- **Subagent hits context limit**: This typically happens with LARGE/X-LARGE categories. If a subagent reports it couldn't finish, resume it with the `resume` parameter and instruct it to continue from where it left off.
+- **Subagent finds no .NET files**: The category may not be ported yet. Mark all items as MISSING and note the empty state.
+- **Conflicting writes to jetstream.md**: If running JS sub-passes in parallel, use the section-header approach described above. If conflicts occur, run sub-passes sequentially instead.
+- **Go files changed upstream**: If the Go reference was updated (`golang/nats-server/`), run a full INITIAL analysis to catch new symbols. Check with: `git log --oneline -5 -- golang/`