dohertj2/natsnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
3ddc5cb1a2eb27692045a328bdeda159c4b5c434
natsnet/docs/plans/2026-02-27-batch-32-js-cluster-meta-implementation-plan.md
Joseph Doherty f8dce79ac0 Add batch plans for batches 31-36 (rounds 16-18) 
Generated design docs and implementation plans via Codex for:
- Batch 31: Raft Part 2
- Batch 32: JS Cluster Meta
- Batch 33: JS Cluster Streams
- Batch 34: JS Cluster Consumers
- Batch 35: JS Cluster Remaining
- Batch 36: Stream Lifecycle

All plans include mandatory verification protocol and anti-stub guardrails.
Updated batches.md with file paths and planned status.
2026-02-27 17:01:31 -05:00

14 KiB
Raw Blame History

Batch 32 JS Cluster Meta Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Port and verify Batch 32 (JS Cluster Meta) from server/jetstream_cluster.go with strict evidence gates for both features and tests.

Architecture: Implement mapped cluster-meta methods in three source-ordered feature groups (<=20 IDs each) across NatsServer, JetStream, JetStreamCluster, RecoveryUpdates, and Account. Then port mapped backlog tests in three waves. Every promotion (stub -> complete -> verified) is gated by build/test evidence, stub scans, and chunked tracker updates (<=15 IDs per update).

Tech Stack: .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (porting.db)

Design doc: docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md

Batch 32 Scope

  • Batch ID: 32
  • Name: JS Cluster Meta
  • Dependencies: 27, 31
  • Go source: golang/nats-server/server/jetstream_cluster.go
  • Features: 58
  • Tests: 36

If dotnet is not available on PATH, run commands with /usr/local/share/dotnet/dotnet instead.

Primary source files:

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs
  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs
  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs
  • Create/Modify: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs

Primary test files:

  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs

MANDATORY VERIFICATION PROTOCOL

NON-NEGOTIABLE: Every feature ID and test ID in Batch 32 must pass this protocol.

Dependency Preflight Gate (before any status change)

  1. Validate dependencies and readiness:
dotnet run --project tools/NatsNet.PortTracker -- batch show 27 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 31 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch show 32 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- batch ready --db porting.db
  1. Only when Batch 32 is ready, start it:
dotnet run --project tools/NatsNet.PortTracker -- batch start 32 --db porting.db
  1. Capture clean baseline:
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/

Per-Feature Verification Loop (REQUIRED per feature ID)

  1. Inspect mapped feature and Go span:
dotnet run --project tools/NatsNet.PortTracker -- feature show <feature_id> --db porting.db
  1. Mark only the active feature as stub (tracker in-progress marker):
dotnet run --project tools/NatsNet.PortTracker -- feature update <feature_id> --status stub --db porting.db
  1. Add/adjust a focused test that fails for the intended behavior.
  2. Run focused test; confirm expected failure.
  3. Implement minimal production change for that feature.
  4. Run Stub Detection Check (below).
  5. Run Build Gate (below).
  6. Run Test Gate (below).
  7. If all pass, mark feature complete. Promote to verified only at task checkpoint.

Stub Detection Check (REQUIRED after each feature loop and each test loop)

Run against changed C# files only:

changed_files=$(git diff --name-only -- dotnet/src/ZB.MOM.NatsNet.Server dotnet/tests/ZB.MOM.NatsNet.Server.Tests | rg "\.cs$" || true)
if [ -n "$changed_files" ]; then
  echo "$changed_files" | xargs rg -n "(NotImplementedException|// TODO|// PLACEHOLDER|Assert\.True\(true\)|Assert\.Pass|=>\s*default;|=>\s*null;|return\s+null;\s*$|return\s+0;\s*$|return\s+false;\s*$)"
fi

Any match in mapped methods/tests blocks promotion.

Build Gate (REQUIRED)

Run dotnet build dotnet/:

  • after each feature loop,
  • after each test loop,
  • before every status batch-update,
  • at every checkpoint between tasks.

Test Gate (REQUIRED)

Minimum gates:

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~JetStream"
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ --filter "FullyQualifiedName~ImplBacklog"

For every mapped backlog test method:

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~<ClassName>.<MethodName>" \
  --verbosity normal

Verify discovery and pass (Passed: 1, Failed: 0) before promotion.

Status Update Protocol (HARD LIMIT: <=15 IDs per batch-update)

  • Allowed transitions: deferred/not_started -> stub -> complete -> verified
  • Never include more than 15 IDs in one update command.
  • Never mark verified without passing stub scan + build gate + test gate.
  • Never update IDs outside the currently active task.

Templates:

dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

dotnet run --project tools/NatsNet.PortTracker -- \
  test batch-update --ids "<max 15 ids>" --set-status <stub|complete|verified> --db porting.db --execute

Checkpoint Protocol Between Tasks (REQUIRED)

At the end of every task before starting the next:

  1. Run Stub Detection Check.
  2. Run Build Gate.
  3. Run targeted Test Gate for touched classes.
  4. Run full unit tests:
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
  1. Apply status promotions (complete/verified) for that task only (<=15 IDs per command).
  2. Commit checkpoint.

ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

Forbidden Patterns (Features + Tests)

Any of these in mapped/touched files is disallowed:

  • throw new NotImplementedException()
  • Empty method bodies for mapped feature methods
  • Placeholder comments (// TODO, // PLACEHOLDER, // later)
  • Fake-pass assertions (Assert.True(true), Assert.Pass())
  • Trivial string/self assertions unrelated to production behavior
  • Unconditional constant returns in mapped methods without state/input usage (return false;, return 0;, return null;, return string.Empty;)
  • Catch-and-ignore blocks that suppress real failures just to pass tests

Hard Limits

  • Max features per implementation task group: 20
  • Max IDs per feature batch-update: 15
  • Max IDs per test batch-update: 15
  • One active feature loop at a time
  • One active test loop at a time
  • No verified status without checkpoint evidence
  • Mandatory checkpoint between tasks

If You Get Stuck (REQUIRED)

  1. Stop work on that specific ID.
  2. Do not keep or add placeholder/stub code.
  3. Mark the item deferred immediately with explicit reason:
dotnet run --project tools/NatsNet.PortTracker -- feature update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- test update <id> --status deferred --override "blocked: <specific reason>" --db porting.db
  1. Add a short note in tracker/test comment with concrete blocker details.
  2. Continue with the next unblocked ID.

Deferred-with-reason is valid progress. Stubs are not.

Feature Groups (max ~20 each)

Group A (20) - Unsupported assignments + server/meta state entry points

IDs:
1520,1521,1522,1523,1524,1525,1526,1527,1528,1529,1530,1531,1532,1533,1534,1535,1536,1537,1538,1539

Group B (20) - Health/leadership checks + clustering enable + assignment checks

IDs:
1540,1541,1542,1543,1544,1545,1546,1547,1548,1549,1550,1551,1552,1553,1554,1555,1556,1557,1558,1559

Group C (18) - Stream/consumer leadership internals + inflight tracking + recovery/orphans

IDs:
1560,1561,1562,1563,1564,1565,1566,1567,1568,1569,1570,1571,1572,1573,1574,1575,1576,1577

Test Waves

Wave T1 (13) - Core cluster tests (files 1/2)

IDs:
772,774,775,791,809,810,811,817,853,914,993,1014,1028

Wave T2 (12) - Advanced cluster tests (files 3/4 + long)

IDs:
1060,1088,1098,1106,1109,1122,1128,1136,1194,1211,1212,1217

Wave T3 (11) - Cross-domain integration-facing backlog tests

IDs:
1406,1453,1454,1457,1465,1528,2225,2390,2459,2489,2689

Task 1: Preflight and Baseline

Files:

  • Read: docs/plans/2026-02-27-batch-32-js-cluster-meta-design.md
  • Read: golang/nats-server/server/jetstream_cluster.go

Step 1: Run dependency preflight gate (27/31/32 + ready list).
Step 2: Start batch 32 only if ready.
Step 3: Capture baseline build and full test run.
Step 4: Apply checkpoint protocol before feature work.

Task 2: Implement Feature Group A (20)

Files:

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs
  • Create/Modify: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs

Step 1: Set Group A to stub in two chunks (15 + 5).
Step 2: Execute per-feature verification loop for each ID.
Step 3: Run checkpoint protocol.
Step 4: Promote passing IDs to complete and verified in <=15 chunks.

Task 3: Implement Feature Group B (20)

Files:

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs
  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs
  • Create/Modify: dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.JetStreamClusterMeta.cs

Step 1: Set Group B to stub in two chunks (15 + 5).
Step 2: Execute per-feature verification loop for each ID.
Step 3: Run checkpoint protocol.
Step 4: Promote passing IDs to complete and verified in <=15 chunks.

Task 4: Implement Feature Group C (18)

Files:

  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamClusterTypes.cs
  • Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/JetStreamTypes.cs

Step 1: Set Group C to stub in two chunks (15 + 3).
Step 2: Execute per-feature verification loop for each ID.
Step 3: Run checkpoint protocol.
Step 4: Promote passing IDs to complete and verified in <=15 chunks.

Task 5: Port Test Wave T1 (13)

Files:

  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests1.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests2.Impltests.cs

Step 1: Set T1 test IDs to stub (single chunk of 13).
Step 2: Execute per-test verification loop for each ID (single-test proof required).
Step 3: Run class-level filters for JetStreamClusterTests1 and JetStreamClusterTests2.
Step 4: Run checkpoint protocol and promote passing IDs in <=15 chunk.

Task 6: Port Test Wave T2 (12)

Files:

  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests3.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterTests4.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamClusterLongTests.Impltests.cs

Step 1: Set T2 test IDs to stub (single chunk of 12).
Step 2: Execute per-test verification loop for each ID.
Step 3: Run class-level filters for touched classes.
Step 4: Run checkpoint protocol and promote passing IDs.

Task 7: Port Test Wave T3 (11)

Files:

  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamSuperClusterTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamLeafNodeTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamEngineTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/MqttHandlerTests.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs
  • Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs
  • Create/Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/RaftNodeTests.Impltests.cs

Step 1: Set T3 test IDs to stub (single chunk of 11).
Step 2: Execute per-test verification loop for each ID.
Step 3: Run class-level filters for all touched backlog classes.
Step 4: Run checkpoint protocol and promote passing IDs.

Task 8: Final Batch 32 Verification and Closure

Files:

  • Modify: porting.db
  • Update: reports/current.md (generated)

Step 1: Final gates:

dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/
dotnet run --project tools/NatsNet.PortTracker -- audit --type features --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- audit --type tests --db porting.db

Step 2: Resolve any remaining stub statuses:

  • Promote to verified with evidence, or
  • Set to deferred with explicit blocker reason.

Step 3: Complete batch:

dotnet run --project tools/NatsNet.PortTracker -- batch complete 32 --db porting.db

Step 4: Refresh report:

./reports/generate-report.sh
Reference in New Issue View Git Blame Copy Permalink