natsnet/docs/plans/2026-02-27-batch-11-filestore-init-implementation-plan.md

Batch 11 FileStore Init Implementation Plan

For Codex: REQUIRED SUB-SKILL: Use executeplan to implement this plan task-by-task.

Goal: Implement and verify all Batch 11 FileStore Init features/tests with real behavior parity to server/filestore.go, then close the batch with evidence-backed status updates.

Architecture: Execute Batch 11 in three vertical feature groups (17, 16, 6 features), each followed by targeted test waves and strict gates. Keep implementation centered in JetStream/FileStore + MessageBlock + FileStoreTypes, and use backlog test classes as the only status evidence source. Every status change is chunked, logged, and tied to passing build/test output.

Tech Stack: .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (porting.db)

Design doc: docs/plans/2026-02-27-batch-11-filestore-init-design.md

---

I'm using writeplan to create the implementation plan.

Batch Inputs

• Batch: 11 (FileStore Init)
• Dependency: Batch 8
• Features: 39
• Tests: 123
• Go source focus: golang/nats-server/server/filestore.go

Feature groups (max ~20 each):

• Group 1 (17): 955,956,957,958,960,961,962,963,964,965,966,967,968,969,970,972,974
• Group 2 (16): 975,976,978,979,980,989,990,998,1057,1152,1153,1154,1158,1163,1164,1165
• Group 3 (6): 1200,1235,1239,1248,1261,1262

Primary test classes:

• JetStreamFileStoreTests (116 mapped tests)
• ConcurrencyTests1 (5 mapped tests)
• ConcurrencyTests2 (2 mapped tests)

---

MANDATORY VERIFICATION PROTOCOL

NON-NEGOTIABLE: every task below must follow this protocol exactly.

Per-Feature Verification Loop (REQUIRED for every feature ID)

1. Read source intent from Go:
   • dotnet run --project tools/NatsNet.PortTracker -- feature show <FEATURE_ID> --db porting.db
   • Read exact Go span in golang/nats-server/server/filestore.go.
2. Write minimal C# implementation in mapped target files.
3. Build immediately:
   • dotnet build dotnet/
4. Run related tests (method/class filter) for that feature's behavior.
5. Only then include the feature in complete/verified candidate lists.

Stub Detection Check (REQUIRED after each feature subgroup and test class)

Run all scans below; any hit is a blocker:

# Production stubs/placeholders
rg -n "NotImplementedException|TODO|PLACEHOLDER|throw new NotSupportedException" \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream -g '*.cs'

# Empty method bodies in FileStore-related source
rg -n "^\s*(public|private|internal|protected).*\)\s*\{\s*\}$" \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs

# Test stubs/placeholders
rg -n "NotImplementedException|Assert\.True\(true\)|Assert\.Pass|// TODO|// PLACEHOLDER|ShouldBe\(true\);" \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs

Build Gate (REQUIRED after each feature group)

dotnet build dotnet/ must pass before moving to next group or updating status.

Test Gate (REQUIRED before marking features verified)

All related tests for the changed feature group must pass:

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests1" \
  --verbosity normal

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests2" \
  --verbosity normal

Status Update Protocol (REQUIRED, evidence-backed)

• Max 15 IDs per feature batch-update or test batch-update call.
• Status flow:
  • Features: deferred -> stub -> complete -> verified
  • Tests: deferred -> stub -> verified (or deferred with explicit reason)
• Save evidence output before each status update (recommended path: /tmp/batch11-evidence/).
• Never mark verified without matching passing output for the exact scope.

Example chunked update:

/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "955,956,957,958,960,961,962,963,964,965,966,967,968,969,970" \
  --set-status complete --db porting.db --execute

Checkpoint Protocol Between Tasks (REQUIRED)

After each major task (each group features+tests):

1. dotnet build dotnet/
2. Run the three backlog test-class gates above.
3. Commit checkpoint:
   • git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db
   • git commit -m "feat(batch11): complete group <N> filestore init + tests"

Do not start the next task until checkpoint passes.

---

ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

Forbidden Patterns

Do not introduce or leave any of the following in feature or test code:

• throw new NotImplementedException(...)
• Empty method bodies { } for mapped feature methods
• // TODO or // PLACEHOLDER in implemented Batch 11 methods/tests
• Tests that only assert constants/string literals unrelated to production behavior
• Assert.True(true), Assert.Pass(), or equivalent always-pass constructs

Hard Limits

• Max ~20 features per task group (already fixed as 17/16/6)
• Max 15 IDs per status update command
• Max one feature group per verification/update cycle
• Zero stub-scan hits allowed before any complete/verified updates
• No deferred item may be left without a concrete reason

If You Get Stuck

1. Stop implementation for that item.
2. Mark it deferred with explicit reason (infra, missing dependency, non-portable behavior, etc.).
3. Do not add placeholder logic or fake tests.
4. Continue with next item in the same group.

Example:

/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- \
  feature update <ID> --status deferred --db porting.db \
  --override "blocked: requires <specific dependency/infra>"

Use the same pattern for tests.

---

Task 1: Batch 11 Working Set and Evidence Scaffold

Files:

• Modify: porting.db
• Create: /tmp/batch11-evidence/

Step 1: Confirm batch and dependency state

Run:

/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 11 --db porting.db
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db

Expected: Batch 11 pending, depends on 8, 39 features, 123 tests.

Step 2: Start batch

Run:

/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch start 11 --db porting.db

Expected: batch marked in-progress (if dependency checks pass).

Step 3: Mark Group 1 features/tests as stub (chunked <=15 IDs)

Run chunked feature batch-update ... --set-status stub and test batch-update ... --set-status stub for Group 1 scope. Expected: only scoped IDs moved to stub.

Step 4: Commit scaffold checkpoint

Run:

git add porting.db
git commit -m "chore(batch11): start batch and stage group1 ids"

Task 2: Group 1 Feature Implementation (17 features)

Files:

• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs
• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs
• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStoreTypes.cs

Step 1: Implement constructors/bootstrap methods

Feature IDs: 955, 956, 960

Step 2: Implement block lock + buffer pool methods

Feature IDs: 957, 958, 967, 968, 969, 970

Step 3: Implement encryption key lifecycle + metadata write

Feature IDs: 961, 962, 963, 964, 965, 966, 972

Step 4: Implement message block recovery path

Feature ID: 974

Step 5: Apply Mandatory Verification Protocol

• Run per-feature loop for all 17 IDs.
• Run stub scans.
• Run dotnet build dotnet/.

Step 6: Mark features complete in chunks (<=15 IDs)

Use feature batch-update commands with evidence log references.

Task 3: Group 1 Test Porting and Verification

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs

Step 1: Port Group 1-related tests (wave A)

Primary IDs: 351,352,353,355,357,358,359,360,361,362,369,370,371,374,375,376,377,380,381,388,397,398,400,401,421,422,423,424,425,426,427,429,430,431,432,433,434,439,443,444,532

Step 2: Run class filter and verify pass count growth

Run:

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal

Expected: no failures for newly ported tests.

Step 3: Run stub scans + build gate

Apply mandatory scans and dotnet build dotnet/.

Step 4: Update test statuses to verified (<=15 IDs/chunk)

Use test batch-update in chunks with evidence.

Step 5: Verify Group 1 features to verified

Only after test gate is green.

Task 4: Group 1 Checkpoint

Files:

• Modify: porting.db

Step 1: Full checkpoint gate

• dotnet build dotnet/
• class test gates for JetStreamFileStoreTests, ConcurrencyTests1, ConcurrencyTests2

Step 2: Commit

git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db
git commit -m "feat(batch11): complete group1 filestore init"

Task 5: Group 2 Feature Implementation (16 features)

Files:

• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs
• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs

Step 1: Implement lost-data and rebuild-state logic

Feature IDs: 975, 976, 978, 979, 980

Step 2: Implement tracking/copy/flusher helpers

Feature IDs: 989, 990, 998, 1057

Step 3: Implement size/header/subject helper funcs

Feature IDs: 1152, 1153, 1154, 1158, 1163, 1164, 1165

Step 4: Run mandatory protocol (per-feature loop + build gate + scans)

Step 5: Mark features complete (chunked <=15)

Task 6: Group 2 Test Porting and Verification

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs

Step 1: Port Group 2-related tests (wave B)

Primary IDs: 364,366,367,406,408,410,414,415,417,419,420,436,441,446,447,450,453,454,456,467,468,469,471,480,481,484,487,488,489,504,505,506,507,508,509,510,511,512,513,514,515,516,517,520,521,522,524,525,526,536,537,538,539,540,541,542,543,556,558,560,563,564,565,576,585,592,2452,2453,2462,2491,2501

Step 2: Run test gate for JetStreamFileStoreTests + concurrency classes

Step 3: Run mandatory scans + build gate

Step 4: Mark tests verified in <=15 ID chunks

Step 5: Mark Group 2 features verified after gate passes

Task 7: Group 2 Checkpoint

Files:

• Modify: porting.db

Step 1: Full build + three class test gates

Step 2: Commit checkpoint

git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db
git commit -m "feat(batch11): complete group2 filestore state and utility paths"

Task 8: Group 3 Feature Implementation (6 features)

Files:

• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs
• Modify: dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs

Step 1: Implement timestamp/init semaphore helpers

Feature IDs: 1200, 1235

Step 2: Implement consumer header/state decode

Feature IDs: 1239, 1248

Step 3: Implement atomic file-write helpers

Feature IDs: 1261, 1262

Step 4: Run mandatory protocol and mark features complete (chunked <=15)

Task 9: Group 3 Test Porting and Verification

Files:

• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs
• Modify: dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs

Step 1: Port Group 3-related tests (wave C)

Primary IDs: 385,386,387,393,395,396,402,440,2427,2447

Step 2: Run class gates and verify all wave C tests pass

Step 3: Run mandatory scans + build gate

Step 4: Mark tests verified (<=15 IDs/chunk)

Step 5: Mark Group 3 features verified (<=15 IDs/chunk)

Task 10: Batch 11 Final Closure

Files:

• Modify: porting.db
• Generate: reports/current.md

Step 1: Final full verification

Run:

dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests1" \
  --verbosity normal
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests2" \
  --verbosity normal

Expected: all relevant tests pass, zero failures.

Step 2: Final stub audit (src + tests)

Run mandatory stub scans again; expected zero matches.

Step 3: Complete batch

/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch complete 11 --db porting.db

Expected: success with no unverified blockers.

Step 4: Regenerate report and final commit

./reports/generate-report.sh
git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db reports/
git commit -m "feat(batch11): complete filestore init feature and test port"

---

Execution Notes

• Always run status updates in explicit chunks and record matching evidence.
• If any test is infra-blocked, keep deferred with reason; never replace with placeholder assertions.
• If unexpected unrelated workspace changes appear during execution, stop and ask for direction before proceeding.