natsnet/docs/plans/2026-02-27-batch-11-filestore-init-implementation-plan.md

# Batch 11 FileStore Init Implementation Plan

> **For Codex:** REQUIRED SUB-SKILL: Use `executeplan` to implement this plan task-by-task.

**Goal:** Implement and verify all Batch 11 FileStore Init features/tests with real behavior parity to `server/filestore.go`, then close the batch with evidence-backed status updates.

**Architecture:** Execute Batch 11 in three vertical feature groups (17, 16, 6 features), each followed by targeted test waves and strict gates. Keep implementation centered in `JetStream/FileStore` + `MessageBlock` + `FileStoreTypes`, and use backlog test classes as the only status evidence source. Every status change is chunked, logged, and tied to passing build/test output.

**Tech Stack:** .NET 10, C# latest, xUnit 3, Shouldly, NSubstitute, PortTracker CLI, SQLite (`porting.db`)

**Design doc:** `docs/plans/2026-02-27-batch-11-filestore-init-design.md`

---

I'm using `writeplan` to create the implementation plan.

## Batch Inputs

- Batch: `11` (`FileStore Init`)
- Dependency: Batch `8`
- Features: `39`
- Tests: `123`
- Go source focus: `golang/nats-server/server/filestore.go`

Feature groups (max ~20 each):
- Group 1 (17): `955,956,957,958,960,961,962,963,964,965,966,967,968,969,970,972,974`
- Group 2 (16): `975,976,978,979,980,989,990,998,1057,1152,1153,1154,1158,1163,1164,1165`
- Group 3 (6): `1200,1235,1239,1248,1261,1262`

Primary test classes:
- `JetStreamFileStoreTests` (116 mapped tests)
- `ConcurrencyTests1` (5 mapped tests)
- `ConcurrencyTests2` (2 mapped tests)

---

## MANDATORY VERIFICATION PROTOCOL

> **NON-NEGOTIABLE:** every task below must follow this protocol exactly.

### Per-Feature Verification Loop (REQUIRED for every feature ID)

1. Read source intent from Go:
   - `dotnet run --project tools/NatsNet.PortTracker -- feature show <FEATURE_ID> --db porting.db`
   - Read exact Go span in `golang/nats-server/server/filestore.go`.
2. Write minimal C# implementation in mapped target files.
3. Build immediately:
   - `dotnet build dotnet/`
4. Run related tests (method/class filter) for that feature's behavior.
5. Only then include the feature in `complete/verified` candidate lists.

### Stub Detection Check (REQUIRED after each feature subgroup and test class)

Run all scans below; any hit is a blocker:

```bash
# Production stubs/placeholders
rg -n "NotImplementedException|TODO|PLACEHOLDER|throw new NotSupportedException" \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream -g '*.cs'

# Empty method bodies in FileStore-related source
rg -n "^\s*(public|private|internal|protected).*\)\s*\{\s*\}$" \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs \
  dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs

# Test stubs/placeholders
rg -n "NotImplementedException|Assert\.True\(true\)|Assert\.Pass|// TODO|// PLACEHOLDER|ShouldBe\(true\);" \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs
```

### Build Gate (REQUIRED after each feature group)

`dotnet build dotnet/` must pass before moving to next group or updating status.

### Test Gate (REQUIRED before marking features `verified`)

All related tests for the changed feature group must pass:

```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests1" \
  --verbosity normal

dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests2" \
  --verbosity normal
```

### Status Update Protocol (REQUIRED, evidence-backed)

- Max **15 IDs** per `feature batch-update` or `test batch-update` call.
- Status flow:
  - Features: `deferred -> stub -> complete -> verified`
  - Tests: `deferred -> stub -> verified` (or `deferred` with explicit reason)
- Save evidence output before each status update (recommended path: `/tmp/batch11-evidence/`).
- Never mark `verified` without matching passing output for the exact scope.

Example chunked update:

```bash
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- \
  feature batch-update --ids "955,956,957,958,960,961,962,963,964,965,966,967,968,969,970" \
  --set-status complete --db porting.db --execute
```

### Checkpoint Protocol Between Tasks (REQUIRED)

After each major task (each group features+tests):

1. `dotnet build dotnet/`
2. Run the three backlog test-class gates above.
3. Commit checkpoint:
   - `git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog porting.db`
   - `git commit -m "feat(batch11): complete group <N> filestore init + tests"`

Do not start the next task until checkpoint passes.

---

## ANTI-STUB GUARDRAILS (NON-NEGOTIABLE)

### Forbidden Patterns

Do not introduce or leave any of the following in feature or test code:

- `throw new NotImplementedException(...)`
- Empty method bodies `{ }` for mapped feature methods
- `// TODO` or `// PLACEHOLDER` in implemented Batch 11 methods/tests
- Tests that only assert constants/string literals unrelated to production behavior
- `Assert.True(true)`, `Assert.Pass()`, or equivalent always-pass constructs

### Hard Limits

- Max ~20 features per task group (already fixed as 17/16/6)
- Max 15 IDs per status update command
- Max one feature group per verification/update cycle
- Zero stub-scan hits allowed before any `complete`/`verified` updates
- No deferred item may be left without a concrete reason

### If You Get Stuck

1. Stop implementation for that item.
2. Mark it `deferred` with explicit reason (infra, missing dependency, non-portable behavior, etc.).
3. Do **not** add placeholder logic or fake tests.
4. Continue with next item in the same group.

Example:

```bash
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- \
  feature update <ID> --status deferred --db porting.db \
  --override "blocked: requires <specific dependency/infra>"
```

Use the same pattern for tests.

---

### Task 1: Batch 11 Working Set and Evidence Scaffold

**Files:**
- Modify: `porting.db`
- Create: `/tmp/batch11-evidence/`

**Step 1: Confirm batch and dependency state**

Run:
```bash
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch show 11 --db porting.db
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch list --db porting.db
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
```
Expected: Batch 11 pending, depends on 8, 39 features, 123 tests.

**Step 2: Start batch**

Run:
```bash
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch start 11 --db porting.db
```
Expected: batch marked in-progress (if dependency checks pass).

**Step 3: Mark Group 1 features/tests as stub (chunked <=15 IDs)**

Run chunked `feature batch-update ... --set-status stub` and `test batch-update ... --set-status stub` for Group 1 scope.
Expected: only scoped IDs moved to `stub`.

**Step 4: Commit scaffold checkpoint**

Run:
```bash
git add porting.db
git commit -m "chore(batch11): start batch and stage group1 ids"
```

### Task 2: Group 1 Feature Implementation (17 features)

**Files:**
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStoreTypes.cs`

**Step 1: Implement constructors/bootstrap methods**

Feature IDs: `955, 956, 960`

**Step 2: Implement block lock + buffer pool methods**

Feature IDs: `957, 958, 967, 968, 969, 970`

**Step 3: Implement encryption key lifecycle + metadata write**

Feature IDs: `961, 962, 963, 964, 965, 966, 972`

**Step 4: Implement message block recovery path**

Feature ID: `974`

**Step 5: Apply Mandatory Verification Protocol**

- Run per-feature loop for all 17 IDs.
- Run stub scans.
- Run `dotnet build dotnet/`.

**Step 6: Mark features complete in chunks (<=15 IDs)**

Use `feature batch-update` commands with evidence log references.

### Task 3: Group 1 Test Porting and Verification

**Files:**
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`

**Step 1: Port Group 1-related tests (wave A)**

Primary IDs:
`351,352,353,355,357,358,359,360,361,362,369,370,371,374,375,376,377,380,381,388,397,398,400,401,421,422,423,424,425,426,427,429,430,431,432,433,434,439,443,444,532`

**Step 2: Run class filter and verify pass count growth**

Run:
```bash
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal
```
Expected: no failures for newly ported tests.

**Step 3: Run stub scans + build gate**

Apply mandatory scans and `dotnet build dotnet/`.

**Step 4: Update test statuses to verified (<=15 IDs/chunk)**

Use `test batch-update` in chunks with evidence.

**Step 5: Verify Group 1 features to verified**

Only after test gate is green.

### Task 4: Group 1 Checkpoint

**Files:**
- Modify: `porting.db`

**Step 1: Full checkpoint gate**

- `dotnet build dotnet/`
- class test gates for `JetStreamFileStoreTests`, `ConcurrencyTests1`, `ConcurrencyTests2`

**Step 2: Commit**

```bash
git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db
git commit -m "feat(batch11): complete group1 filestore init"
```

### Task 5: Group 2 Feature Implementation (16 features)

**Files:**
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs`

**Step 1: Implement lost-data and rebuild-state logic**

Feature IDs: `975, 976, 978, 979, 980`

**Step 2: Implement tracking/copy/flusher helpers**

Feature IDs: `989, 990, 998, 1057`

**Step 3: Implement size/header/subject helper funcs**

Feature IDs: `1152, 1153, 1154, 1158, 1163, 1164, 1165`

**Step 4: Run mandatory protocol (per-feature loop + build gate + scans)**

**Step 5: Mark features complete (chunked <=15)**

### Task 6: Group 2 Test Porting and Verification

**Files:**
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`

**Step 1: Port Group 2-related tests (wave B)**

Primary IDs:
`364,366,367,406,408,410,414,415,417,419,420,436,441,446,447,450,453,454,456,467,468,469,471,480,481,484,487,488,489,504,505,506,507,508,509,510,511,512,513,514,515,516,517,520,521,522,524,525,526,536,537,538,539,540,541,542,543,556,558,560,563,564,565,576,585,592,2452,2453,2462,2491,2501`

**Step 2: Run test gate for JetStreamFileStoreTests + concurrency classes**

**Step 3: Run mandatory scans + build gate**

**Step 4: Mark tests verified in <=15 ID chunks**

**Step 5: Mark Group 2 features verified after gate passes**

### Task 7: Group 2 Checkpoint

**Files:**
- Modify: `porting.db`

**Step 1: Full build + three class test gates**

**Step 2: Commit checkpoint**

```bash
git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db
git commit -m "feat(batch11): complete group2 filestore state and utility paths"
```

### Task 8: Group 3 Feature Implementation (6 features)

**Files:**
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`
- Modify: `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs`

**Step 1: Implement timestamp/init semaphore helpers**

Feature IDs: `1200, 1235`

**Step 2: Implement consumer header/state decode**

Feature IDs: `1239, 1248`

**Step 3: Implement atomic file-write helpers**

Feature IDs: `1261, 1262`

**Step 4: Run mandatory protocol and mark features complete (chunked <=15)**

### Task 9: Group 3 Test Porting and Verification

**Files:**
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
- Modify: `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`

**Step 1: Port Group 3-related tests (wave C)**

Primary IDs:
`385,386,387,393,395,396,402,440,2427,2447`

**Step 2: Run class gates and verify all wave C tests pass**

**Step 3: Run mandatory scans + build gate**

**Step 4: Mark tests verified (<=15 IDs/chunk)**

**Step 5: Mark Group 3 features verified (<=15 IDs/chunk)**

### Task 10: Batch 11 Final Closure

**Files:**
- Modify: `porting.db`
- Generate: `reports/current.md`

**Step 1: Final full verification**

Run:
```bash
dotnet build dotnet/
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.JetStreamFileStoreTests" \
  --verbosity normal
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests1" \
  --verbosity normal
dotnet test dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ \
  --filter "FullyQualifiedName~ZB.MOM.NatsNet.Server.Tests.ImplBacklog.ConcurrencyTests2" \
  --verbosity normal
```
Expected: all relevant tests pass, zero failures.

**Step 2: Final stub audit (src + tests)**

Run mandatory stub scans again; expected zero matches.

**Step 3: Complete batch**

```bash
/usr/local/share/dotnet/dotnet run --project tools/NatsNet.PortTracker -- batch complete 11 --db porting.db
```
Expected: success with no unverified blockers.

**Step 4: Regenerate report and final commit**

```bash
./reports/generate-report.sh
git add dotnet/src/ZB.MOM.NatsNet.Server/JetStream \
  dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog \
  porting.db reports/
git commit -m "feat(batch11): complete filestore init feature and test port"
```

---

## Execution Notes

- Always run status updates in explicit chunks and record matching evidence.
- If any test is infra-blocked, keep deferred with reason; never replace with placeholder assertions.
- If unexpected unrelated workspace changes appear during execution, stop and ask for direction before proceeding.