Add batch plans for batches 6-7, 9-12, 16-17 (rounds 4-7)
Generated design docs and implementation plans via Codex for: - Batch 6: Opts package-level functions - Batch 7: Opts class methods + Reload - Batch 9: Auth, DirStore, OCSP foundations - Batch 10: OCSP Cache + JS Events - Batch 11: FileStore Init - Batch 12: FileStore Recovery - Batch 16: Client Core (first half) - Batch 17: Client Core (second half) All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status.
This commit is contained in:
114
docs/plans/2026-02-27-batch-11-filestore-init-design.md
Normal file
114
docs/plans/2026-02-27-batch-11-filestore-init-design.md
Normal file
@@ -0,0 +1,114 @@
|
||||
# Batch 11 FileStore Init Design
|
||||
|
||||
**Date:** 2026-02-27
|
||||
**Batch:** 11 (`FileStore Init`)
|
||||
**Dependency:** Batch 8 (`Store Interfaces`)
|
||||
**Scope:** 39 features + 123 tests mapped to `server/filestore.go`
|
||||
|
||||
---
|
||||
|
||||
## 1. Context and Constraints
|
||||
|
||||
- Batch 11 is currently `pending` with all 39 features and all 123 tests in `deferred` status.
|
||||
- Current .NET `JetStreamFileStore` implementation is mostly a delegation shell to `JetStreamMemStore`; most Batch 11 methods are not present yet.
|
||||
- Existing backlog tests for this area include placeholder-style tests that must not be treated as verification evidence.
|
||||
- Planning only in this session: produce design + implementation plan documents, no runtime code changes.
|
||||
|
||||
## 2. Success Criteria
|
||||
|
||||
- Implement all Batch 11 feature IDs with real C# behavior matching Go intent for `filestore.go` initialization-related scope.
|
||||
- Port and verify all Batch 11 tests into real behavioral tests (no placeholder assertions).
|
||||
- Enforce strict verification and anti-stub guardrails before any status updates.
|
||||
- Preserve dependency order and use PortTracker updates with evidence-backed chunks.
|
||||
|
||||
## 3. Approach Options
|
||||
|
||||
### Option A (Recommended): Vertical feature groups with co-evolving tests
|
||||
|
||||
Implement features in 3 groups (17, 16, 6 features), each followed by targeted test waves and status updates.
|
||||
|
||||
- Pros: bounded risk, quick feedback, easier rollback and review, aligns with dependency and guardrail requirements.
|
||||
- Cons: repeated build/test cycles increase total command count.
|
||||
|
||||
### Option B: Implement all 39 features first, then all 123 tests
|
||||
|
||||
- Pros: fewer context switches between prod/test files.
|
||||
- Cons: very high integration risk; defects surface late; hard to attribute failures to specific changes.
|
||||
|
||||
### Option C: Utilities-first (helpers, codecs, file writers) then constructors/recovery
|
||||
|
||||
- Pros: helper APIs stabilize early.
|
||||
- Cons: delayed validation of end-to-end init behavior; tends to accumulate partial implementations.
|
||||
|
||||
### Recommendation
|
||||
|
||||
Use **Option A**. This gives the strongest control over regressions and makes status updates defensible under audit.
|
||||
|
||||
## 4. Proposed Design
|
||||
|
||||
### 4.1 Feature Workstreams (max ~20 features/group)
|
||||
|
||||
- **Group 1: Store creation + encryption bootstrap + metadata + block recovery (17 features)**
|
||||
IDs: `955, 956, 957, 958, 960, 961, 962, 963, 964, 965, 966, 967, 968, 969, 970, 972, 974`
|
||||
- **Group 2: Lost-data/state rebuild + tracking/utilities (16 features)**
|
||||
IDs: `975, 976, 978, 979, 980, 989, 990, 998, 1057, 1152, 1153, 1154, 1158, 1163, 1164, 1165`
|
||||
- **Group 3: Init semaphore + consumer decode + atomic file writes (6 features)**
|
||||
IDs: `1200, 1235, 1239, 1248, 1261, 1262`
|
||||
|
||||
### 4.2 File Boundaries
|
||||
|
||||
- Primary production files:
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStore.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/MessageBlock.cs`
|
||||
- `dotnet/src/ZB.MOM.NatsNet.Server/JetStream/FileStoreTypes.cs`
|
||||
- Primary test files:
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/JetStreamFileStoreTests.Impltests.cs`
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests1.Impltests.cs`
|
||||
- `dotnet/tests/ZB.MOM.NatsNet.Server.Tests/ImplBacklog/ConcurrencyTests2.Impltests.cs`
|
||||
|
||||
### 4.3 Data/Control Flow to Port
|
||||
|
||||
- `newFileStoreWithCreated` bootstrap flow:
|
||||
- validate config and storage mode
|
||||
- apply dynamic block sizing defaults
|
||||
- verify/create directories and writable checks
|
||||
- initialize hash/checksum state and encryption metadata
|
||||
- recover persisted stream/message state
|
||||
- enforce limits and age/timer setup
|
||||
- start background flush/state loops
|
||||
- Supporting flows:
|
||||
- message block buffer pooling and recycling
|
||||
- block encryption key generation/recovery
|
||||
- lost data tracking and rebuild state accounting
|
||||
- consumer state binary header/decode path
|
||||
- atomic write helpers with optional sync and directory sync
|
||||
|
||||
### 4.4 Error Handling Design
|
||||
|
||||
- Preserve failure semantics from Go intent:
|
||||
- reject invalid storage config early
|
||||
- propagate IO/crypto errors with contextual messages
|
||||
- fail closed for corrupt headers / decode failures
|
||||
- avoid silent fallback to placeholder behavior
|
||||
- Ensure lock discipline for shared mutable state and block-level operations.
|
||||
|
||||
### 4.5 Test Design
|
||||
|
||||
- Replace placeholder backlog tests with behavior-focused tests derived from Go cases.
|
||||
- Keep benchmark-mapped tests as either:
|
||||
- converted deterministic assertions where appropriate, or
|
||||
- deferred with explicit infrastructure/perf rationale.
|
||||
- Include focused regression tests for encryption key lifecycle, metadata integrity, state recovery, and consumer state decoding.
|
||||
|
||||
## 5. Risks and Mitigations
|
||||
|
||||
- **Risk:** large `filestore.go` surface can lead to stub creep.
|
||||
**Mitigation:** mandatory stub scans for both src and tests, hard stop on matches.
|
||||
- **Risk:** hidden behavior coupling across utility methods.
|
||||
**Mitigation:** group-based checkpoints with full build + targeted suite each cycle.
|
||||
- **Risk:** status drift (marking verified without evidence).
|
||||
**Mitigation:** evidence logs required for every feature/test update, max 15 IDs per update.
|
||||
|
||||
## 6. Design Outcome
|
||||
|
||||
This design uses a strict vertical execution model with heavy verification gates. It is structured to be executed by a plan that enforces per-feature validation loops, anti-stub guardrails, and evidence-based PortTracker updates.
|
||||
Reference in New Issue
Block a user