c05d93618e
Generated design docs and implementation plans via Codex for: - Batch 23: Routes - Batch 24: Leaf Nodes - Batch 25: Gateways - Batch 26: WebSocket - Batch 27: JetStream Core - Batch 28: JetStream API - Batch 29: JetStream Batching - Batch 30: Raft Part 1 All plans include mandatory verification protocol and anti-stub guardrails. Updated batches.md with file paths and planned status.
142 lines
6.3 KiB
Markdown
142 lines
6.3 KiB
Markdown
# Batch 28 JetStream API Design
|
|
|
|
**Date:** 2026-02-27
|
|
**Batch:** 28 (`JetStream API`)
|
|
**Scope:** 55 features + 2 unit tests from `server/jetstream_api.go` / `server/jetstream_test.go`
|
|
**Dependencies:** batches `5` (`JetStream Errors`), `27` (`JetStream Core`)
|
|
|
|
## Problem
|
|
|
|
Batch 28 ports the JetStream API request/response surface, including subject dispatch, request decoding, account usage tracking, stream/consumer management handlers, delayed API responses, and API audit publication. This is the behavioral bridge between JetStream core state (Batch 27) and external API subjects (`$JS.API.*`).
|
|
|
|
## Context Findings
|
|
|
|
### Required Command Findings
|
|
|
|
- `batch show 28 --db porting.db`
|
|
- Batch 28 is `pending`
|
|
- 55 features and 2 tests are all `deferred`
|
|
- Go source scope is `server/jetstream_api.go`
|
|
- `batch list --db porting.db`
|
|
- Batch 28 depends on batches 5 and 27
|
|
- Batch 36 (`Stream Lifecycle`) depends on batch 28, so this is an unblocker batch
|
|
- `report summary --db porting.db`
|
|
- Features verified: 1271 / 3673
|
|
- Unit tests verified: 430 / 3257
|
|
- Large deferred backlog confirms need for strict anti-stub controls
|
|
- `batch ready --db porting.db`
|
|
- Ready batches currently: 0, 1, 5, 8
|
|
- Batch 28 is not startable until batch 27 is complete (and batch 5 remains satisfied)
|
|
|
|
### Batch 28 Mapping Shape
|
|
|
|
Feature distribution by mapped .NET class:
|
|
|
|
- `NatsServer`: 43 features (~4006 Go LOC)
|
|
- `JetStreamApi`: 6 features (~91 Go LOC)
|
|
- `Account`: 4 features (~56 Go LOC)
|
|
- `JsAccount`: 1 feature (~23 Go LOC)
|
|
- `JetStreamEngine`: 1 feature (~86 Go LOC)
|
|
|
|
Complexity hot spots (Go LOC):
|
|
|
|
- `#1501 Server.jsConsumerCreateRequest` (224)
|
|
- `#1477 Server.jsStreamInfoRequest` (223)
|
|
- `#1498 Server.processStreamRestore` (207)
|
|
- `#1504 Server.jsConsumerInfoRequest` (204)
|
|
- `#1484 Server.jsLeaderServerStreamMoveRequest` (162)
|
|
|
|
Test mapping:
|
|
|
|
- `#1716 TestIsJSONObjectOrArray` -> `JetStreamEngineTests.IsJSONObjectOrArray_ShouldSucceed` (depends on feature `#1488`)
|
|
- `#1719 TestJetStreamDelayedAPIResponses` -> `JetStreamEngineTests.JetStreamDelayedAPIResponses_ShouldSucceed` (includes dependency on feature `#1400` from batch 27)
|
|
|
|
## Approaches
|
|
|
|
### Approach A: Single-file monolithic port in one `NatsServer` partial
|
|
|
|
- Put almost all Batch 28 methods into one large `NatsServer` file.
|
|
- Pros: fast to scaffold.
|
|
- Cons: high review/debug risk, weak separation for helper classes (`JetStreamApi`, `JetStreamEngine`), poor maintainability.
|
|
|
|
### Approach B (Recommended): Layered API pipeline with grouped handler waves
|
|
|
|
- Keep class boundaries aligned with mapping (`JetStreamApi`, `JetStreamEngine`, `NatsServer`, `Account`, `JsAccount`).
|
|
- Implement by feature groups (~20 max each) using API lifecycle phases: dispatch/helpers -> stream leader/metadata handlers -> message/consumer/snapshot handlers.
|
|
- Pros: strongest audit alignment, lower regression risk, clean status-evidence workflow.
|
|
- Cons: more upfront planning discipline.
|
|
|
|
### Approach C: Signature-first sweep then fill behavior later
|
|
|
|
- Add method signatures quickly, defer real logic.
|
|
- Pros: apparent short-term progress.
|
|
- Cons: violates anti-stub policy and creates status inflation risk.
|
|
|
|
Decision: **Approach B**.
|
|
|
|
## Proposed Design
|
|
|
|
### 1. API Pipeline Architecture
|
|
|
|
Implement a deterministic request pipeline:
|
|
|
|
1. Subject mapping + dispatch (`GenerateJSMappingTable`, `ApiDispatch`, routed request processing)
|
|
2. Request metadata extraction (`GetRequestInfo`, `UnmarshalRequest`, subject parsing helpers)
|
|
3. Account checks/tracking (`CheckJetStream`, `TrackAPI`, `TrackAPIErr`, reservation/limits checks)
|
|
4. Handler execution (stream, consumer, leader/control operations)
|
|
5. Response publishing (immediate or delayed) and advisory audit emission
|
|
|
|
### 2. Component Boundaries
|
|
|
|
- `JetStreamApi` class: pure/static helpers (`GenerateJSMappingTable`, subject parsers, JSON-shape helpers, delayed queue helper)
|
|
- `JetStreamEngine` class: API entry on engine (`ApiDispatch`)
|
|
- `NatsServer` partials: handler implementations and response transport
|
|
- `Account` / `JsAccount`: API counters, enablement checks, tiered reservation and non-clustered limits checks
|
|
|
|
### 3. Handler Grouping Strategy
|
|
|
|
Use 3 feature groups (max ~20):
|
|
|
|
1. **Group A (20):** IDs `1452,1454-1472`
|
|
Dispatch, delayed responder core, request parsing, account tracking, account info, naming helpers.
|
|
2. **Group B (18):** IDs `1473-1490`
|
|
Stream CRUD/list/info plus leader step-down, peer/remove/move operations and request-shape validators.
|
|
3. **Group C (17):** IDs `1491-1507`
|
|
Message delete/get, consumer unpin/pause/create/list/info/delete, purge/restore/snapshot, API advisory.
|
|
|
|
### 4. Testing Strategy
|
|
|
|
- Tracked tests first-class:
|
|
- `#1716` (JSON object/array detector)
|
|
- `#1719` (delayed API response sequencing/cancellation/reload behavior)
|
|
- Add focused helper tests where needed for deterministic per-feature verification, but only promote tracked IDs through PortTracker workflow.
|
|
- Keep infra-heavy scenarios deferred with explicit reasons when prerequisites from batch 27/runtime harness are missing.
|
|
|
|
### 5. Error Handling and Deferred Policy
|
|
|
|
- Use explicit API error responses (`JsApiError`-based) for request validation and unavailable-state paths.
|
|
- Any blocked item is explicitly marked `deferred` with reason; no placeholder production logic, no fake-pass tests.
|
|
|
|
## Risks and Mitigations
|
|
|
|
- **Risk:** Batch 28 execution before dependencies complete.
|
|
- **Mitigation:** hard preflight gate (`batch show 5`, `batch show 27`, then `batch start 28`).
|
|
- **Risk:** High-LOC handlers become partially ported.
|
|
- **Mitigation:** per-feature loop, stub scan, build/test gates before status promotion.
|
|
- **Risk:** Tracker status drift from actual code quality.
|
|
- **Mitigation:** max-15 ID updates, evidence checklist per update chunk, checkpoint between tasks.
|
|
|
|
## Success Criteria
|
|
|
|
- Batch 28 features are `verified` or explicitly `deferred` with concrete blocker reasons.
|
|
- Batch 28 tests are `verified` or explicitly `deferred` with concrete blocker reasons.
|
|
- No new stub patterns in touched source/test files.
|
|
- Build and required test gates pass at each checkpoint.
|
|
- Status updates are chunked and evidence-backed.
|
|
|
|
## Non-Goals
|
|
|
|
- Executing Batch 28 implementation in this document.
|
|
- Reworking unrelated JetStream/Raft architecture outside mapped Batch 28 behavior.
|
|
- Changing porting tracker mappings unless audit requires explicit remap.
|