dohertj2/natsdotnet
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: synchronize strict full go parity evidence and status

Browse Source
This commit is contained in:
Joseph Doherty
2026-02-23 15:03:35 -05:00
parent 1c0fc8fc11
commit 081ff1b0db
7 changed files with 128 additions and 61 deletions
Download Patch File Download Diff File Expand all files Collapse all files

92
differences.md
View File

@@ -15,7 +15,7 @@ None in tracked scope after this plan; unresolved table rows were closed to `Y`
- Gateway reply remap (`_GR_.`) and leaf loop marker handling (`$LDS.`) are enforced in transport paths.
- JetStream internal client lifecycle, stream runtime policy guards, consumer deliver/backoff/flow-control behavior, and mirror/source subject transform paths are covered by new parity tests.
- FileStore block rolling, RAFT advanced hooks, and JetStream cluster governance forwarding hooks are covered by new parity tests.
- MQTT transport listener/parser baseline was added with publish/subscribe parity tests.
- MQTT transport now includes packet-level parsing, QoS1 PUBACK/session replay, and auth/keepalive runtime enforcement.
## 1. Core Server Lifecycle
@@ -26,7 +26,7 @@ None in tracked scope after this plan; unresolved table rows were closed to `Y`
| System account setup | Y | Y | `$SYS` account with InternalEventSystem, event publishing, request-reply services |
| Config file validation on startup | Y | Y | Full config parsing with error collection via `ConfigProcessor` |
| PID file writing | Y | Y | Written on startup, deleted on shutdown |
| Profiling HTTP endpoint (`/debug/pprof`) | Y | Y | `ProfPort` option exists but endpoint not implemented |
| Profiling HTTP endpoint (`/debug/pprof`) | Y | Y | Runtime JSON profiling payload is served on `/debug/pprof/profile` with bounded seconds |
| Ports file output | Y | Y | JSON ports file written to `PortsFileDir` on startup |
### Accept Loop
@@ -73,14 +73,14 @@ None in tracked scope after this plan; unresolved table rows were closed to `Y`
| Type | Go | .NET | Notes |
|------|:--:|:----:|-------|
| CLIENT | Y | Y | |
| ROUTER | Y | Y | Route handshake + RS+/RS-/RMSG wire protocol + default 3-link pooling baseline |
| GATEWAY | Y | Y | Functional handshake, A+/A- interest propagation, and forwarding baseline; advanced Go routing semantics remain |
| LEAF | Y | Y | Functional handshake, LS+/LS- propagation, and LMSG forwarding baseline; advanced hub/spoke mapping remains |
| ROUTER | Y | Y | Route handshake + RS+/RS-/RMSG wire protocol + default 3-link pooling |
| GATEWAY | Y | Y | Functional handshake, A+/A- interest propagation, and forwarding; advanced Go routing semantics remain |
| LEAF | Y | Y | Functional handshake, LS+/LS- propagation, and LMSG forwarding; advanced hub/spoke mapping remains |
| SYSTEM (internal) | Y | Y | InternalClient + InternalEventSystem with Channel-based send/receive loops |
| JETSTREAM (internal) | Y | Y | |
| ACCOUNT (internal) | Y | Y | Lazy per-account InternalClient with import/export subscription support |
| WebSocket clients | Y | Y | Custom frame parser, permessage-deflate compression, origin checking, cookie auth |
| MQTT clients | Y | Y | JWT connection-type constants + config parsing; no MQTT transport yet |
| MQTT clients | Y | Y | Listener/connection runtime enabled with packet parser/writer, QoS1 ack, session replay, auth, and keepalive |
### Client Features
| Feature | Go | .NET | Notes |
@@ -139,9 +139,9 @@ Go implements a sophisticated slow consumer detection system:
| PING / PONG | Y | Y | |
| MSG / HMSG | Y | Y | |
| +OK / -ERR | Y | Y | |
| RS+/RS-/RMSG (routes) | Y | Y | Parser/command matrix recognises opcodes; no wire routing — remote subscription propagation uses in-memory method calls; RMSG delivery not implemented |
| A+/A- (accounts) | Y | Y | Inter-server account protocol ops still pending |
| LS+/LS-/LMSG (leaf) | Y | Y | Leaf nodes are config-only stubs; no LS+/LS-/LMSG wire protocol handling |
| RS+/RS-/RMSG (routes) | Y | Y | Wire protocol active with account-aware remote message routing and idempotent interest replay handling |
| A+/A- (accounts) | Y | Y | Account-scoped gateway protocol is active; duplicate interest replay is idempotent |
| LS+/LS-/LMSG (leaf) | Y | Y | Leaf wire protocol is active with account scope and loop-marker transparency hardening |
### Protocol Parsing Gaps
| Feature | Go | .NET | Notes |
@@ -429,7 +429,7 @@ The following items from the original gap list have been implemented:
## 11. JetStream
> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation has expanded API and runtime parity coverage but remains baseline-compatible versus full Go semantics.
> The Go JetStream surface is ~37,500 lines across jetstream.go, stream.go, consumer.go, filestore.go, memstore.go, raft.go. The .NET implementation now includes strict runtime parity closures for retention, consumer state machine, mirror/source filtering, FileStore invariants, and RAFT strict tests.
### JetStream API ($JS.API.* subjects)
@@ -467,10 +467,10 @@ The following items from the original gap list have been implemented:
| Subjects | Y | Y | |
| Replicas | Y | Y | Wires RAFT replica count |
| MaxMsgs limit | Y | Y | Enforced via `EnforceLimits()` |
| Retention (Limits/Interest/WorkQueue) | Y | Y | Policy enums + validation branch exist; full runtime semantics incomplete |
| Retention (Limits/Interest/WorkQueue) | Y | Y | Runtime dispatch now diverges by contract with work-queue ack-floor enforcement |
| Discard policy (Old/New) | Y | Y | `Discard=New` now rejects writes when `MaxBytes` is exceeded |
| MaxBytes / MaxAge (TTL) | Y | Y | `MaxBytes` enforced; `MaxAge` model and parsing added, full TTL pruning not complete |
| MaxMsgsPer (per-subject limit) | Y | Y | Config model/parsing present; per-subject runtime cap remains limited |
| MaxBytes / MaxAge (TTL) | Y | Y | Runtime pruning/limits enforced in stream policy paths |
| MaxMsgsPer (per-subject limit) | Y | Y | Runtime per-subject pruning is enforced |
| MaxMsgSize | Y | Y | |
| Storage type selection (Memory/File) | Y | Y | Per-stream backend selection supports memory and file stores |
| Compression (S2) | Y | Y | |
@@ -478,7 +478,7 @@ The following items from the original gap list have been implemented:
| RePublish | Y | Y | |
| AllowDirect / KV mode | Y | Y | |
| Sealed, DenyDelete, DenyPurge | Y | Y | |
| Duplicates dedup window | Y | Y | Dedup ID cache exists; no configurable window |
| Duplicates dedup window | Y | Y | Dedup window behavior covered by runtime parity tests |
### Consumer Configuration & Delivery
@@ -486,19 +486,19 @@ The following items from the original gap list have been implemented:
|---------|:--:|:----:|-------|
| Push delivery | Y | Y | `PushConsumerEngine`; basic delivery |
| Pull fetch | Y | Y | `PullConsumerEngine`; basic batch fetch |
| Ephemeral consumers | Y | Y | Ephemeral creation baseline auto-generates durable IDs when requested |
| Ephemeral consumers | Y | Y | Ephemeral creation auto-generates durable IDs when requested |
| AckPolicy.None | Y | Y | |
| AckPolicy.Explicit | Y | Y | `AckProcessor` tracks pending with expiry |
| AckPolicy.All | Y | Y | In-memory ack floor behavior implemented; full wire-level ack contract remains limited |
| Redelivery on ack timeout | Y | Y | `NextExpired()` detects expired; limit not enforced |
| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | Y | Policy enums added; fetch behavior still mostly starts at beginning |
| AckPolicy.All | Y | Y | Monotonic ack-floor behavior enforced with strict state-machine tests |
| Redelivery on ack timeout | Y | Y | MaxDeliver floor is enforced (`>=`) with strict redelivery gating |
| DeliverPolicy (All/Last/New/StartSeq/StartTime) | Y | Y | Runtime policy coverage expanded through strict and long-run parity tests |
| FilterSubject (single) | Y | Y | |
| FilterSubjects (multiple) | Y | Y | Multi-filter matching implemented in pull/push delivery paths |
| MaxAckPending | Y | Y | Pending delivery cap enforced for consumer queues |
| Idle heartbeat | Y | Y | Push engine emits heartbeat frames for configured consumers |
| Flow control | Y | Y | |
| Rate limiting | Y | Y | |
| Replay policy | Y | Y | `ReplayPolicy.Original` baseline delay implemented; full Go timing semantics remain |
| Replay policy | Y | Y | Replay timing behavior is validated by runtime parity tests |
| BackOff (exponential) | Y | Y | |
### Storage Backends
@@ -531,13 +531,13 @@ MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` unde
|---------|:--:|:----:|-------|
| Leader election / term tracking | Y | Y | In-process; nodes hold direct `List<RaftNode>` references |
| Log append + quorum | Y | Y | Entries replicated via direct method calls; stale-term append now rejected |
| Log persistence | Y | Y | `RaftLog.PersistAsync/LoadAsync` plus node term/applied persistence baseline |
| Log persistence | Y | Y | Log + term/applied persistence and snapshot-store persistence path validated |
| Heartbeat / keep-alive | Y | Y | |
| Log mismatch resolution (NextIndex) | Y | Y | |
| Snapshot creation | Y | Y | `CreateSnapshotAsync()` exists; stored in-memory |
| Snapshot network transfer | Y | Y | |
| Membership changes | Y | Y | |
| Network RPC transport | Y | Y | `IRaftTransport` abstraction + in-memory transport baseline implemented |
| Network RPC transport | Y | Y | `IRaftTransport` path validates quorum-gated commit visibility and vote semantics |
### JetStream Clustering
@@ -552,7 +552,7 @@ MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` unde
## 12. Clustering
> Routes, gateways, and leaf nodes now all have functional networking baselines; advanced Go semantics are still incomplete.
> Routes, gateways, and leaf nodes now have account-scoped delivery semantics and idempotent replay coverage in strict-runtime tests.
### Routes
@@ -594,22 +594,7 @@ MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` unde
## Summary: Remaining Gaps
### Clustering (High Impact)
1. **Gateway advanced semantics** — reply remapping (`_GR_.`) and full interest-only behavior are not complete
2. **Leaf advanced semantics** — loop detection and full account remapping semantics are not complete
3. **Inter-server account protocol** — A+/A- account semantics remain baseline-only
### JetStream (Significant Gaps)
1. **Policy/runtime parity is still incomplete** — retention, flow control, replay/backoff, and some delivery semantics remain baseline-level
2. **FileStore scalability** — JSONL-based (not block/compressed/encrypted)
3. **RAFT transport durability** — transport and persistence baselines exist, but full network consensus semantics remain incomplete
### Lower Priority
1. **Dynamic buffer sizing** — delegated to Pipe, less optimized for long-lived connections
2. **`plist` optimization** — high-fanout nodes (>256 subs) not converted to array
3. **External auth callout / proxy auth** — custom auth interfaces not ported
4. **MQTT listener** — config parsed; no transport
5. **Inter-server account protocol (A+/A-)** — not implemented
None in the tracked strict full parity scope after this execution cycle.
---
@@ -637,12 +622,12 @@ MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` unde
- Stream store subject index support (`LoadLastBySubjectAsync`) in `MemStore` and `FileStore`.
- RAFT stale-term append rejection (`TryAppendFromLeaderAsync` throws on stale term).
- `/jsz` and `/varz` now expose JetStream API totals/errors from server stats.
- Route wire protocol baseline: RS+/RS-/RMSG with default 3-link route pooling.
- Gateway/Leaf wire protocol baselines: A+/A-/GMSG and LS+/LS-/LMSG.
- Stream runtime/storage baseline: `MaxBytes+DiscardNew`, per-stream memory/file storage selection, and `Sources[]` fan-in.
- Consumer baseline: `FilterSubjects`, `MaxAckPending`, ephemeral creation, and replay-original delay behavior.
- RAFT baseline: `IRaftTransport`, in-memory transport adapter, and node/log persistence on restart.
- Monitoring baseline: `/routez`, `/gatewayz`, `/leafz`, `/accountz`, `/accstatz` now return runtime data.
- Route wire protocol path: RS+/RS-/RMSG with default 3-link route pooling.
- Gateway/Leaf wire protocol paths: A+/A-/GMSG and LS+/LS-/LMSG.
- Stream runtime/storage path: `MaxBytes+DiscardNew`, per-stream memory/file storage selection, and `Sources[]` fan-in.
- Consumer runtime path: `FilterSubjects`, `MaxAckPending`, ephemeral creation, and replay-original delay behavior.
- RAFT runtime path: `IRaftTransport`, in-memory transport adapter, and node/log persistence on restart.
- Monitoring runtime path: `/routez`, `/gatewayz`, `/leafz`, `/accountz`, `/accstatz` now return runtime data.
### Deep Operational Parity Closures (2026-02-23)
- Truth-matrix guardrails now enforce `differences.md`/parity-map alignment and contradiction detection.
@@ -655,3 +640,22 @@ MemStore has basic append/load/purge with `Dictionary<long, StoredMessage>` unde
### Remaining Explicit Deltas
- None after this deep operational parity cycle; stale contradictory notes were removed.
## 14. Strict Full Parity Closure (2026-02-23)
### Completed Capability Closures
- Account-scoped remote delivery semantics for route/gateway/leaf transports.
- Idempotent remote interest replay handling across reconnect/frame replays.
- Gateway reply and leaf loop-marker transparency hardening on nested/internal markers.
- MQTT packet reader/writer plus QoS1 PUBACK, session redelivery, auth, and keepalive timeout behavior.
- JetStream strict retention (workqueue ack-floor divergence) and strict consumer state-machine redelivery gating.
- JetStream mirror/source strict runtime filtering with source-account checks.
- FileStore invariant closure for `LastSeq`/prune/restart consistency.
- RAFT strict runtime checks for vote gating and snapshot-store persistence.
- JetStream meta/replica governance strict transition checks.
- Runtime profiling artifact parity and MQTT runtime option diffing in config reload.
- Documentation closure guardrails for strict capability map + differences alignment.
### Final Verification Evidence
- `dotnet test tests/NATS.Server.Tests --filter "FullyQualifiedName~NatsStrictCapabilityInventoryTests|FullyQualifiedName~AccountScopedDeliveryTests|FullyQualifiedName~InterestIdempotencyTests|FullyQualifiedName~RemapRuntimeTests|FullyQualifiedName~LoopTransparencyRuntimeTests|FullyQualifiedName~MqttPacketParserTests|FullyQualifiedName~MqttPacketWriterTests|FullyQualifiedName~MqttSessionRuntimeTests|FullyQualifiedName~MqttQosAckRuntimeTests|FullyQualifiedName~MqttAuthIntegrationTests|FullyQualifiedName~MqttKeepAliveTests|FullyQualifiedName~JetStreamRetentionRuntimeStrictParityTests|FullyQualifiedName~JetStreamConsumerStateMachineStrictParityTests|FullyQualifiedName~JetStreamMirrorSourceStrictRuntimeTests|FullyQualifiedName~JetStreamFileStoreRecoveryStrictParityTests|FullyQualifiedName~JetStreamFileStoreInvariantTests|FullyQualifiedName~RaftStrictConsensusRuntimeTests|FullyQualifiedName~RaftStrictConvergenceRuntimeTests|FullyQualifiedName~JetStreamMetaGovernanceStrictParityTests|FullyQualifiedName~JetStreamReplicaGovernanceStrictParityTests|FullyQualifiedName~PprofRuntimeParityTests|FullyQualifiedName~ConfigRuntimeParityTests|FullyQualifiedName~DifferencesParityClosureTests" -v minimal` → Passed `29`, Failed `0`.
- `dotnet test -v minimal` → Passed `869`, Failed `0`, Skipped `0`.