natsnet/docs/plans/2026-02-27-batch-21-events-msgtrace-design.md

Batch 21 Events + MsgTrace Design

Date: 2026-02-27
Batch: 21 (Events + MsgTrace)
Scope: Design only. No implementation in this document.

Problem

Batch 21 contains high-fanout eventing and tracing behavior used by system account messaging, server stats, admin requests, and distributed message tracing.

Current tracker scope from batch show 21:

• Features: 118 (all deferred)
• Tests: 9 (all deferred)
• Dependencies: batches 18, 19
• Go sources: server/events.go, server/msgtrace.go

Context Findings

• Existing .NET code already has many event and trace DTO/types in:
  • dotnet/src/ZB.MOM.NatsNet.Server/Events/EventTypes.cs
  • dotnet/src/ZB.MOM.NatsNet.Server/MessageTrace/MsgTraceTypes.cs
• Runtime behavior methods mapped in Batch 21 are largely missing from NatsServer and ClientConnection.
• Existing backlog tests for Events/MsgTrace include placeholder-style assertions and must be replaced with behavior-valid tests for mapped Batch 21 IDs.
• test show mapping confirms target class/methods for all 9 tests:
  • Events: 6 methods in EventsHandlerTests
  • MsgTrace: 2 methods in MessageTracerTests
  • Concurrency: 1 method in ConcurrencyTests1

Assumptions

• Batch 21 work begins only after dependencies (batches 18 and 19) are complete enough to compile and run related tests.
• We preserve existing project structure and naming patterns (NatsServer.*.cs, ClientConnection.*.cs, ImplBacklog/*.Impltests.cs).
• No new integration infrastructure is introduced in this batch; infra-blocked items remain deferred with explicit reasons.

Approaches

Approach A: Monolithic implementation in existing large files

Implement all methods directly in NatsServer.cs, ClientConnection.cs, ClientTypes.cs, and NatsServerTypes.cs.

Trade-offs:

• Pros: Minimal file creation.
• Cons: Very high merge conflict risk, poor reviewability, difficult verification per feature group.

Approach B (Recommended): Partial-file segmentation by runtime domain

Add focused partial/runtime files for events and msgtrace behavior while leaving DTO/type files intact.

Trade-offs:

• Pros: Enables clear feature-group boundaries, easier per-group build/test loops, lower risk of accidental regressions.
• Cons: Requires a few new files and up-front structure decisions.

Approach C: Test-only first, then backfill features

Start by rewriting the 9 tests to force implementation behavior.

Trade-offs:

• Pros: Fast feedback for mapped tests.
• Cons: Batch has 118 features and only 9 mapped tests; test-first alone leaves large behavior surface unvalidated.

Recommended Design

Use Approach B.

Code organization

Implement runtime behavior in small, targeted files:

• dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.Events.cs (core eventing/send loops)
• dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.Events.System.cs (system subscriptions/requests)
• dotnet/src/ZB.MOM.NatsNet.Server/NatsServer.Events.Admin.cs (reload/kick/ldm/debug/OCSP event paths)
• dotnet/src/ZB.MOM.NatsNet.Server/ClientConnection.MsgTrace.cs (client-side trace enable/init helpers)
• dotnet/src/ZB.MOM.NatsNet.Server/MessageTrace/MsgTraceRuntime.cs (runtime trace mutation/send behavior)
• plus targeted edits in:
  • dotnet/src/ZB.MOM.NatsNet.Server/NatsServerTypes.cs (ServerInfo capability helpers)
  • dotnet/src/ZB.MOM.NatsNet.Server/ClientTypes.cs (ClientInfo helper projections)
  • dotnet/src/ZB.MOM.NatsNet.Server/Accounts/Account.cs (Account.statz)

Behavior domains

• Domain 1: Capability and metadata helpers (ServerInfo, ClientInfo, hashing helpers).
• Domain 2: Internal system message send/receive loops and event state lifecycle.
• Domain 3: Remote server/account tracking and statsz/advisory publication.
• Domain 4: System subscription wiring and request handlers (connsz/statsz/idz/nsubs/reload/kick/ldm).
• Domain 5: OCSP advisory events and misc utility wrappers.
• Domain 6: Message trace runtime (trace enablement, header extraction, event aggregation, publish path).

Test design

Replace mapped placeholder tests with behavior checks that assert:

• System subscription registration and unsubscribe behavior.
• Connection update timer/sweep behavior under local/remote account changes.
• Remote latency update validation and bad payload handling.
• MsgTrace connection-name normalization and trace-header parsing correctness.
• No-race JetStream compact scenario behavior for mapped test ID 2412.

Execution model

Port features in 6 groups (<=20 IDs each), then tests in 2 waves, with strict per-feature verification and anti-stub gates.

Risks and Mitigations

• Risk: Placeholder tests can pass while behavior is wrong.
  • Mitigation: Mandatory anti-stub checks and per-test evidence before status updates.
• Risk: Large eventing surface can regress unrelated server behavior.
  • Mitigation: Build gate after each feature group + full unit-test checkpoint between tasks.
• Risk: Some tests require runtime topology not available in unit test harness.
  • Mitigation: keep deferred with explicit blocker reason; do not stub.

Success Criteria

• All 118 Batch 21 feature IDs moved through stub -> complete -> verified only with build + test evidence.
• All 9 Batch 21 test IDs either verified with real assertions or deferred with explicit blocker reason.
• No new stubs (NotImplementedException, empty bodies, TODO placeholders) in touched feature/test files.
• Batch 21 can be completed with batch complete 21 once all IDs satisfy status requirements.