Commit Graph

17 Commits

Author SHA1 Message Date
Joseph Doherty
0f8f34afaa feat: add entry application pipeline for meta and stream RAFT groups (Gap 2.7)
Extend ApplyEntry in JetStreamMetaGroup with PeerAdd/PeerRemove dispatch
to the existing Task 12 peer management methods. Add StreamMsgOp (Store,
Remove, Purge) and ConsumerOp (Ack, Nak, Deliver, Term, Progress) enums
plus ApplyStreamMsgOp and ApplyConsumerEntry methods to StreamReplicaGroup.
Extend ApplyCommittedEntriesAsync to parse smsg:/centry: command prefixes
and route to the new apply methods. Add MetaEntryType.PeerAdd/PeerRemove
enum values. 35 new tests in EntryApplicationTests.cs, all passing.
2026-02-25 08:38:21 -05:00
Joseph Doherty
7e4a23a0b7 feat(cluster): implement leadership transition with inflight cleanup
Add ProcessLeaderChange(bool) method and OnLeaderChange event to JetStreamMetaGroup.
Refactor StepDown() to delegate inflight clearing through ProcessLeaderChange,
enabling subscribers to react to leadership transitions.
Go reference: jetstream_cluster.go:7001-7074 processLeaderChange.
2026-02-25 02:08:37 -05:00
Joseph Doherty
63d4e43178 feat(cluster): add structured inflight proposal tracking with ops counting
Replace the simple string-keyed inflight dictionaries with account-scoped
ConcurrentDictionary<string, Dictionary<string, InflightInfo>> structures.
Adds InflightInfo record with OpsCount for duplicate proposal tracking,
TrackInflight/RemoveInflight/IsInflight methods for streams and consumers,
and ClearAllInflight(). Updates existing Propose* methods to use $G account.
Go reference: jetstream_cluster.go:1193-1278.
2026-02-25 02:05:53 -05:00
Joseph Doherty
1a9b6a9175 feat(cluster): add stream/consumer assignment processing with validation
Add 5 validated Process* methods to JetStreamMetaGroup for stream and
consumer assignment processing: ProcessStreamAssignment, ProcessUpdateStreamAssignment,
ProcessStreamRemoval, ProcessConsumerAssignment, and ProcessConsumerRemoval.
Each returns a bool indicating success, with validation guards matching
Go reference jetstream_cluster.go:4541-5925. Includes 12 new unit tests.
2026-02-25 01:57:43 -05:00
Joseph Doherty
7f9ee493b6 feat(cluster): add JetStreamClusterMonitor for meta RAFT entry processing
Implements the background loop (Go: monitorCluster) that reads RaftLogEntry
items from a channel and dispatches assignStream, removeStream, assignConsumer,
removeConsumer, and snapshot operations to JetStreamMetaGroup.

- Add five public mutation helpers to JetStreamMetaGroup (AddStreamAssignment,
  RemoveStreamAssignment, AddConsumerAssignment, RemoveConsumerAssignment,
  ReplaceAllAssignments) used by the monitor path
- JetStreamClusterMonitor: async loop over ChannelReader<RaftLogEntry>, JSON
  dispatch, ILogger injection with NullLogger default, malformed entries logged
  and skipped rather than aborting the loop
- WaitForProcessedAsync uses Monitor.PulseAll for race-free test synchronisation
  with no Task.Delay — satisfies slopwatch SW003/SW004 rules
- 10 new targeted tests all pass; 1101 cluster regression tests unchanged
2026-02-25 01:54:04 -05:00
Joseph Doherty
9fdc931ff5 feat(cluster): add MetaSnapshotCodec with S2 compression and versioned format
Implements binary codec for meta-group snapshots: 2-byte little-endian version
header followed by S2-compressed JSON of the stream assignment map. Adds
[JsonObjectCreationHandling(Populate)] to StreamAssignment.Consumers so the
getter-only dictionary is populated in-place during deserialization. 8 tests
covering round-trip, compression ratio, field fidelity, multi-consumer restore,
version rejection, and truncation guard.

Go reference: jetstream_cluster.go:2075-2145 (encodeMetaSnapshot/decodeMetaSnapshot)
2026-02-25 01:46:32 -05:00
Joseph Doherty
d286349262 fix: re-enable leader check in API router and fix stepdown simulation
Re-enabled the leader check in JetStreamApiRouter.Route() that was
commented out during B8 agent work. Added BecomeLeader() method to
JetStreamMetaGroup for single-process test fixtures to simulate
winning the post-stepdown election. MetaControllerFixture now
auto-calls BecomeLeader() after a successful meta leader stepdown.
2026-02-24 18:02:54 -05:00
Joseph Doherty
1257a5ca19 feat(cluster): rewrite meta-group, enhance stream RAFT, add Go parity tests (B7+B8+B9+B10)
- JetStreamMetaGroup: validated proposals, inflight tracking, consumer counting, ApplyEntry dispatch
- StreamReplicaGroup: ProposeMessageAsync, LeaderChanged event, message/sequence tracking, GetStatus
- PlacementEngine tests: cluster affinity, tag filtering, storage ordering (16 tests)
- Assignment serialization tests: quorum calc, has-quorum, property defaults (16 tests)
- MetaGroup proposal tests: stream/consumer CRUD, leader validation, inflight (30 tests)
- StreamRaftGroup tests: message proposals, step-down events, status (10 tests)
- RAFT Go parity tests + JetStream cluster Go parity tests (partial B11 pre-work)
2026-02-24 17:23:57 -05:00
Joseph Doherty
a323715495 feat(cluster): add stream/consumer assignments, placement engine, and meta proposal workflow (B7+B8+B9)
- RaftGroup, StreamAssignment, ConsumerAssignment types matching Go structs
  (jetstream_cluster.go:154-266)
- PlacementEngine.SelectPeerGroup: topology-aware peer selection with cluster
  affinity, tag filtering, exclude tags, and storage-weighted sorting
  (Go ref: selectPeerGroup at line 7212)
- JetStreamMetaGroup: backward-compatible rewrite with full assignment tracking,
  consumer proposal workflow, and delete operations
- 41 new tests in ClusterAssignmentAndPlacementTests
2026-02-24 17:13:28 -05:00
Joseph Doherty
a41d0f453c feat(cluster): add placement engine with topology-aware peer selection (B9-prep) 2026-02-24 17:09:32 -05:00
Joseph Doherty
824e0b3607 feat(raft): add membership proposals, snapshot checkpoints, and log compaction (B4+B5+B6)
- ProposeAddPeerAsync/ProposeRemovePeerAsync: single-change-at-a-time membership
  changes through RAFT consensus (Go ref: raft.go:961-1019)
- RaftLog.Compact: removes entries up to given index for log compaction
- CreateSnapshotCheckpointAsync: creates snapshot and compacts log in one operation
- DrainAndReplaySnapshotAsync: drains commit queue, installs snapshot, resets indices
- Pre-vote protocol skipped (Go NATS doesn't implement it either)
- 23 new tests in RaftMembershipAndSnapshotTests
2026-02-24 17:08:59 -05:00
Joseph Doherty
7116988d03 feat(jetstream): add API leader forwarding and stream purge options (C7+C8)
C7: JetStreamApiRouter now checks leadership before mutating operations.
Non-leader nodes return error code 10003 with a leader_hint field.
JetStreamMetaGroup gains IsLeader() and Leader for cluster-aware routing.

C8: StreamApiHandlers.HandlePurge accepts PurgeRequest options (filter,
seq, keep). StreamManager.PurgeEx implements subject-filtered purge,
sequence-based purge, keep-last-N, and filter+keep combinations.
2026-02-24 15:22:22 -05:00
Joseph Doherty
148ff9ebb6 feat: implement jetstream governance runtime parity semantics 2026-02-23 14:54:30 -05:00
Joseph Doherty
b41e6ff320 feat: execute post-baseline jetstream parity plan 2026-02-23 12:11:19 -05:00
Joseph Doherty
f46b331921 feat: complete remaining jetstream parity implementation plan 2026-02-23 10:16:16 -05:00
Joseph Doherty
c87661800d feat: add stream replica groups and leader stepdown 2026-02-23 06:17:30 -05:00
Joseph Doherty
23216d0a48 feat: integrate jetstream meta-group placement 2026-02-23 06:16:01 -05:00