Implements AssignmentCodec with JSON serialization for StreamAssignment
and ConsumerAssignment, plus Snappy compression helpers for large payloads.
Adds 12 tests covering round-trips, error handling, compression, and a
golden fixture for format stability.
Add Version property to StreamAssignment and ConsumerAssignment so future-version
entries from newer cluster peers are gracefully skipped rather than applied incorrectly.
ProcessStreamAssignment and ProcessConsumerAssignment now reject entries with
Version > CurrentVersion (1), incrementing SkippedUnsupportedEntries for operator
visibility. Add MetaEntryType.Unknown with a no-op ApplyEntry handler so unrecognised
RAFT entry types never crash the cluster. Version 0 is treated as current for
backward compatibility with pre-versioned assignments.
Implement IsMember, SetPreferred, RemovePeer, AddPeer, CreateRaftGroup
factory, IsUnderReplicated, and IsOverReplicated on RaftGroup. Add 22
RaftGroupLifecycleTests covering all helpers and quorum size calculation.
Extends PlacementEngine.SelectPeerGroup with three new capabilities ported
from jetstream_cluster.go:7212 selectPeerGroup:
- JetStreamUniqueTag enforcement: PlacementPolicy.UniqueTag (e.g. "az")
ensures no two replicas share the same value for a tag with that prefix,
matching Go's uniqueTagPrefix / checkUniqueTag logic.
- MaxAssetsPerPeer HA limit: peers at or above their asset ceiling are
deprioritised (moved to fallback), not hard-excluded, so selection still
succeeds when no preferred peers remain.
- Weighted scoring: candidates sorted by
score = AvailableStorage - (CurrentAssets * assetCostWeight)
(DefaultAssetCostWeight = 1 GiB) replacing the raw-storage sort, with a
custom weight parameter for testing.
10 new tests in TopologyPlacementTests.cs cover all three features and their
edge cases. All 30 PlacementEngine tests continue to pass.
Add ProcessAddPeer/ProcessRemovePeer to JetStreamMetaGroup for
peer-driven stream reassignment. Includes AddKnownPeer/RemoveKnownPeer
tracked in a HashSet, RemovePeerFromStream, RemapStreamAssignment with
replacement-peer selection from the known pool, and DesiredReplicas on
RaftGroup for under-replication detection. Go ref: jetstream_cluster.go:2290-2439.
Extend ApplyEntry in JetStreamMetaGroup with PeerAdd/PeerRemove dispatch
to the existing Task 12 peer management methods. Add StreamMsgOp (Store,
Remove, Purge) and ConsumerOp (Ack, Nak, Deliver, Term, Progress) enums
plus ApplyStreamMsgOp and ApplyConsumerEntry methods to StreamReplicaGroup.
Extend ApplyCommittedEntriesAsync to parse smsg:/centry: command prefixes
and route to the new apply methods. Add MetaEntryType.PeerAdd/PeerRemove
enum values. 35 new tests in EntryApplicationTests.cs, all passing.
Add ProcessLeaderChange(bool) method and OnLeaderChange event to JetStreamMetaGroup.
Refactor StepDown() to delegate inflight clearing through ProcessLeaderChange,
enabling subscribers to react to leadership transitions.
Go reference: jetstream_cluster.go:7001-7074 processLeaderChange.
Replace the simple string-keyed inflight dictionaries with account-scoped
ConcurrentDictionary<string, Dictionary<string, InflightInfo>> structures.
Adds InflightInfo record with OpsCount for duplicate proposal tracking,
TrackInflight/RemoveInflight/IsInflight methods for streams and consumers,
and ClearAllInflight(). Updates existing Propose* methods to use $G account.
Go reference: jetstream_cluster.go:1193-1278.
Add 5 validated Process* methods to JetStreamMetaGroup for stream and
consumer assignment processing: ProcessStreamAssignment, ProcessUpdateStreamAssignment,
ProcessStreamRemoval, ProcessConsumerAssignment, and ProcessConsumerRemoval.
Each returns a bool indicating success, with validation guards matching
Go reference jetstream_cluster.go:4541-5925. Includes 12 new unit tests.
Implements the background loop (Go: monitorCluster) that reads RaftLogEntry
items from a channel and dispatches assignStream, removeStream, assignConsumer,
removeConsumer, and snapshot operations to JetStreamMetaGroup.
- Add five public mutation helpers to JetStreamMetaGroup (AddStreamAssignment,
RemoveStreamAssignment, AddConsumerAssignment, RemoveConsumerAssignment,
ReplaceAllAssignments) used by the monitor path
- JetStreamClusterMonitor: async loop over ChannelReader<RaftLogEntry>, JSON
dispatch, ILogger injection with NullLogger default, malformed entries logged
and skipped rather than aborting the loop
- WaitForProcessedAsync uses Monitor.PulseAll for race-free test synchronisation
with no Task.Delay — satisfies slopwatch SW003/SW004 rules
- 10 new targeted tests all pass; 1101 cluster regression tests unchanged
Implements binary codec for meta-group snapshots: 2-byte little-endian version
header followed by S2-compressed JSON of the stream assignment map. Adds
[JsonObjectCreationHandling(Populate)] to StreamAssignment.Consumers so the
getter-only dictionary is populated in-place during deserialization. 8 tests
covering round-trip, compression ratio, field fidelity, multi-consumer restore,
version rejection, and truncation guard.
Go reference: jetstream_cluster.go:2075-2145 (encodeMetaSnapshot/decodeMetaSnapshot)
Re-enabled the leader check in JetStreamApiRouter.Route() that was
commented out during B8 agent work. Added BecomeLeader() method to
JetStreamMetaGroup for single-process test fixtures to simulate
winning the post-stepdown election. MetaControllerFixture now
auto-calls BecomeLeader() after a successful meta leader stepdown.
- RaftGroup, StreamAssignment, ConsumerAssignment types matching Go structs
(jetstream_cluster.go:154-266)
- PlacementEngine.SelectPeerGroup: topology-aware peer selection with cluster
affinity, tag filtering, exclude tags, and storage-weighted sorting
(Go ref: selectPeerGroup at line 7212)
- JetStreamMetaGroup: backward-compatible rewrite with full assignment tracking,
consumer proposal workflow, and delete operations
- 41 new tests in ClusterAssignmentAndPlacementTests
- ProposeAddPeerAsync/ProposeRemovePeerAsync: single-change-at-a-time membership
changes through RAFT consensus (Go ref: raft.go:961-1019)
- RaftLog.Compact: removes entries up to given index for log compaction
- CreateSnapshotCheckpointAsync: creates snapshot and compacts log in one operation
- DrainAndReplaySnapshotAsync: drains commit queue, installs snapshot, resets indices
- Pre-vote protocol skipped (Go NATS doesn't implement it either)
- 23 new tests in RaftMembershipAndSnapshotTests
Joseph Doherty