feat: add JetStream cluster replication and leaf node solicited reconnect

Add JetStream stream/consumer config and data replication across cluster
peers via $JS.INTERNAL.* subjects with BroadcastRoutedMessageAsync (sends
to all peers, bypassing pool routing). Capture routed data messages into
local JetStream stores in DeliverRemoteMessage. Fix leaf node solicited
reconnect by re-launching the retry loop in WatchConnectionAsync after
disconnect.

Unskips 4 of 5 E2E cluster tests (LeaderDies_NewLeaderElected,
R3Stream_NodeDies_PublishContinues, Consumer_NodeDies_PullContinuesOnSurvivor,
Leaf_HubRestart_LeafReconnects). The 5th (LeaderRestart_RejoinsAsFollower)
requires RAFT log catchup which is a separate feature.
This commit is contained in:
Joseph Doherty
2026-03-13 01:02:00 -04:00
parent ab805c883b
commit 3445a055eb
8 changed files with 164 additions and 5 deletions
@@ -32,4 +32,15 @@ public static class JetStreamApiSubjects
public const string ConsumerLeaderStepdown = "$JS.API.CONSUMER.LEADER.STEPDOWN.";
public const string DirectGet = "$JS.API.DIRECT.GET.";
public const string MetaLeaderStepdown = "$JS.API.META.LEADER.STEPDOWN";
// Internal replication subjects for cluster-wide JetStream state propagation.
// These are NOT part of the public API — they are used between cluster peers
// to replicate mutating operations (stream/consumer create/delete/purge).
// Go reference: jetstream_cluster.go — internal replication via RAFT proposals.
public const string InternalPrefix = "$JS.INTERNAL.";
public const string InternalStreamCreate = "$JS.INTERNAL.STREAM.CREATE.";
public const string InternalStreamDelete = "$JS.INTERNAL.STREAM.DELETE.";
public const string InternalStreamPurge = "$JS.INTERNAL.STREAM.PURGE.";
public const string InternalConsumerCreate = "$JS.INTERNAL.CONSUMER.CREATE.";
public const string InternalConsumerDelete = "$JS.INTERNAL.CONSUMER.DELETE.";
}