Vendors the NORMATIVE (proto2) Eclipse Tahu sparkplug_b.proto and compiles it
in .Contracts with build-time Grpc.Tools, message-only (GrpcServices="None") --
Sparkplug rides MQTT, there is no gRPC service, so no Grpc.Core.Api is pulled in.
Provenance rides in the file header: repo, path, pinned commit
5736e404889d4b95910613040a99ba79589ffb13, permalink, git blob SHA-1
bf72ab5f..., content SHA-256 4432c5c4..., EPL-2.0, and a re-verify recipe. The
body below line 38 is byte-for-byte upstream; the blob SHA-1 matches the tree
entry at that commit, so the copy is provably genuine and not reconstructed.
Chose the proto2 file over upstream's sibling sparkplug_b_c_sharp.proto: the
sibling is a lossy proto3 restatement that drops explicit presence, and protoc
generates valid C# from proto2 into the identical namespace
(Org.Eclipse.Tahu.Protobuf, PascalCased from the package -- no
csharp_namespace option). Presence matters downstream: an NBIRTH legitimately
carries seq = 0 and a DATA metric legitimately omits `name`, so Has{Seq,Name,
Alias,IsNull,Datatype} is the difference between "absent" and "present, zero".
.Contracts stays transport-free: Google.Protobuf is a serialization dependency
with a framework-only graph, Grpc.Tools is PrivateAssets=all, and the resolved
graph is exactly {Google.Protobuf, Grpc.Tools, Core.Abstractions} -- no MQTTnet.
Codegen tests pin the round-trip, the namespace, the Sparkplug field numbers,
the DataType spec indices, and protoc's enum-name mangling (UInt64 -> Uint64,
UUID -> Uuid) that Task 17's datatype map has to spell correctly.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
MQTTnet 5 does not throw on an unsuccessful CONNACK — it returns the reason in
MqttClientConnectResult.ResultCode and v5 removed v4's ThrowOnNonSuccessfulConnectResponse,
so ConnectCoreAsync's unconditional SetState(Connected) sealed a wrong-password deployment
green: DriverState.Healthy / HostState.Running on a session that never authenticated.
Caught by the Task-13 Mosquitto fixture; 240 offline tests missed it.
ConnectCoreAsync now inspects the CONNACK and raises MqttConnectRejectedException.
Credentials/identity/protocol/Last-Will refusals are unrecoverable and set Faulted, which
stops the reconnect supervisor; transient refusals (ServerUnavailable, ServerBusy,
QuotaExceeded, UseAnotherServer, ConnectionRateExceeded, and anything unrecognised) stay
Reconnecting under backoff. MqttDriverProbe now shares the rejection wording, so the
AdminUI Test-connect button and the running driver can no longer disagree.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
C1 (Critical) — DisposeAsync went straight to TeardownAsync with no lifecycle-gate
wait, reopening the orphaned-connection class. Interleaving: ReinitializeAsync's
rebuild branch holds the gate mid-InitializeCoreAsync, past its own teardown but
before `_connection = connection`; an ungated dispose sees a null connection, does
nothing, and sets `_disposed`; the rebuild then publishes a live connection plus a
host-probe loop that every later dispose short-circuits past. Not proven reachable
today (DriverInstanceActor.PostStop calls the gated ShutdownAsync), but MqttDriver
publicly implements IAsyncDisposable, which invites `await using`.
- DisposeAsync now takes the gate with the same bounded wait + fallback teardown
ShutdownAsync uses, and sets `_disposed` BEFORE the wait.
- InitializeCoreAsync re-checks `_disposed` after the connect and disposes the
connection it just built rather than publishing it — this closes the residual
window on the bounded-timeout fallback path.
- The doc comment no longer claims parity with ShutdownAsync it did not have, and
stops conflating "don't Dispose() the semaphore object" with "don't WaitAsync".
I1 (Important) — the SameSession == false rebuild branch had no test driving it.
Adds three: an endpoint-changing delta rebuilds and Faults on a refused connect;
an ingest-only delta (MaxPayloadBytes) ALSO rebuilds, pinning that IngestIdentity
is nested inside SessionIdentity; and DisposeAsync serializes behind a lifecycle
operation parked at a new internal BeforeConnectHookForTests seam (mirroring
MqttConnection's AfterConnectHookForTests, which exists for the same race class).
Minors: comments recording that IngestIdentity names no Sparkplug field and must
grow one in P2 (tasks 21/22), and why _options/_subscriptions/_authoredRawPaths
get looser memory discipline than _health/_hostState.
Falsifiability: reverting DisposeAsync to the ungated form reddens exactly the new
serialization test ("Shouldly.ShouldAssertException : raced"); forcing SameSession
to always-true reddens exactly the two new rebuild tests. 240/240 MQTT tests pass;
forced rebuild of the driver project is 0 warnings under TreatWarningsAsErrors.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Task 9. Adds MqttDriverFactoryExtensions (DriverTypeName = DriverTypeNames.Mqtt,
direct deserialization of MqttDriverOptions — no intermediate DTO, mirroring
OpcUaClientDriverFactoryExtensions), the DriverTypeNames.Mqtt constant, and both
host wiring sites: the factory in DriverFactoryBootstrap.Register and the probe in
AddOtOpcUaDriverProbes (the admin-node path Program.cs calls in its hasAdmin
block — a probe wired only on driver nodes makes Test-connect silently dead).
Carried-forward items:
- Converges every MQTT config-parsing seam onto ONE JsonSerializerOptions —
MqttJson.Options, in .Contracts alongside MqttDriverOptions. It replaces the
probe's internal JsonOpts and the browser's separate private copy; the factory,
the probe, MqttDriver.ParseOptions and MqttDriverBrowser now all parse through
it. .Contracts is the only assembly all four consumers reference, and the
browser's reference to the runtime .Driver project is a documented layering
exception scheduled for removal — anchoring the options there would resurrect
the duplicate the day it goes away.
- Replaces the "Mqtt" literals in MqttDriverProbe, MqttDriverBrowser and
MqttDriver with the constant (string value unchanged).
- Tightens MqttDriverProbeTests.ProbeAsync_EnumAsName: it asserted only
Ok == false + non-empty message, which is also exactly what a JSON-parse
failure produces — so it stayed green under the very regression it names.
It now asserts the probe got past the parse and reached the network.
Falsifiability: deleting JsonStringEnumConverter from MqttJson.Options reddens 9
tests across 4 suites, including the tightened probe test (message becomes
"Config JSON is invalid: The JSON value could not be converted to
MqttProtocolVersion") — which the pre-fix assertions would have passed.
Also references the MQTT driver from Core.Abstractions.Tests so
DriverTypeNamesGuardTests' reflective bin scan discovers the new factory and the
constant/factory parity check stays honest.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Composes MqttConnection + MqttSubscriptionManager + LastValueCache into the
IDriver / ITagDiscovery / ISubscribable / IReadable / IHostConnectivityProbe /
IRediscoverable capability set.
- Discovery replays ONLY the authored raw tags (RediscoverPolicy = Once,
SupportsOnlineDiscovery = false) — a chatty broker can never auto-provision.
- Composition order is register -> AttachTo -> ConnectAsync; the manager's
Reconnected handler is passed through unwrapped so its throw-on-total-failure
still tears a deaf session down.
- ReinitializeAsync applies a tag-only delta in place and never faults on a bad
one; a session-changing delta rebuilds.
- ReadAsync serves the last-value cache and degrades per reference
(BadWaitingForInitialData), never throwing for the batch.
- FlushOptionalCachesAsync is a no-op: the last-value cache IS IReadable.
- Adds MqttDriverOptions.RawTags (the authored-tag delivery mechanism every
other driver's options DTO already has) and promotes MaxPayloadBytes from a
manager ctor knob to an operator-facing key.
- Converges on MqttDriverProbe.JsonOpts rather than a second, divergent
JsonSerializerOptions.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Test-connect probe for the Mqtt driver type: parses MqttDriverOptions
(shared enum-as-name JsonSerializerOptions), opens a bounded MQTT CONNECT
via MqttConnection.BuildClientOptions with a distinct -probe-{guid8}
client-id suffix, and classifies the outcome. Live-probed against
MQTTnet 5.2.0.1603 to confirm the real contract: a broker-rejected
CONNACK is a MqttClientConnectResult.ResultCode, never a thrown
exception, and timeout classification checks the deadline token itself
rather than switching on exception type (which varies unpredictably
between MqttConnectingFailedException and bare OperationCanceledException
for the same frozen-peer shape).
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
C1 (CRITICAL). AuthoredTable routed any wildcard-authored tag into `Wildcards`
and never into the concrete topic index. But the two paths that reconstruct
state from a SUBSCRIBED FILTER STRING — OnReconnectedAsync's re-subscribe set
and DegradeTopic — both looked up that concrete-only index, and
`_subscribedTopics` keys on the wildcard PATTERN ("f/+/temp"). So:
- reconnect resolved zero filters for a wildcard tag, hit a silent early
return, and issued NO subscribe. The cache keeps its last Good value, so the
tag never turns Bad — it just stops updating forever behind a connection
reporting healthy. Exactly the silent-death mode MqttConnection's own remarks
warn about, left unguarded for wildcards.
- DegradeTopic missed too, so a SUBACK rejection of a wildcard filter degraded
nothing and left the tag at BadWaitingForInitialData.
Fix: AuthoredTable now carries `ByFilter` (EVERY def, keyed by its authored
topic filter, wildcards included) alongside `ByExactTopic` (concrete only) and
`Wildcards`. Delivery matches an incoming published topic — which never carries
a wildcard — so it keeps using ByExactTopic + the comparer scan. Every path
keyed on a filter string uses ByFilter. The distinction is documented on the
record. The silent early return is now a Warning naming the orphaned topics.
Also in this pass:
- I2: bounded decode on the dispatcher thread. A body over `MaxPayloadBytes`
(default 1 MiB, ctor-settable) is refused BEFORE any GetString/JsonDocument
parse and degrades its own tags. Unbounded decode on the shared dispatcher is
paid by every subscription, not just the offending topic. NOTE: promoting this
to an operator-facing MqttDriverOptions key (+ WithMaximumPacketSize) needs a
.Contracts edit this task is scoped out of; flagged in the XML docs.
- I3: integral-valued reals now coerce to integer tags. JS/Python edge gateways
serialize an integer as 5.0, and TryGetInt32/int.TryParse are syntactic — an
Int32 tag fed by such a gateway silently never received data. Parsed via
decimal so integrality and range are exact across Int64/UInt64. A genuinely
fractional 5.5 is still refused, never rounded.
- I4: the JSON document is parsed ONCE per message and shared across the whole
fan-out (the documented "one document, one JSONPath per signal" shape was
re-parsing per tag). Zero-alloc via a ref struct when no Json tag matches.
- I5: pinned the documented JSON-string-holding-a-number coercion end-to-end.
- Minor: Register now prunes `_subscribedTopics` / `_handleByRawPath` entries for
tags a redeploy dropped, so a deleted topic stops being re-subscribed forever.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Three review findings, all "unbounded work/memory from untrusted broker input"
rather than correctness bugs - but this points at a live plant broker.
1. InferPayload decoded the WHOLE payload on MQTTnet's dispatcher thread before
trimming to 64 chars. The node cap never engages here: a multi-MB blob on an
already-known topic creates no node, it just updates one, so the cost repeats
per message forever. Now at most PayloadInspectMaxBytes (512) are examined.
A blind slice can cut a multi-byte sequence, which the strict decoder would
report as "binary" - TrimToUtf8Boundary backs the window off to the last whole
sequence so good UTF-8 is never mislabelled. A clipped payload is String by
construction rather than inferred from a partial view.
2. The node cap bounded COUNT, not bytes. MQTT topics run to ~65KB, so 50k nodes
near that ceiling is a multi-GB tree. Topics now bounded at 1024 chars and
segments at 256, rejected whole (a truncated topic is a DIFFERENT topic the
picker would commit as a tag address) and surfaced via an __oversized__ marker
kept distinct from __truncated__ - the operator's remedy differs.
3. Control characters in a snippet would break the picker's single-line row;
Trim() only strips the ends. Now scrubbed to spaces.
Each guard was falsified independently by neutering it and confirming exactly one
test reddens. That found a real gap: the oversized-topic test was passing via the
SEGMENT bound, leaving the whole-topic bound untested - the test now uses many
short segments so only the topic bound can reject it.
Also records the .Driver project reference as a known, deliberate exception to the
.Browser -> .Contracts pattern, with its cost (AdminUI gains Core + Polly + Serilog)
and the clean fix (a leaf transport project; NOT a move into .Contracts, which is
deliberately transport-free).
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
MqttSubscriptionManager is the P1 plain-MQTT ingest path: it holds the authored
RawPath → MqttTagDefinition table behind the shared EquipmentTagRefResolver,
establishes one SUBSCRIBE per distinct authored topic, and turns each inbound
message into a LastValueCache update plus an OnDataChange notification.
Load-bearing choices, each pinned by a falsified test:
- The published reference IS the RawPath (def.Name), never the topic and never
the TagConfig blob — DriverHostActor's dual-namespace fan-out is RawPath-keyed,
so any other key passes every unit test and delivers nothing in production.
- One message fans out to EVERY tag on its topic; several tags routinely share a
topic with different jsonPaths, and stopping at the first match silently starves
the rest.
- An unauthored topic raises nothing: MQTT ingest never auto-provisions.
- Wildcard-authored topics (accepted by the parser, warned at deploy) match via
MQTTnet's own MqttTopicFilterComparer, so '+'/'#' behave as a broker would.
Concrete topics stay a single lookup; the wildcard scan runs only when one exists.
- Retained seeds are honoured natively on MQTT 5.0 (retain handling) AND filtered
client-side on the retain flag, because 3.1.1 brokers always replay.
- Nothing throws on MQTTnet's dispatcher thread: a malformed payload degrades that
tag (BadDecodingError / BadTypeMismatch), and a throwing OnDataChange subscriber
is contained without starving the tags behind it.
- The manager issues the FIRST subscribe itself — Reconnected deliberately does not
fire after the initial ConnectAsync.
- Reconnect re-subscribe: total failure THROWS (MqttConnection then tears the
session down and retries under backoff, rather than serving a connected-but-deaf
driver); a partial SUBACK rejection degrades only the denied refs, because
retrying forever over one ACL-denied topic would take every tag dark.
MqttConnection gains a MessageReceived observer (fired on the dispatcher, throwing
observers contained) and a bounded SubscribeAsync under its own linked-CTS deadline
— deliberately not MqttClientOptions.Timeout, which already governs every MQTTnet
op. A refused filter is an outcome, not an exception; only the SUBSCRIBE itself
failing throws. Classify() is parameterised by operation name (messages unchanged
for connect).
JSONPath is a documented subset ($, $.a.b, $['a'], $.a[0]); filters/slices/
recursive descent report a miss rather than pulling in a JSONPath package for an
expression form that selects a set where a tag needs one scalar.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
MQTT has no discovery protocol, so the AdminUI address picker learns the topic
space the only way there is: subscribe to the wildcard and accumulate what
arrives. MqttBrowseSession serves that observation as a segment tree (split on
'/'); MqttDriverBrowser opens it with a distinct "{clientId}-browse-{guid8}"
identity under a 5-30 s clamped budget.
Browsing publishes NOTHING - the load-bearing safety property, since the picker
runs against a live plant broker. Every outbound message must route through the
single PublishAsync seam that counts into PublishCountForTest; P2's operator-
triggered Sparkplug rebirth is the one intended exception. Sparkplug mode fails
fast rather than serving a raw spBv1.0 topic tree that looks like a metric tree.
Deviation from the plan's ref list: the project also references .Driver so the
browse CONNECT reuses MqttConnection.BuildClientOptions (TLS posture, CA-pin
chain validator, credentials). A second copy would be a security divergence.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Closes the connect-vs-connect defect: MQTTnet throws (and raises no DisconnectedAsync)
on a connect-while-connected, so a caller's ConnectAsync and the reconnect supervisor
corrupted each other in both directions. Both paths now check first; when the supervisor
finds the session already restored it stands down WITHOUT firing Reconnected.
Reconnected becomes Func<CancellationToken, Task>, fed from the lifetime token and capped
at ConnectTimeoutSeconds, so a hung re-subscribe can no longer park the supervisor.
Connected is published only after the re-subscribe succeeds; a supervisor that dies now
reports Faulted instead of Reconnecting forever.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
LastValueCache bridges MQTT's subscribe-first push model to the OPC UA
server's polled IReadable.ReadAsync: the subscription path calls Update()
per RawPath, the read path calls Read(). Read never throws; an unseen
RawPath returns BadWaitingForInitialData (0x80320000) rather than an
exception or null, so a batch covering many references degrades per-ref
instead of failing wholesale.
Deviates from the plan's GoodNoData snippet: GoodNoData is reserved
repo-wide for "the historian window held no samples" (NullHistorianDataSource,
OtOpcUaNodeManager HistoryRead paths); BadWaitingForInitialData is the
established convention for "no live value observed yet" (CalculationDriver,
VirtualTagEngine, FOCAS, AddressSpaceApplier). Keyed by RawPath per the v3
driver-reference identity, not a topic/JSON-path-derived key.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Review follow-ups on MqttConnection (Task 3):
- Every certificate test asserted rejection, so the accept branch was
unreachable-by-regression. Adds a leaf genuinely issued by the pinned CA and
asserts acceptance.
- ValidateAgainstPinnedCa never seeded ChainPolicy.ExtraStore from the incoming
chain, so a leaf behind an intermediate delivered during the handshake failed
despite a legitimate path to the pinned root. Seeds from both the incoming
chain's elements and its ExtraStore; CustomRootTrust still means only the
pinned roots may terminate the chain.
- A DisposeAsync racing an in-flight connect escaped as an unclassified
exception; it now folds into ObjectDisposedException.
- Promotes the single-caller concurrency invariant into the type remarks, with
the accurate blast radius (a leaked live connection, not a benign throw).
Serialising the lifecycle remains Task 4's job.
- X509Chain.Build can throw; an exception escaping a TLS validation callback is
an opaque handshake crash, so it is caught and refused.
- Adds a connect-retry test (Task 4's reconnect loop reuses the instance) and a
disposed-then-connect test.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Task 2 review follow-up. The plan specified MqttEquipmentTagParser.TryParse(reference)
with def.Name = the TagConfig blob, and told us to mirror a type named
ModbusEquipmentTagParser. Both are plan defects:
- EquipmentTagRefResolver documents that a v3 driver reference "is now always a
RawPath" and that the blob-parse fallback is retired. Keying Name by the blob
would make OnDataChange publish under a reference that never matches the
RawPath-keyed fan-out in DriverHostActor - silently dead in production with
every unit test still green.
- ModbusEquipmentTagParser does not exist. The six sibling drivers all use
<Driver>TagDefinitionFactory.FromTagConfig(tagConfig, rawPath, out def).
Changes:
- Rename MqttEquipmentTagParser -> MqttTagDefinitionFactory; TryParse(reference,
out def) -> FromTagConfig(tagConfig, rawPath, out def) setting Name: rawPath,
matching ModbusTagDefinitionFactory's structure, param docs and guard order.
- Pin the identity contract with a dedicated test so a regression to blob-keying
goes red.
- Read qos with the same strictness as the enums: a present-but-invalid qos
("high" / 1.5 / 5 / null) now rejects the tag and is warned by Inspect,
instead of being silently absorbed into the driver-level default and handing
the operator a weaker delivery guarantee than the one they authored.
- No ToTagConfig inverse: the siblings carry one solely for their Driver.<X>.Cli
project, the MQTT plan defines none, and the AdminUI editor template
references no driver factory. Recorded as an explicit YAGNI call in the type
doc rather than added speculatively.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Code review of f22db5d8 (approved-with-findings): the existing round-trip
tests only exercised explicit JSON payloads, so nothing failed if UseTls /
AllowUntrustedServerCertificate or the §5.1 numeric defaults were flipped.
Adds a defaults-pinning test plus a partial-JSON test that omits the TLS
knobs entirely (the production case — an operator config that just doesn't
mention TLS must not silently land insecure).
Also overrides the record's PrintMembers so ToString() no longer prints
Password in plaintext (verified RED before the fix: the new test failed
with the raw password in the rendered string). OpcUaClientDriverOptions has
the same unredacted-ToString() shape but is out of scope for this task per
the coordinator's note; flagging as a possible follow-up.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
Joseph Doherty