Reverses the conservative call from the previous commit at the user's direction,
and they were right: "surface it on a counter first" left the wire lying. An
MXAccess event can carry Good quality while its payload is a kind MxValueDecoder
has no mapping for, so the old behaviour published null-under-Good — which a
client cannot tell apart from a genuine null reading. A value the gateway DID
send would be consumed as a value it did not, with the only evidence in a log
nobody is tailing. Bad data is the honest wire representation of lost data.
BadTypeMismatch is an existing StatusCodeMap constant, deliberately reused rather
than written inline: StatusCodeParityTests finds status constants by reflecting
over Good/Uncertain/Bad-prefixed uint fields and checking them against the pinned
SDK, so an inline literal would be invisible to that guard (Gitea #497, where 16
wrong constants had accumulated across 6 drivers exactly this way).
Logging is now once per DISTINCT value kind rather than once per pump. The set is
bounded by the MxValue oneof enum so a broken producer still cannot flood, but a
second unrepresentable kind appearing later gets its own line — warn-once-per-pump
would have silently swallowed it, which is the same class of bug as the null this
commit fixes.
The test's fixture matters more than its assertion: the undecodable event carries
Quality = 192 (OPC DA Good), so a decoder that gave up silently would publish Good
and fail. Paired with a positive control — a decodable value at the same quality
must still publish Good — so it cannot be satisfied by a pump that marks
everything Bad. Falsified by reverting the status to Good: the test reds, and only
that test.
Verified: 331 Galaxy tests green, Core.Abstractions 298 (the parity guard),
solution builds 0 errors. Docs corrected in the two places that recorded the old
decision as deliberate.
Closes the two follow-ups left open by the MxGateway 0.2.0 bump. Reading the
gateway's own contract first changed what each of them is, and one of my earlier
claims was simply wrong -- both corrections are recorded below.
Replay gap
----------
A sentinel MxEvent carries family = UNSPECIFIED, so it fell through EventPump's
family switch into the default drop: we silently ignored a documented MUST ("you
missed events -- discard local state and re-snapshot"). It is now intercepted and
answered by re-issuing SubscribeBulk for every tracked subscription, since
MXAccess delivers an initial value per item on subscribe -- re-subscribing IS the
re-snapshot. It does not restart the event pump (the stream carrying the sentinel
is healthy, and the handler runs on that stream's own read loop, so disposing the
pump from there would deadlock on itself), it does not block the read loop, and
re-entry is gated so a burst collapses to one pass instead of a subscribe storm
per sentinel.
Handled on the READ loop, not in the dispatch loop, because everything past that
point is subject to newest-dropped channel pressure -- and "you missed events" is
the one signal whose loss is unrecoverable. Pinned by a test that fills the
channel and still expects the sentinel.
⚠️ This cannot fire in the shipped configuration, and I did not pretend otherwise
or paper over it by enabling resume. The gateway emits the sentinel only for a
stream opened with after_worker_sequence > 0; GatewayGalaxySubscriber passes 0,
which the gateway treats as "fresh stream, not a resume -- plain attach, no
replay, no sentinel". Enabling resume is a larger and genuinely delicate change:
the sequence is only meaningful within ONE gateway session, our reconnect
recreates the session, and replaying a remembered sequence against a fresh
session sets the server's live watermark ABOVE every event it is about to send --
silencing the stream instead of recovering it. That deserves its own change with
a live gate, which I cannot run from here.
Sparse arrays -- correcting the last commit message
---------------------------------------------------
I wrote that the bump "makes handling it possible" on the decode path. That is
wrong: MxSparseArray is documented write-only, the gateway expands it before
forwarding, and "the worker never receives or produces it". So the decode path
is unreachable by contract and the write path has no use for it either -- it is a
wire compaction for a WHOLE-array write, not a partial update (unmentioned
indices are RESET, deliberately, because the gateway cannot read-modify-write
without racing the provider). It is therefore the wrong primitive for an OPC UA
IndexRange write, which expects untouched elements preserved -- and WriteRequest
carries no IndexRange in the first place.
It is decoded anyway, as contract defence: the alternative if a future gateway
ever emits one is a silent null published under Good quality. Fill semantics
mirror the gateway's own SparseArrayExpander exactly (epoch for Time, empty
string, Integer widening to long[] when any element is int64) so a value that
round-trips through a write reads back as the same CLR array.
Also: the decoder is now total by contract, since it runs inside the dispatch
loop where an escaping exception kills dispatch for the life of the pump. A
malformed payload degrades instead of throwing.
A real defect found in the same code
------------------------------------
A stream the GATEWAY completes -- a closed or evicted session -- ends the RPC
without an error, so nothing threw, the supervisor was never notified, and the
pump exited silently. GetHealth() kept reporting Healthy, which also means the
#527 comms-loss watchdog could never trip, because it requires a non-Healthy
state. The driver went permanently deaf with no signal anywhere. This one is
live, unlike the two above. Now reported as a fault; the falsification control is
the existing CleanShutdown_DoesNotInvokeOnStreamFault, since our own disposal
also ends the stream and must NOT be reported.
Value kinds we cannot represent are now counted and warned once rather than
published as an indistinguishable null. The status is deliberately NOT downgraded
to Bad: nothing can reach that branch today, and changing what an OPC UA client
sees for a live shape is not a change to make without a gate. The counter exists
so the next person has evidence instead of a guess.
Verified: 330 Galaxy tests green (12 new), plus Galaxy.Browser 21, Runtime 558,
Core 351; full solution builds 0 errors with the warning census unchanged. Every
new test was falsified first -- disabling the gap interception reds 3 of 4 (the
throwing-callback test passes vacuously when the callback never runs, by
construction), disabling sparse decode plus the representability signal reds all
7 decoder tests, and both were restored and re-verified green.
Docs: the Galaxy.Performance invariant was stated as received = dispatched +
dropped + in-flight, which this breaks -- a sentinel is received but never enters
the channel -- so it now reads + replay_gaps. Galaxy.md gains the three
operational notes; DriverLifecycle.md records that "reaches a non-Healthy state"
is an obligation a driver can silently fail to meet, which is the general shape
of the clean-end bug.
Foundation task of the AdminUI UI sweep. CSS/link/doc only — no behavior change
beyond restoring styles that were built but never served.
- App.razor: link the RCL's scoped-CSS bundle
(_content/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.bundle.scp.css).
It is produced on every compile and served by the Host, but nothing referenced
it, so ContextMenu.razor.css was dead at runtime and context menus rendered
in-flow with no position:fixed.
- site.css: three additions.
* .btn / .btn-sm sizing as Bootstrap CSS-VARIABLE overrides (never box
properties — .btn-group depends on Bootstrap's own .btn rules for its seams
and end-cap radii). Theme 0.3.1 ships no .btn rule, so Bootstrap's 1rem
applied against the theme's 0.9rem body. Upstreaming candidate, same as the
ignitionoee router's app.css.
* .notice-alert — red variant of the kit's amber .notice, on the --bad/
--bad-bg/--bad-border tokens. The 13 existing call sites hack this with
style="border-color:var(--alert)", and no --alert token exists anywhere, so
the tint silently no-ops today. Call-site migration is a separate task.
* .tree-twist — shared 18px expander column for the tree components, replacing
repeated inline style="width:18px". Rows are d-flex, so flex-basis is what
actually holds the column. Adoption is a separate task.
- ContextMenu.razor.css: correct the header comment — the kit is light-only
(no dark block in Theme 0.3.1, site.css, or this file). No rule changes.
- docs/Telemetry.md: record that driver-health is keyed per driver instance
across the cluster, not per Akka member (the wire record carries no per-member
identity). The fact previously lived only in Hosts.razor page copy.
- docs/Configuration.md: new audit-log subsection recording that EventId gives
cross-restart dedup, enforced by the filtered unique index
UX_ConfigAuditLog_EventId, so a retried apply inserts no duplicate row;
CorrelationId joins a row to its originating workflow. Previously only in the
historical v2 rebuild plan and ClusterAudit page copy.
IWritable shipped behind write.enabled and docs/drivers/MTConnect.md told the operator to
set it, but MTConnectDriverForm — the only editor DriverFormMaps maps for this type, with
no raw-JSON escape hatch and no driver-instance API — exposed no write section, so the
documented enablement path was a hand edit of DriverInstance.DriverConfig in the config DB.
- Write-back section: switch + form text stating it is observation injection, that the
Agent's AllowPut is an independent second gate, and that it does not by itself make a
node writable over OPC UA (Tag.AccessLevel still composes CurrentWrite).
- Write added to the mirrored MTConnectConfigDto and to the form model, written in BOTH
states — an omitted block would leave a previously-authored true standing, so the switch
could never turn writing off again.
- The false 'MTConnect is read-only' header comment is gone; the doc names the new section.
No bUnit here, so the model/serialization seam is unit-tested both sides: the AdminUI test
pins the emitted bytes, and MTConnectFactoryTests parses that same document and asserts
MTConnectDriverOptions.Write.Enabled. Live page verification remains a rig step.
The two value-bearing fields were read asymmetrically and nothing said why.
whereValue went through ReadScalar (string/number/boolean — the design doc's own
wide-row example authors it as the number 7); keyValue went through the string-only
ReadString, so a numerically-authored key refused the tag outright. Nothing downstream
justified the split: both are BOUND as DbParameters and both are matched back as text —
SqlPollReader.IndexByKey stringifies the key cell precisely so an authored key can face
an INTEGER key column.
The refusal was also mis-explained. Inspect branched on ReadString(root, "keyValue") is
null, so a numerically-authored key drew "Sql KeyValue tag has no 'keyValue'" — telling
the operator the field is missing when it is right there in the blob, the #549 shape of
an authoring surface disagreeing with the runtime about WHY.
keyValue now uses ReadScalar; Inspect mirrors it with HasScalar and says what a usable
key is; SqlTagConfigModel loads it with the shared TagConfigJson.GetScalarText (which
also replaces SqlRowSelectorModel's private copy), so the editor is not left stricter
than the parser. An object/array/null key is still refused. docs/drivers/Sql.md updated.
RED first: 4 failing in Driver.Sql.Tests.
-003 is a documented Won't Fix: the mitigating control is the ConfigEditor page
policy on the only two surfaces that reach a browse session (/raw and /uns),
there is no HTTP endpoint for browse, and PageAuthorizationGuardTests pins both
pages to that exact policy. Recorded in docs/security.md under Control-Plane
Authorization and pointed at from SqlDriverBrowser's class summary.
PreferredTimestampColumn nominated the first column mapping to DriverDataType
.DateTime, and that set includes date-only types (SQL Server / PostgreSQL
'date'). A table whose first date/time column is a 'date' committed
rowSelector.topByTimestamp = <date>, making the wide-row read ORDER BY <date>
DESC LIMIT 1 - which ties for every row written that day, so 'the newest row'
became storage order among the ties.
Time-bearing candidates now outrank date-only ones; a date-only column is still
stated when it is all the table has, so such a table stays bindable. Oracle's
DATE carries a time of day and is exempt - pinned by a control dialect that
differs from the SQLite one only in the Provider it reports.
The comments and docs that described Driver.MTConnect-003 as live at HEAD would
have been wrong the moment 64be41d8 landed:
- code-reviews/Driver.MTConnect/findings.md — Driver.MTConnect-003 Resolved,
with the fixing commit; module open-finding count 5 -> 4.
- code-reviews/Driver.MTConnect.Contracts/findings.md — the one clause inside
Contracts-001's prose that asserted 003 was Open.
- docs/drivers/MTConnect.md — the coercion-precedence warning now states that a
browse-committed tag resolves at step 1 CARRYING the step-3 answer, and that
step 3 remains the rescue for a hand-authored / CSV / API blob with no type.
- MTConnectDriver.BuildBinding, MTConnectDataTypeInference and
MTConnectDriverOptions.RawTags remarks — same correction, each keeping a short
historical note of what the defect was so a reader meeting the old blobs still
understands them.
Comment-only in src; the driver suite (538) and AdminUI suite (1474) re-run green.
Findings from the per-module review of Driver.Mqtt at b990adae
(code-reviews/Driver.Mqtt/findings.md records all ten; these are the fixed
ones).
Driver.Mqtt-001 (High) — MqttSubscriptionManager._subscribedTopics is a claim
about the broker session it established filters on, and AttachTo never cleared
it, but the manager OUTLIVES a session rebuild (AdoptOptions keeps it whenever
the ingest identity is unchanged). After a driver-host reconnect
(ShutdownAsync + InitializeAsync, the #527 watchdog's retry) or a
ReinitializeAsync that only changed the endpoint/credentials, the host's
re-subscribe resolved an empty pending set, sent no SUBSCRIBE at all, and left
the driver Connected/Healthy on a session carrying ZERO subscriptions — no
error, no Bad status, values frozen until the new session happened to drop.
AttachTo now forgets them, mirroring SparkplugIngestor.AttachTo's reset.
Driver.Mqtt-002 (High) — InitializeCoreAsync published _connection and
_primaryHost over whatever the driver already held, so a second initialize
stranded a connected, self-supervising broker session (plus a second Sparkplug
primary-host claim) that no later dispose could reach. Reachable without any
teardown of ours via DriverInstanceActor.AdoptConfigDuringInit. Now tears down
first, under the gate both callers already hold.
Driver.Mqtt-003 (Medium) — connectTimeoutSeconds is [Range]-annotated but
unvalidated on the runtime path; 0 makes every deadline fire immediately and a
negative value throws ArgumentOutOfRangeException out of ShutdownAsync and
DisposeAsync. Normalized once, at the seam every options instance passes.
Also: ShutdownAsync keeps the live last-message instant (-004), three usings
left behind by #512 removed (-005), stale remarks corrected (-009, -010), and
docs/drivers/Mqtt.md now distinguishes the two kinds of reconnect.
Tests 708 -> 713. All three regression tests were confirmed to fail against
the un-fixed code, with two controls that pass in either state.
Driver.Sql-001 (Medium): SqlCatalogLoader.ReadDefaultSchemaAsync documented a
fallback to the sole visible schema when the server reports no default schema and
did not implement it, so a login with no DEFAULT_SCHEMA (or a Postgres search_path
that resolves to nothing) rejected EVERY unqualified authored table at the catalog
gate — a whole-driver BadNodeIdUnknown outage against a healthy database. Fixed;
SqlCatalogDefaultSchemaTests written red first, with a control proving the
ambiguous multi-schema case still refuses to guess.
Driver.Sql-003 (Medium): docs/drivers/Sql.md spelled the WideRow row selector at
the top level, but the parser reads those keys only inside a nested rowSelector —
a tag authored from the doc is refused and reads BadNodeIdUnknown (the #552 class,
via the documentation). Also removed the documented-but-nonexistent pollIntervalMs
per-tag field.
Driver.Sql-007 (Low): dropped the unused Microsoft.Data.SqlClient PackageReference
left behind by the .Dialect extraction (#526).
Open: 002 (no IN-list chunking vs the 2100/1000 provider bind caps), 004, 005, 006,
008, 009, 010.
Driver.MTConnect-001 (Medium): StreamVerdict.Reopen reset the reconnect ladder
unconditionally. The InvalidDataException catch that routes an Agent OUT_OF_RANGE
to a /current re-baseline also catches an unparseable chunk and an over-sized
multipart part - neither of which a re-baseline can resolve - so the pump reopened
and failed with no delay at any point: two HTTP requests and one Warning per
iteration, forever. The Reopen arm now uses the same discriminator the transient
arm does (did THIS connection deliver a chunk?), so a buffer roll behind a working
stream still reopens instantly while a re-baseline that resolved nothing advances
BackoffFor.
Driver.MTConnect-002 (Medium): MTConnectDriverProbe requested {AgentUri}/probe
while MTConnectAgentClient requests {AgentUri}/{DeviceName}/probe, so a mistyped or
since-renamed deviceName showed GREEN on Test Connect and then Faulted the driver
at InitializeAsync (an unknown device path answers MTConnectError under HTTP 200).
The probe's minimal DTO now reads deviceName - the rule is 'which keys compose the
URL', not 'which keys are cheap to read'.
Driver.MTConnect-008 (Low): corrected two stale claims that a since-changed
RawBrowseCommitMapper invalidated (see Driver.MTConnect-003, which remains open and
belongs in AdminUI), and made DiscoverAsync log the options snapshot it
deliberately read once instead of the field.
Tests: 538 pass (534 base + 4 new, each red before its fix); AdminUI.Tests 1466 pass.
`RawBrowseCommitMapper.BuildSqlTagConfig` emitted a WideRow blob carrying only
table + columnName. A wide-row tag must also say WHICH row it reads, so both
`SqlEquipmentTagParser` (the runtime authority) and `SqlTagConfigModel.Validate`
(the authoring gate) refused it — a browse-committed Sql tag was never readable,
first silently as BadNodeIdUnknown and, since #551 put the gate on the import,
loudly at commit.
`SqlBrowseSession` now states a fourth address field: the table's preferred
date/time-typed column, computed from the columns its attributes call ALREADY
read (no extra catalog round-trip). Candidacy is decided by
`ISqlDialect.MapColumnType` reporting `DriverDataType.DateTime` — never by column
name, which would order a wide-row read by a string column that merely looks like
a timestamp. Multiple candidates tie-break on FIRST IN CATALOG ORDER (every
dialect's column SQL orders by ordinal position), so the pick is stable across
calls and servers; it is a visible, editable default the typed editor round-trips.
No candidate (a table with no date/time column) is an operator-actionable refusal,
not an invented selector: the only remaining selector is a whereColumn/whereValue
pair whose value is data, not schema, so `DescribeUncommittableLeaf` refuses that
leaf by name, names the table, and points at the tag editor.
The session and the mapper now agree via `SqlBrowseAddressFields` constants rather
than two independently spelled string literals — a disagreement there is silent,
and is exactly the shape this defect had.
`RawTagConfigGateTests`' Sql leg flips from "records the gap" to "browse-commit
output passes the gate", with the no-candidate case pinned as its own test.
Blazor does not serialize OnParametersSetAsync against its own outstanding
task, and the modal's `_open` bool cannot tell two opens apart. ResetAndResolveAsync
awaits LoadMergedProbeConfigAsync and ResolveGroupContextAsync and then wrote
_driverType/_mergedConfig/_effectiveDeviceId/_groupPrefix/_canBrowse/_disabledReason
with no staleness re-check, so a close-reopen onto another device let the first
open's late continuation install its device's browse context under the displayed
modal — "Connect & browse" would silently dial the wrong controller and the commit
would land under the wrong tag-group.
Ports the per-open `_openGeneration` counter proven in the two sibling modals
(RawManualTagEntryModal, RawTagModal, 2026-08-05) to every post-await write here:
after both awaits, in the catch's error banner, and in the finally. The finally is
the one departure from the sibling shape and it is load-bearing — this modal resolves
inside a try/finally, so a stale `return` still runs the tail and would drop the LIVE
open's spinner mid-resolve.
Pinned by four RawBrowseModalStaleOpenTests facts, each observed red before the guard
and each re-falsified afterwards by reverting only its own guard. The modal calls
StateHasChanged on the open path, which throws without a render handle, so the test
attaches an inert Renderer subclass (file-local BL0006 pragma); [Parameter]s are set
by reflection per the BL0005 convention.
AdminUI suite 1117/1117 green.
The mxaccessgw OnWriteComplete correlation is live (server b948e69, worker
53f69cd — all four unary write kinds), so every write reply's statuses[0] now
carries the real COM commit outcome. Live-gating the cutover on windev found
and fixed three defects:
- StatusCodeMap.FromMxStatus truncated the correlated failure row's MxStatus
detail code to a byte (1007 & 0xFF = 239 -> OPC DA Good band), mapping a
REFUSED write to Good. New invariant: a row that is not IsSuccess() never
maps Good — details > 0xFF (and contradictory Good-band substatuses) route
by category (SECURITY_ERROR -> BadUserAccessDenied, named const covered by
StatusCodeParityTests; COMMUNICATION/CONFIGURATION keep their codes).
- The borrow-from-subscription-registry write optimization could never commit:
a subscription handle is plain-advised, and MXAccess refuses both the
no-login write through it (SECURITY_ERROR detail 1008, live-captured) and
the supervisory advise upgrade (E_INVALIDARG) — one advise per HANDLE, not
per attribute. Removed: writer subscribedHandleSource ctor param + borrow
branch, GalaxyDriver wiring, SubscriptionRegistry.TryResolveItemHandle +
its forward index and test file. The writer always AddItems its own
supervisory-advised handle, which commits fine while the same attribute is
subscribed (live-proven; pinned by the new write-while-subscribed smoke).
- (gateway-side, fixed there) the correlation initially covered WriteSecured
only, while the dominant no-login path emits plain Write.
Live gate, all legs PASSED against windev: happy write Good off a confirmed
status row with galaxy.writes.unconfirmed + advise_failed both FLAT; read-only
refusal surfaces Bad-family (the #5 node-revert input Galaxy could never
produce before); WriteCompletionWaitMilliseconds=0 control -> provisional Good
+ metered unconfirmed (override removed after, restoration re-verified). New
env-gated smokes: Live_correlated_write_statuses_confirm_commit_and_surface_refusal,
Live_write_to_subscribed_tag_commits_via_own_handle (replaces the borrow
smoke), GatewayGalaxyWriteMeterLiveTests (run filtered — meters are
process-wide).
The connect path was the one capability call the resilience pipeline never
saw, so a driver that could not connect contributed zero breaker input and
the /hosts resilience panel read 'All targets healthy' against a dead
endpoint. InitializeAsync now routes through the invoker as
DriverCapability.Initialize, keyed on the driver-instance id.
- No pipeline retry: the actor's RetryConnect timer is the retry loop
(parser forces retryCount overrides to 0, same funnel as write-shaped).
- Breaker threshold 2, not the Read-sized 5: the breaker samples 30 s and
the reconnect timer fires every 10 s, so at most three attempts fit one
window — threshold 5 could never open. Pinned by its own test.
- An open breaker fast-fails into the same InitializeFailed path as a real
connect failure; the 15 s break window costs at most one extra tick.
The best-effort teardown ShutdownAsync stays unwrapped.
- Companion fix: DriverCapabilityInvokerFactory.Create now invalidates the
instance's tracker snapshots with its pipelines — a respawn during an
outage discarded the only breaker whose OnClosed could clear the OPEN
flag, stranding a false 'Breaker OPEN' chip forever.
Live gate PASSED on docker-dev (image symbol-verified per container):
rtu-gate against dead 10.100.0.35:5021 opened its Initialize breaker on
both centrals with the red 'BREAKER OPEN' chip on /hosts; a socat
5021→5020 forwarder recovered it (both breakers closed within 1 s, panel
back to 'All targets healthy'); healthy MAIN-modbus recorded zero breaker
events. Full suite 50/50 green. Evidence:
docs/plans/2026-08-07-adminui-followups-and-live-gates.md §7.
The two items deferment.md §0.1 named as the remaining untracked engineering.
adminui-followups (19 pending / 0 completed) — CLOSED by the overlap audit the plan
itself asks for first, with no code written. All 19 tasks are shipped, superseded by
v3, or moot. WS1's premise had also expired: it assumed driver-embedded tag lists in
DriverConfig are the runtime source of truth, which v3 ended (the artifact populates
RawTags). Its capability shipped in the v3 shape — TagConfigEditorMap + DeviceFormMap,
both verified reachable from RawTagModal/DeviceModal rather than merely present.
Live gates on docker-dev (image rebuilt: the rig was running a build 3 h older than the
last code commit; parity asserted per container plus a positive check that the symbols
under test are in the shipped DLLs):
- R2-03 T11 PASSED — build 0 errors, Runtime.Tests 555/555, RoslynVirtualTagEvaluator 20/20.
- R2-03 T12 PASSED — VT 42 -> 0x80020000 -> 42, staged WITHOUT a deployment (a Modbus
register flipped on the fixture) so the VT child survived; a deploy-driven trip creates
a fresh child with no last value, which is correct-by-design and masked the result twice.
P7 confirmed: two byte-identical redeploys, zero disturbance.
- driver-reconfigure Task 3 PASSED — a faulted driver adopted a corrected config and
connected with the node's startedAt unchanged and RestartCount=0.
- R2-10 T10 still OPEN, but the recorded cause was wrong. Not ConnectionBackoff
throttling (the breaker opens on 2 failures / 30 s): the dead-endpoint failure mode
never enters the resilience pipeline. IDriver.InitializeAsync is not invoker-wrapped
(DriverInstanceActor.cs:707) and ModbusDriver swallows per-tag failures in BOTH
directions, so even the plan's fallback trigger cannot work. Live-proven: a driver
timing out every read against a blackhole reported ConsecutiveFailures=0 while the
panel read "All targets healthy."
Finding from T12: the Bad publish's documented "carrying the last-known value" is
unobservable by any client — subscription and a raw Read service call both return NULL.
The actor is correct; every existing failure assertion checked StatusCode alone, so the
value half was asserted nowhere. Adds
VirtualTagActorTests.The_Bad_result_carries_the_last_known_value, which proves the actor
emits it. The documentation is what overstates.
Re-gates Akka peer-to-peer secret replication on the current three-mesh docker-dev
rig and supersedes the 2026-07-18 six-node gate (lmxopcua#482, c878fbbd) as evidence.
That gate proved convergence to all five peers on the old single fleet-wide mesh;
after mesh Phase 6 (2839deb5) partitioned the rig into three independent 2-node
meshes, six-node convergence would be a FAILURE, not a pass. Phase 6's own leg 6
recorded "PASS (by fallback criterion)" — it inferred pair-locality structurally and
never seeded a secret. This replaces that inference with direct evidence.
All four checks PASS, on master @ 21009874 with one image (OtOpcUa.Host.dll
a434b04f byte-identical on all six nodes), Secrets libs 0.2.3:
1. Pair convergence — seeded on site-a-1 alone, on site-a-2 in <=7s (budget 60s),
byte-identical ciphertext + kek_id + updated_utc, decrypt-verified on the partner.
2. No cross-pair propagation — absent from site-b and central across 8 samples /
240s / 48 announce intervals, and clean in the reverse direction too. Isolation
controls seeded on site-b-1 and central-1 converged within their OWN pairs, so
the absence is isolation rather than a dead replicator.
3. Delete propagates within the pair — issued on site-a-2 (the receiving node, so
this also covers the reverse intra-pair direction); tombstone on site-a-1 in <1s.
4. Clean pair restart (both together) — mesh re-formed in 7s, no resurrection of the
tombstone, live secret survived and decrypted correctly on both nodes.
Zero secret-related warnings or errors fleet-wide across the whole run. Satisfies the
scadaproj#3 precondition for the OtOpcUa production topology (Akka pair-local).
Secrets:Replication:Enabled remains default-false in the product; the rig KEK is dev-only.
Rig config: SqlitePath moved onto each node's existing per-node /app/data volume. The
appsettings default resolves to /app, inside the image's writable container layer, so
the store was destroyed by any `up -d --build` recreate and was unreachable from the
host — seeding or inspecting a node meant stopping the container to docker cp the file
out. Still one private store per node; replication is still what syncs them.
Residuals in the gate doc: the live-push path is not covered (anti-entropy is what a
driver-only node can be gated on), and appsettings.json still carries a stale
"KNOWN NON-FUNCTIONAL against 0.2.0 / do not enable" comment that this gate disproves
— left uncorrected here because this run is config-only.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
A CNC can report duplicate sibling names — the wonder Makino returns two
axes both called AA — which made the Axes/ subtree untrustworthy: the
browser offered two identical folders the commit gate had to refuse, and
the poll loop's name-keyed snapshot cache let the second axis silently
overwrite the first. Spindles and servo loads had the same collision in
their own keying, plus a resolve-to-first-match on the spindle read path.
Scheme: every member of a collision set gets a 1-based occurrence-ordinal
suffix in slot order (AA_1, AA_2); unique names stay bare. Chosen over the
alternatives deliberately:
- Names, not slot numbers, stay the primary identity — the segment map is
re-derived from the live device each session, so a name-keyed path
follows the physical axis across config changes, where a slot-qualified
path would silently rebind on any axis-list shift and rename every axis
on every machine for one pathological CNC.
- The WHOLE collision set is suffixed: bare AA ceases to exist when
ambiguous, so a tag authored against it reads BadNodeIdUnknown rather
than one arbitrary axis's data — the #534/#540 silent-wrongness class.
- Occurrence ordinal, not absolute slot: inserting/removing an unrelated
axis never renames an existing duplicate pair. If the collision-set
membership itself changes, segments change on the next browse and stale
authored segments read Bad, never another axis's data.
One pure helper (FixedTreeNames.Disambiguate, collision-proof against
genuine names by suffix escalation) feeds FocasFixedTreeCache.AxisSegments
/ SpindleSegments, and the emitter, axis snapshot keys, servo-load keys
(cnc_rdsvmeter answers in slot order, so the k-th AA load belongs to the
k-th AA axis) and the spindle read resolution all consume those — they can
no longer disagree. The nameless-spindle S{n} fallback applies before
disambiguation so a fallback name colliding with a real one is handled
like any other duplicate.
Tests are red-first on the pre-fix driver (all five driver-level tests
fail on master's code): browse emits AA_1/AA_2 and no bare AA, each
duplicate reads its own position and servo load, duplicate spindles
resolve per slot, bare AA reads Bad. The wire half runs live against
focas-mock (duplicate names survive the managed wire parse and publish as
suffixed segments); the mock's dynamic store is name-keyed so the
distinct-values half stays with the offline tests. FOCAS 327/327,
integration 11/11, FOCAS.Cli 52, AdminUI 1113, ControlPlane 231.
Closes#548.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
The summary list still described the pre-#520 world: NodeAcl/PermissionTrie
'built and unit-tested but never wired', reads/browse/subscribe/HistoryRead
carrying no authorization at all, and UserName tokens authenticated by LDAP
bind. The body had been updated by #520 stages 5-7 but the summary, the
Anonymous auth-table row, the LDAP bind-flow paragraph (still routing the OPC
UA UserName path through the deleted LdapOpcUaUserAuthenticator), the
'sourced from Security:Ldap:GroupToRole' role-source section, and the
'Dispatch gate (no such gate exists)' section all contradicted it — a reader
who stopped at any of them got the pre-#520 answer.
All now state the flag-gated reality: with Security:Acl:Enabled false (the
default) the only enforcement is the two coarse server-wide role gates and
reads are ungated; with it true, scoped NodeAcl grants decide every
data-plane operation via DataPlaneAuthorizer -> TriePermissionEvaluator.
Data-plane roles derive from cluster-scoped NodeAcl grants
(ApiKeyInterimRoleMapper); GroupToRole and LdapGroupRoleMapping feed AdminUI
login only. The acl-design.md pointer no longer calls the design 'unshipped
work'.
Closes#547.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Ran on master @ 0b194003, post-deletion image d2a290fc5711 (provenance-guarded).
PSK matrix 7/7 incl. unset-key boot refusal; command matrix with Restart/Reconnect
driven from the real /hosts panel (#546 closed the 1A composition carry-forward by
direct evidence); exactly-once ack matrix sealed 6/6; central-kill + site-kill drills
honest (one loud WRN, +2:00 deadlines, boot-replay, no retro-ack); mid-deploy
coordinator kill: recovered coordinator fired the deadline at exactly +2:00, all
landed acks preserved, zero dupes; boundary: 3 established :4053 associations, all
pair-internal, zero site↔central; full-rig restart transport grep = 0 on all six
(one non-mesh observation: central pair reproduced the documented guard-off
cold-start split-brain, healed by single-node rejoin); MeshGrpcBoundaryTests 3/3;
terminal full suite RC=0 direct capture, 50/50 suites, 9,333/0.
I-4 cold-PSK-cache re-carried out of the program (unmeasurable on docker-dev).
No new defects.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
#542 filed FleetStatusBroadcaster as blind to site nodes since the Phase 6 per-cluster mesh
split: it derives "the fleet" from central's own Akka gossip ring, which after the split is
just central's own pair. Scoping the cross-mesh rewrite found the deeper truth — the whole
chain is dead end to end, so this deletes it rather than reviving it.
- Zero consumers. FleetStatusBroadcaster published FleetStatusChanged on the DPS
"fleet-status" topic; FleetStatusSignalRBridge forwarded it to FleetStatusHub
("/hubs/fleet-status"); nothing anywhere connected. The AdminUI bans self-connecting
HubConnections, no browser JS subscribes, and /fleet (Fleet.razor) reads
NodeDeploymentState from the ConfigDb plus IClusterRoleInfo on a 5 s poll instead.
- Its second input never existed. DriverHostStatusHeartbeat had no production sender —
only the broadcaster's own test told it — so CurrentRevision was always null and the
30 s staleness -> Degraded flip could never fire.
- Post-Phase-6 DPS is pair-local, so even the central-only view it did produce was an
accident of topology rather than a design.
The fleet visibility that actually works is unchanged and already cross-mesh-correct:
/fleet over NodeDeploymentState, and the four Phase 5 per-node telemetry dials.
Precedent for deleting inert machinery rather than activating it: #522 (driver stability
tiers), #507 (discovered-node injection), #525 (GetMemoryFootprint).
Deleted: FleetStatusBroadcaster (+ its nested DriverHostStatusHeartbeat), its admin
singleton registration and FleetStatusBroadcasterKey, FleetStatusSignalRBridge and its
key, FleetStatusHub and its route mapping, and the FleetStatusChanged / FleetNodeStatus /
FleetNodeHealth messages. Both Fleet/ directories survive — ControlPlane keeps
ClusterNodeAddressReconciler, Commons keeps GetDiagnostics. Docs swept where they claimed
a fleet-status channel exists (Telemetry.md's Phase 5 non-migrated channels,
Configuration.md, CLAUDE.md); completed-program plan docs under docs/plans and docs/v2 are
left as the historical record they are.
Build 0 errors, 868 warnings — byte-identical to master's rebuild baseline.
ControlPlane.Tests 231/231, AdminUI.Tests 1109/1109.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
OtOpcUa runs five internal gRPC planes between its own processes, all of
them cleartext HTTP/2 (h2c) with a shared-bearer interceptor: the live
telemetry stream, config-serve artifact fetch, LocalDb pair sync, mesh
commands and mesh deploy acks. The bearer therefore crosses the wire in
the clear on every one of them. That was true and undocumented; the two
mesh planes are command surfaces, so it is worth saying out loud rather
than leaving to be rediscovered per plane.
docs/security.md gains "Internal gRPC planes (h2c) — accepted cleartext
posture" under Transport Security (whose existing content is entirely
about the OPC UA endpoint). It carries a five-row table — service name,
listener config key, credential config key, interceptor — the honest
read-vs-write breakdown per plane, and the decision itself.
Decision (issue #545, option 1): risk ACCEPTED 2026-08-06, once, for all
five planes together, on the trusted-link assumption every plane already
makes (VPN / dedicated WAN). Per-cluster PSKs already bound blast radius
on the two command planes — a captured SITE-A key cannot drive SITE-B —
and production keys arrive as ${secret:...} references that must never be
reused across clusters. Recorded future work: a UseTls/scheme option,
which if built must cover all five planes together, since a mixed posture
is harder to reason about than either uniform answer.
docs/Configuration.md's MeshTransport section gains one cross-reference
paragraph pointing at the new subsection.
Docs only — no code, no behaviour change. Every service name, config key,
interceptor name and shared-listener claim was verified against the
source; ports 4056/4055 are phrased as the docker-dev rig's values of
Telemetry:GrpcListenPort / ConfigServe:GrpcListenPort, not as hardcoded
ports.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Task 3.1: 7 ordered deletion steps + MeshPaths review fix, spec+code reviews closed.
Task 3.2: docs sweep + accuracy review, scadaproj index pushed on its main.
DoD full-suite: run 3 exit 0 (50 suites, 9326 passed); runs 1-2 each hit one
pre-existing unrelated timing flake (3 distinct tests, all untouched by the
branch, 9/9 green in isolation). Also records the exit-code-capture trap found
in run 1 and its retroactive caveat on Phase 2's check-8 wording.
Merge to master is user-owned — STOPPED at the Phase 3 DoD.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Review of the ClusterClient-deletion docs sweep (406b1ae5) flagged two
pre-existing staleness items in the swept files, unrelated to ClusterClient
but caught while touching the same passages:
- docs/Telemetry.md: the Phase 5 telemetry section's Status line still said
"code-complete ... live gate has not yet run" — the gate PASSED 2026-07-23
(merge 35552a96), already recorded in CLAUDE.md. Updated to match.
- docs/v2/config-db-schema.md: the AkkaPort/GrpcPort note asserted in the
present tense that "central shares a gossip ring with every node," which
Phase 6 (2026-07-24) ended — central now runs one 2-node Akka mesh per
application Cluster and reaches a site pair only over gRPC. Reworded the
premise to past tense so it no longer contradicts the ClusterClient-deleted
parenthetical sitting right after it.
docs-only; dotnet build ZB.MOM.WW.OtOpcUa.slnx stays green (0 errors).
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Task 3.2 of the ClusterClient->gRPC migration. Sweeps the docs that still
described the Akka ClusterClient transport as live, following its Phase 3
deletion (this branch): MeshTransport:Mode is now Dps (pair-local DPS) or
Grpc (cross-mesh unary calls) only, ClusterClient fails validation loudly,
CentralContactPoints is deleted (not superseded), and the split-topology
validator requires exactly Grpc rather than ClusterClient.
- docs/Configuration.md: rewrite the MeshTransport section — table,
dark-switch/PSK-seeding prose, listener sharing (:4056 driver-role,
:4055 admin-role), remove the "shipped dark, cutover pending" framing.
- docs/Redundancy.md: replace the Dps/ClusterClient transport table with
Dps/Grpc, fix the Phase 6 CentralCommunicationActor + validator bullets.
- docs/Telemetry.md: fix stale ClusterClient cross-refs, add a note that
the driver-role telemetry listener also hosts NodeCommandService.
- docs/v2/config-db-schema.md: one-line "(retired: ...)" annotation on the
historical AkkaPort/GrpcPort note — left as a historical record otherwise.
- CLAUDE.md: rewrite "## Mesh command transport (MeshTransport)" for the
post-deletion world (no ClusterClient/receptionist, per-node gRPC dial,
fire-and-forget semantics unchanged) and fix two Phase 6 bullets.
deferment.md has no mesh-transport mentions; left untouched.
docs-only change; `dotnet build ZB.MOM.WW.OtOpcUa.slnx` stays green (0
errors) after the edit.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
Joseph Doherty