08e33fc7ecc47247c179be623c80c745c8a17ed8
94 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
c51c59987b |
fix(galaxy): publish an unrepresentable value as BadTypeMismatch, not a good null
Reverses the conservative call from the previous commit at the user's direction, and they were right: "surface it on a counter first" left the wire lying. An MXAccess event can carry Good quality while its payload is a kind MxValueDecoder has no mapping for, so the old behaviour published null-under-Good — which a client cannot tell apart from a genuine null reading. A value the gateway DID send would be consumed as a value it did not, with the only evidence in a log nobody is tailing. Bad data is the honest wire representation of lost data. BadTypeMismatch is an existing StatusCodeMap constant, deliberately reused rather than written inline: StatusCodeParityTests finds status constants by reflecting over Good/Uncertain/Bad-prefixed uint fields and checking them against the pinned SDK, so an inline literal would be invisible to that guard (Gitea #497, where 16 wrong constants had accumulated across 6 drivers exactly this way). Logging is now once per DISTINCT value kind rather than once per pump. The set is bounded by the MxValue oneof enum so a broken producer still cannot flood, but a second unrepresentable kind appearing later gets its own line — warn-once-per-pump would have silently swallowed it, which is the same class of bug as the null this commit fixes. The test's fixture matters more than its assertion: the undecodable event carries Quality = 192 (OPC DA Good), so a decoder that gave up silently would publish Good and fail. Paired with a positive control — a decodable value at the same quality must still publish Good — so it cannot be satisfied by a pump that marks everything Bad. Falsified by reverting the status to Good: the test reds, and only that test. Verified: 331 Galaxy tests green, Core.Abstractions 298 (the parity guard), solution builds 0 errors. Docs corrected in the two places that recorded the old decision as deliberate. |
||
|
Joseph Doherty
|
4769044f88 |
feat(galaxy): handle the replay-gap sentinel and sparse-array values
Closes the two follow-ups left open by the MxGateway 0.2.0 bump. Reading the
gateway's own contract first changed what each of them is, and one of my earlier
claims was simply wrong -- both corrections are recorded below.
Replay gap
----------
A sentinel MxEvent carries family = UNSPECIFIED, so it fell through EventPump's
family switch into the default drop: we silently ignored a documented MUST ("you
missed events -- discard local state and re-snapshot"). It is now intercepted and
answered by re-issuing SubscribeBulk for every tracked subscription, since
MXAccess delivers an initial value per item on subscribe -- re-subscribing IS the
re-snapshot. It does not restart the event pump (the stream carrying the sentinel
is healthy, and the handler runs on that stream's own read loop, so disposing the
pump from there would deadlock on itself), it does not block the read loop, and
re-entry is gated so a burst collapses to one pass instead of a subscribe storm
per sentinel.
Handled on the READ loop, not in the dispatch loop, because everything past that
point is subject to newest-dropped channel pressure -- and "you missed events" is
the one signal whose loss is unrecoverable. Pinned by a test that fills the
channel and still expects the sentinel.
⚠️ This cannot fire in the shipped configuration, and I did not pretend otherwise
or paper over it by enabling resume. The gateway emits the sentinel only for a
stream opened with after_worker_sequence > 0; GatewayGalaxySubscriber passes 0,
which the gateway treats as "fresh stream, not a resume -- plain attach, no
replay, no sentinel". Enabling resume is a larger and genuinely delicate change:
the sequence is only meaningful within ONE gateway session, our reconnect
recreates the session, and replaying a remembered sequence against a fresh
session sets the server's live watermark ABOVE every event it is about to send --
silencing the stream instead of recovering it. That deserves its own change with
a live gate, which I cannot run from here.
Sparse arrays -- correcting the last commit message
---------------------------------------------------
I wrote that the bump "makes handling it possible" on the decode path. That is
wrong: MxSparseArray is documented write-only, the gateway expands it before
forwarding, and "the worker never receives or produces it". So the decode path
is unreachable by contract and the write path has no use for it either -- it is a
wire compaction for a WHOLE-array write, not a partial update (unmentioned
indices are RESET, deliberately, because the gateway cannot read-modify-write
without racing the provider). It is therefore the wrong primitive for an OPC UA
IndexRange write, which expects untouched elements preserved -- and WriteRequest
carries no IndexRange in the first place.
It is decoded anyway, as contract defence: the alternative if a future gateway
ever emits one is a silent null published under Good quality. Fill semantics
mirror the gateway's own SparseArrayExpander exactly (epoch for Time, empty
string, Integer widening to long[] when any element is int64) so a value that
round-trips through a write reads back as the same CLR array.
Also: the decoder is now total by contract, since it runs inside the dispatch
loop where an escaping exception kills dispatch for the life of the pump. A
malformed payload degrades instead of throwing.
A real defect found in the same code
------------------------------------
A stream the GATEWAY completes -- a closed or evicted session -- ends the RPC
without an error, so nothing threw, the supervisor was never notified, and the
pump exited silently. GetHealth() kept reporting Healthy, which also means the
#527 comms-loss watchdog could never trip, because it requires a non-Healthy
state. The driver went permanently deaf with no signal anywhere. This one is
live, unlike the two above. Now reported as a fault; the falsification control is
the existing CleanShutdown_DoesNotInvokeOnStreamFault, since our own disposal
also ends the stream and must NOT be reported.
Value kinds we cannot represent are now counted and warned once rather than
published as an indistinguishable null. The status is deliberately NOT downgraded
to Bad: nothing can reach that branch today, and changing what an OPC UA client
sees for a live shape is not a change to make without a gate. The counter exists
so the next person has evidence instead of a guess.
Verified: 330 Galaxy tests green (12 new), plus Galaxy.Browser 21, Runtime 558,
Core 351; full solution builds 0 errors with the warning census unchanged. Every
new test was falsified first -- disabling the gap interception reds 3 of 4 (the
throwing-callback test passes vacuously when the callback never runs, by
construction), disabling sparse decode plus the representability signal reds all
7 decoder tests, and both were restored and re-verified green.
Docs: the Galaxy.Performance invariant was stated as received = dispatched +
dropped + in-flight, which this breaks -- a sentinel is received but never enters
the channel -- so it now reads + replay_gaps. Galaxy.md gains the three
operational notes; DriverLifecycle.md records that "reaches a non-Healthy state"
is an obligation a driver can silently fail to meet, which is the general shape
of the clean-end bug.
|
||
|
Joseph Doherty
|
b8acabfa5f |
merge: SQL driver health review findings resolved (Driver.Sql-004/-005/-006/-008/-010)
# Conflicts: # docs/drivers/Sql.md |
||
|
Joseph Doherty
|
ead8a97511 |
merge: MTConnect review findings resolved (Driver.MTConnect-004 write-back authoring, -005/-006/-007)
# Conflicts: # code-reviews/Driver.MTConnect/findings.md |
||
|
Joseph Doherty
|
38dee82e29 | merge: SQL contracts review findings resolved (Driver.Sql.Contracts-001 scalar keyValue, -002 fleet-wide numeric-enum rejection) | ||
|
Joseph Doherty
|
298ad04fba |
feat(adminui): author MTConnect write-back on the driver form (Driver.MTConnect-004)
IWritable shipped behind write.enabled and docs/drivers/MTConnect.md told the operator to set it, but MTConnectDriverForm — the only editor DriverFormMaps maps for this type, with no raw-JSON escape hatch and no driver-instance API — exposed no write section, so the documented enablement path was a hand edit of DriverInstance.DriverConfig in the config DB. - Write-back section: switch + form text stating it is observation injection, that the Agent's AllowPut is an independent second gate, and that it does not by itself make a node writable over OPC UA (Tag.AccessLevel still composes CurrentWrite). - Write added to the mirrored MTConnectConfigDto and to the form model, written in BOTH states — an omitted block would leave a previously-authored true standing, so the switch could never turn writing off again. - The false 'MTConnect is read-only' header comment is gone; the doc names the new section. No bUnit here, so the model/serialization seam is unit-tested both sides: the AdminUI test pins the emitted bytes, and MTConnectFactoryTests parses that same document and asserts MTConnectDriverOptions.Write.Enabled. Live page verification remains a rig step. |
||
|
Joseph Doherty
|
678aa3aa53 |
fix(sql): read keyValue with the same scalar read as whereValue (Driver.Sql.Contracts-001)
The two value-bearing fields were read asymmetrically and nothing said why. whereValue went through ReadScalar (string/number/boolean — the design doc's own wide-row example authors it as the number 7); keyValue went through the string-only ReadString, so a numerically-authored key refused the tag outright. Nothing downstream justified the split: both are BOUND as DbParameters and both are matched back as text — SqlPollReader.IndexByKey stringifies the key cell precisely so an authored key can face an INTEGER key column. The refusal was also mis-explained. Inspect branched on ReadString(root, "keyValue") is null, so a numerically-authored key drew "Sql KeyValue tag has no 'keyValue'" — telling the operator the field is missing when it is right there in the blob, the #549 shape of an authoring surface disagreeing with the runtime about WHY. keyValue now uses ReadScalar; Inspect mirrors it with HasScalar and says what a usable key is; SqlTagConfigModel loads it with the shared TagConfigJson.GetScalarText (which also replaces SqlRowSelectorModel's private copy), so the editor is not left stricter than the parser. An object/array/null key is still refused. docs/drivers/Sql.md updated. RED first: 4 failing in Driver.Sql.Tests. |
||
|
Joseph Doherty
|
9d4760fd2f | docs(driver.sql): topByTimestamp is only defined when the ordering column is unique (Driver.Sql-008) | ||
|
Joseph Doherty
|
300abde4bb | merge: SQL browser/probe review findings resolved (Driver.Sql-009, Driver.Sql.Browser-001/-002/-003) | ||
|
Joseph Doherty
|
a8cfc886a3 |
fix(sql-browser): rank the row-selector candidate so a date-only column is a last resort (Driver.Sql.Browser-001)
PreferredTimestampColumn nominated the first column mapping to DriverDataType .DateTime, and that set includes date-only types (SQL Server / PostgreSQL 'date'). A table whose first date/time column is a 'date' committed rowSelector.topByTimestamp = <date>, making the wide-row read ORDER BY <date> DESC LIMIT 1 - which ties for every row written that day, so 'the newest row' became storage order among the ties. Time-bearing candidates now outrank date-only ones; a date-only column is still stated when it is all the table has, so such a table stays bindable. Oracle's DATE carries a time of day and is exempt - pinned by a control dialect that differs from the SQLite one only in the Provider it reports. |
||
|
Joseph Doherty
|
1c67d019e5 |
docs(mtconnect): describe the fixed browse-commit coercion type (#560)
The comments and docs that described Driver.MTConnect-003 as live at HEAD would
have been wrong the moment
|
||
|
Joseph Doherty
|
d68009106b | merge: Mqtt runtime + Transport code review — deaf-reconnect subscription reset (High), teardown-before-reinit session leak (High), timeout normalization, dead CA-pin warning; 12 findings | ||
|
Joseph Doherty
|
978eeda753 | merge: MTConnect family code review — sample-pump hot-loop backoff, probe deviceName path, stale-premise docs; 9 findings (High browse-commit String coercion recorded cross-module) | ||
|
Joseph Doherty
|
2b4b3fcbbf |
fix(mqtt): session-rebind subscriptions, re-init teardown, timeout clamp
Findings from the per-module review of Driver.Mqtt at
|
||
|
Joseph Doherty
|
69ad376f0d |
review(Driver.Sql): findings + fix the missing default-schema fallback, the rowSelector doc, and a dead package ref
Driver.Sql-001 (Medium): SqlCatalogLoader.ReadDefaultSchemaAsync documented a fallback to the sole visible schema when the server reports no default schema and did not implement it, so a login with no DEFAULT_SCHEMA (or a Postgres search_path that resolves to nothing) rejected EVERY unqualified authored table at the catalog gate — a whole-driver BadNodeIdUnknown outage against a healthy database. Fixed; SqlCatalogDefaultSchemaTests written red first, with a control proving the ambiguous multi-schema case still refuses to guess. Driver.Sql-003 (Medium): docs/drivers/Sql.md spelled the WideRow row selector at the top level, but the parser reads those keys only inside a nested rowSelector — a tag authored from the doc is refused and reads BadNodeIdUnknown (the #552 class, via the documentation). Also removed the documented-but-nonexistent pollIntervalMs per-tag field. Driver.Sql-007 (Low): dropped the unused Microsoft.Data.SqlClient PackageReference left behind by the .Dialect extraction (#526). Open: 002 (no IN-list chunking vs the 2100/1000 provider bind caps), 004, 005, 006, 008, 009, 010. |
||
|
Joseph Doherty
|
6c63bf8fab |
fix(mtconnect): back off a /sample re-baseline that resolved nothing; scope Test Connect to the configured device
Driver.MTConnect-001 (Medium): StreamVerdict.Reopen reset the reconnect ladder
unconditionally. The InvalidDataException catch that routes an Agent OUT_OF_RANGE
to a /current re-baseline also catches an unparseable chunk and an over-sized
multipart part - neither of which a re-baseline can resolve - so the pump reopened
and failed with no delay at any point: two HTTP requests and one Warning per
iteration, forever. The Reopen arm now uses the same discriminator the transient
arm does (did THIS connection deliver a chunk?), so a buffer roll behind a working
stream still reopens instantly while a re-baseline that resolved nothing advances
BackoffFor.
Driver.MTConnect-002 (Medium): MTConnectDriverProbe requested {AgentUri}/probe
while MTConnectAgentClient requests {AgentUri}/{DeviceName}/probe, so a mistyped or
since-renamed deviceName showed GREEN on Test Connect and then Faulted the driver
at InitializeAsync (an unknown device path answers MTConnectError under HTTP 200).
The probe's minimal DTO now reads deviceName - the rule is 'which keys compose the
URL', not 'which keys are cheap to read'.
Driver.MTConnect-008 (Low): corrected two stale claims that a since-changed
RawBrowseCommitMapper invalidated (see Driver.MTConnect-003, which remains open and
belongs in AdminUI), and made DiscoverAsync log the options snapshot it
deliberately read once instead of the field.
Tests: 538 pass (534 base + 4 new, each red before its fix); AdminUI.Tests 1466 pass.
|
||
|
Joseph Doherty
|
7498d39e5b |
fix(sql): browse-commit states a topByTimestamp row selector (#552)
`RawBrowseCommitMapper.BuildSqlTagConfig` emitted a WideRow blob carrying only table + columnName. A wide-row tag must also say WHICH row it reads, so both `SqlEquipmentTagParser` (the runtime authority) and `SqlTagConfigModel.Validate` (the authoring gate) refused it — a browse-committed Sql tag was never readable, first silently as BadNodeIdUnknown and, since #551 put the gate on the import, loudly at commit. `SqlBrowseSession` now states a fourth address field: the table's preferred date/time-typed column, computed from the columns its attributes call ALREADY read (no extra catalog round-trip). Candidacy is decided by `ISqlDialect.MapColumnType` reporting `DriverDataType.DateTime` — never by column name, which would order a wide-row read by a string column that merely looks like a timestamp. Multiple candidates tie-break on FIRST IN CATALOG ORDER (every dialect's column SQL orders by ordinal position), so the pick is stable across calls and servers; it is a visible, editable default the typed editor round-trips. No candidate (a table with no date/time column) is an operator-actionable refusal, not an invented selector: the only remaining selector is a whereColumn/whereValue pair whose value is data, not schema, so `DescribeUncommittableLeaf` refuses that leaf by name, names the table, and points at the tag editor. The session and the mapper now agree via `SqlBrowseAddressFields` constants rather than two independently spelled string literals — a disagreement there is silent, and is exactly the shape this defect had. `RawTagConfigGateTests`' Sql leg flips from "records the gap" to "browse-commit output passes the gate", with the no-candidate case pinned as its own test. |
||
|
Joseph Doherty
|
a486a009dc | docs(mtconnect): state the driverDataType-over-dataType precedence and the editor's normalisation (#550) | ||
|
Joseph Doherty
|
290117660d |
docs: truth sweep — reconcile stale deferment/archreview/plan records with shipped state
- deferment.md: node ACLs no longer authored-but-inert (#520 shipped through stage 7, opt-in Security:Acl:Enabled); failure-visibility trio mostly closed (01/S-1 + 03/S4 fixed, 06/S-1 partial); bulkhead + WriteIdempotentAttribute closed by R2-02; Sql Postgres+Oracle dialects shipped ( |
||
|
Joseph Doherty
|
8b10d32728 |
fix(focas): disambiguate duplicate sibling axis/spindle names in the fixed tree (#548)
A CNC can report duplicate sibling names — the wonder Makino returns two
axes both called AA — which made the Axes/ subtree untrustworthy: the
browser offered two identical folders the commit gate had to refuse, and
the poll loop's name-keyed snapshot cache let the second axis silently
overwrite the first. Spindles and servo loads had the same collision in
their own keying, plus a resolve-to-first-match on the spindle read path.
Scheme: every member of a collision set gets a 1-based occurrence-ordinal
suffix in slot order (AA_1, AA_2); unique names stay bare. Chosen over the
alternatives deliberately:
- Names, not slot numbers, stay the primary identity — the segment map is
re-derived from the live device each session, so a name-keyed path
follows the physical axis across config changes, where a slot-qualified
path would silently rebind on any axis-list shift and rename every axis
on every machine for one pathological CNC.
- The WHOLE collision set is suffixed: bare AA ceases to exist when
ambiguous, so a tag authored against it reads BadNodeIdUnknown rather
than one arbitrary axis's data — the #534/#540 silent-wrongness class.
- Occurrence ordinal, not absolute slot: inserting/removing an unrelated
axis never renames an existing duplicate pair. If the collision-set
membership itself changes, segments change on the next browse and stale
authored segments read Bad, never another axis's data.
One pure helper (FixedTreeNames.Disambiguate, collision-proof against
genuine names by suffix escalation) feeds FocasFixedTreeCache.AxisSegments
/ SpindleSegments, and the emitter, axis snapshot keys, servo-load keys
(cnc_rdsvmeter answers in slot order, so the k-th AA load belongs to the
k-th AA axis) and the spindle read resolution all consume those — they can
no longer disagree. The nameless-spindle S{n} fallback applies before
disambiguation so a fallback name colliding with a real one is handled
like any other duplicate.
Tests are red-first on the pre-fix driver (all five driver-level tests
fail on master's code): browse emits AA_1/AA_2 and no bare AA, each
duplicate reads its own position and servo load, duplicate spindles
resolve per slot, bare AA reads Bad. The wire half runs live against
focas-mock (duplicate names survive the managed wire parse and publish as
suffixed segments); the mock's dynamic store is name-keyed so the
distinct-values half stays with the offline tests. FOCAS 327/327,
integration 11/11, FOCAS.Cli 52, AdminUI 1113, ControlPlane 231.
Closes #548.
Claude-Session: https://claude.ai/code/session_014WNM4vjoVksyyBraTXSZE1
|
||
|
Joseph Doherty
|
dd900f970a |
fix(hosts): a driver that never built a tag table reports NO ANSWER, not zero skips
Found on the docker-dev live gate, not in tests. Two drivers that faulted during initialise (rtu-gate, opcua1) each rendered "0 tags" on /hosts against two authored tags apiece, because a ledger that had never committed returned SkippedTagReport.Empty. A zero report is a positive claim -- "this driver skipped nothing" -- and making it on behalf of a driver that never got as far as looking is exactly the quiet inaccuracy #541 exists to remove. GetSkippedTags() and SkippedTagLedger.Snapshot() now return null until the first committed build, so the column renders the same em-dash it already used for a driver that cannot report at all. Both are 'no answer'; neither is 'nothing skipped'. Knock-on: a FOCAS driver whose only init threw now reports null rather than a zero report, which is strictly more honest than the assertion the test originally made. |
||
|
Joseph Doherty
|
39db75083d |
feat(drivers): every driver reports the authored tags it skipped (Gitea #541)
A driver builds its RawPath -> definition table at init and tolerates a
per-tag miss: an unmappable TagConfig, an unparseable address, a
capability-matrix rejection. That tolerance is right -- one bad tag must not
take down the other few thousand. What was missing is any statement of the
fact afterwards. A skip is not an error and happens before any read, so the
driver stays Healthy, errors/5 min stays 0, Last error stays empty, and Last
read advances live off the tags that did map. On the wonder Makino in #540
that reading held while focas-z-34184 ignored 20 of its 22 tags.
New optional capability ISkippedTagReporter, backed by a shared
SkippedTagLedger so the reset/commit/cap rules are one implementation rather
than one per driver. The report -- authored count, exact skipped count, and a
capped sample of RawPath + reason -- rides DriverHealthChanged to a new Tags
column on /hosts, over the Phase 5 telemetry stream. No table: a driver-only
node has no ConfigDb to write to, the lesson from #521.
The issue named eight drivers. It is twelve -- every driver in
DriverTypeNames.All has an authored-tag skip path. Calculation (no scriptId),
MTConnect (no readable DataItem id), Galaxy (no attributeRef) and OpcUaClient
(no nodeId) were unlisted, and OpcUaClient's was the quietest of the twelve:
NamespaceMap drops such an entry with a bare `continue` and not even a log
line. SkippedTagReporterFleetTests pins the invariant reflectively over every
IDriver in src/Drivers/, so a thirteenth cannot re-introduce the silence.
Sql drops a tag at TWO points and its report is the union of both: a blob
that does not map, AND an identifier the 8.1 catalog gate rejects against the
live catalog. The second is what schema drift produces -- a renamed column, a
dropped view -- months after a deployment sealed green, and reporting only
the first would have that driver claim "0 skipped" while polling half its
tags, which is worse than reporting nothing because the column would then be
actively reassuring.
Deliberately additive to driver state, never a change of it. Latching
non-Healthy for a config problem would satisfy both halves of the #527
comms-loss conjunction -- 90 s without a successful read AND non-Healthy --
and a driver whose tags were ALL skipped has no successful reads by
construction, so it would reconnect-loop forever against a healthy device.
Three details that are load-bearing rather than incidental:
- The skip digest is part of the publish fingerprint. A skipped tag leaves
every other fingerprint component identical to a clean driver's, so without
it PublishHealthSnapshot's dedup swallows exactly the publish carrying the
signal -- the trap that already bit the rediscovery and host-status
channels. The digest covers the sample, not just the counts: a redeploy
that fixes one tag and breaks another leaves both counts unchanged.
- The count is exact; only the sample is capped, and the tooltip says so when
it is showing fewer than it counted. A truncated list that looked complete
would have an operator fix twenty tags and believe the job done.
- The ledger stages and commits, so an init that throws part-way leaves the
previous report standing. A partial snapshot would read as "we now skip
fewer tags", the opposite of the truth.
The proto field needs no companion presence flag, unlike host_statuses: a
singular message field carries real presence in proto3, and a has_ flag would
collide with protoc's generated HasSkippedTags.
Verified red-first three ways: dropping SkippedTagDigest from the fingerprint
fails the two dedup tests, removing the seam from one driver fails the fleet
guard, and dropping the catalog-gate leg fails the Sql union test. Full
solution builds with zero new warnings against
|
||
|
Joseph Doherty
|
b61fe38823 |
fix(focas): browse-committed fixed-tree tags are readable (Gitea #540)
Authoring FOCAS tags through /raw -> Browse device... produced tags the driver
could never read. Every committed tag read BadNodeIdUnknown forever, and the
failure was silent end to end: the AdminUI accepted it, the deploy gate sealed
green, and the driver reported Healthy with 0 errors while ignoring the tags.
Net effect: the whole fixed-tree surface could not be published through any v3
authoring path. Found cutting wonder-app-vd03 over to v3 against a real Makino
Pro 5 / Fanuc 30i, where 0 of 20 committed tags read while two hand-authored
MACRO: tags on the same device read Good.
A captured browse leaf's id IS its DriverAttributeInfo.FullName, and the fixed
tree emitted "{deviceHost}/{path}" there -- an internal cache key, not an
address. Both authoring paths wrote it verbatim into TagConfig.address, and
FocasAddress.TryParse accepts no such form, so the driver skipped the tags at
init.
Fixed at the emitter rather than by stripping the host at commit, which fixes
the browse-commit flow and the manual picker's tree-select together and leaves
one canonical address form:
- FocasAddress gains a fourth kind, FIXED:<path>, and moves to Contracts so the
AdminUI and ControlPlane can parse without referencing the runtime driver
(#526). A FIXED: read is served from the fixed-tree snapshot the poll loop
already maintains -- no area read, no connection. Writes refuse: nothing in
that tree is a writable location on the CNC.
- The address carries no device. A tag's device comes from its Device row, and
a second routing channel for one fact is what created this defect.
- An authored FIXED: tag arms the fixed-tree poll loop by itself. Enabled
defaults false and the browser force-enables it only for a browse, so the
operator would otherwise browse, commit, deploy, and land on a driver whose
loop never starts -- every tag Bad with a correct address.
- Fixed-tree values are now dated from the last answered poll and go
BadCommunicationError once it is stale (8 cadences, floored at 30s). The poll
loop swallows a failed tick by design, so a dead CNC would otherwise serve its
last snapshot as Good indefinitely -- harmless while nothing could bind to the
tree, not harmless now.
Guardrails, so this class cannot recur silently:
- FocasTagConfigModel.Validate() runs the real parser instead of a non-empty
check, and FocasTagDefinitionFactory.Inspect() warns at the deploy gate, which
covers CSV/API config that never touches the modal.
- The manual picker offered axis/spindle/program/status "parameter groups" --
none of which is a FOCAS addressing space -- and its own default value,
axis:5, was unusable. It now composes PMC / PARAM: / MACRO: through
FocasAddress, so what it emits is parseable by construction.
Tests assert addresses ROUND-TRIP through the parser rather than matching an
expected string: the previous FocasAddressBuilderTests pinned "axis:5" exactly,
which is how a green suite coexisted with a picker nobody could use. Verified
red-first against the old emitter. FOCAS 308, AdminUI 1095, ControlPlane 169,
Core 581 -- all pass, 0 warnings.
The remaining half of #540 -- a driver reporting Healthy/0 errors while
skipping tags at init -- is fleet-wide (eight drivers do this) and is filed
separately rather than patched per-driver.
|
||
|
Joseph Doherty
|
d54f3c8342 | merge: Sql driver Postgres + Oracle providers (managed Npgsql/ODP.NET, dialect-owned binding, live-gated fixtures) | ||
|
Joseph Doherty
|
203000a80a | docs: MTConnect write-back recorded; stale MQTT write-through/browse-gap records reconciled | ||
|
Joseph Doherty
|
08c1728a98 | feat(sql-adminui): Postgres + Oracle in the provider dropdown; docs | ||
|
Joseph Doherty
|
b815446940 |
refactor(drivers): delete GetMemoryFootprint / FlushOptionalCachesAsync (#525)
Both `IDriver` members had no production call site. Their only consumer was `MemoryTracking`, deleted with the driver-tier recycle machinery in #522, and `GetMemoryFootprint`'s doc-comment claimed "polled every 30s by Core" — which had not been true since v3, even before #522 removed the poller. Owner decision on #525 was (a) delete rather than (b) build the process-wide watchdog that would consume them. The driver-side implementations are in git history and were mostly reusable; what was always missing is the trigger, and a member nobody calls is how the misleading doc-comment survived this long. Removed from `IDriver`, all 12 driver implementations, and every test stub — plus the support code that existed only to feed them, which is the part a member-only deletion would have left behind as dead weight: - MTConnect's three per-entry byte-weight constants, its cached `ProbeModelDataItemCount`, and `CountDataItems` — a full device-tree walk run on every cache fill to populate a number with no remaining reader. - OpcUaClient's `_discoveredNodeCount` field and the discovery-pass write to it. - MQTT's `ApproxBytesPerAuthoredTag`. Two behavioural notes, neither a change in what the software does: - `AbCipTemplateCache` now has no production reset path. Its only caller was AB CIP's flush, which itself had no caller — so nothing changes except how visible the fact is. Recorded on the type, along with the correction that its comment's claim about a device `ReinitializeAsync` also flushing it was never true. A UDT redefined on a live controller keeps its stale shape until the driver instance is rebuilt. - MTConnect's `/probe` cache is still dropped, by `InvalidateProbeModel()` on an Agent restart. That is now `internal` rather than `private` so the tests which used the flush purely to empty the cache drive the real path instead — they now cover a case that actually occurs in production. Several unrelated doc comments cited `FlushOptionalCachesAsync` as the canonical CAS example and are repointed to it. Tests: the ones that existed only to assert on the removed members are deleted (3 OpcUaClient, 3 Galaxy, 2 MQTT, 1 MTConnect, 1 AbCip). Worth noting one was vacuous — `FlushOptionalCachesAsync_resets_footprint_counter` asserted a footprint of 0 before the flush and 0 after with no discovery in between, so it would have passed against a flush that did nothing at all. Full suite green except 7 failures verified identical on master before and after (4 AbLegacy integration with the compose profile down, 3 Host.IntegrationTests), checked by name rather than assumed. Docs corrected where they described this as live: `DriverLifecycle.md` (which carried the #525 placeholder), `drivers/AbCip.md`, `drivers/AbLegacy.md`, and `v2/driver-stability.md` — whose "polls this every 30 s" + per-driver `DriverConfig` memory budget were never built as written and are now banner-ed as design intent rather than description. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
c74ccc55b5 |
feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):
Template → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
DataSet → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
PropertySet → node metadata, never tags — it describes a metric rather than being one
Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.
**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.
**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).
**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.
The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.
Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.
Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.
Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
7f2108dcf2 |
feat(mqtt): Sparkplug primary-host STATE publishing, Primary-gated (#511)
`ActAsPrimaryHost` was accepted by config and by the AdminUI form and did nothing but
log a warning. It now delivers the whole three-part contract or none of it: a retained
`ONLINE` on `spBv1.0/STATE/{hostId}`, a matching retained `OFFLINE` armed as the MQTT
Last Will, and an explicit `OFFLINE` on a clean stand-down.
**The claim is a dedicated connection, not a message.** A Will can only be registered in
the CONNECT packet — it cannot be added to a live session, changed, or withdrawn. So a
claim that is acquired and released at runtime cannot ride the long-lived ingest session.
`SparkplugPrimaryHost` opens its own connection (client id `{clientId}-spbhost`, forced
clean-session) when the node gains Primary and closes it when the node loses Primary; the
connection's lifetime *is* the claim, so every leg follows from its lifecycle. Ingest is
untouched by a role change — no dropped subscriptions, no lost birth cache, no data gap at
the Secondary->Primary transition.
**Primary-gated** (owner decision, recorded on the issue). Both pair nodes subscribe
independently, so an ungated claim means both claim the same host id — and the sharp
failure is not the two fighting over the retained STATE, it is that the SECONDARY's death
fires its Will and publishes OFFLINE while the Primary is healthy, stopping every edge node
in the group. Registering the Will only on whichever node is Primary at connect time fails
the other way: a node that later gains Primary would publish ONLINE with no Will behind it.
New seam: `IDriverPrimaryRoleSource` (`Core.Abstractions`) carrying the *data-plane* verdict
(`PrimaryGatePolicy.ShouldServiceAsPrimary`), not the alarm-drain verdict — an unknown role
with a driver peer present must deny. Registered unconditionally by `AddOtOpcUaRuntime`,
deliberately NOT alongside `IRedundancyRoleView`, which `AddAlarmHistorian` registers only
when that section is enabled: piggybacking would leave a pair with no alarm historian
falling back to "claim unconditionally". Polled on a 1s timer rather than evented — a
process singleton raising events at per-instance drivers outlives every driver it holds.
Accepted trade: a brief edge-visible OFFLINE window during failover. That is a genuine
state transition, not a glitch to hide.
Live-gated 5/5 against the Mosquitto fixture, including the Will leg — `AbandonForTests`
drops the socket with no DISCONNECT, because a clean disconnect makes the broker discard
the Will by protocol rule, so a test driven through `StopAsync` would observe the
stand-down OFFLINE and pass against an implementation that never armed a Will. Falsified:
removing the Will turns that leg and only that leg red.
Also corrects two stale claims found en route: the AdminUI form still said MQTT tags are
read-only (#508 shipped), and the driver doc still said nothing consumes `IRediscoverable`.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
72be19225d |
feat(mqtt): write-through (IWritable) — Plain publish + Sparkplug NCMD/DCMD (#508)
MqttDriver implemented no IWritable, so every MQTT node materialized read-only
regardless of what the operator authored. It now publishes.
Plain -> the tag's writeTopic; UTF-8 text (Raw/Scalar) or a bare JSON
literal (Json)
Sparkplug -> DCMD for a device metric, NCMD for an edge-node metric, carrying
the metric BY NAME with an explicit Datatype and no seq
Both at QoS 0, retain:false. Retain is deliberately not configurable — a retained
command is replayed by the broker on every future subscribe, including the
device's own reconnect, so one stale write would be re-applied forever.
writeTopic is a new, separate TagConfig key, and its ABSENCE is meaningful: the
write is refused BadNotWritable and the node is discovered ViewOnly. The issue's
"do not skip" warning drove the design — the one outcome worse than read-only
nodes is a node that looks writable, returns Good and publishes nothing. Reusing
the read topic would be worse still: it would echo back and read as a confirmed
device value. Browse-time writability is computed by the SAME encoder the write
path uses, so a node cannot advertise writability it would then refuse.
Good means published, not applied — MQTT has no application-level write ack, so
this is Galaxy's optimistic-Good posture and inherits its property: a failed
DEVICE write can never be surfaced. A failed PUBLISH is reported Bad; the
optimism covers what the device did with the command, never whether it was sent.
WriteIdempotent stays default-off.
The encoder is pure and lives in .Contracts, so both wire shapes are testable
without a broker. Unrepresentable values and types with no Sparkplug encoding are
refused rather than coerced — a Reference must not silently become a String on
the wire, and "not-a-number" must not become 0 on a live plant.
LIVE GATE PASSED against the real Mosquitto fixture on 10.100.0.35: 21/21
integration tests, including 6 new ones that observe the published message with
an INDEPENDENT broker subscriber — a fake asserting "publish was called" could
not tell the feature working from the feature publishing to the wrong topic.
The gate earned its keep: the NCMD test initially failed because NCMD is a SHARED
channel — the driver publishes its own rebirth request (Node Control/Rebirth) on
the very topic an edge-node metric write uses, and it arrived first. Correct
driver behaviour, wrong test; the watcher now matches on metric name. Documented,
because anything consuming that topic needs to know. DCMD has no contention.
Offline: 24 encoder + 9 driver tests. The four new status constants are NAMED so
StatusCodeParityTests can see them (#497/#519) — it went 258 -> 262 and validated
each against the pinned SDK.
Also corrects the Mqtt.md "Known gaps" table: prunes the six rows closed in this
round, and fixes the #507 row, which still described rediscovery as inert after
that issue was closed as superseded and its injection path deleted.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
cbb882293b |
refactor(drivers): delete the inert driver-tier recycle machinery (#522)
The tier system was documented, operator-authorable, and inert. Deleted rather than activated,
because its premise is gone rather than merely unused.
"Tier C" meant a driver running out-of-process behind an IDriverSupervisor that could restart its
Host without tearing down the OPC UA session. No such process exists anywhere: Galaxy reaches
MXAccess over gRPC to the external mxaccessgw sidecar (PR 7.2 retired the in-process
Galaxy.Host/Proxy/Shared projects) and FOCAS has run in-process since its managed wire client
landed 2026-04-24. Consistently, IDriverSupervisor had ZERO implementations and there was nothing
for one to implement against.
The issue understated the inertness. It says the Tier-C-only protections never engaged, implying
the Tier A/B parts did. They did not: nothing constructs MemoryTracking, MemoryRecycle or
ScheduledRecycleScheduler outside their own unit tests, so the whole Core/Stability recycle layer
was dead — meaning option (a), "pass real tiers", was never a flag flip. It would have meant
writing the wiring that never existed AND arming it.
Deleted: MemoryTracking, MemoryRecycle, ScheduledRecycleScheduler, IDriverSupervisor, the
vestigial DriverTypeRegistry (referenced only by its own tests), and the RecycleIntervalSeconds
knob — which the AdminUI let an operator author and the parser validated while it configured
nothing.
DriverTier itself SURVIVES and is load-bearing: DriverResilienceOptions.GetTierDefaults supplies
the real per-capability timeout/retry/breaker policies via DriverFactoryRegistry.GetTier and
DriverCapabilityInvokerFactory. Only the isolation-and-recycle layer above it is gone.
Deliberately NOT deleted:
- WedgeDetector came along in the same directory and is equally dead in production, but it is
tier-agnostic and is not recycle machinery — it only shares the folder. Restored rather than
swept up in a decision that was not about it.
- IDriver.GetMemoryFootprint() and FlushOptionalCachesAsync() lose their only consumer here.
Removing them touches all 12 drivers and every test stub, so they are documented as
consumerless and filed as #525 instead of buried in this diff.
Compatibility: a deployed ResilienceConfig blob still carrying "recycleIntervalSeconds" parses
cleanly (unknown keys are ignored — guarded by a new test, because a blob that suddenly failed to
parse would fall back to tier defaults and silently discard the operator's real overrides), and
the AdminUI's preserve-unknown-keys bag keeps the key rather than rewriting stored config on an
unrelated edit.
The 01/U-6 knob-inertness guard carried an explicit carve-out admitting RecycleIntervalSeconds was
dormant and out of scope; that carve-out is now gone, so the expected set is literally what the
test's name claims.
Note: Host.IntegrationTests has 2 failures (DriverProbeRegistrationTests.is_idempotent,
PrimaryGateFailoverTests) — verified pre-existing by reproducing both on clean master
|
||
|
Joseph Doherty
|
f3cd685da5 |
docs(drivers): write the two missing driver guides (Sql, Calculation)
Closes the §7.2 priority-3 item. Sql was a shipped, merged driver with NO
user-facing documentation at all -- only design and plan files, which describe
what was intended rather than what an operator needs to author one.
Sql.md covers: SQL Server as the only constructed provider (the other four
SqlProvider names are reserved, and authoring one fails rather than falling
back), the KeyValue / WideRow tag models with worked TagConfig, Query being
rejected end-to-end, the config table with real defaults read from
SqlDriverOptions, connectionStringRef never carrying credentials into an
artifact, per-source query grouping and its case-sensitive group keys, and the
value-bound-vs-identifier-validated split that makes it injection-safe.
Calculation.md covers: what a pseudo-driver is and why it occupies a driver slot,
a Calculation-vs-VirtualTags decision table (the two are easy to confuse), the
literal-only ctx.GetTag dependency extraction and the runtime-built-path trap,
the two deploy gates, the all-deps-arrived publish gate, and the Good->Bad-once
error semantics.
Two stale claims found by checking against source rather than inheriting them:
- The driver README's Calculation row still carried G-1 ("no typed config form,
RunTimeout unauthorable"), fixed on 2026-07-27 by CalculationDriverForm.
- docs/Raw.md's "one auto-created default Engine device" NEVER SHIPPED. I had
copied it into the new doc before checking; "Engine" has zero occurrences in
src/ and nothing special-cases Calculation at device creation. Corrected in
both files. The config key was also wrong in my first draft: runTimeoutMs, an
int in milliseconds, not a TimeSpan string -- and it is ignored on reinit
because the evaluator's timeout is constructor-fixed.
Full solution: 49 suites pass; the 2 failures are the known pre-existing
environment ones (AbLegacy.IntegrationTests docker fixture, Host.IntegrationTests).
OpcUaClient.IntegrationTests, which was silently failing, is now green.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
adce27e7fa |
refactor(drivers): delete the dead discovered-node injection path (§8.2, #507)
#507 was filed as "injection inert in v3, re-migrate onto the raw subtree". Reading it, the retained code is not revivable: it resolves equipment from EquipmentNode.DriverInstanceId UNION EquipmentTags, and BOTH are structurally empty in v3 (AddressSpaceComposer always constructs EquipmentNode with a null DriverInstanceId; DeploymentArtifact hard-codes an empty EquipmentTags set). Removing the guard would have changed a log line and injected nothing. So it is deleted rather than fixed, and #507 closes as superseded by /raw browse-commit. Removed: HandleDiscoveredNodes, PartitionDiscoveredByDeviceHost, ShouldWarnPartition, PlansRoutingEqual, ApplyDiscoveredPlansForDriver, _discoveredByDriver, the redeploy re-inject tail, DiscoveredNodeMapper, DiscoveredInjection, AddressSpaceApplier.MaterialiseDiscoveredNodes, OpcUaPublishActor.MaterialiseDiscoveredNodes, and the Runtime-local copies of CapturingAddressSpaceBuilder/DiscoveredNode. The connect-time discovery loop goes too (StartDiscovery, RediscoverTick, HandleRediscoverAsync, DiscoveredNodesReady, TriggerRediscovery). With injection gone it had no consumer, and leaving it would either dead-letter or keep browsing real devices up to ~15x per connect to drop the result. ITagDiscovery itself stays — the /raw browse picker drives it through Commons/Browsing/DiscoveryDriverBrowser, which never went through the actor, so the picker is unaffected. deferment.md §4.4 called the 18 DiscoveryInjectionDormantV3 tests "Real — blocked on #507". They are not: they assert an equipment-rooted graft (EquipmentRootNodeId == "EQ-1") that v3 cannot produce, so they could never have unskipped as written. Deleted along with DiscoveredNodeMapperTests, the CapturingAddressSpaceBuilder tests, and the MaterialiseDiscoveredNodes tests in AddressSpaceApplierTests/OpcUaPublishActorTests. Runtime.Tests skipped: 31 -> 13. IHostConnectivityProbe is deliberately NOT resolved here. GetHostStatuses() has no production call site and nothing writes a DriverHostStatus row, but the table was re-created in the v3 initial migration, so deleting on inference would be wrong. Split to Gitea #521 with both options costed. CLAUDE.md, docs/drivers/{Galaxy,TwinCAT,MTConnect}.md updated — they described the seam as dead. Build clean; Runtime.Tests 476 passed / 13 skipped; OpcUaServer.Tests 362 passed / 4 skipped. |
||
|
Joseph Doherty
|
e08855fb9d |
docs: source-verified deferment register + correct 17 drifted docs
Adds deferment.md — a source-verified inventory of new-driver status, all 17
open issues, in-code deferrals, open live gates and plan bookkeeping. Every
claim was checked against src/ and git, not against documentation.
Headline finding: three subsystems are authored, persisted and shipped but
never executed —
* Node ACLs: IPermissionEvaluator/TriePermissionEvaluator have zero
production consumers and OtOpcUaNodeManager never references them, yet
ClusterAcls.razor authors NodeAcl rows and ConfigComposer.cs:51 ships them
in every artifact. An authored deny rule has no effect.
* IRediscoverable / IHostConnectivityProbe raise into the void (#518/#507);
nothing ever writes a DriverHostStatus row.
* DriverTypeRegistry is vestigial, so no factory passes a tier and every
driver runs Tier A with the Tier-C protections dormant.
Also records the Calculation driver as picker-visible but unauthorable
(DriverConfigModal has no case) — the "registered but unauthorable" class
recurring after the Sql picker defect — and notes that no parity test guards
DriverConfigModal/DeviceModal, which is why it survived review.
Documentation corrections (source-verified):
* ReadWriteOperations.md claimed "a denied read never hits the driver" via
four types that do not exist in src/. Bannered + struck through; a security
review reading that page would have concluded a per-node ACL gate exists.
* CLAUDE.md: the Change Detection sentence was false (DriverHost consumes
nothing); the mesh Phase 4 and Phase 5 live gates and the auto-down 1-vs-1
gate had all PASSED; the ScriptedAlarmState table was already dropped.
* driver-expansion tracking: Modbus RTU and SQL poll are merged, not
pending — its command table would have made someone rebuild two shipped
drivers in a fresh worktree.
* drivers/README.md: dead DriverTypeRegistry paragraph, retired
SystemPlatform namespace kind, missing Sql + Calculation rows, missing
Modbus RTU-over-TCP transport.
* TwinCAT.md/Galaxy.md promised an address-space rebuild that never happens.
* Historian.md gained the #491 unproven-value-capture pointer.
* IncrementalSync.md and AddressSpace.md bannered as wholesale v2-era (the
rewrite is recorded as still owed, not done here); four v2 status docs
bannered as historical; three wrong-name-for-a-live-type fixes.
Left deliberately untouched: the secrets NoOpSecretReplicator line, which
makes a security claim and needs the real behaviour identified rather than a
rename.
|
||
|
Joseph Doherty
|
71ccccef7c |
docs(mtconnect): P1 driver guide + tracking-doc update, defer write-back (Task 22)
- docs/drivers/MTConnect.md: getting-started guide for the P1 Agent MVP — config keys read off MTConnectDriverOptions/ConfigDto, capability surface, RawPath->dataItemId coercion precedence, UNAVAILABLE->BadNoCommunication + the rest of the quality-code table, CONDITION-as-String, browse-via-the- universal-browser, the fixture recipe (links the existing IntegrationTests Docker README instead of duplicating it), and a Known limitations section covering the two pre-existing fleet-wide gaps this build surfaced (IRediscoverable/IHostConnectivityProbe have no server consumer; no driver form blocks Save on validation). Records write-back (MTConnect Interfaces), P1.5 (native alarms, TIME_SERIES arrays, EVENT->enum), and P2 (SHDR ingest) as deferred per design doc SS3.6/SS9. Documents where the build diverged from the design: TrakHound MTConnect.NET was dropped entirely (no XML formatter in the pinned packages, no injectable HTTP handler) in favor of a hand-rolled System.Xml.Linq client, namespace-agnostic on LocalName. - docs/drivers/README.md: adds MTConnect to the ground-truth driver table, the per-driver doc list, and the fixture coverage-map list. - docs/plans/2026-07-24-driver-expansion-tracking.md: marks the MTConnect Wave-2 row Done (22/23 tasks; Task 21 live gate tracked separately in the plan file, not touched here), corrects mtconnect/cppagent -> mtconnect/agent in the fixture note, and links the new driver guide. Deferred-cleanup decision (Task 1 review follow-up): removed GenerateDocumentationFile/NoWarn CS1591 from Driver.MTConnect.Contracts's csproj to match all eight sibling .Contracts projects, none of which carry it. Verified both the Contracts project alone and the full solution still build clean (0 warnings, 0 errors) with the flags gone — the existing XML doc comments don't depend on GenerateDocumentationFile to compile. |
||
|
Joseph Doherty
|
a0b5095b3b |
docs(drivers): un-stale the MQTT README rows + track every known gap as an issue
The cross-driver README still described MQTT as pre-P2 — "Sparkplug B is P2
(`mode: SparkplugB` is a placeholder)" and "Sparkplug B has no fixture yet (P2)".
Both were false as of the P2 merge (
|
||
|
Joseph Doherty
|
d5bd4226ee |
docs(mqtt): Sparkplug P2 live-verified; MQTT driver complete
Task 26 — the P2 milestone gate. Ran the live /run verification on an isolated
docker-dev rig (project otopcua-mqtt, ports 9210/4850-4851/14350, image built
from this branch) against the real Mosquitto TLS+auth broker and the C#
Sparkplug edge-node simulator on 10.100.0.35, and recorded the result.
The gate found three defects. Two are the same defect class the P1 gate found
twice — a hand-maintained AdminUI surface left behind by a driver-side feature —
and the third is a pre-existing cross-driver bug that only a Sparkplug flow
could surface.
1. CRITICAL, FIXED — MqttDriverForm still shipped its P1 Sparkplug PLACEHOLDER.
Switching Mode to SparkplugB rendered a "not available yet" notice and NO
Group ID field, so with Sparkplug ingest fully shipped there was still no
way to author a Sparkplug driver from the AdminUI at all. Sparkplug.GroupId
is the driver's entire subscription filter (spBv1.0/{GroupId}/#): blank ⇒
connected, Healthy, ingesting nothing. Now authors all five Sparkplug keys,
MERGES over the existing sub-object rather than replacing it (so a key a
newer driver adds inside it survives an older AdminUI), leaves it untouched
in Plain mode, and validates the group id as the topic segment it is.
Pinned by 8 MqttDriverFormModelTests cases, verified falsifiable — 8 RED
with the fix stubbed out.
2. PRE-EXISTING + CROSS-DRIVER, FIXED — every node label in the shared
DriverBrowseTree was an <a href="#">. In a Blazor Web App, blazor.web.js's
enhanced-navigation click interceptor resolves a bare "#" against
<base href="/">, so clicking ANY browse-tree node label navigated the whole
AdminUI to "/", tore down the circuit, and destroyed the hosting modal —
losing the browse session AND the tag selection. @onclick:preventDefault
does not help: it suppresses the browser's default action, not Blazor's own
interceptor. Labels are now <button type="button">. Dates to the component's
introduction (2026-05-28) and affects EVERY driver's picker; it surfaced
only now because choosing a rebirth scope is the first flow that requires
clicking a label rather than the ▶ toggle (always a button, always fine).
3. FIXED (mitigation) — the browse tree rendered exactly once, at open, and
never again, while OpenAsync returns as soon as the SUBSCRIBE is granted.
On Sparkplug that means it is almost always empty forever, because births
are never retained. Added a Refresh button that re-reads the root against
the SAME session (nothing reconnects, nothing observed is lost, the tag
selection is kept, only the armed rebirth scope is cleared).
Live gate result (all against the real broker + simulator, group OtOpcUaSim):
- driver authored end-to-end through the fixed MqttDriverForm; blob is
Mode:SparkplugB + Sparkplug.GroupId:OtOpcUaSim with enums as names
- browse: BROWSER OPEN chip, Sparkplug-only rebirth panel, tree renders
Group → EdgeNode → [Device] → Metric with the device folder Filler1 and
node-level metric leaves SIDE BY SIDE, and "Node Control/Rebirth" as ONE
leaf (the metric-name-is-not-a-topic-segment rule holding)
- rebirth: group scope names the group + the 32-node cap, Cancel published
NOTHING (simulator log unchanged); edge-node scope → "1 command published"
and the simulator logged "rebirth NCMD #1 received; republishing metadata";
group scope → "2 commands published", both edge nodes re-birthed; a device
and a metric both resolve UP to "edge node EdgeA"; the panel does NOT render
for a Plain MQTT device
- browse-commit wrote bindable tuples with no topic/address key —
{"groupId","edgeNodeId","metricName","deviceId"} for the device metric and
deviceId ABSENT (not blank) for the node-level one, datatypes inherited from
the birth (Int64 / Float)
- deployed (Sealed), and both nodes served live changing values through OPC UA
at the 2 s cadence — Good 0x00000000, no BadNodeIdUnknown
- death→STALE: stopping the simulator drove both nodes to 0x80000000 with an
empty value ~2 s later, and restarting it recovered them to Good on the new
birth (FillCount back to its birth-declared 1000, then counting)
- rediscovery fired exactly TWICE (once per authored scope: OtOpcUaSim/EdgeA
7 metrics, OtOpcUaSim/EdgeA/Filler1 3 metrics) and was then gated across
many subsequent births/rebirths — the anti-storm change gate working, and
EdgeB correctly filtered out as an unauthored scope
- tag editor: opens in SparkplugB (mode inference on reopen) with all four
descriptor fields populated; blank group → "A Sparkplug group ID is
required."; "Plant/1" rejected; "Node Control/Rebirth" ACCEPTED and saved
with the slash whole; dataType override persists and its key is ABSENT again
after selecting "(from birth certificate)"
Two things are NOT verified, and are recorded rather than claimed:
- Rediscovery is INERT in v3 and this is a platform gap, not an MQTT one:
nothing subscribes to IRediscoverable.OnRediscoveryNeeded and
DriverHostActor.HandleDiscoveredNodes hard-returns. A DBIRTH introducing a
metric does NOT change the OPC UA tree; redeploy. Verified from the driver's
log line only, exactly as far as it can be verified.
- A metric name containing "/" cannot be BROWSE-COMMITTED: the derived raw tag
Name is a RawPath segment and may not contain "/", so the spec-mandatory
"Node Control/Rebirth" is refused ("Row 3: Name must not contain '/'"). The
refusal is loud and all-or-nothing, never a silently mis-bound tag, and
Manual entry is the working path. Fixing it needs a name-sanitisation policy
with a collision answer, so it was recorded rather than guessed at.
Also left open + documented: an empty browse tree cannot arm a rebirth (the
scope comes from a tree click, and the session side already accepts a bare
{group}/{edgeNode} — only a UI path is missing); _canRebirth is captured at
browse-open; the tag editor injects the Plain-only payloadFormat key into a
Sparkplug blob (cosmetic — the factory routes on the tuple).
Suites: offline 1528 passed / 0 failed (581 Driver.Mqtt + 809 AdminUI + 138
Core.Abstractions); live 15/15 against the broker + simulator.
Docs: docs/drivers/Mqtt.md brought fully up to date for P2 (Sparkplug config
sub-object, both tag shapes, the ingest state machine, rediscovery, browse +
rebirth + Refresh, a Known-gaps table); Mqtt-Test-Fixture.md gains the simulator
and drops its "Sparkplug has no fixture" claims; infra/README.md §3 gains the
simulator row; CLAUDE.md gains the simulator endpoint facts; the tracking doc
marks MQTT/Sparkplug COMPLETE. docker-dev/docker-compose.mqtt.yml is the
isolated-rig overlay the gate ran on.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
8d9155682d |
fix(mqtt): browse-commit writes a bindable address + a Request-rebirth affordance
Two gaps the Task 23/24 review found, both blocking Task 26's live gate.
Gap 1 — browse-commit produced a silently dead MQTT tag. RawBrowseCommitMapper
had no `Mqtt` case, so a committed leaf fell through to the generic
`{"address": …}` key. Neither MqttTagDefinitionFactory entry point reads
`address`: the tag deployed clean and reported BadNodeIdUnknown forever, with
no signal at commit time. Predates Task 23 (Plain was affected too), but Task 23
built the Sparkplug metric tree precisely so an operator could browse and commit
a binding.
The address is a DESCRIPTOR, not a reference string, and it cannot be recovered
from the browse node id: `{group}/{node}[/{device}]::{metric}` where a metric
name legitimately contains `/` (`Node Control/Rebirth`) — the ambiguity
MetricSeparator's remarks already name. So the session STATES it, via a new
`AttributeInfo.AddressFields` seam, and the mapper reads it. Which keys are
emitted is also how the mapper learns Plain vs Sparkplug — the driver type
reaching it is just `Mqtt`, and the mode lives on the driver config. Key names
are single-sourced in the new `MqttTagConfigKeys` (producer, mapper, factory),
with the literals pinned by a test so a symmetric rename cannot silently unbind
already-persisted blobs. A leaf with no stated address is refused at commit in
words rather than committed dead.
Gap 2 — RequestRebirthAsync had no UI. Task 23 shipped it backend-only; Task 26
step 1 assumes the button, and Task 25's runbook notes the picker tree stays
empty until a birth lands, so it is the only way to fill it on demand. Added to
the /raw browse modal: scope = the clicked tree node (device/metric resolve up to
their edge node, group fans out and is refused whole past 32), an explicit
two-click confirm naming the resolved scope and its consequence, the outcome or
the refusal shown rather than swallowed. Offered only for a Sparkplug session
(new `IRebirthCapableBrowseSession.RebirthAvailable` — one session class serves
both modes, so a type test alone would draw a button that can only throw) and
only to a DriverOperator; the server-side gate remains the boundary.
Tests: MQTT 545 → 581, AdminUI 781 → 800. The round-trip tests feed the emitted
blob to the REAL factory and were falsified by mutating the emitted key name.
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
92ba120964 |
feat(mqtt): MqttDriverForm + MqttDeviceForm — the driver/device config forms P1 omitted
The P1 live gate had to insert MQTT driver config directly via SQL: DriverConfigModal and DeviceModal both fell through to "No typed config form for driver type Mqtt" with no raw-JSON fallback, so broker host/port/TLS/credentials were unauthorable from the AdminUI. Task 12 built the *tag* editor; these are the driver + device surfaces. MqttDriverForm authors the whole MqttDriverOptions connection surface (host/port/clientId, TLS + CA pin, credentials, protocol version/clean-session/keep-alive, connect timeout, reconnect backoffs, mode, maxPayloadBytes, and the Plain sub-object). The Sparkplug sub-object stays P2 — a marked placeholder mirroring MqttTagConfigEditor's stub, with any existing sparkplug keys preserved untouched. The connection is authored on the DRIVER, not the device — unlike Modbus/S7/OpcUaClient and contrary to what docs/drivers/Mqtt.md said. DriverDeviceConfigMerger merges a device's keys up only when the driver has exactly ONE device, so a device-authored broker connection vanishes silently the moment a second device is added. MqttDeviceForm is therefore informational (the GalaxyDeviceForm shape) and round-trips DeviceConfig verbatim; it flags legacy connection keys left on a device by a pre-form deployment, because those still win via merge-up. Serialization goes through the ONE shared MqttJson.Options instance (Task 9's decision), never a fifth per-form copy — pinned by an assertion that an ordinal-only reader CANNOT bind the emitted blob, and by extending the fleet-wide DriverPageJsonConverterTests guard to resolve a form's serializer from an explicit external-instance registry when it has no _jsonOpts field. Dropping the converter from MqttJson.Options reddens 3 tests and leaves the symmetric round-trip green — the Task 9 lesson, reproduced. RawTags is stripped from the emitted blob (the deploy artifact owns it) and unknown top-level keys survive a load->save. Validation mirrors the driver's own [Range] bounds inline, and ToOptions() additionally clamps, so an ignored error still cannot persist a connectTimeoutSeconds:0 driver-brick. A non-blank clientId raises the P1 live-gate warning inline (fixed ids make redundant pair nodes evict each other while both report Healthy). AdminUI 761/761 green (was 730), TWAE-clean; Driver.Mqtt 266/266 green. Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
07a4a5ff5e |
docs(mqtt): P1 plain-MQTT milestone live-verified; endpoint recorded
Task 14 — the P1 milestone gate. Ran the live /run verification on a docker-dev
rig against the real Mosquitto TLS+auth fixture, and recorded the result.
The gate did its job: it found MQTT was unauthorable in production.
RawDriverTypeDialog's driver-type list is a hand-maintained array that nobody
added MQTT to, so the /raw "New driver" picker never offered it — with every
other layer (contracts, driver, browser, factory, host registration, typed tag
editor) complete and 266 green unit tests. Nothing tied that array to
DriverTypeNames and AdminUI has no bUnit, so no offline test could see it.
Fixed, plus RawDriverTypeDialogParityTests: a reflection parity guard that
fails for ANY DriverTypeNames entry missing from the picker. Verified
falsifiable — RED with "…cannot be authored from the /raw New-driver picker,
so they are unreachable in production: Mqtt" before the one-line fix.
A second gap is left OPEN and documented, not fixed (no task ever covered it):
MqttDriverForm / MqttDeviceForm do not exist, and DriverConfigModal/DeviceModal
have no raw-JSON fallback — so an operator can create an MQTT driver but cannot
author its broker endpoint or credentials. P1 is not operator-complete until
those two razor forms land. The gate authored config via SQL to get past it.
Live gate result (isolated 2-node MAIN stack, image built from this branch,
fixture at 10.100.0.35:8883, AllowUntrustedServerCertificate rather than
pinning the CA into the rig):
- typed MQTT tag editor renders and round-trips; Json shows the JSON-path
field, Raw/Scalar hide it; wildcard topic a/+/c blocked at save; data-type
list offers Float64 and no Double; qos/retainSeed absent on "(driver
default)" and qos:2 written as a number; isHistorized/historianTagname
survive a topic-only edit; SparkplugB renders the P2 placeholder and
switching back keeps Plain values; jsonPath is guidance not a gate in all
three shapes ($ seeded only for a brand-new blank tag, blank stays absent,
clearing saves fine)
- deployed, and both editor-written blobs resolved and served changing live
values through OPC UA (22.8 / 1629, StatusCode Good) — no BadNodeIdUnknown
- the bespoke #-observation browser drove a real broker session and rendered
the fixture's actual topic tree (line1/{counter,state,temperature},
noise/chatter, retained/seed); a Modbus device's picker still correctly
reports "Browsing unavailable"
Also found live and documented (not a code change): both nodes of a redundant
pair run the driver, so a FIXED clientId makes them evict each other forever —
the broker logs "already connected, closing old connection" and both nodes
reconnect every ~2s while still reporting Healthy. Unset (the default) is
correct; confirmed 0 reconnects/60s on both nodes after removing it.
Suites: offline 1134 passed / 0 failed (266 Driver.Mqtt + 730 AdminUI incl. the
3 new guards + 138 Core.Abstractions); live 7/7 against the fixture.
Docs: new docs/drivers/Mqtt.md + Mqtt-Test-Fixture.md (the per-driver + fixture
convention every other driver follows), MQTT rows in docs/drivers/README.md,
TestConnectProbes.md, root README.md, and infra/README.md §3.
CLAUDE.md drift corrected while adding the MQTT endpoints:
- the "every fixture carries project: lmxopcua" claim was false — only the
MQTT fixture does; the filter returns one stack, not the fleet
- lmxopcua-fix.ps1 is Windows-VM-only; on macOS drive the host over ssh/rsync
(and the docker-dev rig itself runs locally under OrbStack)
Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW
|
||
|
Joseph Doherty
|
2e4ccf7fe9 |
chore(localdb): phase-2 DoD sweep
Build: 0 errors solution-wide, and 0 warnings from every project this branch
touches. The ~816 solution-wide warnings are pre-existing xUnit1051 /
OTOPCUA0001 / CS86xx in untouched driver + client test projects.
Tests: full solution run compared against a full run on a detached worktree at
the pre-branch baseline
|
||
|
Joseph Doherty
|
7fc3b19961 | docs: R2-11 TagConfig consolidation + strictness rollout decisions | ||
|
Joseph Doherty
|
2124f21ab6 |
docs(historian-gateway): document gateway backend, config keys, EnsureTags hook, known gates; retire Wonderware from docs
v2-ci / build (pull_request) Failing after 38s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (pull_request) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (pull_request) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (pull_request) Has been skipped
HistorianGateway is now the sole historian backend (read + alarm SendEvent + continuous WriteLiveValues). Document the final state and retire the Wonderware sidecar from the docs/config/labels: - CLAUDE.md: rewrite the Historian section — ServerHistorian / ContinuousHistorization / AlarmHistorian config keys, the IHistorianProvisioning EnsureTags hook, the GatewayAlarmHistorianWriter SendEvent path + ReadEvents dependency on gateway RuntimeDb:EventReadsEnabled=true, gateway-side prerequisites (RuntimeDb flags + historian:read/write/tags:write scopes), migration note, and two KNOWN-LIMITATION callouts (live-validation gate + empty historized-ref-set recorder follow-on). - appsettings.json: fix the stale ServerHistorian block (Host/Port/SharedSecret/ ServerCertThumbprint -> Endpoint/ApiKey/UseTls/AllowUntrustedServerCertificate/ CaCertificatePath/CallTimeout, keep MaxTieClusterOverfetch); add a disabled ContinuousHistorization block; prune the orphaned Wonderware keys from AlarmHistorian (keep the SQLite knobs). ApiKey env-supplied via ServerHistorian__ApiKey (commented; valid strict JSON via _comment keys). - README.md + docs (Historian.md, AlarmHistorian.md, Configuration.md, ServiceHosting.md, DriverLifecycle.md, drivers/README.md, Uns.md, VirtualTags.md, AlarmTracking.md, Client.UI.md, README.md, TestConnectProbes.md): retire the Wonderware historian backend from current-backend descriptions; fix the stale ServerHistorian/AlarmHistorian config tables (now gateway shape); convert drivers/Historian.Wonderware.md to a retired stub pointing at the gateway. - Source/UI labels (descriptive text only, no behavior change): OtOpcUaServerHostedService.cs, HistoryPaging.cs, OtOpcUaSdkServer.cs, HistorianAdapterActor.cs, VirtualTagModal.razor, ScriptedAlarmModal.razor, AlarmsHistorian.razor now name the HistorianGateway backend. Build clean (0 errors); AdminUI.Tests green (514 passed). Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii |
||
|
Joseph Doherty
|
5f0a52864c |
feat(focas): real FANUC 30i/31i-B PDU-v3 support (live-validated on a 31i-B)
First real FOCAS hardware contact (Makino Pro 5 / 31i-B @ 10.201.31.5). A full
v3 data-PDU capture corrected the initial diagnosis: the v3 block envelope is
identical to v1, so only specific payload structs / request math / one client
robustness gap were wrong — not "framing rewrites".
Fixes (all re-validated live through the fixed driver):
- version gate: accept inbound PDU {1,3}, keep emitting v1 (FocasWireProtocol).
- cnc_rdtimer: 8-byte {minute,msec} payload is little-endian (ParseTimer) — the
only decode with an in-range msec field.
- pmc_rdpmcrng: request range widened to the data-type byte width
(end = start + width - 1) so a Word/Long isn't truncated to 0 values
(was spurious BadOutOfRange); decode extracted to ParsePmcRange.
- cnc_rdsvmeter: per-axis LOADELM is 8 bytes (not 12) and names come from the
0x0089 block — ParseServoMeters fixes the misaligned 655360 garbage. Also the
"hang" was NetworkStream.ReadAsync not aborting a stalled socket: ReadExactlyAsync
now disposes the stream on cancellation so a stalled peer can't wedge a poll loop.
- cnc_rddynamic2: contract guard rejecting axis < 1 (driver poll already 1-based).
- FocasDriverProbe: run a real wire session (initiate + cnc_statinfo) instead of
degrading to Ok=true "TCP reachability only" when FWLIB is absent — a bare TCP
listener no longer reports HEALTHY.
cnc_rdparam (0x000e) is unsupported on this control — EW_FUNC across 14
request-framing variants x 4 known-present params; needs a reference FWLIB trace
or is restricted. Deferred (deployed config uses macros, not parameters).
Tests: FOCAS suite 234 green (+16), full solution builds 0 errors. Raw v3
captures checked in under tests/.../Fixtures/v3/. Capture tools under scripts/focas/.
Docs: docs/plans/2026-06-25-focas-pdu-v3-{30i-b-support,implementation-plan}.md,
docs/drivers/FOCAS.md, docs/v2/focas-version-matrix.md,
docs/deployments/wonder-app-vd03-makino-z-34184.md.
|
||
|
Joseph Doherty
|
c8ab8fc348 |
test(abcip): Emulate-tier nested-UDT live-gate smoke + docs (backlog #6)
Add AbCipEmulateNestedUdtTests (skip-gated, AB_SERVER_PROFILE=emulate) to close the live-gate gap for nested-struct UDT discovery via CIP Template Object (class 0x6C) threaded by commits 3d8ce4e8/d203f31c. Compiles + skips cleanly against ab_server (no CIP Template Object service). Update docs/drivers/AbCip.md nested-struct section to record the shipped decode path, the Emulate-only live-gate, and offline unit coverage. |
||
|
Joseph Doherty
|
163f57b6bc | docs: within-timestamp tie-cluster paging + AbCip/TwinCAT UDT member discovery | ||
|
Joseph Doherty
|
56ccaa797c | docs(drivers): B/I/O + BOOL-within-word bit writes (RMW) | ||
|
Joseph Doherty
|
b7dfb5aff2 |
docs(s7): wide-type/Timer-Counter support — CLI help + driver-specs + S7 driver doc
Drop the "not yet implemented / BadNotSupported" stale note from all three S7 CLI --type option descriptions (ReadCommand, WriteCommand, SubscribeCommand) and replace with accurate help listing the full supported type set, byte-anchored addressing for wide types, and Timer/Counter read-only status. docs/v2/driver-specs.md §5: add Supported Data Types table, Byte-Anchored Addressing table (DBB/MB/IB/QB + examples), Timer/Counter read section with the Counter-BCD known-limitation, and Deferrals list. docs/drivers/S7.md: expand Data types to a full table, add "Wide types & Timer/Counter" section (byte-anchored addressing, Timer/Counter read-only, Counter BCD known-limitation, deferrals), update Address forms table and 1-D array Deferrals note. |
||
|
Joseph Doherty
|
05c7e86f0c | docs(phase4c): cross-driver 1-D array support + isArray/arrayLength keys + coverage matrix | ||
|
Joseph Doherty
|
08a6551360 |
docs(phase4b): Modbus driver-type canonical + Galaxy nesting + FOCAS auto-scale (managed-backend caveat)
v2-ci / build (push) Failing after 37s
v2-ci / unit-tests (tests/Core/ZB.MOM.WW.OtOpcUa.Cluster.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.ControlPlane.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Runtime.Tests) (push) Has been skipped
v2-ci / unit-tests (tests/Server/ZB.MOM.WW.OtOpcUa.Security.Tests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.Host.IntegrationTests) (push) Has been skipped
v2-ci / integration (tests/Server/ZB.MOM.WW.OtOpcUa.OpcUaServer.IntegrationTests) (push) Has been skipped
|