d7c9ead64e8ba01eed3fdd4a693de04b8b5d2a21
423 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
dd900f970a |
fix(hosts): a driver that never built a tag table reports NO ANSWER, not zero skips
Found on the docker-dev live gate, not in tests. Two drivers that faulted during initialise (rtu-gate, opcua1) each rendered "0 tags" on /hosts against two authored tags apiece, because a ledger that had never committed returned SkippedTagReport.Empty. A zero report is a positive claim -- "this driver skipped nothing" -- and making it on behalf of a driver that never got as far as looking is exactly the quiet inaccuracy #541 exists to remove. GetSkippedTags() and SkippedTagLedger.Snapshot() now return null until the first committed build, so the column renders the same em-dash it already used for a driver that cannot report at all. Both are 'no answer'; neither is 'nothing skipped'. Knock-on: a FOCAS driver whose only init threw now reports null rather than a zero report, which is strictly more honest than the assertion the test originally made. |
||
|
Joseph Doherty
|
39db75083d |
feat(drivers): every driver reports the authored tags it skipped (Gitea #541)
A driver builds its RawPath -> definition table at init and tolerates a
per-tag miss: an unmappable TagConfig, an unparseable address, a
capability-matrix rejection. That tolerance is right -- one bad tag must not
take down the other few thousand. What was missing is any statement of the
fact afterwards. A skip is not an error and happens before any read, so the
driver stays Healthy, errors/5 min stays 0, Last error stays empty, and Last
read advances live off the tags that did map. On the wonder Makino in #540
that reading held while focas-z-34184 ignored 20 of its 22 tags.
New optional capability ISkippedTagReporter, backed by a shared
SkippedTagLedger so the reset/commit/cap rules are one implementation rather
than one per driver. The report -- authored count, exact skipped count, and a
capped sample of RawPath + reason -- rides DriverHealthChanged to a new Tags
column on /hosts, over the Phase 5 telemetry stream. No table: a driver-only
node has no ConfigDb to write to, the lesson from #521.
The issue named eight drivers. It is twelve -- every driver in
DriverTypeNames.All has an authored-tag skip path. Calculation (no scriptId),
MTConnect (no readable DataItem id), Galaxy (no attributeRef) and OpcUaClient
(no nodeId) were unlisted, and OpcUaClient's was the quietest of the twelve:
NamespaceMap drops such an entry with a bare `continue` and not even a log
line. SkippedTagReporterFleetTests pins the invariant reflectively over every
IDriver in src/Drivers/, so a thirteenth cannot re-introduce the silence.
Sql drops a tag at TWO points and its report is the union of both: a blob
that does not map, AND an identifier the 8.1 catalog gate rejects against the
live catalog. The second is what schema drift produces -- a renamed column, a
dropped view -- months after a deployment sealed green, and reporting only
the first would have that driver claim "0 skipped" while polling half its
tags, which is worse than reporting nothing because the column would then be
actively reassuring.
Deliberately additive to driver state, never a change of it. Latching
non-Healthy for a config problem would satisfy both halves of the #527
comms-loss conjunction -- 90 s without a successful read AND non-Healthy --
and a driver whose tags were ALL skipped has no successful reads by
construction, so it would reconnect-loop forever against a healthy device.
Three details that are load-bearing rather than incidental:
- The skip digest is part of the publish fingerprint. A skipped tag leaves
every other fingerprint component identical to a clean driver's, so without
it PublishHealthSnapshot's dedup swallows exactly the publish carrying the
signal -- the trap that already bit the rediscovery and host-status
channels. The digest covers the sample, not just the counts: a redeploy
that fixes one tag and breaks another leaves both counts unchanged.
- The count is exact; only the sample is capped, and the tooltip says so when
it is showing fewer than it counted. A truncated list that looked complete
would have an operator fix twenty tags and believe the job done.
- The ledger stages and commits, so an init that throws part-way leaves the
previous report standing. A partial snapshot would read as "we now skip
fewer tags", the opposite of the truth.
The proto field needs no companion presence flag, unlike host_statuses: a
singular message field carries real presence in proto3, and a has_ flag would
collide with protoc's generated HasSkippedTags.
Verified red-first three ways: dropping SkippedTagDigest from the fingerprint
fails the two dedup tests, removing the seam from one driver fails the fleet
guard, and dropping the catalog-gate leg fails the Sql union test. Full
solution builds with zero new warnings against
|
||
|
Joseph Doherty
|
4e22f02434 | test(focas): thread the test cancellation token through the two new Task.Delay waits | ||
|
Joseph Doherty
|
b61fe38823 |
fix(focas): browse-committed fixed-tree tags are readable (Gitea #540)
Authoring FOCAS tags through /raw -> Browse device... produced tags the driver
could never read. Every committed tag read BadNodeIdUnknown forever, and the
failure was silent end to end: the AdminUI accepted it, the deploy gate sealed
green, and the driver reported Healthy with 0 errors while ignoring the tags.
Net effect: the whole fixed-tree surface could not be published through any v3
authoring path. Found cutting wonder-app-vd03 over to v3 against a real Makino
Pro 5 / Fanuc 30i, where 0 of 20 committed tags read while two hand-authored
MACRO: tags on the same device read Good.
A captured browse leaf's id IS its DriverAttributeInfo.FullName, and the fixed
tree emitted "{deviceHost}/{path}" there -- an internal cache key, not an
address. Both authoring paths wrote it verbatim into TagConfig.address, and
FocasAddress.TryParse accepts no such form, so the driver skipped the tags at
init.
Fixed at the emitter rather than by stripping the host at commit, which fixes
the browse-commit flow and the manual picker's tree-select together and leaves
one canonical address form:
- FocasAddress gains a fourth kind, FIXED:<path>, and moves to Contracts so the
AdminUI and ControlPlane can parse without referencing the runtime driver
(#526). A FIXED: read is served from the fixed-tree snapshot the poll loop
already maintains -- no area read, no connection. Writes refuse: nothing in
that tree is a writable location on the CNC.
- The address carries no device. A tag's device comes from its Device row, and
a second routing channel for one fact is what created this defect.
- An authored FIXED: tag arms the fixed-tree poll loop by itself. Enabled
defaults false and the browser force-enables it only for a browse, so the
operator would otherwise browse, commit, deploy, and land on a driver whose
loop never starts -- every tag Bad with a correct address.
- Fixed-tree values are now dated from the last answered poll and go
BadCommunicationError once it is stale (8 cadences, floored at 30s). The poll
loop swallows a failed tick by design, so a dead CNC would otherwise serve its
last snapshot as Good indefinitely -- harmless while nothing could bind to the
tree, not harmless now.
Guardrails, so this class cannot recur silently:
- FocasTagConfigModel.Validate() runs the real parser instead of a non-empty
check, and FocasTagDefinitionFactory.Inspect() warns at the deploy gate, which
covers CSV/API config that never touches the modal.
- The manual picker offered axis/spindle/program/status "parameter groups" --
none of which is a FOCAS addressing space -- and its own default value,
axis:5, was unusable. It now composes PMC / PARAM: / MACRO: through
FocasAddress, so what it emits is parseable by construction.
Tests assert addresses ROUND-TRIP through the parser rather than matching an
expected string: the previous FocasAddressBuilderTests pinned "axis:5" exactly,
which is how a green suite coexisted with a picker nobody could use. Verified
red-first against the old emitter. FOCAS 308, AdminUI 1095, ControlPlane 169,
Core 581 -- all pass, 0 warnings.
The remaining half of #540 -- a driver reporting Healthy/0 errors while
skipping tags at init -- is fleet-wide (eight drivers do this) and is filed
separately rather than patched per-driver.
|
||
|
Joseph Doherty
|
75a057f268 |
fix(sql): Test Connect must construct Postgres and Oracle dialects too
SqlDriverProbe.DefaultDialectFor was left SqlServer-only when the provider gates were widened, so AdminUI Test Connect failed for both new providers with InvalidOperationException while Deploy and browse worked. Found by the live chrome/AdminUI gate the plan deferred, not by any test: every case in SqlDriverProbeTests builds its probe through ForTest(), which injects the dialect and never reaches the production switch. Widen the switch arm-for-arm with SqlDriverFactoryExtensions.CreateDialect, and cover the gate directly via an internal seam — plus a reserved-provider falsification control and an end-to-end production-constructor case. |
||
|
Joseph Doherty
|
d54f3c8342 | merge: Sql driver Postgres + Oracle providers (managed Npgsql/ODP.NET, dialect-owned binding, live-gated fixtures) | ||
|
Joseph Doherty
|
9675823cd6 |
test(sql): env-gated Postgres + Oracle integration suites
Two disposable docker fixtures (postgres:17-alpine on 15432, gvenzl/oracle-free: 23-slim on 11521) shaped like the existing mssql one — project label, distinct container name, healthcheck, one-shot seed service — plus a live suite per provider, gated on SQL_PG_* / SQL_ORACLE_* and skipping clean offline. Each suite covers what a DIALECT can get wrong, rather than re-running the provider-agnostic read semantics already pinned against SQL Server and SQLite: liveness + catalog load, a KeyValue read, the provider's own row-limit syntax via topByTimestamp (LIMIT 1 / FETCH FIRST 1 ROWS ONLY, where T-SQL emits TOP 1), identifier case-folding through the design §8.1 catalog gate, and the AdminUI browse walk driven through SqlDriverBrowser. Two seed decisions are load-bearing rather than scaffolding: - The Postgres schema is owned by a role the suite is NOT; the connecting login holds USAGE + SELECT only. That is the exact provisioning PostgresDialect's ListSchemasSql exists for (information_schema.schemata lists only schemas the role OWNS, so a granted-not-owning poller would see none and every tag would be rejected). A fixture that owned its own schema would pass either way. - Every identifier in both seeds is created UNQUOTED, so the catalogs hold lower-case (Postgres) and UPPER-case (Oracle) spellings. The case-folding legs author the opposite case and pin SqlCatalog.TryMatch's REAL contract — exact ordinal first, then a UNIQUE case-insensitive match whose catalog spelling is substituted — which on both providers is what makes mixed-case authoring work at all, since both dialects emit case-sensitive double-quoted identifiers. Connection strings are composed through dialect.Factory.CreateConnectionString- Builder() and the abstract DbConnectionStringBuilder indexer, so the csproj keeps its no-direct-provider-reference discipline; every key used was verified accepted through that indexer. The csproj gains only Driver.Sql.Browser, for the browse leg. Live-run evidence (local OrbStack, both images arm64): Postgres 5/5 green; Oracle 6/6 green after the ORA-01745 dialect fix this suite found. Offline the whole project is 33 skipped / 0 failed. |
||
|
Joseph Doherty
|
ec90b7c416 |
fix(sql): Oracle bind names must not be reserved words (ORA-01745)
Found by the new OracleReadTests against a real Oracle 23 Free: EVERY Oracle
driver instance failed its Initialize-time catalog load, and every AdminUI browse
of a table's columns threw, with
ORA-01745: invalid host/bind variable name
Oracle rejects a bind variable named after a reserved word, and the catalog
readers bind a parameter whose base name is `table` (SqlCatalogLoader /
SqlBrowseSession bind `schema` + `table`), so `ListColumnsSql` could not execute
at all. The provider was structurally non-functional.
OracleDialect now namespaces every marker/bind name through one `p_` constant
(`:p_k0` / `p_k0`), which a reserved word can never collide with, and its catalog
SQL is respelled to match. Prefixing everything rather than carrying a
reserved-word list is deliberate: the list is long and version-dependent, and a
future base name (`date`, `user`, `comment`, `level`…) would silently re-open the
same hole. Marker and bind name are derived from the same constant, so they
cannot drift apart.
No offline test could have caught this — only a server parses the SQL — so the
regression test asserts the invariant that made it possible (a bind name is never
the bare base name), with `table` called out explicitly.
Live evidence: 6/6 OracleReadTests red with ORA-01745 before, 6/6 green after,
against gvenzl/oracle-free:23-slim.
|
||
|
Joseph Doherty
|
b4480bf45a |
test(mtconnect): live write legs against the AllowPut fixture
The fixture Agent flips to AllowPut = true (its comment rewritten: writes are in
scope now that the driver implements IWritable) and the suite grows three
env-gated write legs, all still Assert.Skip-clean with no Agent reachable.
Devices.xml gains a DataItem, and that is the deliberate part. The write target
has to be one the SHDR adapter NEVER feeds, or the assertion races the stream --
but it must NOT be the existing never-fed fixture_block: the first successful
write takes an item off UNAVAILABLE for the life of the container, which would
silently rob Read_maps_an_unavailable_observation_to_BadNoCommunication of its
subject. So fixture_writeback (free-text EVENT, PROGRAM_COMMENT) is a separate,
dedicated target, overridable via MTCONNECT_WRITE_TARGET for a run pointed at a
real machine tool.
The three legs:
* Round trip -- driver write accepted, then the injected value read back off
/current over plain HTTP (not through the parser under test) AND through the
driver's own ReadAsync. The value carries a TickCount64 suffix: the Agent
keeps an injected observation for the container's life, so a constant would
be satisfied by a previous run's leftover.
* Agent refusal -> BadCommunicationError. The design doc doubted this leg was
constructible ("an id in the probe but refused by the agent"); it is, and the
mechanism is BuildWriteTargets. Write targets are derived from the tags the
BINDING serves, with the device resolved from the probe model and falling
back to the configured deviceName -- so an authored tag naming an id the
Agent never declared, on a driver whose deviceName IS set, resolves to a real
URL and is POSTed. That is exactly the shape a stale tag has after a DataItem
is renamed. The second half of the test is the falsifiability control: the
SAME unknown id with NO deviceName has no URL and must be refused locally as
BadNodeIdUnknown. Two different codes is what proves the first write actually
reached the Agent rather than being short-circuited into a look-alike.
* Disabled gate -- write.enabled absent returns BadNotWritable AND /current is
unchanged from the snapshot taken just before, which is what distinguishes
"refused" from "reported refused and sent anyway".
RAN LIVE, not just compiled: the fixture was brought up locally (OrbStack,
MTCONNECT_AGENT_HOST_PORT=5555 -- macOS AirPlay squats :5000) and the whole
suite passed 15/15, 0 skipped. Evidence beyond the green: the Agent reports
<ProgramComment dataItemId="fixture_writeback">OTOPCUA-WRITE-2CBCB530</...> and
its log carries "Error processing request:
/OtFixtureCnc?otopcua_no_such_data_item_2CBCB3D9=refuse-me" -- the refusal leg
demonstrably went to the wire. Offline (no Agent) the suite is 15 skipped, 0
failed; build 0 warnings / 0 errors.
Note for anyone reading agent.cfg: BufferSize = 14 is an EXPONENT (2^14 = 16384
observations), not a 14-deep ring, and /current is answered from the Agent's
per-DataItem checkpoint anyway -- so eviction is not a hazard for a write
read-back even on this busy fixture stream.
LiveOptions/ConfigJson/LiveDriverAsync gained writeEnabled + deviceName
parameters, both defaulted off/null so every pre-existing test still runs
against the shipped read-only driver whatever the Agent's AllowPut says. The
config JSON sources write.enabled FROM the options object, so the parsed config
can never disagree with the options a test thinks it built.
|
||
|
Joseph Doherty
|
429ead837f |
feat(sql): construct Postgres and Oracle providers; MySql/Odbc stay reserved
Widens the two provider gates so an authored `{"provider":"Postgres"}` or
`{"provider":"Oracle"}` binds that provider's own dialect instead of failing
"not available in this build":
- `SqlDriverFactoryExtensions.CreateDialect` gains a Postgres and an Oracle
arm. There is deliberately no fallback arm — a default resolving to
SqlServerDialect would construct happily and then emit T-SQL at a Postgres
or Oracle server. The rejection message now names the three supported and
the two still-reserved providers.
- `SqlDriverBrowser.DefaultDialectSelector` widens identically, so a provider
the runtime constructs is also browsable; MySql/Odbc keep resolving to null
and the caller keeps reporting them as unavailable.
Tests: the existing rejected-provider controls flip from Postgres to MySql
(factory) and Odbc (browser) so the reserved half stays falsifiable, and the
new legs assert the bound dialect TYPE rather than mere construction —
"no throw" alone would pass against a SqlServer fallback.
Doc-comment sweep (contract addition, one line each): the "v1 constructs
SqlServer only" / "reserved for P2" claims in SqlProvider, SqlDriverConfigDto,
ISqlDialect.Provider and SqlServerDialect are now false and were corrected in
the same change.
|
||
|
Joseph Doherty
|
b833dda673 |
fix(mtconnect): a throwing value formatter cannot escape the write batch
TryFormatValue's catch filter named only FormatException / InvalidCastException / OverflowException, and the call sits OUTSIDE WriteAsync's per-slot try. But Convert.ToString falls back to the value's OWN ToString/IFormattable - arbitrary caller code that may throw anything - so any other exception escaped WriteAsync uncaught and failed the entire batch, breaking the documented never-throws per-slot contract. Broadened to a catch-all; the slot gets BadTypeMismatch and the exception is logged at Warning (TryFormatValue became an instance method to reach _logger). One unformattable value now costs its own slot and nothing more. Falsification checked, not assumed: restoring the narrow filter makes the new test fail with a raw "System.InvalidOperationException : this formatter is broken" propagating out of WriteAsync - i.e. it fails for the contract-breach reason, not on an assertion. Two tests added: - a value whose ToString throws InvalidOperationException is refused BadTypeMismatch while the *following* good slot still writes (the load-bearing half - it proves the batch survived). - mid-batch cancellation: cancellation injected from inside slot 1's own formatting, so slot 0 has provably already reached the Agent. It keeps its real Good outcome while slots 1-2 fill BadCommunicationError. Reporting a landed write as failed is worse than reporting a failure, and a fill that started at 0 instead of i would do exactly that. Tests: 534 passed / 0 failed (was 532). Solution build: 0 errors, no warnings in any MTConnect project. |
||
|
Joseph Doherty
|
74435801bb | feat(sql): OracleDialect (ODP.NET managed, BindByName) | ||
|
Joseph Doherty
|
b97b3a3604 | feat(sql): PostgresDialect (Npgsql) | ||
|
Joseph Doherty
|
b29623cf99 |
feat(mtconnect): IWritable via agent observation writes
MTConnectDriver now implements IWritable over the Task-1 agent-client seam (WriteObservationAsync -> the Agent's AllowPut POST surface). A write is OBSERVATION INJECTION, not machine command: it sets what the Agent reports for a DataItem, is visible to every consumer of that Agent, and is overwritten by the next adapter/SHDR update. Off at two independent gates, both off by default - the Agent's own AllowPut and the driver's write.enabled (Task 0). Per-slot refusal order (never throws, strict N-in/N-out batch shape): write disabled -> BadNotWritable no live client -> BadNotConnected (ReadAsync's existing convention) reference unresolved -> BadNodeIdUnknown no resolvable device -> BadNodeIdUnknown CONDITION category -> BadNotWritable null / non-scalar value -> BadTypeMismatch agent refusal OR throw -> BadCommunicationError OperationCanceledException fills the remaining slots and returns. The Agent's refusal text is logged at Warning and never enters DriverHealth.LastError (type-only discipline); a refused write leaves health untouched. Device-name resolution: TagBinding gains WriteTargetsByDataItemId, built in BuildBinding from a probe-model walk that remembers which device each DataItem was declared under (name, or id when nameless) plus its category. Falls back to options.DeviceName; unresolvable -> BadNodeIdUnknown rather than a guess. Values render via Convert.ToString(..., InvariantCulture) - so a Boolean writes the invariant string "True"/"False", NOT "1"/"0". Arrays are refused rather than rendered: Convert.ToString would emit the literal "System.Double[]", which the Agent stores happily and which is invisible until read-back. FINDING - the CurrentWrite bit does NOT key on SecurityClassification, and the discovery stamping below is therefore cosmetic. Traced end to end: a browsed leaf's SecurityClass survives into the browse session's AttributeInfo (CapturedTreeBrowseSession.cs:53) and stops there - RawBrowseModal never reads it, RawBrowseCommitMapper.MapLeaf hardcodes TagAccessLevel.Read, and Tag has no SecurityClass column. AddressSpaceComposer:746/:809 sets Writable purely from Tag.AccessLevel == ReadWrite; AddressSpaceApplier ANDs !IsArray; OtOpcUaNodeManager.ComposeAccessLevel:2002 composes CurrentWrite from that alone. Driver capability is never consulted at compose time either - IWritable is checked only at runtime in DriverInstanceActor:755. Consequences: (a) raw tags already committed keep their authored AccessLevel, so nothing this commit does makes an existing node writable; (b) an operator must still set the tag to ReadWrite for an MTConnect write to be reachable over OPC UA; (c) conversely a ReadWrite tag on a non-IWritable driver has always advertised CurrentWrite and failed only at the actor - pre-existing, not touched here. Discovery stamps CanWrite(writeEnabled, dataItem) = write.enabled && !CONDITION -> Operate, else ViewOnly - kept honest so the browse never advertises a tier the write path would refuse (MQTT's CanWrite principle), with the caveat above recorded verbatim at the call site. Status-code constants are spelled WITHOUT MqttDriver's "Status" prefix, to match the two this file already declares (BadCommunicationError, BadNotConnected). StatusCodeParityTests strips either spelling and passes. TDD: WriteAsync landed first as a NotImplementedException scaffold - 19 of 20 new tests red on behaviour, 1 (the capability assertion) green on the scaffold - then implemented. MTConnectFactoryTests' "not_writable" characterization test is rewritten to pin the new truth, and is the guard for the concrete-return contract the factory remark explains. Tests: 532 passed / 0 failed (was ~509). Full solution build: 0 errors, 0 warnings; OTOPCUA0001 clean. |
||
|
Joseph Doherty
|
1da770253e |
refactor(sql): dialect-owned parameter markers, bind names and command configuration
ISqlDialect gains ParameterMarker (the token in the SQL text), ParameterBindName
(the DbParameter.ParameterName) and ConfigureCommand (per-command provider setup,
where Oracle will set BindByName). SQL Server and the test SQLite dialect answer
"@"+name for both spellings and no-op the hook, so SQL Server behaviour is
byte-identical — every existing expected-SQL assertion passes untouched.
SqlGroupPlanner drops its hardcoded @k/@w consts and mints base names (k0..kN, w)
that the dialect spells; SqlQueryPlan carries the emitted markers AND the matching
bind names, so SqlPollReader keeps copying the plan's pairing rather than
re-deriving a provider convention (the two diverge on ODP.NET: ":k0" in the text,
bare "k0" on the parameter). SqlCatalogLoader and SqlBrowseSession bind
schema/table through the dialect, so their bind names cannot drift from the
markers in the dialect-owned catalog SQL. Every parameter-carrying command site
calls ConfigureCommand before its first parameter.
Controlled by a new test-only ColonMarkerDialect (":name" markers, bare bind
names) that really executes against the SQLite fixture. Each leg was falsified
against the pre-change behaviour: the planner emitted "@k0", the reader bound
":k0", and a hardcoded "@schema" in the loader failed catalog load outright.
|
||
|
Joseph Doherty
|
3b8b5a77e1 | fix(mtconnect): share MTConnectError text extraction; pin hollow-error refusal | ||
|
Joseph Doherty
|
9f17d7efa2 | feat(mtconnect): agent-client observation write (HTTP POST, AllowPut surface) | ||
|
Joseph Doherty
|
b5e39e1f32 | feat(mtconnect): parse the write.enabled config block | ||
|
Joseph Doherty
|
6b3da93151 |
fix(browse): commit the browsed data type, and fix the probe classification the fixture was hiding (#536)
A browse-committed AB CIP / TwinCAT / FOCAS tag bound with its model's DEFAULT
data type, not the one the device reported. A browsed REAL committed as
`{"tagPath":"Pump_1_Speed","dataType":"DInt"}` — and those three drivers decode
the wire bytes by that field, not by the raw row's OPC UA DataType — so the tag
read four bytes of a float through an integer decoder and published a plausible
wrong number with no error raised anywhere. FOCAS was missed when #536 was filed.
The type is CARRIED, not derived. Reverse-mapping the leaf's driver-agnostic
DriverDataType looks like the smaller fix and is wrong: SInt/Int/DInt/USInt/UInt
all collapse into Int32 on the way out of the driver, so that version would bind
a 16-bit INT as a 32-bit DINT while appearing to fix the REAL. Drivers now state
their own type name on DriverAttributeInfo.NativeDataType, which reaches the
picker as AttributeInfo.AddressFields["dataType"] and the commit mapper from
there. A leaf that states nothing keeps the model default — the pre-#536
behaviour, kept deliberately as the fallback rather than a guess.
Numeric strings are refused before parsing: Enum.TryParse accepts "9" for any
enum and Enum.IsDefined then confirms it, so an unguarded parse would bind a tag
to a type nobody named. A test caught this in the first version of the guard.
--- and the other half, which the fixture was hiding ---
Test Connect on a device with NO authored tags — every AB CIP device at the
moment an operator first presses the button — reported "CIP handshake failed"
against a healthy controller. That was written down as a fixture limitation
("ab_server doesn't implement @raw_cpu_type"). Both halves of that were wrong:
* @raw_cpu_type is not a libplctag special tag at all (only @raw, @tags and
@udt/ are), so it goes out as an ordinary symbolic name that no controller
defines, and the controller's "tag does not exist" IS the successful answer.
* ab_server answered an unresolvable tag path with CIP 0x03 where a Logix
controller answers 0x04 — ErrorBadParam vs ErrorNotFound, which the probe
classifies oppositely. Fixed in vendored patch 0002.
* With the fixture then telling the truth, the real defect surfaced:
AbCipDriverProbe.IsReachableException matched the native C status spellings
("STATUS_NOT_FOUND"), but libplctag.NET throws with the MANAGED enum name
("ErrorNotFound"). Those substrings can never match, so the reachable
classification could not fire on the exception path — which is the path every
no-authored-tags probe takes. This was failing on real hardware too, not just
against the sim. It now recovers the Status and defers to IsReachableStatus,
so there is one classification rather than two that can drift.
Rejected: switching the fallback to @tags. libplctag builds a tag-listing tag
with NO network I/O, so it reports OK against any TCP listener — measured green
against a Mosquitto broker.
Verification: full solution suite green. Falsification runs confirm exactly the
intended tests go red — 9 for the type carry, 3 (the reachable statuses only) for
the probe classification. Live on docker-dev: browse-committed TestString/
Motor_Run/TestSINT now store String/Bool/SInt, alongside last session's
Pump_1_Speed still holding the wrong DInt. TestSINT is the falsifier — its row
DataType is Int32, identical to TestDINT's, so no reverse map could have produced
"SInt".
Known consequence: pre-existing browse-committed tags keep their wrong dataType;
the fix applies to new commits only.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
c2c3c52274 |
fix(abcip): unblock the controller-browse gate, and fix the decoder it exposed (#468)
The AbCip controller browse could never be verified end to end: libplctag's ab_server answers CIP service 0x55 (Get Instance Attribute List) on the Symbol Object class with "unsupported", so the `@tags` walk had no backend. #468's own definition of done offered a sim that serves `@tags` as an alternative to real hardware; this is that. The fixture now applies a vendored patch teaching ab_server to serve the Symbol Object listing, with partial-transfer paging. Correctness is checked against libplctag's own `list_tags_logix` rather than against our decoder — our decoder is the thing under test, so its agreement would prove nothing. The clone is also pinned to a concrete tag: the Dockerfile called `--branch release` "pinned" while tracking a branch that moves. That immediately exposed a real defect. CipSymbolObjectDecoder advanced to an even byte boundary after each symbol name, but entries are packed with no padding — libplctag walks them by sizeof(entry) + string_len, and a captured reply confirms it. Any odd-length name therefore desynchronised every entry behind it, and odd-length names are ordinary. In the live browse below, `Motor_Run` is 9 bytes and first on the wire, so the old decoder would have shown that one tag and silently dropped the other eight. Every existing multi-entry test passed throughout, because the test builder padded exactly as the decoder did: the two agreed with each other and neither agreed with a controller. The builder no longer pads, and the layout is pinned by a captured reply (CipSymbolObjectWireCaptureTests) instead of by a second statement of the assumption. Restoring the alignment step reddens exactly the three new tests and none of the 29 others. Live gate on docker-dev, all three of #468's criteria: - populated tree — all nine seeded tags render as leaves, in wire order - refresh re-captures - leaf commit — Pump_1_Speed + TestDINT saved with their CIP tag paths The gate also surfaced two things it was not asked about: browse-commit drops the discovered data type, so a browsed REAL commits as a DINT and the driver decodes it as one (filed as #536), and Test Connect fails against this fixture regardless of the device because it probes `@raw_cpu_type` (documented). TwinCAT and FOCAS still have no browsable backend; #468's AbCip half is closed, theirs is not. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
01538464b7 |
fix(abcip,ablegacy): honour the authored EtherNet/IP port (#534)
Both libplctag drivers carried Port on their tag-create params and consumed it nowhere. The runtimes set Tag.Gateway to the bare host, and libplctag does not fail on a portless gateway — it substitutes 44818. So an authored port was not ignored in the harmless sense; the connection was silently REDIRECTED to whatever answers on the default port. On a plant network that is another PLC, and the driver would read its tags and report them Good. This is the root cause of #534's four AbLegacy integration failures, and the explanation for every observation in that issue: with each PCCC family moved to its own port (44819/44820/44821), the driver still dialled 44818 and kept talking to the AB CIP fixture's ControlLogix container. Hence PCCC-against-ControlLogix errors under every server-path / client-cip-path permutation — the permutations were never reaching a PCCC server at all. Established by a direct libplctag 1.5.2 probe outside the repo, which answered #534's open question: PCCC works fine against ab_server for all three families, with path="1,0" AND with path=null (only path="" throws ErrorBadData). Since the library could reach the simulators and the driver could not, the difference had to be in our Tag construction, and Gateway was the only field that differed. Same probe confirmed libplctag accepts host:port in the gateway attribute, which is what the fix emits. AbCip has the identical defect and is fixed alongside. It has never been observable there: every AbCip fixture profile binds 44818, so no test that driver has could distinguish "uses the authored port" from "always uses 44818". A fixture that only ever exercises the default value cannot falsify a rule about non-default values, however many tests point at it. The port is now emitted unconditionally rather than only when non-default, so the value does not depend on libplctag's own default; a port <= 0 still falls back to the bare host rather than producing "host:0". The load-bearing guard is the AbLegacy integration suite, which addresses each family on a non-default port and cannot pass unless the port is honoured — 4/4 green, having never once passed before. Unit tests in both drivers pin the composition for machines with no fixtures. Falsified both ways: reverting the runtime line reddens all 4 integration tests; dropping the port > 0 guard reddens exactly the two fallback cases. AbCip integration 10/13 (3 pre-existing skips), AbCip unit 345, AbLegacy unit 219. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
b13bdc154a |
test(ablegacy): deploy the PCCC fixture, one port per family (#534)
The AbLegacy integration suite has never run against a simulator. /opt/otopcua-ablegacy/ did not exist on the Docker host — the stack was written but never deployed — and every ab_server across BOTH ab fixtures bound :44818. So the AB CIP fixture's ControlLogix container (up for two weeks) answered this suite's TCP reachability probe, the suite declined to skip, and it ran SLC500-mode PCCC against a ControlLogix. Four failures that read as "fixture missing". Two structural problems, both fixed here: - The theory parameterises over all three PCCC families, but only one container could ever exist on the shared port, so at least two cases failed on every conceivable run. The suite was un-greenable by construction, and AB_LEGACY_COMPOSE_PROFILE existed to paper over it by filtering the theory down to whichever container happened to be up. - A reachability probe that cannot tell one simulator from another is not a skip gate. It answered "is some ab_server listening", which is a different question from the one the caller is asking, and reported the difference as a test failure. Each family now has its own host port — 44819 slc500, 44820 micrologix, 44821 plc5 — so all three run together and alongside AbCip on 44818 (ab_server still listens on 44818 inside each container; only the host mapping differs). Skip is per-family via SkipReasonFor(profile); the class-level gate runs if ANY family is up so one down container cannot silently skip the other two. AB_LEGACY_COMPOSE_PROFILE now applies only in AB_LEGACY_ENDPOINT-pinned real-hardware mode, where the family genuinely is a property of the one device. This finally makes true what the fixture's XML doc has claimed from the start — that it and the AB CIP fixture can point at different containers simultaneously. Deployed to /opt/otopcua-ablegacy/ on 10.100.0.35; all three containers running. The 4 tests still FAIL, and are left red on purpose — see #534. Now aimed at correct-family simulators, the PCCC exchange itself returns BadCommunicationError for every server-path / client-cip-path permutation, reproduced outside the test harness via the driver CLI. That is a true signal about the driver (or about libplctag's PCCC support against ab_server) that skipping would hide. What changed here is that the failure is now diagnostic instead of an artifact of a port collision. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
b815446940 |
refactor(drivers): delete GetMemoryFootprint / FlushOptionalCachesAsync (#525)
Both `IDriver` members had no production call site. Their only consumer was `MemoryTracking`, deleted with the driver-tier recycle machinery in #522, and `GetMemoryFootprint`'s doc-comment claimed "polled every 30s by Core" — which had not been true since v3, even before #522 removed the poller. Owner decision on #525 was (a) delete rather than (b) build the process-wide watchdog that would consume them. The driver-side implementations are in git history and were mostly reusable; what was always missing is the trigger, and a member nobody calls is how the misleading doc-comment survived this long. Removed from `IDriver`, all 12 driver implementations, and every test stub — plus the support code that existed only to feed them, which is the part a member-only deletion would have left behind as dead weight: - MTConnect's three per-entry byte-weight constants, its cached `ProbeModelDataItemCount`, and `CountDataItems` — a full device-tree walk run on every cache fill to populate a number with no remaining reader. - OpcUaClient's `_discoveredNodeCount` field and the discovery-pass write to it. - MQTT's `ApproxBytesPerAuthoredTag`. Two behavioural notes, neither a change in what the software does: - `AbCipTemplateCache` now has no production reset path. Its only caller was AB CIP's flush, which itself had no caller — so nothing changes except how visible the fact is. Recorded on the type, along with the correction that its comment's claim about a device `ReinitializeAsync` also flushing it was never true. A UDT redefined on a live controller keeps its stale shape until the driver instance is rebuilt. - MTConnect's `/probe` cache is still dropped, by `InvalidateProbeModel()` on an Agent restart. That is now `internal` rather than `private` so the tests which used the flush purely to empty the cache drive the real path instead — they now cover a case that actually occurs in production. Several unrelated doc comments cited `FlushOptionalCachesAsync` as the canonical CAS example and are repointed to it. Tests: the ones that existed only to assert on the removed members are deleted (3 OpcUaClient, 3 Galaxy, 2 MQTT, 1 MTConnect, 1 AbCip). Worth noting one was vacuous — `FlushOptionalCachesAsync_resets_footprint_counter` asserted a footprint of 0 before the flush and 0 after with no discovery in between, so it would have passed against a flush that did nothing at all. Full suite green except 7 failures verified identical on master before and after (4 AbLegacy integration with the compose profile down, 3 Host.IntegrationTests), checked by name rather than assumed. Docs corrected where they described this as live: `DriverLifecycle.md` (which carried the #525 placeholder), `drivers/AbCip.md`, `drivers/AbLegacy.md`, and `v2/driver-stability.md` — whose "polls this every 30 s" + per-driver `DriverConfig` memory budget were never built as written and are now banner-ed as design intent rather than description. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
803f4e6efe |
refactor(sql): split ISqlDialect/SqlServerDialect into a .Dialect leaf (#526)
`Driver.Sql.Browser` referenced the runtime `Driver.Sql` project. The AdminUI references that browser, so the whole SQL driver runtime — and `ZB.MOM.WW.OtOpcUa.Core` through it — sat in the AdminUI's build/deploy graph. This is the same layering violation #512 removed from the MQTT browser, in a second driver, and it is why #512's expected removal of `Core.dll` from the AdminUI's output did not happen. MEASURED, both directions, on a clean AdminUI build: before: ZB.MOM.WW.OtOpcUa.Core.dll + ZB.MOM.WW.OtOpcUa.Driver.Sql.dll present after: both absent The old csproj comment's premise was sound and survives — the dialect's catalog SQL *is* the browse engine, so it must be shared rather than duplicated. What was wrong was the conclusion that sharing it required referencing the runtime driver. `ISqlDialect`/`SqlServerDialect` depend on nothing in `Driver.Sql`, so they move to a `.Dialect` leaf that both consume. This is the `.Transport` shape, not the cheap one: `ISqlDialect` exposes `DbProviderFactory` and `SqlServerDialect` returns `SqlClientFactory.Instance`, so `.Contracts` (deliberately provider-free) cannot hold them. Assembly name is `.Dialect`; the NAMESPACE stays `ZB.MOM.WW.OtOpcUa.Driver.Sql`, matching the `Driver.Mqtt.Transport` precedent — the extraction cost zero `using` changes. Two dependencies the cut edge surfaced as the implicit ones they were: - The AdminUI itself read `SqlDriver.DriverTypeName` in three places — the tag editor map, the tag config validator, and the Sql address picker. All three now read `DriverTypeNames.Sql`, which is exactly what that constant resolves to and exactly the repoint `SqlDriverBrowser`'s own comment had deferred until a factory was registered against the shared constant. One exists now. - `SqlDriverFormContractTests` asserts the AdminUI form's emitted JSON against the runtime factory that reads it, so it genuinely needs the runtime driver. `AdminUI.Tests` now references it explicitly. A test project referencing a runtime driver is fine — tests are not in the AdminUI's deploy graph. New guard: `DriverBrowserLayeringTests` (Commons.Tests) scans the csproj graph of every `*.Browser` project in the tree, transitively. It reads csproj files rather than loaded assemblies because the cost is a build-graph cost — the compiler emits no assembly reference for an unused `ProjectReference`, so `MqttBrowserLayeringTests`' assembly-reference approach cannot see that case. The two are complementary and each now points at the other. Falsified three ways, each red then green again: the direct reference re-added to the browser; the same reference one hop out via `.Dialect` (proving the walk is really transitive); and both controls — browsers-discovered and references-parsed — which exist so an empty discovery cannot pass the absence assertion vacuously. #526's claim that the reference "carries no comment at all" was wrong; it carried a detailed one from the scaffold commit. Corrected on the issue and in the design doc's project table, which had recorded the deviation as deliberate. Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
c74ccc55b5 |
feat(mqtt): flatten Sparkplug Template/DataSet metrics to sibling tags (#515)
`SparkplugCodec` handled scalars; `DataSet`, `Template` and `PropertySet` were skipped
with a warning. They now flatten to addressable tags (owner decision, recorded on the
issue):
Template → sibling metrics, `/`-joined, recursively: Motor → Motor/Speed, Motor/Stator/Temp
DataSet → one metric per cell, {column}/{rowIndex}: Recipe → Recipe/Step/0, Recipe/Target/1
PropertySet → node metadata, never tags — it describes a metric rather than being one
Flattening was chosen over OPC UA Structures because flattened scalars are consumable by
any client with no type dictionary, and the result mirrors how the `/raw` tree works.
**The row index is always present, including on row 0.** Emitting a bare column name for
a single-row table and an indexed one otherwise would make a tag's name depend on how many
rows the plant published — a table growing from one row to two would rename every tag
already authored against it.
**A DATA metric carries the parent's alias and no name**, so member names cannot be
composed until the alias table has resolved the parent. Expansion therefore happens in
`SparkplugIngestor.PublishStructuredMembers` — after resolution, at the single funnel both
the DATA and birth-value paths already pass through — not at decode time. The alias table
installs a binding per member so browse enumerates them and a DDATA member that omits its
datatype falls back to the birth's (guessing from the CLR type would read a signed Int32 of
-42 out of its unsigned field as 4294967254).
**Skip-and-warn survives, and refusal is whole** — the explicit requirement on the issue.
A template definition, an unnamed member, an extension value, a column/type mismatch, a
short row, or a payload past MaxDepth(5)/MaxMembers(512) refuses the ENTIRE metric. A
silently half-flattened UDT looks exactly like a complete one in the picker.
The browse picker offers the members and deliberately NOT the structured parent, which has
no value and no servable datatype. The rediscovery signature likewise tracks members rather
than the parent, or a UDT gaining a member would leave the signature unmoved.
Member paths contain `/`, which #509 already solved: the slashed name travels intact as
`metricName` while the derived tag Name is coerced to a legal RawPath segment.
Live-gated 3/3 against the Mosquitto fixture with the project-owned simulator extended to
publish a nested Template and a 2x2 DataSet. Falsified at three points: dropping the member
bindings, not expanding the parent, and refusing to flatten templates each turn exactly the
expected tests red.
Also fixes a regression from the #511 commit in the same branch: `AddOtOpcUaDriverFactories`
resolved `IDriverPrimaryRoleSource` with `GetRequiredService` but only `AddOtOpcUaRuntime`
registered it, so a graph wiring driver factories without the full runtime failed at
resolution. Two Host integration suites do exactly that and caught it; both extensions now
TryAdd the same concrete `RedundancyRoleView`, so the registration is order-independent and
the fail-loud resolve is kept.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
7f2108dcf2 |
feat(mqtt): Sparkplug primary-host STATE publishing, Primary-gated (#511)
`ActAsPrimaryHost` was accepted by config and by the AdminUI form and did nothing but
log a warning. It now delivers the whole three-part contract or none of it: a retained
`ONLINE` on `spBv1.0/STATE/{hostId}`, a matching retained `OFFLINE` armed as the MQTT
Last Will, and an explicit `OFFLINE` on a clean stand-down.
**The claim is a dedicated connection, not a message.** A Will can only be registered in
the CONNECT packet — it cannot be added to a live session, changed, or withdrawn. So a
claim that is acquired and released at runtime cannot ride the long-lived ingest session.
`SparkplugPrimaryHost` opens its own connection (client id `{clientId}-spbhost`, forced
clean-session) when the node gains Primary and closes it when the node loses Primary; the
connection's lifetime *is* the claim, so every leg follows from its lifecycle. Ingest is
untouched by a role change — no dropped subscriptions, no lost birth cache, no data gap at
the Secondary->Primary transition.
**Primary-gated** (owner decision, recorded on the issue). Both pair nodes subscribe
independently, so an ungated claim means both claim the same host id — and the sharp
failure is not the two fighting over the retained STATE, it is that the SECONDARY's death
fires its Will and publishes OFFLINE while the Primary is healthy, stopping every edge node
in the group. Registering the Will only on whichever node is Primary at connect time fails
the other way: a node that later gains Primary would publish ONLINE with no Will behind it.
New seam: `IDriverPrimaryRoleSource` (`Core.Abstractions`) carrying the *data-plane* verdict
(`PrimaryGatePolicy.ShouldServiceAsPrimary`), not the alarm-drain verdict — an unknown role
with a driver peer present must deny. Registered unconditionally by `AddOtOpcUaRuntime`,
deliberately NOT alongside `IRedundancyRoleView`, which `AddAlarmHistorian` registers only
when that section is enabled: piggybacking would leave a pair with no alarm historian
falling back to "claim unconditionally". Polled on a 1s timer rather than evented — a
process singleton raising events at per-instance drivers outlives every driver it holds.
Accepted trade: a brief edge-visible OFFLINE window during failover. That is a genuine
state transition, not a glitch to hide.
Live-gated 5/5 against the Mosquitto fixture, including the Will leg — `AbandonForTests`
drops the socket with no DISCONNECT, because a clean disconnect makes the broker discard
the Will by protocol rule, so a test driven through `StopAsync` would observe the
stand-down OFFLINE and pass against an implementation that never armed a Will. Falsified:
removing the Will turns that leg and only that leg red.
Also corrects two stale claims found en route: the AdminUI form still said MQTT tags are
read-only (#508 shipped), and the driver doc still said nothing consumes `IRediscoverable`.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
72be19225d |
feat(mqtt): write-through (IWritable) — Plain publish + Sparkplug NCMD/DCMD (#508)
MqttDriver implemented no IWritable, so every MQTT node materialized read-only
regardless of what the operator authored. It now publishes.
Plain -> the tag's writeTopic; UTF-8 text (Raw/Scalar) or a bare JSON
literal (Json)
Sparkplug -> DCMD for a device metric, NCMD for an edge-node metric, carrying
the metric BY NAME with an explicit Datatype and no seq
Both at QoS 0, retain:false. Retain is deliberately not configurable — a retained
command is replayed by the broker on every future subscribe, including the
device's own reconnect, so one stale write would be re-applied forever.
writeTopic is a new, separate TagConfig key, and its ABSENCE is meaningful: the
write is refused BadNotWritable and the node is discovered ViewOnly. The issue's
"do not skip" warning drove the design — the one outcome worse than read-only
nodes is a node that looks writable, returns Good and publishes nothing. Reusing
the read topic would be worse still: it would echo back and read as a confirmed
device value. Browse-time writability is computed by the SAME encoder the write
path uses, so a node cannot advertise writability it would then refuse.
Good means published, not applied — MQTT has no application-level write ack, so
this is Galaxy's optimistic-Good posture and inherits its property: a failed
DEVICE write can never be surfaced. A failed PUBLISH is reported Bad; the
optimism covers what the device did with the command, never whether it was sent.
WriteIdempotent stays default-off.
The encoder is pure and lives in .Contracts, so both wire shapes are testable
without a broker. Unrepresentable values and types with no Sparkplug encoding are
refused rather than coerced — a Reference must not silently become a String on
the wire, and "not-a-number" must not become 0 on a live plant.
LIVE GATE PASSED against the real Mosquitto fixture on 10.100.0.35: 21/21
integration tests, including 6 new ones that observe the published message with
an INDEPENDENT broker subscriber — a fake asserting "publish was called" could
not tell the feature working from the feature publishing to the wrong topic.
The gate earned its keep: the NCMD test initially failed because NCMD is a SHARED
channel — the driver publishes its own rebirth request (Node Control/Rebirth) on
the very topic an edge-node metric write uses, and it arrived first. Correct
driver behaviour, wrong test; the watcher now matches on metric name. Documented,
because anything consuming that topic needs to know. DCMD has no contention.
Offline: 24 encoder + 9 driver tests. The four new status constants are NAMED so
StatusCodeParityTests can see them (#497/#519) — it went 258 -> 262 and validated
each against the pinned SDK.
Also corrects the Mqtt.md "Known gaps" table: prunes the six rows closed in this
round, and fixes the #507 row, which still described rediscovery as inert after
that issue was closed as superseded and its injection path deleted.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
7f53bd96ec |
fix(mqtt): close #509, #510, #512, #513, #514
Five MQTT tracker items — four AdminUI/authoring defects plus the .Browser layering exception. No runtime driver behaviour changes. #513 payloadFormat in a Sparkplug blob. MqttTagConfigModel.ToJson wrote the key unconditionally. Sharper than the issue states: PayloadFormat is the one Plain field with a NON-NULLABLE typed backing, so the write did not preserve a retyped tag's leftover — it INVENTED "payloadFormat":"Json" in a Sparkplug tag that never had one. Now Plain-only; preserve-unknown-keys is pinned unchanged. #509 slashed Sparkplug metric names. The mapper already carried metricName verbatim; only the tag Name (a RawPath segment) tripped RawPaths.ValidateSegment, which refused the batch whole — including Node Control/Rebirth, mandatory on every Sparkplug edge node. ToRawSegment coerces the Name and leaves the address untouched. Coercion is not injective, so DescribeNameCollisions refuses a colliding set BY NAME: silently merging two metrics into one tag would be worse than the refusal it replaces. Applies to every driver, not just MQTT — the argument is about what a Name is, not about who produced it. The sanitiser is pinned against RawPaths.ValidateSegment rather than a copied charset. #510 stale rebirth affordance. _canRebirth was probed once at browse-open and sessions are reaped for idleness, so a modal left open offered an action whose token no longer resolved. Re-probed on arm AND on confirm (the reap can land between the two, and confirm is the one path that publishes to a live broker); the affordance retires with an explanation instead of vanishing or erroring. #514 rebirth on a quiet plant. The tree is birth-driven and births are never retained, so a plant that birthed before the window opened showed an empty tree — and the scope came only from a tree selection, so the one action that would fill the tree could not be armed. The session already accepted a bare {group}/{edgeNode} ("the prime rebirth target"); this adds the affordance. Edge-node scope only, not a group broadcast. The rule is extracted to SparkplugRebirthScope so it is testable — this repo has no bUnit — and is advisory: the session re-resolves and refuses before publishing. #512 .Browser -> .Driver layering. TWO extractions were needed, not the one the issue anticipated: it costed this as "the client-options builder", but the browse session decodes births itself, so moving only that would have left the reference exactly where it was. - MqttClientOptionsFactory (TLS, PEM CA-pin validator, credentials, protocol mapping) -> new leaf .Transport project. NOT .Contracts: that is deliberately transport-free and MqttClientOptions is an MQTTnet type, so .Contracts would hand MQTTnet to ControlPlane. This is the fix the .Browser csproj comment had already prescribed, against the issue's suggestion. - The Sparkplug protocol layer (SparkplugTopic/MessageType, SparkplugCodec, RebirthRequester) -> .Contracts, beside the Tahu wire schema it encodes. Still exactly one copy of the TLS/CA-pin logic; MqttBrowserLayeringTests guards the graph both ways. Measured, not assumed: Driver.Mqtt.dll, Polly.Core and Serilog are now absent from the AdminUI's output. Core.dll remains, because Driver.Sql.Browser has the SAME violation, undocumented — the old comment blamed this browser for Core and was wrong. Filed separately rather than fixed in passing. Full suite green apart from the 7 known pre-existing failures (4 AbLegacy integration, compose profiles not up; 3 Host.IntegrationTests). Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
2d20aeb779 |
fix(opcua,drivers): carry driver status sub-codes to clients (#519) + health-write ordering (#517)
#519 — a driver's OPC UA status code now reaches a client with its sub-code intact. DriverInstanceActor projected the driver's uint onto a 3-state Good/Uncertain/Bad enum and OtOpcUaNodeManager re-expanded Bad to a bare 0x80000000, so BadNoCommunication, BadTypeMismatch and BadNotSupported were indistinguishable on the wire and every driver's status constants (incl. the 16 #497 had just corrected) were diagnostic in logs only. Resolved by option (b): the raw uint rides AttributeValuePublished -> AttributeValueUpdate -> IOpcUaAddressSpaceSink.WriteValue, OpcUaQuality is deleted, and severity classification is a helper (OpcUaStatus.IsGood/IsUncertain/IsBad). The surface was wider than the issue costed (41 files, not "the sink and 3 implementations"): OpcUaQuality had non-driver producers with no uint to carry. Each got its natural code — BadInternalError for a failed virtual-tag script, BadNoCommunication for driver comms loss — and ScriptedAlarmHostActor's pair of mutually-inverse uint<->enum mappers is deleted, having round-tripped a code the engine already held. AddressSpaceApplier's removal writes deliberately keep the generic Bad: not losing a code the driver had is this issue; inventing one for the deploy path is a separate call. Behaviour change: WriteAlarmQuality fires its Part 9 event on a status-code change rather than a severity-bucket change. While the annotation was 3-state those were the same predicate; now a move between two Bad reasons changes what a subscriber reads. New SDK-free named constants live on Commons.OpcUa.OpcUaStatus, and StatusCodeParityTests is widened to cover them — that guard is reflection-driven, so it can only see a named constant, never an inline literal. #517 — the symptom is real but the issue named the wrong path and its fix was unsafe. ShutdownAsync cannot race: DriverInstanceActor calls it from PostStop, after the mailbox drains. (Core/Hosting/DriverHost, the one class with a concurrent GetHealth, has no production call site.) InitializeAsync can: it runs on a fire-and-forget Task.Run, and MTConnectDriver writes Healthy then starts loops that can throw into a catch which disposes the client before writing Faulted. Moving the write earlier, as suggested, inverts the terminal state — teardown drains in-flight loops that write health as they unwind, so a late loop could leave Healthy at rest with every connection closed. The fix is a bracket (conservative before, terminal after), applied to MTConnect's init-failure path only; shutdown keeps its single trailing write, and the rule plus the reachability table are documented on IDriver.ShutdownAsync. CalculationDriver.GetHealth returned a hard-coded Healthy that consulted nothing — Healthy before initialize, after shutdown and after dispose. Now derived from _running. Its test, named Health_is_always_Connected, asserted the defect and is replaced. Every new guard was falsified by breaking the production code and watching it go red. The two sub-code tests each assert two different Bad codes, since a single-code assertion could pass against a hard-coded constant under the old projection. Pre-existing failures, verified unchanged on a stashed tree and not chased: 4 in Driver.AbLegacy.IntegrationTests (slc500/micrologix/plc5 fixtures not up) and 3 in Host.IntegrationTests. Closes #517 Closes #519 Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo |
||
|
Joseph Doherty
|
4624141a5d |
test+docs: retire the stale EquipmentTagsDarkBatch4 skips, guard the deploy-time
tag gate, and fix three silently-broken OpcUaClient integration tests
Continues working through deferment.md's remaining items.
SKIPPED TESTS (13 -> 0 for this reason). The skip said "dark until Batch 4";
Batch 4 shipped as v3.0 and RETIRED the equipment-tag path rather than lighting
it, so every one of them was waiting on something that had already happened and
gone the other way. Resolved individually rather than in bulk:
- REVIVED onto the raw/UNS shape (6): four DriverHostActor primary-gate cases,
the cluster-scoping rebuild, and the real-SDK dual-namespace materialisation
E2E. The behaviour never went dark, only the v2 seeding did.
- FOLDED into a live parity test (3): DeploymentArtifactRawUnsParityTests
already compared RawTagPlan record-equal and RawTagPlan carries array /
historize / alarm intent, so widening its corpus by two tags covers what
three empty placeholder suites had been promising. Those suites are deleted.
- DELETED (4): subjects genuinely retired -- equipment-namespace EquipmentTags,
and both dot-joint {{equip}}.X token suites (that resolver was deleted in
Batch 3; the slash-joint form is covered by DeploymentArtifactEquipRefParityTests).
Falsified, not assumed: removing the seeded UnsTagReference turns two revived
primary-gate tests red; re-homing the SITE-A node to MAIN turns the scoping test
red. Widening a parity corpus also needed direct value assertions -- parity alone
cannot distinguish "both seams right" from "both seams blind".
Runtime.Tests skips 13->3, OpcUaServer.Tests 4->1; all remaining are deliberate
EquipmentDeviceBindingRetired tombstones.
G-7 (deploy-time TagConfig gate). MQTT shipped an Inspect() nobody ever called;
now wired. Replaced the hand-maintained list with a reflection guard, because the
existing test enumerates driver types by hand and so guards renames but can never
notice a gap. NOTE: the first version of that guard was hollow -- it discovered
candidates via GetReferencedAssemblies(), which is circular, since the compiler
elides a reference nothing in the IL uses. Removing MQTT from the map also removed
its Contracts assembly from the manifest, and the guard passed green with the bug
reintroduced. Rewritten to scan the output directory; re-falsified and it now
fails naming MqttTagDefinitionFactory. Residual recorded at the code: Sql,
MTConnect, Calculation and OpcUaClient have no Contracts-side Inspect at all, so
the deploy gate stays weaker than the AdminUI's authoring gate for them.
OPCUACLIENT INTEGRATION TESTS (3 of 4 failing, on master too). Not fixture rot:
the simulator was healthy and Client.CLI read the very same node Good. v3 made the
driver resolve every reference through its authored RawTags table, so a bare
"ns=3;s=StepUp" fails LOCALLY at the resolver with BadNodeIdInvalid and never
reaches the wire -- the tests were exercising the unresolved-reference guard.
Fixed by seeding OpcPlcProfile.RawTags in the deploy artifact's TagConfig shape
and addressing by RawPath. 4/4 pass.
DOCS. Applied the Phase7* -> AddressSpace* rename across the four live docs
(historical bannered files deliberately untouched), resolving the three that are
not renames to their real members. Found en route that the earlier banner pass
missed three files: VirtualTags.md, OpcUaServer.md and ScriptedAlarms.md all
still presented the test-scaffolding GenericDriverNodeManager as the production
dispatch path. All three now carry the correction.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
|
||
|
Joseph Doherty
|
5184a2e107 |
fix(drivers): complete the Sql + FOCAS in-place config re-parse (#516)
The first #516 pass deliberately left these two out, because each derives more than options from config: Sql its ISqlDialect and resolved connection string, FOCAS the IFocasClientFactory its Backend key selects — all injected at construction. Adopting new options alone would poll a NEW tag set through the OLD database/backend, and a half-applied config change is worse than a discarded one because it looks like it worked. Rather than accept that exclusion, this fixes the blocker. Each factory now exposes a ParseBinding returning EVERY config-derived dependency as one value, and the driver adopts them atomically: - Sql: ApplyBinding takes (options, dialect, connectionString) and rebuilds everything downstream — provider factory, Endpoint, and the SqlPollReader that captures all three. It runs BEFORE BuildTagTable, so the tag table and the connection it is polled over always come from the same revision. A test-injected DbProviderFactory survives a rebind (_explicitFactory), so a re-derived dialect cannot displace what a test passed in. - FOCAS: options and backend move as a pair in InitializeAsync. Re-resolving Sql's connection string on reinit is a side benefit: a rotated credential is picked up without a process restart. Drivers constructed directly get no rebinder and keep their constructor binding, so every existing "{}"-passing lifecycle test is unaffected by construction rather than by luck. The tests pin ATOMICITY, not merely that a re-parse happened — a test checking only options would have passed against the broken version. Confirmed by simulating the half-fix (adopt options, skip the backend): 2 of 3 FOCAS tests go red. Reverting Sql's rebind turns its connection-string test red. Sql also asserts that a reinit which cannot resolve its new connection leaves the previous binding whole rather than half-adopting. All 12 drivers now honour a changed config in place. Sql.Tests 226, FOCAS.Tests 275 pass. |
||
|
Joseph Doherty
|
d32d89c340 |
fix(drivers): stop silently discarding driver config edits (§8.3, #516)
Five drivers ignored the config handed to them on reinitialize, serving the
options their constructor captured — and the deployment sealed green anyway.
Fixed on both halves, as chosen.
Seam (load-bearing): DriverSpawnPlanner routes a changed DriverConfig to
ToStop + ToSpawn instead of an in-place delta, making the factory the single
parse authority. This REVERSES the deliberate decision documented at
DriverSpawnPlan.cs:49-50 ("a pure DriverConfig change stays an in-place delta
— no reconnect"). That reasoning was correct about the resilience pipeline and
wrong about the driver. The accepted price is a reconnect per config edit.
Per-driver, and this split was not anticipated:
- Re-parse in place — Modbus, AbLegacy, OpcUaClient. ParseOptions extracted
from each factory, called from InitializeAsync behind a HasConfigBody guard
so "{}" still keeps the constructor options.
- Respawn-only, deliberately NOT re-parsed — Sql and FOCAS. Each builds more
than options from config (Sql's dialect + resolved connection string,
FOCAS's client-factory backend, both injected at construction), so adopting
new options alone would run a NEW tag set against an OLD connection. Half a
re-parse is worse than none. SqlDriver's doc-comment asserted the opposite
premise — "config parsing belongs to the factory, which builds a fresh
instance" — which was false when written and is true only now.
Two green seals removed from ApplyChildDelta: it overwrote the cached Spec
synchronously BEFORE the child dequeued the message, so the host believed the
new config was live and the next reconcile computed no delta, sealing the
drift permanently; and it Tell'd with no Receive<ApplyResult> registered, so a
failed reinit — including Galaxy's deliberate NotSupportedException —
dead-lettered.
Exposed (not created) by the change: a factory throw is a CONFIG error, and
SpawnChild catches it and silently substitutes a stub. Only a brand-new driver
could reach that before; an ordinary config edit can now. Raised Warning ->
Error with an actionable message. It still does not fail the deployment —
doing so would let one malformed driver block a fleet deploy, so that is a
follow-up rather than a drive-by.
Tests: every pre-existing reinit test in these suites passes "{}", the exact
input a guarded re-parser treats as "keep constructor options" — blind to this
defect by construction. New tests pass CHANGED json; the Modbus one was
verified falsifiable by deleting the re-parse (goes red). Two tests asserted
the old behaviour and were rewritten, including the positive control in
DriverHostActorUnreadableArtifactTests, whose observable moved from
UnsubscribeAsync to ShutdownAsync now that teardown happens by stopping the
child rather than emptying its desired set.
Remaining full-suite failures are pre-existing and environmental, verified
identical on the pre-change tree: Host.IntegrationTests (3) and
Driver.AbLegacy.IntegrationTests (4, docker fixture-gated).
|
||
|
Joseph Doherty
|
cf9ce91e51 |
test(mtconnect): live-Agent docker fixture + integration suite (Tasks 19+20)
The first and only thing in the MTConnect workstream that exercises the driver
against a real Agent; everything else runs on canned XML.
Fixture (Docker/): two services, both stock images with this folder bind-mounted.
- `agent` mtconnect/agent:2.7.0.12 on :5000. NOTE the repo name: the source
project is mtconnect/cppagent but the PUBLISHED image is
mtconnect/agent; mtconnect/cppagent does not exist on Docker Hub.
- `adapter` a stdlib SHDR feeder. Required, not optional: the agent image ships
only the binary + schemas/styles, so a lone Agent answers /probe and
then reports every observation UNAVAILABLE forever, which can prove
nothing about reads or streaming.
Devices.xml seeds one DataItem per inference branch (SAMPLE+units, TIME_SERIES,
PART_COUNT/LINE_NUMBER, controlled vocab, DATA_SET, CONDITION), gives every named
item `name != id` (the inverse of the hand-authored unit fixtures), and leaves one
EVENT permanently unfed so UNAVAILABLE -> BadNoCommunication has a live subject.
Suite (12 tests) asserts STRUCTURALLY against the Agent's own /probe response, never
by literal id, so it survives an edit to Devices.xml and can be pointed at a real
machine tool via MTCONNECT_AGENT_ENDPOINT. It skips cleanly (12/12) when no Agent
answers, and each test carries a hard [Fact(Timeout)] so a half-up fixture cannot
wedge a build.
Three findings from bringing the fixture up, each now pinned in a comment:
- agent.cfg must be pure ASCII; one non-ASCII byte in a COMMENT makes the config
parser reject the whole file with a bare "Failed / Stopped at line: N" and exit.
- `sampleCount` is an attribute of a TIME_SERIES observation, not of a DataItem
declaration. A real Agent meeting one on a declaration DROPS THE ENTIRE DATA ITEM
from the device model, so the inference only ever sees null and a live TIME_SERIES
tag is always variable-length. Asserted, not ignored.
- The Agent's own <Agent> self-model publishes update-rate SAMPLEs that tick whether
or not any adapter is attached. Excluding them is load-bearing: with them included
the "live stream delivered a changed value" test passed with the fixture's data
source deliberately stopped.
MTConnectError-under-HTTP-200 is NOT covered live and cannot be: cppagent 2.7 answers
an unknown device 404 and an out-of-range sequence 400, each with a well-formed error
body. That shape stays canned-only, and the suite says so.
|
||
|
Joseph Doherty
|
51c8c7f30b |
fix(mtconnect): bind RawTags and key the data plane by RawPath
The driver could not receive data after a real deploy. The runtime hands every
driver its authored tags as a RawTags array (RawPath identity + TagConfig blob,
injected by DriverDeviceConfigMerger) and then subscribes, reads, and routes
published values by RawPath. MTConnectDriverConfigDto declared no RawTags
property, so System.Text.Json silently dropped the array, and every reference
the server passed missed an index keyed by DataItem id: every value came back
BadNodeIdUnknown, with the whole suite green because every test handed the
driver its own Tags array directly.
- MTConnectDriverOptions gains RawTags; the config DTO binds it (verbatim — a
bad blob is skipped per-tag at binding time, never fails the deployment).
- A new TagBinding, derived from the options and swapped with them in one write,
carries RawPath -> dataItemId, its one-to-many reverse, and the definition set
the observation index coerces against. Subscribe/Read resolve through it; the
fan-out expands each touched dataItemId into every RawPath bound to it and
publishes under THAT reference — the line the blocker turned on.
- Coercion type precedence: the blob's driverDataType/dataType (both spellings —
the AdminUI's MTConnectTagConfigModel writes the latter), else a Tags entry for
the same DataItem, else the Agent's own /probe declaration via
MTConnectDataTypeInference (what makes a browse-committed {"address": id} blob
publish as a number), else String. A present-but-invalid type rejects that tag.
- An unmapped reference falls through unchanged, so the dataItemId-keyed CLI
surface and every existing test keep working, and an unclaimed RawPath reports
Bad rather than throwing.
15 new tests build their config through the real DriverDeviceConfigMerger; all
four mutations (publish the dataItemId, drop the RawTags binding, drop the
derived coercion type, drop the reverse-map expansion) are caught.
|
||
|
Joseph Doherty
|
c232a3ce67 |
fix(mtconnect): cap CancelAfter overflow + stop leaking ex.Message in probe fallback (review follow-up)
Two review findings on Task 14's MTConnectDriverProbe, both red-first:
- CancelAfter(effectiveTimeout) sat before the try block with only a low-end
floor on a non-positive timeout, never a high-end cap. A pathological
timeout (e.g. TimeSpan.FromDays(1000)) threw an uncaught
ArgumentOutOfRangeException straight out of CancelAfter's ~49.7-day legal
range, violating the probe's own "Never throws" contract. Not reachable
through today's only caller (AdminOperationsActor clamps to [1,60]s and
wraps the call), but a landmine for any future caller without that clamp.
Fixed by clamping effectiveTimeout to a new MaxTimeout (10 minutes).
- The final `catch (Exception ex) { return new(false, ex.Message, null); }`
forwarded ex.Message verbatim for any exception type not enumerated above
it, breaking the redaction discipline every other catch was audited for
(AgentUri may carry userinfo credentials). Not exploitable by any
currently-reachable exception type, but an open door for a future one.
Fixed to build a message from the already-redacted safeUri + the
exception's type name instead of its message.
Also: a genuine non-2xx stub-handler test (prior HttpRequestException
coverage was connection-refused only), a credentialed-agentUri +
successful-probe redaction test, and a one-line doc remark on the accepted
reachable-vs-deployable gap.
|
||
|
Joseph Doherty
|
ba89d6d0ff |
feat(mtconnect): driver factory delegating to the single config parser (Task 15)
The factory carries NO config DTO of its own: CreateInstance delegates to MTConnectDriver.ParseOptions, the single authority for the config document. A second DTO would drift from the parse the runtime's ReinitializeAsync path uses, and the drift would only surface at deploy time. Construction is connection-free (agentClientFactory: null) because the Wave-0 universal browser builds a throwaway instance per browse probe, and the returned instance is never narrowed — its five capability interfaces ARE the runtime's dispatch surface. Pinned by tests asserting all five plus the deliberate absence of IWritable. DriverTypeName is the literal "MTConnect"; Task 16 adds DriverTypeNames.MTConnect and the host registration that must equal it. |
||
|
Joseph Doherty
|
23cd47d54b | feat(mtconnect): IDriverProbe one-shot /probe reachability (Task 14) | ||
|
Joseph Doherty
|
dfbcb0e7f0 |
fix(mtconnect): close the silent-refusal + teardown-wedge defects in the pump (Task 11 review)
C1 (critical): a latched "this endpoint cannot stream" verdict could end with a GREEN driver holding a permanently silent subscription. DriverInstanceActor handles a tag-set change as Unsubscribe-then-Subscribe with no re-initialize; the unsubscribe cleared the stream-degraded flag, the resubscribe was refused by the latch without a word, and one successful read then reported Healthy. StartSampleStreamCore now re-asserts the degradation and logs a Warning naming the remedy, and teardown no longer clears the flag while the latch is set. NOT promoted to Faulted: DriverHealthReport maps any Faulted driver to a /readyz 503, so that would de-ready the whole node over one misconfigured Agent whose reads are perfectly healthy. Faulted stays for the case that earns it. I1: teardown waits are bounded (5 s) and honour the caller's token. They were unbounded and ignored it, so DriverInstanceActor's 5 s budget was inert and PostStop — which blocks a dispatcher thread on ShutdownAsync while holding the lifecycle semaphore — could be wedged forever by one blocking subscriber. The cancellation source is disposed only when the loop is provably gone. Applied to StopProbeLoopAsync too: Task 13 reproduced the identical shape, and closing the class in one method while leaving the other as the pattern to copy is worse than not closing it. I2: the reconnect ladder was monotonic for the process lifetime, so ~9 unrelated drops pinned the driver at MaxBackoffMs forever. A stream that delivered before dropping now starts a fresh ladder (at attempt 1, not 0, so MinBackoffMs is still honoured). I3: MaxBackoffMs <= 0 clamped every delay to zero — an operator-authorable reconnect spin. Treated as unset, like a multiplier that cannot grow. I4: the session published instanceId without NextSequence, so after a restart it held the new agent's id beside the dead one's cursor, and a gap/OUT_OF_RANGE re-baseline (id unchanged) never published the cursor at all. Both move in one CAS, and the pump publishes its cursor as it exits. I5: OnDataChange was a multicast Invoke — the first throwing handler aborted the rest of the list, starving every later subscriber. Now walked by hand with the catch inside the loop. The test that claimed to cover this registered the recorder BEFORE the thrower, so it passed regardless; swapped, it was red. Faults are latched to one Warning per stream generation (Debug thereafter) so a consistently-throwing consumer cannot flood the log. Also: the pump restarts after an unexpected fault (IsCompleted, not null); a device-scope that matches nothing is a Warning, not a Debug tally; a device or component with neither name nor id is skipped rather than emitting a blank path segment; DiscoverAsync's remark no longer claims DriverInstanceActor retries (it does not for a Once driver, and injection is dormant in v3); and two flake seeds in the fake are gone (Dispose re-read its own counter; _disposeCts was disposed under a racing SampleAsync). 439/439. Every changed behaviour falsified by mutation. |
||
|
Joseph Doherty
|
fa0e40796f | feat(mtconnect): host-connectivity probe + instanceId rediscover (Task 13) | ||
|
Joseph Doherty
|
13dd9fcd70 | feat(mtconnect): ITagDiscovery streams tree + SupportsOnlineDiscovery opt-in (Task 12) | ||
|
Joseph Doherty
|
9a67ed918a |
feat(mtconnect): ISubscribable /sample pump + ring-buffer re-baseline (Task 11)
One shared /sample long poll behind every subscription handle. Subscribe fires initial data per reference from the primed /current (Part 4 convention) and returns without waiting on the stream; the pump runs on its own task under its own token — never the caller's Subscribe token, which is disposed the moment that call returns. The pump owns every way the sequence can break: - sequence gap, measured against the RUNNING cursor (the previous chunk's nextSequence). Comparing against the opening `from` reports a gap on every chunk once the buffer rolls — a /current re-baseline storm. - OUT_OF_RANGE under HTTP 200 (InvalidDataException out of SampleAsync), which IsSequenceGap cannot see at all — without it a driver recovers from falling a little behind but not from falling a lot. - agent restart, checked BEFORE the gap because a restart resets sequences and so usually trips the gap check too; it clears the index (the held values describe a device model that no longer exists) and tells the subscribers. - every chunk advances the cursor, heartbeats included. Re-baseline calls CurrentAsync directly on the client the pump already holds: routing it through ReinitializeAsync would take the non-reentrant lifecycle semaphore from inside a pump a lifecycle method may already be awaiting, and hang the driver with no exception and no log. StopSampleStreamAsync is filled in (same call sites) and InitializeAsync now stops the stream before teardown too, so a re-Initialize on a live instance cannot leave a pump enumerating a disposed client. MTConnectStreamEnded/TimeoutException reconnect under a geometric backoff with a 100 ms growth floor (MinBackoffMs defaults to 0, and 0 x multiplier is still 0). MTConnectStreamNotSupportedException does NOT retry — it latches, reports loudly, and is cleared only by a re-initialize. Health precedence is explicit: /current and /sample fail independently, so each path owns a flag, clears only its own, and Healthy requires both clear. Neither can paper over the other's failure. 29 new tests, all deterministic — no sleeps, no wall-clock assertions. The fake grew per-generation sample streams (so a reconnect has somewhere to land), scripted /current answers, scripted sample failures, and a chunk-consumed signal that also fires when the consumer abandons the stream. 375/375 green. |
||
|
Joseph Doherty
|
a127954cab |
fix(mtconnect): review remediation for the driver lifecycle (Task 9)
Important 1 - the probe-model triple was read outside _lifecycle while every
write happened under it. Packed (Client, ProbeModel, ProbeModelDataItemCount)
into one immutable AgentSession swapped by a single Volatile.Write, matching
what _index already does, rather than locking the readers: both await the
network, so taking _lifecycle would stall a shutdown for a whole
RequestTimeoutMs and would widen the non-reentrancy hazard. GetOrFetch and
Flush re-publish via CompareExchange so a late re-cache cannot undo a
concurrent teardown/re-init, and a client disposed mid-fetch now surfaces as
the same "not connected" InvalidOperationException browse already handles
instead of a raw ObjectDisposedException.
Important 2 - HasConfigBody decided emptiness by matching the literals "{}" /
"[]", so "{ }", "{\n}" and every pretty-printed empty document fell through to
ParseOptions, failed the required-AgentUri check, and turned a semantically
empty config into a cold-start fault. Now parsed: an object/array with no
elements (or a bare null) is empty; malformed text is deliberately NOT empty so
ParseOptions produces the real quoted error rather than silently starting on
stale options.
Important 3 - documented the _lifecycle non-reentrancy hazard in the code, on
both the semaphore and StopSampleStreamAsync, naming Task 11's re-baseline as
the specific path that would deadlock and giving the CurrentAsync-directly
pattern that avoids it.
Important 4 - removed the Initialize/Reinitialize asymmetry rather than
documenting it. Both now share one rule via ResolveIncomingOptions: an
unreadable config document never destroys working state, and faults only a
driver that had none. InitializeAsync previously tore down before parsing, so a
bad document on a live instance destroyed a healthy client - the exact outcome
ReinitializeAsync was written to avoid.
Minors: SafeDispose traces the swallowed disposal fault at Debug (a client that
cannot be released is how a handle leak starts); the RequirePositive test is a
Theory over all four timing knobs, not just RequestTimeoutMs (arch-review
01/S-6); the footprint test no longer overclaims "is zero before initialize".
CannedAgentClient gains a one-shot ProbeGate so a lifecycle change landing
mid-request is deterministic - no timers. 346/346 (329 + 17). Six mutations
verified: fetch-CAS, disposed-translation, flush retired-session guard,
literal HasConfigBody, teardown-before-parse, and two dropped RequirePositive
calls each fail only their own tests.
|
||
|
Joseph Doherty
|
1bfc1722b3 |
feat(mtconnect): IReadable via /current, ordered per-ref snapshots (Task 10)
One deadline-bounded /current per batch, answered out of that one whole-device document: N references cost one round-trip, one snapshot each, in request order, duplicates included. DEVIATION (deliberate, from the plan's implied shared-index write): the response is indexed into a PER-CALL snapshot index, never into the driver's shared observation index. That index is written by Task 11's /sample pump and is last-write-wins with no timestamp arbitration; a read folding its /current into it would be a second writer racing the first, and /current is frequently the OLDER document (the Agent composes it while the pump's newer delta is in flight). Losing that race rolls a subscribed value backwards and leaves it there until the data item next changes. Both paths coerce through the same MTConnectObservationIndex logic, so a read and a subscription report a given observation identically -- the read path is simply a pure function of (document, authored tags). Failure posture is the inverse of InitializeAsync: nothing but genuine caller cancellation crosses the capability boundary. Unreachable Agent / blown deadline / unusable answer => every ref codes BadCommunicationError and the driver degrades (never downgrading Faulted); no client at all (pre-initialize, faulted, post-shutdown) => BadNotConnected with health untouched. The index's BadWaitingForInitialData / BadNodeIdUnknown / BadNoCommunication / BadNotSupported distinctions pass through unmodified. 21 tests pinning arity, order, duplicates, empty batch, one-round-trip, each Bad code, read-time freshness, the anti-clobber invariant, the deadline, and cancellation-propagates-but-agent-failure-does-not. CannedAgentClient gains a cancellation-observing CurrentGate (still no timers or sleeps of its own). Test csproj takes the AbCip/S7 OTOPCUA0001 NoWarn -- this suite calls the capability directly by design. 329/329 green. |
||
|
Joseph Doherty
|
4fe0af3693 |
feat(mtconnect): driver shell IDriver lifecycle over agent-client seam (Task 9)
MTConnectDriver implements IDriver only: connection-free ctor, initialize (deadline-bounded /probe + /current, caching the device model, the agent instanceId, and a primed observation index), reinitialize, shutdown, health, footprint, and cache flush. Decisions worth knowing: - The driverConfigJson argument is HONOURED, not ignored. DriverInstanceActor delivers a config change only via ReinitializeAsync(json) on the live instance, so a driver reading only its ctor options would turn every MTConnect config edit into a silent no-op that still sealed the deployment green. ParseOptions lives on the driver; Task 15's factory must delegate to it rather than deserialize a second DTO. - /probe ok but priming /current failing is Faulted, not Healthy-with-no-data: the index IS the read surface and the /sample cursor comes from /current. - ReinitializeAsync rebuilds the client whenever ANY option the client reads at construction changed - AgentUri, DeviceName, RequestTimeoutMs, HeartbeatMs, SampleIntervalMs, SampleCount - not just the first two. The timing knobs are baked into the deadlines, the watchdog window, and the /sample query string. - An unparseable config faults a cold start but leaves a RUNNING driver serving its previous valid configuration (the deployment fails instead). - IMTConnectAgentClient now extends IDisposable rather than the driver doing `as IDisposable`, which would silently no-op against a non-disposable fake and leak an HttpClient + socket handler per re-deploy. Sync, not async: every resource behind the seam is sync-disposable, and the async unwind belongs to the SampleAsync enumerator the pump owns. - Flush drops the probe/browse cache and keeps the index; every model consumer goes through GetOrFetchProbeModelAsync so a flush cannot wedge browse. Tests: CannedAgentClient (shared, deterministic - channel-backed scripted /sample with no timers, call counts, disposal tracking, failure injection) + 27 lifecycle tests. 308/308 in the project. |
||
|
Joseph Doherty
|
908bd7ba4a |
fix(mtconnect): deterministic concurrency test + DATA_SET -> BadNotSupported
The concurrency test asserted a post-Apply post-condition from inside the race window: dev1_pos is authored, so a reader beating the first Apply correctly observes BadWaitingForInitialData, which the test's two-member "legal" set omitted. Consistently red (10/10) rather than intermittent. Rewritten to assert only properties that hold at every instant — the whole (status, value, source timestamp) triple against the three legal snapshots, which is the real torn-snapshot check — plus a read counter guarding a vacuous pass and one deterministic closing Apply for an exact end state. The index is unchanged here; BadWaitingForInitialData is a designed state. Also codes structured (DATA_SET / TABLE) observations BadNotSupported now that the parser supplies MTConnectObservation.IsStructured, so a two-entry data set no longer surfaces as the Good string "12". |
||
|
Joseph Doherty
|
5ba9f1be6f |
fix(mtconnect): make a /sample stream end loud, and correct the gap contract (Task 7 review)
Code review of c46540ae approved the framing algorithm but moved the risk onto the contract around it. C1 (critical). SampleAsync returned normally on three different events: EOF, the closing --boundary--, and a response that was not multipart at all (where it yielded one parsed snapshot and finished). The seam documents the stream as yielding until ct is cancelled, so a Task 11 pump written to that contract would silently drop its subscription or hot-loop reconnecting; the non-multipart leg reported a configuration error as a healthy finished stream, which is the #485 quiet-successful-termination shape aimed at the very next task. Every non-cancellation end now throws: MTConnectStreamEndedException (ConnectionClosed / ClosingBoundary - transient) or MTConnectStreamNotSupportedException (configuration - reconnecting reproduces it forever), sharing one catchable base. The non-multipart body is still parsed first so an Agent's own MTConnectError text wins, but the document is NOT yielded. I1. IsSequenceGap moved to IMTConnectAgentClient (static) and its first parameter is now expectedFrom. The old name and doc were wrong for every chunk after the first - the correct comparand is the PREVIOUS chunk's NextSequence, and comparing against the opening "from" argument reports a gap on every chunk once the ring buffer rolls, i.e. an endless /current re-baseline storm. Both doc blocks also now record that cppagent answers from < firstSequence with an OUT_OF_RANGE MTConnectError under HTTP 200, which surfaces as InvalidDataException and NOT as a gap - so Task 11 must treat that parse failure as a re-baseline trigger too. I2. Every multipart test served the whole body from one buffer, so every framing test completed in a single ReadAsync - the split-boundary path, the split-header path and the multi-fill loop were correct by inspection only. A ChunkedStream double (N bytes per read, N = 1/3/7) now drives the fixtures through a split transport, plus a >16 KB document that forces a buffer resize. Falsifiability: removing either scan overlap, or collapsing the fill loop, fails ONLY the new chunked tests. I3. Disposal mid-enumeration surfaces as OperationCanceledException via an internal dispose-linked token, not a raw ObjectDisposedException that Task 9's re-init would inflict on an enumerating pump. I5. HeartbeatMs, SampleIntervalMs and SampleCount join RequestTimeoutMs in construction-time positive-value validation. All three reach the query string and an Agent answers heartbeat=0 with an HTTP-200 MTConnectError that names no config key. I4/M2. The no-Content-length framing fallback logs a one-shot warning (the client takes an optional ILogger); a multipart response with no boundary parameter fails fast instead of degrading into a watchdog timeout. M5/M6. Shared element/attribute reading rules extracted to MTConnectXml; the probe parser documents why it alone does not trim BOM/whitespace. The literal U+FEFF in source became a '' escape. Also, from Task 8: MTConnectObservation gained IsStructured (default false), set from element.HasElements. A DATA_SET/TABLE observation's <Entry> children concatenate through element.Value to nonsense ("12" for two entries) that the index would publish as Good, and only the parser can tell that apart from a legitimate space-bearing Message. False for CONDITION observations - their value comes from the element name, so child content cannot corrupt it. 275/275 green in this suite's own files. |
||
|
Joseph Doherty
|
ac0a284055 | feat(mtconnect): observation index + UNAVAILABLE->BadNoCommunication (Task 8) | ||
|
Joseph Doherty
|
cf43f8d0ab |
feat(mtconnect): current/sample parse + sequence-gap detection (Task 7)
Adds the /current and /sample legs to MTConnectAgentClient: - MTConnectStreamsParser: hand-rolled System.Xml.Linq parse of an MTConnectStreams document into MTConnectStreamsResult. Observations are every element child of <Samples>/<Events>/<Condition>, keyed by dataItemId (the element NAME is the data item's type in PascalCase, so a fixed name set would drop observations). A CONDITION's value is its element name, not its text - <Normal/> is empty - and <Unavailable/> normalizes onto the same UNAVAILABLE sentinel a Sample/Event carries as text, so Task 8's no-comms mapping sees one token. Timestamps are normalized to DateTimeKind.Utc explicitly. - IsSequenceGap(requestedFrom, chunk) => chunk.FirstSequence > requestedFrom, so Task 11's pump can re-baseline after a ring-buffer overflow. Equality is NOT a gap. - SampleAsync: ResponseHeadersRead + a hand-rolled multipart/x-mixed-replace frame reader (Content-length fast path, boundary-scan fallback), bounded by a heartbeat watchdog derived from HeartbeatMs so a silent Agent faults instead of wedging the pump - the S7 frozen-peer shape (arch-review R2-01). The long poll gets its own HttpClient with an infinite Timeout so the unary /probe + /current deadline stays bounded. Task 0 package decision, option (c): both TrakHound PackageReferences and their PackageVersions are dropped. MTConnectHttpClientStream takes a URL and dials its own socket (no handler/stream seam, so it cannot serve a socket-free unit suite), and it emits already-parsed documents through the formatter lookup Task 6 proved is missing - framing-only reuse was never actually on offer. The driver now depends on nothing but the BCL. Recorded in the plan's Task 0 CORRECTION block. Also folds in two Task 6 review carry-overs: the stale IMTConnectAgentClient doc comment claiming a TrakHound-backed implementation, and a probe test constructing a mixed-namespace vendor extension (<x:Pump> with default-namespace <DataItems> children) that backs the parser's ignore-the-namespace claim with a test. 187/187 green. |
||
|
Joseph Doherty
|
1990d21733 | feat(mtconnect): agent client /probe parse to device model (Task 6) | ||
|
Joseph Doherty
|
b13cf5c6e8 | fix(mtconnect): options carry the authored tag definitions (Task 2 follow-up) |