9227d03ca88a551217fe022b753ae923ef565672
2889 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
9227d03ca8 |
feat(mtconnect): make an MTConnect driver creatable + configurable in /raw (Task 18 Part B)
An MTConnect driver instance could not be authored at all: RawDriverTypeDialog
(the only DriverInstance creation surface) offered a hardcoded 9-entry list
without it, and DriverConfigModal's default: branch renders a warning with no
raw-JSON fallback — so even a hand-inserted row could not be configured and
DriverConfig stayed "{}", on which ParseOptions throws "missing required
AgentUri".
Adds the typed MTConnectDriverForm (the consistent sibling pattern) covering
agentUri / deviceName / the four polling knobs / probe / reconnect, wires the
dispatch case, and offers the type in the dialog.
- All decisions live in the static MTConnectDriverForm.BuildConfigJson +
MTConnectFormModel (plain C#), because this project has no bUnit.
- The wire shape mirrors the driver's private MTConnectDriverConfigDto, NOT
MTConnectDriverOptions: the latter's TimeSpan probe knobs would serialise as
"00:00:05" and never bind to the driver's integer intervalMs.
- arch-review 01/S-6: a non-positive timing knob is REMOVED from the document
rather than written, so ParseOptions substitutes the driver's own positive
default instead of the driver faulting at Initialize on RequirePositive.
Removal (not just skipping) also stops a stale positive value surviving and
disagreeing with what the form shows.
- Unknown top-level keys survive a load/save, so the driver's hand-authored
tags[] surface is not silently deleted by opening the form.
- _jsonOpts carries JsonStringEnumConverter, satisfying the
DriverPageJsonConverterTests fleet guard properly rather than by exemption.
New RawDriverTypeDialogCoverageTests resolves the offered set through
DriverTypeNames.All, so the next registered driver fails here until the dialog
offers it.
|
||
|
Joseph Doherty
|
115a43dd47 |
feat(mtconnect): typed AdminUI tag-config editor + browse round-trip (Task 18 Part A)
Registers the MTConnect typed tag editor in TagConfigEditorMap /
TagConfigValidator (keyed off DriverTypeNames.MTConnect, never a literal) and
adds the razor shell over Task 17's MTConnectTagConfigModel.
The shell stays trivial: the /probe-sourced mtCategory/mtType/mtSubType/units
row is rendered disabled + readonly with NO binding of any kind, and no
inferred-type hint is shown — MTConnectDataTypeInference.Infer() also keys on
the DataItem `representation`, which the tag-config model does not carry, so
calling it here would agree with the browse picker for ordinary items and
silently disagree for exactly the DATA_SET/TABLE/TIME_SERIES ones.
Also fixes the picker -> editor round trip, which was broken in both
directions:
- RawBrowseCommitMapper.BuildTagConfig had no MTConnect branch, so a committed
tag landed as the generic {"address": "<dataItemId>"} while the model read
only `fullName`. The data plane kept working (the driver accepts all three id
spellings), so the only symptom was an empty DataItem-id field in the editor
— and saving from that state stamped fullName:"" over the tag.
- MTConnectTagConfigModel.FromJson now accepts dataItemId/address as fallbacks
(fixing tags committed before this change) and ToJson normalises onto
`fullName`, dropping the aliases so the two spellings cannot drift.
|
||
|
Joseph Doherty
|
51c8c7f30b |
fix(mtconnect): bind RawTags and key the data plane by RawPath
The driver could not receive data after a real deploy. The runtime hands every
driver its authored tags as a RawTags array (RawPath identity + TagConfig blob,
injected by DriverDeviceConfigMerger) and then subscribes, reads, and routes
published values by RawPath. MTConnectDriverConfigDto declared no RawTags
property, so System.Text.Json silently dropped the array, and every reference
the server passed missed an index keyed by DataItem id: every value came back
BadNodeIdUnknown, with the whole suite green because every test handed the
driver its own Tags array directly.
- MTConnectDriverOptions gains RawTags; the config DTO binds it (verbatim — a
bad blob is skipped per-tag at binding time, never fails the deployment).
- A new TagBinding, derived from the options and swapped with them in one write,
carries RawPath -> dataItemId, its one-to-many reverse, and the definition set
the observation index coerces against. Subscribe/Read resolve through it; the
fan-out expands each touched dataItemId into every RawPath bound to it and
publishes under THAT reference — the line the blocker turned on.
- Coercion type precedence: the blob's driverDataType/dataType (both spellings —
the AdminUI's MTConnectTagConfigModel writes the latter), else a Tags entry for
the same DataItem, else the Agent's own /probe declaration via
MTConnectDataTypeInference (what makes a browse-committed {"address": id} blob
publish as a number), else String. A present-but-invalid type rejects that tag.
- An unmapped reference falls through unchanged, so the dataItemId-keyed CLI
surface and every existing test keep working, and an unclaimed RawPath reports
Bad rather than throwing.
15 new tests build their config through the real DriverDeviceConfigMerger; all
four mutations (publish the dataItemId, drop the RawTags binding, drop the
derived coercion type, drop the reverse-map expansion) are caught.
|
||
|
Joseph Doherty
|
b0046d6959 |
feat(mtconnect): typed AdminUI tag-config model (Task 17)
MTConnectTagConfigModel: pure FromJson/ToJson/Validate for the MTConnect tag TagConfig JSON blob, mirroring ModbusTagConfigModel/OpcUaClientTagConfigModel. Fields: fullName (DataItem@id, required), dataType (DriverDataType override, written as an enum name string), mtCategory/mtType/mtSubType/units (probe metadata), mtDevice/mtComponent (author context). Preserves unknown JSON keys across load->save (isHistorized/historianTagname and anything else). Guards against the enum-serialization trap (numeric dataType would fault the driver at deploy) both on write (always TagConfigJson.Set via enum name) and on read (Validate() rejects a dataType that was stored as a bare digit string, since Enum.TryParse silently accepts numeric text). AdminUI.csproj gains a ProjectReference to Driver.MTConnect.Contracts, matching the existing POCO-only driver-Contracts pattern used by the other six typed tag editors. Scope: pure model only (Task 17). The razor editor shell + TagConfigEditorMap/ TagConfigValidator registration is Task 18. |
||
|
Joseph Doherty
|
c232a3ce67 |
fix(mtconnect): cap CancelAfter overflow + stop leaking ex.Message in probe fallback (review follow-up)
Two review findings on Task 14's MTConnectDriverProbe, both red-first:
- CancelAfter(effectiveTimeout) sat before the try block with only a low-end
floor on a non-positive timeout, never a high-end cap. A pathological
timeout (e.g. TimeSpan.FromDays(1000)) threw an uncaught
ArgumentOutOfRangeException straight out of CancelAfter's ~49.7-day legal
range, violating the probe's own "Never throws" contract. Not reachable
through today's only caller (AdminOperationsActor clamps to [1,60]s and
wraps the call), but a landmine for any future caller without that clamp.
Fixed by clamping effectiveTimeout to a new MaxTimeout (10 minutes).
- The final `catch (Exception ex) { return new(false, ex.Message, null); }`
forwarded ex.Message verbatim for any exception type not enumerated above
it, breaking the redaction discipline every other catch was audited for
(AgentUri may carry userinfo credentials). Not exploitable by any
currently-reachable exception type, but an open door for a future one.
Fixed to build a message from the already-redacted safeUri + the
exception's type name instead of its message.
Also: a genuine non-2xx stub-handler test (prior HttpRequestException
coverage was connection-refused only), a credentialed-agentUri +
successful-probe redaction test, and a one-line doc remark on the accepted
reachable-vs-deployable gap.
|
||
|
Joseph Doherty
|
a107a1a8b9 |
feat(mtconnect): host registration + DriverTypeNames const + guard parity (Task 16)
Register the MTConnect factory + probe with the Host, add the DriverTypeNames.MTConnect constant, and keep DriverTypeNamesGuardTests green. - DriverTypeNames: new MTConnect const + appended to All. - DriverFactoryBootstrap: MTConnectDriverFactoryExtensions.Register in Register(...) at the default Tier A (managed HttpClient/XML, in-process; Tier C is the only tier arming process-recycle, wrong here), and TryAddEnumerable(IDriverProbe -> MTConnectDriverProbe) in AddOtOpcUaDriverProbes so the probe reaches admin-only nodes (the admin-pinned Test-Connect singleton) without double-registering on a fused admin,driver node. - Host.csproj: ProjectReference to Driver.MTConnect (required to compile the Register call). - Core.Abstractions.Tests.csproj: ProjectReference to Driver.MTConnect so the guard's bin scan discovers the factory. This and the constant MUST land together -- verified RED (2/4 facts) with the constant alone. - DriverProbeRegistrationTests: MTConnect added to AdminUiDriverTypeKeys; its idempotency fact asserts distinct-probe-count and goes RED without this (and RED if TryAddEnumerable is downgraded to AddSingleton). Reconciles all four "MTConnect" literals onto the one constant: the factory's DriverTypeName, MTConnectDriver.DriverType, and MTConnectDriverProbe.DriverType now all alias DriverTypeNames.MTConnect (no circular reference -- Core.Abstractions is a leaf both driver projects already reference). This is the repo's TwinCat/Focas anti-drift convention. |
||
|
Joseph Doherty
|
ba89d6d0ff |
feat(mtconnect): driver factory delegating to the single config parser (Task 15)
The factory carries NO config DTO of its own: CreateInstance delegates to MTConnectDriver.ParseOptions, the single authority for the config document. A second DTO would drift from the parse the runtime's ReinitializeAsync path uses, and the drift would only surface at deploy time. Construction is connection-free (agentClientFactory: null) because the Wave-0 universal browser builds a throwaway instance per browse probe, and the returned instance is never narrowed — its five capability interfaces ARE the runtime's dispatch surface. Pinned by tests asserting all five plus the deliberate absence of IWritable. DriverTypeName is the literal "MTConnect"; Task 16 adds DriverTypeNames.MTConnect and the host registration that must equal it. |
||
|
Joseph Doherty
|
23cd47d54b | feat(mtconnect): IDriverProbe one-shot /probe reachability (Task 14) | ||
|
Joseph Doherty
|
dfbcb0e7f0 |
fix(mtconnect): close the silent-refusal + teardown-wedge defects in the pump (Task 11 review)
C1 (critical): a latched "this endpoint cannot stream" verdict could end with a GREEN driver holding a permanently silent subscription. DriverInstanceActor handles a tag-set change as Unsubscribe-then-Subscribe with no re-initialize; the unsubscribe cleared the stream-degraded flag, the resubscribe was refused by the latch without a word, and one successful read then reported Healthy. StartSampleStreamCore now re-asserts the degradation and logs a Warning naming the remedy, and teardown no longer clears the flag while the latch is set. NOT promoted to Faulted: DriverHealthReport maps any Faulted driver to a /readyz 503, so that would de-ready the whole node over one misconfigured Agent whose reads are perfectly healthy. Faulted stays for the case that earns it. I1: teardown waits are bounded (5 s) and honour the caller's token. They were unbounded and ignored it, so DriverInstanceActor's 5 s budget was inert and PostStop — which blocks a dispatcher thread on ShutdownAsync while holding the lifecycle semaphore — could be wedged forever by one blocking subscriber. The cancellation source is disposed only when the loop is provably gone. Applied to StopProbeLoopAsync too: Task 13 reproduced the identical shape, and closing the class in one method while leaving the other as the pattern to copy is worse than not closing it. I2: the reconnect ladder was monotonic for the process lifetime, so ~9 unrelated drops pinned the driver at MaxBackoffMs forever. A stream that delivered before dropping now starts a fresh ladder (at attempt 1, not 0, so MinBackoffMs is still honoured). I3: MaxBackoffMs <= 0 clamped every delay to zero — an operator-authorable reconnect spin. Treated as unset, like a multiplier that cannot grow. I4: the session published instanceId without NextSequence, so after a restart it held the new agent's id beside the dead one's cursor, and a gap/OUT_OF_RANGE re-baseline (id unchanged) never published the cursor at all. Both move in one CAS, and the pump publishes its cursor as it exits. I5: OnDataChange was a multicast Invoke — the first throwing handler aborted the rest of the list, starving every later subscriber. Now walked by hand with the catch inside the loop. The test that claimed to cover this registered the recorder BEFORE the thrower, so it passed regardless; swapped, it was red. Faults are latched to one Warning per stream generation (Debug thereafter) so a consistently-throwing consumer cannot flood the log. Also: the pump restarts after an unexpected fault (IsCompleted, not null); a device-scope that matches nothing is a Warning, not a Debug tally; a device or component with neither name nor id is skipped rather than emitting a blank path segment; DiscoverAsync's remark no longer claims DriverInstanceActor retries (it does not for a Once driver, and injection is dormant in v3); and two flake seeds in the fake are gone (Dispose re-read its own counter; _disposeCts was disposed under a racing SampleAsync). 439/439. Every changed behaviour falsified by mutation. |
||
|
Joseph Doherty
|
fa0e40796f | feat(mtconnect): host-connectivity probe + instanceId rediscover (Task 13) | ||
|
Joseph Doherty
|
13dd9fcd70 | feat(mtconnect): ITagDiscovery streams tree + SupportsOnlineDiscovery opt-in (Task 12) | ||
|
Joseph Doherty
|
9a67ed918a |
feat(mtconnect): ISubscribable /sample pump + ring-buffer re-baseline (Task 11)
One shared /sample long poll behind every subscription handle. Subscribe fires initial data per reference from the primed /current (Part 4 convention) and returns without waiting on the stream; the pump runs on its own task under its own token — never the caller's Subscribe token, which is disposed the moment that call returns. The pump owns every way the sequence can break: - sequence gap, measured against the RUNNING cursor (the previous chunk's nextSequence). Comparing against the opening `from` reports a gap on every chunk once the buffer rolls — a /current re-baseline storm. - OUT_OF_RANGE under HTTP 200 (InvalidDataException out of SampleAsync), which IsSequenceGap cannot see at all — without it a driver recovers from falling a little behind but not from falling a lot. - agent restart, checked BEFORE the gap because a restart resets sequences and so usually trips the gap check too; it clears the index (the held values describe a device model that no longer exists) and tells the subscribers. - every chunk advances the cursor, heartbeats included. Re-baseline calls CurrentAsync directly on the client the pump already holds: routing it through ReinitializeAsync would take the non-reentrant lifecycle semaphore from inside a pump a lifecycle method may already be awaiting, and hang the driver with no exception and no log. StopSampleStreamAsync is filled in (same call sites) and InitializeAsync now stops the stream before teardown too, so a re-Initialize on a live instance cannot leave a pump enumerating a disposed client. MTConnectStreamEnded/TimeoutException reconnect under a geometric backoff with a 100 ms growth floor (MinBackoffMs defaults to 0, and 0 x multiplier is still 0). MTConnectStreamNotSupportedException does NOT retry — it latches, reports loudly, and is cleared only by a re-initialize. Health precedence is explicit: /current and /sample fail independently, so each path owns a flag, clears only its own, and Healthy requires both clear. Neither can paper over the other's failure. 29 new tests, all deterministic — no sleeps, no wall-clock assertions. The fake grew per-generation sample streams (so a reconnect has somewhere to land), scripted /current answers, scripted sample failures, and a chunk-consumed signal that also fires when the consumer abandons the stream. 375/375 green. |
||
|
Joseph Doherty
|
a127954cab |
fix(mtconnect): review remediation for the driver lifecycle (Task 9)
Important 1 - the probe-model triple was read outside _lifecycle while every
write happened under it. Packed (Client, ProbeModel, ProbeModelDataItemCount)
into one immutable AgentSession swapped by a single Volatile.Write, matching
what _index already does, rather than locking the readers: both await the
network, so taking _lifecycle would stall a shutdown for a whole
RequestTimeoutMs and would widen the non-reentrancy hazard. GetOrFetch and
Flush re-publish via CompareExchange so a late re-cache cannot undo a
concurrent teardown/re-init, and a client disposed mid-fetch now surfaces as
the same "not connected" InvalidOperationException browse already handles
instead of a raw ObjectDisposedException.
Important 2 - HasConfigBody decided emptiness by matching the literals "{}" /
"[]", so "{ }", "{\n}" and every pretty-printed empty document fell through to
ParseOptions, failed the required-AgentUri check, and turned a semantically
empty config into a cold-start fault. Now parsed: an object/array with no
elements (or a bare null) is empty; malformed text is deliberately NOT empty so
ParseOptions produces the real quoted error rather than silently starting on
stale options.
Important 3 - documented the _lifecycle non-reentrancy hazard in the code, on
both the semaphore and StopSampleStreamAsync, naming Task 11's re-baseline as
the specific path that would deadlock and giving the CurrentAsync-directly
pattern that avoids it.
Important 4 - removed the Initialize/Reinitialize asymmetry rather than
documenting it. Both now share one rule via ResolveIncomingOptions: an
unreadable config document never destroys working state, and faults only a
driver that had none. InitializeAsync previously tore down before parsing, so a
bad document on a live instance destroyed a healthy client - the exact outcome
ReinitializeAsync was written to avoid.
Minors: SafeDispose traces the swallowed disposal fault at Debug (a client that
cannot be released is how a handle leak starts); the RequirePositive test is a
Theory over all four timing knobs, not just RequestTimeoutMs (arch-review
01/S-6); the footprint test no longer overclaims "is zero before initialize".
CannedAgentClient gains a one-shot ProbeGate so a lifecycle change landing
mid-request is deterministic - no timers. 346/346 (329 + 17). Six mutations
verified: fetch-CAS, disposed-translation, flush retired-session guard,
literal HasConfigBody, teardown-before-parse, and two dropped RequirePositive
calls each fail only their own tests.
|
||
|
Joseph Doherty
|
1bfc1722b3 |
feat(mtconnect): IReadable via /current, ordered per-ref snapshots (Task 10)
One deadline-bounded /current per batch, answered out of that one whole-device document: N references cost one round-trip, one snapshot each, in request order, duplicates included. DEVIATION (deliberate, from the plan's implied shared-index write): the response is indexed into a PER-CALL snapshot index, never into the driver's shared observation index. That index is written by Task 11's /sample pump and is last-write-wins with no timestamp arbitration; a read folding its /current into it would be a second writer racing the first, and /current is frequently the OLDER document (the Agent composes it while the pump's newer delta is in flight). Losing that race rolls a subscribed value backwards and leaves it there until the data item next changes. Both paths coerce through the same MTConnectObservationIndex logic, so a read and a subscription report a given observation identically -- the read path is simply a pure function of (document, authored tags). Failure posture is the inverse of InitializeAsync: nothing but genuine caller cancellation crosses the capability boundary. Unreachable Agent / blown deadline / unusable answer => every ref codes BadCommunicationError and the driver degrades (never downgrading Faulted); no client at all (pre-initialize, faulted, post-shutdown) => BadNotConnected with health untouched. The index's BadWaitingForInitialData / BadNodeIdUnknown / BadNoCommunication / BadNotSupported distinctions pass through unmodified. 21 tests pinning arity, order, duplicates, empty batch, one-round-trip, each Bad code, read-time freshness, the anti-clobber invariant, the deadline, and cancellation-propagates-but-agent-failure-does-not. CannedAgentClient gains a cancellation-observing CurrentGate (still no timers or sleeps of its own). Test csproj takes the AbCip/S7 OTOPCUA0001 NoWarn -- this suite calls the capability directly by design. 329/329 green. |
||
|
Joseph Doherty
|
4fe0af3693 |
feat(mtconnect): driver shell IDriver lifecycle over agent-client seam (Task 9)
MTConnectDriver implements IDriver only: connection-free ctor, initialize (deadline-bounded /probe + /current, caching the device model, the agent instanceId, and a primed observation index), reinitialize, shutdown, health, footprint, and cache flush. Decisions worth knowing: - The driverConfigJson argument is HONOURED, not ignored. DriverInstanceActor delivers a config change only via ReinitializeAsync(json) on the live instance, so a driver reading only its ctor options would turn every MTConnect config edit into a silent no-op that still sealed the deployment green. ParseOptions lives on the driver; Task 15's factory must delegate to it rather than deserialize a second DTO. - /probe ok but priming /current failing is Faulted, not Healthy-with-no-data: the index IS the read surface and the /sample cursor comes from /current. - ReinitializeAsync rebuilds the client whenever ANY option the client reads at construction changed - AgentUri, DeviceName, RequestTimeoutMs, HeartbeatMs, SampleIntervalMs, SampleCount - not just the first two. The timing knobs are baked into the deadlines, the watchdog window, and the /sample query string. - An unparseable config faults a cold start but leaves a RUNNING driver serving its previous valid configuration (the deployment fails instead). - IMTConnectAgentClient now extends IDisposable rather than the driver doing `as IDisposable`, which would silently no-op against a non-disposable fake and leak an HttpClient + socket handler per re-deploy. Sync, not async: every resource behind the seam is sync-disposable, and the async unwind belongs to the SampleAsync enumerator the pump owns. - Flush drops the probe/browse cache and keeps the index; every model consumer goes through GetOrFetchProbeModelAsync so a flush cannot wedge browse. Tests: CannedAgentClient (shared, deterministic - channel-backed scripted /sample with no timers, call counts, disposal tracking, failure injection) + 27 lifecycle tests. 308/308 in the project. |
||
|
Joseph Doherty
|
908bd7ba4a |
fix(mtconnect): deterministic concurrency test + DATA_SET -> BadNotSupported
The concurrency test asserted a post-Apply post-condition from inside the race window: dev1_pos is authored, so a reader beating the first Apply correctly observes BadWaitingForInitialData, which the test's two-member "legal" set omitted. Consistently red (10/10) rather than intermittent. Rewritten to assert only properties that hold at every instant — the whole (status, value, source timestamp) triple against the three legal snapshots, which is the real torn-snapshot check — plus a read counter guarding a vacuous pass and one deterministic closing Apply for an exact end state. The index is unchanged here; BadWaitingForInitialData is a designed state. Also codes structured (DATA_SET / TABLE) observations BadNotSupported now that the parser supplies MTConnectObservation.IsStructured, so a two-entry data set no longer surfaces as the Good string "12". |
||
|
Joseph Doherty
|
5ba9f1be6f |
fix(mtconnect): make a /sample stream end loud, and correct the gap contract (Task 7 review)
Code review of c46540ae approved the framing algorithm but moved the risk onto the contract around it. C1 (critical). SampleAsync returned normally on three different events: EOF, the closing --boundary--, and a response that was not multipart at all (where it yielded one parsed snapshot and finished). The seam documents the stream as yielding until ct is cancelled, so a Task 11 pump written to that contract would silently drop its subscription or hot-loop reconnecting; the non-multipart leg reported a configuration error as a healthy finished stream, which is the #485 quiet-successful-termination shape aimed at the very next task. Every non-cancellation end now throws: MTConnectStreamEndedException (ConnectionClosed / ClosingBoundary - transient) or MTConnectStreamNotSupportedException (configuration - reconnecting reproduces it forever), sharing one catchable base. The non-multipart body is still parsed first so an Agent's own MTConnectError text wins, but the document is NOT yielded. I1. IsSequenceGap moved to IMTConnectAgentClient (static) and its first parameter is now expectedFrom. The old name and doc were wrong for every chunk after the first - the correct comparand is the PREVIOUS chunk's NextSequence, and comparing against the opening "from" argument reports a gap on every chunk once the ring buffer rolls, i.e. an endless /current re-baseline storm. Both doc blocks also now record that cppagent answers from < firstSequence with an OUT_OF_RANGE MTConnectError under HTTP 200, which surfaces as InvalidDataException and NOT as a gap - so Task 11 must treat that parse failure as a re-baseline trigger too. I2. Every multipart test served the whole body from one buffer, so every framing test completed in a single ReadAsync - the split-boundary path, the split-header path and the multi-fill loop were correct by inspection only. A ChunkedStream double (N bytes per read, N = 1/3/7) now drives the fixtures through a split transport, plus a >16 KB document that forces a buffer resize. Falsifiability: removing either scan overlap, or collapsing the fill loop, fails ONLY the new chunked tests. I3. Disposal mid-enumeration surfaces as OperationCanceledException via an internal dispose-linked token, not a raw ObjectDisposedException that Task 9's re-init would inflict on an enumerating pump. I5. HeartbeatMs, SampleIntervalMs and SampleCount join RequestTimeoutMs in construction-time positive-value validation. All three reach the query string and an Agent answers heartbeat=0 with an HTTP-200 MTConnectError that names no config key. I4/M2. The no-Content-length framing fallback logs a one-shot warning (the client takes an optional ILogger); a multipart response with no boundary parameter fails fast instead of degrading into a watchdog timeout. M5/M6. Shared element/attribute reading rules extracted to MTConnectXml; the probe parser documents why it alone does not trim BOM/whitespace. The literal U+FEFF in source became a '' escape. Also, from Task 8: MTConnectObservation gained IsStructured (default false), set from element.HasElements. A DATA_SET/TABLE observation's <Entry> children concatenate through element.Value to nonsense ("12" for two entries) that the index would publish as Good, and only the parser can tell that apart from a legitimate space-bearing Message. False for CONDITION observations - their value comes from the element name, so child content cannot corrupt it. 275/275 green in this suite's own files. |
||
|
Joseph Doherty
|
ac0a284055 | feat(mtconnect): observation index + UNAVAILABLE->BadNoCommunication (Task 8) | ||
|
Joseph Doherty
|
cf43f8d0ab |
feat(mtconnect): current/sample parse + sequence-gap detection (Task 7)
Adds the /current and /sample legs to MTConnectAgentClient: - MTConnectStreamsParser: hand-rolled System.Xml.Linq parse of an MTConnectStreams document into MTConnectStreamsResult. Observations are every element child of <Samples>/<Events>/<Condition>, keyed by dataItemId (the element NAME is the data item's type in PascalCase, so a fixed name set would drop observations). A CONDITION's value is its element name, not its text - <Normal/> is empty - and <Unavailable/> normalizes onto the same UNAVAILABLE sentinel a Sample/Event carries as text, so Task 8's no-comms mapping sees one token. Timestamps are normalized to DateTimeKind.Utc explicitly. - IsSequenceGap(requestedFrom, chunk) => chunk.FirstSequence > requestedFrom, so Task 11's pump can re-baseline after a ring-buffer overflow. Equality is NOT a gap. - SampleAsync: ResponseHeadersRead + a hand-rolled multipart/x-mixed-replace frame reader (Content-length fast path, boundary-scan fallback), bounded by a heartbeat watchdog derived from HeartbeatMs so a silent Agent faults instead of wedging the pump - the S7 frozen-peer shape (arch-review R2-01). The long poll gets its own HttpClient with an infinite Timeout so the unary /probe + /current deadline stays bounded. Task 0 package decision, option (c): both TrakHound PackageReferences and their PackageVersions are dropped. MTConnectHttpClientStream takes a URL and dials its own socket (no handler/stream seam, so it cannot serve a socket-free unit suite), and it emits already-parsed documents through the formatter lookup Task 6 proved is missing - framing-only reuse was never actually on offer. The driver now depends on nothing but the BCL. Recorded in the plan's Task 0 CORRECTION block. Also folds in two Task 6 review carry-overs: the stale IMTConnectAgentClient doc comment claiming a TrakHound-backed implementation, and a probe test constructing a mixed-namespace vendor extension (<x:Pump> with default-namespace <DataItems> children) that backs the parser's ignore-the-namespace claim with a test. 187/187 green. |
||
|
Joseph Doherty
|
d3aaa4a411 | docs(mtconnect): correct the Task 0 decision — TrakHound cannot parse XML as pinned | ||
|
Joseph Doherty
|
1990d21733 | feat(mtconnect): agent client /probe parse to device model (Task 6) | ||
|
Joseph Doherty
|
b13cf5c6e8 | fix(mtconnect): options carry the authored tag definitions (Task 2 follow-up) | ||
|
Joseph Doherty
|
79e30d1ead | fix(mtconnect): infer numeric EVENT types from the probe's UPPER_SNAKE spelling (Task 3 follow-up) | ||
|
Joseph Doherty
|
3ca8ddf6f7 | test(mtconnect): canned probe/current/sample XML fixtures incl. forced-gap (Task 4) | ||
|
Joseph Doherty
|
c0729ae5c0 | feat(mtconnect): pure data-type inference table + golden test (Task 3) | ||
|
Joseph Doherty
|
992ffe5620 | feat(mtconnect): IMTConnectAgentClient seam + neutral parse DTOs (Task 5) | ||
|
Joseph Doherty
|
18742ea92b | feat(mtconnect): driver options + tag definition records (Task 2) | ||
|
Joseph Doherty
|
e36b920113 | build(mtconnect): scaffold Contracts+Driver+Tests projects, add to slnx (Task 1) | ||
|
Joseph Doherty
|
3b92b7cc83 | docs(mtconnect): record TrakHound-vs-hand-rolled client decision (Task 0) | ||
|
Joseph Doherty
|
a0b5095b3b |
docs(drivers): un-stale the MQTT README rows + track every known gap as an issue
The cross-driver README still described MQTT as pre-P2 — "Sparkplug B is P2
(`mode: SparkplugB` is a placeholder)" and "Sparkplug B has no fixture yet (P2)".
Both were false as of the P2 merge (
|
||
|
Joseph Doherty
|
394558e6c2 |
docs: the umbrella-index edit belongs on scadaproj's main, and must be pushed
The cross-repo propagation rule said WHAT to update but not WHERE. scadaproj is a separate repo with its own checkout, so committing there inherits whatever branch it happens to be on — usually an unrelated feature branch, sometimes one with no upstream — and the index then disagrees with reality for as long as that branch stays unmerged. Observed 2026-07-27: the Sql poll driver's index entry had sat unpushed on feat/overview-dashboard since 2026-07-24 despite this rule, and the Mqtt driver's entry was about to join it. Both recovered by cherry-picking onto main. Records the explicit procedure and says to cherry-pick stranded index commits rather than merge the branch they are stuck on. The canonical statement now also lives in scadaproj's own CLAUDE.md (main @ 75f267c). Claude-Session: https://claude.ai/code/session_01GASWkNEi68FSCtvr6rLoEW |
||
|
Joseph Doherty
|
c3a2b0f773 |
Merge branch 'feat/mqtt-sparkplug-driver'
# Conflicts: # src/Core/ZB.MOM.WW.OtOpcUa.Core.Abstractions/DriverTypeNames.cs # src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Components/Shared/Drivers/DriverConfigModal.razor # src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/EndpointRouteBuilderExtensions.cs # src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigEditorMap.cs # src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/Uns/TagEditors/TagConfigValidator.cs # src/Server/ZB.MOM.WW.OtOpcUa.AdminUI/ZB.MOM.WW.OtOpcUa.AdminUI.csproj # src/Server/ZB.MOM.WW.OtOpcUa.Host/Drivers/DriverFactoryBootstrap.cs |
||
|
Joseph Doherty
|
123ddc3f40 |
fix(scripteditor): complete + hover absolute RawPaths (#490)
The issue reports a missing feature. The cause is a stale projection, and the
consequence was worse than the gap described.
`ScriptTagCatalog` still emitted `Tag.TagConfig.FullName` — the pre-v3 driver wire
address. Since v3 the mux key is the **RawPath**: `AddressSpaceComposer` harvests the
`ctx.GetTag("…")` literals into `DependencyRefs`, those become `VirtualTagActor`
dependency refs registered with `DependencyMuxActor`, whose `_byRef` is keyed Ordinal
by `AttributeValuePublished.FullReference` — and a driver's wire ref IS the RawPath
(`DriverHostActor._nodeIdByDriverRef` is keyed `(DriverInstanceId, RawPath)`). The
`{{equip}}/<RefName>` form already substitutes to RawPaths at both compose seams.
So the catalog was wrong in BOTH directions, not merely incomplete: completion
offered paths that could never resolve at runtime, while a CORRECT absolute RawPath
got no completion and hovered as "not a known configured tag path". The file's own
comment admitted the raw/UNS catalog was left for "Batch 2/3" and it never came.
Now projected through the shared `RawPathResolver` — the same byte-parity authority
`DraftValidator` and `AddressSpaceComposer` use — so a suggested path is identical to
the one the deploy gate and the runtime compute. A tag with a broken ancestry chain
resolves to null and is omitted: the runtime could not route it either, so offering
it would be a lie. Hover additionally gains the owning `DriverInstanceId`, which the
topology now makes available (it was always null before).
Editor-accepts <=> publish-accepts is preserved: no diagnostic keys off this catalog
(`OTSCRIPT_EQUIPREF` only marks `{{equip}}` paths), so this changes completion and
hover only — it cannot newly reject a script.
The existing tests pinned the v2 contract and were re-authored: a RawPath only exists
if the RawFolder -> DriverInstance -> Device -> TagGroup -> Tag chain does, so the
seed now builds one. Added coverage for group-nested paths, a driver at the cluster
root (no folder segment), a broken chain being omitted rather than guessed, and the
pre-v3 FullName no longer resolving.
Live-gated on docker-dev (rig rebuilt, both central nodes):
- completion, empty literal -> every RawPath incl. group-nested
`opcua1/plc/OpcPlc/Telemetry/Basic/AlternatingBoolean`
- completion, prefix `pymod` -> `pymodbus/plc/HR200X`, `pymodbus/plc/ImportedTag`
- hover `pymodbus/plc/HR200X` -> "Tag · Type UInt16 · Driver MAIN-modbus"
- hover an unknown path -> "Not a known configured tag path"
AdminUI 742/742; script-analysis 55/55; solution builds 0 errors.
|
||
|
Joseph Doherty
|
0f38f48679 |
merge: Modbus RTU-over-TCP transport (#495)
Wave 1 of the driver-expansion program. The branch was complete and live-gated but had never been merged; it sat 15 commits ahead and 50 behind master. Purely additive behind the existing IModbusTransport seam — the application protocol (function codes, codecs, planner, coalescing, write path, probe, materialisation, HistoryRead) is identical across TCP and RTU; only the wire framing differs ([addr][PDU][CRC-16], no MBAP, no TxId). Zero changes to codecs/planner/health. Selected by a new Transport config field (ModbusTransportMode: Tcp | RtuOverTcp), routed through ModbusTransportFactory (throws on unknown mode), with the string-enum guard on options/DTO/factory and a Transport selector on the AdminUI Modbus form. ModbusSocketLifecycle was extracted from ModbusTcpTransport (behaviour-preserving) and is composed by both transports. Descoped by design: direct-serial (System.IO.Ports) and Modbus ASCII — RTU buses are reached exclusively via a serial->Ethernet gateway. No per-tag config changes; the existing per-tag UnitId already makes a multi-drop bus work. Re-validated against current master at merge time (the branch's own gate predates 50 commits of master): no overlapping files, clean merge, solution builds 0 errors, and Modbus 344/344, Modbus.Addressing 161/161, AdminUI 740/740, Runtime 507/507, Configuration 131/131 all pass. Confirmed the Transport selector is still reachable through master's current authoring path (DriverConfigModal -> ModbusDriverForm), so the branch's live /run gate result still applies. |
||
|
Joseph Doherty
|
0e93587b1c |
merge: test-correctness, config-secret and AdminUI fixes (#503 #501 #493 #499 #488 #505)
Six issues closed, one new one found, filed and fixed.
- #503 Galaxy EventPump flake — root cause was cross-test measurement leakage on a
STATIC Meter, not a timing budget. 15/15 clean under the issue's own contention
recipe. Three further defects in the same test fixed on the way, including a
"held" dispatch loop that was never held (async void on EventHandler<T>) and an
assertion built from a derived quantity, which could only fail spuriously.
- #501 Three absence assertions given positive controls (a fourth site the issue
did not list). Proven falsifiable by breaking the production code: both breaks
passed under the old AwaitAssert form.
- #493 docker-dev seed backfills ClusterNode.GrpcPort; upgrade step documented.
Deliberately no EF migration — the right value is per-node config the DB cannot
derive, and a wrong dial target is worse than a NULL that is already skipped.
- #499 Sql credentials refused in ClusterNode.DriverConfigOverridesJson, gated at
the SAVE rather than the deploy: node overrides never enter the artifact, so a
deploy gate would not stop the credential being stored. Live-verified.
- #488 Periodic desired-vs-actual subscription reconcile (option A), gated on
ISubscribable so a non-subscribable driver is not Subscribe-spammed forever.
- #505 The AdminUI cluster-node editor had never worked at all — HTTP 500 on load
(InputNumber<byte>) and Save a silent no-op (NodeId regex forbids the ':' every
NodeId contains), with no ValidationSummary to reveal it. Pre-existing on master,
invisible to 739 green AdminUI tests, found only by going to drive the page.
Also closed #492 (already fixed by
|
||
|
Joseph Doherty
|
4cc0215bb4 |
fix(adminui): the cluster-node editor was dead — 500 on load, silent no-op on save
Both defects are PRE-EXISTING (present on master, unrelated to #499) and were found live-verifying the #499 guard, which lives on this page and was unreachable without them fixed. 1. **HTTP 500 on load.** `FormModel.ServiceLevelBase` was `byte` to match the entity, and `InputNumber<T>`'s static constructor throws outright for `System.Byte` — "The type 'System.Byte' is not a supported numeric type". That escapes `BuildRenderTree` unhandled, which in Blazor takes the WHOLE page down, not one field. `/clusters/{id}/nodes/{nodeId}` and `.../nodes/new` have never rendered. Now `int` with the same `[Range(0, 255)]` and a cast on the way to the entity, so the stored type is unchanged. (Same failure mode as #504: an unhandled throw inside the render tree is a page-kill.) 2. **Save did nothing, silently.** `NodeId` was validated against `^[A-Za-z0-9_-]+$`, which forbids the colon — but every NodeId in this system is `host:port` (`ClusterRoleInfo` and `ConfigPublishCoordinator` derive it as `member.Address.Host:Port`, the seed writes `central-1:4053`, and `NodeDeploymentState.NodeId` is FK-bound to it). So every existing node failed validation, `OnValidSubmit` never ran, and the button was inert. Pattern now allows `:` and `.` (hostnames may be dotted). 3. **Added the missing `<ValidationSummary/>`.** The form had a `DataAnnotationsValidator` but nothing rendering its output, so a validation failure produced no feedback at all — which is precisely how (2) stayed invisible. This is the change that stops the next such defect being silent. Live-verified on docker-dev (both central nodes rebuilt, since :9200 round-robins the pair): page 500 → 200 on both routes, and a real edit now persists. |
||
|
Joseph Doherty
|
4b9bcddbcc |
feat(drivers): periodic reconcile of desired vs actual subscriptions (#488)
Option A from the issue — the cheap consistency check, no interface change. The desired ref set was re-applied on a Connected TRANSITION (`ResubscribeDesired`) and on a deploy's `SetDesiredSubscriptions`, and nowhere else. A subscription lost while the driver STAYED Connected — a subscribe that failed and was never retried, a handle dropped down an error path — left the actor subscribed to nothing while still reporting Healthy, indefinitely. `ReconcileSubscription` now rides the existing 30 s `health-poll` tick: while Connected, a driver holding a desired set but no live handle is re-subscribed. Gated on `ISubscribable`. A driver that cannot subscribe at all can still be handed a desired set, and without the guard it would be told `Subscribe` every tick and fail every tick, forever. This is a state-machine consistency check, NOT a probe — it can only see that this actor holds no handle, never that a handle is present but dead server-side, because `ISubscriptionHandle` is opaque. That is option B, and it stays deferred until a real occurrence shows the handle outliving the subscription; it would need a liveness member implemented across all eight drivers with different subscription mechanics. A redundant `Subscribe` is possible (a tick queued ahead of an already-queued one observes the same no-handle state) and is harmless: `HandleSubscribeAsync` is a `ReceiveAsync`, so no tick is processed mid-subscribe, and it drops any prior subscription before establishing the new one. Suppressing it would need a pending-subscribe flag threaded through every self-tell site — more state than the race is worth. `healthPollInterval` becomes an optional Props parameter, mirroring the existing `rediscoverInterval` seam, so a test can drive the reconcile without waiting 30 s. Three tests. The reconcile itself is proven by a stub whose FIRST subscribe throws: nothing else would ever retry it, so a second attempt can only come from the reconcile — disabling `ReconcileSubscription` turns it red. The two guard tests are absence assertions and are bounded by a positive control rather than a sleep: a counting health publisher makes ticks PROCESSED observable, so "still one subscribe" means the reconcile ran and declined, not that the timer had not fired. Explicitly not deploy-time drift (#486) — re-asserting an already-correct desired set would not have helped there. |
||
|
Joseph Doherty
|
2193d9f2e4 |
fix(config): refuse to store a Sql credential in a node config override (#499)
`ClusterNode.DriverConfigOverridesJson` is the third surface a Sql driver's config is persisted on, and the only one nothing gated. It is a map keyed by `DriverInstanceId` merged onto the cluster-level `DriverConfig`, so a literal `connectionString` pasted there leaks exactly as one pasted into the driver — the leak #498 closed for `DriverInstance.DriverConfig` and `Device.DeviceConfig`. Gated at the SAVE, not at the deploy — a deliberate departure from both options the issue offered: - `DraftSnapshot` carries no `ClusterNode` rows, and adding them would widen the snapshot and every builder of one for a single rule about a value that never enters the artifact. - More to the point, a deploy gate is the wrong instrument here. Node overrides are not in the artifact, so blocking a deploy would not stop the credential being stored — it is already in the database and replicated by then. Refusing the save is the only point where "refuse to store it" is literally true, which is #498's own framing: discarding a secret on read is not the same as refusing to store it. The check moves into a shared `SqlCredentialGuard` so the two enforcement points cannot drift; `DraftValidator` now calls it instead of its own private helper. Kept deliberately narrow — the `Sql` driver's `connectionString` only, and only for instance ids that ARE Sql drivers. The issue asked whether to generalise to credential-shaped keys across every driver type; not doing that, for the same reason #498 did not: a broader sweep would start refusing configs that are legitimate today for drivers which never made an indirect-credential guarantee, which is a regression rather than defence in depth. Widen per driver, as each gains its own contract. The error names the offending driver instance(s) and NEVER the value — it reaches the AdminUI and the audit trail. 14 new cases cover both halves, including the case variants (System.Text.Json binds `ConnectionString` to `connectionString`, so a case variant is the same key, not a bypass), non-Sql ids being ignored, and every blank/malformed/non-object shape. |
||
|
Joseph Doherty
|
8b7ea3213d |
fix(rig,docs): backfill ClusterNode.GrpcPort and document the upgrade step (#493)
`GrpcPort` is a nullable column added by Phase 5. Every insert in the docker-dev seed is INSERT-if-not-exists — the right shape for a re-runnable seed, but it means a new nullable column never reaches rows an earlier version of the file created. On a persisted volume all six rows kept NULL, central found no dial targets, and `TelemetryDial:Mode=Grpc` connected to nothing. (`AkkaPort` escaped this only because it is NOT NULL with a default of 4053.) Seed now backfills `GrpcPort IS NULL AND CreatedBy = 'docker-dev-seed'` to 4056. NULL only — it must not overwrite a port an operator set deliberately on a long-lived dev volume, and NULL is the unambiguous "predates the column" marker. Deliberately NOT doing the suggested EF data migration for real deployments. The correct value is each node's own `Telemetry:GrpcListenPort`, which is per-deployment configuration the database cannot derive; a migration could only invent one, and a WRONG dial target is worse than the NULL the dialer already skips explicitly with a throttled warning. The rig can be backfilled because there the port is fixed by docker-compose.yml and therefore actually knowable. `docs/Telemetry.md` gains an upgrade section stating the prerequisite, the symptoms (the throttled skip log, LISTEN with no ESTABLISHED, the pill never going live) and the UPDATE to run before flipping any node to Grpc — plus why there is no migration. |
||
|
Joseph Doherty
|
240d7aa8aa |
fix(test): give the three absence assertions a positive control (#501)
`AwaitAssert` returns on its FIRST success, so an assertion that is already true at t=0 passes instantly and spends none of its duration. All three sites asserted an absence against a collection that starts empty, so none of them ever gave the handler the time their comments claimed. Replaced with mailbox-ordering positive controls rather than sleeps: - `RouteNativeAlarmAck_unknown_node_is_dropped` — follow the unmapped ack with a MAPPED one. The host forwards via `Tell` and the child handles `RouteAlarmAck` with `ReceiveAsync` (which suspends its own mailbox), so once the control reaches the driver the unmapped one has provably been handled. - `RouteNativeAlarmAck_on_non_primary_is_dropped` — the role itself is the control: return the node to Primary and re-send. Comments (`gated` / `control`) are the discriminator, so a leaked ack is visible rather than merged into a count. - `PeerProbeSupervisorTests.Single_node_snapshot_spawns_no_children` — a FOURTH site the issue did not list. It says the other `ChildCount.ShouldBe(0)` waits are each preceded by a `ShouldBe(1)`; that is true of three of the four, but not this one, which asserts the initial state. Now followed by a two-node snapshot: the count reaching 1 proves the single-node snapshot was processed, and a local-node child would have made it 2. The issue warned these may turn red, which would be a real defect surfacing. They did not — the routing is correct. Proven by breaking it deliberately instead: disabling the Primary gate reddens the Secondary test, and routing unmapped ids to an arbitrary mapping reddens the unknown-node test. Under the old form both breaks passed. |
||
|
Joseph Doherty
|
95295210b0 |
fix(test): scope the EventPump counter capture to its own pump (#503)
Root cause was cross-test measurement leakage, not a timing budget. EventPump's `Meter` is STATIC — one instance per process — and a `MeterListener` can only select by *meter* in `InstrumentPublished`. `StartMeterCapture` therefore received measurements from every `EventPump` alive anywhere in the assembly. xUnit runs test classes in parallel and several build pumps (`EventPumpStreamFaultTests`, the `Driver-X` test in this same file), so a sibling's events landed in these counters. That is why the flake needed a busy box: it needed two tests to overlap. Measured directly during the gate — `Received = 11` in a run that emitted 10. The old `Received == Dispatched + Dropped + InFlight` assertion is what turned the leak into a failure. `InFlight` is DERIVED as `Received - Dispatched - Dropped`, so the identity is a tautology that cannot fail on a real defect — only on a foreign increment landing between its four separate `Interlocked.Read` calls. Hence the reported subject, `counters.Received`. It is deleted, along with the `InFlight` member, and replaced by direct assertions on the three MEASURED counters. Three further defects in the same test, all found on the way: - **The "held" dispatch loop was never held.** `OnDataChange` is `EventHandler<T>` — void-returning — so `async (_, _) => await gate` is async VOID: it returned to the dispatch loop at the first await and blocked nothing. Measured `Dispatched = 2..3`, never 0. Drops happened only when the producer happened to outrun the consumer, so the arrangement the doc comment describes was fiction and the assertions rode on scheduling luck. Now a synchronous `ManualResetEventSlim.Wait()` on the loop's own thread, pinned by `Dispatched.ShouldBe(0)`. - **Fixed `Task.Delay(150)`** replaced by a presence poll on the drop count (#500's rule). Dropped starts at 0, so it cannot be satisfied by the initial state. - **Emission is staged** — one event, wait for the dispatcher to enter the handler, then the other nine — so the arithmetic is exact (`10 - 1 held - 2 buffered = 7`) rather than scheduler-dependent. The gate release moved into `finally`, ahead of disposal: `DisposeAsync` awaits the dispatch loop, which is now genuinely parked on that gate, so a failed assertion would otherwise hang instead of failing. Verified: 15/15 clean under the exact contention the issue measured (Runtime + AdminUI + Core concurrently) — where the half-fixed version still failed on run 4. Falsifiability: restoring the async-void handler turns the new assertions red. |
||
|
Joseph Doherty
|
e3155fbbf5 |
merge: never slice a DB-sourced string with the range operator (#504)
`@s.SourceHash[..12]` on an unvalidated nvarchar(64) threw out of BuildRenderTree and took the whole /scripts page to an HTTP 500. Replaced with a shared, null/short-safe `DisplayText.Abbreviate`, applied to all five unguarded range-operator slices over DB-sourced strings in the AdminUI. Live-verified on docker-dev: /scripts 500 -> 200 with the short hashes rendered in full; long hashes still truncate with an ellipsis on the other pages. |
||
|
Joseph Doherty
|
47d148daf9 |
fix(adminui): never slice a DB-sourced string with the range operator (#504)
`Scripts.razor` rendered a hash prefix as `@s.SourceHash[..12]…`. `SourceHash`
is a plain nvarchar(64) with no length floor, so any value shorter than 12
characters threw ArgumentOutOfRangeException out of BuildRenderTree — unhandled
in Blazor, which takes the WHOLE page to an HTTP 500, not just the offending
row. Live-reproduced on docker-dev, where two Script rows carry `h1` and
`h-abs-hr200`: /scripts was a hard 500.
New `Components/Shared/DisplayText.Abbreviate(string?, int)`:
- null/blank renders the Admin UI's em-dash placeholder rather than throwing;
- a value already within budget renders verbatim with NO ellipsis, so the
display never implies text that isn't there (the old markup appended "…"
unconditionally, outside the slice — `h1` would have shown as `h1…`);
- a maxLength < 1 is clamped, so a display bug cannot become a page crash.
Applied to every unguarded range-operator slice over a DB-sourced string found
by sweeping the AdminUI for `[..N]`:
Scripts.razor SourceHash (the reported crash)
Certificates.razor Thumbprint (read off the on-disk store)
Deployments.razor x2 RevisionHash
Clusters/ClusterOverview.razor RevisionHash
The other `[..N]` hits are safe and untouched: Guid.ToString("N") slices
(always 32 chars) and the `Length > 60 ? [..60]` ternaries that self-guard.
NOTE — the issue text blamed the docker-dev seed; that was wrong and is
corrected on #504. `seed-clusters.sql` inserts only ServerCluster, ClusterNode
and LdapGroupRoleMapping. The short-hash rows came from earlier live-gate
authoring. A freshly seeded rig is fine; the trigger is any Script row that did
not go through ScriptEdit / UnsTreeService.HashSource — REST-API authored,
hand-inserted, or migrated in.
Tests: 14 new in DisplayTextTests, covering the short/null/blank/clamped cases
and a never-throws sweep over every value x budget combination. AdminUI suite
739/739.
Live-verified on docker-dev (AdminUI has no bUnit): /scripts 500 -> 200 across
both central nodes, rendering `hash=h1` and `hash=h-abs-hr200` in full;
/deployments still truncates 64-char hashes at 12 with the ellipsis;
/certificates and /clusters/MAIN unchanged.
|
||
|
Joseph Doherty
|
755ae1aa3f |
merge: re-assert non-normal scripted-alarm conditions after a (re)load (#487)
Closes the redeploy-reset gap on Part 9 scripted-alarm condition nodes — the alarm analogue of the VirtualTag ReassertValue fix. A full address-space rebuild re-materialises every condition node fresh, while the engine's reload produces EmissionKind.None for an alarm whose state did not change — so nothing wrote the node. Live-reproduced on docker-dev: a cleared- but-still-UNACKNOWLEDGED alarm came back reporting acknowledged, dropping Retain to false, so it vanished from ConditionRefresh entirely. The operator's outstanding alarm silently disappeared from the alarm list on deploy. ScriptedAlarmHostActor.ReassertConditionNodes now writes one AlarmStateUpdate per alarm not in the Part 9 no-event position, sourced from the new ScriptedAlarmEngine.GetProjections(). Node-only by construction — the projection type carries no EmissionKind, so it cannot become an alerts row or a duplicate historian record. Live gate PASSED (pre-fix image vs. rebuilt, central-2): condition absent pre-fix, present + Unacknowledged + Retain=True post-fix, stamped with its own LastTransitionUtc; /alerts empty across two re-asserts with the panel proven live by a real ACTIVATED transition. |
||
|
Joseph Doherty
|
549c656489 |
fix(alarms): re-assert non-normal scripted-alarm conditions after a (re)load (#487)
A deploy that triggers a FULL address-space rebuild clears `_alarmConditions` and re-materialises every condition node fresh — inactive, acked, confirmed, unshelved. The engine reloads its persisted state and re-derives Active from the predicate, so it ends up correct; but there is no transition to report, so `LoadAsync` yields `EmissionKind.None` and `OnEngineEmission` drops it. Nothing writes the node. Live-reproduced on the docker-dev rig against the pre-fix image: an alarm that had fired and cleared but was still UNACKNOWLEDGED came back from a rebuild reporting Acknowledged with Retain=false — so it disappeared from ConditionRefresh entirely, while the engine still held Unacknowledged. An operator's outstanding alarm silently vanishes from the alarm list on deploy. `ScriptedAlarmHostActor.ReassertConditionNodes` closes it. After each load it reads the new `ScriptedAlarmEngine.GetProjections()` and sends one `AlarmStateUpdate` per alarm NOT in the Part 9 no-event position. Load-bearing properties: - Node-only. It sends `AlarmStateUpdate` and nothing else — never the `alerts` topic, never the telemetry hub. `alerts` is the historization path, so a row per alarm per deploy would append a duplicate historian/AVEVA record every time anyone deploys. This is why it reads a projection rather than re-emitting: `ScriptedAlarmProjection` carries no `EmissionKind`, so there is nothing for the emission machinery — or a future refactor — to mistake for a transition. The issue's sketch said `GetState(alarmId)` + `LoadedAlarmIds`; that alone would have clobbered the node's severity and message, so the projection also carries the definition's severity, the resolved message template, and the last-observed worst-input quality. - Only non-normal alarms. One in the no-event position already matches what the materialise built, so a never-fired alarm stays byte-identical to a deploy without this pass (including its Message, which the materialise seeds with the display name), and an all-normal deploy writes nothing at all. - Timestamp is the condition's own `LastTransitionUtc`, not the deploy instant. - No duplicate Part 9 events: `WriteAlarmCondition`'s delta-gate compares against the node's CURRENT state, so a re-assert onto a freshly materialised node is a genuine delta (fires once — correct, the rebuild reset what clients see) while one onto a surgically-preserved node is suppressed. Tests: 6 new host tests + 6 new engine tests. Falsifiability checked by disabling the call — 4 of the 6 host tests go red; the other 2 are absence guards against an over-broad fix and pass either way by design. Live gate (docker-dev, central-2, pre-fix image vs. rebuilt image): - pre-fix: condition absent from ConditionRefresh after a rebuild; - post-fix: present, Unacknowledged, Retain=True, stamped with its own LastTransitionUtc rather than the restart instant; - /alerts stayed empty across two re-asserts, with the panel proven live by a real ACTIVATED transition immediately afterwards — no duplicate history. |
||
|
Joseph Doherty
|
28c2866710 |
merge: Sql driver follow-ups #496/#497/#498 + the Runtime.Tests flake #500 (fix/sql-driver-followups)
Four commits closing the three follow-ups the Sql poll driver left behind, plus the intermittent Runtime.Tests failure found while verifying them. - #497 corrects 16 wrong OPC UA status-code constants across 6 drivers (the issue named 3) and adds StatusCodeParityTests, a reflection guard checking every driver's hard-coded uint against the pinned SDK. That guard is what found the other 13. - #498 fails the deploy when a Sql driver's persisted config carries a literal connectionString — the write-side half of a guarantee only enforced on read. - #496 implements the design 8.1 catalog gate: authored table/column names are resolved against the live catalog at Initialize and replaced with the catalog's own spelling, so quoting becomes the backstop it was documented to be rather than the sole defence. - #500 fixes the Runtime.Tests flake: four ordering/logic defects plus a class of tight presence budgets, now routed through a documented RuntimeActorTestBase.PresenceBudget. Verified: full solution builds, all 41 unit-test projects green, Sql integration suite 21/21 against the real SQL Server on 10.100.0.35, and 30 consecutive Runtime.Tests runs clean against a measured 13% baseline. Still open by design: #499 (ClusterNode.DriverConfigOverridesJson is a third config persistence surface DraftValidator cannot see) and #501 (two alarm-ack tests use AwaitAssert for an absence assertion, so they prove nothing; the rewrite may legitimately turn them red). |
||
|
Joseph Doherty
|
154171f48c |
test(runtime): fix the intermittent Runtime.Tests failure — 4 ordering/logic defects + the presence-budget class (#500)
The reported symptom was "Runtime.Tests occasionally reports Failed: 1" with no test
name. Instrumenting 30 full-assembly runs reproduced it at 13% (4 runs) and showed it
was never ONE flaky test: five failures across three distinct tests in that batch, and
five more distinct tests over the verification rounds that followed.
Two hypotheses were tested and DISPROVED by measurement before anything was changed:
- Cluster-formation timeout. Every test in this assembly forms a real single-node Akka
cluster over TCP in RuntimeActorTestBase's constructor and waits 5 s for MemberStatus.Up
— 219 formations per run. Instrumented: idle max 960 ms, under-load max 1470 ms, zero
timeouts. A 3.4x margin; not the cause.
- Ephemeral-port exhaustion from that bind churn. TIME_WAIT measured at ~0 throughout.
Not the cause.
FOUR GENUINE ORDERING/LOGIC DEFECTS (none is a timeout):
1. VirtualTagHostActorTests.ApplyVirtualTags_respawns_child_when_plan_changes_in_place
The test loops to accept RegisterInterest/UnregisterInterest in EITHER order — and
then asserts on mux.LastSender, which DEPENDS on that order. Under the [Register,
Unregister] interleaving LastSender is the DYING child and the assertion fails. Now
captures the sender of the Register message as it arrives. Fully deterministic; no
timing involved. (Swept the other four LastSender sites: all drain the Unregister
first, so their ordering is fixed. Only this one was unsafe.)
2. DriverHostActorWriteRoutingTests.Primary_routes_write_via_uns_nodeid_to_driver_by_rawpath
ApplyAck marks the end of the APPLY, not the point a write can succeed: the child is
still in Connecting, which deliberately fast-fails writes ("driver not connected"),
and the NodeId->driver reverse map has not been pushed. Now retries until accepted.
This does not weaken the assertion — every rejection branch replies WITHOUT reaching
the driver, so Writes.Count.ShouldBe(1) still means exactly what it did.
3. HistorianAdapterActorTests.Redundancy_snapshot_without_local_node_...
One 500 ms constant served both presence budgets (AwaitAssert) and absence windows
(ExpectNoMsg). Different quantities that happened to share a number, so the presence
budget could not be raised without slowing every absence check. Split into
AssertTimeout (5 s) and Settle (500 ms).
4. ContinuousHistorizationRecorderTests.Retry_after_writer_failure_eventually_acks
Waited on `outbox == 0`, which is ALSO true before the first append — so the poll
could win the race against the recorder and return having observed the INITIAL state,
after which the CallCount>=2 check outside the block failed against a recorder that
had not run. Both conditions are now polled together with the retry count as the
discriminator. (The preceding test in the same file documents this exact trap.)
THE PRESENCE-BUDGET CLASS:
After those four, three consecutive 30-run rounds each still failed once — on a
DIFFERENT test, in a DIFFERENT file, every time (OpcUaPublishActorApplyFailureTests 2 s,
OpcUaPublishActorRebuildTests 2 s, OpcUaPublishActorTests 500 ms). That is a
distribution, not a defect, and fixing it one test at a time was the wrong method.
RuntimeActorTestBase now carries a documented PresenceBudget (15 s) and 57 sub-3 s
AwaitAssert/AwaitCondition durations across four files route through it.
Why this is safe rather than sloppy: a presence budget is an upper bound before giving
up, not a wait — these helpers poll and return the instant the condition holds. Raising
one costs nothing on the happy path and CANNOT make a genuinely failing assertion pass;
it only changes how quickly a real breakage reports. Absence windows are the opposite
(the elapsed time IS the assertion) and were deliberately left untouched with their own
short, individually-calibrated literals.
ScriptedAlarmHostActorTests is a special case, diagnosed rather than merely raised: its
8 s budget was sized for message passing but actually waits on a REAL Roslyn compile
(ApplyScriptedAlarms -> ScriptedAlarmEngine.LoadAsync compiles every predicate). Cold
script compiles are the slowest and most variable thing in the assembly, so that class
gets 30 s with the reason recorded.
Verification: 30 consecutive full-assembly runs, 30 passed / 0 failed, against a 13%
baseline. Full solution builds; all 41 unit-test projects green.
NOT fixed here, filed as #501: DriverHostActorNativeAlarmAckRoutingTests:90 and :116 use
AwaitAssert for an ABSENCE assertion, so they return instantly and prove nothing. They
are excluded from the budget change on purpose — a bigger number does not fix them, and
rewriting them to settle-then-assert may legitimately turn them red.
|
||
|
Joseph Doherty
|
4dc2ad8084 |
feat(sql): implement the design §8.1 catalog identifier-validation gate (#496)
Until now ISqlDialect.QuoteIdentifier was the SOLE defence for authored
table/column names: an identifier went from the TagConfig blob straight into a
command text, bracket-quoted. The residual risk was bounded — a hostile name is
quoted into one nonexistent object, the query fails after the connection opens,
and the tag Bad-codes — but "bounded" is not "filtered", and the design promised
a filter. Both ISqlDialect and SqlServerDialect carried a doc paragraph saying so.
SqlCatalogGate + SqlCatalogLoader now resolve every authored identifier against
the live catalog at Initialize, and REPLACE it with the catalog's own spelling.
The identifier text in an emitted poll query is therefore a string this driver
read back out of ListSchemas/ListTables/ListColumns — not operator input. Quoting
becomes the backstop it was documented to be.
Decisions worth knowing before touching this:
- Substitution, not just validation. Matching is exact-ordinal first, then a
UNIQUE case-insensitive hit: SQL Server's default collation is CI so case
variants have always worked, and rejecting them would break valid deployments.
An ambiguous CI match under a case-sensitive collation is refused rather than
guessed — picking one would publish another column's data under the operator's
node, which is worse than rejecting the tag.
- Charset check BEFORE catalog lookup. Each identifier goes through
QuoteIdentifier for its rejection rules first, so a name carrying a control or
Unicode format character is rejected WITHOUT its value being echoed into a log
line (Trojan-Source). A name that passes is safe to render, which is why
catalog-miss messages do name it — an operator hunting a typo has to see what
they wrote. Both halves are pinned by tests.
- A rejected tag keeps its node. It is dropped from the POLLED table but stays in
the AUTHORED table, so it still materializes and reads BadNodeIdUnknown —
§8.1's specified outcome. My first wiring dropped it from both, which deleted
the node instead; an existing test (ReinitializeAsync_recoversFromFaulted)
caught it. A status code can only be published by a node that exists, and a
missing address-space entry is far harder to diagnose than a Bad quality.
- An unreadable catalog FAULTS Initialize; it does not reject every tag. That is
the absence of evidence about the tags, not evidence against them — rejecting
all of them would serve a confidently-empty address space and send the operator
hunting typos that do not exist. Zero visible schemas is treated the same way,
because that is exactly what a missing GRANT looks like. Faulting lands
DriverInstanceActor in Reconnecting with its retry timer running.
- Bounded load: one schema list, one default-schema scalar, then one ListTables
per distinct authored schema and one ListColumns per distinct authored relation
— never a full catalog enumeration, and nothing after Initialize. Every
authored name reaches the catalog queries as a bound @schema/@table parameter,
so building the allow-list cannot itself be an injection vector.
- ISqlDialect gains DefaultSchemaSql ("SELECT SCHEMA_NAME()"). An unqualified
`TagValues` must resolve in whatever schema the SERVER reports; hardcoding dbo
would be a silent lie on any estate that maps service accounts to their own
default schema. It is a query, not a constant, because the answer is
per-connection.
- Accepted v1 limitation: a 3-part db.schema.table (or linked-server name)
addresses a catalog this connection cannot enumerate, so it cannot be
allow-listed and is rejected with a message pointing at the fix — expose the
data through a view in the connected database.
VerifyLivenessAsync's wall-clock pattern is extracted to RunBoundedAsync and
reused, so the new I/O gets the same R2-01 / STAB-14 protection rather than a
second hand-rolled copy. (It also fixes a latent bug in the extracted code: the
OperationCanceledException arm detached the wrong task.)
Tests: 24 pure gate tests + 9 end-to-end driver tests against the real SQLite
catalog + 3 new live tests against the real SQL Server on 10.100.0.35 (21 in that
suite now pass, exercising the real SELECT SCHEMA_NAME() + INFORMATION_SCHEMA
path). Verified falsifiable: bypassing the gate turns exactly the three rejection
assertions red and leaves the rest green.
SqlInjectionRegressionTests is deliberately NOT rewritten to expect
BadNodeIdUnknown. It drives SqlPollReader directly, below the gate, and pins the
quoting backstop on its own — defence in depth is only worth the name if each
layer holds independently. Rewriting those assertions would delete the backstop's
only coverage and leave the gate a single point of failure. Its scope note, which
said the gate does not exist, is updated to say why it stays where it is.
|
||
|
Joseph Doherty
|
1919a8e538 |
feat(config): reject a persisted Sql connectionString at the deploy gate (#498)
The Sql driver's "a pasted literal connection string is never persisted" guarantee was
enforced only on the READ path: SqlDriverConfigDto has no connectionString property and
UnmappedMemberHandling.Skip drops the key on deserialization. Nothing stopped it being
WRITTEN. Config blobs are schemaless JSON columns and the fallback authoring pattern for
a driver without a typed form is a raw-JSON textarea, so an operator pasting
{"connectionString":"Server=...;Password=..."} would put a live credential in the
ConfigDb — persisted, replicated to every node's artifact cache, readable by anyone with
config access — while the runtime silently ignored it and the driver failed to connect.
Discarding a secret on read is not the same as refusing to store it.
DraftValidator.ValidateSqlConnectionStringNotPersisted fails the deploy
(SqlConnectionStringPersisted) when a Sql-typed driver instance's DriverConfig, or the
DeviceConfig of any device beneath it, carries a top-level connectionString key.
- Both surfaces are checked because DeviceConfig is merged onto DriverConfig before the
DTO sees it — a credential pasted into the device blob is the identical leak.
- The key is matched case-insensitively: System.Text.Json binds ConnectionString to a
connectionString property by default, so an ordinal match would leave the obvious
bypass open.
- Only the top level is scanned; the DTO is flat, so a nested occurrence cannot bind and
is not the mistake this rule exists to catch.
- The message never echoes the value — validation errors reach the AdminUI, the deploy
log and the audit trail, and repeating the string there would leak the credential the
rule is refusing to store. Pinned by a test.
- Malformed/blank/non-object JSON never throws: a parse failure would take down every
other rule in the same pass.
16 tests. Verified falsifiable — with the rule disabled the 6 positive assertions fail
and the 10 negatives stay green, so none of them passes vacuously.
Design §8.2's "if an inline connectionString is ever permitted (dev convenience only),
the AdminUI redacts it" carve-out is withdrawn in the same change: redaction only hides a
credential that has already been written.
Not covered, and filed as a follow-up: ClusterNode.DriverConfigOverridesJson is a third
persistence surface for a driver config blob, but it is not part of DraftSnapshot, so
this validator cannot see it.
|
||
|
Joseph Doherty
|
b95b99ba8e |
fix(drivers): correct 16 wrong OPC UA status-code constants + guard them by reflection (#497)
Every value verified against Opc.Ua.StatusCodes in the pinned SDK (1.5.378.106) by reflection, not by copying siblings. All are client-visible: OPC UA clients branch on status. The three named in #497: - Historian.Gateway SampleMapper "BadNoData" 0x800E0000 -> 0x809B0000 (0x800E0000 is BadServerHalted) - Galaxy "BadTimeout" 0x800B0000 -> 0x800A0000 (0x800B0000 is BadServiceUnsupported) - TwinCAT BadTypeMismatch 0x80730000 -> 0x80740000 (0x80730000 is BadWriteNotSupported) BadTypeMismatch was wrong in three MORE drivers the issue did not name — FOCAS, AbLegacy and AbCip carried the same 0x80730000. Every call site is a FormatException/InvalidCastException conversion failure, so the name was right and the value was wrong in all four. Adding the reflection guard then surfaced nine further defects offline tests had never checked: - Galaxy GoodLocalOverride 0x00D80000 -> 0x00960000 (not a UA code at all) - Galaxy UncertainLastUsableValue 0x40A40000 -> 0x40900000 - Galaxy UncertainSensorNotAccurate 0x408D0000 -> 0x40930000 - Galaxy UncertainEngineeringUnitsExceeded 0x408E0000 -> 0x40940000 - Galaxy UncertainSubNormal 0x408F0000 -> 0x40950000 - TwinCAT BadOutOfService 0x80BE0000 -> 0x808D0000 (was BadProtocolVersionUnsupported) - TwinCAT BadInvalidState 0x80350000 -> 0x80AF0000 (was BadAttributeIdInvalid) - AbCip/AbLegacy GoodMoreData 0x00A70000 -> 0x00A60000 (was GoodCommunicationEvent) - Historian.Gateway GatewayQualityMapper Good_LocalOverride 0x00D80000 -> 0x00960000 Galaxy's whole Uncertain block read as though the OPC DA quality byte could be shifted into the UA substatus position; it cannot — the substatuses are an unrelated enumeration. GatewayQualityMapper already held the correct table, which is what the Galaxy values are now reconciled against. Guard: StatusCodeParityTests (Core.Abstractions.Tests, which already project- references every driver) reflects over every `const uint` in the deployed ZB.MOM.WW.OtOpcUa.Driver.*.dll whose name reads as a status code, and asserts it equals Opc.Ua.StatusCodes.<name>. Discovery is by convention, so a new driver assembly referenced by that project is covered with no edit here. It went red on all nine defects above before the fixes and now checks 109 constants green. Because reflection can only see a NAMED constant, two inline literals were hoisted so the guard can reach them: Galaxy's BadTimeout/Bad into StatusCodeMap, and GatewayQualityMapper's 15-entry table into a new HistorianStatusCodes. An inline `0x800B0000u, // BadTimeout` at a call site is exactly how the Galaxy defect survived. The drivers stay SDK-free by design; only the test project references Opc.Ua.Core, as the oracle. Also corrected: tests that pinned the wrong values (Galaxy StatusCodeMapTests, SampleMapperTests, GatewayQualityMapperTests — all written from the same bad source), a mislabelled comment in DriverInstanceActorTests, and stale constants in two design docs, one of them the unexecuted MTConnect driver design that would have propagated BadNoData's wrong value into a new driver. The CLI's SnapshotFormatter value->name table was checked against the SDK and is correct; no change needed. |