The two value-bearing fields were read asymmetrically and nothing said why.
whereValue went through ReadScalar (string/number/boolean — the design doc's own
wide-row example authors it as the number 7); keyValue went through the string-only
ReadString, so a numerically-authored key refused the tag outright. Nothing downstream
justified the split: both are BOUND as DbParameters and both are matched back as text —
SqlPollReader.IndexByKey stringifies the key cell precisely so an authored key can face
an INTEGER key column.
The refusal was also mis-explained. Inspect branched on ReadString(root, "keyValue") is
null, so a numerically-authored key drew "Sql KeyValue tag has no 'keyValue'" — telling
the operator the field is missing when it is right there in the blob, the #549 shape of
an authoring surface disagreeing with the runtime about WHY.
keyValue now uses ReadScalar; Inspect mirrors it with HasScalar and says what a usable
key is; SqlTagConfigModel loads it with the shared TagConfigJson.GetScalarText (which
also replaces SqlRowSelectorModel's private copy), so the editor is not left stricter
than the parser. An object/array/null key is still refused. docs/drivers/Sql.md updated.
RED first: 4 failing in Driver.Sql.Tests.
-003 is a documented Won't Fix: the mitigating control is the ConfigEditor page
policy on the only two surfaces that reach a browse session (/raw and /uns),
there is no HTTP endpoint for browse, and PageAuthorizationGuardTests pins both
pages to that exact policy. Recorded in docs/security.md under Control-Plane
Authorization and pointed at from SqlDriverBrowser's class summary.
ListSchemasSql has two consumers and the remarks only justified the divergence
from the catalog gate's viewpoint. SqlBrowseSession.RootAsync renders every row
as an expandable schema, so an Oracle browse offers every database user. The
narrower ALL_TABLES/ALL_VIEWS query is named, along with why it is not taken
here: it narrows the gate too, and that is a poll-path behaviour change only a
real Oracle server can gate.
Inspect parsed the TagConfig blob once itself (root kind + the present-but-discarded
changeTriggered/timerIntervalMs checks) and then called TryReadTagConfig(reference, out _),
which parsed the same string from scratch again for the scriptId rule. The field-by-field
read is now TryReadRoot(JsonElement), which both entry points share, so Inspect consults
the one authority over the root it already has. The public string-based TryReadTagConfig
signature other modules consume is unchanged.
Pure refactor: added CalculationTagDefinitionFactoryInspectTests (17 cases pinning the
exact wording, the rejection-vs-advisory split and their order, and the
never-stricter-than-the-runtime-read invariant). They pass unchanged against the
pre-refactor code — verified by stashing the production change.
ISqlDialect gains MaxBindParameters — the per-command ceiling the seam was
missing while modelling everything else that differs per backend: 2098 usable
of SQL Server's 2100 (sp_executesql spends two), 1000 for Oracle's IN-list
limit (ORA-01795, tighter than its 64k bind count), 65535 for the Postgres
Bind message.
SqlGroupPlanner.AddKeyValuePlans now emits several plans for one group when
its distinct keys exceed that ceiling, instead of one statement the provider
refuses after the connection opened — which left every tag in an EAV group
publishing BadCommunicationError on every poll, permanently, and reported the
database endpoint as down. Chunks share the group key; members follow their
own key so a shared keyValue never splits. SqlPollReader is unchanged: it runs
one command per plan and slices each plan's own Members back, proven by a
fixture read at a ceiling of 1.
OnDependencyValue ran every change-triggered dependent's script INLINE, under _lock,
via TimedScriptEvaluator.RunAsync(...).GetAwaiter().GetResult() — on the Akka dispatcher
thread of Runtime's DependencyConsumerMuxAdapter, for up to RunTimeout (2 s default) per
arrival, with the state lock held for the whole run.
The callback now records the value, PREPARES the affected tags under the lock (definition
+ warm-up gate + dependency snapshot taken at trigger time) and queues them for a single
serialized evaluation pump. _lock is a short-held state lock again; a new _evalGate
serializes the pump against the timer ticks and is taken only by evaluation workers, so
no host thread can wait on a script. Lock order is always _evalGate -> _lock.
Semantics preserved: per-tag ordering (one FIFO pump), the warm-up gate BEFORE the
evaluator runs, dedup/Bad-transition/recovery publishes, and — because the dependency
snapshot is taken when the trigger fires rather than when the pump catches up — a burst
of arrivals still evaluates against the values that triggered it.
Driver.MTConnect-005/-006/-007, each TDD'd red-first.
- -005: _options was the one field at the StartCoreAsync commit point published by a
plain write and read lock-free from ReadAsync/WriteAsync/DiscoverAsync/HostName/health.
Now Volatile.Write/Volatile.Read like _session/_binding/_announcedInstanceId, pinned by
a source scan (MTConnectOptionsPublicationTests) that fails on any unfenced touch.
- -006: TryFormatValue guarded null and Array, but Convert.ToString renders ANY
non-IConvertible value as its type name under a Good status. Guard is now the property
the remark relies on; the accepted set (string/bool/numeric/DateTime) is unchanged.
- -007: MTConnectAgentClient.Dispose was a check-then-set on a non-volatile bool, so two
racing disposals both passed and the loser cancelled an already-disposed CTS. Now an
Interlocked.Exchange claim over an int, with the request seams reading it volatile.
(1) DisposeAsync disposed _gate unconditionally. SemaphoreSlim.Dispose does not
complete pending async waiters, and the in-flight call's Release() then threw
ObjectDisposedException from inside the semaphore (swallowed by the finally), so
a caller parked in WaitAsync was never released and its call never returned. The
gate is now left alive - the pattern SqlPollReader documents - so the waiter is
admitted and meets the session's own guarded ThrowIf(_disposed, this).
(2) LastUsedUtc advanced only on completion, so a slow catalog call (the AdminUI
allows 20 s) was invisible to the TTL reaper for its whole duration and a session
at the edge of its TTL could be disposed mid-call. It is now stamped on entry
too.
PreferredTimestampColumn nominated the first column mapping to DriverDataType
.DateTime, and that set includes date-only types (SQL Server / PostgreSQL
'date'). A table whose first date/time column is a 'date' committed
rowSelector.topByTimestamp = <date>, making the wide-row read ORDER BY <date>
DESC LIMIT 1 - which ties for every row written that day, so 'the newest row'
became storage order among the ties.
Time-bearing candidates now outrank date-only ones; a date-only column is still
stated when it is all the table has, so such a table stays bindable. Oracle's
DATE carries a time of day and is exempt - pinned by a control dialect that
differs from the SQLite one only in the Provider it reports.
Driver.Mqtt-006 (Low, concurrency). DisposeAsync disposed _gate while StopAsync
cancels the reconcile loop WITHOUT awaiting it, so a ReconcileAsync already in
flight could be inside _gate.WaitAsync, or on its way to the finally-release,
when the semaphore went away. The release then threw ObjectDisposedException
naming SemaphoreSlim, logged as a reconcile failure — telling an operator
nothing about what happened and hiding whatever the reconcile was really
failing on.
Left undisposed, matching MqttConnection's documented rule for its own lifecycle
primitives and its stated rationale: a late callback racing teardown should be a
benign no-op rather than an ObjectDisposedException naming the wrong type. Two
lifecycle types in one module disagreeing about this is what gets copied into
the third. Nothing is leaked — the gate holds no timer and no registration — and
ClaimAsync still re-checks _disposed under it, so no claim can be created after
dispose either way.
Test is deterministic rather than timing-dependent: the gate is taken before the
connect starts, so a MiniBroker in blackhole mode (TCP accepted, CONNECT never
answered) accepting the socket proves the claim is inside the gate when dispose
runs. RED: log.Captured contained "ObjectDisposedException".
Driver.Mqtt-007 (Low, error handling). OnReconnectedAsync throws when a
re-subscribe is granted zero filters — MqttConnection's contract is that a
throwing Reconnected subscriber tears the fresh session down and retries under
backoff, because a connected-but-deaf driver is the one outcome this path must
never serve. But when the live subscribed topics resolved to zero filters in the
first place it logged a warning and returned, reaching the same "nothing was
re-established" state by a different route and giving it the opposite answer.
Throw there too, and say so in the remarks so the type states ONE rule. It costs
no other tag — every live topic is in the set that resolved to nothing, so there
is nothing left flowing to protect, which is the whole reason the partial-
rejection case is exempt — and it is not permanent: the retry runs against
whatever Register has published by then.
Test reaches the state the only way production can: Register prunes
_subscribedTopics of the topics its new table no longer carries, but a SUBSCRIBE
already in flight records its own topic AFTER that prune ran, so a deploy landing
inside the round-trip leaves a live topic no authored tag resolves to. Paired
with the control that a reconnect with no live subscriptions at all stays a
no-op — throwing there would tear down the session of every driver whose tags
have not been subscribed yet.
Driver.Mqtt.Transport-002 (Low, correctness). Credentials were attached only
when Username was non-empty, so a config carrying a Password and no Username
connected anonymously with the secret silently discarded — the deployment reads
as authenticated and is not, which is the direction an operator cannot see from
the config.
Warned rather than sent. MQTT 3.1.1 may not carry a password without a
username, so the password-only CONNECT is not available on every protocol
version this factory serves, and synthesizing one under V500 only would change
the wire behaviour of deployments that connect happily today. The configured
values stay authoritative; the drop just stops being silent. The secret is
never formatted into the message.
Tests: the warning fires and does NOT contain the password, plus the falsifying
control that an ordinary anonymous config (neither credential set) stays silent
— without it the fix could warn on every connect and still pass.
NAMEDATALEN-1 (63) and Oracle 12.2's 128 are byte limits in the database
encoding, not character counts, so a 32-character Cyrillic name passed
ident.Length and was then silently truncated (Postgres) or rejected by the
server (Oracle). Both dialects now measure Encoding.UTF8.GetByteCount and say
'bytes' in the rejection. SqlServerDialect deliberately keeps ident.Length —
sysname is nvarchar(128), a real character limit — pinned by a control test.
SqlDriverProbe surfaced 'Config JSON is invalid: ' + ex.Message, where
SqlDriverBrowser.ParseDocument deliberately withholds the identical message
because the malformed text may itself carry a connection string. The two read
the same blob shape - the AdminUI Sql picker composes a document with a
top-level connectionString literal - so a malformed paste reaches both. The
probe now reports the failure class only, matching the browser.
The comments and docs that described Driver.MTConnect-003 as live at HEAD would
have been wrong the moment 64be41d8 landed:
- code-reviews/Driver.MTConnect/findings.md — Driver.MTConnect-003 Resolved,
with the fixing commit; module open-finding count 5 -> 4.
- code-reviews/Driver.MTConnect.Contracts/findings.md — the one clause inside
Contracts-001's prose that asserted 003 was Open.
- docs/drivers/MTConnect.md — the coercion-precedence warning now states that a
browse-committed tag resolves at step 1 CARRYING the step-3 answer, and that
step 3 remains the rescue for a hand-authored / CSV / API blob with no type.
- MTConnectDriver.BuildBinding, MTConnectDataTypeInference and
MTConnectDriverOptions.RawTags remarks — same correction, each keeping a short
historical note of what the defect was so a reader meeting the old blobs still
understands them.
Comment-only in src; the driver suite (538) and AdminUI suite (1474) re-run green.
Driver.Mqtt.Transport-001 (Medium, Security). allowUntrustedServerCertificate
and caCertificatePath are independent settings, and when both are set the hatch
wins SILENTLY: ConfigureTls installs the accept-everything validator and
returns before the pin is consulted. The combination is not exotic — the hatch
is what an operator turns on while standing a broker up and the pin is what
they add afterwards — and the failure is silent in the direction that matters:
the config reads as pinned, and nothing validates anything.
Both knobs are explicit, so the flag stays authoritative (overriding it would
break a deployment that genuinely wants the hatch, and refusing to build would
fault a driver over a config we have been accepting). Saying so once, at the
seam that resolves the conflict, is the proportionate answer. No behaviour
change; the factory's optional ILogger already exists for this class of
deferred diagnostic and never formats credentials.
CalculationDriverProbe.Opts (Test-Connect) omitted AllowTrailingCommas and
ReadCommentHandling that CalculationDriverFactoryExtensions.JsonOptions
(spawn) and CalculationDriver.ReinitJsonOptions (in-place ApplyDelta) both
set, so a config blob the real deploy path tolerates could fail Test-Connect
as "invalid JSON" even though deploying it would succeed. Aligned the three
option sets. Also adds CalculationDriverProbeTests.cs, which previously did
not exist at all for this driver type's Test-Connect surface.
Part of code-reviews/Driver.Calculation/findings.md (-003, -004).
Code review of Driver.Calculation found two real defects, both TDD-fixed:
- CalculationDriver's ctor tried to reuse its ILogger<CalculationDriver> as the
evaluator's ILogger<CalculationEvaluator> via an `is` check that can never
succeed (unrelated closed generics), so CalculationEvaluator always ran with
a NullLogger regardless of what the driver was constructed with. Fixed by
having CalculationEvaluator accept the non-generic ILogger instead.
- EvaluateTag called the evaluator BEFORE checking whether every declared
dependency had arrived, so a multi-dependency change-triggered calc tag
evaluated its script against a partial snapshot on every warm-up tick.
VirtualTagContext.GetTag returns a Bad snapshot (.Value == null) for a
missing dep, so the idiomatic `(double)ctx.GetTag("x").Value` cast throws
NullReferenceException — caught and logged as a spurious "script threw"
script-log Warning on ordinary deploy/restart warm-up, for the standard
multi-input calc-tag shape. Fixed by gating evaluation itself on
"all deps arrived", not just the publish.
See code-reviews/Driver.Calculation/findings.md for full detail (5 findings:
001-004 Resolved, 005 Open/cross-module).
Findings from the per-module review of Driver.Mqtt at b990adae
(code-reviews/Driver.Mqtt/findings.md records all ten; these are the fixed
ones).
Driver.Mqtt-001 (High) — MqttSubscriptionManager._subscribedTopics is a claim
about the broker session it established filters on, and AttachTo never cleared
it, but the manager OUTLIVES a session rebuild (AdoptOptions keeps it whenever
the ingest identity is unchanged). After a driver-host reconnect
(ShutdownAsync + InitializeAsync, the #527 watchdog's retry) or a
ReinitializeAsync that only changed the endpoint/credentials, the host's
re-subscribe resolved an empty pending set, sent no SUBSCRIBE at all, and left
the driver Connected/Healthy on a session carrying ZERO subscriptions — no
error, no Bad status, values frozen until the new session happened to drop.
AttachTo now forgets them, mirroring SparkplugIngestor.AttachTo's reset.
Driver.Mqtt-002 (High) — InitializeCoreAsync published _connection and
_primaryHost over whatever the driver already held, so a second initialize
stranded a connected, self-supervising broker session (plus a second Sparkplug
primary-host claim) that no later dispose could reach. Reachable without any
teardown of ours via DriverInstanceActor.AdoptConfigDuringInit. Now tears down
first, under the gate both callers already hold.
Driver.Mqtt-003 (Medium) — connectTimeoutSeconds is [Range]-annotated but
unvalidated on the runtime path; 0 makes every deadline fire immediately and a
negative value throws ArgumentOutOfRangeException out of ShutdownAsync and
DisposeAsync. Normalized once, at the seam every options instance passes.
Also: ShutdownAsync keeps the live last-message instant (-004), three usings
left behind by #512 removed (-005), stale remarks corrected (-009, -010), and
docs/drivers/Mqtt.md now distinguishes the two kinds of reconnect.
Tests 708 -> 713. All three regression tests were confirmed to fail against
the un-fixed code, with two controls that pass in either state.
Driver.Sql-001 (Medium): SqlCatalogLoader.ReadDefaultSchemaAsync documented a
fallback to the sole visible schema when the server reports no default schema and
did not implement it, so a login with no DEFAULT_SCHEMA (or a Postgres search_path
that resolves to nothing) rejected EVERY unqualified authored table at the catalog
gate — a whole-driver BadNodeIdUnknown outage against a healthy database. Fixed;
SqlCatalogDefaultSchemaTests written red first, with a control proving the
ambiguous multi-schema case still refuses to guess.
Driver.Sql-003 (Medium): docs/drivers/Sql.md spelled the WideRow row selector at
the top level, but the parser reads those keys only inside a nested rowSelector —
a tag authored from the doc is refused and reads BadNodeIdUnknown (the #552 class,
via the documentation). Also removed the documented-but-nonexistent pollIntervalMs
per-tag field.
Driver.Sql-007 (Low): dropped the unused Microsoft.Data.SqlClient PackageReference
left behind by the .Dialect extraction (#526).
Open: 002 (no IN-list chunking vs the 2100/1000 provider bind caps), 004, 005, 006,
008, 009, 010.
Driver.Historian.Gateway-001 (High): AlarmEventMapper wrote the alarm's
Severity wire property as the .NET enum name (e.g. "High"), but
EventMapper's read side int.TryParse()s that same property and silently
falls back to the minimum severity on any unparseable value - so every
alarm event's severity round-tripped through this module's own mappers
collapsed to 1 (Low), contradicting AlarmHistorianEvent.Severity's own
documented contract ("mapped to Historian's numeric priority"). Now
mapped through a numeric priority mirroring the existing
NativeAlarmProjector.MapSeverity convention (200/500/700/900).
Driver.Historian.Gateway-002 (Low): GatewayTagProvisioner never disposed
its IHistorianGatewayClient, unlike its GatewayHistorianDataSource /
GatewayAlarmHistorianWriter siblings. Now implements IAsyncDisposable so
the DI-owned singleton's gRPC channel closes gracefully at shutdown.
Both TDD: new/corrected tests confirmed RED against the pre-fix code.
SparkplugRebirthScope (the AdminUI's typed input) rejects '+'/'#' and its own
remarks say it is 'advisory, not the gate' because MqttBrowseSession
.ResolveRebirthTargets 'refuses an unusable one BEFORE any message leaves'. That
claim was false: the bare {group}/{edgeNode} branch — honoured on purpose, since a
node that has not birthed is the prime rebirth target — gated on IsUsableLabel,
which checks length and control characters only. A scope of 'Plant1/#' therefore
published spBv1.0/Plant1/NCMD/# , a PUBLISH the MQTT spec forbids and a conformant
broker answers by closing the browse connection; '+' would have fanned one
operator click across every matching edge node.
The check is a separate IsUsableRebirthSegment rather than a tightening of
IsUsableLabel: that helper also judges Sparkplug METRIC names, which are arbitrary
publisher-chosen UTF-8 and never part of a topic, so tightening it there would drop
legitimate metrics from the tree. Cases 1 and 2 cannot reach this shape — their ids
come off SparkplugTopic.TryParse, which already rejects both characters.
Driver.Mqtt.Tests 724 (was 708).
Driver.MTConnect-001 (Medium): StreamVerdict.Reopen reset the reconnect ladder
unconditionally. The InvalidDataException catch that routes an Agent OUT_OF_RANGE
to a /current re-baseline also catches an unparseable chunk and an over-sized
multipart part - neither of which a re-baseline can resolve - so the pump reopened
and failed with no delay at any point: two HTTP requests and one Warning per
iteration, forever. The Reopen arm now uses the same discriminator the transient
arm does (did THIS connection deliver a chunk?), so a buffer roll behind a working
stream still reopens instantly while a re-baseline that resolved nothing advances
BackoffFor.
Driver.MTConnect-002 (Medium): MTConnectDriverProbe requested {AgentUri}/probe
while MTConnectAgentClient requests {AgentUri}/{DeviceName}/probe, so a mistyped or
since-renamed deviceName showed GREEN on Test Connect and then Faulted the driver
at InitializeAsync (an unknown device path answers MTConnectError under HTTP 200).
The probe's minimal DTO now reads deviceName - the rule is 'which keys compose the
URL', not 'which keys are cheap to read'.
Driver.MTConnect-008 (Low): corrected two stale claims that a since-changed
RawBrowseCommitMapper invalidated (see Driver.MTConnect-003, which remains open and
belongs in AdminUI), and made DiscoverAsync log the options snapshot it
deliberately read once instead of the field.
Tests: 538 pass (534 base + 4 new, each red before its fix); AdminUI.Tests 1466 pass.
A wildcard writeTopic was published verbatim. MQTT-3.3.2-2 forbids a wildcard in a
PUBLISH topic and a conformant broker answers a protocol violation by closing the
connection — which for this driver is the one connection every subscription lives
on, so a single write to a mis-authored tag took the whole driver dark. The
Sparkplug side had the same hole plus a worse one: the binding tuple IS the
NCMD/DCMD topic, so a '/' in groupId/edgeNodeId/deviceId silently addressed a
different edge node than the tag names.
MqttWriteEncoder now refuses both as NoWriteDestination (-> BadNotWritable); the
read path is untouched. metricName is exempt on purpose — it rides in the payload
and Sparkplug's own canonical names use '/'.
MqttTagDefinitionFactory.Inspect gained the matching ADVISORIES (never rejections
— the mappers accept these blobs, and the gate must never be stricter than the
runtime). Both rules were already hard errors in the AdminUI's typed editor, but
that is one authoring path of several: the raw-JSON textarea, CSV import and the
API all reach the deploy gate directly, so the authoring surface was stricter than
the gate — the drift class the #549-#559 arc exists to close.
Shared shape rules single-sourced in the new internal MqttAddressRules.
Driver.Mqtt.Tests 724 (was 708); AdminUI.Tests 1466; ControlPlane.Tests 261.
FromTagConfig and FromSparkplugTagConfig each carried a framing pre-check
(`tagConfig[0] != '{'`) evaluated on the UNTRIMMED string, so a TagConfig blob
with leading whitespace was refused by MQTT alone and the tag read
BadNodeIdUnknown forever — while the other six structural-parse drivers
(Modbus, S7, AbCip, AbLegacy, TwinCAT, FOCAS) hand the identical blob to
JsonDocument.Parse, which skips leading whitespace per the JSON spec.
The clause is dropped (the blank guard stays). JsonDocument.Parse plus the
root-ValueKind check now decide framing, exactly as for the six siblings: an
array/number/string root fails ValueKind, non-JSON is a JsonException. The gate
was a validator, not a discriminator — no caller (MqttSubscriptionManager,
SparkplugIngestor, MqttTagConfigModel, Inspect) used the first byte to pick a
format — so verdicts are identical for every input except whitespace-padded
JSON objects, which now get their content's verdict.
Inspect needs no logic change (it has trimmed since #556 and hands the RAW
reference to the mappers), but its comments claimed the padded-blob refusal was
production-reachable; rewritten. That shape was also MQTT's only vehicle for the
catch-all backstop, which is now unreachable-today like FOCAS's — so its test is
replaced by the same count-relation pin, and the backstop code stays as
defence-in-depth.
Tests: both mappers pinned by padded-vs-unpadded DEFINITION equality (a record,
so field-wise), padded non-object/malformed negative controls, an Inspect
verdict+inspection parity theory over the accepted AND rejected corpus with an
anti-vacuity control, and a padded MQTT corpus entry in the AdminUI
authoring/runtime parity suite.
#556 item 1 — leading whitespace defeated every Inspect's pre-filter. Seven
factories (Modbus/S7/AbCip/AbLegacy/TwinCAT/FOCAS/Mqtt) tested
`reference[0] != '{'`, so a padded blob returned EMPTY and a tag the driver
refuses deployed clean. The check now trims first, matching the idiom
Sql/Calculation/MTConnect/OpcUaClient already used. The raw reference is still
what is handed to the mapper — the driver reads the stored blob, not a trimmed
copy — so the gate keeps mirroring the runtime rather than becoming lenient.
Both directions are pinned: a padded blob the mapper accepts still draws
nothing, and one that is not object-framed AFTER trimming still draws nothing
(it is not an equipment-tag TagConfig at all).
Six of the seven mappers hand the blob to a whitespace-tolerant
JsonDocument.Parse, so their verdict is unchanged by padding. MQTT is the
exception: both its readers carry the same framing gate, so a padded blob is
genuinely refused — which is why the untrimmed pre-filter was a real defect
there and not a cosmetic mismatch.
#556 item 2 — FOCAS and MQTT appended advisories unconditionally and then gated
the catch-all rejection message on `warnings.Count == 0`, so an advisory (which
fires on ACCEPTED blobs too, and says nothing about a refusal) silently
swallowed the only message naming a refusal nothing else can. The two classes
now accumulate in SEPARATE lists and the backstop gates on the rejection count
alone — structural, not string matching. MQTT's swallow was reachable in
production (a padded blob its readers refuse while every describe branch finds
the parsed document well-formed); FOCAS's backstop is currently unreachable by
audit, so its guard is protective.
#557 item 2 — decided and pinned: KEEP the union verdict. Evidence:
FromSparkplugTagConfig hard-codes PayloadFormat=Json and never consults the key;
only MqttSubscriptionManager (the Plain ingest path) decodes by it; and
MqttTagConfigModel.ToJson deliberately omits the key entirely in Sparkplug mode
(#513). So a Sparkplug driver genuinely serves the tag and rejecting it would be
stricter than the runtime. The cost — served behaviour depends on the driver's
MqttMode, which no tag row records — is paid by an ADVISORY naming the key and
the mode dependency, not by changing the verdict, so both gates keep agreeing.
The same invalid value on a blob the union REFUSES stays a rejection; the class
is decided by the accept gate, never by the key.
Falsified: reverting Modbus's trim reddens the padded-rejection theory;
neutering MQTT's backstop guard to the pre-#556 shape reddens the
advisory-plus-unnameable-rejection test; disabling the #557 advisory reddens the
contract pin; and for FOCAS, removing a describe branch alone stays green (the
backstop covers it) while additionally neutering the guard goes red.
`OpcUaClientTagDefinitionFactory.TryReadNodeId` still honours a legacy PascalCase
`NodeId` when the v3 camelCase `nodeId` is wholly absent, but
`OpcUaClientTagConfigModel` read only `nodeId` — so a legacy blob was SERVED by
the driver yet opened with an empty reference field and could not be saved
untouched. Authoring stricter than the reader, which #554 names as never
legitimate: the editor was blocking config the runtime honours.
Fixed with the #550 pattern (read both spellings, normalise on save), NOT by
retiring the spelling from the reader — that would break every legacy blob the
driver currently serves at the next deploy, a data-plane regression traded for
an editor asymmetry.
The precedence is PRESENCE-shadows, not first-non-blank. The reader's
`TryGetProperty("nodeId", out n) || TryGetProperty("NodeId", out n)`
short-circuits on the mere PRESENCE of the camelCase key and then checks
string-and-non-blank ONCE, so a blank / non-string / JSON-null `nodeId` beside a
valid legacy `NodeId` is a REJECTED blob, not a fallback. Copying MTConnect's
`FirstNonBlank` walk here would have made authoring WEAKER than the reader —
the same defect in mirror image.
- `NodeIdKeys` drives both halves, so the read and the drop cannot drift.
- `ToJson` writes `nodeId` and drops `NodeId`; the TagModal history-merge keys
(`isHistorized` / `historianTagname`) are disjoint and untouched.
- `Validate`'s required-reference fast path is now precedence-aware by
construction, so it agrees with the #554 parse consult across the whole
rejection surface.
- The `TagConfigNonEnumParityTests` exemption for OpcUaClient is REMOVED — its
anti-vacuity guard `Every_deliberately_stricter_entry_is_still_exactly_that`
went red on the fix, which is this change's falsification control. The legacy
shapes moved into the corpus, where both fleet theories now cover them.
Tests: a nine-row precedence theory driving the authoring verdict against the
READER's verdict per shape, on the original blob AND on the `ToJson()` output;
no-retype (legacy loads showing its reference, saves canonical, same node id
before and after); no-drift (output never carries `NodeId`). Each half was
reverted in isolation to confirm it reddens tests of its own: read-only-revert
=> 6 red incl. the fleet parity theory; drop-only-revert => 4 red.
Suites: AdminUI.Tests 1459/1459, ControlPlane.Tests 261/261,
Driver.OpcUaClient.Tests 143/143, Driver.OpcUaClient.Browser.Tests 10/10.
Two guards enumerated the same RawBrowseCommitMapper.BuildTagConfig seam against
two hand-authored lists, which can only drift. The older one's list was also
misnamed: DriverDispatchMapParityTests.NonBrowsableDriverTypes really meant
"types that produce the generic key" — AbLegacy is genuinely non-browsable yet
correctly absent from it, because it has a typed branch.
Retire the older fact. RawBrowseCommitMapperParityTests.GenericKeyExemptions is
now the single hand-authored list for this seam: named for what it holds, one
written reason per entry, and checked against a DI-derived browsable set rather
than a second list. The one assertion the retired fact made that nothing here
covered — set EQUALITY, i.e. an exemption must still DESCRIBE the mapper — moves
across as Every_generic_key_exemption_still_commits_to_the_generic_key. Without
it, a driver that gained a typed branch would keep a silently inert exemption.
Item 2: OpcUaClientDriverBrowser and GalaxyDriverBrowser returned string
literals ("OpcUaClient", "GalaxyMxGateway") where Sql/Mqtt use DriverTypeNames.
The values matched, and nothing guarded that. Repoint both at the constants —
DriverTypeNames.Galaxy carries the identical value — and add
Every_registered_browser_is_keyed_by_a_declared_driver_type_name over the same
AddAdminUI() read-back. Compared ORDINALLY: BrowserSessionService indexes on the
exact persisted string, and case-only drift (TwinCat/TwinCAT) is what
DriverTypeNames exists to end. Browser projects are out of reach of
DriverTypeNamesGuardTests, which pins the driver FACTORIES' names.
Falsified, four ways, each red naming the offender and green on revert:
- #555 re-run: disabling the Sql branch reddens the two enumeration facts
("...no dedicated BuildTagConfig branch...: Sql"), the same catch the
retired fact made.
- giving Modbus a typed branch reddens ONLY the new stale-exemption fact,
proving it is not redundant with the four that were already there.
- SqlDriverBrowser.DriverType => "SQL" and Galaxy => "GalaxyMXGateway" each
redden the new browser-key guard by name — case-only typos, which an
OrdinalIgnoreCase compare would have waved through.
`RawBrowseCommitMapper.BuildSqlTagConfig` emitted a WideRow blob carrying only
table + columnName. A wide-row tag must also say WHICH row it reads, so both
`SqlEquipmentTagParser` (the runtime authority) and `SqlTagConfigModel.Validate`
(the authoring gate) refused it — a browse-committed Sql tag was never readable,
first silently as BadNodeIdUnknown and, since #551 put the gate on the import,
loudly at commit.
`SqlBrowseSession` now states a fourth address field: the table's preferred
date/time-typed column, computed from the columns its attributes call ALREADY
read (no extra catalog round-trip). Candidacy is decided by
`ISqlDialect.MapColumnType` reporting `DriverDataType.DateTime` — never by column
name, which would order a wide-row read by a string column that merely looks like
a timestamp. Multiple candidates tie-break on FIRST IN CATALOG ORDER (every
dialect's column SQL orders by ordinal position), so the pick is stable across
calls and servers; it is a visible, editable default the typed editor round-trips.
No candidate (a table with no date/time column) is an operator-actionable refusal,
not an invented selector: the only remaining selector is a whereColumn/whereValue
pair whose value is data, not schema, so `DescribeUncommittableLeaf` refuses that
leaf by name, names the table, and points at the tag editor.
The session and the mapper now agree via `SqlBrowseAddressFields` constants rather
than two independently spelled string literals — a disagreement there is silent,
and is exactly the shape this defect had.
`RawTagConfigGateTests`' Sql leg flips from "records the gap" to "browse-commit
output passes the gate", with the no-candidate case pinned as its own test.
Gitea #551.
1. CSV import ran no driver-typed config gate. Its typed columns were always
strictly parsed, but the free-form TagConfigJson column reached the database
unread, so a CSV could author a tag the driver rejects — one that deploys
green and reads BadNodeIdUnknown forever. RawTagCsvMapper.Parse now names the
offending row in the staged review grid, and RawTreeService enforces the same
gate on CreateTag/UpdateTag/ImportTags: a form cannot be the guarantee (the
API and any future caller reach none of them, and this repo has no bUnit).
A rejected import row is an ERROR — the batch inserts nothing and the row is
named — never a skip.
Note the brief's premise that manual entry rides ImportTagsAsync is wrong:
RawManualTagEntryModal commits row-by-row via CreateTagAsync, which is why
the gate had to land on all three write paths.
2/3. The seven pre-#549 Inspect methods ran a hand-list of DescribeInvalidEnum
checks, so every rejection cause not on the list was invisible. They are now
ACCEPT-GATE-FIRST (the MTConnect/Sql/OpcUaClient/Calculation shape): ask the
driver's own reader for the verdict, then explain the refusal, with a
catch-all so "the parse rejects but Inspect said nothing" is unreachable.
That closes S7's absent dataType, AbCip members[], Modbus stringByteOrder and
the type-dependent guards, plus causes the issue did not name (absent
region/address/tagPath/symbolPath/topic, TwinCAT Structure, MQTT's Sparkplug
tuple). FOCAS and MQTT keep advisories about ACCEPTED blobs, so they
diagnose rather than return empty.
Authoring half: ModbusTagConfigModel gains stringByteOrder;
AbCipTagConfigModel asks the new AbCipTagDefinitionFactory.DescribeInvalidMember;
S7TagConfigModel captures whether the SOURCE blob declared dataType and asks
S7TagDefinitionFactory.DataTypeKey/DeclaresDataType. One authority per rule.
4. Doc sweep: the six factory Inspect summaries, EquipmentTagConfigInspector's
class remarks and AdminOperationsActor's comment all claimed invalid enums are
"silently defaulted by the lenient runtime" — false since R2-11 Phase C made
the runtime reads strict. They describe rejections now.
MTConnectTagDefinitionFactory.TryFromTagConfig resolves the coercion type as
FirstNonBlank(blob.DriverDataType, blob.DataType) — driverDataType OUT-RANKS
dataType — while MTConnectTagConfigModel read and wrote dataType alone. That
broke in both directions at once:
* a blob carrying "driverDataType": "Strng" was accepted at authoring while
the runtime rejected the tag (the residual slice of #549), and
* a VALID driverDataType silently overrode whatever type the editor displayed
and saved, so the editor showed a lie and a save RETYPED the tag.
Both halves land together, mirroring the existing IdKeys pattern:
* FromJson reads DataTypeKeys = [driverDataType, dataType] in the runtime's
precedence, with the runtime's own per-key semantics — absent/JSON-null and
present-but-blank fall through, a non-string token stops the walk (the
runtime deserialise throws JsonException on it and rejects the whole tag),
and the raw TOKEN is captured so Validate() judges what the driver judges.
* ToJson drops every non-canonical DataTypeKeys entry and writes dataType as
the enum NAME, so the alias can never out-rank the key the editor writes on
the next load.
* Validate() now names the offending spelling rather than always saying
"dataType".
Doing only one half silently retypes existing tags on their next save; the
falsification runs recorded in the PR confirm each half-fix reddens a distinct
test.
MTConnectTagConfigModel read `dataType` with TagConfigJson.GetEnum, which falls
back to String on ANY unrecognised value. A misspelled name ("Strng") from a
hand-authored blob, a CSV import or the API was therefore accepted at authoring
and silently rewritten, while the driver's strict parse rejected it and the tag
died with BadNodeIdUnknown. Found by the G-7 parity tests' falsifiability
control, which had to use the digits-only shape to have a case that failed.
The rule is not restated in the AdminUI. MTConnectTagDefinitionFactory exposes
its parse as `IsAcceptedDataTypeName`, documented as the one authority the
driver, the deploy gate and the authoring model all consult; the model's
Validate() asks it about the raw value the blob carries. The old digits-only
message is NOT retained as a special case: the runtime draws no distinction
between a bare number and a typo — one predicate, one Inspect warning — so a
second message here would be a second statement of a rule this model does not
own. The raw value is captured as the JSON TOKEN, because the runtime
deserialises the key into a `string?` DTO field and an unquoted number throws
there too.
The survey found the same asymmetry at six more editors, contrary to the
assumption that those runtimes silently default: R2-11 Phase C moved every
driver's TagConfig enum read to TagConfigJson.TryReadEnumStrict, which REJECTS
the tag on a present-but-invalid string, while the editors kept loading them
with a silent fallback. Modbus (region/dataType/byteOrder), S7, AbCip, AbLegacy,
TwinCAT and FOCAS now check their enums in Validate() through the runtime's own
TagConfigJson.DescribeInvalidEnumText — added alongside a public
ClassifyEnumText that TryReadEnum now delegates to, so the JSON-token door and
the string door reach one implementation. Mqtt and Sql already carried
equivalent strict checks and are untouched.
Care taken NOT to become stricter than the runtime: a present-but-non-string
token stays accepted for those six (TryReadEnum calls it absent), as does an
absent key, each pinned by a test.
Also adds a direct ProjectReference to Core.Abstractions from the AdminUI — the
save gate now leans on a type it was consuming only transitively.
G-7 wired OpcUaClientTagDefinitionFactory.Inspect into the deploy-time TagConfig
gate, which gave ControlPlane a ProjectReference to
Driver.OpcUaClient.Contracts — and with it, transitively, the whole OPC UA client
stack (5 OPCFoundation packages) into every host that loads the control plane.
NamespaceMap was the only reason Contracts carried
OPCFoundation.NetStandard.Opc.Ua.Client: its entire public surface is
ISession/NamespaceTable/NodeId-typed. It moves to a new leaf project,
ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient.Session, referenced by the runtime .Driver
and by .Browser. Contracts is left with Core.Abstractions and NOTHING else —
the same shape as Calculation.Contracts.
The namespace stays ZB.MOM.WW.OtOpcUa.Driver.OpcUaClient, so no consumer's
`using` changed. The .Session -> .Contracts direction is deliberate: the blob
read NamespaceMap needs is OpcUaClientTagDefinitionFactory.TryReadNodeId, the
one authority it shares with the deploy gate, and that must stay in the
dependency-light half.
Why a new project rather than folding NamespaceMap into the runtime .Driver and
having .Browser reference it (the Driver.Mqtt.Browser #512-style deliberate
exception): the AdminUI project-references .Browser and NOT .Driver, so that
shape would pull the OpcUaClient driver runtime — plus Core,
Opc.Ua.Configuration and Secrets.Abstractions — into the AdminUI's build/deploy
graph. That is exactly the violation #512 spent two extractions removing from
Mqtt.Browser; re-introducing it to save one csproj is a bad trade.
Guard: ContractsIsDependencyLightTests (Driver.OpcUaClient.Tests) asserts the
Contracts manifest names no Opc.Ua* assembly, plus a counterpart asserting
.Session does. An ABSENCE assertion is sound on the manifest where a presence
one is not — the compiler elides an unused reference, but it MUST emit one the
moment Opc.Ua-typed code is added. Falsified by temporarily re-adding the
package plus an Opc.Ua.NodeId-typed member: red on Opc.Ua.Types, green again on
removal.
Evidence — `dotnet list ControlPlane package --include-transitive | grep -i opc`:
before: OPCFoundation.NetStandard.Opc.Ua.{Client,Configuration,Core,
Security.Certificates,Types} 1.5.378.106
after: none (and 0 Opc.Ua*.dll in ControlPlane's output directory)
Deferment-register G-7: these four were validated by the AdminUI's TagConfigValidator at
authoring but by nothing at deploy, so config arriving by CSV import or API met no gate at
all. The deploy gate dispatches to a driver's *TagDefinitionFactory.Inspect in its Contracts
assembly, and none of the four had one.
Rather than mirror the AdminUI models (a mirror of an accept/reject boundary drifts silently,
with the gate describing a shape the driver no longer reads), each Inspect is built on the
driver's own runtime parse:
- Sql wraps the existing SqlEquipmentTagParser, which already lived in Sql.Contracts. The
P3-deferred Query model stays refused; Inspect surfaces the refusal rather than weakening it.
- MTConnect's blob reader MOVED out of MTConnectDriver (it was private, with a private DTO)
into MTConnectTagDefinitionFactory; the driver now calls it. Behaviour unchanged.
- OpcUaClient's nodeId read MOVED off NamespaceMap into OpcUaClientTagDefinitionFactory;
NamespaceMap now calls it. Behaviour unchanged.
- Calculation gains a Contracts assembly it never had. Only the JSON reading moved there
(CalculationTagDefinitionFactory.TryReadTagConfig); building a CalculationTagDefinition
still needs EquipmentScriptPaths from Commons and stays in the driver, so the split is at
the blob boundary. The new project is deliberately reference-free — a calc blob is strings,
a bool and an int, with no Core.Abstractions type in it.
Each Inspect runs the parse FIRST and returns empty on success, so "never stricter than the
runtime" is structural rather than a promise: a warning is only reachable for a blob the
driver would have refused and then skipped silently. Calculation additionally reports two
present-but-silently-discarded fields (a non-boolean changeTriggered reads as true; a
non-positive-integer timerIntervalMs is dropped), and deliberately does NOT warn about a
trigger-less tag or a sub-50ms timer — the editor refuses both, the runtime honours both, and
the deploy gate follows the runtime.
Joseph Doherty