Commit Graph

1259 Commits

Author SHA1 Message Date
Joseph Doherty 7891b3cb54 docs(reviews): record driver fixture + H1 gates (lmxopcua#761)
v2-ci / unit-tests (push) Has been cancelled
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 21:09:48 -05:00
Joseph Doherty 7cbeb9a7cc docs(reviews): record docker-dev Wave-4 gates A/B/C/E (lmxopcua#761)
Gates A (#731), B (#732), C (#733) and E (#734) run on the local docker-dev
rig against origin/master 1b9c36f3 on 2026-11-09: all four PASS. Records the
procedure, the measured thread/fd/RSS, subscription and deploy-status
numbers per gate, plus six non-blocking observations.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 21:09:02 -05:00
Joseph Doherty 1b9c36f38d docs(plans): OPC UA SDK 2.0 upgrade spike catalogue (lmxopcua#782)
v2-ci / unit-tests (push) Waiting to run
Catalogue of what breaks moving from 1.5.378.106 to 2.0.1: restore
break (MEL 10.0.12), 42 -> 103 compile errors in the first two waves,
root-cause groups with API mappings, PeerAddress replacement for the
reflection-based remote endpoint resolver, effort and risks.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 20:32:50 -05:00
Joseph Doherty c5df43d594 fix(historian): ContinuousHistorization CommitMode code default is PerEntry, matching appsettings/docker-dev/fallback (lmxopcua#776)
v2-ci / unit-tests (push) Has been cancelled
The Lane H2 remediation made Periodic(100 ms) the options-class default, but
appsettings.json, docker-dev and Program.cs's bad-value fallback all used
PerEntry, so effective behaviour never changed and the docs contradicted the
shipped config. The options default is now PerEntry; Periodic stays opt-in
(<= CommitIntervalMs of crash loss in exchange for throughput). Program.cs now
uses ContinuousHistorizationOptions.ResolveCommitMode() so the fallback shares
the single DefaultCommitMode constant. docs/Historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 20:27:06 -05:00
Joseph Doherty 7d989fb529 docs(security): script authoring and deploy are Administrator OR Designer (lmxopcua#779)
v2-ci / unit-tests (push) Has been cancelled
The docs said script authoring was Admin-gated (VirtualTags.md "Known
resource limits") and that a Designer "cannot publish" (security.md roles
table). The code admits Designer to both: Scripts, ScriptEdit, EquipmentPage
(virtual tags, scripted alarms), /api/script-analysis/* and Deployments all
carry AdminUiPolicies.ConfigEditor = RequireRole("Administrator",
"Designer"). That is a recorded decision, not drift: AdminUI-001 (98b27fc1)
loosened the analysis backend from FleetAdmin to Administrator,Designer to
match the page gate, and R2-05 converged the same pages on ConfigEditor
"with the same semantics". So the docs follow the code:

- VirtualTags.md: the primary control is ConfigEditor (Administrator or
  Designer), which also gates deploy.
- security.md roles table: Designer = the whole ConfigEditor surface incl.
  scripts and deploy; Administrator = Designer + the FleetAdmin-only surface
  (role grants, API keys, certificate actions, manual failover) — the old
  "publish, cluster/node CRUD" were ConfigEditor all along.
- 2026-06-09 Monaco plan: its FleetAdmin gate is marked superseded.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:44:35 -05:00
Joseph Doherty 9e8b5b0ea7 fix(scripting): stop unbounded script recursion from killing the server (lmxopcua#779)
Reproduced first, out of process (ScriptRecursionContainmentTests re-launches
the xunit v3 test assembly to run one recursive script through
ScriptEvaluator + ScriptRunGuard, so the runner never executes one): before
this change every shape — local function, mutual local functions,
self-referencing Func<>, anonymous self-application, recursion through LINQ,
an expression-bodied void lambda, an anonymous method — ended the child with
exit 134 (SIGABRT, "Stack overflow."). A StackOverflowException cannot be
caught, so in production one script takes the whole node down.

Fix: ScriptEvaluator.Compile now rewrites the validated wrapper
(ScriptStackGuardInjector, after ForbiddenTypeAnalyzer) so the script body and
every lambda, anonymous method and local function starts with
ScriptStackGuard.Check(), which throws ScriptStackExhaustedException when
RuntimeHelpers.TryEnsureSufficientExecutionStack reports the stack nearly
spent. The script then fails like any throwing script (Bad, script log) and
the process lives.

Why a runtime check, not an analyzer rule: recursion is not decidable from
syntax (anonymous self-application never names itself), and bounded recursion
must keep working. Expression-bodied lambdas/local functions become blocks
(`return e;` or `e;`, chosen from the bound symbol, so the same overload
binds); expression-tree and async lambdas are left alone. Limits (documented
on ScriptStackGuard and in docs/VirtualTags.md): depth depends on the thread's
stack size; a recursion that never re-enters script code is not seen; a script
may catch the exception, which only unwinds it.

Tests: ScriptRecursionContainmentTests (7 shapes, child process each; all 7
fail with exit 134 when the rewrite is disabled), ScriptStackGuardInjectionTests
(bounded recursion and every rewritten lambda/local-function shape keep their
value; the expression-tree case fails if those lambdas are rewritten).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:44:35 -05:00
Joseph Doherty a84d38707b fix(virtual-tags): evaluate scripts off the Akka dispatcher thread (lmxopcua#779)
VirtualTagActor called the blocking IVirtualTagEvaluator.Evaluate, and
RoslynVirtualTagEvaluator blocked on the run guard with
.GetAwaiter().GetResult(), so every script run parked a dispatcher thread for
up to its 2 s budget (a first-use Roslyn compile did too). This is the #750 P0
refactor for virtual tags.

- IVirtualTagEvaluator.EvaluateAsync (default: a completed task over the
  snapshot overload, so the null evaluator and fakes are unchanged).
- RoslynVirtualTagEvaluator implements it without blocking: passthrough and
  compile failures complete synchronously, a first-use compile goes to the
  pool, the run is awaited. The blocking overload now delegates to it.
- VirtualTagActor: a completed task is applied in the same turn, as before;
  an incomplete one switches to an "evaluating" behaviour that stashes every
  dependency/reassert message until a private completion message arrives,
  then applies the result and unstashes. Each trigger is still evaluated
  against its own inputs, results are applied/published in trigger order and
  the dedup/Bad-transition state sees the same sequence. The dependency map is
  not touched while a run reads it.

Calculation and scripted-alarm serialisation is NOT relaxed: neither blocks a
thread, and running independent scripts in parallel is not semantics-neutral
(pool consumption bound, cross-tag publish order, timer/pump exclusion for
Calc; state-machine/persist boundary, transition timestamps and the
store-failure path for alarms). Recorded in docs/VirtualTags.md and the #750
design's P0 entry.

Tests: VirtualTagActorAsyncEvaluationTests — on a one-thread dispatcher a slow
script no longer starves a sibling actor and the slow actor answers Identify
(fails when the actor calls the blocking Evaluate); triggers during a run are
evaluated afterwards in order against their own inputs; a reassert during a
run is applied after it; a faulted evaluation takes the evaluator-threw path
(the ordering/reassert tests fail with the stash removed).
RoslynVirtualTagEvaluatorAsyncTests — a slow script and a cold compile return
an incomplete task (fail without the override); cheap outcomes and timeouts
match the blocking overload.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:44:35 -05:00
Joseph Doherty d683b83156 fix(scripting): give scripted-alarm predicates the configured run budget, not 250 ms (lmxopcua#779)
DriverHostActor built ScriptedAlarmEngine without a scriptTimeout, so every
production alarm predicate fell back to TimedScriptEvaluator.DefaultTimeout
(250 ms) while virtual tags and Calculation tags ran on 2 s. History shows no
decision behind it: 250 ms is the generic Phase-7 A.4 default, the VT
evaluator moved to 2 s in F8b (219d10a2) and the alarm engine was simply never
given a value.

- New node-wide Scripting:RunTimeout (default 2 s, clamped to (0, 60 s],
  non-positive -> default, each clamp a startup Warning) on ScriptingOptions.
- The VT evaluator and the alarm engine default to it; DriverHostActor now
  builds the engine through CreateScriptedAlarmEngine, which passes it
  explicitly. Calculation keeps its per-driver runTimeoutMs.
- Docs: ScriptEditor (config table), ScriptedAlarms, VirtualTags.

Tests: ScriptedAlarmEngineBudgetTests (a 600 ms predicate now completes and
activates; default and configured budgets), ScriptedAlarmEngineBudgetWiringTests
(the host-built engine), ScriptingOptionsBindingTests (bind + clamps). The
engine and wiring tests fail with the old 250 ms default restored.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:44:35 -05:00
Joseph Doherty 2331f7ed14 feat(sql): opt-in writes — allowWrites + per-tag writable, single-row parameterised UPDATE (lmxopcua#753)
v2-ci / unit-tests (push) Waiting to run
allowWrites is no longer inert: SqlDriver implements IWritable, default off.

- Gates (outermost first): node Writable + WriteOperate (server), driver
  allowWrites, the tag's own TagConfig "writable": true (strict JSON boolean;
  anything else refuses the tag, and the deploy gate says why), and a model that
  names exactly one row — KeyValue, or WideRow by where-pair. topByTimestamp and
  Query are BadNotWritable.
- SqlWritePlanner: the only statement a write can become is
  UPDATE <table> SET <value col> = @v WHERE <key col> = @k, built from the
  catalog-gate-canonical identifiers (dialect-quoted) with value and key bound;
  value-before-key marker order so positional ODBC binds correctly. No authored
  write SQL, INSERT, UPSERT or procedure.
- Executed in a transaction that commits only when exactly one row was touched:
  0 rows -> BadNoData (never inserts), >1 -> rolled back, BadConfigurationError,
  unknown count -> rolled back, BadNotSupported. Bounded by operationTimeout;
  provider errors -> BadCommunicationError; health untouched.
- Discovery advertises Operate only where a write could succeed.
- AdminUI: driver form's Allow writes switch is live; tag editor gains a
  per-tag write switch; model validates writable strictly (parity corpus).
- Factory no longer warns "allowWrites ignored"; it logs whether writes are on.
- Tests: SqlWriteTests (planner SQL/bind order/injection/coercion, every gate,
  readback, absent key, two-row rollback, gate-rejected tag, discovery);
  MySqlReadTests write legs against InnoDB (keyed row only, two-row rollback,
  server-side grant refusal, default-off). Seed adds a setpoints table and the
  one UPDATE grant.

(lmxopcua#753)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:30:31 -05:00
Joseph Doherty 3c9ab4b668 feat(sql): Odbc provider — ANSI SQL:2008 profile over System.Data.Odbc (lmxopcua#753)
SqlProvider.Odbc is now constructed; every enum member is a working backend.

- OdbcDialect: System.Data.Odbc 10.0.14 (no vulnerable packages), ANSI double-
  quote quoting, positional `?` markers, INFORMATION_SCHEMA catalog (system
  schemas of the common backends excluded by name — the live gate caught
  psqlODBC listing pg_catalog), FETCH FIRST n ROWS ONLY limits, a conservative
  1000-parameter chunk. Positional binding is safe because every caller adds
  parameters in marker order (planner IN list, catalog schema-then-table).
- Named queries on a positional provider are checked by `?` count against the
  declared params (literals/comments ignored), since there is no name to match.
- Supported backends are those whose SQL accepts that profile (PostgreSQL via
  psqlODBC proven, Snowflake-like); SQL Server/MySQL/DB2 should use their native
  provider. The host needs the ODBC driver manager + vendor driver.
- Reserved-provider controls in factory/probe/browser tests now use an
  undeclared ordinal (99); the AdminUI form offers Odbc and a test pins the
  dropdown theory to SqlDialects.Supported.
- Tests: OdbcDialectTests (unit, any host). OdbcReadTests + Docker/odbc/
  run-odbc-gate.sh run the built suite in a throwaway Linux SDK container with
  unixODBC + psqlODBC against a throwaway seeded PostgreSQL: KV IN-list
  positional binding, where-pair, topByTimestamp via FETCH FIRST, a two-
  parameter named query, and the browse walk. macOS has no vendor ODBC driver,
  so there the suite skips.

(lmxopcua#753)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:30:31 -05:00
Joseph Doherty 63372e757a feat(sql): MySql provider over MySqlConnector, one shared provider map (lmxopcua#753)
SqlProvider.MySql is now a constructed backend instead of a reserved name.

- MySqlDialect (Driver.Sql.Dialect): MySqlConnector 2.6.2 (MIT, fully managed,
  no vulnerable packages), backtick quoting with MySQL's 64-char / no-trailing-
  space rules, information_schema catalog SQL with the database as the schema
  (system databases excluded, aliases pinned), trailing LIMIT, a type map that
  also reads the reader-side spellings (INT UNSIGNED -> UInt32, BOOL -> Boolean).
- SqlDialects: the one provider -> dialect map. The factory, the Test Connect
  probe and the browse picker each carried their own switch (and drifted once);
  they now all read this, and their refusals list the supported/reserved sets
  from it.
- AdminUI driver form offers MySql.
- Tests: MySqlDialectTests (golden quoting/catalog/type map/planner SQL); the
  provider-gate tests in factory, probe, browser and form now treat MySql as
  supported and Odbc as the reserved control. MySqlReadTests + Docker/mysql
  (throwaway mysql:8.4, SELECT-only login): liveness, KV/wide-row/topByTimestamp,
  INT keys vs authored text, case folding against case-sensitive table names,
  catalog rejection, browse walk, factory+probe, named query with a bound
  parameter, and a writing stored function whose effect the rolled-back
  transaction discards.

(lmxopcua#753)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:30:31 -05:00
Joseph Doherty a68593e714 feat(sql): named-query tag model — read-only SELECTs declared on the driver (lmxopcua#753)
SqlTagModel.Query (design §5.4) is now served end to end instead of refused.
The driver config declares `queries` (name -> sql, params, maxRows); a tag names
the query, the result `column` it reads and optionally a `rowKey` picking its row.

- Parser/deploy gate: query + column required, rowKey all-or-nothing; Inspect
  explains each refusal. Table is not read for this model.
- SqlReadOnlyStatement: single SELECT/WITH only, no data/schema-changing keyword
  outside literals/comments (refuses SELECT INTO, FOR UPDATE, DML CTEs). Shared by
  the factory and the AdminUI driver form (SqlNamedQueryRules.Describe), so the
  form refuses what the driver refuses. One bad entry fails the driver.
- Parameters are always bound; the factory refuses a declared parameter whose
  dialect marker the statement never uses.
- Reader: one execution per query per poll, inside a transaction that is always
  rolled back; result columns matched exact-then-unique-CI; no row -> BadNoData,
  ambiguous row / missing column / over maxRows -> BadConfigurationError.
- Catalog gate passes Query tags through (nothing is quoted into SQL); a
  Query-only driver skips the catalog load. A tag naming an undeclared query is
  skipped with the reason on /hosts.
- AdminUI: typed editor offers Query (query/column/rowKey); driver form edits
  `queries` as JSON and keeps the last valid set on a typo.

(lmxopcua#753)

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:30:31 -05:00
Joseph Doherty 9e4a26f888 docs(scripting): design an out-of-process, pre-emptible script runner (lmxopcua#750)
v2-ci / unit-tests (push) Has been cancelled
Design only. Documents why today's in-process ScriptRunGuard can abandon but
not stop a runaway script (file:line evidence), and proposes a sharded
worker pool with an isolation worker under a ScriptRunnerActor, a
named-pipe + protobuf frame protocol carrying full input snapshots,
soft/hard deadlines with process kill, kill attribution and quarantine,
worker sandboxing (scrubbed env, pinned assemblies, Job Object, heap
limit), an HMAC'd content-addressed PE cache, performance budgets,
failure-quality semantics, a flag-gated InProcess/Shadow/OutOfProcess
rollout, a test plan, open decisions and an assumptions register.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 17:04:12 -05:00
Joseph Doherty 0702b31029 fix(alarms): refuse a keyed native ack with no Galaxy identity synchronously, from a cached key set (lmxopcua#765)
v2-ci / unit-tests (push) Waiting to run
Item 3 of #765. A keyed (API-key) session whose key had no Galaxy ack identity
got Good and then a revert, because whether the key has an identity lives in
the Secrets store, which must not be read under the SDK lock.

GalaxyAckIdentityDirectory (Driver.Galaxy) keeps a node-wide set of the key ids
that have a galaxy-ack-user secret. It lists this node's Secrets store
(metadata only, nothing decrypted) and is read lock-free.
GalaxyDriver.GetAcknowledgeRefusal (IAlarmAcknowledgePreflight, published
through NativeAlarmAckPreflightView as in #760) now refuses a keyed session
whose key the directory reports Absent, when the driver has no fallback ack
account. A refusal also asks for a refresh.

Fails open: before the first refresh, once the snapshot is older than
StaleAfter, and from a signalled identity-secret change until the next
refresh, the lookup answers Unknown and the ack takes the old Good-then-revert
path.

Refresh: GalaxyAckIdentityRefreshService runs on driver nodes, every
Drivers:Galaxy:AckIdentityCache:RefreshIntervalSeconds (default 30;
StaleAfterSeconds 0 = three intervals; validated at start). It also refreshes
early on secret changes. The Secrets library has no change event, so
SecretsRegistration decorates ISecretCacheInvalidator, which is called by
AdminUI /api-keys, /admin/secrets and the replication appliers. API-key
changes need no event: a revoked key is already denied before the pre-check.

Tests: Galaxy.Tests GalaxyAckIdentityDirectoryTests (16) and Host.Tests
GalaxyAckIdentityCacheWiringTests (9). Seven separate mutations each turn at
least one test red: pre-check ignores the directory, factory drops it, no
invalidator decoration, no staleness check, no change-pending check, no
refresh on refusal, loop never wakes early.
Docs: Galaxy.md, AlarmTracking.md, Configuration.md, .claude/rules/alarms.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:38:57 -05:00
Joseph Doherty 380ec541c2 feat(cli): historyread --events reads event history over the wire (lmxopcua#776)
v2-ci / unit-tests (push) Has been cancelled
Item 1 of #776 (the B2 over-the-wire ask): Client.CLI had no way to issue a
HistoryReadEvents. `historyread --events` now sends ReadEventDetails to an
event notifier and prints one line per event in select-clause order.
--fields picks the select clauses (default EventId, EventType, Time,
SourceName, Severity, Message, ConditionName (on ConditionType), SourceNode);
--of-type adds an OfType where-clause. --events is exclusive with --modified /
--aggregate, and --fields / --of-type require it.

Client.Shared gains IOpcUaClientService/ISessionAdapter.HistoryReadEventsAsync
(default-implemented as NotSupported for other implementations) backed by the
real session adapter through HistoryReadPaging, so continuation points are
followed up to --max and a Bad node status is surfaced, not printed as empty.

Tests: CLI tests over the fake service (default filter, --fields/--of-type
filter, option conflicts); an over-the-wire test drives the shared client's
event read against the in-process server — the default fields come back
non-null, OfType filters, an invalid where-clause is BadEventFilterInvalid.
The other #776 items are decisions and are left open.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:17:24 -05:00
Joseph Doherty e790938f94 fix(historian): QuestDB modified reads keep a landed change whose marker was lost; H2 review NITs (lmxopcua#778)
QuestDB modified-history reads dropped every still-pending trail row that was
not the latest at its source time, so V0->V1 (applied marker never written)
followed by V1->V2 hid the V0->V1 change for good. A pending row below the
latest is now served when the NEXT trail row at that time recorded a different
prior value (kind, value, status or the prior's server_ts) - the next change saw
what this one wrote; an orphan of a change that never landed leaves the same
prior for the next row and stays hidden. QuestDB window functions reject
VARCHAR, so the next row's prior comes from a LEFT JOIN on (ts, lead(id)).
The updater's comment and marker-failure log no longer claim modified reads
"still serve it" unconditionally.

Regression tests (QuestDB 10.0.1 throwaway container): marker lost then a
replace / delete at the same time; an orphan followed by a marker-lost retry
then a later change (orphan still dropped); the random marker-mix fuzz now
varies the recorded priors. All four fail on the old filter.

NITs:
- SqlServerHistorianValueWriter: a SqlException raised while the token is
  cancelled (SqlClient's "Operation cancelled by user") is rethrown as
  OperationCanceledException instead of booked as a failed write. Integration
  test holds a TABLOCKX on Sample and cancels mid-INSERT; it returned false before.
- SqlServerHistorianDataUpdater.DeleteRawBatches is an Interlocked counter.
- HistoryReadEvents: an event filter whose SDK validation throws (unknown
  operator) failed with the generic BadUnexpectedError; the node manager now
  re-throws any failed event-filter validation as BadEventFilterInvalid.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 16:17:24 -05:00
Joseph Doherty 29665c1872 feat(opcua): HistoryReadEvents projects EventType, ConditionName, SourceNode; honours the where-clause (lmxopcua#746)
HistoryReadEvents projected only EventId / SourceName / Time / ReceiveTime /
Message / Severity: a select clause asking for EventType, ConditionName or
SourceNode (what most generic alarm viewers ask for) got nulls, and the
event filter's where-clause (an OfType) was ignored.

- HistoricalEvent gains optional EventTypeName, ConditionName and AlarmId.
  SqlServer, QuestDB and Gateway event reads fill them from what their
  alarm writers already store (AlarmTypeName / AlarmName / AlarmId; the
  gateway event Type and its AlarmName / AlarmId properties).
- The node manager maps the type name to its Part 9 event type (unknown or
  non-event => AlarmConditionType), projects ConditionName, and resolves
  SourceNode to the materialised condition the alarm id names (the live
  events' SourceNode), else null.
- The where-clause is evaluated per event with the SDK content filter
  through an IFilterTarget (OfType via the type tree; other operators
  against the projected fields), after the page's cursor is taken, so
  paging is unchanged.

Tested in the node manager, over a real client session (the Client.CLI has
no `historyread --events` mode yet; filed as a follow-up), and by alarm
writer -> event read round trips on SqlServer and QuestDB.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:47 -05:00
Joseph Doherty 096a3d5b3c docs(historian): backend capability matrix; warn on Gateway continuous historization (lmxopcua#746)
The Gateway backend's gaps against SqlServer/QuestDb were documented
piecemeal. The backend-selection table in docs/Historian.md now also covers
the aggregate set (and the native time-weighted opt-in), emulated reverse
and end-only reads, the client-side events cap, retention and the health
probe.

Continuous historization on Gateway silently loses sample quality (stored
Good), comms-loss outage markers and non-numeric values. Building its live
writer, which only happens when ContinuousHistorization is enabled, now
logs one startup Warning that names the gaps and the alternatives.

The review asked for the matrix in the AdminUI backend picker, but no such
picker exists: the backend is configuration-only. Filed as a follow-up.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:47 -05:00
Joseph Doherty a5557375ee perf(historian): recorder appends off its mailbox; Periodic commit default (lmxopcua#746)
The continuous-historization recorder awaited every outbox append (an fsync
under PerEntry) on its mailbox, so ingestion was capped at 1/fsync_latency
bursts per second while the mux kept telling into an unbounded, unmetered
mailbox. GetStatus also suspended the mailbox on an unbounded outbox count.

- Gated ingest bursts go to a bounded channel (64 bursts) drained by one
  ordered append loop, which commits every burst waiting behind a slow
  append in a single AppendBatchAsync (FIFO kept). Only a full queue holds
  the mailbox (backpressure, bounded memory). Counters are Interlocked; the
  loop nudges the drain. PostStop flushes queued bursts within the shared
  5 s stop budget.
- GetStatus reads counters on the actor thread and pipes a count bounded by
  a 10 s timeout (-1 when it does not answer); it no longer suspends the
  mailbox. RecorderStatus gains PendingAppends, MailboxDepth and
  AppendBackpressureWaits; two gauges publish pending appends and mailbox
  depth.
- ContinuousHistorization:CommitMode defaults to Periodic (100 ms). The
  Host's appsettings.json, its unparseable-value fallback and docker-dev
  still say PerEntry: filed as a Host follow-up (outside this lane).

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 1a7a8f6f22 fix(historian): QuestDB modification trail gets an applied marker and a dedup key (lmxopcua#746)
QuestDB has no multi-table transaction, so a HistoryUpdate writes its trail
row before the sample change. A failed or crashed change left an orphan
trail row that modified reads served as a change that happened, and a trail
INSERT re-sent after a client timeout duplicated (the table had no key).

- sample_modified gains `pending` and DEDUP UPSERT KEYS(ts, tag,
  modified_id). The trail row is written pending; after the change lands
  the same row is re-upserted pending = false (the applied marker).
- A transient trail INSERT failure is re-sent once with the same ids; a
  first attempt that did apply converges on the key.
- ReadModifiedAsync serves a pending row only when it is the latest trail
  row at its time and the sample there was written at or after the
  modification (a crash between change and marker). The orphan of a change
  that never landed, typically followed by the client's retry, is dropped.
  ASOF JOIN over the window's samples of one tag; the resume filter sits
  outside the window function so it never cuts a time's partition.
- The bootstrapper migrates an older trail (ADD COLUMN pending, DEDUP
  ENABLE) and waits for it to apply (bounded 10 s); old rows read
  pending = false and stay visible. Until applied, or with
  AutoCreateSchema=false, the old format is used.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 9c3c9b0d61 perf(historian): page SQL Server HistoryUpdate DeleteRaw (lmxopcua#746)
DeleteRaw deleted an unbounded range in one statement and transaction with
OUTPUT INTO the modification trail. Over years of a 1 Hz tag that is tens of
millions of rows deleted and audited at once: lock escalation on Sample
(blocking the live writer), transaction-log growth, and a 30 s
CommandTimeout that rolled the whole thing back.

It now runs DELETE TOP (4000) batches, each its own transaction with its
trail rows in the same statement (every deleted row is still audited),
cancellable between batches. A failure part-way leaves the committed batches
deleted and reports the failure; re-issuing completes it (documented, same
as QuestDB's paged tombstoning). The existing atomicity tests still hold per
batch.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty bc2f5e1889 perf(historian): index QuestDB event.source and sample_modified.tag (lmxopcua#746)
Event reads filtered on source and every modified-value read filtered on tag
scanned that symbol column in every partition of the window; only
sample.tag carried an index (#683).

Both columns are now SYMBOL INDEX in the DDL, and the bootstrapper's index
migration is generalised over QuestDbTables.SymbolIndexes (sample.tag,
event.source, sample_modified.tag): a missing index is added with the same
idempotent ALTER ... ADD INDEX, Warning-only when AutoCreateSchema is false
or the ALTER is refused. Integration tests: new tables carry all three; a
pre-index table set gains them on boot and event reads answer the same.

This reverses #683's documented choice to leave the two unindexed;
docs/Historian.md carries the reasoning.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty d44e58dcd6 perf(historian): SQL Server retention deletes only for tags with eligible rows (lmxopcua#746)
The SqlServer purge read every TagId and issued at least one DELETE TOP per
table per tag on every pass, even when nothing was eligible: 50000 tags made
100000 statements per hourly purge, serialised on one connection under the
applock.

One pre-filter query per table (EXISTS seek on the clustered key) now picks
the tags holding a row older than the cutoff; only those get deletes. A pass
with nothing eligible sends just the Event delete. The mid-command
cancellation test now accepts the pre-filter SELECT as the blocked command,
since it is the first statement to touch Sample.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 74afc3f90a perf(historian): batch alarm-history writes; pipeline gateway SendEvent (lmxopcua#746)
All three alarm-history writers inserted one event per round trip, serially
(SqlServer also opened a pooled connection per event), so a 10000-event
alarm storm drained at 10000 x RTT while the backlog sat in LocalDb.

- Sql.Common.AlarmBatchWrite: one statement per chunk; a chunk that fails
  PermanentFail-class is re-sent event by event so only the poison event
  dead-letters; a transient failure retries the chunk; a cancellation never
  dead-letters.
- SqlServer: multi-row insert-if-absent over a VALUES set (150 events per
  statement, under the 2100-parameter limit), duplicate ids in a batch
  written once; a duplicate-key race isolates per event, where it is Ack.
  No new table type, so no schema migration (the review suggested a TVP;
  the VALUES form gets the same one-round-trip-per-chunk without one).
- QuestDb: multi-row INSERTs of 500 events; pre-1970 events still
  dead-letter unsent.
- Gateway: no batch RPC exists, so sends are pipelined, at most 8 in
  flight, outcomes kept in input order.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty ee314d1b9e perf(historian): one QuestDB WAL wait per drain pass, with an exponential poll (lmxopcua#746)
The QuestDB live writer waited for WAL visibility after every tag group, and
the wait polled wal_tables() every 10 ms. With DrainConcurrency = 8 that was
up to eight pollers at ~100 queries/s each against a database whose WAL
apply was already behind.

New Runtime seam IHistorianPassWriter (WriteLiveValuesInPassAsync +
FlushAsync). The recorder writes every group of a pass through it and calls
FlushAsync once after all groups report, before the ack, whenever any group
wrote (an un-acked pass is redelivered and its insert-if-absent check must
see this pass's rows). A flush failure does not fail the pass. The QuestDB
writer implements it: in-pass writes only mark a flush as owed; FlushAsync
waits once. The poll now backs off 10 -> 20 -> ... -> 200 ms, never past the
timeout. Direct WriteLiveValuesAsync callers still wait per call.

Also updates the QuestDB cancelled-write integration test to the rethrow
contract from the previous commit.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 7c5cc9c7e5 fix(historian): live writers rethrow pass cancellation; gateway writes normalised (lmxopcua#746)
Every IHistorianValueWriter caught Exception and returned false, including
OperationCanceledException from the drain's own token. The recorder's guard
("a cancelled pass is not a tag failure") was therefore dead code: at shutdown
mid-pass each cancelled group could be booked as a consecutive failure and,
with a sibling that already wrote, tip a healthy tag toward quarantine.
SqlServer, QuestDb and Gateway writers now rethrow a cancellation of the
caller's token; a cancellation-shaped error on a live token is still an
ordinary failed write.

The Gateway writer relabelled a Local-kind timestamp as UTC instead of
converting it, so the same value could land at two different times on two
backends. Stamped samples now go through LiveWritePreparer (Local -> UTC,
non-finite doubles not sent, duplicate timestamps keep the first); unstamped
samples are still left for the gateway to stamp, in batch order.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty bc7e58ed64 perf(historian): exact tag lookup and a non-thrashing presence cache (lmxopcua#746)
Every Gateway HistoryRead was preceded by a BrowseTagNames catalog stream
keyed by a glob filter (a name with '*' or '?' streamed every match), and the
4096-entry presence cache evicted an arbitrary entry (Keys.First()) when full,
so a sweep over more tags thrashed it.

The presence check is now one exact unary GetTagMetadata (null = absent).
Only the deployed gateway's known misparse of a missing tag (Internal /
Unknown / InvalidDataException) falls back to the exact-name catalog check;
any other failure propagates and never becomes absence. The positive TTL is 5
minutes (was 60 s); a full cache drops expired entries first, else the one
closest to expiry. Wire tests cover the working, the misparsing and the
refusing gateway.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 8dc594d8e4 fix(historian): probe the gateway on a timer for the health snapshot (lmxopcua#746)
The Gateway historian's ProcessConnectionOpen / EventConnectionOpen were
derived from read outcomes only, so a freshly booted node with a healthy
gateway reported the historian closed until a client read, and a node that
only writes (continuous historization) never flipped it.

The data source now runs a ProbeAsync loop (ServerHistorian:Gateway:
ProbeInterval, default 30 s, first probe at start-up; 0 = off). Each probe,
bounded at 10 s, is recorded as the latest connectivity observation; the
latest observation (probe or read) is what the snapshot reports. Probes never
touch the query/failure counters. DisposeAsync stops the loop before
disposing the client.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 0b85c499e3 fix(historian): refuse native Gateway TimeAverage/Total until historiangw#27 (lmxopcua#746)
AVEVA's native TimeAverage / Total are answered with Good-status wrong values
while the gateway sends the interval in the wrong unit (historiangw#27): one
whole-span value on the earliest interval, the rest absent. Nothing on the
wire marks the answer as degraded.

Both are now refused BadAggregateNotSupported (unsent, not counted as a
backend failure, not advertised in AggregateFunctions) unless
ServerHistorian:Gateway:EnableNativeTimeWeighted is true. The default
Gateway set is the ten raw-evaluated aggregates
(GatewayHistorianDataSource.RawEvaluatedAggregates); ServedAggregates stays
the 12 the backend can serve with the flag on. The native-path tests opt in.

Also documents the ServerHistorian:Gateway sub-section (EventDrainBudget from
the previous commit) and the events drain in docs/Historian.md.

Rebased from the unpushed remed/lane-H2 branch (originally authored 2026-10-22
in session_01Ub9hGmYLQvn9XETzw7CMsT) onto current master.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 15:40:46 -05:00
Joseph Doherty 90db63cfaf fix(opcuaclient): don't count idle-session BadNoSubscription as a publish error (lmxopcua#764)
v2-ci / unit-tests (push) Has been cancelled
The SDK keeps sending publish requests on a session that holds no subscription,
and the server answers each with an anonymous (id 0) BadNoSubscription. With
nothing tracked there is nothing to lose, but every answer was counted on
publish_error, so the counter climbed on any idle driver. HandlePublishError now
returns before counting for an anonymous BadNoSubscription while neither the data
nor the alarm subscription set tracks anything. Any other PublishError, and an
anonymous one while something is tracked, is counted and handled as before.

Not addressed from #764: the Republish/state probe for a delete slower than the
5 s grace, and the #728 wire test's fixed delays.

Tests: OpcUaClientIdlePublishErrorTests (idle session: not counted, with a
BadTooManyPublishRequests positive control; tracked subscription: still counted).
The idle test fails without the fix. Driver.OpcUaClient.Tests 295/0.
Docs: drivers/OpcUaClient.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:03:59 -05:00
Joseph Doherty d59e76dd9d feat(security): per-client pre-auth API-key miss budget; widen IOpcUaUserAuthenticator with the remote endpoint (lmxopcua#751)
The #733 miss budget (MissBurst / MissRefillPerSecond) was process-wide, because
the impersonation seam did not carry the client's address, so one noisy client
could drain it for everybody while the known-id set was unavailable.

- IOpcUaUserAuthenticator gains AuthenticateUserNameAsync(username, password,
  IPEndPoint? remoteEndpoint, ct) (default: ignore it and delegate); the server
  calls it. IApiKeyStore gains the matching FindAsync(..., IPAddress?, ...).
- The SDK (1.5.378) exposes no client address publicly. SecureChannelRemoteEndpoint
  reads it from the TCP listener's private channel map by reflection (through the
  protected TransportListeners, exposed by OtOpcUaSdkServer), keyed by the
  session's secure channel id. Fail-soft: null => process-wide accounting only.
- BoundedApiKeyStore charges a miss to the client's own bucket (keyed by IP
  address) and then to the process-wide one, which stays as the backstop. A
  refusal by either takes from neither; a hit refunds both. New options
  Security:ApiKeys:MissBurstPerEndpoint (10) / MissRefillPerSecondPerEndpoint (2),
  0 disables; telemetry outcome endpoint_miss_budget. At most 4096 addresses are
  tracked; idle buckets are pruned and an address beyond the cap is charged
  process-wide only.

Tests: BoundedApiKeyStorePerEndpointTests (noisy client vs quiet one, process-wide
backstop, refund + refill, off switch, authenticator forwards the address) and
over the wire ApiKeyMissBudgetPerEndpointTests (activation hands the authenticator
the loopback endpoint; real authenticator + bounded store refuse a client's 3rd
and 4th unknown key without a store read). Each new behaviour test fails with its
fix mutated out. Host.Tests 224/0, OpcUaServer.Tests 1019/1 (the one failure is the
wall-clock NodeManagerLockTimeBenchmarkTests under suite load; passes alone).
Docs: security.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:03:59 -05:00
Joseph Doherty f54922f94c fix(alarms): revert a native ack that reaches no driver; refuse native acks while the host is Stale (lmxopcua#765)
Residuals of #760:

1. An ack routed to a disabled / not-yet-spawned driver, or to a stubbed child
   (driver does not run on this node, or its factory rejected the config), was
   dropped and left the condition Acknowledged. DriverHostActor now reverts it
   through the same NativeAlarmProjector.TryRevertAcknowledge path as a refused
   ack.

2. A native ack arriving while the host is Stale was dropped before the projector
   recorded it, so it could never be reverted. Stale now marks the whole
   NativeAlarmAckPreflightView unavailable (SetHostUnavailable), so the node
   manager refuses every native ack synchronously; BecomeSteady clears it.

3. (keyed sessions without a Galaxy identity) is not addressed.

Tests: DriverHostActorNativeAlarmTests' client-ack test and the FOCAS respawn
theory now start the host's own driver (AckingAlarmDriver; a factory building a
real FocasDriver per spawn over per-instance fake CNCs, with a real config-change
respawn) - both fail against the fix in their old shape. New: ack to a disabled
driver reverts, ack to a stubbed driver reverts, a Stale host refuses through the
pre-check view - all three fail with the fix mutated out. Runtime.Tests 943/0/3.
Docs: AlarmTracking.md, .claude/rules/alarms.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 14:03:59 -05:00
Joseph Doherty 7f29a75edb feat(mqtt): read Sparkplug B array metrics as 1-D OPC UA arrays (lmxopcua#754)
v2-ci / unit-tests (push) Has been cancelled
R06:M-5 made Sparkplug *Array leaves unbindable because the ingestor refused
them. The ingestor now decodes the packed bytes_value of all 13 array types
(new SparkplugArrayCodec: LE fixed widths, int64-ms DateTime, uint32-count +
MSB-first BooleanArray, NUL-terminated UTF-8 StringArray; golden vectors are
the spec's examples, reproduced byte-for-byte by SparkplugNet 1.3.10) and
publishes a typed 1-D array through the scalar coercion, all-or-nothing.

Shape uses the fleet's one array model: the tag's platform isArray/arrayLength
keys (ValueRank=1, ArrayDimensions=[arrayLength] or [0] = variable length).
The driver reads them with TagConfigIntent's exact semantics and never rejects
on them, so composer, driver, authoring model and deploy inspector agree
(MqttArrayShapeParityTests). Shape mismatch (array metric on a scalar tag, or
the reverse) is BadTypeMismatch - a String tag no longer gets the packed
buffer as base64; a malformed body or a length differing from a declared
arrayLength is BadDecodingError (MTConnect #627 verdict). Template/DataSet
members of an array type take the same path. Arrays stay read-only (encoder
refuses them; MQTT is not in DriverArrayWriteSupport).

Browser: array leaves are bindable again and state isArray in AddressFields;
the commit mapper writes "isArray": true (no arrayLength). Discovery reports
IsArray/ArrayDim. Docs: Mqtt.md "Array metrics", design table, drivers rule.

Tests: codec (golden + malformed), ingest through the real codec + Dispatch
(the project's MiniBroker never forwards a PUBLISH, so there is no in-process
broker harness), browse, discovery, commit mapper, parity corpus. Mutating the
fix back out fails 18 MQTT + 9 AdminUI tests.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:58:38 -05:00
Joseph Doherty 5db01053e8 docs(redundancy): stop claiming RedundancySupport/ServerUriArray are exposed; record why #717 is blocked (lmxopcua#717)
v2-ci / unit-tests (push) Waiting to run
Bounded, docs-only part of #717. The standard non-transparent metadata itself is
NOT implemented here: the owner-selected contract is Warm with an explicit
degraded standby (and None standalone), today's Secondary serves Good data at
ServiceLevel 240, and the design/spec/plan for that standby behaviour is still
pending on the issue - advertising Warm now would misdescribe the server.

- docs/Redundancy.md: the "deferred pending an SDK object-type upgrade" claim
  was wrong; the pinned SDK 1.5.378.106 can build the graph
  (CreateMainNodeManagerFactory + AddBehaviourToPredefinedNode). Records what is
  and is not advertised (i=3709 SDK default, no i=11314) and the blocker.
- .claude/rules/cluster-topology.md: said each node exposes RedundancySupport
  and ServerUriArray; it does not.
- docs/OpcUaServer.md: same false claim about the SDK exposing them.

Interop gates A4/A5b stay unmet; no product code changed.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:12:17 -05:00
Joseph Doherty 8e8d4cdf13 fix(alarms): surface a refused native alarm ack synchronously, revert one that fails later (lmxopcua#760)
The native-ack path is fire-and-forget after the Part 9 Acknowledge returns, so
a Galaxy ack refused for lack of a credential (#749), or rejected by MXAccess,
left the client with Good and the condition showing Acknowledged while Galaxy
was never acked. Both options from the issue, layered:

1. Synchronous refusal for what the driver can decide without I/O. New optional
   IAlarmAcknowledgePreflight (Core.Abstractions); GalaxyDriver refuses an
   anonymous session when the driver has no AlarmAck fallback account.
   DriverHostActor publishes a condition NodeId -> owning-driver snapshot into
   NativeAlarmAckPreflightView (lock-free, volatile); the node manager's new
   NativeAlarmAckPreflight seam, consulted after the authz and Primary gates,
   returns BadUserAccessDenied with the reason - nothing routed, applied or
   reported. A keyed session is not pre-checked (its identity lives in the
   Secrets store, which must not be read under the SDK lock).

2. Revert for everything else. RouteAlarmAck carries the condition NodeId;
   DriverInstanceActor replies AlarmAckFailed on a refusal, a backend failure,
   a driver without an alarm source, or an ack dropped while connecting /
   reconnecting. DriverHostActor re-projects the condition unacknowledged via
   NativeAlarmProjector.TryRevertAcknowledge, but only while the local ack is
   still pending - any driver transition projected since then wins. A disabled
   or unspawned driver is not reverted (unchanged).

GatewayGalaxyAlarmAcknowledger now throws DriverOperationRefusedException on a
non-zero native hresult (wrong password, no ack permission) instead of only
warning, so that case reverts too; as a refusal it neither retries nor trips
the breaker.

Tests (each fails with its fix mutated out): projector revert/supersede,
DriverHostActor refused + failed ack -> unacked AlarmStateUpdate (successful
ack control), preflight view snapshot, node-manager preflight unit test,
over-the-wire ConditionMethodEventTests (BadUserAccessDenied, no route, no
event; positive control), Galaxy preflight + hresult refusal.
Docs: Galaxy.md, AlarmTracking.md, ScriptedAlarms.md, .claude/rules/alarms.md.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 13:12:17 -05:00
Joseph Doherty 0217d36cf4 fix(opcuaclient): stale BadNoSubscription after a resubscribe is not a subscription loss (lmxopcua#763, lmxopcua#728)
v2-ci / unit-tests (push) Has been cancelled
HandlePublishError treated any anonymous BadNoSubscription (subscription id 0)
as loss of a tracked subscription whenever anything was tracked. A publish the
SDK queued while the session briefly held no subscription (the driver had just
deleted its last one) is answered BadNoSubscription and can be handled after the
replacement is registered, so a routine unsubscribe->resubscribe declared the new
subscription lost: Degraded health, a BadNoCommunication quality-only annotation
on every alarm condition and ConnectionLost (host re-initialise). That annotation
is the extra Unspecified event the #728 ConditionRefresh wire test saw.

- A BadNoSubscription that names a subscription id now counts only if the id is
  tracked (same rule as BadSubscriptionIdInvalid).
- An anonymous BadNoSubscription within StaleNoSubscriptionGrace (5 s) of a
  delete that left the driver tracking nothing is ignored (Debug log). The window
  is stamped on removal and restarted when the delete completes; failed-subscribe
  removes stamp it too; it resets with the session. A delete that leaves another
  subscription tracked opens no window. A genuine loss inside the window is still
  caught by the PublishingStopped heuristic; only the immediate ConnectionLost is
  forgone.
- OpcUaClientStaleNoSubscriptionTests: stale error after alarm and data
  resubscribe (fail without the fix), id-attributed stale error, plus genuine-loss
  controls (after the grace, no recent delete, non-last delete, session release),
  each mutation-checked.
- The ConditionRefresh wire test now records Kind, AlarmType, message, severity,
  quality, restatement and timestamp of every post-resubscribe event on failure.

Wire test locally: 1/10 passing before (all 9 failures the 0x80310000
ConditionType quality-only annotation), 12/12 after.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:47:58 -05:00
Joseph Doherty e31888c9fd docs(redundancy): rewrite phase 6.3 interop runbook for the current model and correct standard NodeIds (lmxopcua#716)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:30:03 -05:00
Joseph Doherty 8ddf737a2a docs: reconcile stale deferrals in ScriptEditor.md and Uns.md (lmxopcua#759)
Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 12:27:52 -05:00
Joseph Doherty acfb4fbaca fix(redundancy): a failed OPC UA endpoint drops ServiceLevel to 0 and steps the Primary down (lmxopcua#762)
v2-ci / unit-tests (push) Has been cancelled
IOpcUaEndpointState (review-remediation lane C3) recorded whether the SDK started, but only the
health check read it: a fused node whose endpoint failed (port 4840 taken, ...) still advertised
250/240 and stayed driver Primary, so the Primary-gated data plane ran on a node no client could
reach while the reachable peer refused writes as Secondary.

- Move IOpcUaEndpointState/OpcUaEndpointState/OpcUaEndpointStatus from Host to Commons.OpcUa so
  the Runtime can consume it.
- ServiceLevelCalculator: new NodeHealthInputs.EndpointListening (default true). false -> 0,
  ahead of every other input including the +10 Primary bonus.
- OpcUaPublishActor: takes the endpoint state, feeds EndpointListening (Failed only; NotStarted /
  Stopped are boot/shutdown windows) on the DB-backed and DB-less paths, and the legacy role-only
  seam returns 0 for a failed endpoint too.
- New OpcUaEndpointStepDownActor (one per driver node): a Primary (ClusterActiveNode.SelfIsActive,
  the election's own primitive) with a Failed endpoint gracefully Leaves its mesh - the manual
  failover mechanism - so the peer becomes the oldest Up driver member, hosts the singletons and is
  Primary. The election itself is not bent. Guarded on a peer having recently reported its own
  endpoint listening (EndpointReport on the new opcua-endpoint-state DPS topic), so a pair with
  both endpoints failed does not bounce the role and restart nodes forever, and a lone node never
  shuts itself down.
- Runtime now references ZB.MOM.WW.Health.Akka (ClusterActiveNode).
- Docs: Redundancy.md tier table + new "Endpoint start failure: Primary step-down" section;
  cluster-topology rule; HealthEndpoints / OpcUaEndpointHealthCheck comments (F-C1 landed).

Tests: ServiceLevelCalculatorTests (+5), OpcUaPublishActorEndpointStateTests (7: DB-less,
DB-backed, legacy seam, NotStarted/Stopped do not demote, a later failure demotes on the next
tick), OpcUaEndpointStepDownActorTests (3, real two-node mesh: holds while the peer is not
listening then leaves and the peer becomes Primary; failed Secondary and healthy Primary never
leave). Each was mutation-checked: removing the calculator/legacy guard, the Leave, the
peer-listening guard or the Primary guard fails the corresponding tests.
2026-11-09 12:16:37 -05:00
Joseph Doherty dc0a24f224 feat(galaxy): operator-authenticated alarm ack — send Galaxy credential to mxaccessgw (#749)
v2-ci / unit-tests (push) Failing after 1h8m43s
mxaccessgw 488649c+ acknowledges by AuthenticateUser as the operator and
writing <alarm>.AckMsg, so AcknowledgeAlarmRequest now needs a real Galaxy
operator_user + operator_domain + operator_password. The OPC UA data plane is
API-key/anonymous and carries no Galaxy password, so the driver resolves the
account per ack:

1. the session's API key's Galaxy ack identity, set on AdminUI /apikeys and
   held ONLY in the encrypted Secrets store
   (otopcua/apikeys/<keyId>/galaxy-ack-{user,password}); a key with a user but
   no password secret is refused, never silently downgraded;
2. else the driver instance's AlarmAck fallback account (DriverConfig
   FallbackUser/FallbackDomain/FallbackPasswordSecretRef; secret:/${secret:}/
   env:/file:/dev: only — a literal is refused);
3. else DriverOperationRefusedException: nothing is sent, never an empty
   operator_user. Refusals are excluded from the resilience retry/breaker.

- Pins ZB.MOM.WW.MxGateway.Client/Contracts 0.2.1 -> 0.3.0 (vendored into
  nuget-packages/ from the Gitea feed, per the local-mxgw source mapping).
- The session ApiKeyId (not a secret) rides NativeAlarmAck ->
  RouteNativeAlarmAck -> RouteAlarmAck -> AlarmAcknowledgeRequest; the password
  is resolved in the driver and never travels through actor messages.
- The OPC UA principal + key id are appended to the AckMsg comment
  ("[OPC UA: <principal>, key <id>]") and logged, so a shared fallback
  account does not lose who acknowledged.
- Password never logged: GalaxyAckCredential.ToString omits it, the
  acknowledger scrubs it from every gateway message it echoes and withholds
  any exception that echoes it; audit rows carry only the key id.
- AdminUI: /apikeys "Ack identity" editor (set/remove; delete-key clears it),
  Galaxy driver form gains the fallback section.
- Synchronous OPC UA status for a refusal is follow-up #760 (the native ack
  path is fire-and-forget after the Part 9 method returns).

Tests: request mapping (password populated), password absent from logs and
exception text (mutation-checked), key-mapped / fallback / refused paths at
resolver and driver level, breaker non-trip, key-id propagation through node
manager and DriverHostActor, AdminUI service + form round-trip.

Closes #749
2026-11-06 16:23:00 -05:00
Joseph Doherty 7ec5b95d24 docs: point markdown backlogs at Gitea issues (status lives in Gitea)
v2-ci / unit-tests (push) Waiting to run
2026-11-06 15:31:43 -05:00
Joseph Doherty cf62007678 fix(arrays): Modbus row honours the FC16/FC03 protocol ceilings, the RMW read cap and the address-space end
Review r0 of Integration-W3:
- I1: the effective FC16 cap is min(configured maxRegistersPerWrite, 123). The
  driver trusts a larger configured cap and sends an over-ceiling FC16 (past 127
  registers its byte-count byte wraps), which every conformant device refuses, so
  such an array is never advertised writable.
- I2: a BitInRegister[] write reads its span first in one unchunked FC03, so it is
  also bounded by min(configured maxRegistersPerRead, 125) (absent/0 = 125).
- m2: every span (registers and coils) must end at or before address 65535.
- m1: the String-node wording says it converts only when the text parses, which
  the server cannot promise (DateTime never converts).
- m4: the parity device now enforces FC01/02 2000, FC03/04 125 (or a smaller
  device read limit), FC15 1968, FC16 123, FC23 121/125, the FC15/FC16 byte-count
  field (exception 03) and the address-space end (exception 02).

Tests: parity rows for each ceiling/cap/span boundary against the real write
path, and Core.Abstractions row tests. The configured-cap test that asserted a
cap of 200 lifts the ceiling now asserts the opposite.

Refs #741

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 17:32:57 -04:00
Joseph Doherty 8699614d2f Merge commit '82380d2b' into remed/integration-w3 2026-10-22 17:21:38 -04:00
Joseph Doherty 82380d2bee Merge lane G1: OpcUaClient + MTConnect driver remediation (#743)
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 17:16:20 -04:00
Joseph Doherty 5533de6281 feat(arrays): Modbus String[]/BitInRegister[] array writes reachable; table follows the resolver
The Modbus lane (#741 U3) gave the driver String[] and BitInRegister[] holding-register array encoders, and
moved address resolution to ModbusTagAddressResolution (keys first, a bit region coerced to Bool, strict key
reads), but DriverArrayWriteSupport still listed both array shapes read-only (follow-up F1), so a client
write was BadNotWritable before it reached the driver.

Modbus row now mirrors the driver end to end:
- Keys shape only (region + address both present). A string-only addressString tag is resolved by the
  driver with its own family through a parser Core.Abstractions cannot reference, so it stays read-only
  (pinned: the conservative direction).
- Strict reads as the resolver: address an integer 0-65535; a present region / dataType / byteOrder /
  stringByteOrder string must name a member.
- Coils: any dataType (coerced to Bool), <= 1968 coils (FC15).
- HoldingRegisters (FC16, never Bool): numeric, String ((stringLength+1)/2 registers, stringLength >= 1),
  BitInRegister (1 register, bitIndex 0-15, span ending <= 65535), all within maxRegistersPerWrite.
  FC23 is opt-in and never changes the answer.
- Elements convert (ScadaBridge int[] -> Int16 stays writable); only a String / DateTime node over a
  numeric or bit element is left out, since such an element never converts.

Tests: the parity test's PendingF1 exemption and its guard are gone; the parity matrix is now region x
element type x node type against the real WriteAsync with a spec-conformant device (FC15 > 1968 refused),
plus shape edges at every cap boundary (numeric, String, BitInRegister, coil), strict key reads, FC23
on/off, string-only, the ScadaBridge int[] shape and U3 wire bytes. Core.Abstractions row tests and a
composer test (String[] and BitInRegister[] writable; over-cap, no-stringLength and string-only read-only).
Docs: table XML doc, Uns.md, modbus-addressing.md, drivers/Modbus.md, data-plane rules.

Refs #741

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 17:10:00 -04:00
Joseph Doherty 419ecb3ce9 fix(mtconnect): a delivered /sample chunk clears an unanswered-write degradation (review r2 N-5)
Coordinator ruling: a delivered chunk proves the Agent is alive (the analogue of
OpcUaClient's good keep-alive), so TryMarkStreamDelivered - generation-checked
under _subscriptionLock - also clears _writeDegraded. The degradation no longer
sticks forever on a subscription-only driver. Test: unanswered write -> Degraded,
stays Degraded with nothing else happening (positive control), the next chunk ->
Healthy; red with the clear removed. Round-2 doc text corrected.

Refs #743

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 17:08:44 -04:00
Joseph Doherty bc1f341e8a fix(mtconnect): Unsupported latch inside the checked section; live-index reads do not clear a write degradation (review r1 N-4, N-5)
- N-4: the /sample Unsupported latch is set inside TryDegradeStream's
  generation-checked critical section (latchUnsupported), so a superseded pump can
  no longer latch it on the session that replaced it.
- N-5: a read served from the live pump index clears only the read degradation -
  it is stream evidence, like a chunk, which by the I-3 ruling does not clear an
  unanswered write. The 'cannot stick' wording in the XML doc and
  docs/drivers/MTConnect.md now states the latch-until-next-answered-/current-read-
  or-write behaviour, incl. the subscription-only consequence. Tested (chunk +
  live-index read keep Degraded; an answered write clears it).

Refs #743

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 16:58:37 -04:00
Joseph Doherty c39bcd6e24 fix(opcuaclient): unresolvable acks warn instead of tripping the ack breaker; shutdown holds subscription management (review r1 N-1, N-2, N-3)
- N-1 (Important): m3's BadNodeIdUnknown throw for an ack with no resolvable
  condition counted toward the host's AlarmAcknowledge breaker (threshold 5), so
  re-acks of one unresolvable condition blocked every genuine ack. It now logs a
  Warning and returns; only a no-reply transport failure throws. Test through the
  real CapabilityInvoker: six unresolvable acks, then a genuine ack that is still
  delivered (red with the throw restored).
- N-2: ShutdownAsync holds _subscriptionGate for the whole teardown (in-flight
  subscribes finish first, bounded by the budget), so a subscribe arriving during
  the delete phase finds the session released instead of creating a subscription
  on the closing session. Red-tested.
- N-3: the shutdown-side cert-validation detach is generation-checked under
  _probeLock with the map / endpoint clear.

Refs #743

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 16:58:37 -04:00
Joseph Doherty 92dea29547 Merge lane Modbus: Modbus driver remediation (#741)
Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 16:53:49 -04:00
Joseph Doherty 863a00c25d Merge master (75b7effb) into remed/lane-G1
Refs #743

Claude-Session: https://claude.ai/code/session_01Ub9hGmYLQvn9XETzw7CMsT
2026-10-22 16:52:43 -04:00