Commit Graph

1458 Commits

Author SHA1 Message Date
Joseph Doherty e9064dbdf4 Merge fix/815-credential-ref-scoping
# Conflicts:
#	.claude/rules/drivers.md
#	docs/drivers/Heidenhain.md
2026-11-11 02:08:24 -05:00
Joseph Doherty 4563940792 fix(security): warn-only compatibility for deployed drivers + upgrade preflight (#815)
User decision: save, Test Connect, live browse and the deploy gate keep
refusing out-of-namespace credential references, but an ALREADY-DEPLOYED
driver must not fault at the upgrade restart or (Galaxy) at the next
gateway reconnect, since ReopenAsync re-resolves Gateway.ApiKeySecretRef.

- LegacyCredentialReferenceTracker (Core.Abstractions): the one, deprecated,
  call-site-chosen compatibility path. Running Galaxy (InitializeAsync,
  ReopenAsync, ack fallback) and OpcUaClient (session open) pass it and keep
  the pre-#815 semantics (case-insensitive prefixes, Galaxy file:, OpcUaClient
  literal pass-through) with a rate-limited Warning (driver/field/allowed
  form, never the value) and a credential-compat:<field> diagnostics entry
  (both drivers now implement IDriverDiagnostics). Browsers and probes pass
  none and stay hard-fail; Heidenhain has none.
- Deploy-gate message names the migration.
- Upgrade preflight: CredentialReferencePreflightService (admin nodes, once
  after ApplicationStarted, ConfigDb only, resolves nothing) logs a Warning
  per Draft row / Deployed-artifact finding; the /raw banner shows them plus
  Running drivers reporting warn-only use (driver-health diagnostics).
- docs/security.md: exact replacement table (incl. the old form default
  env:MX_API_KEY, file:, un-namespaced secret:, casing changes), the
  ReopenAsync reconnect behaviour, the warn-only window and its removal.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 02:07:33 -05:00
Joseph Doherty d3285af6b3 Merge feat/830-heidenhain-followups 2026-11-11 01:55:10 -05:00
Joseph Doherty 9aff90f884 fix(opcuaclient): never pass checkDomain for a None endpoint; document both SDK check sites (#816 review)
SDK 2.0.1 checks the server host name in TWO places: Session.OpenAsync (after
the chain is trusted) and DefaultSessionFactory.CreateChannelAsync, against the
certificate in the GetEndpoints reply and before any security-mode test. opc-plc
sends a certificate on its None endpoint, so `connect -u
opc.tcp://10.100.0.35:50000 --validate-host-name` was refused on an
unauthenticated channel, where a host check proves nothing.

All three call sites now pass checkDomain only when the selected endpoint's
SecurityMode != None: OpcUaEndpointSelection.CheckDomainFor for the driver
(OpenSessionOnEndpointAsync, so the failover sweep too) and the AdminUI browser;
the same expression in Client.Shared ConnectToEndpointAsync (shared by
failover). New live tests: driver None + flag on vs :50000 reads; browser None +
flag on vs the NAT fixture browses; the secure + flag refusals still hold.

Docs: name both check sites; a zone-qualified IPv6 host never matches (FindDomain
compares the zone too), so link-local needs validateServerHostName=false; the
match is loose (first label alone, subject DC= parts count, no wildcards), which
caveats the CA recommendation.
2026-11-11 01:53:01 -05:00
Joseph Doherty e6cba4bc18 fix(adminui): Heidenhain authoring UX — CSV preview runs the save-path row rule, dictated data type is locked, credential error shown once (#830)
- RawTagRowRules: the ONE driver-dispatched authority on Tag-ROW rules (access level /
  Tag.DataType). RawTreeService's save path, the CSV import's staged review grid
  (RawTagCsvMapper) and the manual-entry grid all call it; no call site names a driver.
  A ReadWrite or mistyped Heidenhain row is now invalid IN THE PREVIEW instead of only at Commit.
- Manual-entry grid + tag modal: when the driver dictates the data type for the row's
  address (RawTagRowDefaults.DictatedDataType), the field is a locked read-only value instead
  of a select that commit silently replaced; the grid runs the row rule client-side and no
  longer rewrites the row at commit.
- Driver/device modals: the credential refusal is shown once (the modal's save-gate notice);
  HeidenhainDriverFormModel / HeidenhainDeviceModel.Validate report only ranges / endpoint.
- docs/drivers/Heidenhain.md: #831 alarm conditions + rediscovery, #830 authoring behaviour.

Tests: HeidenhainAuthoringUxTests (CSV preview == ImportTagsAsync row-for-row, dictated type
is what the save path accepts, single credential surface, ALARM: seeding, CSV alarm rows,
AlarmMs round-trip); HeidenhainAuthoringTests updated for the single-notice rule.
2026-11-11 01:37:51 -05:00
Joseph Doherty 2b0eed1a8f feat(heidenhain): controller alarms as native Part 9 conditions; rediscovery on identity change (#831)
- ALARM: selectors (HeidenhainAlarmSelector, Contracts): ALARM:*, ALARM:OTHER,
  ALARM:CLASS=<r>[/GROUP=<r>][/NUMBER=<r>] (<r> = n | lo..hi; negative numbers allowed).
  An ALARM: address and the cross-driver alarm object only work together — the factory
  refuses either alone, so editor, save path, CSV and deploy inspector agree. Data type
  fixed Boolean; never a value (ReadAsync BadNotSupported), never in a data subscription.
- HeidenhainDriver implements IAlarmSource: one condition per ALARM: tag at its RawPath,
  driven by the SAME allowlisted R_RI 27/28 walk on the control's single loop, on the new
  Polling.AlarmMs cadence (default 2000 ms, only for controls carrying an ALARM: tag).
  FOCAS #647 transition rules (Raise / Retrigger / Clear), identity = class+group+number+text,
  deterministic display order, told-record on the driver instance, recorded only while a
  handler listens. A failed walk (refusal, overflow, comms loss) is quality-only, never a clear.
- Acknowledge is LOCAL-ONLY: AcknowledgeAsync is a no-op success (FOCAS/MTConnect posture);
  Lsv2ReadOnlyPolicy is unchanged — no ack or write request exists.
- Opt-in per tag, no driver flag (FOCAS #647 / MTConnect #626 precedent); Alarms/* arrays unchanged.
- IRediscoverable: a reconnect whose R_VR identity (model / NC / PLC / options) differs from
  the previous session raises OnRediscoveryNeeded once (advisory re-browse chip). No extra traffic.
- AdminUI: Heidenhain joins NativeAlarmModel's alarm drivers; the typed editor fixes ALARM:
  tags at Boolean and seeds/drops the alarm object with the selector; driver form gains
  "Alarm conditions (ms)". (HeidenhainDriverFormModel also carries the #830 credential-notice change.)

Tests: HeidenhainAlarmTests (grammar, pairing, rules, simulator e2e with the captured iTNC 530
alarm record, local-only ack = zero bytes sent, quality-only outage/refusal, rediscovery),
Runtime.Tests HeidenhainNativeAlarmEndToEndTests (real DriverHostActor condition), a
DraftValidator case, and the NativeAlarmDriverSetGuard now scans the Heidenhain driver.
2026-11-11 01:37:41 -05:00
Joseph Doherty ae91e6b7f7 feat(opcuaclient): opt-in server-certificate host-name check; NAT fixture keeps its PKI (#816, #818)
#816 finding: the SDK checks that the server certificate names the host only
when a session is created with checkDomain:true. The driver, the AdminUI
browser and Client.Shared all called the 8-argument
DefaultSessionFactory.CreateAsync, which forwards checkDomain:false — so no
host-name check ever ran (why a trusted NAT cert for *.invalid browsed fine
over an IP in #813). When enabled the SDK (Session.OpenAsync ->
ValidateDomains -> FindDomain) compares ConfiguredEndpoint.EndpointUrl.IdnHost
— the CONFIGURED host, because OpcUaEndpointSelection/EndpointSelector
rewrote the advertised one — against the cert's DNS/IP SANs; secured
sessions only, and the auto-accept handler accepts a mismatch.

Policy: certificateValidation.validateServerHostName, tri-state. Unset =
OpcUaCertificateValidationOptions.DefaultValidateServerHostName = false, the
pre-#816 behaviour, so nothing changes on upgrade; true = validate the
configured host; false = explicit NAT/port-publish opt-out. Threaded into the
driver, the AdminUI browser (same JSON) and the AdminUI form (a select that
keeps "unset" unset), Client.Shared ConnectionSettings.ValidateServerHostName
and the CLI --validate-host-name. A driver refusal names the opt-out.

#818: the NAT fixture's /app/pki is on the opc-plc-nat-pki volume (verified:
same thumbprint across --force-recreate), its PUBLIC cert is committed as
Docker/nat/opc-plc-nat-server.der, and NatPublishedEndpointBrowseTests run
every secure browse on a throwaway PKI (internal browser seam): refusal with
nothing trusted, positive with the cert trusted (unset/false host check), and
a BadCertificateHostNameInvalid refusal with the check on. The trusted cert is
OPCUA_NAT_SERVER_CERT or the committed one (skip on thumbprint mismatch);
OPCUA_NAT_SERVER_CERT_TRUSTED is retired. CI exports its throwaway NAT
container's public cert. ServerHostNameValidationTests pin the driver against
the plain opc-plc. MIN_EXECUTED not raised (needs a hosted run): +6 in
v2-integration (440 -> 446), documented in scripts/ci/README.md.
2026-11-11 01:33:00 -05:00
Joseph Doherty a224fd6391 fix(security): scope operator-authored credential references per driver (#815)
A Designer could point a Galaxy / OpcUaClient endpoint at a host they control,
set a credential field to any reference the node can resolve
(env:ConnectionStrings__ConfigDb, env:ServerHistorian__ApiKey, secret:<any>,
Galaxy file:<any path>) and click Test Connect / live browse; the node resolved
the value and sent it to that host.

One shared rule in Core.Abstractions (CredentialReferences +
DriverCredentialPolicy) generalises the #803 Heidenhain namespace:
- Galaxy: secret:galaxy/... | secret:otopcua/galaxy/... | env:OTOPCUA_GALAXY_... |
  legacy env:GALAXY_MXGW_...; file: refused; dev:/literal API key unchanged.
- OpcUaClient: secret:opcuaclient/... | env:OTOPCUA_OPCUACLIENT_...; literals unchanged.
- Heidenhain adopts the shared parser/resolver (namespace unchanged).
Enforced at the AdminUI save path (DriverConfigSaveGate), the deploy gate
(CredentialReferenceDeployChecks, code CredentialReferenceOutOfScope), Test
Connect (AdminOperationsActor, before any probe), live browse
(BrowserSessionService, before any browser) and again at resolve time in every
driver/browser (fail closed, before any store/env read).

Co-Authored-By: Claude <noreply@anthropic.com>
2026-11-11 01:28:58 -05:00
Joseph Doherty 5bbe1e3c7a fix(heidenhain): resolve the probe's secret resolver lazily — eager resolution crash-looped central nodes at boot (#803)
v2-ci / unit-tests (push) Successful in 1h9m7s
The Test-Connect probe set is resolved inside the AdminOperations Akka starter; resolving
ISecretResolver there walks AddZbSecrets -> IAuditWriter -> ActorAuditWriter, whose ctor needs
AuditWriterActorKey (registered later in the same builder). Found by the docker-dev live gate.
Regression test: resolving the probe set never resolves ISecretResolver (fails before the fix).
Also: TagRowProblem doc comment re-attached; Heidenhain credential help text names the
secret:heidenhain/ and env:OTOPCUA_HEIDENHAIN_ namespaces.
2026-11-11 00:27:37 -05:00
Joseph Doherty 7af9c40e41 Merge feat/803-heidenhain-lsv2 2026-11-11 00:03:43 -05:00
Joseph Doherty 54f58c583c fix(heidenhain): refuse literal credentials and writable/mistyped tags at the save path (#803 review)
- Review 1: RawTreeService Create/UpdateDriver and Create/UpdateDevice refuse a Heidenhain
  config whose password fields are not blank / secret:heidenhain/... / env:OTOPCUA_HEIDENHAIN_...
  (DriverConfigSaveGate over HeidenhainOptionsParser.DescribeCredentialProblems: top level,
  Access, Devices[], case-insensitive keys; the value is never echoed), so a literal password
  never reaches the draft DB. The driver and device modals show the reason and disable Save.
- Review 8: CreateTag/UpdateTag/ImportTags (tag modal, manual entry, CSV import, browse commit)
  refuse a ReadWrite Heidenhain tag or a Tag.DataType that disagrees with what the driver
  publishes (HeidenhainTagRules: catalog type, Float64 => Double, or the binding's dataType).
  The tag modal and manual-entry grid fix the access level at Read and derive the data type from
  the address (RawTagRowDefaults). New deploy gate HeidenhainTagDataTypeMismatch.
- docs/drivers/Heidenhain.md: credential namespace convention + why, the save-path and deploy
  enforcement, optional-role credential behaviour, BadConfigurationError, backoff/ordering notes.

Tests: HeidenhainSavePathTests (service save path; red with the gates disabled) and new
DraftValidatorHeidenhainTests cases.
2026-11-10 23:54:32 -05:00
Joseph Doherty e39a125069 fix(acl): close the #808 review findings — reference variables, owner folder chain, NUL keys, unknown ids (#808)
Review of af5c9744:
1. UNS reference variables (s=Area/Line/Equip/Eff) of a moved or renamed
   equipment stayed readable/writable under the OLD names' grants: they now
   resolve as ReferenceVariable and are authorized at their split path only
   while it equals their live equipment folder's placement (live chain ==
   generation chain) + one name; otherwise denied (Read and Write).
2. TryPlaceScriptedCondition now also requires the owner folder's live chain
   to match the generation (AuthorizeScriptedCondition gains ownerFolderChain),
   so a moved equipment's alarm/children/commands follow its folder.
3. Identifiers containing NUL are denied before the decision cache, keeping
   client ids disjoint from the NUL-prefixed resolved-scope keys (a client
   could otherwise poison a real folder's cached verdict for its key shard).
4. An id no live node holds is Unknown: placed at [\0unknown] so only
   Cluster / Namespace-uns grants decide it (a removed opaque id can no longer
   split onto a same-named narrow grant; narrow grantees get no NodeDeleted for
   removed UNS folders/vtags/conditions — documented); 4-segment
   reference-shaped ids keep their split.
5. A wire test drives placements from DeploymentArtifact.ParseDataPlaneAuthority.

Baseline updates: #797 owner tests (unit + wire) and Sdk2RepeatedAlarmAuthority
now install folder scopes; Sdk2HistoryAuthority's UNS alias used the RAW path
as its UNS id (never minted in production) and now carries EqA/HistoryAlias.
Each new guard was falsified (reverting it fails its test).
2026-11-10 23:44:11 -05:00
Joseph Doherty 3641d56154 feat(heidenhain): AdminUI driver/device forms, typed tag editor, CSV + browse-commit (#803)
- HeidenhainDriverForm over HeidenhainDriverFormModel (JsonObject bag preserves Devices and
  unknown keys; no enum fields; Validate asks HeidenhainOptionsParser) + DriverConfigFormMap.
- HeidenhainDeviceForm/Model (Host, Port 19000, optional per-device password references)
  + DeviceFormMap. Literal passwords are refused in both forms.
- RawDriverTypeDialog entry; HeidenhainTagConfigEditor over HeidenhainTagConfigModel, whose
  Validate is the driver's own TryFromTagConfig; catalog leaves derive dataType + isArray;
  registered in TagConfigEditorMap + TagConfigValidator.
- CsvColumnMap (HhAddress/HhDataType/HhIsArray) and RawBrowseCommitMapper branch.
- Guards: DriverPageJsonConverterTests external-serializer entry, TagConfigNonEnumParityTests
  runtime map + corpus + freshly-authored rows, TagConfigRoundTrip, HeidenhainAuthoringTests.
2026-11-10 23:17:26 -05:00
Joseph Doherty af5c9744b7 fix(acl): place opaque-id UNS folders, virtual tags and scripted conditions for every operation (#808)
UNS Area/Line/Equipment folders are minted under their opaque entity ids
(s=UA-/UL-/EQ-...), virtual tags under s=EQ-.../Name and scripted conditions
(+ #629 children) under s=<ScriptedAlarmId>[/...], while every UNS grant is
planted at a NAME path. DataPlaneAuthorizer split those ids, so with
Security:Acl:Enabled a narrow (Area/Line/Equipment) grant could not Browse,
Read, subscribe or HistoryRead them, and Server-object delivery never reached
it — so a narrow key never held the EventId Ack/Confirm/AddComment need.

- The generation carries UnsFolderScopes (folderId -> path + id chain), built
  by DeploymentArtifact from the same AclScopePathResolver as the grants; ids
  two entity kinds share are dropped; plumbed through IDataPlaneAuthoritySink.
- OtOpcUaNodeManager.ResolveUnsAclSubject reports live facts (folder id
  ancestry / scripted owner binding); attached by the Authorizer setter.
- Evaluate places the subject against the CURRENT generation before the
  decision cache, which now caches the walk keyed on the resolved scope.
  Folders require live chain == generation chain (a moved equipment is denied
  under old and new line until the rebuild). Owned nodes (vtags, conditions,
  children) get owner path + a reserved segment, so Tag grants and their
  Browse transit never expose them. Conditions share TryPlaceScriptedCondition
  with the #797 command gate.
- Server-object visibility of a scripted condition is decided on the
  condition alone, so a moved alarm cannot reach its old owner there.

Tests: UnsFolderScopeWireTests (opaque ids, end-to-end browse -> EventId ->
Acknowledge on an Equipment grant; sibling denied; Tag-only grant; moved alarm
and moved equipment stale windows). Rig tests now install folder scopes
explicitly (ServerEventRig.FolderScopes); the #797 recreate test deliberately
asserts the moved alarm is invisible to the Cluster observer in the window.
2026-11-10 23:12:46 -05:00
Joseph Doherty 2b45694f91 feat(heidenhain): read-only, credential and endpoint deploy gates + TagConfig inspector (#803)
- HeidenhainDeployChecks (wired into DraftValidator): HeidenhainTagWritable refuses a
  ReadWrite access level on any Heidenhain tag, so no node is ever advertised writable;
  HeidenhainCredentialPersisted refuses a literal password in DriverConfig/DeviceConfig
  (only blank / secret: / env: references; the value is never echoed);
  HeidenhainConfigInvalid for out-of-range settings or a tagged device without an endpoint.
- EquipmentTagConfigInspector maps Heidenhain to HeidenhainTagDefinitionFactory.Inspect.
- Driver validates only devices that carry tags (an unconfigured default Device1 no longer
  faults the driver).
2026-11-10 23:10:36 -05:00
Joseph Doherty d50ee809e3 feat(heidenhain): native read-only LSV2 driver + in-process simulator (#803)
Native LSV2 TCP transport (default port 19000; no Java/Python/vendor SDK), ported from
the ignitionoee v0.20.3 reference driver:

- Lsv2ReadOnlyPolicy: payload-aware outbound allowlist (A_LG INSPECT/DNC/DATA, R_VR, R_PR,
  R_RI 22-28/30-34/51, bounded R_MB, R_DP), applied in Lsv2Codec.Frame before any byte is
  written; no public raw-command API.
- Lsv2Client: serialized transactions, one monotonic deadline per transaction, EOF/oversize/
  unexpected/trickle close the client; Close never waits on the gate.
- Lsv2Data / HeidenhainBindingCodec: reference decoders, R_PR byte-capacity + element-index
  rules, next-area cap, DATA Boolean 00/01/FF, strict non-truncating coercion.
- HeidenhainSession: per-batch wire cache, refusal suppression, provisional DNC denial retry,
  alarm cursor (57 = end, overflow = BadOutOfRange), capability strings.
- HeidenhainDeviceRuntime: one loop per control, generation-guarded publication, all-node
  BadNoCommunication on comms loss, bounded exponential reconnect.
- HeidenhainDriver: cached ReadAsync, push subscriptions deduped on (value,status), static
  catalog discovery (no connect), host status, diagnostics, skipped-tag report. No IWritable.
- Probe (INSPECT login + version) and factory registered in DriverFactoryBootstrap (probe on
  admin nodes too); DriverTypeNames.Heidenhain.

Tests: 159 unit tests against an in-process LSV2 simulator incl. the Java fixtures' captured
iTNC 530 replies as golden vectors; env-gated LiveIntegration category for real hardware.
2026-11-10 23:08:30 -05:00
Joseph Doherty 4620e542e8 fix(acl): require the live scripted-alarm owner to match the generation's deployed owner (#797)
Review of 3d1b86b4: OpcUaPublishActor installs a new authority generation
before applying its address space, and an apply that throws keeps the old one.
A scripted alarm moved from equipment A to B therefore stayed commandable with
A's grant under B's generation (briefly before the rebuild, or indefinitely
after a failed apply).

The generation now carries ScriptedAlarmOwners (ScriptedAlarmId -> EquipmentId,
decoded by ParseDataPlaneAuthority from the artifact's ScriptedAlarms rows, the
same rows the composition materialises from). AuthorizeScriptedCondition denies
unless the node manager's live binding names the same owner as the generation:
a moved alarm is denied under the old owner's, the new owner's and a Cluster
grant until the address space re-binds it; an alarm the generation no longer
deploys is denied. Install/apply order unchanged.

Tests: window (a) generation moved / binding stale => both owners and Cluster
denied, then rebuild under the new owner => new owner allowed, old denied;
window (b) apply failed after install => denied across clock and further
generations, then a successful rebuild restores the new owner only; undeployed
alarm denied; artifact decode of the owner map.
2026-11-10 21:42:03 -05:00
Joseph Doherty 3d1b86b451 fix(acl): authorize scripted alarm commands at their owning equipment's scope (#797)
Scripted condition NodeIds are opaque ScriptedAlarmIds, so splitting them as a
topology path could never match an Equipment grant: a narrow equipment grant was
denied while a Cluster grant passed.

- OtOpcUaNodeManager binds each LIVE scripted condition instance to the
  EquipmentId it was materialised under (only when it hangs under that folder);
  dropped on remove / drop-recreate / equipment-subtree removal / rebuild; a
  same-id re-materialise under another equipment re-parents and re-binds.
- DataPlaneAuthoritySnapshot carries EquipmentScopePaths (EquipmentId ->
  [uns, Area, Line, Equipment]) decoded from the artifact by the same
  AclScopePathResolver call that plants Equipment grants.
- DataPlaneAuthorizer.AuthorizeScriptedCondition re-reads key, grant and
  generation per call (uncached) and evaluates at the owner path; unbound,
  replaced or unplaceable owners deny under every grant. No id-split or
  realm-root fallback.
- Native conditions (RawPath) and the ACL-off role gate are unchanged.
  NodeIds and AlarmCommand.AlarmId are unchanged.

Sdk2OpaqueScriptedScopeBaselineTests deliberately flipped: it pinned the denial
and now pins owner-grant allow / sibling deny / cluster allow. Follow-up #808
(folder ids + Server-object event visibility for narrow UNS grants).
2026-11-10 21:23:57 -05:00
Joseph Doherty 19610c6812 fix: share SDK2 host disposal completion and close unadopted servers 2026-11-10 17:06:27 -05:00
Joseph Doherty cd60e469a1 docs: remove stale SDK 1 peer and calculator descriptions 2026-11-10 14:02:47 -05:00
Joseph Doherty ef8c2d1919 docs: clarify aggregate registration and SDK identity ownership 2026-11-10 12:48:41 -05:00
Joseph Doherty 41e65f6a22 fix(opcua): restore alarm component reference after SDK creation 2026-11-10 12:14:35 -05:00
Joseph Doherty a21104d27c fix(opcua): scope application audit receive permissions to validation 2026-11-10 11:41:31 -05:00
Joseph Doherty 09ac0e58cb fix(opcua): preserve repeated alarm methods and owned write reverts 2026-11-10 11:04:53 -05:00
Joseph Doherty ac5e9d84f0 fix: propagate SDK request cancellation to historian calls 2026-11-10 10:39:39 -05:00
Joseph Doherty 9712e66ff4 fix: retain certificate ownership and optional alarm workflows 2026-11-10 10:37:28 -05:00
Joseph Doherty 08354231f8 fix: register compatibility aggregates once during server startup 2026-11-10 10:32:13 -05:00
Joseph Doherty b71620145b fix: preserve SDK role identity and continuation authority lifecycle 2026-11-10 10:30:18 -05:00
Joseph Doherty a5bed28765 fix: preserve baseline OPC UA aggregate algorithms under SDK 2.0 2026-11-10 10:24:19 -05:00
Joseph Doherty 3dcb035f89 fix: preserve history sentinels and alarm SDK metadata 2026-11-10 10:21:41 -05:00
Joseph Doherty f9a006dab4 fix: initialize SDK telemetry and preserve ByteString boundaries 2026-11-10 10:11:32 -05:00
Joseph Doherty 9af1a824b3 refactor: migrate OPC UA server SDK 2.0 compilation and boundaries 2026-11-10 10:04:27 -05:00
Joseph Doherty d2076e75c6 build: pin OPC UA SDK 2.0.1 and migration tooling 2026-11-10 09:51:58 -05:00
Joseph Doherty 2a8e2981fe feat(scripting): ScriptWorker isolation on Windows: restricted worker identity, pipe ACL, firewall + PeCacheDir installer (lmxopcua#780)
v2-ci / unit-tests (push) Has been cancelled
Owner decisions §14 #5/#6 as built (design §12.5):

- Scripting:Worker:Identity (default unset, Windows only, fail-closed validation): each worker
  starts under a restricted token derived from the host's. Every privilege but SeChangeNotify is
  removed; the restricting SIDs are NT SERVICE\OtOpcUaScriptWorker's SID + Everyone + Users +
  RESTRICTED; the token and default DACLs are host + SYSTEM + identity + RESTRICTED. The worker
  is created suspended, put in its Job Object before it runs, inherits only its stdio pipes, gets
  an ACL'd work dir as TEMP, and runs with DOTNET_EnableDiagnostics=0. The identity is granted on
  the host's window station and desktop, because USER32 init made coreclr.dll fail with
  ERROR_DLL_INIT_FAILED on windev. The logon SID is not used because the host's process DACL
  gives it PROCESS_VM_READ.
- The worker pipe on Windows always has an explicit protected DACL: host account + identity
  only. There is no owner ACE, so an elevated host's Administrators gets no access.
- The worker treats access denied on its parent-pid poll as alive; the Job Object covers that case.
- scripts/install/Install-ScriptWorkerIsolation.ps1 + ScriptWorkerIsolation.psm1 are idempotent
  and support -WhatIf, -Uninstall and -PlanOnly. They create the disabled anchor service, the
  PeCacheDir ACL (protected, no Users, key and markers host-only, pe\ Modify for the identity),
  read+execute on script-worker\, a ReadData deny on the install root, the outbound firewall
  block on the worker exe, and optionally the host-service env.
- Tests: the dry-run harness scripts/install/tests/Test-ScriptWorkerIsolation.ps1 (46
  assertions, mutation-checked); ScriptWorkerIdentityTests; WorkerIsolationTests, which cover
  the pipe ACL, token access and a real restricted worker running a script, all green on windev.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 08:46:23 -05:00
Joseph Doherty 03686eb5d7 feat(scripting): Scripting:Runner=Shadow — in-process results published, each compiled run mirrored to the ScriptWorker pool and compared (lmxopcua#780 P2)
- ShadowScriptRunner (Core.Scripting): IScriptShadow.Mirror copies the inputs and queues the shadow run on the
  pool; compares value (structural), CLR type, status and failure class (compile / budget / threw+message /
  unsupported_type); runner-fault shadow outcomes (Unavailable, worker busy) are not compared. Bounded:
  Scripting:Shadow:MaxInFlight (256) and one per script; drops counted (behind | key_busy).
- Metrics otopcua.script.shadow.{compared,mismatch{script},dropped,inflight}; one rate-limited Warning per
  script and reason with a diff summary (Scripting:Shadow:MismatchLogsPerMinute, re-armed at the seam's deploy).
- The VirtualTag, Calculation and scripted-alarm seams recognise the shadow and keep their in-process path
  byte-for-byte (no coalescer, one-at-a-time pump, in-process engine), mirroring after each compiled run.
- ScriptRunnerSelection builds the shadow over an OutOfProcessScriptRunner whose lines carry
  ScriptRunnerRole=Shadow; Status() is null under Shadow so readiness never sees the shadow pool; the
  script-runner health entry stays Healthy and reports the shadow counters as data.
- docker-dev/docker-compose.script-runner-shadow.yml; docs/VirtualTags.md "Shadow mode".
- Tests: comparison table, per-script metric + once-per-reason log, rate limit, hung/blocking/throwing shadow
  never reaches the caller, MaxInFlight drop accounting, input copy; real-worker divergent + unsupported-type
  scripts counted against exactly those scripts, a worker-only runaway killed/quarantined there with every
  published value intact, alarm transitions identical to in process across a kill -9 of the shadow worker.
  Mutation-proved (16 mutants, 15 red; the one survivor added an assertion).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 07:23:06 -05:00
Joseph Doherty 0796928504 feat(scripting): batch submit, exact per-item deadlines, scripted-alarm seam on the out-of-process runner (lmxopcua#780)
v2-ci / unit-tests (push) Waiting to run
- IScriptRunner.RunBatch: one EvalBatch frame per worker for every evaluation ready to run; per-item, streamed results.
  The default (InProcessScriptRunner) is RunAsync per item. Used by the Calculation pump (dequeue all, split at a
  repeated tag, publish in dequeue order), the VirtualTag seam (ScriptBatchCoalescer, no timer) and the scripted-alarm
  engine (whole dirty set). In process the pump and the engine stay one at a time, unchanged.
- Exact per-item deadlines: the worker hands work to an idle evaluation thread or defers it (EvalResults
  deferred_eval_ids); the host stops that run's clock and restarts T then G when it starts (started_eval_ids). A run
  queued behind a runaway is neither timed out nor struck and is a victim, not the culprit, at the kill.
- Scripted-alarm seam behind Scripting:Runner:Seams; unset Seams now means VirtualTag + Calculation, so alarms run out of
  process only when listed. Load still compiles each predicate (to PE bytes only) so a broken one fails the load; passes
  batch and apply in the serial order under the gate. Script faults freeze the condition BadInternalError; runner faults
  freeze it BadResourceUnavailable and re-evaluate on recovery; a dead worker never clears an alarm. IScriptRunnerProvider
  carries the runner to DriverHostActor.
- Perf harness: batch throughput (RunBatch vs RunAsync x n) and alarm-pass latency (in process / batched / serial);
  design §12.3 records macOS + windev. On windev the serial out-of-process alarm pass misses 250 ms from ~500 alarms;
  batched p99 <= 53 ms at 1,000.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 06:37:36 -05:00
Joseph Doherty 000d7f5591 fix(virtualtags): route VirtualTag outputs through the dependency mux so VT->VT reads work (lmxopcua#791)
v2-ci / unit-tests (push) Waiting to run
Proven broken first: VirtualTagToVirtualTagFlowTests (real mux + host + Roslyn
evaluator) failed on master - VT A evaluated, VT B reading A by its key never did.
VT outputs only reached their OPC UA node; the mux carried driver values only.

- VirtualTagHostActor also sends each real result (not re-asserts) and a disabled
  tag's BadOutOfService to the mux as VirtualTagOutput, keyed by the VT's UNS
  NodeId identifier {EquipmentId}/{Name}. Publish path unchanged (single source).
- DependencyMuxActor pins VT outputs (retained with no subscriber, no grace,
  immune to SourceRemoved) and replays them; VirtualTagOutputsRemoved releases
  keys that leave the deployed set.
- DraftValidator: blocking VirtualTagDependencyCycle (A->B->A, self-read, loops
  through Calculation tags, {{equip}} refs resolved); disabled VTs are not nodes.
  EquipReferenceUnresolved names the key form when {{equip}}/<name> is a VT.
- Runtime backstop: the host re-detects VT cycles per apply and does not route
  outputs of tags on one.
- AdminUI ScriptTagCatalog lists VTs by key (the bare leaf name never resolved).
- docs/VirtualTags.md documents the routing, key, retention and cycle rules.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 05:17:00 -05:00
Joseph Doherty f201f16fd4 fix(virtualtags): a disabled VirtualTag keeps its node and reads BadOutOfService (lmxopcua#787, owner decision)
v2-ci / unit-tests (push) Has been cancelled
Revises 963b0d66, which dropped a disabled VirtualTag from the runtime plan so a
bound client got BadNodeIdUnknown. Now the plan carries Enabled (part of value
equality): AddressSpaceComposer.Compose and DeploymentArtifact.BuildEquipmentVirtualTagPlans
keep the row with Enabled=false (byte-parity), the applier materialises the node as
normal, and VirtualTagHostActor spawns no VirtualTagActor for it but publishes
BadOutOfService (new OpcUaStatus constant) with a null value stamped at deploy, on
every apply. Enabled flips are node-irrelevant to the change classifier; enabled->disabled
stops the child, disabled->enabled resumes evaluation. Validator exemption and name
uniqueness unchanged. Scripted alarms keep their not-materialised convention.
Tests updated; new wire test DisabledVirtualTagWireTests. docs/VirtualTags.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 04:48:57 -05:00
Joseph Doherty 8582ce91bc perf(scripting): cut out-of-process runner host CPU; compile timeout from compile start; quiet start-failure logs (lmxopcua#789, #790, #788)
v2-ci / unit-tests (push) Has been cancelled
#789 (partial, further tuning deferred): combining inline frame writer, buffered
frame reader, one timer per evaluation for T and T+G, slot release on the reader
before the caller resumes, a non-spinning per-connection result dispatcher, and
blocking pipe I/O on Unix (both ends) so the socket never enters the socket
engine; TerminateJobObject kill on Windows. macOS host CPU at 10k evals/s K=2:
228-240 % -> 79 % (InProcess 69 %, IPC share +10 vs budget 25); windev 153 % ->
123 % (+115, still a miss). Harness reports the "vs InProcess" share, warm-cache
worker RSS, and times kill latency from the soft-timeout answer.

#790: the worker sends LoadStarted; the compile timeout is armed then, so a load
queued behind other compiles is never killed or struck.

#788: one Error with stack per down period, rate-limited Warnings with attempt
count, one Information on recovery.

Design doc §8.2 notes + §12.2 record.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 04:25:08 -05:00
Joseph Doherty 963b0d66fc fix(virtualtags): a disabled VirtualTag is no longer deployed or evaluated (lmxopcua#787)
v2-ci / unit-tests (push) Waiting to run
VirtualTag.Enabled was authorable but ignored by both runtime-plan seams.
AddressSpaceComposer.Compose and DeploymentArtifact.BuildEquipmentVirtualTagPlans
now drop disabled rows (byte-parity), so no node is materialised and no
VirtualTagActor is spawned - the same outcome as a disabled scripted alarm.
DraftValidator exempts a disabled VT from VirtualTagNoEffectiveTrigger.
Calculation tags have no Enabled flag, so no equivalent gap. docs/VirtualTags.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 03:15:22 -05:00
Joseph Doherty 1ead8f458a feat(scripting): self-contained script worker publish, docker-dev drill, §8 perf harness (lmxopcua#780 P1)
v2-ci / unit-tests (push) Waiting to run
- A self-contained Host publish (-r win-x64 --self-contained, the Windows installs) now publishes the
  ScriptWorker self-contained for the same RID into script-worker/. The launcher starts its apphost
  directly when runtimeconfig.json lists includedFrameworks. A framework-dependent worker (docker-dev)
  keeps running through the dotnet muxer.
- docker-dev/docker-compose.script-runner.yml: OutOfProcess for the central pair's VT + Calc seams.
  Live drill recorded in design §12.1 (runaway kill + quarantine, kill -9 recovery, open circuit
  with /health/ready still 200, metrics).
- ScriptRunnerPerfTests (Category=Perf, opt-in via OTOPCUA_SCRIPT_PERF=1): InProcess vs OutOfProcess
  at K=1/2/4. The accepted 5k evals/s/node gate passes on macOS and windev. The host-CPU, RSS and
  Windows latency budget misses are tracked as #789.
- Fix: the single-flight slot is released before the awaiting caller resumes. Before, a caller that
  started its next run right after the previous one returned could be refused with "previous
  evaluation still running", which the seams publish as Bad.
- Fix: the worker compiles on dedicated threads, not the thread pool. A burst of cold loads starved
  the heartbeat, and the host killed the shard (found on windev). Load-timeout queueing is #790.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 02:53:53 -05:00
Joseph Doherty f50b29abae feat(scripting): PE cache, quarantine-race fix, recovery, readiness for the out-of-process runner (lmxopcua#780 P1)
v2-ci / unit-tests (push) Failing after 1h12m10s
Second P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md; still opt-in behind
Scripting:Runner=OutOfProcess (InProcess unchanged).

- Compiled-PE cache (design §7, §14 #8): Scripting:Worker:PeCacheDir holds <dir>/pe/<id>.pe entries authenticated
  with HMAC-SHA256 under a per-node random key (<dir>/pe-cache.key, created by the host, 0600/0700 on Unix), handed to
  the worker in Hello. Entry id = SHA-256(compile fingerprint + kind + source); the fingerprint covers the Roslyn
  version, the Core.Scripting build (analyzer, wrapper, stack guard), the sandbox reference set and the runtime. Edited,
  truncated, other-key or renamed entries are rejected, deleted and recompiled; the worker never loads unverified bytes.
  Pruned by recency at startup (30 d / 50 000). ScriptEvaluator splits into CompileToImage + FromImage. Measured on
  40 scripts, restart to every first result: macOS 820 ms no cache / 840 ms empty / 85 ms warm;
  windev (Windows) 9.5 s / 9.4 s / 1.0 s.
- Quarantine race: WorkerConnection runs exit attribution synchronously BEFORE completing the evaluations/loads it held
  (and before Exited), and the shard passes it straight to the runner — a kill strike is in place before the killed
  key's slot is released.
- Supervision: a dead worker restarts by itself after its backoff; Unavailable outcomes carry ScriptRunOutcome.Recovery
  (-> VirtualTagEvalResult.RetryWhen) and the VT actor / Calculation driver re-evaluate once against current inputs when
  it completes. Isolation worker stops after Scripting:Worker:IsolationIdleTimeout (10 min). Keys compiling when a
  worker dies are suspects too (isolation finds a poison compile and quarantines it). Startup orphan sweep by launch
  marker: kills only a process whose marker's host is gone and whose pid + start time + image match.
- Crash-loop circuit (Scripting:Worker:CircuitOpenAfter, 2 min): runner Unavailable, one Error, evaluations answered
  at once while shards keep retrying at the cap.
- Observability (§10): otopcua.script.runner.state, worker.up/inflight/queue_depth/heap_bytes (shard),
  ipc.roundtrip + eval.duration (kind), compile{cache}, isolation.suspects, exits reason=idle. `script-runner` health
  check on the ready tier, at worst Degraded (§14 #12: degraded but still ready).

Left on #780: self-contained worker publish, installer (account, pipe ACL, firewall), docker-dev live drill,
scripted-alarm seam, the §4.5 per-seam batch submit, the §8 perf harness.

Tests: SupervisionTests, PeCacheStoreTests/PeCacheWorkerTests (real workers), ScriptRunnerHealthCheckTests, VT + Calc
recovery seam tests; 14 mutations each turned their test red.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-10 00:20:24 -05:00
Joseph Doherty 69a6e0b4c8 feat(scripting): out-of-process pre-emptible script runner behind Scripting:Runner (lmxopcua#780 P1)
v2-ci / unit-tests (push) Waiting to run
Opt-in (Scripting:Runner=OutOfProcess; InProcess stays the default and its code path is unchanged). Implements
the P1 slice of docs/plans/2026-11-09-out-of-process-script-runner-design.md with the owner's §14 decisions:

- Options: Scripting:Runner (InProcess | Shadow | OutOfProcess), Scripting:Runner:Seams, Scripting:Worker:*
  (Shards 2, HardGrace max(1 s,T), HeapLimitBytes 512 MiB, QuarantineAfterKills 2, UnavailableGrace 5 s,
  IsolationProbationRuns 20, PeCacheDir validated absolute + outside the content root (fail closed),
  ExecutablePath). Shadow is not implemented yet: runs InProcess with a startup Warning.
- Core.Scripting: IScriptRunner seam + InProcessScriptRunner (today's guard, the parity baseline).
- Core.Scripting.Protocol: protobuf envelopes, 4-byte LE framing (zero/oversize/truncated/garbage refused),
  Variant carrying the exact CLR type; unsupported types are explicit failures.
- ScriptWorker app: compiles in the worker, P-thread scheduler, host-driven soft cancel, fail-fast
  watchdog, heartbeats, ctx.Logger forwarding, exits on pipe EOF / parent gone, forbidden-assembly
  self-check; Akka and Grpc.Core.Api excluded from its folder. Shipped in script-worker/ (Build + Publish).
- Core.Scripting.OutOfProcess: K shards + isolation worker, soft T / hard T+G kill, attribution
  (deadline kill = one key; crash = suspects -> isolation; isolation crash = strike), kill-strike
  quarantine, backoff 250 ms -> 30 s, fail closed, cleared allow-listed env, nonce + pid handshake on a
  current-user pipe, Windows Job Object.
- VirtualTag + Calculation seams use the runner when selected; a runner-side failure (worker restarted /
  runner unavailable) publishes BadResourceUnavailable (0x80040000) — ScadaBridge#112. Script faults keep
  BadInternalError. The scripted-alarm seam stays in process (alarms last).

Tests: new ZB.MOM.WW.OtOpcUa.ScriptWorker.Tests (Category=ScriptWorker, real processes; green on macOS and
on windev/Windows), seam tests in Host/Runtime/Calculation suites; mutation-checked.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 23:33:08 -05:00
Joseph Doherty 2da58d0360 build(deps): OPC Foundation SDK 1.5.378.106 -> 1.5.378.182 (lmxopcua#782 interim hop)
v2-ci / unit-tests (push) Has been cancelled
Behaviour-neutral hop: pin the SDK AggregateManager default to our
TreatUncertainAsBad=false (SDK .182 flipped it to true), parity guard now an
explicit deliberate-divergence assertion plus a booted-server check; adapt to
obsolete FindPredefinedNode(Type)/TraceConfiguration; Microsoft.Extensions.
Logging.Abstractions 10.0.7 -> 10.0.8 (SDK floor); stall wire test's late-flush
positive control is best-effort (.176 preserves publish queue ordering).

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:47:58 -05:00
Joseph Doherty 42c73bfb83 fix(adminui): confirm before deleting role grants, clusters, nodes and ACLs (lmxopcua#785)
v2-ci / unit-tests (push) Waiting to run
The four Delete buttons deleted on the first click. They now open the shared
RawConfirmDialog naming the item; only Confirm deletes. Adds bUnit (test-only)
render tests: modal opens without deleting, Cancel keeps the item, Confirm deletes.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 22:22:47 -05:00
Joseph Doherty 80118a01c5 fix(deploy): persist which nodes never acked on a deploy timeout (lmxopcua#784)
In the normal timeout branch (some acks landed) the deployment row was set to
TimedOut with a null FailureReason, so GET /api/deployments/{id} could not say
which node to fix; only the central log named it. The branch now persists a
reason naming the silent nodes ("Timed out waiting for acks (n/m landed). No ack
from: ..."), bounded to 20 named nodes plus a count of the rest, and still
backstopped by the 2048-char Truncate.

Tests: the reason names the silent node and not the acked one; on a 25-node
silence it names 20, counts 5 and fits the column. Mutation-checked: dropping
the SetFailureReason fails both, dropping the bound fails the second.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 21:30:17 -05:00
Joseph Doherty 43f61eba2e fix(mesh): stop reporting override-keyed clusters as REVOKED (lmxopcua#784)
The docker-dev centrals logged "Mesh PSK secret ot-mesh-psk-<cluster> ... is
absent (or blank) in the store; the key is REVOKED" for every cluster on every
contact refresh, while every gRPC dispatch and ack authenticated.

Investigated: not an auth bypass. The rig supplies each cluster's key through
MeshTransport:ClusterPsks (central side, equal to that cluster's nodes'
MeshTransport:GrpcPsk) and seeds nothing in the store. TryGetPsk/Matches consult
the override first on every call, so enforcement was on; only RefreshAsync read
the store for those clusters regardless and logged a revocation that never took
effect. RefreshAsync now skips clusters covered by a non-blank override, so the
REVOKED warning is only logged for a cluster that genuinely has no key.

Test pins it with a positive control (a cluster with no override is still read,
warned about and refused) and enforcement on the override key (wrong token
rejected). Mutation-checked: removing the skip fails the test.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 21:30:17 -05:00
Joseph Doherty d9b95dcf56 fix(virtualtags): consume VirtualTag.TimerIntervalMs; add VirtualTagNoEffectiveTrigger gate (lmxopcua#783)
v2-ci / unit-tests (push) Waiting to run
Choice: implement the timer (option a). The field was designed and promised as a
working trigger: the entity doc says ChangeTriggered and TimerIntervalMs together
realize the trigger model, the DB check constraint requires ChangeTriggered OR a
timer >= 50, the AdminUI virtual-tag modal exposes "TimerIntervalMs (optional)" and
UnsTreeService rejects a tag with neither, and the 2026-06-08 design specified
"ChangeTriggered OR TimerIntervalMs>=50". Only the runtime never honoured it (the
standalone TimerTriggerScheduler was deleted in #603), so a timer-only tag sat at
BadWaitingForInitialData forever.

- EquipmentVirtualTagPlan carries ChangeTriggered/TimerIntervalMs (in equality, so
  an edit respawns the child); both the composer and the artifact decode fill them.
- VirtualTagActor schedules a repeating tick (PreStart, cancelled in PostStop,
  clamped to the 50 ms floor). ChangeTriggered=false makes the timer the only
  trigger. A tick during an in-flight async run (#779) is dropped, not stashed, so
  slow scripts never build a backlog of runs.
- DraftValidator VirtualTagNoEffectiveTrigger (hard error, same severity as
  CalculationNoEffectiveTrigger): no timer and no change trigger with a literal
  ctx.GetTag read.

Tests: VirtualTagActorTimerTests, DraftValidatorVirtualTagTriggerTests, artifact
decode test. Mutation-checked: no scheduling, stashing ticks, no cancel in
PostStop, and removing the validator call each fail a test.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 21:27:59 -05:00
Joseph Doherty c5df43d594 fix(historian): ContinuousHistorization CommitMode code default is PerEntry, matching appsettings/docker-dev/fallback (lmxopcua#776)
v2-ci / unit-tests (push) Has been cancelled
The Lane H2 remediation made Periodic(100 ms) the options-class default, but
appsettings.json, docker-dev and Program.cs's bad-value fallback all used
PerEntry, so effective behaviour never changed and the docs contradicted the
shipped config. The options default is now PerEntry; Periodic stays opt-in
(<= CommitIntervalMs of crash loss in exchange for throughput). Program.cs now
uses ContinuousHistorizationOptions.ResolveCommitMode() so the fallback shares
the single DefaultCommitMode constant. docs/Historian.md updated.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 20:27:06 -05:00
Joseph Doherty 241318e7e5 feat(deploy): reject Sql Query tags naming undeclared queries; warn on Writable/writable disagreement (lmxopcua#781)
v2-ci / unit-tests (push) Has been cancelled
Item 5: DraftValidator now emits SqlQueryNotDeclared (always blocking) for a Query tag whose
query is absent from the merged driver config's queries map, via the driver's own tag parse.
Item 3: StartDeployment appends a non-blocking warning to the accepted message when the Tag
entity's AccessLevel and the Sql blob's writable flag disagree.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
2026-11-09 18:08:54 -05:00