Commit Graph

1486 Commits

Author SHA1 Message Date
Joseph Doherty 8dcc55f633 test: address review feedback — cover WaitForAsync predicate overload; clarify ChildrenAccessor parity row; dedup object-method exclusion 2026-06-17 11:16:33 -04:00
Joseph Doherty b88f04ec2d fix(siteruntime): normalize routed WaitForAttribute response value for cross-process transport 2026-06-17 11:10:17 -04:00
Joseph Doherty adc8ee4afa test(scriptanalysis): parity test fails on any unmirrored runtime accessor method 2026-06-17 11:09:00 -04:00
Joseph Doherty bee295d3ee fix(central-ui): mirror WaitForAttribute on inbound-script analysis RouteTarget
Add WaitForAttribute(attributeName, targetValue, timeout, cancellationToken)
to InboundScriptHost.RouteTarget and SandboxInboundScriptHost.RouteTarget,
mirroring the shipped runtime signature in RouteHelper. Eliminates the false
CS error the editor raised against valid Route.To("X").WaitForAttribute(...)
calls in inbound API method scripts. Test asserts the call diagnoses clean
under ScriptKind.InboundApi.
2026-06-17 11:04:13 -04:00
Joseph Doherty a1186685a9 fix(scriptanalysis): mirror WaitAsync/WaitForAsync on CompileAttributeAccessor
Adds the four missing overloads (value + predicate × WaitAsync + WaitForAsync)
to CompileAttributeAccessor so template/call scripts that use Attributes.WaitAsync
or Attributes.WaitForAsync pass design-time Roslyn validation.  Covers both root
scope and composed/child scope (Children["x"].Attributes.WaitAsync) automatically
since CompileCompositionAccessor.Attributes already returns CompileAttributeAccessor.
2026-06-17 11:03:24 -04:00
Joseph Doherty dc43a3f0f6 docs(plans): add WaitAsync compile-surface mirror-gaps plan + tasks 2026-06-17 11:00:41 -04:00
Joseph Doherty c2e89e9d40 fix(central-ui): never render DB connection strings on Integration Definitions list
Connection strings carry credentials; the Database Connections tab rendered the
full string (text + title tooltip) for any Design/Admin user. Replace with a
non-sensitive 'hidden — edit to view' hint so it never reaches the browser DOM.
Connection strings remain editable on the create/edit form. Adds a bUnit
regression guard asserting the seeded secret is absent from the rendered list.
2026-06-17 10:51:18 -04:00
Joseph Doherty af54c8ad11 merge: integrate WaitAsync/M5-audit (parallel session) with galaxy array-write + inbound-timeout fixes 2026-06-17 09:28:15 -04:00
Joseph Doherty bf2f481bb4 fix(siteruntime): normalize routed script return value for cross-process transport
A routed inbound-API call (Route.To(inst).Call(script)) runs the script on
the Site and returns its value to Central inside RouteToCallResponse, which
crosses the Central<->Site PROCESS boundary. A script's natural
'return new { ... }' is a compiler-generated anonymous type that Akka's
cross-process serializer cannot reconstruct on the receiving node, so the
reply was silently dropped and the caller's Route.To().Call() Ask timed out
at 30s with 'Script execution timed out' -- even though the script completed
and all device writes committed.

DeploymentManagerActor.RouteInboundApiCall now projects the routed return
value to a plain CLR graph (Dictionary/List/string/long/double/bool/null)
via a JSON round-trip before placing it in RouteToCallResponse. The graph
round-trips the wire and re-serializes to the same JSON shape the inbound
API expects for the HTTP body / ReturnDefinition validation.

Diagnosed live: IpsenMESMoveIn writes committed + site_events showed the
IpsenMoveIn script completed in ~0.6s, yet the inbound POST returned 500 at
30s; Central's Akka serializer logged 'Writing value of type
<>f__AnonymousType0`1 as Json' at the timeout moment.

379/379 SiteRuntime tests green.
2026-06-17 09:19:12 -04:00
Joseph Doherty 11534089b9 docs(siteruntime): mark WaitAsync deferred items implemented (§3/§4.2/§6) + fast-path throwing-predicate test 2026-06-17 09:15:42 -04:00
Joseph Doherty c482cac110 feat(siteruntime): unpack routed RouteToWaitForAttributeRequest into InstanceActor (spec §6 site half) 2026-06-17 09:10:08 -04:00
Joseph Doherty 61048a4ecf feat(siteruntime): WaitForAsync/WaitResult + quality-gated WaitAsync (spec §3, §4.2) 2026-06-17 09:05:12 -04:00
Joseph Doherty 0f6da8a106 feat(inbound): routed Route.To().WaitForAttribute — contract + central path (spec §6) 2026-06-17 09:02:21 -04:00
Joseph Doherty cd15426b21 docs(siteruntime): plan for WaitAsync deferred items (WaitForAsync, quality-gated, routed §6) 2026-06-17 08:58:05 -04:00
Joseph Doherty 04e97f4a87 fix(siteruntime): harden WaitAsync — no spurious match on quality republish, guard throwing predicate, Ask-timeout returns false 2026-06-17 08:44:03 -04:00
Joseph Doherty eeb6210151 fix(dcl): address MXAccess array attributes with trailing "[]" on write
MXAccess silently no-ops a whole-array write unless the item reference
ends in "[]" (e.g. "<object>.MoveInWorkOrderNumbers[]") — the COM Write
returns success but the value never commits. Reads work either way, so
the bug surfaced only on writes. Mirror the AVEVA MES Camstar API, which
registers array tags as "<object>.<attr>[]" (scalars have no brackets).

WriteAsync now resolves/advises/writes array values against tag + "[]"
(scalars unchanged), keeping the original tag for result mapping. Adds
IsArrayValue matching the ToMxValue/PadArrayToDeclaredSizeAsync array set.

Verified live via mxwrtest against the deployed gateway: bare ref write
ok but read-back unchanged; "[]" ref write commits (read-back changes,
fresh source timestamp). No RealMxGatewayClient unit harness exists (the
gRPC session is concrete) — consistent with how the sibling supervisory/
pad/encode fixes are verified.
2026-06-17 08:32:03 -04:00
Joseph Doherty 75ffa09b8f feat(siteruntime): event-driven Attributes.WaitAsync attribute-change helper
Adds InstanceActor one-shot waiter registry (fast-path + change-match + scheduled
timeout self-eviction), threads per-script timeout token through ScriptRuntimeContext,
and exposes Attributes.WaitAsync(value|predicate, timeout). Replaces handshake busy-poll.
Implements spec docs/plans/2026-06-17-waitfor-attribute-change-helper-spec.md §3-§5;
§6 routed variant + WaitForAsync + quality-only mode deferred.
2026-06-17 08:25:06 -04:00
Joseph Doherty b89d69a008 docs(siteruntime): add WaitAsync attribute-change helper spec 2026-06-17 08:14:09 -04:00
Joseph Doherty 8cbecdec0e fix(inbound): materialize array params as typed lists, not JsonElement
An inbound /api array parameter was materialized as List<object?> whose
elements were raw System.Text.Json.JsonElement. When such a value is routed
Central->Site and a template script assigns it to a List-typed Galaxy
attribute (recv.Attributes[name] = Parameters[name]), the script-side encode
stalls (the attribute codec JSON-serializing JsonElement items) and the array
write never reaches the DCL — the Ipsen MoveIn array writes hung 30s while
scalars succeeded.

ParameterValidator.MaterializeArray now builds a strongly-typed list per the
declared element schema (List<string>/long/double/bool); arrays with no
declared scalar element type materialize each element to its CLR value
(MaterializeJsonValue) so no raw JsonElement survives. Typed lists serialize
cleanly across nodes and encode to a canonical JSON array, which the
InstanceActor decodes back to the typed list for the device write.
2026-06-17 07:34:39 -04:00
Joseph Doherty 45b23476fc fix(dcl): pad List writes to the Galaxy array's full declared size
Even with correct array encoding (30d07b9), Ipsen MoveIn array writes still
hung: the Galaxy MES-receiver arrays are fixed-size SAFEARRAYs (e.g.
MoveInWorkOrderNumbers = SAFEARRAY(VT_BSTR) dimensions:[50]) and MXAccess only
accepts a write that supplies ALL slots. ScadaBridge sent just the N elements
the MES provided (1-2), so the COM write blocked. Verified on the live gateway:
a full-size (50) constructed array writes via WriteBulk in ~34ms; a short one
does not.

RealMxGatewayClient.WriteAsync now, for a list value, reads the tag's current
array to learn its slot count and pads the value to that length with
element-type defaults (empty string / 0 / false / default) — the caller's
values fill slots 0..N-1, the rest are cleared. The PLC reads the valid count
from a separate scalar (MoveInNumberWorkOrders). If the size can't be
determined (read fails / not an array) the value is written unpadded and a
warning is logged. Scalars are unaffected.
2026-06-17 06:34:05 -04:00
Joseph Doherty 30d07b91f4 fix(dcl): write List attributes as real MXAccess arrays (not a stringified List)
The Ipsen MoveIn e2e (after the supervisory-advise fix landed scalar writes)
exposed a second blocker: writes to List-typed attributes
(MoveInWorkOrderNumbers / MoveInPartNumbers, List<string>) hung at the 30s
device-write timeout while scalar writes succeeded.

InstanceActor.HandleSetDataAttribute already decodes a List attribute's
canonical JSON into a typed List<T> before the write (so the DCL can push a
real array), but RealMxGatewayClient.ToMxValue only had scalar cases — a
List<T> fell through to Convert.ToString and wrote the garbage string
"System.Collections.Generic.List`1[System.String]" to the array Galaxy
node, which the gateway's COM write rejected/blocked.

Add IReadOnlyList<bool|int|long|float|double|string|DateTimeOffset|DateTime>
cases that call the client package's typed array encoders
(VT_ARRAY|VT_BSTR etc.); List<DateTime> is mapped to DateTimeOffset. Covers
every element type AttributeValueCodec produces.
2026-06-17 06:07:40 -04:00
Joseph Doherty b5333d0f15 fix(dcl): default MxGateway advises to supervisory when WriteUserId==0
Writes through the MxGateway data connection (e.g. the Ipsen MoveIn flow
writing MES-receiver attributes) hung ~30s and changed nothing, while reads
of the same attributes worked. Root cause: MXAccess only accepts a write on
an item that holds a SUPERVISORY advise; the write path did AddItem +
WriteBulk with no advise and the monitoring subscription used a plain Advise,
so the worker's synchronous COM Write blocked until the gateway command
timeout. (Plain, non-secured writes need no user/login.) Verified live: with a
supervisory advise the write returns ok in ~22ms; without it it does not.

When the connection has no MXAccess write-user context (WriteUserId == 0) it
now behaves as a supervisory client: every advise defaults to
AdviseSupervisory — both the monitoring subscription (SubscribeAsync) and the
write path — so one connection can read and write. A supervisory advise still
delivers OnDataChange (the worker treats either advice kind as sufficient for
updates) so monitoring is unaffected, and the worker's UnAdvise tears down
either kind, so unsubscribe is unchanged. AdviseSupervisory is issued as a raw
MxCommandKind.AdviseSupervisory via the session's Invoke (the client package
exposes only plain Advise). The advise runs at most once per handle via a
Lazy<Task> so a concurrent first-time subscribe+write on the same new handle
both await the same advise (neither writes before it completes); a faulted
advise is evicted so the next write retries. Dropped on unsubscribe. A
configured non-zero WriteUserId keeps the prior plain-advise behaviour.
2026-06-17 05:52:30 -04:00
Joseph Doherty 639e331db1 test+docs(m5): M5.7 — de-date 2 EndToEnd purge tests (closes #52); document T3-T8 in Component-AuditLog/-CLI/README/CLAUDE
Tests: anchor SeedOccurredAt() to a fixed thresholdAnchor (2026-01-20) and compute
RetentionDays dynamically (UtcNow - anchor + 1d) so the threshold always sits near
Jan 20 2026, between the Jan-15 "old" seed (purged) and Apr-15/Jun-15 "kept" seeds.
Seed dates stay within the explicit pf_AuditLog_Month boundary range (Jan 2026 –
Dec 2027) — relative-from-now offsets landed before 2026-01-01 (the catch-all
partition, invisible to GetPartitionBoundariesOlderThanAsync). Both tests confirmed
passing; all 284 AuditLog tests green.

Docs:
- Component-AuditLog.md: per-channel retention overrides (T3, PerChannelRetentionDays
  + bounded DELETE + AuditLogPurge:ChannelPurgeBatchSize); ParentExecutionId tag-cascade
  now spans alarm-triggered + nested CallScript/CallShared + inbound→routed (T4, "no
  further spawn points deferred"); per-node stuck KPIs for Notification Outbox +
  Site Call Audit (T6); T7 structured response-capture increments (request headers in
  Extra.requestHeaders, AuditInboundCeilingHits counter, per-method SkipBodyCapture);
  T8 CLI audit tree; T1 hash-chain + T2 Parquet explicitly marked deferred to v1.x.
- Component-CLI.md + README.md: document audit tree --execution-id <guid> and
  audit backfill-source-node --sentinel/--before/--batch with exact options verified
  against AuditCommands.cs; update Interactions to list new endpoints.
- CLAUDE.md: update audit-log design-decision bullets for T3 per-channel retention,
  T4 tag-cascade complete, T6 per-node KPIs, T7 inbound capture increments, T8 tree
  command; clarify T1/T2 remain deferred to v1.x.
2026-06-16 22:26:09 -04:00
Joseph Doherty 1b63d6751f fix(audit): M5 integration — add BackfillSourceNodeAsync to 5 test stubs (M5.5+M5.6 interface collision after cherry-pick recovery) 2026-06-16 22:11:10 -04:00
Joseph Doherty 50b674accc feat(audit): M5.5 per-channel retention overrides via purge-role bounded delete (T3) 2026-06-16 22:05:08 -04:00
Joseph Doherty 55630b48b6 feat(audit): M5.6 SourceNode sentinel backfill (purge-role) + CLI + runbook note (T5) 2026-06-16 22:02:21 -04:00
Joseph Doherty d4ec84d5fb fix(inbound): log swallowed scope-creation failure + test scope disposal on script throw 2026-06-16 22:00:10 -04:00
Joseph Doherty daff1446d8 feat(inbound): expose read-only Database helper on InboundScriptContext 2026-06-16 22:00:10 -04:00
Joseph Doherty 16fc62bfa0 test(inbound): add namespace + Query() coverage for InboundDatabaseHelper 2026-06-16 22:00:10 -04:00
Joseph Doherty 90ac746fdc feat(inbound): read-only InboundDatabaseHelper for inbound scripts 2026-06-16 22:00:10 -04:00
Joseph Doherty f63055c296 chore(inbound): task-persistence for Ipsen MES MoveIn plan 2026-06-16 22:00:10 -04:00
Joseph Doherty 6fbf3e71fd docs(inbound): implementation plan for Ipsen MES MoveIn 2026-06-16 22:00:10 -04:00
Joseph Doherty d8ccad6f54 docs(inbound): design for Ipsen MES MoveIn -> reactor MES receiver
SAPID(+side) -> BTDB Machine.SAPID -> Code -> instance; inbound script does
the lookup via a new scoped read-only DB helper, then routes to a new T1
template script that gates on MoveInReadyFlag and writes the MoveIn to the
correct Left/Right MES receiver. -LT deferred.
2026-06-16 22:00:10 -04:00
Joseph Doherty de2968b03d fix(audit): M5.3 review I-1 — use the request-start options snapshot for header redaction (no mid-request reload split) 2026-06-16 21:49:43 -04:00
Joseph Doherty 20760014c2 feat(audit): M5.4 ParentExecutionId tag-cascade for alarm + nested calls (T4) 2026-06-16 21:42:14 -04:00
Joseph Doherty 209f368cb5 feat(audit): M5.2 per-node stuck-count KPIs (T6) — repo per-node aggregation, actor message pair, CentralUI tiles 2026-06-16 21:34:14 -04:00
Joseph Doherty a07ff28f10 feat(audit): M5.3 response-capture increments — request headers, ceiling-hits counter, per-method body opt-out (T7)
1. Request headers in Extra JSON (AuditWriteMiddleware): adds a `requestHeaders`
   object to the existing Extra JSON alongside remoteIp/userAgent; headers whose
   names appear in AuditLogOptions.HeaderRedactList (Authorization, X-Api-Key,
   Cookie, Set-Cookie by default) are replaced with "<redacted>" using
   OrdinalIgnoreCase matching — same policy as ScadaBridgeAuditRedactor.

2. AuditInboundCeilingHits counter: new IAuditInboundCeilingHitsCounter interface
   + NoOpAuditInboundCeilingHitsCounter default; AuditCentralHealthSnapshot
   implements the interface (Interlocked field, thread-safe) and exposes
   AuditInboundCeilingHits on IAuditCentralHealthSnapshot; AddAuditLog registers
   the NoOp default, AddAuditLogCentralMaintenance forwards to the snapshot;
   AuditWriteMiddleware accepts the counter as an optional ctor arg and increments
   it once per request where either the request or response body hit the cap.

3. Per-method SkipBodyCapture opt-out: adds SkipBodyCapture bool to
   PerTargetRedactionOverride; AuditWriteMiddleware consults the per-target
   override map at the start of InvokeAsync (before EnableBuffering) and, when
   set, skips body read + capture entirely — the audit row still emits with
   headers/metadata but null RequestSummary/ResponseSummary; truncation flags
   are also cleared so the ceiling-hits counter is not bumped for opted-out methods.
2026-06-16 21:23:07 -04:00
Joseph Doherty 0569c5ff23 feat(audit): M5.1 audit tree endpoint + CLI audit tree (T8)
Add GET /api/audit/tree endpoint that accepts executionId query param,
authenticates via HTTP Basic + LDAP (OperationalAudit permission), calls
IAuditLogRepository.GetExecutionTreeAsync, and returns a JSON array of
ExecutionTreeNode. Returns 400 for missing/invalid GUID, 401/403 as normal.

Add `scadabridge audit tree --execution-id <guid> [--format table|json]`
CLI subcommand in AuditCommands.Build(). Adds AuditTreeHelpers with:
  - BuildUrl: constructs the relative URL + query string
  - RunTreeAsync: calls the endpoint, dispatches to table or JSON renderer
  - WriteTable: indented ASCII tree (root → children, [*] marks queried node)
  - WriteJson: pretty-printed JSON array pass-through

Tests: 7 new ManagementService endpoint tests (valid id, empty, 400, 401,
403, Viewer allowed, wrong role), 18 new CLI tests (parse, render, HTTP
error codes, JSON output, multi-level indentation, queried-node marker).
2026-06-16 21:20:54 -04:00
Joseph Doherty 0e9bcbb676 docs(m5): implementation plan + tasks for audit hardening T3-T8 2026-06-16 21:12:52 -04:00
Joseph Doherty 1b7600fcb3 docs(m5): design — audit hardening T3-T8 (T1 hash-chain + T2 Parquet stay deferred) 2026-06-16 21:10:21 -04:00
Joseph Doherty e77e209b8a docs(m4): fix bundle CLI example option names in transport design §13 (--output/--input, not --out/positional) 2026-06-16 20:37:51 -04:00
Joseph Doherty dd545281e6 docs(m4.3): reconcile CLI README + Component-CLI to registered commands/options (document bundle group; fix option drift) 2026-06-16 20:30:40 -04:00
Joseph Doherty 0780c2e49e docs(m4.4): clear stale deferred/no-op markers for shipped features (relay, bundle-import audit, M5 redaction, audit drill-in, Transport CLI, traceability)
- SiteCallAudit/ServiceCollectionExtensions.cs: drop "still deferred" note on relay; point to SiteCallAuditActor where it lives
- Transport/Import/BundleImporter.cs: update "Only LoadAsync implemented" to reflect all three phases shipped
- SiteRuntime/Scripts/AuditingDbCommand.cs: replace two M5-deferred redaction comments with accurate references to AuditLogOptions.PerTargetOverrides
- SiteRuntime/Scripts/ScriptRuntimeContext.cs: replace "M5 will layer redaction" note with accurate description of shipped redactor
- CentralUI/AuditLogPage.razor.cs: replace "Bundle C wires… no-op seam" with accurate description of HandleRowSelected implementation
- docs/plans/2026-05-24-transport-design.md §13: update from "CLI Deferred / not built in v1" to reflect shipped BundleCommands.cs; update Open Questions entry
- docs/plans/2026-05-24-transport.md: convert Out-of-Scope "Do NOT build CLI" reminder to a factual note that it shipped
- docs/plans/2026-05-24-transport.md.tasks.json: flip all 30 tasks from pending → done (entire Transport feature shipped)
2026-06-16 20:30:29 -04:00
Joseph Doherty 13605d3dfd docs(m4.1): reconcile Config-DB AuditLog schema + Commons (AuditEvent/ApiKey/SiteCall/NotificationType) to shipped code 2026-06-16 20:29:16 -04:00
Joseph Doherty c3b046457e docs(m4.2): reconcile InboundAPI (Bearer/audit-timing/type-validation), Security (cookie session, role names), Notification (Email-only, AuditKind vocab) to code 2026-06-16 20:27:43 -04:00
Joseph Doherty 9106efafd8 Merge main (DCL alarm fixes 06ef177..9b78e60) into M3 branch 2026-06-16 20:20:27 -04:00
Joseph Doherty fb5f14e04f docs(m3): document Script Analysis component (#25); reconcile consumer specs + README/CLAUDE component list 2026-06-16 20:05:24 -04:00
Joseph Doherty 069757209a fix(scriptanalysis): M3.6 — full-framework analysis refs close forbidden-type-in-allowed-ns blind spot; pin Process/Stopwatch; fix stale codec test; drop dead ContainsInCode 2026-06-16 20:00:28 -04:00
Joseph Doherty 9b78e6071d fix(dcl): identify MxGateway native alarms by object-relative reference
Surface native (Galaxy/MxGateway) alarms by their object-relative reference
(e.g. "Z28061.HeartbeatTimeoutAlarm") instead of the gateway's full provider
reference ("Galaxy!<area>.<object>.<alarm>"). The area is already preserved in
Category and the object reference is globally unique within the galaxy, so the
full provider prefix added only noise to the alarm identity operators see.

MxGatewayAlarmMapper.MapTransition/MapSnapshot now set SourceReference from
SourceObjectReference, falling back to AlarmFullReference only when the gateway
omits the object reference. +2 mapper tests; full DCL suite green (158).
2026-06-16 19:46:44 -04:00
Joseph Doherty cf935d5744 refactor(centralui): M3.5 ScriptAnalysisService uses shared deny-list + delegates trust verdict 2026-06-16 19:40:03 -04:00