Commit Graph

1460 Commits

Author SHA1 Message Date
Joseph Doherty 04e97f4a87 fix(siteruntime): harden WaitAsync — no spurious match on quality republish, guard throwing predicate, Ask-timeout returns false 2026-06-17 08:44:03 -04:00
Joseph Doherty 75ffa09b8f feat(siteruntime): event-driven Attributes.WaitAsync attribute-change helper
Adds InstanceActor one-shot waiter registry (fast-path + change-match + scheduled
timeout self-eviction), threads per-script timeout token through ScriptRuntimeContext,
and exposes Attributes.WaitAsync(value|predicate, timeout). Replaces handshake busy-poll.
Implements spec docs/plans/2026-06-17-waitfor-attribute-change-helper-spec.md §3-§5;
§6 routed variant + WaitForAsync + quality-only mode deferred.
2026-06-17 08:25:06 -04:00
Joseph Doherty b89d69a008 docs(siteruntime): add WaitAsync attribute-change helper spec 2026-06-17 08:14:09 -04:00
Joseph Doherty 639e331db1 test+docs(m5): M5.7 — de-date 2 EndToEnd purge tests (closes #52); document T3-T8 in Component-AuditLog/-CLI/README/CLAUDE
Tests: anchor SeedOccurredAt() to a fixed thresholdAnchor (2026-01-20) and compute
RetentionDays dynamically (UtcNow - anchor + 1d) so the threshold always sits near
Jan 20 2026, between the Jan-15 "old" seed (purged) and Apr-15/Jun-15 "kept" seeds.
Seed dates stay within the explicit pf_AuditLog_Month boundary range (Jan 2026 –
Dec 2027) — relative-from-now offsets landed before 2026-01-01 (the catch-all
partition, invisible to GetPartitionBoundariesOlderThanAsync). Both tests confirmed
passing; all 284 AuditLog tests green.

Docs:
- Component-AuditLog.md: per-channel retention overrides (T3, PerChannelRetentionDays
  + bounded DELETE + AuditLogPurge:ChannelPurgeBatchSize); ParentExecutionId tag-cascade
  now spans alarm-triggered + nested CallScript/CallShared + inbound→routed (T4, "no
  further spawn points deferred"); per-node stuck KPIs for Notification Outbox +
  Site Call Audit (T6); T7 structured response-capture increments (request headers in
  Extra.requestHeaders, AuditInboundCeilingHits counter, per-method SkipBodyCapture);
  T8 CLI audit tree; T1 hash-chain + T2 Parquet explicitly marked deferred to v1.x.
- Component-CLI.md + README.md: document audit tree --execution-id <guid> and
  audit backfill-source-node --sentinel/--before/--batch with exact options verified
  against AuditCommands.cs; update Interactions to list new endpoints.
- CLAUDE.md: update audit-log design-decision bullets for T3 per-channel retention,
  T4 tag-cascade complete, T6 per-node KPIs, T7 inbound capture increments, T8 tree
  command; clarify T1/T2 remain deferred to v1.x.
2026-06-16 22:26:09 -04:00
Joseph Doherty 1b63d6751f fix(audit): M5 integration — add BackfillSourceNodeAsync to 5 test stubs (M5.5+M5.6 interface collision after cherry-pick recovery) 2026-06-16 22:11:10 -04:00
Joseph Doherty 50b674accc feat(audit): M5.5 per-channel retention overrides via purge-role bounded delete (T3) 2026-06-16 22:05:08 -04:00
Joseph Doherty 55630b48b6 feat(audit): M5.6 SourceNode sentinel backfill (purge-role) + CLI + runbook note (T5) 2026-06-16 22:02:21 -04:00
Joseph Doherty de2968b03d fix(audit): M5.3 review I-1 — use the request-start options snapshot for header redaction (no mid-request reload split) 2026-06-16 21:49:43 -04:00
Joseph Doherty 20760014c2 feat(audit): M5.4 ParentExecutionId tag-cascade for alarm + nested calls (T4) 2026-06-16 21:42:14 -04:00
Joseph Doherty 209f368cb5 feat(audit): M5.2 per-node stuck-count KPIs (T6) — repo per-node aggregation, actor message pair, CentralUI tiles 2026-06-16 21:34:14 -04:00
Joseph Doherty a07ff28f10 feat(audit): M5.3 response-capture increments — request headers, ceiling-hits counter, per-method body opt-out (T7)
1. Request headers in Extra JSON (AuditWriteMiddleware): adds a `requestHeaders`
   object to the existing Extra JSON alongside remoteIp/userAgent; headers whose
   names appear in AuditLogOptions.HeaderRedactList (Authorization, X-Api-Key,
   Cookie, Set-Cookie by default) are replaced with "<redacted>" using
   OrdinalIgnoreCase matching — same policy as ScadaBridgeAuditRedactor.

2. AuditInboundCeilingHits counter: new IAuditInboundCeilingHitsCounter interface
   + NoOpAuditInboundCeilingHitsCounter default; AuditCentralHealthSnapshot
   implements the interface (Interlocked field, thread-safe) and exposes
   AuditInboundCeilingHits on IAuditCentralHealthSnapshot; AddAuditLog registers
   the NoOp default, AddAuditLogCentralMaintenance forwards to the snapshot;
   AuditWriteMiddleware accepts the counter as an optional ctor arg and increments
   it once per request where either the request or response body hit the cap.

3. Per-method SkipBodyCapture opt-out: adds SkipBodyCapture bool to
   PerTargetRedactionOverride; AuditWriteMiddleware consults the per-target
   override map at the start of InvokeAsync (before EnableBuffering) and, when
   set, skips body read + capture entirely — the audit row still emits with
   headers/metadata but null RequestSummary/ResponseSummary; truncation flags
   are also cleared so the ceiling-hits counter is not bumped for opted-out methods.
2026-06-16 21:23:07 -04:00
Joseph Doherty 0569c5ff23 feat(audit): M5.1 audit tree endpoint + CLI audit tree (T8)
Add GET /api/audit/tree endpoint that accepts executionId query param,
authenticates via HTTP Basic + LDAP (OperationalAudit permission), calls
IAuditLogRepository.GetExecutionTreeAsync, and returns a JSON array of
ExecutionTreeNode. Returns 400 for missing/invalid GUID, 401/403 as normal.

Add `scadabridge audit tree --execution-id <guid> [--format table|json]`
CLI subcommand in AuditCommands.Build(). Adds AuditTreeHelpers with:
  - BuildUrl: constructs the relative URL + query string
  - RunTreeAsync: calls the endpoint, dispatches to table or JSON renderer
  - WriteTable: indented ASCII tree (root → children, [*] marks queried node)
  - WriteJson: pretty-printed JSON array pass-through

Tests: 7 new ManagementService endpoint tests (valid id, empty, 400, 401,
403, Viewer allowed, wrong role), 18 new CLI tests (parse, render, HTTP
error codes, JSON output, multi-level indentation, queried-node marker).
2026-06-16 21:20:54 -04:00
Joseph Doherty 0e9bcbb676 docs(m5): implementation plan + tasks for audit hardening T3-T8 2026-06-16 21:12:52 -04:00
Joseph Doherty 1b7600fcb3 docs(m5): design — audit hardening T3-T8 (T1 hash-chain + T2 Parquet stay deferred) 2026-06-16 21:10:21 -04:00
Joseph Doherty e77e209b8a docs(m4): fix bundle CLI example option names in transport design §13 (--output/--input, not --out/positional) 2026-06-16 20:37:51 -04:00
Joseph Doherty dd545281e6 docs(m4.3): reconcile CLI README + Component-CLI to registered commands/options (document bundle group; fix option drift) 2026-06-16 20:30:40 -04:00
Joseph Doherty 0780c2e49e docs(m4.4): clear stale deferred/no-op markers for shipped features (relay, bundle-import audit, M5 redaction, audit drill-in, Transport CLI, traceability)
- SiteCallAudit/ServiceCollectionExtensions.cs: drop "still deferred" note on relay; point to SiteCallAuditActor where it lives
- Transport/Import/BundleImporter.cs: update "Only LoadAsync implemented" to reflect all three phases shipped
- SiteRuntime/Scripts/AuditingDbCommand.cs: replace two M5-deferred redaction comments with accurate references to AuditLogOptions.PerTargetOverrides
- SiteRuntime/Scripts/ScriptRuntimeContext.cs: replace "M5 will layer redaction" note with accurate description of shipped redactor
- CentralUI/AuditLogPage.razor.cs: replace "Bundle C wires… no-op seam" with accurate description of HandleRowSelected implementation
- docs/plans/2026-05-24-transport-design.md §13: update from "CLI Deferred / not built in v1" to reflect shipped BundleCommands.cs; update Open Questions entry
- docs/plans/2026-05-24-transport.md: convert Out-of-Scope "Do NOT build CLI" reminder to a factual note that it shipped
- docs/plans/2026-05-24-transport.md.tasks.json: flip all 30 tasks from pending → done (entire Transport feature shipped)
2026-06-16 20:30:29 -04:00
Joseph Doherty 13605d3dfd docs(m4.1): reconcile Config-DB AuditLog schema + Commons (AuditEvent/ApiKey/SiteCall/NotificationType) to shipped code 2026-06-16 20:29:16 -04:00
Joseph Doherty c3b046457e docs(m4.2): reconcile InboundAPI (Bearer/audit-timing/type-validation), Security (cookie session, role names), Notification (Email-only, AuditKind vocab) to code 2026-06-16 20:27:43 -04:00
Joseph Doherty 9106efafd8 Merge main (DCL alarm fixes 06ef177..9b78e60) into M3 branch 2026-06-16 20:20:27 -04:00
Joseph Doherty fb5f14e04f docs(m3): document Script Analysis component (#25); reconcile consumer specs + README/CLAUDE component list 2026-06-16 20:05:24 -04:00
Joseph Doherty 069757209a fix(scriptanalysis): M3.6 — full-framework analysis refs close forbidden-type-in-allowed-ns blind spot; pin Process/Stopwatch; fix stale codec test; drop dead ContainsInCode 2026-06-16 20:00:28 -04:00
Joseph Doherty 9b78e6071d fix(dcl): identify MxGateway native alarms by object-relative reference
Surface native (Galaxy/MxGateway) alarms by their object-relative reference
(e.g. "Z28061.HeartbeatTimeoutAlarm") instead of the gateway's full provider
reference ("Galaxy!<area>.<object>.<alarm>"). The area is already preserved in
Category and the object reference is globally unique within the galaxy, so the
full provider prefix added only noise to the alarm identity operators see.

MxGatewayAlarmMapper.MapTransition/MapSnapshot now set SourceReference from
SourceObjectReference, falling back to AlarmFullReference only when the gateway
omits the object reference. +2 mapper tests; full DCL suite green (158).
2026-06-16 19:46:44 -04:00
Joseph Doherty cf935d5744 refactor(centralui): M3.5 ScriptAnalysisService uses shared deny-list + delegates trust verdict 2026-06-16 19:40:03 -04:00
Joseph Doherty 64d6ac7288 refactor(siteruntime): M3.3 ValidateTrustModel delegates to shared ScriptAnalysis + compile-surface parity test 2026-06-16 19:37:50 -04:00
Joseph Doherty 14bd25196a feat(templateengine): M3.2 deploy gate delegates to shared ScriptAnalysis (real compile + authoritative forbidden-API) 2026-06-16 19:36:03 -04:00
Joseph Doherty 784fee7b07 refactor(inboundapi): M3.4 ForbiddenApiChecker delegates to shared ScriptAnalysis validator 2026-06-16 19:35:43 -04:00
Joseph Doherty 361e7f41ba fix(dcl): broadcast SnapshotComplete sentinel to all alarm subscribers
The MxGateway alarm mapper emits the SnapshotComplete framing sentinel with
empty SourceReference/SourceObjectReference. HandleAlarmTransitionReceived
routed every transition by prefix match against the subscriber's source, so
the empty-ref sentinel ('' .StartsWith("<src>.") == false) was dropped for
any specific source. The NativeAlarmActor buffers snapshot conditions and only
flushes them on SnapshotComplete, so statically-active native alarms delivered
only in the initial snapshot (no later live transition) never surfaced.

Broadcast the SnapshotComplete sentinel to all alarm subscribers (bypassing the
source match + type filter) so each NativeAlarmActor's snapshot swap completes.
Adds a regression test using the real empty-ref sentinel against a specific
(prefix) source.
2026-06-16 19:33:41 -04:00
Joseph Doherty 069c0e0b1a fix(scriptanalysis): M3.1 review — Pass 2 self-sufficient descent, pin nested-forbidden + nameof cases, drop dead code 2026-06-16 19:29:59 -04:00
Joseph Doherty 4f2b17ce6d feat(scriptanalysis): M3.1 shared trust validator + compiler + compile surfaces + tests 2026-06-16 19:18:39 -04:00
Joseph Doherty 0cc8642cfa docs(m3): implementation plan + tasks for shared ScriptAnalysis consolidation 2026-06-16 19:09:12 -04:00
Joseph Doherty 8e99f22b24 docs(m3): design — shared ScriptAnalysis project consolidating the 4 trust-model analyzers 2026-06-16 19:07:32 -04:00
Joseph Doherty 06ef1779bd fix(dcl): deliver initial-read seed value after subscription registration
DataConnectionActor seeded a tag's initial value by Tell-ing TagValueReceived
from HandleSubscribe's background task, which runs BEFORE HandleSubscribeCompleted
registers the instance's tags in _subscriptionsByInstance. HandleTagValueReceived's
fan-out then found no subscriber and dropped the value. A tag that soon gets a
data-change notification recovers, but a STATIC tag (e.g. an idle MES field that
never changes) was left Uncertain forever — the dropped seed was its only value.

Seeds now ride back on SubscribeCompleted and are delivered after registration,
reusing HandleTagValueReceived's generation guard, fan-out and quality accounting.
+1 regression test (DCL026).
2026-06-16 18:42:28 -04:00
Joseph Doherty 33af948651 Merge feature/native-typed-json: native-typed JSON for List attribute values + data normalization
List values now encode as native-typed JSON ([10,20], [true,false], ISO dates;
strings stay quoted) via AttributeValueCodec; Decode reads both native and the
earlier array-of-strings form for every element type. Already-persisted old-form
data is normalized on the fly: idempotent central startup normalizer
(ListValueNormalizer), active site-SQLite normalization on InstanceActor
override-load, and normalize-on-import in the bundle importer. Instance-override
writes now stamp ElementDataType (#93/M3). Full solution 0/0; feature-targeted
tests green. Plan: docs/plans/2026-06-16-native-typed-json.md.
2026-06-16 18:36:07 -04:00
Joseph Doherty dc9f31537a docs: record final-review follow-ups (deployed-snapshot normalization gap I-1; CLI native-form help example) 2026-06-16 18:34:34 -04:00
Joseph Doherty c53b621b85 docs: mark native-typed JSON feature complete; update Component-Commons codec note
NJ-6: full solution builds 0/0; feature-targeted tests green (Commons codec 38,
TemplateEngine InstanceService 17, ConfigDB normalizer 8, Transport serializer 12,
SiteRuntime InstanceActor 47). Component-Commons now describes the native-typed
List encoding + read-both decode + the three normalization paths. #93/M3 folded in.
2026-06-16 18:27:10 -04:00
Joseph Doherty feeae1371e fix(multivalue): NJ-3/NJ-4/NJ-5 review fixes
- NJ-3: widen per-row catch to Exception (an STJ encode failure can't abort startup); drop dead null-guard already excluded by the SQL filter
- NJ-4: capture logger/instanceName in locals for the fire-and-forget normalize continuation (match the sibling pattern in this actor)
- NJ-5: emit a warn-log when a malformed List value is imported verbatim; thread an optional ILogger<BundleImporter> to the sync re-import site
2026-06-16 18:25:42 -04:00
Joseph Doherty f4b101b532 feat(db): idempotent startup normalizer rewriting List values to native JSON 2026-06-16 17:50:19 -04:00
Joseph Doherty e3d804a1a6 feat(transport): normalize List attribute values to native JSON on import 2026-06-16 17:50:05 -04:00
Joseph Doherty 5841cec958 feat(siteruntime): normalize old-form List static overrides to native JSON on load 2026-06-16 17:49:21 -04:00
Joseph Doherty bf80ca1388 test(commons): NJ-1 review — backward-compat decode tests for old-form Float/DateTime + assert DateTime list is quoted-string array 2026-06-16 17:38:57 -04:00
Joseph Doherty abe8832e9e feat(template): stamp ElementDataType on instance attribute overrides
Set existingOverride.ElementDataType and newOverride.ElementDataType from
templateAttr.ElementDataType in both the update and create branches of
SetAttributeOverrideAsync, so the persisted InstanceAttributeOverride row
always carries the element type for later central normalizer use (#93/M3).
2026-06-16 17:33:15 -04:00
Joseph Doherty 180d55482b feat(commons): native-typed JSON for List values; Decode reads both forms 2026-06-16 17:32:40 -04:00
Joseph Doherty 69f7c526d0 docs: implementation plan for native-typed JSON List values + normalization
6 tasks (NJ-1..NJ-6): native codec + read-both decode; stamp override
ElementDataType (#93/M3); idempotent central startup normalizer; site
override-load normalization; normalize-on-import; integration + docs.
2026-06-16 17:13:14 -04:00
Joseph Doherty d312dfb139 fix(management): honor DisableLogin on the Basic-Auth CLI surfaces
DisableLogin only swapped the cookie auth scheme (AutoLoginAuthenticationHandler),
which covers the interactive UI. The CLI authenticates POST /management, the audit
REST endpoints, and the SignalR debug-stream hub with HTTP Basic, and each ran its
own hardcoded Basic->LDAP check that ignored DisableLogin. In a login-disabled (e.g.
no-LDAP) deployment that locked the CLI out: every call returned 401 AUTH_FAILED.

Add ManagementAuthenticator, which centralizes the management/CLI auth flow:
when ScadaBridge:Security:Auth:DisableLogin is true it synthesizes the same dev
principal as AutoLoginAuthenticationHandler (configured user, all roles, system-wide)
and bypasses Basic->LDAP; otherwise the unchanged Basic->LDAP flow runs. Wired into
ManagementEndpoints (delegates), AuditEndpoints (delegates), and DebugStreamHub
(bypass branch). +6 unit tests; ManagementService.Tests green (140).
2026-06-16 17:12:17 -04:00
Joseph Doherty 91b1aa1275 docs: design for native-typed JSON List attribute values + data normalization
Encode emits native-typed JSON ([10,20], [true,false], ISO dates); Decode reads
both old (array-of-strings) and new forms. Existing data normalized via an
idempotent central MS SQL startup normalizer, active site SQLite normalization in
the InstanceActor override-load path, and normalize-on-import for bundles.
Approved via brainstorming (Approach B, thorough).
2026-06-16 17:08:38 -04:00
Joseph Doherty cdf0a199cb Merge feature/multivalue-attribute: structured multi-value (List) attributes
First-class DataType.List (homogeneous list of a scalar ElementDataType) round-tripping
through authoring, flatten, site runtime, OPC UA read+write, gRPC streaming, validation,
management API, CLI, Transport bundles, and Central UI (TemplateEdit + InstanceConfigure).

Canonical AttributeValueCodec (JSON, invariant culture); in-memory typed List<T> vs
persisted/streamed JSON; idempotent migration; element type fixed by base. 255
feature-targeted tests; full solution builds 0/0. Plan: docs/plans/2026-06-16-multivalue-attribute.md.
2026-06-16 16:51:36 -04:00
Joseph Doherty 94be5e813b fix(siteruntime): decode List value to typed array before DCL write (OPC UA array write path) 2026-06-16 16:48:28 -04:00
Joseph Doherty 734c161383 docs: mark multi-value (List) attribute feature complete; document DataType.List + ElementDataType in Component-Commons
MV-15 integration checkpoint: full solution builds 0/0; feature-targeted tests
green across Commons, TemplateEngine, SiteRuntime, DataConnectionLayer,
Communication, Transport, ManagementService, CLI, CentralUI (255 tests).
2026-06-16 16:34:56 -04:00
Joseph Doherty ca9ee5ea2a fix(ui): MV-14 review — surface SetAttributeOverride failures in InstanceConfigure save loop (no false success toast) 2026-06-16 16:32:28 -04:00