docs(archreview): add architecture-review fix plans (P0 initiative baseline)
This commit is contained in:
Joseph Doherty
@@ -0,0 +1,82 @@
|
||||
# Architecture Review Fix Plans — Master Tracker
|
||||
|
||||
**Created:** 2026-07-08
|
||||
**Source:** [`archreview/00-OVERALL.md`](../00-OVERALL.md) and domain reports 01–08
|
||||
**Plans:** 8 implementation plans, **192 tasks** (191 net of one cross-plan duplicate — see Dedupe Rulings)
|
||||
|
||||
Every plan follows the TDD bite-sized-task format and ships a co-located `.tasks.json` manifest. Execute a plan with the `superpowers-extended-cc:executing-plans` skill (pass the plan path); it reads/updates the `.tasks.json`. **When a plan's status changes, update the table below** — this file is the single source of truth for initiative progress.
|
||||
|
||||
## Status Board
|
||||
|
||||
| Plan | Domain | Tasks | Done | Status | Findings coverage |
|
||||
|------|--------|------:|-----:|--------|-------------------|
|
||||
| [PLAN-01](PLAN-01-cluster-host-failover.md) | Cluster, Host & Failover | 23 | 0 | ⬜ Not started | 28 findings: 20 fixed, 3 → other plans, 3 accepted, 2 roadmap-deferred |
|
||||
| [PLAN-02](PLAN-02-communication-store-and-forward.md) | Communication & S&F | 24 | 0 | ⬜ Not started | 28 findings: 25 fixed, 1 → plan 01, 1 deferred (serializer swap), 1 won't-fix |
|
||||
| [PLAN-03](PLAN-03-site-runtime-dcl.md) | Site Runtime & DCL | 26 | 0 | ⬜ Not started | 28 actionable: 25 fixed, 3 deferred with rationale |
|
||||
| [PLAN-04](PLAN-04-data-audit-backbone.md) | Data & Audit Backbone | 26 | 0 | ⬜ Not started | 27 items: 22 fixed, 4 deferred, 1 won't-fix (incl. NodeB reconcile from report 08) |
|
||||
| [PLAN-05](PLAN-05-templates-deployment-transport.md) | Templates, Deployment & Transport | 25 | 0 | ⬜ Not started | 33 line-items: 27 fixed, 6 deferred with rationale |
|
||||
| [PLAN-06](PLAN-06-edge-integrations.md) | Edge Integrations | 24 | 0 | ⬜ Not started | 24 findings: 21 fixed, 3 deferred/won't-fix |
|
||||
| [PLAN-07](PLAN-07-ui-management-security.md) | UI, Management & Security | 33 | 0 | ⬜ Not started | 24 items: 20 fixed, 3 partial (deferrals documented), 1 won't-fix |
|
||||
| [PLAN-08](PLAN-08-conventions-tests.md) | Conventions & Tests | 11 | 0 | ⬜ Not started | 13 covered, 4 owned by other plans, ~20 triaged into deferred-work register |
|
||||
|
||||
Status values: ⬜ Not started · 🟨 In progress · ✅ Complete · ⏸ Blocked
|
||||
|
||||
## P0 — Do These First (regardless of plan order)
|
||||
|
||||
1. **PLAN-01 Task 1** — enable the SBR downing provider (the system-wide Critical; one line + parsed-config test).
|
||||
2. **PLAN-01 Tasks 3–4** — the two-node kill-test rig (`TwoNodeClusterFixture`); integration-proves Task 1 and later PLAN-02 work.
|
||||
3. **PLAN-07 Tasks 1–3** — `DisableLoginGuard` + scrub `deploy/wonder-app-vd03` (Critical; unauthenticated prod surface).
|
||||
4. **PLAN-02 Tasks 1–2** — ClusterClient generation-suffixed names (Critical crash-loop dropping routing to all sites).
|
||||
5. **PLAN-05 Task 1** — bundle import consumes `BaseTemplateName` (Critical silent data loss).
|
||||
6. **PLAN-08 Task 1** — CLI.Tests into the slnx (protects every subsequent plan's test runs).
|
||||
|
||||
## Cross-Plan Dependency Map
|
||||
|
||||
**Hard-ish sequencing (execute earlier):**
|
||||
- **PLAN-01 Tasks 5–7 → PLAN-02 Tasks 3–4.** PLAN-02's standby S&F delivery gate is a `Func<bool>` seam with marked swap points; it works standalone but its final wiring consumes PLAN-01's `ClusterActivityEvaluator` / `SelfIsPrimary` (singleton-host semantics).
|
||||
- **PLAN-01 Tasks 3–4 → PLAN-02 integration proof.** The two-node rig is how PLAN-02's Critical fixes get integration-verified.
|
||||
- **PLAN-04 Task 13 (additive `RequestedBy` message change) before any PLAN-07 sender refactors** touching the same messages.
|
||||
|
||||
**Soft seams (either order works, coordinate):**
|
||||
- **PLAN-05 Task 14 (`IScriptArtifactChangeBus`) ↔ PLAN-06 Task 3 (`InboundScriptExecutor.InvalidateMethod`).** PLAN-06 Task 1's per-request revision check is the designed correctness fallback, so there is no hard sequencing; when both land, wire the bus to the invalidate seam.
|
||||
- **PLAN-03's `DeployCompileValidator` should adopt PLAN-05's compile-cache abstraction** once it exists (later refactor, not a blocker).
|
||||
- **PLAN-03 Task 8's stuck-thread watchdog pattern is reusable by PLAN-06 Task 4** (inbound orphan counter).
|
||||
|
||||
**EF Core migrations — serialize globally.** Migration-adding tasks exist in PLAN-04 (Tasks 6, 8), PLAN-06 (Tasks 8, 19), and possibly PLAN-07 (secured-write expiry). All share one model snapshot: **never run two migration tasks concurrently**, and always build first (never `--no-build` on `dotnet ef migrations add` — known empty-scaffold gotcha).
|
||||
|
||||
**Shared-file merge surfaces (don't run these plans' overlapping tasks concurrently):**
|
||||
| File / area | Plans touching it |
|
||||
|---|---|
|
||||
| `ManagementActor.cs` | PLAN-07 (15 tasks, serialized within plan), PLAN-05 (management-path publish), PLAN-06 (method-update paths) |
|
||||
| `deploy/wonder-app-vd03/appsettings.Central.json` | PLAN-07 (auth), PLAN-01 (Cluster/Node sections) |
|
||||
| `NotificationOutboxActor.cs` | PLAN-04, PLAN-06 (SMS retry policy) |
|
||||
| `StoreAndForwardService.cs` / site tracking store | PLAN-02 (primary owner), PLAN-04 Task 15 (keyset query) |
|
||||
| `Host/Program.cs` + service registration | PLAN-01, PLAN-08 (options validation) |
|
||||
| `SiteHealthReport` (Commons record, additive fields) | PLAN-03 Task 6, PLAN-04 health-event fields |
|
||||
| Transport `BundleImporter.cs` | PLAN-05 (primary, serialized chain), PLAN-07 (import idempotency note) |
|
||||
|
||||
## Dedupe Rulings (one owner per finding)
|
||||
|
||||
| Finding | Owner | Others must skip |
|
||||
|---|---|---|
|
||||
| CLI.Tests into slnx | **PLAN-08 Task 1** (also retires CLAUDE.md gotcha + memory note) | PLAN-07 Task 33 — **skip as duplicate**; mark `skipped-duplicate` in its tasks.json |
|
||||
| `DisableLogin: true` artifact | **PLAN-07 Tasks 1–3** | PLAN-01 references only |
|
||||
| AuditLog reconciliation NodeB failover | **PLAN-04 Task 18** | PLAN-08 pointer only |
|
||||
| `SmsConfiguration.MaxRetryCount` unused | **PLAN-06 Task 18** | PLAN-08 pointer only |
|
||||
| Transport `AreaName: null` | **PLAN-05 Task 7** | PLAN-08 pointer only |
|
||||
| Role-casing normalization | **PLAN-07 Task 10 chain** | PLAN-08 pointer only |
|
||||
| Two-node failover rig | **PLAN-01 Tasks 3–4** | PLAN-08 Task 8 is a documented placeholder; PLAN-02 consumes |
|
||||
| Script-artifact invalidation contract | **PLAN-05 Task 14** (design + publisher) | PLAN-06 implements the inbound consumer |
|
||||
|
||||
## Recommended Execution Waves
|
||||
|
||||
- **Wave 0 (P0 list above).** Independent files; all six items can run in parallel. ~1 focused session.
|
||||
- **Wave 1 — stability spine.** Rest of PLAN-01 (active-node semantics, drains); PLAN-02 Tasks 3–9 (standby gate, gRPC channel keying, sweep bounding); PLAN-03 Tasks 1–4 (reconnect leak, compile gate); PLAN-04 Tasks 1–5 (purge chains); PLAN-05 import chain Tasks 2–6; PLAN-06 Tasks 1–4 + 8; PLAN-07 Tasks 4–9 (deploy scoping, secret elision).
|
||||
- **Wave 2 — remainder.** All other tasks per intra-plan `blockedBy` edges; migration tasks serialized globally; docs-update tasks land with their code tasks (repo rule: doc + code + tests together).
|
||||
- **Wave 3 — closeout.** PLAN-08 Task 11 (deferred-work register), each plan's doc-sweep task, then a full `dotnet build ZB.MOM.WW.ScadaBridge.slnx` + `dotnet test` (which now includes CLI.Tests) + `bash docker/deploy.sh` + the two-node drill from PLAN-01.
|
||||
|
||||
**Execution hygiene:** work in a dedicated worktree (not the primary checkout); one plan per executor session where possible; for parallel implementers use pathspec-scoped commits and verify every commit landed on HEAD afterward (both are known failure modes in this repo's history). Re-verify a finding's line numbers before editing — reports and plans cite the code as of 2026-07-08.
|
||||
|
||||
## Deferred / Won't-Fix Registry (initiative-level)
|
||||
|
||||
Consolidated in **PLAN-08 Task 11** → `docs/plans/2026-07-08-deferred-work-register.md` (triages the 23-item inventory from report 08 plus each plan's coverage-table deferrals: serializer swap, hash-chain/Parquet, aggregated live alarm stream, central cert-trust persistence, runtime script sandbox, clustered-GUID benchmark, TLS profile, Dockerfile HEALTHCHECK, etc.). Nothing from the review is silently dropped: every report finding is either a task, an ownership pointer, or a registered deferral with rationale.
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,29 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-01-cluster-host-failover.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: Enable the SBR downing provider in BuildHocon", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: Raise the cluster-leave CoordinatedShutdown phase timeout above the drain budget", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 3, "subject": "Task 3: Two-node in-process cluster fixture (the reusable failover rig)", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 4, "subject": "Task 4: SBR behavioral proof — kill the oldest node, assert downing + singleton migration", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 5, "subject": "Task 5: ClusterActivityEvaluator — single oldest-member \"active node\" definition", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 6, "subject": "Task 6: Rewire ActiveNodeGate, AkkaClusterNodeProvider and Primary labels onto the evaluator", "status": "pending", "blockedBy": [5] },
|
||||
{ "id": 7, "subject": "Task 7: Oldest-based /health/active check + require DB reachability for Active", "status": "pending", "blockedBy": [5] },
|
||||
{ "id": 8, "subject": "Task 8: Divergence proof — restarted first node must NOT become active again", "status": "pending", "blockedBy": [3, 6] },
|
||||
{ "id": 9, "subject": "Task 9: CentralSingletonRegistrar helper (manager + drain task + proxy)", "status": "pending", "blockedBy": [2] },
|
||||
{ "id": 10, "subject": "Task 10: Route all seven central singletons through the registrar (adds missing outbox/ingest drains)", "status": "pending", "blockedBy": [9] },
|
||||
{ "id": 11, "subject": "Task 11: SiteHealthReportAck message + ack semantics in the communication actors", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 12, "subject": "Task 12: Acked async health-report transport so counter-restore actually fires", "status": "pending", "blockedBy": [11] },
|
||||
{ "id": 13, "subject": "Task 13: Metrics-staleness signal + status-transition timestamps in the aggregator", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 14, "subject": "Task 14: Surface metrics-stale + offline-since on the Health dashboard", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 15, "subject": "Task 15: Fix the metrics-port seed in appsettings.Site.json + close the validator gap", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 16, "subject": "Task 16: Single-node installs — explicit AllowSingleNodeCluster instead of phantom seeds", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 17, "subject": "Task 17: stop_grace_period for all app containers (both topologies)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 18, "subject": "Task 18: Evict deleted sites from the health aggregator", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 19, "subject": "Task 19: Rate-limit dead-letter warnings", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 20, "subject": "Task 20: Down-if-alone recovery — exit on unexpected ActorSystem termination + service restart actions", "status": "pending", "blockedBy": [10] },
|
||||
{ "id": 21, "subject": "Task 21: Docker failover drill script (manual/CI kill test against the real topology)", "status": "pending", "blockedBy": [1, 17] },
|
||||
{ "id": 22, "subject": "Task 22: Reconcile REQ-HOST-6 (Akka.Hosting) with the hand-rolled bootstrap", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 23, "subject": "Task 23: Docs & deploy-config cleanup batch (Traefik doc, NodeName overlays, TLS roadmap note)", "status": "pending", "blockedBy": [7, 16] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-02-communication-store-and-forward.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: Generation-suffixed, sanitized ClusterClient actor names", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: Guard client creation in the refresh loop + real-factory address-edit lifecycle test", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 3, "subject": "Task 3: Active-node delivery gate on the S&F retry sweep", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 4, "subject": "Task 4: Wire the delivery gate in the Host + align the S&F design doc", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 5, "subject": "Task 5: Key the gRPC client cache by (site, endpoint)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 6, "subject": "Task 6: Endpoint-safe unsubscribe in DebugStreamBridgeActor + concurrent-session regression test", "status": "pending", "blockedBy": [5] },
|
||||
{ "id": 7, "subject": "Task 7: Bound the retry sweep with a batch LIMIT", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 8, "subject": "Task 8: Per-target short-circuit in the retry sweep", "status": "pending", "blockedBy": [7] },
|
||||
{ "id": 9, "subject": "Task 9: Parallel per-target lanes with a small concurrency cap", "status": "pending", "blockedBy": [8] },
|
||||
{ "id": 10, "subject": "Task 10: Ordered, observable replication dispatch", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 11, "subject": "Task 11: Upsert-based standby applies for Add/Park/Requeue", "status": "pending", "blockedBy": [7, 10] },
|
||||
{ "id": 12, "subject": "Task 12: deferToSweep enqueue mode + immediate sweep kick", "status": "pending", "blockedBy": [9] },
|
||||
{ "id": 13, "subject": "Task 13: Unstrand and unblock Notify.Send (maxRetries 0 + deferToSweep) + docs", "status": "pending", "blockedBy": [12] },
|
||||
{ "id": 14, "subject": "Task 14: Park (don't silently drop) corrupt buffered notifications", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 15, "subject": "Task 15: Wire-format pin tests for cross-cluster contracts", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 16, "subject": "Task 16: Replicate heartbeat marks to the peer central node", "status": "pending", "blockedBy": [2] },
|
||||
{ "id": 17, "subject": "Task 17: S&F SQLite hygiene — WAL + busy_timeout", "status": "pending", "blockedBy": [11] },
|
||||
{ "id": 18, "subject": "Task 18: Epoch-ms due-check (replace per-row julianday parsing)", "status": "pending", "blockedBy": [17] },
|
||||
{ "id": 19, "subject": "Task 19: Decouple audit-observer notification from the sweep (ordered channel)", "status": "pending", "blockedBy": [12] },
|
||||
{ "id": 20, "subject": "Task 20: Buffer resync storage primitives (GetAllMessagesAsync / ReplaceAllAsync)", "status": "pending", "blockedBy": [18] },
|
||||
{ "id": 21, "subject": "Task 21: Peer-join S&F buffer resync (anti-entropy)", "status": "pending", "blockedBy": [10, 20] },
|
||||
{ "id": 22, "subject": "Task 22: Conventions & observability cleanup batch (Lows)", "status": "pending", "blockedBy": [16] },
|
||||
{ "id": 23, "subject": "Task 23: Oldest-parked-age health metric (parked-retention aging signal)", "status": "pending", "blockedBy": [20] },
|
||||
{ "id": 24, "subject": "Task 24: Pin the reconciliation cursor boundary contract (>= at the site read)", "status": "pending", "blockedBy": [] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-03-site-runtime-dcl.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: OpcUaDataConnection — dispose + detach the previous client on reconnect (S1)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: MxGatewayDataConnection — dispose previous client + cancel previous event loop on reconnect (S1)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 3, "subject": "Task 3: Pure deploy-time compile validator for a flattened config (S3 groundwork)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 4, "subject": "Task 4: Reject the deployment on site-side compile failure per spec (S3)", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 5, "subject": "Task 5: ScriptExecutionScheduler gauges — queue depth, busy threads, oldest-busy age (S2/UA5)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 6, "subject": "Task 6: Surface scheduler stats through ISiteHealthCollector + SiteHealthReport (S2/UA5)", "status": "pending", "blockedBy": [5] },
|
||||
{ "id": 7, "subject": "Task 7: Periodic scheduler-stats reporter hosted service (S2/UA5)", "status": "pending", "blockedBy": [5, 6] },
|
||||
{ "id": 8, "subject": "Task 8: Stuck-script watchdog — name the script whose thread never came back (S2)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 9, "subject": "Task 9: ScriptActor — trigger-expression evaluation off the dispatcher, coalesced (P1)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 10, "subject": "Task 10: AlarmActor — async trigger-expression evaluation on the script scheduler (P1)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 11, "subject": "Task 11: InstanceActor — retry failed/lost tag subscriptions per connection (S4/UA6)", "status": "pending", "blockedBy": [4, 8] },
|
||||
{ "id": 12, "subject": "Task 12: NativeAlarmActor — retry on lost subscribe response (S4)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 13, "subject": "Task 13: DeploymentManager — retry the startup config load (S5)", "status": "pending", "blockedBy": [4] },
|
||||
{ "id": 14, "subject": "Task 14: Watch all Instance Actors; clean the map on unexpected termination (S6)", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 15, "subject": "Task 15: Fail the in-flight deployment when its Instance Actor dies during init (S6)", "status": "pending", "blockedBy": [14] },
|
||||
{ "id": 16, "subject": "Task 16: DataConnectionActor — capture _adapter into locals before background tasks (S7)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 17, "subject": "Task 17: DCL — tag→instances reverse index for the value fan-out hot path (P3)", "status": "pending", "blockedBy": [16] },
|
||||
{ "id": 18, "subject": "Task 18: InstanceActor — per-attribute child-subscription routing (P2)", "status": "pending", "blockedBy": [9, 11] },
|
||||
{ "id": 19, "subject": "Task 19: SiteStorageService — WAL journal mode + busy timeout (S8)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 20, "subject": "Task 20: NativeAlarmActor — coalesce per-transition SQLite upserts (P4)", "status": "pending", "blockedBy": [12, 19] },
|
||||
{ "id": 21, "subject": "Task 21: Persist native-alarm display metadata for rehydration (UA4)", "status": "pending", "blockedBy": [20] },
|
||||
{ "id": 22, "subject": "Task 22: CertStoreActor — export trusted certs (thumbprint + DER) for reconciliation (UA1 groundwork)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 23, "subject": "Task 23: Cert-trust reconcile-on-join — singleton pushes trusted set to (re)joining site node (UA1)", "status": "pending", "blockedBy": [15, 22] },
|
||||
{ "id": 24, "subject": "Task 24: Wire ConfigFetchRetryCount into the standby replication fetch (UA2)", "status": "pending", "blockedBy": [11] },
|
||||
{ "id": 25, "subject": "Task 25: Low-severity code cleanups — S9, S10, C2, C6", "status": "pending", "blockedBy": [9, 17] },
|
||||
{ "id": 26, "subject": "Task 26: Design-doc updates — C3, C4, C5 + behavior notes for landed changes", "status": "pending", "blockedBy": [1, 2, 4, 7, 8, 11, 13, 15, 18, 21, 23, 24, 25] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
@@ -0,0 +1,774 @@
|
||||
# Data & Audit Backbone Fixes Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Defuse the scale-dependent operational time bombs in the central data/audit layer (missing site SQLite retention purge, partition-switch timeout self-lock, unindexed KPI scans, table-wide execution-tree scans, hardcoded-zero backlog KPI) and close the audit-integrity/consistency gaps (append-only role drift, frozen SiteCalls progress fields, missing operator-Retry audit rows, reconciliation dead-ends, NodeA-only reconciliation) identified in `archreview/04-data-audit-backbone.md`.
|
||||
|
||||
**Architecture:** All fixes stay inside the existing component boundaries: site-side purge lives in the AuditLog component's site half (SqliteAuditWriter + a new hosted retention job), central maintenance hardening lives in AuditLogRepository/AuditLogPurgeActor, index/DDL changes ride EF Core migrations in ConfigurationDatabase (with idempotent SQL scripts for production), and contract changes (keyset cursor, RequestedBy, SiteEntry fallback endpoint) are strictly additive per the repo's message-evolution rule. Design docs are updated in the same task as the code they describe (repo rule: doc + code + tests travel together).
|
||||
|
||||
**Tech Stack:** C#/.NET 8, Akka.NET (actors/TestKit), EF Core + SQL Server (central), Microsoft.Data.Sqlite (site), xUnit. Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per-project: `dotnet test tests/<project> --filter <name>`. MSSQL-backed integration tests need `cd infra && docker compose up -d`. **EF migrations gotcha (repo-documented):** always build first and run `dotnet ef migrations add <Name>` WITHOUT `--no-build`, or it scaffolds an empty migration off the stale DLL.
|
||||
|
||||
## Findings Coverage
|
||||
|
||||
| # | Report finding (severity) | Task(s) |
|
||||
|---|---------------------------|---------|
|
||||
| S1 | Site SQLite 7-day retention purge missing (High) | 1, 2, 3 |
|
||||
| S2 | `SwitchOutPartitionAsync` no CommandTimeout / log-only failure (High) | 4, 5 |
|
||||
| S3 | Append-only DB-role enforcement DDL-only; purger grants insufficient (High) | 6, 7 |
|
||||
| S4 | SiteCalls upsert freezes RetryCount/LastError at rank (Medium) | 11 |
|
||||
| S5 | No EF `EnableRetryOnFailure` (Medium) | 12 |
|
||||
| S6 | Operator Retry on parked notification emits no audit row (+SiteCalls relay identity) (Medium) | 13, 14 |
|
||||
| S7 | Reconciliation edge cases dead-ended (cursor pin; permanent abandonment) (Medium) | 15, 16, 17 |
|
||||
| S8 | NotificationOutboxRepository dual SQLite/T-SQL dialect — provider parity (Medium) | 24 (documented convention decision: MSSQL-only SQL + MSSQL fixtures is the norm for new repositories; existing dual-dialect kept as a documented legacy exception — rewriting the shipped repository + its whole test suite is not worth the churn) |
|
||||
| S9 | Purge actors' first tick waits a full interval (Low) | 22 |
|
||||
| S10 | `SqliteAuditWriter.Dispose` sync-over-async (Low) | Won't-fix — report itself notes the thread-pool-hop mitigation is correct and documented; flagged-only |
|
||||
| P1 | SiteCalls KPI predicates full-scan — no `TerminalAtUtc` index (High) | 8 |
|
||||
| P2 | `GetExecutionTreeAsync` scans entire AuditLog (High) | 9 |
|
||||
| P3 | AuditLog clustered key leads with random GUID (Medium) | Deferred — report says "worth a deliberate benchmark before the table gets big"; changing it is a full-table rebuild. Task 24 records it as a tracked design decision in Component-ConfigurationDatabase.md so it is no longer untracked |
|
||||
| P4 | KpiSample volume + unbatched purge (Medium) | 19 (batched purge); per-node sampling cadence recorded as tracked follow-up in 24 |
|
||||
| P5 | Catch-all `(@p IS NULL OR col=@p)` predicates in SiteCalls query (Medium) | 21 |
|
||||
| P6 | Outbox offset paging + double query + LIKE; KPI 5-7 round trips; unbounded oldest materialization (Medium) | 20 (KPI single query + bounded oldest); offset→keyset conversion Deferred — it changes the page/TotalCount UI contract for a table whose live-queue portion stays small under Task 19/existing purge; recorded in 24 |
|
||||
| P7 | Non-persisted `IngestedAtUtc` computed column re-parse (Low) | 23 (predicate-ban comment guard) |
|
||||
| P8 | `MarkForwardedAsync`/`MarkReconciledAsync` per-id IN lists near SQLite param limit (Low) | 23 |
|
||||
| C1 | Design docs drifted (enum lists, Teams, per-channel overrides, SiteCalls schema) (Medium) | 24 |
|
||||
| C2 | `AuditLogRow` entity lives in ConfigurationDatabase not Commons (Medium) | 24 (document the deviation — the rationale in the report is accepted: contract type is external `ZB.MOM.WW.Audit.AuditEvent`, row is a persistence-only projection) |
|
||||
| C3 | Mixed timestamp CLR types across sibling tables (Low) | 24 (convention note: new tables use `DateTime`+Utc converter) |
|
||||
| C4 | Uncharted KPI metric names are string literals (Low) | 23 |
|
||||
| U1 | `backlogTotal` KPI history permanently zero | 10 |
|
||||
| U2 | Site SQLite retention purge | 1, 2, 3 |
|
||||
| U3 | Hash-chain tamper evidence / Parquet archival | No action — explicitly deferred to v1.x per Component-AuditLog.md; report confirms "consistent with plan; no drift" |
|
||||
| U4 | SecuredWrite audit rows leave `SourceNode` NULL; doc under-promises per-channel keys | SourceNode fix Deferred — already a logged follow-up in Component-AuditLog.md:154-157 and the SecuredWrite emission path belongs to plan 07's domain. Doc under-promise fixed in 24 |
|
||||
| U5 | Reconciliation cursor keyset upgrade untracked | 15, 16 (SiteCalls keyset implemented); AuditLog-pull keyset recorded as tracked follow-up in Component-AuditLog.md (24) |
|
||||
| U6 | Dispatcher throughput ceiling (sequential delivery) | 25 |
|
||||
| U7 | Test-coverage gaps mirror findings | Covered inside tasks 1, 4, 10 (the report's three named gaps each get a test) |
|
||||
| X1 | AuditLog reconciliation dials site NodeA only (flagged in report 08, owned here) | 18 |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Site SQLite retention purge — `PurgeExpiredAsync` on the writer
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 4, 6, 8, 11, 12, 19, 20, 21
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Services/ISiteAuditQueue.cs` (add method)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs` (implement; the `auto_vacuum = INCREMENTAL` pragma at ~line 149 already anticipates this)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterRetentionTests.cs` (create)
|
||||
|
||||
1. Write the failing test (pattern-match the fixture setup from `SqliteAuditWriterBacklogStatsTests.cs`):
|
||||
|
||||
```csharp
|
||||
public class SqliteAuditWriterRetentionTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task PurgeExpired_DeletesForwardedAndReconciled_OlderThanCutoff()
|
||||
{
|
||||
await using var writer = CreateWriter(); // same in-memory/temp-file helper the sibling tests use
|
||||
var oldForwarded = await WriteEventAsync(writer, occurredAtUtc: DateTime.UtcNow.AddDays(-10));
|
||||
var oldReconciled = await WriteEventAsync(writer, occurredAtUtc: DateTime.UtcNow.AddDays(-10));
|
||||
var oldPending = await WriteEventAsync(writer, occurredAtUtc: DateTime.UtcNow.AddDays(-10));
|
||||
var fresh = await WriteEventAsync(writer, occurredAtUtc: DateTime.UtcNow);
|
||||
await writer.MarkForwardedAsync(new[] { oldForwarded, oldReconciled, fresh });
|
||||
await writer.MarkReconciledAsync(new[] { oldReconciled });
|
||||
|
||||
var purged = await writer.PurgeExpiredAsync(DateTime.UtcNow.AddDays(-7));
|
||||
|
||||
Assert.Equal(2, purged); // oldForwarded + oldReconciled
|
||||
// Hard ForwardState invariant: the old *Pending* row survives on age alone.
|
||||
var pending = await writer.ReadPendingAsync(10);
|
||||
Assert.Contains(pending, e => e.EventId == oldPending);
|
||||
// Fresh forwarded row inside the window survives.
|
||||
var stats = await writer.GetBacklogStatsAsync();
|
||||
Assert.Equal(1, stats.PendingCount);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task PurgeExpired_IsIdempotent()
|
||||
{
|
||||
await using var writer = CreateWriter();
|
||||
var id = await WriteEventAsync(writer, occurredAtUtc: DateTime.UtcNow.AddDays(-10));
|
||||
await writer.MarkForwardedAsync(new[] { id });
|
||||
Assert.Equal(1, await writer.PurgeExpiredAsync(DateTime.UtcNow.AddDays(-7)));
|
||||
Assert.Equal(0, await writer.PurgeExpiredAsync(DateTime.UtcNow.AddDays(-7)));
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter SqliteAuditWriterRetentionTests` — expect FAIL (method does not exist).
|
||||
3. Add to `ISiteAuditQueue`: `Task<int> PurgeExpiredAsync(DateTime olderThanUtc, CancellationToken ct = default);` with XML doc stating the hard invariant (`Pending` rows are never purged on age alone — only `Forwarded`/`Reconciled`). Implement in `SqliteAuditWriter` under `_writeLock` (mirror `MarkForwardedAsync`'s lock + `ObjectDisposedException.ThrowIf` shape), one transaction:
|
||||
|
||||
```sql
|
||||
CREATE TEMP TABLE IF NOT EXISTS purge_ids AS
|
||||
SELECT EventId FROM audit_forward_state
|
||||
WHERE ForwardState IN ('Forwarded','Reconciled') AND OccurredAtUtc < $cutoff;
|
||||
DELETE FROM audit_forward_state WHERE EventId IN (SELECT EventId FROM purge_ids);
|
||||
DELETE FROM audit_event WHERE EventId IN (SELECT EventId FROM purge_ids);
|
||||
DROP TABLE purge_ids;
|
||||
```
|
||||
|
||||
Delete the sidecar first (it is the FK child of `audit_event`). `$cutoff` uses the same round-trip `"o"` timestamp format the writer already uses. Return `changes()` from the `audit_event` DELETE. After COMMIT, run `PRAGMA incremental_vacuum;` (the pragma the ctor set up — reference the existing comment at `SqliteAuditWriter.cs:145` and update it: the purge now exists).
|
||||
4. Run the filter again — expect PASS. Also run the whole site suite: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter "FullyQualifiedName~Site"` — expect PASS.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Services/ISiteAuditQueue.cs src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterRetentionTests.cs && git commit -m "feat(audit-log): add site SQLite PurgeExpiredAsync honoring the ForwardState invariant"`
|
||||
|
||||
### Task 2: Site retention options
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1, 4, 6, 8, 11, 12, 19, 20, 21
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SiteAuditRetentionOptions.cs`
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SiteAuditRetentionOptionsTests.cs` (create)
|
||||
|
||||
1. Failing test:
|
||||
|
||||
```csharp
|
||||
public class SiteAuditRetentionOptionsTests
|
||||
{
|
||||
[Theory]
|
||||
[InlineData(0, 1)] // clamp floor
|
||||
[InlineData(7, 7)] // default passthrough
|
||||
[InlineData(365, 90)] // clamp ceiling (spec: min 1, max 90)
|
||||
public void ResolvedRetentionDays_Clamps(int configured, int expected)
|
||||
=> Assert.Equal(expected, new SiteAuditRetentionOptions { RetentionDays = configured }.ResolvedRetentionDays);
|
||||
|
||||
[Fact]
|
||||
public void Defaults_Are_SevenDays_DailyPurge_FiveMinuteInitialDelay()
|
||||
{
|
||||
var o = new SiteAuditRetentionOptions();
|
||||
Assert.Equal(7, o.ResolvedRetentionDays);
|
||||
Assert.Equal(TimeSpan.FromHours(24), o.ResolvedPurgeInterval);
|
||||
Assert.Equal(TimeSpan.FromMinutes(5), o.InitialDelay);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter SiteAuditRetentionOptionsTests` — FAIL.
|
||||
3. Implement the options class following the `SiteCallAuditOptions.ResolvedReconciliationInterval` clamp pattern: `RetentionDays` (default 7; `ResolvedRetentionDays` clamps to `[1, 90]`), `PurgeInterval` (default 24h; `ResolvedPurgeInterval` clamps to min 1 minute — Akka zero-interval spin footgun), `InitialDelay` (default 5 min — short so a daily-restarting node still purges). Config section: `ScadaBridge:AuditLog:SiteRetention`.
|
||||
4. Run filter — PASS.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SiteAuditRetentionOptions.cs tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SiteAuditRetentionOptionsTests.cs && git commit -m "feat(audit-log): SiteAuditRetentionOptions (7-day default, clamped 1-90)"`
|
||||
|
||||
### Task 3: Site retention hosted job + registration + design doc
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 4, 6, 8, 11, 12, 19, 20, 21
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SiteAuditRetentionService.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/ServiceCollectionExtensions.cs` (site registration path, near the `ISiteAuditQueue` registration at ~line 111)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (~lines 412-414, the "Sites: daily site job" bullet)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SiteAuditRetentionServiceTests.cs` (create)
|
||||
|
||||
1. Failing test (fake `ISiteAuditQueue`, short intervals):
|
||||
|
||||
```csharp
|
||||
public class SiteAuditRetentionServiceTests
|
||||
{
|
||||
[Fact]
|
||||
public async Task Tick_Purges_With_RetentionCutoff()
|
||||
{
|
||||
var queue = new RecordingSiteAuditQueue(); // stub: records PurgeExpiredAsync calls
|
||||
var options = Options.Create(new SiteAuditRetentionOptions
|
||||
{ RetentionDays = 7, PurgeInterval = TimeSpan.FromMilliseconds(50), InitialDelay = TimeSpan.Zero });
|
||||
using var svc = new SiteAuditRetentionService(queue, options, NullLogger<SiteAuditRetentionService>.Instance);
|
||||
await svc.StartAsync(CancellationToken.None);
|
||||
await WaitUntilAsync(() => queue.PurgeCalls.Count >= 1, TimeSpan.FromSeconds(5));
|
||||
await svc.StopAsync(CancellationToken.None);
|
||||
var cutoff = queue.PurgeCalls[0];
|
||||
Assert.InRange(cutoff, DateTime.UtcNow.AddDays(-7).AddMinutes(-1), DateTime.UtcNow.AddDays(-7).AddMinutes(1));
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Tick_SwallowsExceptions_AndKeepsTicking()
|
||||
{
|
||||
var queue = new RecordingSiteAuditQueue { ThrowOnFirstCall = true };
|
||||
// ... start, wait for >= 2 calls, assert no unobserved exception
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter SiteAuditRetentionServiceTests` — FAIL.
|
||||
3. Implement `SiteAuditRetentionService : IHostedService, IDisposable` mirroring `SiteAuditBacklogReporter.cs` (same file layout: `System.Threading.Timer`, `InitialDelay` as due time, `ResolvedPurgeInterval` as period, per-tick try/catch that logs and continues, purge count logged at Information). Register in the site path of `ServiceCollectionExtensions`: bind `SiteAuditRetentionOptions` from `ScadaBridge:AuditLog:SiteRetention` and `services.AddHostedService<SiteAuditRetentionService>();`.
|
||||
4. Run filter — PASS. Then `dotnet build ZB.MOM.WW.ScadaBridge.slnx` — expect clean.
|
||||
5. Update `Component-AuditLog.md` ~412-414: replace the aspirational bullet with the shipped mechanism — `SiteAuditRetentionService` hosted job, config keys (`ScadaBridge:AuditLog:SiteRetention:RetentionDays|PurgeInterval`), the `PurgeExpiredAsync` invariant, and the post-purge `incremental_vacuum`.
|
||||
6. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site docs/requirements/Component-AuditLog.md && git commit -m "feat(audit-log): daily site SQLite retention purge job (closes unbounded site DB growth)"`
|
||||
|
||||
### Task 4: Maintenance command timeout on partition switch + per-channel purge
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 8 (different files), 11, 20
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeOptions.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IAuditLogRepository.cs` (additive optional parameter)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs` (`SwitchOutPartitionAsync` ~257-371, `PurgeChannelOlderThanAsync` ~374-445)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeActor.cs` (pass the timeout)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/AuditLogPurgeActorTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Integration/PartitionPurgeTests.cs` (extend, MSSQL)
|
||||
|
||||
1. Failing unit test in `AuditLogPurgeActorTests` (these tests already drive the actor against a mocked `IAuditLogRepository`): assert the actor calls `SwitchOutPartitionAsync(boundary, commandTimeout)` with the options value:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task PurgeTick_Passes_MaintenanceCommandTimeout_To_Repository()
|
||||
{
|
||||
// options: MaintenanceCommandTimeoutMinutes = 45
|
||||
// mock repo: capture the TimeSpan? argument
|
||||
// drive one tick; Assert.Equal(TimeSpan.FromMinutes(45), captured);
|
||||
}
|
||||
```
|
||||
|
||||
Plus an options test: `MaintenanceCommandTimeoutMinutes` default 30, clamped to min 1 (`ResolvedMaintenanceCommandTimeout`).
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter AuditLogPurgeActorTests` — FAIL (no such parameter).
|
||||
3. Implementation:
|
||||
- `AuditLogPurgeOptions`: add `public int MaintenanceCommandTimeoutMinutes { get; set; } = 30;` + `ResolvedMaintenanceCommandTimeout` clamp (min 1 minute), XML-doc'd against the 30s-default-timeout failure mode from the review.
|
||||
- `IAuditLogRepository`: `Task<long> SwitchOutPartitionAsync(DateTime monthBoundary, TimeSpan? commandTimeout = null, CancellationToken ct = default);` and `PurgeChannelOlderThanAsync(..., TimeSpan? commandTimeout = null, ...)` — additive defaults, existing callers compile unchanged.
|
||||
- `AuditLogRepository.SwitchOutPartitionAsync`: when `commandTimeout` is non-null, set `sampleCmd.CommandTimeout = (int)commandTimeout.Value.TotalSeconds;` and wrap the `ExecuteSqlRawAsync` with `_context.Database.SetCommandTimeout(commandTimeout)` (the context is scoped per purge tick — see `AuditLogPurgeActor.OnTickAsync`'s `CreateAsyncScope` — so no restore is needed, but restore the previous value in a `finally` anyway for hygiene). Same `cmd.CommandTimeout` treatment on the per-channel `DELETE TOP` command.
|
||||
- `AuditLogPurgeActor.OnTickAsync`: pass `_purgeOptions.ResolvedMaintenanceCommandTimeout` to both repository calls.
|
||||
4. Run the actor filter — PASS. Extend `PartitionPurgeTests` (MSSQL; `cd infra && docker compose up -d` first) with one case that calls `SwitchOutPartitionAsync(boundary, TimeSpan.FromMinutes(30))` and asserts success — proving the timeout path executes against real SQL Server: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter PartitionPurgeTests`.
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests && git commit -m "fix(audit-log): explicit long CommandTimeout on partition-switch and per-channel purge maintenance paths"`
|
||||
|
||||
### Task 5: Purge-failure health event (not just a log line)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 8, 11, 19, 20, 21
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeFailedEvent.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeActor.cs` (catch block ~198-206 and the boundary-enumeration catch)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditCentralHealthSnapshot.cs` + `IAuditCentralHealthSnapshot.cs` (add `PurgeFailures` counter, mirroring the existing counter members)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (retention section: purge failure is now a health metric)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/AuditLogPurgeActorTests.cs` (extend)
|
||||
|
||||
1. Failing test: mock repository whose `SwitchOutPartitionAsync` throws; subscribe a TestProbe to `AuditLogPurgeFailedEvent` on the EventStream (same pattern the existing tests use for `AuditLogPurgedEvent`); drive a tick; expect one `AuditLogPurgeFailedEvent(boundary, errorMessage, elapsedMs)` and assert the health snapshot's `PurgeFailures` incremented.
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter AuditLogPurgeActorTests` — FAIL.
|
||||
3. Implement: `public sealed record AuditLogPurgeFailedEvent(DateTime MonthBoundary, string Error, long ElapsedMilliseconds);` — publish it in the per-boundary catch (alongside the existing `LogError`) using the already-captured `eventStream`; increment the new snapshot counter (inject `IAuditCentralHealthSnapshot` the same way sibling actors take their counters — follow the existing ctor DI shape). The success event (`AuditLogPurgedEvent`) already exists; this is its symmetric failure twin.
|
||||
4. Run filter — PASS.
|
||||
5. Doc: add to Component-AuditLog.md retention section: "Purge failure publishes `AuditLogPurgeFailedEvent` and increments the `PurgeFailures` central health counter — a silently failing retention job is a health-surface condition, not just a log line."
|
||||
6. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central docs/requirements/Component-AuditLog.md && git commit -m "feat(audit-log): surface partition-purge failure as health event + counter"`
|
||||
|
||||
### Task 6: Fix purger role grants (migration)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1, 2, 3, 11, 20 (NOT with Task 8 — both add migrations, serialize on the model snapshot)
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/<timestamp>_FixAuditPurgerRoleGrants.cs` (scaffolded)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Migrations/` (add `FixAuditPurgerRoleGrantsTests.cs`), plus re-run `AuditLogAppendOnlyGuardTests`
|
||||
|
||||
1. Failing test first — a migration-content guard in the style of the existing migration tests: assert the migration source contains `GRANT CREATE TABLE TO scadabridge_audit_purger` and `GRANT DELETE ON dbo.AuditLog TO scadabridge_audit_purger`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter FixAuditPurgerRoleGrants` — FAIL (file absent).
|
||||
2. Scaffold (repo gotcha — build first, NO `--no-build`):
|
||||
|
||||
```bash
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
cd src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase
|
||||
dotnet ef migrations add FixAuditPurgerRoleGrants
|
||||
```
|
||||
|
||||
Verify the scaffold is model-empty (no schema change) and add the SQL by hand in `Up`:
|
||||
|
||||
```csharp
|
||||
migrationBuilder.Sql(@"
|
||||
IF DATABASE_PRINCIPAL_ID('scadabridge_audit_purger') IS NULL
|
||||
EXEC sp_executesql N'CREATE ROLE scadabridge_audit_purger';
|
||||
-- The switch-out dance CREATEs a staging table; ALTER ON SCHEMA::dbo alone does not
|
||||
-- confer the database-level CREATE TABLE permission (review finding S3).
|
||||
GRANT CREATE TABLE TO scadabridge_audit_purger;
|
||||
-- The per-channel retention override (PurgeChannelOlderThanAsync) is a bounded row
|
||||
-- DELETE on the maintenance path; a segregated purger principal needs this grant.
|
||||
GRANT DELETE ON dbo.AuditLog TO scadabridge_audit_purger;"); // AUDIT-PURGE-ALLOWED: role grant for the maintenance principal
|
||||
```
|
||||
|
||||
`Down`: `REVOKE` both grants (guarded on principal existence).
|
||||
3. Run the new test — PASS. Then run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter AuditLogAppendOnlyGuardTests` — if the CI grep guard trips on the `GRANT DELETE ON dbo.AuditLog` text, extend the guard's allowlist to permit `GRANT`/`DENY`/`REVOKE` statements (they are permission DDL, not data mutation) — keep the guard strict for `DELETE FROM`/`UPDATE `.
|
||||
4. Production script step (repo convention — manual SQL for production): `dotnet ef migrations script AddPendingDeployment FixAuditPurgerRoleGrants --idempotent --output ../../docs/plans/sql/FixAuditPurgerRoleGrants.sql` (adjust the from-migration to the actual latest; review the output).
|
||||
5. MSSQL sanity: with infra up, `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter Migrations` — expect PASS.
|
||||
6. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests docs/plans/sql && git commit -m "fix(config-db): grant CREATE TABLE + scoped DELETE to scadabridge_audit_purger so a segregated maintenance principal can actually run the purge"`
|
||||
|
||||
### Task 7: Honest append-only enforcement story (docs)
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** everything except 24 (same doc files)
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (~417-424, "Security & Tamper-Evidence")
|
||||
- Modify: `docs/requirements/Component-ConfigurationDatabase.md` (~399-402, AuditLog Table Purge section)
|
||||
|
||||
1. Rewrite Component-AuditLog.md §Security "Append-only enforcement" to state reality (review finding S3): the application runs with **one** connection principal for both writer and maintenance paths, so in the default deployment append-only is enforced by (a) the CI grep guard (`AuditLogAppendOnlyGuardTests`) with its single marked DELETE exemption, and (b) code review. The `scadabridge_audit_writer` / `scadabridge_audit_purger` roles are **optional DBA hardening**: a deployment that wants database-level enforcement must provision two logins (runtime → writer role; a maintenance job/second connection → purger role) — and note the purger role now carries `CREATE TABLE` + scoped `DELETE` (Task 6) so it can genuinely execute the switch-out and per-channel purge. Remove the false claim "row-level DELETE is not granted even to purge" (stale since `PerChannelRetentionDays` shipped).
|
||||
2. In Component-ConfigurationDatabase.md fix the same passage's stale "single global value in v1, no per-channel overrides" claim while in the file (coordinates with Task 24 — do this bullet here since it is in the same paragraph; Task 24 skips it if already fixed).
|
||||
3. Verify no other doc contradicts: `grep -rn "audit_writer\|audit_purger" docs/`.
|
||||
4. Commit: `git add docs/requirements/Component-AuditLog.md docs/requirements/Component-ConfigurationDatabase.md && git commit -m "docs(audit-log): document append-only enforcement honestly (CI guard default; DB roles are optional DBA hardening)"`
|
||||
|
||||
### Task 8: SiteCalls filtered non-terminal index (migration)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1, 2, 3, 5, 11 (NOT with Task 6 — migration serialization)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/SiteCallEntityTypeConfiguration.cs` (~66-88; also fix the now-stale "No index — … per-site, not per-node" SourceNode comment)
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/<timestamp>_AddSiteCallsNonTerminalIndex.cs` (scaffolded)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/SchemaConfigurationTests.cs` (extend)
|
||||
|
||||
1. Failing model test (SchemaConfigurationTests already asserts index shapes):
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public void SiteCalls_Has_Filtered_NonTerminal_Index()
|
||||
{
|
||||
var entity = Model.FindEntityType(typeof(SiteCall))!;
|
||||
var index = entity.GetIndexes().Single(i => i.GetDatabaseName() == "IX_SiteCalls_NonTerminal");
|
||||
Assert.Equal(new[] { nameof(SiteCall.CreatedAtUtc) }, index.Properties.Select(p => p.Name));
|
||||
Assert.Equal("[TerminalAtUtc] IS NULL", index.GetFilter());
|
||||
Assert.Equal(new[] { "SourceSite", "SourceNode", "Status" }, index.GetIncludeProperties());
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter SiteCalls_Has_Filtered_NonTerminal_Index` — FAIL.
|
||||
3. Add to `SiteCallEntityTypeConfiguration.Configure`:
|
||||
|
||||
```csharp
|
||||
// NonTerminal backs every "live queue" KPI predicate (TerminalAtUtc IS NULL):
|
||||
// buffered/stuck/oldest counts run every 60 s from the KPI recorder plus every
|
||||
// 10 s Health-dashboard poll. Filtered: the non-terminal population is the live
|
||||
// queue, not the 365-day archive — each count becomes a small seek instead of a
|
||||
// clustered scan (arch-review 04, P1).
|
||||
builder.HasIndex(s => s.CreatedAtUtc)
|
||||
.HasDatabaseName("IX_SiteCalls_NonTerminal")
|
||||
.HasFilter("[TerminalAtUtc] IS NULL")
|
||||
.IncludeProperties(nameof(SiteCall.SourceSite), nameof(SiteCall.SourceNode), nameof(SiteCall.Status));
|
||||
```
|
||||
|
||||
4. Scaffold the migration (build first, NO `--no-build`): `dotnet build ZB.MOM.WW.ScadaBridge.slnx && cd src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase && dotnet ef migrations add AddSiteCallsNonTerminalIndex` — verify the scaffold contains the `CreateIndex` with filter (if it is empty, the build/`--no-build` gotcha bit — delete and redo).
|
||||
5. Run the model test — PASS. With infra up, run the MSSQL-backed repo suite: `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter SiteCallAuditRepositoryTests` — PASS.
|
||||
6. Production script: `dotnet ef migrations script FixAuditPurgerRoleGrants AddSiteCallsNonTerminalIndex --idempotent --output ../../docs/plans/sql/AddSiteCallsNonTerminalIndex.sql`.
|
||||
7. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests docs/plans/sql && git commit -m "perf(site-call-audit): filtered IX_SiteCalls_NonTerminal index — KPI predicates seek the live queue instead of scanning 365 days"`
|
||||
|
||||
### Task 9: Bound the execution-tree edge scan with a root time window
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 11, 13, 18, 19
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs` (`GetExecutionTreeAsync`, ~632-790)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (audit-tree section: document the 7-day traversal bound)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Integration/ExecutionIdCorrelationTests.cs` or the existing execution-tree test file (extend; MSSQL)
|
||||
|
||||
1. Failing test (add to the existing MSSQL execution-tree suite — locate with `grep -rln "GetExecutionTreeAsync" tests/`):
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task ExecutionTree_Bounds_EdgeScan_To_Root_Window()
|
||||
{
|
||||
// Root execution at T0, child at T0+5min → both returned.
|
||||
// Unrelated execution pair at T0+30 days sharing no ancestry → NOT returned (was never returned;
|
||||
// this asserts correctness is preserved), AND a genuine descendant stamped at T0+30 days
|
||||
// (beyond the 7-day traversal bound) is documented-excluded: assert it is absent.
|
||||
}
|
||||
```
|
||||
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter ExecutionTree_Bounds_EdgeScan` — FAIL (descendant beyond bound is currently returned).
|
||||
3. Implementation in `GetExecutionTreeAsync`, between Phase 1 and Phase 2:
|
||||
- New constants: `private static readonly TimeSpan ExecutionTreeWindowSlack = TimeSpan.FromHours(1);` and `private static readonly TimeSpan ExecutionTreeMaxSpan = TimeSpan.FromDays(7);` with a comment: execution trees span minutes, not years; the window enables partition elimination on the Edges anchor (arch-review 04, P2).
|
||||
- Query `SELECT MIN(OccurredAtUtc) FROM dbo.AuditLog WHERE ExecutionId = @root;` (a seek on `IX_AuditLog_Execution`).
|
||||
- If non-NULL, parameterize the Edges CTE anchor: `WHERE ExecutionId IS NOT NULL AND OccurredAtUtc >= @winStart AND OccurredAtUtc < @winEnd` with `@winStart = rootFirst - slack`, `@winEnd = rootFirst + maxSpan`. If NULL (row-less stub root), keep the unbounded form (current behavior) — correctness over speed for the degenerate case.
|
||||
4. Run the execution-tree filter plus the full existing suite: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter "FullyQualifiedName~ExecutionTree|FullyQualifiedName~ExecutionId"` — PASS.
|
||||
5. Doc: Component-AuditLog.md audit-tree section — "tree traversal is bounded to a 7-day window anchored at the root's first event (configurable constant); descendants beyond the window are excluded by design."
|
||||
6. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/AuditLogRepository.cs docs/requirements/Component-AuditLog.md tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests && git commit -m "perf(audit-log): bound GetExecutionTreeAsync edge scan to a root-anchored time window (partition elimination)"`
|
||||
|
||||
### Task 10: Fix the hardcoded-zero backlogTotal KPI history
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 6, 8, 11, 19, 21
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Kpi/IAuditBacklogProvider.cs`
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/Kpi/CentralHealthAuditBacklogProvider.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/ServiceCollectionExtensions.cs` (central registration; locate the file — `grep -n AddSingleton src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/ServiceCollectionExtensions.cs`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Kpi/AuditLogKpiSampleSource.cs`
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Kpi/AuditLogKpiSampleSourceTests.cs` (extend/create), `tests/ZB.MOM.WW.ScadaBridge.HealthMonitoring.Tests` (provider test)
|
||||
|
||||
1. Failing test:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task Collect_Uses_BacklogProvider_When_Present()
|
||||
{
|
||||
var repo = new FakeAuditLogRepository(new AuditLogKpiSnapshot(10, 1, BacklogTotal: 0, DateTime.UtcNow));
|
||||
var source = new AuditLogKpiSampleSource(repo, new FakeBacklogProvider(42));
|
||||
var samples = await source.CollectAsync(DateTime.UtcNow);
|
||||
Assert.Equal(42, samples.Single(s => s.Metric == KpiMetrics.AuditLog.BacklogTotal).Value);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Collect_FallsBack_To_Snapshot_When_Provider_Absent()
|
||||
{
|
||||
// ctor with provider: null → BacklogTotal sample = snapshot value (0) — unchanged behavior
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter AuditLogKpiSampleSource` — FAIL.
|
||||
3. Implementation:
|
||||
- Commons: `public interface IAuditBacklogProvider { long GetPendingBacklogTotal(); }` (XML doc: sums latest per-site `SiteAuditBacklog.PendingCount` health reports; best-effort — sites not yet reporting contribute zero).
|
||||
- HealthMonitoring: `CentralHealthAuditBacklogProvider(ICentralHealthAggregator)` — port the exact summation from `CentralUI/Services/AuditLogQueryService.GetKpiSnapshotAsync` (~lines 145-155: `state.LatestReport?.SiteAuditBacklog?.PendingCount`, `> 0` guard). Register `services.AddSingleton<IAuditBacklogProvider, CentralHealthAuditBacklogProvider>();` in the central HealthMonitoring registration.
|
||||
- `AuditLogKpiSampleSource`: ctor gains `IAuditBacklogProvider? backlogProvider = null` (MS.DI resolves the default when unregistered — sites/tests unaffected); backlog sample value becomes `_backlogProvider?.GetPendingBacklogTotal() ?? snapshot.BacklogTotal`.
|
||||
- Optionally refactor `AuditLogQueryService` to consume the provider (skip if it drags UI DI churn — the live tile already works; the *history* was the broken surface).
|
||||
4. Run filter — PASS. Build the solution.
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.HealthMonitoring src/ZB.MOM.WW.ScadaBridge.AuditLog tests && git commit -m "fix(kpi-history): backlogTotal trend records the real site-backlog aggregate instead of a hardwired zero"`
|
||||
|
||||
### Task 11: SiteCalls same-rank freshness upsert (unfreeze RetryCount/LastError)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 8, 9, 10
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/SiteCallAuditRepository.cs` (rank comment ~27-40; UPDATE predicate ~117-139)
|
||||
- Modify: `docs/requirements/Component-SiteCallAudit.md` (~76-78 upsert contract wording)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Repositories/SiteCallAuditRepositoryTests.cs` (extend; MSSQL — infra up)
|
||||
|
||||
1. Failing tests:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task Upsert_SameRank_NewerUpdatedAt_RefreshesProgressFields()
|
||||
{
|
||||
var id = NewId();
|
||||
await Repo.UpsertAsync(Call(id, "Attempted", retryCount: 1, lastError: "timeout #1", updatedAt: T0));
|
||||
await Repo.UpsertAsync(Call(id, "Attempted", retryCount: 5, lastError: "timeout #5", updatedAt: T0.AddMinutes(5)));
|
||||
var row = await Repo.GetAsync(id);
|
||||
Assert.Equal(5, row!.RetryCount);
|
||||
Assert.Equal("timeout #5", row.LastError);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Upsert_SameRank_OlderUpdatedAt_IsNoOp() { /* retry 5 then out-of-order retry 2 with older stamp → stays 5 */ }
|
||||
|
||||
[Fact]
|
||||
public async Task Upsert_LowerRank_IsStillNoOp() { /* Delivered then Attempted (newer stamp) → stays Delivered */ }
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter SiteCallAuditRepositoryTests` — FAIL (same-rank is a no-op today).
|
||||
3. Change the UPDATE guard from `incomingRank > CASE...` to:
|
||||
|
||||
```sql
|
||||
AND ( {incomingRank} > (CASE Status ... END)
|
||||
OR ({incomingRank} = (CASE Status ... END) AND UpdatedAtUtc < {siteCall.UpdatedAtUtc}) )
|
||||
```
|
||||
|
||||
(repeat the CASE expression — both branches parameterize identically). Update the class comment: "monotonic on rank; within equal rank, newest `UpdatedAtUtc` wins (still idempotent — equal stamps are a no-op — and still regression-proof)". This unfreezes `RetryCount`/`LastError`/`HttpStatus` during the Attempted phase and lets a genuinely-newer terminal correction land.
|
||||
4. Run filter — PASS (all pre-existing monotonic tests must stay green).
|
||||
5. Doc: Component-SiteCallAudit.md upsert paragraph — replace "first-write-wins at each rank" semantics with "insert-if-not-exists, then upsert on newer status rank, with a newest-`UpdatedAtUtc` tiebreaker within equal rank so retrying calls show live RetryCount/LastError".
|
||||
6. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/SiteCallAuditRepository.cs docs/requirements/Component-SiteCallAudit.md tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests && git commit -m "fix(site-call-audit): same-rank freshness tiebreaker — retrying calls no longer freeze RetryCount/LastError at first write"`
|
||||
|
||||
### Task 12: EF Core connection resiliency (`EnableRetryOnFailure`)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 8, 11 (do NOT run concurrently with any task executing MSSQL integration tests — behavior-wide change)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ServiceCollectionExtensions.cs` (~29-34)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogIngestActor.cs` (`OnCachedTelemetryAsync` dual-write transaction ~262-311)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/ServiceCollectionExtensionsTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Integration/CombinedTelemetryIdempotencyTests.cs` (must stay green)
|
||||
|
||||
1. First inventory every user-initiated transaction against `ScadaBridgeDbContext`: `grep -rn "BeginTransaction" src/ --include="*.cs"` — expect `AuditLogIngestActor.OnCachedTelemetryAsync`; treat any others found identically.
|
||||
2. Failing test in `ServiceCollectionExtensionsTests`:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public void AddConfigurationDatabase_Configures_RetryingExecutionStrategy()
|
||||
{
|
||||
var services = new ServiceCollection();
|
||||
services.AddConfigurationDatabase("Server=unused;Database=unused;");
|
||||
using var sp = services.BuildServiceProvider();
|
||||
var options = sp.GetRequiredService<DbContextOptions<ScadaBridgeDbContext>>();
|
||||
var sqlExt = options.Extensions.OfType<SqlServerOptionsExtension>().Single();
|
||||
Assert.NotNull(sqlExt.ExecutionStrategyFactory); // retry strategy configured
|
||||
}
|
||||
```
|
||||
|
||||
3. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter Configures_RetryingExecutionStrategy` — FAIL.
|
||||
4. Implementation:
|
||||
- `options.UseSqlServer(connectionString, sql => sql.EnableRetryOnFailure(maxRetryCount: 5, maxRetryDelay: TimeSpan.FromSeconds(30), errorNumbersToAdd: null))` — comment: covers every read-side path (KPI asks, outbox queries, `Notify.Status`, execution trees) that previously surfaced transient faults raw (arch-review 04, S5).
|
||||
- `OnCachedTelemetryAsync`: wrap the per-entry `BeginTransactionAsync` block in the execution strategy — `var strategy = context.Database.CreateExecutionStrategy(); await strategy.ExecuteAsync(async () => { await using var tx = await context.Database.BeginTransactionAsync(); ... await tx.CommitAsync(); });` — the block is already idempotent (insert-if-not-exists + monotonic upsert), so a retried unit is safe; say so in the comment.
|
||||
- Note in `SwitchOutPartitionAsync`'s comment: the raw batch carries its own server-side BEGIN TRAN/CATCH and is IF-EXISTS-guarded idempotent, so strategy-level replay of the whole batch is safe.
|
||||
5. Run: the new filter → PASS; then with infra up, `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter "CombinedTelemetry"` → PASS (proves the dual-write still commits atomically under the strategy).
|
||||
6. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/ServiceCollectionExtensions.cs src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogIngestActor.cs tests && git commit -m "fix(config-db): EnableRetryOnFailure fleet-wide + execution-strategy wrap of the combined-telemetry dual-write"`
|
||||
|
||||
### Task 13: Audit row for operator Retry on parked notifications
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 8, 9, 11, 18, 19, 21
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Notification/NotificationOutboxQueries.cs` (`RetryNotificationRequest` ~54, `DiscardNotificationRequest` ~69 — additive `string? RequestedBy = null`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs` (`RetryAsync` ~951-977; `BuildNotifyDeliverEvent` ~675-714 gains `string? actorOverride = null`; `EmitTerminalAuditAsync` passes it for discard)
|
||||
- Modify: callers that send these requests — locate with `grep -rln "RetryNotificationRequest(" src/` (expect the Central UI outbox service and/or ManagementActor) and pass the authenticated username
|
||||
- Modify: `docs/requirements/Component-NotificationOutbox.md` (~128)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/NotificationOutboxActorRetryEmissionTests.cs` (create; clone the audit-writer capture rig from `NotificationOutboxActorTerminalEmissionTests.cs`)
|
||||
|
||||
1. Failing test:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task Retry_OnParked_Emits_Submitted_NotifyDeliver_AuditRow_With_Operator()
|
||||
{
|
||||
// seed a Parked notification; send RetryNotificationRequest(corr, id, RequestedBy: "jdoe")
|
||||
// expect RetryNotificationResponse Success
|
||||
// assert captured audit writer received exactly one event:
|
||||
// Kind NotifyDeliver, Status Submitted, Actor "jdoe", CorrelationId == NotificationId
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Retry_AuditFailure_DoesNotFail_TheRetry() { /* throwing audit writer → response still Success (audit is best-effort) */ }
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests --filter RetryEmission` — FAIL.
|
||||
3. Implementation:
|
||||
- Add `string? RequestedBy = null` to both request records (additive-only contract evolution — existing senders compile unchanged).
|
||||
- `BuildNotifyDeliverEvent(..., string? actorOverride = null)`: `actor: actorOverride ?? SystemActor`.
|
||||
- In `RetryAsync`, after `repository.UpdateAsync(notification)`: emit `BuildNotifyDeliverEvent(notification, now, AuditStatus.Submitted, errorMessage: null, actorOverride: request.RequestedBy)` via `_auditWriter.WriteAsync` inside the same try/catch-and-log shape as `EmitTerminalAuditAsync` (audit failure never aborts the retry). `Submitted` = "operator re-queued it" — the forensic gap the review names (`Parked → Attempted → Delivered` with no record of who un-parked).
|
||||
- `DiscardAsync`: pass `request.RequestedBy` through to the terminal row's actor.
|
||||
- Plumb the username at the senders (the UI/management layer knows the authenticated user; pass `null` where no identity exists).
|
||||
4. Run filter + `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests` — PASS.
|
||||
5. Doc: Component-NotificationOutbox.md ~128 — "…each transition emits the corresponding audit row: operator **Retry** emits a `Notification`-channel `NotifyDeliver` row with status `Submitted` and the operator as `Actor`; Discard emits the `Terminal` row (operator as `Actor`)."
|
||||
6. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.NotificationOutbox src/ZB.MOM.WW.ScadaBridge.CentralUI src/ZB.MOM.WW.ScadaBridge.ManagementService docs/requirements/Component-NotificationOutbox.md tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests && git commit -m "feat(notification-outbox): operator Retry/Discard emit audit rows with operator identity"`
|
||||
|
||||
### Task 14: Operator identity on the SiteCalls Retry/Discard relay
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 8, 9, 10, 19, 20
|
||||
**Files:**
|
||||
- Modify: the relay request records — locate with `grep -rn "record RetryParkedOperation\|record DiscardParkedOperation" src/ZB.MOM.WW.ScadaBridge.Commons` (additive `string? RequestedBy = null`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallAuditActor.cs` (relay handlers — see `SiteCallRelayTests.cs` for the handler names)
|
||||
- Modify: `docs/requirements/Component-SiteCallAudit.md` (relay section)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallRelayTests.cs` (extend)
|
||||
|
||||
1. Failing test: relay a Retry for a parked row with `RequestedBy: "jdoe"`; assert the injected central audit writer (same `ICentralAuditWriter` seam `NotificationOutboxActor` uses — confirm with `grep -rn "ICentralAuditWriter" src/ZB.MOM.WW.ScadaBridge.NotificationOutbox src/ZB.MOM.WW.ScadaBridge.Commons`) received one event: `Kind = CachedResolve`, `Status = Submitted`, `Actor = "jdoe"`, `CorrelationId = TrackedOperationId`, channel taken from the stored `SiteCalls` row's `Channel`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests --filter SiteCallRelayTests` — FAIL.
|
||||
2. Implementation: inject `ICentralAuditWriter` into `SiteCallAuditActor` (optional ctor param defaulting to null → no-op, so existing tests construct unchanged); in the Retry/Discard relay handlers, after the site relay succeeds, fetch the row (`ISiteCallAuditRepository.GetAsync`) for its `Channel`, build the event via `ScadaBridgeAuditEventFactory.Create` (mirror `BuildNotifyDeliverEvent`'s shape), write best-effort (try/catch + log — the relay outcome is authoritative, audit never aborts it). Site-side telemetry still records the state change itself; this row adds *who asked*.
|
||||
3. Run filter — PASS.
|
||||
4. Doc: Component-SiteCallAudit.md relay section — the relay now emits a central direct-write audit row carrying operator identity; sites remain the source of truth for the state change.
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.SiteCallAudit docs/requirements/Component-SiteCallAudit.md tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests && git commit -m "feat(site-call-audit): relay Retry/Discard emit operator-identity audit rows"`
|
||||
|
||||
### Task 15: Composite keyset in the SiteCalls reconciliation pull contract (site side)
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 8, 13
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/Protos/sitestream.proto` (additive `string after_id` on the SiteCalls pull request message — locate exact message with `grep -n "PullSiteCalls" src/ZB.MOM.WW.ScadaBridge.Communication/Protos/sitestream.proto`)
|
||||
- Modify: the Commons pull-request message (`grep -rn "record CachedCallReconcileRequest\|PullSiteCallsRequest" src/ZB.MOM.WW.ScadaBridge.Commons`) — additive `string? AfterId = null`
|
||||
- Modify: the site-side pull handler + tracking-store query (`grep -rn "ReadSince\|PullSiteCalls" src/ZB.MOM.WW.ScadaBridge.StoreAndForward src/ZB.MOM.WW.ScadaBridge.Communication --include="*.cs"`) — order by `(UpdatedAtUtc, TrackedOperationId)`, filter `UpdatedAtUtc > since OR (UpdatedAtUtc = since AND TrackedOperationId > afterId)` (when `afterId` present; absent → legacy `>= since` behavior)
|
||||
- Test: site-side handler/store tests (same project as the handler; locate via the greps above)
|
||||
|
||||
1. Failing test on the site store/handler: seed `batchSize + 2` rows sharing one `UpdatedAtUtc`; pull with `since = thatTimestamp, afterId = <id of last row of page 1>`; assert page 2 returns the remaining rows (today it returns the same page). Run the owning test project with `--filter <new test name>` — FAIL.
|
||||
2. Implement the additive proto field (field number = next free; additive-only rule), regenerate/compile, map it through the DTO, and apply the keyset predicate + deterministic `(UpdatedAtUtc, TrackedOperationId)` ordering in the store query. Absent `AfterId` preserves the exact legacy contract (old central against new site: unchanged).
|
||||
3. Run the owning project's full suite — PASS.
|
||||
4. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Communication src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.StoreAndForward tests && git commit -m "feat(site-call-audit): additive (UpdatedAtUtc, TrackedOperationId) keyset in the reconciliation pull contract"`
|
||||
|
||||
### Task 16: Central keyset cursor — un-pin the SiteCalls reconciliation loop
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 5, 8, 9, 10 (requires Task 15)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallAuditActor.cs` (reconciliation loop ~600-670: cursor becomes `(DateTime, string?)`; pin branch ~636-654)
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallReconciliationPinnedChanged.cs` (EventStream record, mirroring `SiteAuditTelemetryStalledChanged`)
|
||||
- Modify: `docs/requirements/Component-SiteCallAudit.md` (reconciliation cursor paragraph)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallAuditReconciliationTests.cs` (extend)
|
||||
|
||||
1. Failing test: fake pull client honoring the keyset; seed `2 × batchSize` rows sharing one `UpdatedAtUtc`; drive one tick; assert **all** rows were upserted (today the tail never reconciles). Second test: a fake *legacy* client that ignores `AfterId` (returns the same page) → assert the actor publishes `SiteCallReconciliationPinnedChanged(siteId, pinned: true)` on the EventStream instead of silently logging. `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests --filter Reconciliation` — FAIL.
|
||||
2. Implement: per-site cursor `(DateTime since, string? afterId)`; after each page set `afterId` to the max `(UpdatedAtUtc, TrackedOperationId)` row's id; pass it on the next pull. Keep the no-progress detection as the legacy-site fallback, but publish the new event (and clear it with `pinned: false` when progress resumes) — the dead-end becomes a health-observable condition.
|
||||
3. Run filter — PASS; run the full SiteCallAudit.Tests suite — PASS.
|
||||
4. Doc: Component-SiteCallAudit.md — cursor is now a composite `(UpdatedAtUtc, TrackedOperationId)` keyset (additive contract field; legacy sites fall back to the timestamp cursor and surface `SiteCallReconciliationPinnedChanged` when pinned). Also add one line to Component-AuditLog.md's reconciliation section: the analogous keyset for the `PullAuditEvents` cursor is a tracked follow-up (same shape; lower urgency because the audit cursor already re-pulls idempotently).
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.SiteCallAudit docs/requirements tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests && git commit -m "fix(site-call-audit): composite keyset cursor eliminates the single-timestamp reconciliation pin; pinned state now a published event"`
|
||||
|
||||
### Task 17: Durable record for permanently-abandoned reconciliation rows
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 8, 11, 19, 20, 21
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditKind.cs` (additive `ReconciliationAbandoned` value)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/SiteAuditReconciliationActor.cs` (abandonment branch ~283-293)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (kind vocabulary + reconciliation section)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/SiteAuditReconciliationActorTests.cs` (extend)
|
||||
|
||||
1. Failing test: repository stub whose `InsertIfNotExistsAsync` throws for one specific EventId across `MaxPermanentInsertAttempts` ticks but accepts everything else; after the abandonment tick, assert a **synthetic** audit event was inserted (new EventId; `Kind = ReconciliationAbandoned`; `DetailsJson`/Extra carrying the abandoned `EventId`, the site id, and the final error) and the cursor advanced. `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter SiteAuditReconciliationActorTests` — FAIL.
|
||||
2. Implement in the `attempts >= MaxPermanentInsertAttempts` branch: alongside the existing `LogCritical`, build a minimal synthetic event (fresh `Guid.NewGuid()` EventId, `OccurredAtUtc = now`, Actor = system, Action `"Audit.ReconciliationAbandoned"`, Category preserved from the lost event where parseable, Extra = `{ abandonedEventId, sourceSiteId, error }` — use the same factory/projection helpers the ingest path uses) and `InsertIfNotExistsAsync` it in its own try/catch (the synthetic row is small and well-formed, so the row-specific failure that killed the original won't recur; if it does, log and continue — never block the cursor twice). The permanent loss is now queryable in the audit log itself, not only in a rotating log file.
|
||||
3. Run filter — PASS.
|
||||
4. Doc: add `ReconciliationAbandoned` to the kind vocabulary in Component-AuditLog.md (Task 24 updates the Commons enum listing — coordinate: this task adds the value, 24 fixes the counts).
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.AuditLog docs/requirements/Component-AuditLog.md tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests && git commit -m "feat(audit-log): permanently-abandoned reconciliation rows leave a durable ReconciliationAbandoned audit record"`
|
||||
|
||||
### Task 18: Reconciliation NodeB failover (owned from report 08)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9, 11, 13, 19, 20, 21
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/ISiteEnumerator.cs` (`SiteEntry` record ~37: additive `string? FallbackGrpcEndpoint = null`; fix the stale "NodeA-only first cut" doc comment)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/SiteEnumerator.cs` (~60-75: populate from `GrpcNodeBAddress`; include sites with blank NodeA but valid NodeB, primary = NodeA else NodeB)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/GrpcPullAuditEventsClient.cs` + `GrpcPullSiteCallsClient.cs` (retry-once-on-fallback)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (reconciliation endpoint-resolution paragraph)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Central/SiteEnumeratorTests.cs`, `GrpcPullAuditEventsClientTests.cs`, `GrpcPullSiteCallsClientTests.cs` (extend)
|
||||
|
||||
1. Failing tests: (a) `SiteEnumeratorTests` — site with both addresses yields `SiteEntry(id, nodeA, FallbackGrpcEndpoint: nodeB)`; site with only NodeB is no longer skipped; (b) `GrpcPullAuditEventsClientTests` — fake invoker throws `Unavailable` for the primary endpoint and succeeds for the fallback → the pull returns the fallback's events (today: collapses to empty). `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter "SiteEnumeratorTests|GrpcPull"` — FAIL.
|
||||
2. Implement: in both pull clients' transport-fault catch (the `Unavailable`/`DeadlineExceeded`/`Cancelled`/`HttpRequestException` set already enumerated there), if `FallbackGrpcEndpoint` is non-blank and differs from the primary, invoke once against it before collapsing to empty; log the failover at Information. The invoker's per-endpoint channel cache already handles the second endpoint.
|
||||
3. Run filter + full Central test folder — PASS.
|
||||
4. Doc: Component-AuditLog.md — reconciliation dials NodeA first, fails over to NodeB per call; during a NodeA outage the loss-recovery safety net stays available.
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog docs/requirements/Component-AuditLog.md tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests && git commit -m "fix(audit-log): reconciliation pulls fail over to site NodeB when NodeA is unreachable"`
|
||||
|
||||
### Task 19: Batch the KpiSample purge
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1, 2, 3, 8, 9, 13, 18
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/KpiHistoryRepository.cs` (`PurgeOlderThanAsync` ~65-71)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IKpiHistoryRepository.cs` (doc update; signature unchanged)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Repositories/KpiHistoryRepositoryTests.cs` (extend)
|
||||
|
||||
1. Failing test: seed samples spread across 5 days; purge with a 2-day-old cutoff; assert the correct rows are gone and the return value equals the total deleted — then assert via an EF command interceptor (or SQL capture, matching how sibling tests inspect SQL) that **more than one** DELETE statement was issued for a multi-window purge. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter KpiHistoryRepositoryTests` — FAIL (single `ExecuteDeleteAsync` today).
|
||||
2. Implement provider-portable time-slice batching (avoids `DELETE TOP` dialect divergence — the convention Task 24 documents):
|
||||
|
||||
```csharp
|
||||
public async Task<int> PurgeOlderThanAsync(DateTime before, CancellationToken cancellationToken = default)
|
||||
{
|
||||
// Time-sliced batches: each DELETE covers at most one hour of samples, capping the
|
||||
// lock/log footprint per statement (arch-review 04, P4 — steady state is ~1 day of
|
||||
// rows/day; after an outage the catch-up would otherwise be one giant transaction).
|
||||
var total = 0;
|
||||
var floor = await _context.KpiSamples.Where(s => s.CapturedAtUtc < before)
|
||||
.MinAsync(s => (DateTime?)s.CapturedAtUtc, cancellationToken);
|
||||
while (floor is not null && floor < before)
|
||||
{
|
||||
var ceiling = floor.Value.AddHours(1) < before ? floor.Value.AddHours(1) : before;
|
||||
total += await _context.KpiSamples
|
||||
.Where(s => s.CapturedAtUtc < ceiling)
|
||||
.ExecuteDeleteAsync(cancellationToken);
|
||||
floor = await _context.KpiSamples.Where(s => s.CapturedAtUtc < before)
|
||||
.MinAsync(s => (DateTime?)s.CapturedAtUtc, cancellationToken);
|
||||
}
|
||||
return total;
|
||||
}
|
||||
```
|
||||
|
||||
3. Run filter — PASS; run `dotnet test tests/ZB.MOM.WW.ScadaBridge.KpiHistory.Tests` (recorder actor contract unchanged) — PASS.
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/KpiHistoryRepository.cs src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/Repositories/IKpiHistoryRepository.cs tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests && git commit -m "perf(kpi-history): time-sliced batched purge replaces the single daily mega-DELETE"`
|
||||
|
||||
### Task 20: NotificationOutbox KPI — single-query aggregation, bounded oldest
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 8, 9, 14, 18, 19
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/NotificationOutboxRepository.cs` (`ComputeKpisAsync` ~239-283, `ComputePerSiteKpisAsync` ~286-334, `ComputePerNodeKpisAsync` ~337-391)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/NotificationOutboxRepositoryPerSiteKpiTests.cs`, `...PerNodeKpiTests.cs`, and the global KPI tests (all must stay green — this is a pure refactor of query shape)
|
||||
|
||||
1. Add one new failing test capturing the *bounded-oldest* behavior: seed 3 non-terminal rows; assert the oldest-pending computation issues no query that materializes more than 1 row (assert via command interceptor / captured SQL: the oldest query is `ORDER BY … OFFSET 0 ROWS FETCH NEXT 1` or `TOP(1)`, not a full non-terminal SELECT). Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter NotificationOutboxRepository` — FAIL (per-site/per-node paths materialize every non-terminal row today, ~lines 312-318, 368-375).
|
||||
2. Refactor each snapshot to:
|
||||
- **Counts:** one conditional-aggregation query per scope — `GroupBy(n => 1)` (global) / `GroupBy(n => n.SourceSiteId)` / `GroupBy(n => n.SourceNode)` with `g.Count(n => <predicate>)` per metric, replacing the 5-7 sequential round trips.
|
||||
- **Oldest pending:** global — `Where(nonTerminal).OrderBy(n => n.CreatedAt).Select(n => (DateTimeOffset?)n.CreatedAt).FirstOrDefaultAsync()`; per-site/per-node — try server-side `GroupBy(...).Select(g => new { g.Key, Oldest = g.Min(n => n.CreatedAt) })` first; if the `DateTimeOffset` converter blocks translation (the review's noted awkwardness), fall back to projecting only `(Key, CreatedAt)` pairs of non-terminal rows (narrow projection, no entity materialization) and reduce in memory — either way the full-entity materialization is gone.
|
||||
3. Run the full NotificationOutboxRepository test set + `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests --filter Kpi` — PASS (identical snapshot values).
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/NotificationOutboxRepository.cs tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests && git commit -m "perf(notification-outbox): single-query KPI aggregation; oldest-pending no longer materializes the live queue"`
|
||||
|
||||
### Task 21: `OPTION (RECOMPILE)` on the SiteCalls catch-all query
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~2 min
|
||||
**Parallelizable with:** 1, 2, 3, 8, 9, 13, 18, 19, 20 (NOT 11 — same file)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/SiteCallAuditRepository.cs` (`QueryAsync` SQL ~186-202)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests/Repositories/SiteCallAuditRepositoryTests.cs` (existing query tests must pass)
|
||||
|
||||
1. Append `OPTION (RECOMPILE)` after the `ORDER BY` line of the `FormattableString` (with a comment: every filter is the `(@p IS NULL OR col=@p)` optional-parameter shape; RECOMPILE lets the optimizer prune dead predicates per invocation and pick `IX_SiteCalls_Status_Updated`/`IX_SiteCalls_NonTerminal` — arch-review 04, P5; per-invocation compile cost is fine at UI-page cadence).
|
||||
2. With infra up: `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests --filter SiteCallAuditRepositoryTests` — PASS.
|
||||
3. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Repositories/SiteCallAuditRepository.cs && git commit -m "perf(site-call-audit): OPTION (RECOMPILE) on the optional-parameter query page"`
|
||||
|
||||
### Task 22: Short first-tick delay on the three purge timers
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1, 2, 6, 8, 19, 20, 21 (NOT 4/5 — AuditLogPurgeActor; NOT 16 — SiteCallAuditActor)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Central/AuditLogPurgeActor.cs` (`PreStart` ~95-100)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallAuditActor.cs` (`StartPurgeTimer` ~374-387)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.KpiHistory/KpiHistoryRecorderActor.cs` (purge timer ~124-128)
|
||||
- Test: extend each actor's existing test file with one fast-first-tick case
|
||||
|
||||
1. Failing test (one per actor, using each suite's existing scheduler/interval-override rig): configure a long interval (24h) and assert a purge tick arrives within the short initial delay window rather than never.
|
||||
2. Implement uniformly: `var initialDelay = interval < ShortFirstTick ? interval : ShortFirstTick;` with `private static readonly TimeSpan ShortFirstTick = TimeSpan.FromMinutes(5);` and a shared comment: purges are idempotent; `initialDelay = interval` meant a node recycled daily would never purge (arch-review 04, S9). Taking `min(interval, 5min)` keeps millisecond test cadences unchanged.
|
||||
3. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter AuditLogPurgeActorTests && dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests --filter Purge && dotnet test tests/ZB.MOM.WW.ScadaBridge.KpiHistory.Tests` — PASS.
|
||||
4. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog src/ZB.MOM.WW.ScadaBridge.SiteCallAudit src/ZB.MOM.WW.ScadaBridge.KpiHistory tests && git commit -m "fix(purge): short first tick on all three purge timers so daily-recycled nodes still purge"`
|
||||
|
||||
### Task 23: Low-severity cleanup batch (param chunking, KPI catalog, computed-column guard)
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 4, 5, 8, 9, 11, 20
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.AuditLog/Site/SqliteAuditWriter.cs` (`MarkForwardedAsync` ~644-664, `MarkReconciledAsync` ~735-753)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Kpi/KpiMetrics.cs` (locate: `grep -rn "class KpiMetrics" src/ZB.MOM.WW.ScadaBridge.Commons`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Kpi/NotificationOutboxKpiSampleSource.cs` (~40-43)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/AuditLogEntityTypeConfiguration.cs` (~158-164)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests/Site/SqliteAuditWriterWriteTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/Kpi` (extend)
|
||||
|
||||
1. **P8 — parameter chunking:** failing test — write 1200 events, `MarkForwardedAsync(all 1200 ids)`, assert all flip to Forwarded (today: one statement with 1200+ parameters risks/violates SQLite's 999 limit as batch sizes grow). Implement: chunk `eventIds` into slices of 500 inside the existing `_writeLock` (one transaction around all chunks so the batch stays atomic); same in `MarkReconciledAsync`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.AuditLog.Tests --filter SqliteAuditWriter` — PASS.
|
||||
2. **C4 — KPI metric catalog:** add `KpiMetrics.NotificationOutbox.StuckCount = "stuckCount"` and `OldestPendingAgeSeconds = "oldestPendingAgeSeconds"` (EXACT existing string values — persisted, renames forbidden); replace the inline literals in `NotificationOutboxKpiSampleSource` with the constants; add a test asserting the constant values equal the historical strings.
|
||||
3. **P7 — computed-column guard:** extend the `IngestedAtUtc` comment in `AuditLogEntityTypeConfiguration`: "NEVER use IngestedAtUtc in a WHERE predicate — non-persisted, it forces a full-table `JSON_VALUE` parse; read-side projection only (arch-review 04, P7)."
|
||||
4. Run both test filters + build — PASS.
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.AuditLog src/ZB.MOM.WW.ScadaBridge.Commons src/ZB.MOM.WW.ScadaBridge.NotificationOutbox src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase tests && git commit -m "chore(data-layer): low-severity cleanups — SQLite param chunking, KPI metric catalog, computed-column predicate guard"`
|
||||
|
||||
### Task 24: Design-doc reconciliation pass
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** all code tasks except 7 (same files); run AFTER 17 (AuditKind gains a value)
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/Component-Commons.md` (~46-47, ~89)
|
||||
- Modify: `docs/requirements/Component-ConfigurationDatabase.md` (~58; timestamp-convention + clustered-key + repository-dialect notes)
|
||||
- Modify: `docs/requirements/Component-SiteCallAudit.md` (~44-51 schema section)
|
||||
- Modify: `docs/requirements/Component-AuditLog.md` (~396-399 per-channel config channel list)
|
||||
- Modify: `docs/requirements/Component-NotificationOutbox.md` (~23 "dedicated blocking-I/O dispatcher" phrasing — align with Task 25 if it has landed, else describe the actual thread-pool off-actor delivery)
|
||||
|
||||
Concrete edits (each verifiable against the code cited in the review):
|
||||
1. Component-Commons.md ~47: `AuditKind` list → the current enum (`grep -A20 "enum AuditKind" src/ZB.MOM.WW.ScadaBridge.Commons/Types/Enums/AuditKind.cs`): the original 10 + `SecuredWriteSubmitted/Approved/Rejected/Executed` (4) + `ReconciliationAbandoned` (Task 17). ~46: `AuditChannel` → 5 values incl. `SecuredWrite`.
|
||||
2. Component-Commons.md ~89: `NotificationType` → "`Email`, `Sms` (shipped 2026-06-19; the Teams plan was evaluated and dropped)".
|
||||
3. Component-ConfigurationDatabase.md ~58: NotificationList Type discriminator → "`Email` / `Sms`".
|
||||
4. Component-ConfigurationDatabase.md ~399: "single global value in v1, no per-channel overrides" → per-channel `AuditLog:PerChannelRetentionDays` overrides exist (skip if Task 7 already fixed it).
|
||||
5. Component-SiteCallAudit.md ~44-51: rewrite the schema table to the shipped columns (`Channel`/`Target`/`AuditStatus`-derived `Status` strings, `SourceNode`, no provenance columns) — align with `SiteCallEntityTypeConfiguration.cs` and Component-ConfigurationDatabase.md:64 (the current source of truth per the review).
|
||||
6. Component-AuditLog.md ~396-399: per-channel retention config enumerates all five channels including `SecuredWrite` (the validator already accepts it — doc under-promised).
|
||||
7. Component-Commons.md / Component-ConfigurationDatabase.md: document the `AuditLogRow` deviation (persistence-only projection of the external `ZB.MOM.WW.Audit.AuditEvent` contract — deliberately not a Commons POCO) and two conventions: **new tables use UTC `DateTime` + the Utc converter** (the `DateTimeOffset` choice on `Notification` is the documented exception that forced in-memory Min workarounds), and **new repositories write MSSQL-only SQL with MSSQL-backed test fixtures** (the NotificationOutboxRepository dual SQLite/T-SQL dialect is a documented legacy exception — finding S8). Record two tracked follow-ups: the AuditLog clustered-key `(OccurredAtUtc, EventId)` benchmark question (P3) and the per-node KPI sampling cadence / outbox keyset-paging deferrals (P4/P6).
|
||||
8. Verify: `grep -rn "Teams" docs/requirements/ | grep -iv "dropped\|evaluated"` returns nothing load-bearing; `grep -n "no per-channel" docs/requirements/Component-ConfigurationDatabase.md` returns nothing.
|
||||
9. Commit: `git add docs/requirements && git commit -m "docs: reconcile design docs with shipped code (enums, Sms, per-channel retention, SiteCalls schema, conventions)"`
|
||||
|
||||
### Task 25: Bounded parallel notification delivery
|
||||
|
||||
**Classification:** high-risk
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 8, 9, 11, 18, 19 (NOT 13 — same actor file; do 13 first)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxOptions.cs` (add `MaxParallelDeliveries`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs` (`RunDispatchPass` ~321-347)
|
||||
- Modify: `docs/requirements/Component-NotificationOutbox.md` (~23 dispatcher paragraph)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/NotificationOutboxActorDispatchTests.cs` (extend)
|
||||
|
||||
1. Failing test: adapter stub whose `Deliver` blocks on a gate and records concurrent-invocation high-water mark; enqueue 8 pending notifications; with `MaxParallelDeliveries = 4` assert high-water ≥ 2 and all 8 reach terminal status; with `MaxParallelDeliveries = 1` assert high-water == 1 (existing sequential behavior preserved). `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests --filter Dispatch` — FAIL.
|
||||
2. Implement: `MaxParallelDeliveries` (default **4**, clamp min 1 — the review's alarm-storm scenario: sequential 5 s SMTP attempts cap throughput at ~0.2/s regardless of batch size). In `RunDispatchPass`, replace the sequential foreach with `SemaphoreSlim(maxParallel)` + `Task.WhenAll`, giving **each notification its own DI scope/repository** (rows are claimed per-notification, so no shared-context concurrency; the existing in-flight pass guard is untouched). Audit emission stays per-notification inside its task.
|
||||
3. Run the full outbox suite: `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests` — PASS (attempt/terminal emission tests prove per-row semantics survived).
|
||||
4. Doc: Component-NotificationOutbox.md dispatcher section — delivery runs off the actor thread on the thread pool with bounded per-sweep parallelism (`MaxParallelDeliveries`, default 4); drop the inaccurate "dedicated blocking-I/O dispatcher" phrasing (doc drift the review flagged).
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.NotificationOutbox docs/requirements/Component-NotificationOutbox.md tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests && git commit -m "perf(notification-outbox): bounded parallel delivery within a dispatch sweep (default 4)"`
|
||||
|
||||
---
|
||||
|
||||
## Dependencies on other plans
|
||||
|
||||
- **Plan 02 (Communication/S&F)** owns the S&F retry-sweep batching — Task 15's site-side tracking-store query touches adjacent code in the StoreAndForward project; coordinate file ownership if both run concurrently.
|
||||
- **Plan 07 (UI/Management/Security)** owns SecuredWrite emission paths — the `SourceNode`-NULL follow-up (U4) stays with them; Task 24 only fixes the doc under-promise.
|
||||
- **Plan 08 (Conventions/Tests)** flagged the reconciliation NodeB gap; it is implemented here (Task 18) — plan 08 should not duplicate it.
|
||||
- Tasks 13/14 plumb `RequestedBy` through UI/ManagementActor senders — if plan 07 refactors those senders simultaneously, land this plan's additive message change first (defaults keep old senders compiling).
|
||||
- Migration serialization is **intra-plan only** (Tasks 6 → 8); no other plan currently adds ConfigurationDatabase migrations, but if one does, coordinate snapshot ordering.
|
||||
|
||||
## Execution order
|
||||
|
||||
**P0 (the operational time bombs — do first, in this order):** 1 → 2 → 3 (site purge chain), 4 → 5 (partition-switch hardening), 6 → 8 (migrations, serialized), 9, 10.
|
||||
**P1:** 11, 12, 13, 15 → 16, 17, 18.
|
||||
**P2 (hardening/cleanup):** 14, 19, 20, 21, 22, 23, 25, then 7 and 24 (docs last so they describe what actually landed).
|
||||
Intra-plan critical path: **1 → 2 → 3** and **15 → 16**; everything else is broadly parallelizable per each task's "Parallelizable with" contract. Finish with a full `dotnet build ZB.MOM.WW.ScadaBridge.slnx && dotnet test ZB.MOM.WW.ScadaBridge.slnx` (infra compose up for the MSSQL suites) before declaring the plan done.
|
||||
@@ -0,0 +1,32 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-04-data-audit-backbone.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: Site SQLite retention purge — PurgeExpiredAsync on the writer", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: Site retention options", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 3, "subject": "Task 3: Site retention hosted job + registration + design doc", "status": "pending", "blockedBy": [1, 2] },
|
||||
{ "id": 4, "subject": "Task 4: Maintenance command timeout on partition switch + per-channel purge", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 5, "subject": "Task 5: Purge-failure health event (not just a log line)", "status": "pending", "blockedBy": [4] },
|
||||
{ "id": 6, "subject": "Task 6: Fix purger role grants (migration)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 7, "subject": "Task 7: Honest append-only enforcement story (docs)", "status": "pending", "blockedBy": [6] },
|
||||
{ "id": 8, "subject": "Task 8: SiteCalls filtered non-terminal index (migration)", "status": "pending", "blockedBy": [6] },
|
||||
{ "id": 9, "subject": "Task 9: Bound the execution-tree edge scan with a root time window", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 10, "subject": "Task 10: Fix the hardcoded-zero backlogTotal KPI history", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 11, "subject": "Task 11: SiteCalls same-rank freshness upsert (unfreeze RetryCount/LastError)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 12, "subject": "Task 12: EF Core connection resiliency (EnableRetryOnFailure)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 13, "subject": "Task 13: Audit row for operator Retry on parked notifications", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 14, "subject": "Task 14: Operator identity on the SiteCalls Retry/Discard relay", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 15, "subject": "Task 15: Composite keyset in the SiteCalls reconciliation pull contract (site side)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 16, "subject": "Task 16: Central keyset cursor — un-pin the SiteCalls reconciliation loop", "status": "pending", "blockedBy": [15] },
|
||||
{ "id": 17, "subject": "Task 17: Durable record for permanently-abandoned reconciliation rows", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 18, "subject": "Task 18: Reconciliation NodeB failover (owned from report 08)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 19, "subject": "Task 19: Batch the KpiSample purge", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 20, "subject": "Task 20: NotificationOutbox KPI — single-query aggregation, bounded oldest", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 21, "subject": "Task 21: OPTION (RECOMPILE) on the SiteCalls catch-all query", "status": "pending", "blockedBy": [11] },
|
||||
{ "id": 22, "subject": "Task 22: Short first-tick delay on the three purge timers", "status": "pending", "blockedBy": [5, 16] },
|
||||
{ "id": 23, "subject": "Task 23: Low-severity cleanup batch (param chunking, KPI catalog, computed-column guard)", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 24, "subject": "Task 24: Design-doc reconciliation pass", "status": "pending", "blockedBy": [7, 17] },
|
||||
{ "id": 25, "subject": "Task 25: Bounded parallel notification delivery", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 26, "subject": "Final verification: full solution build + test sweep (infra up for MSSQL suites)", "status": "pending", "blockedBy": [3, 8, 9, 10, 12, 14, 18, 19, 20, 21, 22, 23, 24, 25] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
@@ -0,0 +1,632 @@
|
||||
# Templates, Deployment & Transport Fix Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Close every finding in `archreview/05-templates-deployment-transport.md` — the Transport import silent-data-loss family (inheritance, script fields, native alarm sources, AreaName), the flattener/collision repeated-composition bug, revision-hash staleness blindness, native-alarm lock enforcement, Roslyn compile cost/leak on read paths, deployment audit/delete defects, script-trust hardening, and the cross-cutting script-artifact invalidation contract.
|
||||
|
||||
**Architecture:** The Transport import apply path (`BundleImporter`) re-implements DTO→entity mapping by hand and drifted from the DTO layer; fixes restore field-for-field fidelity, add the missing inheritance/native-source passes, and single-source the diff/sync equality predicates so they cannot drift again, guarded structurally by a per-entity round-trip equivalence test suite. TemplateEngine fixes convert the shared cycle-guard `visited` set to a recursion-path set (composition is by-slot, not by-template), extend the revision hash/diff to the native-alarm member class, and thread `IsLocked` through `ResolvedNativeAlarmSource`. A new Commons seam (`IScriptArtifactChangeBus` + `ScriptArtifactsChanged`) defines the "script artifact changed → invalidate everywhere" contract; Transport publishes post-commit, plan 06 consumes it in the Inbound API.
|
||||
|
||||
**Tech Stack:** C#/.NET, EF Core (in-memory + MSSQL integration fixtures), Roslyn (`Microsoft.CodeAnalysis.CSharp.Scripting`), xUnit. Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per project: `dotnet test tests/<project>`.
|
||||
|
||||
## Findings Coverage
|
||||
|
||||
| Report finding | Severity | Task(s) |
|
||||
|---|---|---|
|
||||
| Bundle import drops template inheritance edges | Critical | 1, 2 |
|
||||
| Imported scripts lose `MinTimeBetweenRuns`/`ExecutionTimeoutSeconds`; `LockedInDerived` not in DTOs | High | 3, 4 |
|
||||
| Template `NativeAlarmSources` not transported (overrides dangle) | High | 5 |
|
||||
| Flattener drops grandchild members on repeated composition; CollisionDetector blind | High | 9, 10 |
|
||||
| Revision hash and diff omit `NativeAlarmSources` | High | 11, 12 |
|
||||
| Import leaves stale compiled Inbound API handlers | High | 14 (contract + Transport publisher; consumer → **plan 06**) |
|
||||
| Locked native alarm sources overridable at instance level | High | 13 |
|
||||
| Post-success audit failure flips Success→Failed | Medium | 17 |
|
||||
| Deleting `NotDeployed` instance requires live site round-trip | Medium | 18 |
|
||||
| Import blocker heuristic hard-blocks on false positives | Medium | 19 |
|
||||
| `OperationLockManager` single-node in-memory | Medium | 25 (documented invariant); active-node gating **deferred to plans 01/07** (cluster routing ownership) |
|
||||
| Every validation run compiles every script (CPU + assembly leak) | High (perf) | 15, 16 |
|
||||
| `LineDiffer` O((N+M)²) memory, no input cap | Medium (perf) | 22 |
|
||||
| Templates with folder/parent/composition never diff `Identical` | Medium (perf) | 23 |
|
||||
| `LoadAsync` O(bundle) + unbounded session count | Low (perf) | 24 (session cap); manifest-peek `ReadManifestAsync` **Deferred** — already doc-acknowledged (`Component-Transport.md:90`), pure optimisation |
|
||||
| Import apply is one long EF transaction | Low (perf) | **Deferred** — Task 16 removes the dominant cost (Roslyn in `StaleInstanceProbe`); restructuring the single-transaction rollback contract is high-risk for low residual gain |
|
||||
| Deny-list gaps: `Environment.Exit`/`FailFast`/`GetEnvironmentVariable` | Medium (sec) | 21 |
|
||||
| `System.Data` provider namespaces unbounded network hole | Medium (sec) | 21 |
|
||||
| Reflection-gateway member list incomplete | Medium (sec) | 21 |
|
||||
| Transport import bypasses script trust gate | Medium (sec) | 20 |
|
||||
| Instance alarm overrides implement less than spec | Medium | 25 (doc aligned to implementation — adding `DescriptionOverride`/`OnTriggerScriptOverride` columns is a feature, not a fix; decision recorded) |
|
||||
| `ArtifactDiff` equality predicates drift from sync predicates | Medium | 6 |
|
||||
| Spec advertises unimplemented `scripts/` bundle dir; `MaxBundleEntryCount = 4` | Low | 25 |
|
||||
| `TryCompile` surfaces only first violation/error | Low | 24 |
|
||||
| RevisionHash determinism contract fragile | Low | 11 (guard test extended; migration note) |
|
||||
| Underdeveloped 1: template fidelity shipped behind site/instance wave | — | 1–8 |
|
||||
| Underdeveloped 2: no derived-template import test | — | 1, 8 |
|
||||
| Underdeveloped 3: no repeated-composition flattening test | — | 9, 10 |
|
||||
| Underdeveloped 4: deferred-but-load-bearing (rename call-site rewriting, Transport-012 UI, `ReadManifestAsync`) | — | 25 documents the rename limitation; Transport-012 UI + `ReadManifestAsync` **Deferred** (acknowledged backlog, not correctness) |
|
||||
| Underdeveloped 5: preview→apply window has no optimistic concurrency | — | **Deferred** — version fields are documented as reserved (`ArtifactDiff.cs:33-37`); 30-min TTL window accepted, recorded in Task 25 doc sweep |
|
||||
| Underdeveloped 6: three hand-maintained compile-surface mirrors | — | **Deferred** — reflection/representative-script guard tests already exist; source-generator is an improvement, not a defect |
|
||||
| Underdeveloped 7: `SemanticValidator` leaf-name fallback silently accepts wrong-child calls | — | 24 (emit warning) |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Wire template inheritance edges in the import apply path
|
||||
|
||||
**Classification:** high-risk (Critical data-loss fix on the import data contract)
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9, 10, 11, 12, 13, 15, 16, 17, 18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (add `ResolveInheritanceEdgesAsync`; call it from `ApplyAsync` after line 1044 `ResolveCompositionEdgesAsync`; extend Overwrite branch ~1466-1484)
|
||||
- Test (new): `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/InheritanceImportTests.cs`
|
||||
|
||||
1. Write failing test (follow the fixture pattern of `Import/SiteInstanceImportTests.cs` — source context → `BundleExporter.ExportAsync` → target context → `LoadAsync`/`PreviewAsync`/`ApplyAsync`):
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task Import_DerivedTemplate_WiresParentTemplateId()
|
||||
{
|
||||
// Source: base "Pump" + derived "Pump.WaterPump" (ParentTemplateId -> Pump)
|
||||
// with one inherited-from-base attribute on Pump.
|
||||
var bundle = await ExportTemplatesFromSourceAsync("Pump", "Pump.WaterPump");
|
||||
await ImportIntoEmptyTargetAsync(bundle, ResolutionAction.Add);
|
||||
|
||||
var target = await TargetTemplateRepo.GetAllTemplatesAsync();
|
||||
var basePump = target.Single(t => t.Name == "Pump");
|
||||
var derived = target.Single(t => t.Name == "Pump.WaterPump");
|
||||
Assert.Equal(basePump.Id, derived.ParentTemplateId); // FAILS today: null
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task Import_Overwrite_UpdatesParentEdge_AndRenamedBaseIsHonoured()
|
||||
{
|
||||
// Bundle base renamed to "Pump2" via ResolutionAction.Rename; derived's
|
||||
// BaseTemplateName ("Pump") must resolve through the rename map to Pump2.
|
||||
...
|
||||
Assert.Equal(pump2.Id, derived.ParentTemplateId);
|
||||
}
|
||||
```
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests --filter FullyQualifiedName~InheritanceImportTests` → expect **FAIL** (ParentTemplateId null).
|
||||
3. Implement `ResolveInheritanceEdgesAsync(IReadOnlyList<TemplateDto> dtos, Dictionary<(string,string), ImportResolution> resolutionMap, string user, CancellationToken ct)` mirroring `ResolveCompositionEdgesAsync` (line 1993):
|
||||
- For each non-Skip template DTO: compute the template's *persisted* name (Rename → `RenameTo`), load the tracked entity by name.
|
||||
- If `dto.BaseTemplateName is null` → set `ParentTemplateId = null` (an Overwrite from a root-template bundle must clear a stale edge).
|
||||
- Else resolve the base name through the resolution map (`ResolveOrDefault(resolutionMap, "Template", dto.BaseTemplateName)`; Rename → `RenameTo`), then look up by name across staged-imported + pre-existing target templates (same lookup set `ResolveCompositionEdgesAsync` builds). Set `ParentTemplateId`.
|
||||
- Unresolvable base → leave null and emit a `BundleImportBaseTemplateUnresolved` audit row (mirror `BundleImportAlarmScriptUnresolved`), never throw.
|
||||
- Call from `ApplyAsync` immediately after `ResolveCompositionEdgesAsync` (line 1044).
|
||||
4. Run test → expect **PASS**. Run the whole Transport suites: `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.Tests && dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests`.
|
||||
5. Update `docs/requirements/Component-Transport.md` apply-pass list (the second-pass rewire section) to name the inheritance pass alongside alarm-script and composition rewires.
|
||||
6. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Transport tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests docs/requirements/Component-Transport.md && git commit -m "fix(transport): wire template inheritance edges on bundle import (C3) — derived templates no longer land as roots"`
|
||||
|
||||
### Task 2: Import-time acyclicity check over the merged template graph
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 9, 10, 11, 12, 13, 15, 16, 17, 18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (after the Task-1 inheritance pass in `ApplyAsync`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/InheritanceImportTests.cs` (extend)
|
||||
|
||||
1. Write failing test: craft a bundle whose templates form an inheritance cycle against a pre-existing target template (target `A` has parent `B`; bundle overwrites `B` with `BaseTemplateName: "A"`). Assert `ApplyAsync` throws `SemanticValidationException` and the transaction rolled back (target `B.ParentTemplateId` unchanged). Add the composition-cycle twin (bundle composition edge closing a loop through a target template).
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests --filter FullyQualifiedName~InheritanceImportTests` → **FAIL** (cycle persists today; importer performs no acyclicity check).
|
||||
3. Implement: after both rewire passes, load all templates + compositions from the tracked context and run TemplateEngine `CycleDetector.DetectInheritanceCycle` / `DetectCompositionCycle` (Transport already references TemplateEngine). On a detected cycle, throw `SemanticValidationException(new[] {"Import would create a template inheritance/composition cycle: <path>"})` — the existing catch path rolls back and reports.
|
||||
4. Run test → **PASS**. Commit: `git commit -m "fix(transport): reject bundle imports that would persist an inheritance/composition cycle"` (with the touched paths in `git add`).
|
||||
|
||||
### Task 3: Carry `MinTimeBetweenRuns` / `ExecutionTimeoutSeconds` through import
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`BuildTemplate` ~1550-1560; `SyncTemplateScriptsAsync` `changed` predicate + copies ~1836-1851 and the add branch ~1870-1877)
|
||||
- Test (new): `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/TemplateScriptFidelityTests.cs`
|
||||
|
||||
1. Failing test: export a template whose script has `MinTimeBetweenRuns = TimeSpan.FromSeconds(30)` and `ExecutionTimeoutSeconds = 42`; import as Add into empty target, assert both fields on the persisted `TemplateScript`. Second test: Overwrite a target whose script differs *only* in `ExecutionTimeoutSeconds` → assert field updated and a `TemplateScriptUpdated` audit row emitted.
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests --filter FullyQualifiedName~TemplateScriptFidelityTests` → **FAIL**.
|
||||
3. Implement: add `MinTimeBetweenRuns = s.MinTimeBetweenRuns, ExecutionTimeoutSeconds = s.ExecutionTimeoutSeconds` to the `TemplateScript` initializers in `BuildTemplate` and the `SyncTemplateScriptsAsync` add branch; add `|| current.MinTimeBetweenRuns != scriptDto.MinTimeBetweenRuns || current.ExecutionTimeoutSeconds != scriptDto.ExecutionTimeoutSeconds` to the `changed` predicate and copy both fields in the update block.
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(transport): carry script cadence/timeout fields through bundle import (was silently reset to defaults)"`
|
||||
|
||||
### Task 4: Add `LockedInDerived` to template child DTOs and apply paths
|
||||
|
||||
**Classification:** high-risk (bundle data contract change — must stay additive)
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntityDtos.cs` (`TemplateAttributeDto`, `TemplateAlarmDto`, `TemplateScriptDto` — trailing `bool LockedInDerived = false`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs` (export ~55-90; `FromBundleContent` ~330-360)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`BuildTemplate`, three `SyncTemplate*Async` predicates/copies)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Serialization/EntitySerializerTests.cs` (extend), `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/TemplateScriptFidelityTests.cs` (extend)
|
||||
|
||||
1. Failing unit test in `EntitySerializerTests`: entity with `LockedInDerived = true` on an attribute, alarm, and script → `ToBundleContent` DTOs carry it; `FromBundleContent` round-trips it. Failing integration test: import as Add → persisted flags true; old-form JSON (no field) still deserializes with `false` (backward-compat assertion — deserialize a hand-written DTO JSON blob missing the property).
|
||||
2. Run both test projects with `--filter LockedInDerived` → **FAIL** (compile errors first — that counts; fix signatures then re-run for behavioural red).
|
||||
3. Implement: trailing optional parameter on each record (mirrors the `ExecutionTimeoutSeconds = null` precedent — additive, no `schemaVersion` bump needed); populate at export; consume in `BuildTemplate`, `FromBundleContent`, and all three sync helpers (`changed` predicate + copy).
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(transport): transport LockedInDerived on template attributes/alarms/scripts (additive DTO fields)"`
|
||||
|
||||
### Task 5: Transport template `NativeAlarmSources` end-to-end
|
||||
|
||||
**Classification:** high-risk (new bundle payload class + new sync pass)
|
||||
**Estimated implement time:** ~5 min (DTO+export+apply) — if it runs long, split diff wiring into its own commit
|
||||
**Parallelizable with:** 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntityDtos.cs` (new `TemplateNativeAlarmSourceDto`; `TemplateDto` gains trailing `IReadOnlyList<TemplateNativeAlarmSourceDto> NativeAlarmSources` defaulting empty)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs` (export + `FromBundleContent`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`BuildTemplate`; new `SyncTemplateNativeAlarmSourcesAsync` called from the Overwrite branch at ~1482)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/ArtifactDiff.cs` (`CompareTemplate` — `DiffChildren` over native sources)
|
||||
- Test (new): `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/NativeAlarmSourceImportTests.cs`; extend `EntitySerializerTests`
|
||||
|
||||
1. Failing tests:
|
||||
```csharp
|
||||
[Fact] public async Task Import_Add_CarriesTemplateNativeAlarmSources()
|
||||
// template with one source (all fields set incl. IsLocked, IsInherited, LockedInDerived)
|
||||
// → export → import Add → persisted TemplateNativeAlarmSource field-equal.
|
||||
|
||||
[Fact] public async Task Import_Overwrite_SyncsNativeAlarmSources()
|
||||
// target source differs in ConditionFilter; bundle also drops one source and adds one
|
||||
// → assert update/add/delete applied + audit rows (TemplateNativeAlarmSource{Added,Updated,Deleted}).
|
||||
|
||||
[Fact] public async Task Preview_DiffsNativeAlarmSources()
|
||||
// changed source shows a "NativeAlarmSources.<name>" FieldChange; identical → no change row.
|
||||
|
||||
[Fact] public async Task Import_InstanceOverride_NoLongerDangles()
|
||||
// export template-with-source + instance override of it → import → flatten target instance
|
||||
// → resolved config contains the source with the override applied (today: silently dropped).
|
||||
```
|
||||
2. Run `--filter NativeAlarmSourceImportTests` → **FAIL**.
|
||||
3. Implement:
|
||||
- DTO: `record TemplateNativeAlarmSourceDto(string Name, string? Description, string ConnectionName, string SourceReference, string? ConditionFilter, bool IsLocked, bool IsInherited, bool LockedInDerived)`.
|
||||
- Export in `EntitySerializer.ToBundleContent` (include `IsInherited` placeholder rows — the collision detector depends on them) and consume in `FromBundleContent`.
|
||||
- `BuildTemplate`: copy the collection. New `SyncTemplateNativeAlarmSourcesAsync` mirrors `SyncTemplateAttributesAsync` (name-keyed add/update/delete; repository has `DeleteTemplateNativeAlarmSourceAsync`-style members — verify exact name on `ITemplateEngineRepository` and add if missing).
|
||||
- `ArtifactDiff.CompareTemplate`: `DiffChildren(existing.NativeAlarmSources, incoming.NativeAlarmSources, e => e.Name, i => i.Name, NativeAlarmSourcesEqual, "NativeAlarmSources", changes)`.
|
||||
4. Run → **PASS**; run full Transport suites. Update `docs/requirements/Component-Transport.md` transported-entity table + `CLAUDE.md` component #24 blurb (template native alarm sources now travel).
|
||||
5. Commit: `git commit -m "fix(transport): transport template NativeAlarmSources — imports no longer amputate native alarm mirrors nor dangle instance overrides"`
|
||||
|
||||
### Task 6: Single-source per-entity field comparers for diff + sync
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/TemplateChildEquality.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/ArtifactDiff.cs` (replace `AttributesEqual`/`AlarmsEqual`/`ScriptsEqual` at 646-666; `CompareTemplate` passes a script-name resolver for the alarm on-trigger ref)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (three sync `changed` predicates delegate to the same class)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Import/ArtifactDiffTests.cs` (extend)
|
||||
|
||||
1. Failing tests (today's drift, both directions):
|
||||
```csharp
|
||||
[Fact] public void CompareTemplate_AttributeElementDataTypeChange_IsModified() // omitted today → false Identical
|
||||
[Fact] public void CompareTemplate_ScriptExecutionTimeoutChange_IsModified() // omitted today
|
||||
[Fact] public void CompareTemplate_AlarmOnTriggerScriptChange_IsModified() // omitted today
|
||||
[Fact] public void CompareTemplate_LockedInDerivedChange_IsModified()
|
||||
```
|
||||
2. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.Tests --filter ArtifactDiffTests` → **FAIL**.
|
||||
3. Implement `TemplateChildEquality`: one static method per entity comparing the **complete** writable field list (attributes: Value/DataType/ElementDataType/IsLocked/LockedInDerived/Description/DataSourceReference; alarms: + `OnTriggerScriptName` via a `Func<int?, string?> scriptNameById` resolver built from `existing.Scripts`; scripts: + MinTimeBetweenRuns/ExecutionTimeoutSeconds/LockedInDerived; native sources per Task 5). XML-doc each with: *"This is the single source of truth for '<entity> changed'; ArtifactDiff and SyncTemplate\*Async must both call it."* Point `ArtifactDiff` and the three (four after Task 5) sync predicates at it.
|
||||
4. Run Transport unit + integration suites → **PASS**.
|
||||
5. Commit: `git commit -m "refactor(transport): single-source template child equality — diff and overwrite-sync can no longer drift"`
|
||||
|
||||
### Task 7: Export real `AreaName` (finish the half-shipped Area transport)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1–4, 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Export/DependencyResolver.cs` / `ResolvedExport.cs` (aggregate gains `areaNameById` for exported instances; exporter has `ScadaBridgeDbContext` — load `Areas` for the instances' `AreaId`s)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Serialization/EntitySerializer.cs` (~247-256: replace the `AreaName: null` TODO with the lookup)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/SiteInstanceImportTests.cs` (extend) or new `AreaTransportTests.cs`
|
||||
|
||||
1. Failing test: source instance assigned to Area "Line-1" under its site → export → import into target with `--create-missing` site mapping → assert target instance's `AreaId` resolves to an Area named "Line-1" under the target site (the importer's `GetOrCreate` machinery at `BundleImporter.cs:3168` is already built and tested — only the exporter feeds it null).
|
||||
2. Run → **FAIL** (imported `AreaId` is null).
|
||||
3. Implement: extend the export aggregate with the instances' area id→name map (single query over `Areas` filtered to referenced ids) and emit `AreaName: areaNameById.TryGetValue(inst.AreaId ?? -1, out var an) ? an : null`. Delete the TODO comment.
|
||||
4. Run → **PASS**. Update `docs/requirements/Component-Transport.md` (Area-by-name now actually travels; remove any "importer leaves AreaId null" caveat). CLAUDE.md #24 already claims this — now true, no edit needed (verify wording).
|
||||
5. Commit: `git commit -m "fix(transport): export instance AreaName — Area-by-name reconciliation was half-shipped (exporter hardcoded null)"`
|
||||
|
||||
### Task 8: Round-trip export→import equivalence test suite (Theme-3 structural guard)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9–18, 21, 22 (test-only; depends on Tasks 3–5, 7 landing first)
|
||||
**Files:**
|
||||
- Create: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/RoundTripEquivalenceTests.cs`
|
||||
|
||||
1. For **every** transported entity type (Template + attribute/alarm/script/native-source/composition children, SharedScript, ExternalSystem + methods, DatabaseConnection, NotificationList, SmtpConfig, SmsConfig, ApiMethod, TemplateFolder, Site, DataConnection, Instance + override children + bindings), seed a source DB with a *maximally populated* entity (every writable property non-default), export, import into a fresh target, reload, and compare **by reflection**: every public writable property of the entity must be equal, except an explicit per-type exclusion list (`Id`, FK id columns remapped by design, timestamps). The reflection sweep is the point — a future entity property that no DTO carries fails the test instead of silently vanishing (exactly how `LockedInDerived`, cadence/timeout, and native sources were lost).
|
||||
```csharp
|
||||
private static void AssertRoundTripEqual<T>(T source, T imported, params string[] excluded)
|
||||
{
|
||||
foreach (var p in typeof(T).GetProperties().Where(p => p.CanWrite
|
||||
&& !excluded.Contains(p.Name)
|
||||
&& p.PropertyType.IsValueType || p.PropertyType == typeof(string)))
|
||||
Assert.True(Equals(p.GetValue(source), p.GetValue(imported)),
|
||||
$"{typeof(T).Name}.{p.Name} did not survive export→import");
|
||||
}
|
||||
```
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests --filter RoundTripEquivalenceTests` — expect **PASS** if Tasks 1–7 are complete (any failure here is a residual fidelity hole: fix it in the importer/exporter, not the test).
|
||||
3. Commit: `git commit -m "test(transport): per-entity reflection round-trip equivalence suite — structural guard against silent import data loss"`
|
||||
|
||||
### Task 9: Fix repeated-composition member loss in `FlatteningService`
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1–8, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Flattening/FlatteningService.cs` (the four `ResolveComposed*Recursive` methods: attributes ~285-297, alarms ~648-660, native sources ~761-772, scripts ~913-924)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Flattening/FlatteningServiceTests.cs` (extend)
|
||||
|
||||
1. Failing test:
|
||||
```csharp
|
||||
[Fact]
|
||||
public void Flatten_SameTemplateComposedTwice_ResolvesNestedMembersUnderBothSlots()
|
||||
{
|
||||
// X composes Y as "y1" and "y2"; Y composes Z as "z"; Z has attribute "Val",
|
||||
// alarm "Alm", script "Run", native source "Src".
|
||||
var result = Flatten(x);
|
||||
Assert.Contains(result.Attributes, a => a.CanonicalName == "y1.z.Val");
|
||||
Assert.Contains(result.Attributes, a => a.CanonicalName == "y2.z.Val"); // FAILS today
|
||||
// + same assertions for Alarms / Scripts / NativeAlarmSources
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void Flatten_CompositionCycle_StillTerminates() // regression guard for the cycle role of `visited`
|
||||
```
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests --filter Flatten_SameTemplateComposedTwice` → **FAIL** (`y2.z.Val` missing).
|
||||
3. Implement: convert the shared `visited` set into a **recursion-path** set in all four methods. Pattern (identical in each):
|
||||
```csharp
|
||||
// Guard on the recursion PATH, not globally: composition is by-slot, so a template
|
||||
// legitimately appears twice under different prefixes; only a template already on
|
||||
// the CURRENT path is a cycle.
|
||||
var pushed = new List<int>();
|
||||
foreach (var composedTemplate in composedChain)
|
||||
if (path.Add(composedTemplate.Id)) pushed.Add(composedTemplate.Id);
|
||||
try
|
||||
{
|
||||
foreach (var composedTemplate in composedChain)
|
||||
{
|
||||
if (!compositionMap.TryGetValue(composedTemplate.Id, out var nested)) continue;
|
||||
foreach (var n in nested)
|
||||
if (!path.Contains(n.ComposedTemplateId)) // cycle guard
|
||||
ResolveComposed…Recursive(n, $"{prefix}.{n.InstanceName}", …, path);
|
||||
}
|
||||
}
|
||||
finally { foreach (var id in pushed) path.Remove(id); }
|
||||
```
|
||||
(Rename the parameter `visited` → `path` so the semantics are explicit.)
|
||||
4. Run the full TemplateEngine test project → **PASS**, no regressions.
|
||||
5. Commit: `git commit -m "fix(template-engine): repeated composition no longer drops nested members — cycle guard keyed on recursion path, not global template id"`
|
||||
|
||||
### Task 10: Fix `CollisionDetector` blindness under repeated composition
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1–8, 15–18, 21, 22 (different file from Task 9)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/CollisionDetector.cs` (`CollectComposedMembers` ~119-144)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/CollisionDetectorTests.cs` (extend)
|
||||
|
||||
1. Failing test: X composes Y twice (`y1`, `y2`); Y composes Z; X also declares a direct attribute named `y2.z.Val` (path-collision). Assert the detector reports the collision (today the second slot's subtree is never collected, so it cannot).
|
||||
2. Run `--filter CollisionDetectorTests` → **FAIL**.
|
||||
3. Implement the same path-set conversion as Task 9: `visited` becomes the current recursion path — add `template.Id` on entry, remove in `finally`; the early-return then only fires on genuine cycles.
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(template-engine): collision detection sees all slots of a repeatedly-composed template"`
|
||||
|
||||
### Task 11: Add `NativeAlarmSources` to `RevisionHashService` (planned hash migration)
|
||||
|
||||
**Classification:** high-risk (revision-hash change ⇒ staleness flags flip for native-source-bearing instances)
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1–8, 15–18, 21, 22 — **after Task 13** (hash includes `IsLocked`)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Flattening/RevisionHashService.cs` (new `HashableNativeAlarmSource`; `HashableConfiguration` gains the property in alphabetical position)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Flattening/RevisionHashServiceTests.cs` (extend, incl. the alphabetical-guard test)
|
||||
|
||||
1. Failing tests:
|
||||
```csharp
|
||||
[Fact] public void ComputeHash_NativeAlarmSourceChange_ChangesHash() // FAILS today
|
||||
[Fact] public void ComputeHash_NoNativeAlarmSources_HashUnchangedFromBaseline()
|
||||
// pin a literal known-good hash for a native-source-free config so the migration
|
||||
// is surgical: only instances that HAVE native sources flip stale.
|
||||
```
|
||||
2. Run `--filter RevisionHashServiceTests` → **FAIL**.
|
||||
3. Implement: `HashableNativeAlarmSource` with alphabetical properties (`CanonicalName`, `ConditionFilter`, `ConnectionName`, `IsLocked`, `SourceReference`); on `HashableConfiguration` add `public List<HashableNativeAlarmSource>? NativeAlarmSources { get; init; }` in alphabetical slot (between `InstanceUniqueName` and `Scripts`), populated **null-when-empty** so `WhenWritingNull` keeps every native-source-free config's hash byte-identical (no global staleness storm — the migration only flags the instances the fix is about). Extend the `HashableRecords_PropertiesDeclaredAlphabetically` guard to the new record. Add a migration note to the class doc-comment and `docs/requirements/Component-TemplateEngine.md` (revision-hash section): *native-source-bearing instances will report stale once after upgrade; redeploy clears it — deliberate.*
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(template-engine): native alarm sources participate in the revision hash — staleness detection restored (deliberate one-time stale flip for affected instances)"`
|
||||
|
||||
### Task 12: Add `NativeAlarmSources` to `DiffService`
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1–8, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Flattening/DiffService.cs` (add added/removed/changed sweep — file currently has zero `NativeAlarm` references)
|
||||
- Modify: the `ConfigurationDiff` type in Commons (`src/ZB.MOM.WW.ScadaBridge.Commons/Types/Flattening/` — additive section)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Flattening/DiffServiceTests.cs` (extend)
|
||||
|
||||
1. Failing test: two flattened configs differing only in one native source's `SourceReference` → diff reports one changed native source; add/remove cases too.
|
||||
2. Run `--filter DiffServiceTests` → **FAIL**.
|
||||
3. Implement following the exact pattern DiffService uses for Alarms (keyed on `CanonicalName`, field-compare `ConnectionName`/`SourceReference`/`ConditionFilter`/`IsLocked`). Additive `NativeAlarmSources` collection on `ConfigurationDiff` (message-contract rule: additive only).
|
||||
4. Run → **PASS**; also run `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests` (`DeploymentComparisonTests` consume the diff). Commit: `git commit -m "fix(template-engine): DiffService covers native alarm sources — Deployments diff view no longer blind to native-alarm edits"`
|
||||
|
||||
### Task 13: Enforce locks on native alarm source overrides
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1–8, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Types/Flattening/FlattenedConfiguration.cs` (`ResolvedNativeAlarmSource` gains `public bool IsLocked { get; init; }` — additive)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Flattening/FlatteningService.cs` (`ResolveInheritedNativeAlarmSources` ~670-710 sets `IsLocked`; `ApplyInstanceNativeAlarmSourceOverrides` ~775-792 adds the lock check)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`HandleSetInstanceNativeAlarmSourceOverride` ~882-909)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Flattening/FlatteningServiceTests.cs`, ManagementService test project (mirror an existing handler test)
|
||||
|
||||
1. Failing tests:
|
||||
```csharp
|
||||
[Fact] public void Flatten_LockedNativeAlarmSource_IgnoresInstanceOverride()
|
||||
// template source IsLocked=true + instance override repointing ConnectionName
|
||||
// → resolved source keeps template values, Source stays "Template". FAILS today.
|
||||
|
||||
[Fact] public async Task SetNativeAlarmSourceOverride_OnLockedSource_Throws()
|
||||
// ManagementActor handler rejects with ManagementCommandException. FAILS today.
|
||||
```
|
||||
2. Run TemplateEngine + ManagementService test filters → **FAIL**.
|
||||
3. Implement:
|
||||
- `ResolveInheritedNativeAlarmSources`: set `IsLocked = binding.IsLocked || lockedNames.Contains(binding.Name)` on the resolved record (keep `lockedNames` derived-shadow logic).
|
||||
- `ApplyInstanceNativeAlarmSourceOverrides`: after the `TryGetValue`, `if (existing.IsLocked) continue;` — exactly mirroring the attribute (`:309`) and alarm (`:337`) paths.
|
||||
- `HandleSetInstanceNativeAlarmSourceOverride`: resolve `IFlatteningPipeline` from `sp` (ManagementService already references DeploymentManager), flatten the instance, find the source by `cmd.SourceCanonicalName`; throw `ManagementCommandException($"Native alarm source '{…}' is locked at the template level and cannot be overridden.")` when locked, and also reject when the canonical name doesn't resolve at all (prevents creating dangling overrides — matching the flattener's cannot-add rule).
|
||||
4. Run → **PASS**. Update `docs/requirements/Component-TemplateEngine.md` §Native-alarm override rules if wording needs the enforcement point named. Commit: `git commit -m "fix(security): enforce template locks on native alarm source overrides at flatten and management-command level"`
|
||||
|
||||
### Task 14: Script-artifact invalidation contract — Commons seam + Transport publisher (plan-06 handoff)
|
||||
|
||||
**Classification:** high-risk (cross-component contract)
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9–13, 15–18, 21, 22
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Scripting/ScriptArtifactsChanged.cs`
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Commons/Interfaces/IScriptArtifactChangeBus.cs`
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Host/Services/InProcessScriptArtifactChangeBus.cs` (+ singleton registration in the Host's central-role service wiring)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (optional ctor dep `IScriptArtifactChangeBus?`; publish after `tx.CommitAsync` at ~1098)
|
||||
- Create: `docs/plans/2026-07-08-script-artifact-invalidation-contract.md`
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/Import/BundleImporterApplyTests.cs` (extend), new Host unit test for the bus
|
||||
|
||||
1. Failing test: apply a bundle that Overwrites an `ApiMethod`, a `SharedScript`, and a `Template`; a recording fake `IScriptArtifactChangeBus` handed to the importer must receive, **after** commit, one `ScriptArtifactsChanged` per kind with the post-resolution (renamed) names. Second test: a failed apply (semantic validation throw) publishes **nothing**.
|
||||
2. Run → **FAIL** (types don't exist — compile-red, then behavioural red).
|
||||
3. Implement the contract:
|
||||
```csharp
|
||||
// Commons/Messages/Scripting/ScriptArtifactsChanged.cs
|
||||
/// <summary>Advisory, at-least-once, published only AFTER the mutating transaction
|
||||
/// commits. Consumers must invalidate any compiled/cached artifact keyed by these
|
||||
/// names, and must ALSO self-heal without the notification (content-hash checks) —
|
||||
/// this bus is in-process per node; cross-node propagation is the consumer's
|
||||
/// responsibility (see docs/plans/2026-07-08-script-artifact-invalidation-contract.md).</summary>
|
||||
public sealed record ScriptArtifactsChanged(
|
||||
string ArtifactKind, // ScriptArtifactKinds.ApiMethod | SharedScript | Template
|
||||
IReadOnlyList<string> Names, // post-resolution (renamed) names
|
||||
string Source, // "BundleImport" | "Management" | "FailoverActivation"
|
||||
DateTimeOffset OccurredAtUtc);
|
||||
public static class ScriptArtifactKinds { public const string ApiMethod = "ApiMethod"; /* … */ }
|
||||
|
||||
// Commons/Interfaces/IScriptArtifactChangeBus.cs
|
||||
public interface IScriptArtifactChangeBus
|
||||
{
|
||||
void Publish(ScriptArtifactsChanged notification);
|
||||
IDisposable Subscribe(Action<ScriptArtifactsChanged> handler);
|
||||
}
|
||||
```
|
||||
`InProcessScriptArtifactChangeBus`: lock-free copy-on-write handler list; `Publish` swallows-and-logs handler exceptions (a bad consumer must never fail an import). `BundleImporter` collects overwritten/renamed names during apply and publishes once per kind **after** `tx.CommitAsync` (never inside the transaction — a notification for a rolled-back write is worse than a missed one).
|
||||
4. Write the contract doc: semantics above + the **explicit handoff table**: *plan 06 implements (a) the `InboundScriptExecutor` subscription that evicts `_scriptHandlers` and `_knownBadMethods` by name, (b) publishing from the ManagementActor `UpdateApiMethod`/shared-script paths (closes the project-memory gotchas), (c) cross-node/failover freshness (revision-keyed handler cache).* Plan 05 ships the seam + the Transport publisher only.
|
||||
5. Run → **PASS**; `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Commit: `git commit -m "feat(commons/transport): ScriptArtifactsChanged invalidation contract + post-commit publish from bundle import (consumer lands in plan 06)"`
|
||||
|
||||
### Task 15: Cache script-compile verdicts by code hash
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1–14, 17, 18, 21, 22
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Validation/ScriptCompileVerdictCache.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Validation/ScriptCompiler.cs` (`TryCompile` consults the cache)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.TemplateEngine.Tests/Validation/ScriptCompilerTests.cs` (extend)
|
||||
|
||||
1. Failing test:
|
||||
```csharp
|
||||
[Fact]
|
||||
public void TryCompile_SameCodeTwice_SecondCallIsCacheHit()
|
||||
{
|
||||
ScriptCompileVerdictCache.Clear();
|
||||
var c = new ScriptCompiler();
|
||||
c.TryCompile("return 1;", "A");
|
||||
var hitsBefore = ScriptCompileVerdictCache.Hits;
|
||||
var r = c.TryCompile("return 1;", "B"); // same code, different name
|
||||
Assert.True(r.IsSuccess);
|
||||
Assert.Equal(hitsBefore + 1, ScriptCompileVerdictCache.Hits);
|
||||
}
|
||||
```
|
||||
2. Run `--filter ScriptCompilerTests` → **FAIL**.
|
||||
3. Implement: static `ConcurrentDictionary<string,(bool ok, string? error)>` keyed `Convert.ToHexString(SHA256.HashData(Encoding.UTF8.GetBytes(code)))` — the verdict is a pure function of code + policy, so a process-wide cache is sound; the policy is compile-time static. Bound at 4096 entries (on overflow, `Clear()` — simple and safe). Expose `Hits`/`Count`/`Clear()` for tests and diagnostics. `TryCompile` keys the cache on code only; the script *name* is formatted into the returned message at read time (store the name-free error, format on return). This removes both the repeat-CPU and the repeat assembly load: an unchanged script compiles exactly once per process lifetime.
|
||||
4. Run → **PASS**; run the full TemplateEngine test project. Commit: `git commit -m "perf(template-engine): cache script compile verdicts by code hash — unchanged scripts compile once per process"`
|
||||
|
||||
### Task 16: Skip script compilation on read-only staleness/comparison paths
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1–15, 21, 22
|
||||
**Files:**
|
||||
- Modify: `IFlatteningPipeline` + `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/FlatteningPipeline.cs` (`FlattenAndValidateAsync(int instanceId, bool validateScripts = true, CancellationToken ct = default)` — additive default param)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentService.cs` (`GetDeploymentComparisonAsync` ~652 passes `validateScripts: false`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/StaleInstanceProbe.cs` (~34 passes `validateScripts: false`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/DeploymentComparisonTests.cs`, `DeploymentServiceTests.cs` (extend)
|
||||
|
||||
1. Failing test: with a template script that does **not** compile, `GetDeploymentComparisonAsync` must still return a comparison (staleness needs the hash, not a compile) — today `FlattenAndValidateAsync` fails and the comparison errors out; and a mock-pipeline assertion that the comparison/probe paths request `validateScripts: false` while `DeployInstanceAsync` keeps `true`.
|
||||
2. Run `--filter DeploymentComparisonTests` → **FAIL**.
|
||||
3. Implement: `FlatteningPipeline` skips `ValidationService.ValidateScriptCompilation` (only that validator — semantic/structural validation stays) when `validateScripts` is false. Deploy path unchanged (`true` default). This removes 100% of Roslyn work (and the non-collectible `InteractiveAssemblyLoader` loads) from the Deployments page sweep and from `BundleImporter.ComputeStaleInstanceIdsAsync`'s per-instance probing.
|
||||
4. Run DeploymentManager + Transport integration suites → **PASS**. Update `docs/requirements/Component-DeploymentManager.md` (comparison path is hash-only by design; deploy gate remains authoritative).
|
||||
5. Commit: `git commit -m "perf(deployment): staleness/comparison paths no longer Roslyn-compile every script — deploy gate remains the authoritative compile"`
|
||||
|
||||
### Task 17: Isolate the post-success deployment audit write
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1–16, 21, 22 (same file as 16/18 — coordinate order, regions don't overlap)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentService.cs` (~330-332)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/DeploymentServiceTests.cs` (extend)
|
||||
|
||||
1. Failing test: audit-service mock throws on the `"Deploy"` `LogAsync`; site responds Success → assert result is Success and the persisted record's `Status` **stays** `Success` (today the catch at :343 flips it to `Failed` — the exact divergence the surrounding comments prevent, and a violation of "audit-write failure NEVER aborts the user-facing action").
|
||||
2. Run → **FAIL**.
|
||||
3. Implement: wrap the `LogAsync` at :330-332 in its own try/catch that only `_logger.LogError`s (mirror `TryLogLifecycleTimeoutAsync` at :1034). Same guard for any other post-terminal-status audit writes in the lifecycle verbs (audit-only; scan `DisableInstanceAsync`/`EnableInstanceAsync` for the same shape and guard if present).
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(deployment): audit-write fault can no longer flip a committed Success deployment to Failed"`
|
||||
|
||||
### Task 18: Delete `NotDeployed` instances without a site round-trip
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 1–16, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentService.cs` (`DeleteInstanceAsync` ~546-585)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/DeploymentServiceTests.cs` (extend)
|
||||
|
||||
1. Failing test: instance in `NotDeployed`, communication-service mock configured to throw (site unreachable) → `DeleteInstanceAsync` must succeed and remove the record without ever calling the communication service (assert `Verify(..., Times.Never)`); Transport-imported instances (always `NotDeployed`) become deletable against uncommissioned sites.
|
||||
2. Run → **FAIL**.
|
||||
3. Implement: after the lock acquisition, `if (instance.State == InstanceState.NotDeployed)` → skip the site command entirely, delete the central record (reuse the existing guarded delete block), audit `"Delete"` with `new { CommandId = (string?)null, LocalOnly = true }`, return success. Matches `StateTransitionValidator.CanDelete`'s documented rationale (:40-49) — the two components currently disagree about the same design sentence.
|
||||
4. Run → **PASS**. Update the matching sentence in `docs/requirements/Component-DeploymentManager.md`. Commit: `git commit -m "fix(deployment): delete of a NotDeployed instance is central-side only — no site round-trip, unreachable sites can't block cleanup"`
|
||||
|
||||
### Task 19: Import blocker heuristic — local-declaration exclusion, denylist additions, warning severity for template scripts
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 9–18, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`RunSemanticValidationAsync` ~3680-3692; `DetectBlockersAsync` ~751-818; `KnownNonReferenceNames` ~853-874; `CollectCallIdentifiers`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/SemanticValidatorImportTests.cs` (extend)
|
||||
|
||||
1. Failing tests:
|
||||
```csharp
|
||||
[Fact] public async Task Apply_TemplateScriptWithLocalPascalCaseMethod_DoesNotHardBlock()
|
||||
// script: "decimal ComputeRate(decimal x) => x * 2; return ComputeRate(Attributes[\"A\"]);"
|
||||
// → ApplyAsync succeeds (today: SemanticValidationException on 'ComputeRate').
|
||||
|
||||
[Fact] public async Task Apply_TemplateScriptWithRegexAndStringBuilder_DoesNotHardBlock()
|
||||
|
||||
[Fact] public async Task Apply_ApiMethodWithUnresolvedReference_StillHardBlocks()
|
||||
// ApiMethod script calling a genuinely-missing ExternalSystem keeps throwing.
|
||||
```
|
||||
2. Run `--filter SemanticValidatorImportTests` → **FAIL**.
|
||||
3. Implement, three edges:
|
||||
- **Locally-declared names:** before the candidate loop, Roslyn-parse each scanned script (`CSharpSyntaxTree.ParseText(code, new CSharpParseOptions(kind: SourceCodeKind.Script))` — Transport gets it transitively; add the package ref if not), collect `LocalFunctionStatementSyntax`/`MethodDeclarationSyntax` identifier names, and exclude them from `referenced` for that script's contributions.
|
||||
- **Denylist additions:** add to `KnownNonReferenceNames`: `Regex, Match, Matches, IsMatch, Replace, Split, StringBuilder, Append, AppendLine, Parse, TryParse, Format, Join, Abs, Round, Min, Max, Floor, Ceiling, Pow, Sqrt, Json, Serialize, Deserialize, Where, Select, First, FirstOrDefault, Any, All, Count, Sum, Average, OrderBy, OrderByDescending, GroupBy, Distinct, Add, Remove, Clear, Contains, ContainsKey, TryGetValue, StartsWith, EndsWith, Substring, Trim, ToUpper, ToLower, HAVING, VALUES, DELETE, DISTINCT, LIMIT` (comment: *the list will still drift — that is why template-script findings are warnings, below*).
|
||||
- **Severity split:** track which candidate names came from *template* scripts vs *ApiMethod* scripts. Template-script findings become **warnings** — logged + appended to the `ImportResult`/preview blocker list with a non-blocking flag (the deploy gate re-validates authoritatively at deploy time, `ScriptCompiler.TryCompile`); **ApiMethod** findings remain hard errors (`SemanticValidationException`) because no downstream design-time gate exists for them. Apply the identical split in `DetectBlockersAsync` so preview and apply agree.
|
||||
4. Run → **PASS**; full Transport integration suite. Update `docs/requirements/Component-Transport.md` blocker-row section (template-script name findings are advisory; ApiMethod findings block).
|
||||
5. Commit: `git commit -m "fix(transport): blocker heuristic no longer hard-blocks valid imports — local declarations excluded, denylist extended, template-script findings downgraded to warnings"`
|
||||
|
||||
### Task 20: Run the script trust gate on bundle import
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 9–18, 21, 22 (same `BundleImporter` region as Task 19 — do after 19)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`RunSemanticValidationAsync` Pass 2, ~3694+; `DetectBlockersAsync` for preview parity)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.IntegrationTests/SemanticValidatorImportTests.cs` (extend)
|
||||
|
||||
1. Failing test: bundle whose template script (and, second case, ApiMethod script) contains `System.Diagnostics.Process.Start("cmd")` → `ApplyAsync` throws `SemanticValidationException` naming the forbidden API; preview surfaces the same as a blocker row. Today both import clean — the fifth write path with no trust gate.
|
||||
2. Run → **FAIL**.
|
||||
3. Implement: in Pass 2, for every non-Skip template script, shared script, and ApiMethod script body, call `ScriptTrustValidator.FindViolations(code)` (ScriptAnalysis is referenced via TemplateEngine; add the direct project reference to `ZB.MOM.WW.ScadaBridge.Transport.csproj` if needed). Any violation → **hard error** for all three kinds (trust violations are not the false-positive-prone name heuristic — the semantic verdict is authoritative, same severity everywhere). The bodies are already in memory; Task 15's verdict cache keeps repeat cost nil.
|
||||
4. Run → **PASS**. Update `docs/requirements/Component-ScriptAnalysis.md` call-site list (Transport import is now the fifth delegating call site) and `Component-Transport.md`.
|
||||
5. Commit: `git commit -m "fix(security): bundle import runs the script trust gate — forbidden-API scripts rejected at import review, not at runtime"`
|
||||
|
||||
### Task 21: Harden the `ScriptTrustPolicy` deny-list
|
||||
|
||||
**Classification:** high-risk (security policy; false-positive blast radius on existing deployed scripts)
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1–20, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ScriptAnalysis/ScriptTrustPolicy.cs` (`ForbiddenScopes` :36-45, `ReflectionGatewayMembers` :66-88)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ScriptAnalysis.Tests/ScriptTrustValidatorTests.cs` (extend)
|
||||
|
||||
1. Failing tests:
|
||||
```csharp
|
||||
[Theory]
|
||||
[InlineData("Environment.Exit(0);")]
|
||||
[InlineData("Environment.FailFast(\"x\");")]
|
||||
[InlineData("var s = Environment.GetEnvironmentVariable(\"SCADABRIDGE_API_KEY\");")]
|
||||
[InlineData("var c = new Microsoft.Data.SqlClient.SqlConnection(\"Server=attacker\");")]
|
||||
public void FindViolations_Flags(string code) => Assert.NotEmpty(ScriptTrustValidator.FindViolations(code));
|
||||
|
||||
[Theory] // syntactic gateway closure (matters in TPA-degraded fallback mode)
|
||||
[InlineData("typeof(string).Assembly.GetTypes()")]
|
||||
[InlineData("asm.EntryPoint")]
|
||||
public void SyntacticPass_Flags(string code) ...
|
||||
|
||||
[Theory] // must NOT regress legitimate scripts
|
||||
[InlineData("await Task.Delay(1);")]
|
||||
[InlineData("Func<int,int> f = x => x; f.Invoke(3);")] // delegate Invoke stays allowed — see step 3
|
||||
public void FindViolations_Allows(string code) => Assert.Empty(ScriptTrustValidator.FindViolations(code));
|
||||
```
|
||||
2. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ScriptAnalysis.Tests --filter ScriptTrustValidatorTests` → **FAIL**.
|
||||
3. Implement:
|
||||
- `ForbiddenScopes` += `"System.Environment"` (whole type — no legitimate script use; `Environment.NewLine` callers can use `"\n"`), `"System.GC"`, and the ADO.NET provider namespaces: `"Microsoft.Data"`, `"System.Data.SqlClient"`, `"System.Data.Odbc"`, `"System.Data.OleDb"` (scripts reach `DbConnection` only via the `Database` helper, whose `System.Data.Common` abstract types stay allowed — closes the arbitrary-host `new SqlConnection(...)` channel the `System.Net` deny was meant to cover).
|
||||
- `ReflectionGatewayMembers` += `"GetTypes"`, `"EntryPoint"`, `"DeclaredMethods"`, `"DeclaredMembers"`, `"DeclaredConstructors"`, `"DynamicInvoke"`. **Deliberately exclude `"Invoke"`**: the syntactic pass rejects regardless of receiver, and `delegate.Invoke()` is legitimate; `MethodInfo.Invoke` is already caught semantically (`System.Reflection` scope) — record this rationale in the code comment.
|
||||
4. Run → **PASS**; also run the SiteRuntime/InboundAPI representative-script suites if they exercise the surface (`dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter Script` — any newly-flagged legitimate pattern is a deliberate decision to surface, not silently absorb).
|
||||
5. Update `docs/requirements/Component-ScriptAnalysis.md` deny-list table + the `System.Data` posture paragraph. Commit: `git commit -m "fix(security): deny Environment/GC and ADO.NET provider namespaces; close reflection-gateway list (GetTypes/EntryPoint/Declared*/DynamicInvoke)"`
|
||||
|
||||
### Task 22: Bound `LineDiffer` input size
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1–21
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/LineDiffer.cs`
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Import/LineDifferTests.cs` (extend)
|
||||
|
||||
1. Failing test: two 10,000-line inputs with zero common lines → `Diff` returns within a small time budget with `Truncated = true` and a summary-only result (hunk list empty or `<N lines>` placeholders), and never materializes the Myers trace. Assert e.g. total runtime < 2s (today: ~20k rounds × 160 KB V-snapshots ≈ multi-GB transient allocation — the assertion red-lines as timeout/OOM).
|
||||
2. Run `--filter LineDifferTests` → **FAIL** (or pathological slowness — treat as fail).
|
||||
3. Implement: `private const int MaxInputLines = 4000;` at the top of `Diff`: `if (aLines.Length + bLines.Length > MaxInputLines) return SummaryOnlyResult(aLines.Length, bLines.Length);` producing the same shape the `maxLines` output cap already emits (`Truncated = true`, add/remove totals = full line counts, no hunks). The 400-line *output* cap stays; this bounds the *algorithm input* so a bloated or crafted bundle cannot OOM the active central node from the import preview.
|
||||
4. Run → **PASS**. Commit: `git commit -m "fix(transport): cap LineDiffer input size — oversized script diffs degrade to summary instead of O((N+M)^2) memory"`
|
||||
|
||||
### Task 23: Fix `ArtifactDiff` placeholder-identity comparisons
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~4 min
|
||||
**Parallelizable with:** 9–18, 21, 22 (touches `ArtifactDiff` + preview call site — after Tasks 5/6)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/ArtifactDiff.cs` (`CompareTemplate` gains optional `IReadOnlyDictionary<int,string>? folderNameById`, `IReadOnlyDictionary<int,string>? templateNameById`; `FolderNameOf`/`BaseTemplateNameOf`/`CompositionTargetNameOf` at 674-693 use them)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleImporter.cs` (`PreviewAsync` ~379-388 passes the maps it already builds at ~365-366)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/Import/ArtifactDiffTests.cs` (extend)
|
||||
|
||||
1. Failing test: existing template with `FolderId`, `ParentTemplateId`, and one composition; incoming DTO carries the *matching names* → `CompareTemplate(dto, existing, folderMap, templateMap)` classifies **Identical** (today: `"<id:5>" != "Pump"` ⇒ spurious Modified on every re-import, defeating the wizard's auto-skip contract, `Component-Transport.md:222`).
|
||||
2. Run → **FAIL**.
|
||||
3. Implement: resolve real names through the maps, falling back to the `<id:N>` placeholder only when a map is absent/misses (keeps existing unit tests valid). Wire the two maps through from `PreviewAsync`.
|
||||
4. Run Transport unit + integration suites → **PASS**. Commit: `git commit -m "fix(transport): template diffs resolve folder/base/composition names — unchanged structured templates classify Identical again"`
|
||||
|
||||
### Task 24: Low-severity cleanup batch
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none (touches files owned by Tasks 15 and 19 — run after both)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/Import/BundleSessionStore.cs` + `src/ZB.MOM.WW.ScadaBridge.Transport/TransportOptions.cs` (session cap)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Validation/ScriptCompiler.cs` (:42, :47 — full error lists)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.TemplateEngine/Validation/SemanticValidator.cs` (~43-56 — leaf-fallback warning)
|
||||
- Tests: `BundleSessionStoreTests.cs`, `ScriptCompilerTests.cs`, `SemanticValidatorTests` (Validation folder)
|
||||
|
||||
Three concrete edits, each TDD'd:
|
||||
1. **Session cap:** `TransportOptions.MaxConcurrentImportSessions = 8`. Failing test: 9th `Add` throws `InvalidOperationException("Too many concurrent import sessions …")`; removing one admits a new one. Implement the count check in `BundleSessionStore.Add` (:38). Bounds the N×~200 MB decrypted-content pinning.
|
||||
2. **Full error lists:** failing test: script with two forbidden APIs → failure message contains both. Implement: `string.Join("; ", violations)` / `string.Join("; ", errors)` instead of `violations[0]`/`errors[0]` (coordinate with Task 15's cached-error format — store the joined string).
|
||||
3. **Leaf-fallback warning:** failing test: composed script call resolved only by leaf-name fallback yields a `ValidationEntry.Warning(ValidationCategory.…, "call target '…' matched by leaf name only — child path not verified")` instead of silent acceptance. Implement in the fallback branch at `SemanticValidator.cs:43-56` (return-shape permitting; if the method returns bool, thread a warnings collector the way `ValidationService` already aggregates warnings).
|
||||
4. Run all three filters → **PASS**. Commit: `git commit -m "chore: low-severity sweep — import session cap, full compile-error lists, leaf-name-fallback validation warning"`
|
||||
|
||||
### Task 25: Design-doc + CLAUDE.md sync sweep and deferred-decision ledger
|
||||
|
||||
**Classification:** small (docs only)
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** none (run last)
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/Component-Transport.md`, `Component-TemplateEngine.md`, `Component-DeploymentManager.md`, `Component-ScriptAnalysis.md`, `/Users/dohertj2/Desktop/ScadaBridge/CLAUDE.md` (component #24 blurb + Transport bullet in Key Design Decisions)
|
||||
|
||||
No tests; concrete edits (several were partially made inside their behavior tasks — this pass verifies and completes them, then does a stale-cross-reference sweep per repo convention):
|
||||
1. **Component-Transport.md:** (a) remove the `scripts/` bundle-directory section (:45-47) or mark it explicitly *"not implemented — `MaxBundleEntryCount` default 4 would reject it"*; (b) transported-entity list gains template `NativeAlarmSources`, `LockedInDerived`, script cadence/timeout, real `AreaName`; (c) blocker section documents the warning/error severity split (Task 19) and the LineDiffer input cap (Task 22); (d) make the *rename call-site limitation* explicit and load-bearing: *a Rename resolution does not rewrite call sites in importing scripts — the target DB can end up with scripts calling the old name; Pass-1 registers both names so import passes, but deploy-time validation on the target is the real gate*; (e) note the accepted preview→apply window (no optimistic concurrency; version fields reserved — Underdeveloped 5) as a recorded decision.
|
||||
2. **Component-TemplateEngine.md:** align the alarm-override granularity sentence (:113) to the implementation — only Trigger Definition thresholds and Priority Level are instance-overridable; Description and On-Trigger Script reference are **not** (`InstanceAlarmOverride` carries only `TriggerConfigurationOverride`/`PriorityLevelOverride`); record it as the decision (adding the two override columns is future feature work, not spec debt). Add the revision-hash native-source migration note (Task 11) if not already present.
|
||||
3. **Component-DeploymentManager.md:** `NotDeployed` delete is central-side only (Task 18); post-success audit isolation (Task 17); add the explicit `OperationLockManager` invariant paragraph: *locks are per-node in-memory; mutual exclusion assumes management traffic reaches only the active central node (Traefik). Direct standby-port mutations are unsupported — structural rejection of ops on the non-active node is owned by the cluster/UI plans (01/07).*
|
||||
4. **Component-ScriptAnalysis.md:** deny-list additions + `System.Data` provider posture + Transport import as the fifth delegating call site (Tasks 20, 21).
|
||||
5. **CLAUDE.md:** component #24 blurb — add template native-alarm-source transport and confirm the Area-by-name claim is now true; adjust the "Transport (#24, M8)" Key Design Decisions bullet likewise. Per the umbrella-index rule, no `../scadaproj/CLAUDE.md` change is needed (no wire-relationship/stack change) — verify and note in the commit message.
|
||||
6. `git diff` review, then commit: `git commit -m "docs: sync Transport/TemplateEngine/DeploymentManager/ScriptAnalysis specs + CLAUDE.md with the arch-review fix wave (plan 05)"`
|
||||
|
||||
---
|
||||
|
||||
## Dependencies on other plans
|
||||
|
||||
- **Plan 06 (Edge Integrations) — explicit handoff:** Task 14 ships the `ScriptArtifactsChanged` message, `IScriptArtifactChangeBus` seam, in-process bus, contract doc, and the Transport post-commit publisher. Plan 06 implements the consumers: `InboundScriptExecutor` eviction of `_scriptHandlers`/`_knownBadMethods`, publishing from the ManagementActor ApiMethod/shared-script update paths, and cross-node/failover handler freshness. Plan 06 should not start its consumer work until Task 14's contract doc is committed.
|
||||
- **Plans 01/07:** structural rejection of management mutations on the non-active central node (the `OperationLockManager` residual risk) belongs to the cluster-routing/management-surface owners; Task 25 records the invariant only.
|
||||
- **Plan 08:** also flagged the `AreaName: null` half-shipped feature — the fix is owned here (Task 7); plan 08 should not duplicate it.
|
||||
- **Plan 03:** runtime script execution/containment is untouched by this plan (reference only).
|
||||
|
||||
## Execution order
|
||||
|
||||
**P0 (start immediately, in parallel):** Task 1 → 2 (the Critical, sequential), Task 14 (unblocks plan 06), Task 9 → 10 (flattener correctness), Task 11 (after 13), Task 13.
|
||||
**Wave 2 (BundleImporter chain — same file, serialize):** 3 → 4 → 5 → 6 → 19 → 20 → 23, with 7 in parallel (exporter files) and 15/16/17/18/21/22 in parallel (disjoint projects).
|
||||
**Wave 3:** 8 (equivalence suite — after 3–7), 12, 24.
|
||||
**Last:** 25 (doc sweep), then a full `dotnet build ZB.MOM.WW.ScadaBridge.slnx && dotnet test ZB.MOM.WW.ScadaBridge.slnx` plus the direct `dotnet test tests/ZB.MOM.WW.ScadaBridge.CLI.Tests` run (slnx-exclusion gotcha) before declaring the plan done.
|
||||
@@ -0,0 +1,31 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-05-templates-deployment-transport.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: Wire template inheritance edges in the import apply path", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: Import-time acyclicity check over the merged template graph", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 3, "subject": "Task 3: Carry MinTimeBetweenRuns/ExecutionTimeoutSeconds through import", "status": "pending", "blockedBy": [2] },
|
||||
{ "id": 4, "subject": "Task 4: Add LockedInDerived to template child DTOs and apply paths", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 5, "subject": "Task 5: Transport template NativeAlarmSources end-to-end", "status": "pending", "blockedBy": [4] },
|
||||
{ "id": 6, "subject": "Task 6: Single-source per-entity field comparers for diff + sync", "status": "pending", "blockedBy": [5] },
|
||||
{ "id": 7, "subject": "Task 7: Export real AreaName (finish half-shipped Area transport)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 8, "subject": "Task 8: Round-trip export→import equivalence test suite", "status": "pending", "blockedBy": [5, 6, 7] },
|
||||
{ "id": 9, "subject": "Task 9: Fix repeated-composition member loss in FlatteningService", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 10, "subject": "Task 10: Fix CollisionDetector blindness under repeated composition", "status": "pending", "blockedBy": [9] },
|
||||
{ "id": 11, "subject": "Task 11: Add NativeAlarmSources to RevisionHashService (planned hash migration)", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 12, "subject": "Task 12: Add NativeAlarmSources to DiffService", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 13, "subject": "Task 13: Enforce locks on native alarm source overrides", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 14, "subject": "Task 14: Script-artifact invalidation contract — Commons seam + Transport publisher (plan-06 handoff)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 15, "subject": "Task 15: Cache script compile verdicts by code hash", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 16, "subject": "Task 16: Skip script compilation on read-only staleness/comparison paths", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 17, "subject": "Task 17: Isolate the post-success deployment audit write", "status": "pending", "blockedBy": [16] },
|
||||
{ "id": 18, "subject": "Task 18: Delete NotDeployed instances without a site round-trip", "status": "pending", "blockedBy": [17] },
|
||||
{ "id": 19, "subject": "Task 19: Import blocker heuristic — local-declaration exclusion, denylist additions, warning severity", "status": "pending", "blockedBy": [6] },
|
||||
{ "id": 20, "subject": "Task 20: Run the script trust gate on bundle import", "status": "pending", "blockedBy": [19] },
|
||||
{ "id": 21, "subject": "Task 21: Harden the ScriptTrustPolicy deny-list", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 22, "subject": "Task 22: Bound LineDiffer input size", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 23, "subject": "Task 23: Fix ArtifactDiff placeholder-identity comparisons", "status": "pending", "blockedBy": [20] },
|
||||
{ "id": 24, "subject": "Task 24: Low-severity cleanup batch (session cap, full error lists, semantic-validator warning)", "status": "pending", "blockedBy": [15, 19] },
|
||||
{ "id": 25, "subject": "Task 25: Design-doc + CLAUDE.md sync sweep and deferred-decision ledger", "status": "pending", "blockedBy": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
@@ -0,0 +1,820 @@
|
||||
# Edge Integrations Fix Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Fix every finding in `archreview/06-edge-integrations.md` — make the inbound compiled-handler cache self-healing (revision-checked, failover/direct-SQL/import-safe), close both ESG spec gaps (per-system timeout, path templating), bound outbound response buffering and SMS blast radius, wire the dead SMS retry fields, and clear the report's Medium/Low stability, performance, and convention findings across InboundAPI, ExternalSystemGateway, NotificationService/Outbox delivery, and DelmiaNotifier.
|
||||
|
||||
**Architecture:** The core stability fix replaces the name-keyed compiled-handler cache in `InboundScriptExecutor` with a script-revision-checked cache (the fresh `ApiMethod` row is already in hand on every request — one ordinal string compare detects staleness and triggers recompile), plus a public `InvalidateMethod` seam that plan 05's "script artifact changed → invalidate" contract will consume. The ESG per-system timeout is a full vertical slice per repo rules: Commons entity → EF mapping + migration → artifact message contract (additive) → site SQLite column → client honor → management command → CLI → UI → design doc. Notification fixes are localized to the delivery adapters and the outbox dispatcher's retry-policy resolution.
|
||||
|
||||
**Tech Stack:** C#/.NET 9, xUnit + NSubstitute, EF Core migrations (MS SQL central), Microsoft.Data.Sqlite (site), Roslyn scripting, System.CommandLine (CLI), Blazor Server (UI).
|
||||
|
||||
Build: `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Test per-project: `dotnet test tests/<project>`. EF migrations: **build first, never `--no-build`**; `dotnet ef migrations add <Name> --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host`. If the migration scaffolds empty, delete the files and rebuild first.
|
||||
|
||||
## Findings Coverage
|
||||
|
||||
| # | Finding (report section) | Severity | Task(s) |
|
||||
|---|--------------------------|----------|---------|
|
||||
| S1 | Compiled inbound handler cache stale across central failover (+ known-bad mirror image) | High | 1, 2, 3 |
|
||||
| S2 | Method timeout abandons running script; DI scope disposed underneath (use-after-dispose + orphan accumulation, no counter) | Medium | 4 |
|
||||
| S3 | ESG API-key `AuthConfiguration` colon-splitting ambiguous (also silently misparses the JSON shapes the UI placeholder already suggests) | Medium | 15 |
|
||||
| S4 | DelmiaNotifier failover can duplicate a processed notification (at-least-once undocumented) | Medium | 23 |
|
||||
| S5 | SMS adapter keeps sending after first transient — duplicate amplification on retry | Medium | 17 |
|
||||
| S6 | `ErrorClassifier.IsTransient(Exception)` treats any OCE as transient (misuse-prone public predicate) | Low | 16 |
|
||||
| S7 | First-of-many SMTP configuration nondeterministic (`FirstOrDefault` in adapter + retry-policy resolver) | Low | 21 |
|
||||
| S8 | OAuth2 token cache: unsynchronized field visibility (growth is bounded in practice — accepted) | Low | 21 |
|
||||
| S9 | Non-JSON content-type with body yields misleading 400 instead of 415 | Low | 5 |
|
||||
| P1 | Outbound HTTP responses buffered unbounded | Medium | 14 |
|
||||
| P2 | Serial per-recipient Twilio POSTs inside serial dispatch sweep | Medium | 17 (first-transient short-circuit bounds the black-hole worst case; bounded parallelism **deferred** — at current list sizes short-circuit alone caps the sweep at ~1 timeout) |
|
||||
| P3 | Per-delivery TCP+TLS SMTP handshake | Low | **Won't-fix** — deliberate, documented tradeoff with the pooling seam already left open (`MailKitSmtpClientWrapper.cs:12-38`); report itself says "correct tradeoff at current volume" |
|
||||
| P4 | Startup compiles every inbound method serially | Low | 7 |
|
||||
| C1 | Spec drift: per-system ESG timeout specified but not implemented | High | 8, 9, 10, 11, 12 |
|
||||
| C2 | Spec drift: path templates (`/recipes/{id}`) never substituted | Medium | 13 |
|
||||
| C3 | Spec drift: inbound error responses lack documented `code` field (incl. SITE_UNREACHABLE surfacing as generic 500) | Low | 5 |
|
||||
| C4 | OAuth2 SMTP is Microsoft-365-only despite broader spec | Low | 19, 20 |
|
||||
| C5 | `InboundApiEndpointFilter` freezes options at construction | Low | 6 |
|
||||
| U1 | Script trust boundary is static-only (no runtime sandbox) | Underdeveloped | **Deferred** — restricted-ALC/out-of-process containment is an architecture change already logged as deferred to v1.x in the codebase's own deferred-work inventory; interacts with plan 05's Script Analysis ownership. Not a bite-sized fix. |
|
||||
| U2 | No cache-coherence mechanism for compiled handlers | Underdeveloped | 1, 3 (revision-check fallback works standalone; Task 3 provides the consumer seam for plan 05's invalidation contract) |
|
||||
| U3 | Per-SMS retry settings are dead fields (`SmsConfiguration.MaxRetries`/`RetryDelay` never read) | Underdeveloped | 18 |
|
||||
| U4 | Twilio accept-only + `AccountSid` interpolated un-escaped into URI | Underdeveloped | 22 (SID format validation at save). Status-callback webhook / per-recipient delivery state: **Deferred** — documented out-of-scope in `Component-NotificationService.md:93,102`; needs an inbound webhook endpoint (new attack surface) that warrants its own design pass. |
|
||||
| U5 | DelmiaNotifier duplicate-delivery semantics undocumented; no CI artifact for shipped binary | Underdeveloped | 23 (docs). Windows AOT CI artifact: **Deferred** — repo has no Windows CI runner; the design doc's manual-smoke note stands. |
|
||||
| U6 | Test gaps: handler failover/staleness, oversized outbound body, mid-list SMS transient | Underdeveloped | 1, 14, 17 (each task's failing-test step is exactly the missing test) |
|
||||
| U7 | API key rotation procedure unwritten | Underdeveloped | 24 |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Revision-checked inbound handler cache (self-healing across failover / direct SQL / known-bad)
|
||||
|
||||
**Classification:** high-risk (concurrency; changes the serving semantics of every inbound API method)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 6, 7, 8, 17, 18, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs` (lines 26, 38-63, 118-143, 311-331)
|
||||
- Test (create): `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorStalenessTests.cs`
|
||||
|
||||
Design: the cache value becomes `CachedHandler(string? Script, Func<InboundScriptContext, Task<object?>> Handler)`; `_knownBadMethods` becomes `ConcurrentDictionary<string, string>` (method name → the exact script text that failed). On every `ExecuteAsync`, the freshly-fetched `method.Script` is ordinal-compared against the cached script; mismatch → recompile in place (and purge the known-bad record). A known-bad record only short-circuits when the *current* script is the one that failed — a changed script always gets a fresh compile. `RegisterHandler(name, handler)` (test seam) stores `Script = null`, meaning "no staleness info, serve as-is" — existing tests keep passing; all production compiles flow through `CompileAndRegister`, which always records the script text. This fixes failover staleness, direct-SQL-edit staleness, and stale known-bad in one move; marginal per-request cost is one ordinal string compare of a few-KB script.
|
||||
|
||||
Behavioral consequence (deliberate, see Task 2): the DB row becomes authoritative. A saved-but-broken script now returns HTTP 500 "Script compilation failed" on every node instead of the old delegate silently serving on the node that compiled it — honest failure replaces node-divergent stale success.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Write the failing tests:
|
||||
|
||||
```csharp
|
||||
// tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorStalenessTests.cs
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using NSubstitute;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Entities.InboundApi;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Interfaces.Services;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.InboundAPI.Tests;
|
||||
|
||||
/// <summary>
|
||||
/// Arch-review S1: the compiled-handler cache must be revision-checked against the
|
||||
/// freshly-fetched ApiMethod.Script so a standby node (whose cache was populated at
|
||||
/// its own startup) never serves a stale delegate after failover, and a method fixed
|
||||
/// out-of-band is never stuck behind a stale known-bad record.
|
||||
/// </summary>
|
||||
public class InboundScriptExecutorStalenessTests
|
||||
{
|
||||
private readonly InboundScriptExecutor _executor;
|
||||
private readonly RouteHelper _route;
|
||||
|
||||
public InboundScriptExecutorStalenessTests()
|
||||
{
|
||||
_executor = new InboundScriptExecutor(
|
||||
NullLogger<InboundScriptExecutor>.Instance, Substitute.For<IServiceProvider>());
|
||||
_route = new RouteHelper(
|
||||
Substitute.For<IInstanceLocator>(), Substitute.For<IInstanceRouter>());
|
||||
}
|
||||
|
||||
private Task<InboundScriptResult> Run(ApiMethod m) => _executor.ExecuteAsync(
|
||||
m, new Dictionary<string, object?>(), _route, TimeSpan.FromSeconds(10));
|
||||
|
||||
[Fact]
|
||||
public async Task UpdatedScript_RecompilesInsteadOfServingStaleHandler()
|
||||
{
|
||||
// Simulates the failover scenario: this node compiled v1 at startup...
|
||||
var v1 = new ApiMethod("stale", "return 1;") { Id = 1, TimeoutSeconds = 10 };
|
||||
Assert.True(_executor.CompileAndRegister(v1));
|
||||
var r1 = await Run(v1);
|
||||
Assert.Contains("1", r1.ResultJson);
|
||||
|
||||
// ...the active node updated the method (this node never heard about it);
|
||||
// the per-request DB fetch hands ExecuteAsync the NEW row.
|
||||
var v2 = new ApiMethod("stale", "return 2;") { Id = 1, TimeoutSeconds = 10 };
|
||||
var r2 = await Run(v2);
|
||||
|
||||
Assert.True(r2.Success);
|
||||
Assert.Contains("2", r2.ResultJson); // old cache would have returned 1
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task KnownBadMethod_FixedScript_CompilesInsteadOfFastFailing()
|
||||
{
|
||||
// The mirror-image variant: broken at this node's startup...
|
||||
var broken = new ApiMethod("fixme", "%%% not C# %%%") { Id = 2, TimeoutSeconds = 10 };
|
||||
Assert.False(_executor.CompileAndRegister(broken));
|
||||
var r1 = await Run(broken);
|
||||
Assert.False(r1.Success);
|
||||
|
||||
// ...fixed via the management path on the OTHER node; this node's fresh
|
||||
// DB fetch carries the fixed script.
|
||||
var fixedMethod = new ApiMethod("fixme", "return 42;") { Id = 2, TimeoutSeconds = 10 };
|
||||
var r2 = await Run(fixedMethod);
|
||||
|
||||
Assert.True(r2.Success);
|
||||
Assert.Contains("42", r2.ResultJson);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task KnownBadMethod_SameScript_StillFastFails()
|
||||
{
|
||||
var broken = new ApiMethod("stillbad", "%%% not C# %%%") { Id = 3, TimeoutSeconds = 10 };
|
||||
Assert.False(_executor.CompileAndRegister(broken));
|
||||
|
||||
// Same script text → the fast-fail record must hold (no per-request Roslyn).
|
||||
var again = new ApiMethod("stillbad", "%%% not C# %%%") { Id = 3, TimeoutSeconds = 10 };
|
||||
var r = await Run(again);
|
||||
Assert.False(r.Success);
|
||||
Assert.Contains("Script compilation failed", r.ErrorMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public async Task TestSeamRegisterHandler_WithoutScript_IsNotRevisionChecked()
|
||||
{
|
||||
// RegisterHandler (script-less test seam) pins the handler regardless of row content.
|
||||
var m = new ApiMethod("pinned", "return 0;") { Id = 4, TimeoutSeconds = 10 };
|
||||
_executor.RegisterHandler("pinned", async _ => { await Task.CompletedTask; return 99; });
|
||||
var r = await Run(m);
|
||||
Assert.True(r.Success);
|
||||
Assert.Contains("99", r.ResultJson);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
2. Run: `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests --filter InboundScriptExecutorStalenessTests` — expect **FAIL** (`UpdatedScript_RecompilesInsteadOfServingStaleHandler` returns 1; `KnownBadMethod_FixedScript…` fast-fails).
|
||||
|
||||
3. Minimal implementation in `InboundScriptExecutor.cs`:
|
||||
- Add `private sealed record CachedHandler(string? Script, Func<InboundScriptContext, Task<object?>> Handler);` and change `_scriptHandlers` to `ConcurrentDictionary<string, CachedHandler>`.
|
||||
- Change `_knownBadMethods` to `ConcurrentDictionary<string, string>`; `TryRecordBadMethod(string methodName, string script)` stores the failing script text (cap logic unchanged; `_knownBadMethods[methodName] = script` when already present so a *newer* bad script replaces the record).
|
||||
- `RegisterHandler(name, handler)` → `_scriptHandlers[name] = new CachedHandler(null, handler);` (doc-comment it as the script-less test seam, exempt from revision checks).
|
||||
- `Register(methodName, handler)` → `Register(ApiMethod method, handler)` storing `new CachedHandler(method.Script, handler)`.
|
||||
- In `ExecuteAsync`, replace the lookup block (lines 311-331) with:
|
||||
|
||||
```csharp
|
||||
_scriptHandlers.TryGetValue(method.Name, out var cached);
|
||||
// Revision check: a cached handler compiled from different script text is stale
|
||||
// (standby-after-failover, direct SQL edit, bundle import). Null Script = test seam, exempt.
|
||||
if (cached is { Script: not null }
|
||||
&& !string.Equals(cached.Script, method.Script, StringComparison.Ordinal))
|
||||
{
|
||||
_logger.LogInformation(
|
||||
"API method {Method} script changed since it was cached; recompiling", method.Name);
|
||||
cached = null;
|
||||
}
|
||||
|
||||
if (cached == null)
|
||||
{
|
||||
// Fast-fail only when the CURRENT script text is the one already known bad.
|
||||
if (_knownBadMethods.TryGetValue(method.Name, out var badScript)
|
||||
&& string.Equals(badScript, method.Script, StringComparison.Ordinal))
|
||||
return new InboundScriptResult(false, null, "Script compilation failed for this method");
|
||||
|
||||
var (compiled, _) = Compile(method);
|
||||
if (compiled == null)
|
||||
{
|
||||
TryRecordBadMethod(method.Name, method.Script ?? string.Empty);
|
||||
return new InboundScriptResult(false, null, "Script compilation failed for this method");
|
||||
}
|
||||
|
||||
_knownBadMethods.TryRemove(method.Name, out _);
|
||||
cached = new CachedHandler(method.Script, compiled);
|
||||
_scriptHandlers[method.Name] = cached; // last-write-wins; concurrent first-callers race benignly
|
||||
}
|
||||
|
||||
var result = await cached.Handler(context).WaitAsync(cts.Token);
|
||||
```
|
||||
|
||||
- In `CompileAndRegister` update the failure branch to `TryRecordBadMethod(method.Name, method.Script ?? string.Empty)`.
|
||||
|
||||
4. Run the new filter → **PASS**, then the whole project: `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests` → all green (existing `RegisterHandler`-based tests keep passing via the null-script seam).
|
||||
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorStalenessTests.cs && git commit -m "fix(inbound-api): revision-check compiled handler cache against the fresh ApiMethod row — heals failover/direct-SQL/known-bad staleness"`
|
||||
|
||||
### Task 2: DB-authoritative compile semantics — management warning text + design doc
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 6, 7, 8, 17, 18, 23
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`TryCompileAndDescribe`, ~lines 2646-2660: warning strings)
|
||||
- Modify: `docs/requirements/Component-InboundAPI.md` (the "Direct SQL Warning" section, ~lines 129-131, plus the broken-save behavior description)
|
||||
|
||||
Task 1 changed the truth: after a broken save, the *old* handler no longer keeps serving (the per-request revision check bypasses it and the method 500s with a compile error on every node consistently). The management warning and the component doc must say so, or they now lie.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. In `TryCompileAndDescribe`, replace the `isUpdate` warning string with: `"Script saved but failed to compile; the method will return a compilation error on every request until a compiling version is saved. Compilation errors: {detail}"` (the create-path string is already correct). Update the `CompileAndRegister` XML doc-comment in `InboundScriptExecutor.cs` (lines 105-114) if Task 1 did not already: the "previously registered handler is left in place" sentence must be replaced with the DB-authoritative behavior.
|
||||
2. In `Component-InboundAPI.md`, rewrite the Direct SQL Warning section: direct SQL edits are now picked up on the next request via the per-request script revision check (no restart / no management round-trip needed); note that a broken script serves a 500 compile-failure consistently on all central nodes.
|
||||
3. Verify no stale text remains: `grep -rn "previously registered" src/ZB.MOM.WW.ScadaBridge.ManagementService src/ZB.MOM.WW.ScadaBridge.InboundAPI docs/requirements/Component-InboundAPI.md` — every remaining hit must describe the new semantics.
|
||||
4. `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → green; `dotnet test tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests` (some tests may assert the old warning text — update them to the new string).
|
||||
5. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.ManagementService src/ZB.MOM.WW.ScadaBridge.InboundAPI tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests docs/requirements/Component-InboundAPI.md && git commit -m "docs(inbound-api): DB row is authoritative for method scripts — update broken-save warning + Direct SQL Warning for the revision-checked cache"`
|
||||
|
||||
### Task 3: `InvalidateMethod` seam — the Inbound API consumer of plan 05's invalidation contract
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 8, 9, 10, 13-17, 19, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs` (after Task 1)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`HandleDeleteApiMethod`, ~line 2631: swap `RemoveHandler` for `InvalidateMethod`) (after Task 2)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorStalenessTests.cs` (append)
|
||||
|
||||
Plan 05 designs the general "script artifact changed → invalidate everywhere" contract (bundle import is its consumer to wire). This task gives the Inbound API one obvious, complete entry point: `InvalidateMethod(string methodName)` drops **both** the compiled handler and the known-bad record. Even before plan 05 lands, Task 1's revision check makes invalidation a fast-path optimization rather than a correctness requirement — that is the designed fallback.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (append to `InboundScriptExecutorStalenessTests`):
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task InvalidateMethod_DropsHandlerAndKnownBadRecord()
|
||||
{
|
||||
var good = new ApiMethod("inv", "return 5;") { Id = 5, TimeoutSeconds = 10 };
|
||||
Assert.True(_executor.CompileAndRegister(good));
|
||||
var broken = new ApiMethod("invbad", "%%%") { Id = 6, TimeoutSeconds = 10 };
|
||||
Assert.False(_executor.CompileAndRegister(broken));
|
||||
|
||||
_executor.InvalidateMethod("inv");
|
||||
_executor.InvalidateMethod("invbad");
|
||||
|
||||
Assert.Equal(0, _executor.KnownBadMethodCount);
|
||||
// After invalidation the method lazily recompiles from the row — still serves.
|
||||
var r = await Run(good);
|
||||
Assert.True(r.Success);
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests --filter InvalidateMethod_DropsHandlerAndKnownBadRecord` → **FAIL** (no such method).
|
||||
3. Implement:
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Drops the compiled handler AND any known-bad record for the method. The
|
||||
/// consumption seam for the script-artifact invalidation contract (bundle import,
|
||||
/// out-of-band updates, failover activation). Safe to call for unknown names.
|
||||
/// The per-request revision check in ExecuteAsync remains the correctness
|
||||
/// fallback when an invalidation is missed.
|
||||
/// </summary>
|
||||
public void InvalidateMethod(string methodName)
|
||||
{
|
||||
_scriptHandlers.TryRemove(methodName, out _);
|
||||
_knownBadMethods.TryRemove(methodName, out _);
|
||||
}
|
||||
```
|
||||
|
||||
Keep `RemoveHandler` as a one-line delegate to `InvalidateMethod` (public API preserved), and change `HandleDeleteApiMethod` to call `InvalidateMethod(method.Name)`.
|
||||
4. Run filter → **PASS**; run full `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests`.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorStalenessTests.cs && git commit -m "feat(inbound-api): InvalidateMethod seam purging handler + known-bad — consumer entry point for the script-artifact invalidation contract (plan 05)"`
|
||||
|
||||
### Task 4: Abandoned inbound executions — defer DI-scope disposal + orphan telemetry
|
||||
|
||||
**Classification:** high-risk (concurrency; per-request lifecycle)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 8-16, 17-22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs` (`ExecuteAsync` body + finally, lines 333, 388-394) (after Task 3)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Observability/ScadaBridgeTelemetry.cs` (add abandoned-execution counter, mirroring `RecordInboundApiRequest`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorTests.cs` (append)
|
||||
|
||||
On timeout, `WaitAsync` returns but the handler task keeps running; the `finally` disposes the DI scope underneath it → `ObjectDisposedException` background faults, and orphans accumulate invisibly (no rate limiting on this API). Fix: capture the handler task; in `finally`, dispose the scope only if the task completed — otherwise increment an orphan gauge, log, and attach a continuation that observes the fault, disposes the scope, and decrements the gauge.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test:
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task TimedOutScript_ScopeDisposalDeferred_NoOrphanLeak()
|
||||
{
|
||||
// Handler ignores the token and keeps running past the method timeout.
|
||||
var release = new TaskCompletionSource();
|
||||
_executor.RegisterHandler("slow", async _ => { await release.Task; return 1; });
|
||||
var method = new ApiMethod("slow", "unused") { Id = 9, TimeoutSeconds = 1 };
|
||||
|
||||
var result = await _executor.ExecuteAsync(
|
||||
method, new Dictionary<string, object?>(), _route, TimeSpan.FromMilliseconds(50));
|
||||
|
||||
Assert.False(result.Success);
|
||||
Assert.Contains("timed out", result.ErrorMessage);
|
||||
Assert.Equal(1, _executor.AbandonedExecutionCount); // orphan is COUNTED
|
||||
|
||||
release.SetResult(); // orphan finishes; continuation cleans up
|
||||
await Task.Delay(200);
|
||||
Assert.Equal(0, _executor.AbandonedExecutionCount); // ...and DECREMENTED
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests --filter TimedOutScript_ScopeDisposalDeferred_NoOrphanLeak` → **FAIL** (no `AbandonedExecutionCount`).
|
||||
3. Implement in `ExecuteAsync`:
|
||||
- Add `private long _abandonedExecutions;` and `internal long AbandonedExecutionCount => Interlocked.Read(ref _abandonedExecutions);` (public if the health surface needs it later).
|
||||
- Hoist the handler task: `Task<object?>? handlerTask = null;` declared next to `scope`; replace line 333 with `handlerTask = cached.Handler(context); var result = await handlerTask.WaitAsync(cts.Token);`.
|
||||
- Replace the `finally` block:
|
||||
|
||||
```csharp
|
||||
finally
|
||||
{
|
||||
if (handlerTask is null || handlerTask.IsCompleted)
|
||||
{
|
||||
scope?.Dispose();
|
||||
}
|
||||
else
|
||||
{
|
||||
// The handler outlived the request (non-cooperative script after a
|
||||
// timeout/client abort). Do NOT dispose the scope underneath it —
|
||||
// defer cleanup to the task's own completion, and count the orphan
|
||||
// so repeated timeouts of a slow method are visible.
|
||||
Interlocked.Increment(ref _abandonedExecutions);
|
||||
ScadaBridgeTelemetry.RecordInboundAbandonedExecution(method.Name);
|
||||
_logger.LogWarning(
|
||||
"Abandoned inbound execution for method {Method} is still running; DI scope disposal deferred", method.Name);
|
||||
var abandonedScope = scope;
|
||||
_ = handlerTask.ContinueWith(t =>
|
||||
{
|
||||
_ = t.Exception; // observe the fault so it never surfaces as unobserved
|
||||
abandonedScope?.Dispose();
|
||||
Interlocked.Decrement(ref _abandonedExecutions);
|
||||
}, TaskScheduler.Default);
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
- In `ScadaBridgeTelemetry`, add `RecordInboundAbandonedExecution(string method)` incrementing a counter (`scadabridge.inbound_api.abandoned_executions`) with a bounded `method` tag, copying the existing `RecordInboundApiRequest` pattern.
|
||||
4. Run filter → **PASS**; run full project.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs src/ZB.MOM.WW.ScadaBridge.Commons/Observability/ScadaBridgeTelemetry.cs tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundScriptExecutorTests.cs && git commit -m "fix(inbound-api): defer DI-scope disposal to handler completion and count abandoned executions — closes the timeout use-after-dispose"`
|
||||
|
||||
### Task 5: Inbound error contract — machine-readable `code` field + SITE_UNREACHABLE + 415 for non-JSON bodies
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 8-16, 17-22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/EndpointExtensions.cs` (all `Results.Json(new { error = … })` sites: lines 111-113, 153-155, 184-194, 212-214, 253-255)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundScriptExecutor.cs` (`InboundScriptResult` + catch blocks) (after Task 4)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/RouteHelper.cs` (~line 172: typed exception)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundApiEndpointFilter.cs` (503/413 bodies) (after Task 6)
|
||||
- Modify: `docs/requirements/Component-InboundAPI.md` (error-code table, ~lines 91-97)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/EndpointExtensionsTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/EndpointContentTypeTests.cs` (append)
|
||||
|
||||
Spec documents `{"error": …, "code": "SITE_UNREACHABLE"}`; code emits `{error}` only, and routed site-unreachable failures collapse into a generic 500. Implement the spec (the audience is external integrators). Codes: `UNAUTHORIZED` (401), `NOT_APPROVED` (403 — one shared code for both indistinguishable negatives, preserving enumeration-safety), `INVALID_JSON`, `VALIDATION_FAILED`, `UNSUPPORTED_MEDIA_TYPE` (new 415), `BODY_TOO_LARGE` (413), `STANDBY_NODE` (503), `TIMEOUT`, `SITE_UNREACHABLE`, `SCRIPT_ERROR` (500 catch-all).
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing tests (follow the existing patterns in `EndpointExtensionsTests`/`EndpointContentTypeTests`, which drive the endpoint via a test host):
|
||||
- `ErrorBodies_CarryMachineReadableCode`: unauthorized request → body has `code == "UNAUTHORIZED"`; in-scope-but-invalid-JSON body → `code == "INVALID_JSON"`; missing required parameter → `code == "VALIDATION_FAILED"`.
|
||||
- `NonJsonContentType_WithBody_Returns415`: `POST` with `Content-Type: text/plain` and body `"hello"` → HTTP 415, `code == "UNSUPPORTED_MEDIA_TYPE"`. A request with a body and **no** Content-Type header still parses as JSON (lenient, preserves existing callers).
|
||||
- `RoutedSiteUnreachable_Returns500WithSiteUnreachableCode` (unit-level on the executor): stub `IInstanceRouter.RouteToCallAsync` to return `Success=false, ErrorMessage="Site unreachable"`; script does `await Route.To("X").Call("s")`; assert `result.ErrorCode == "SITE_UNREACHABLE"`.
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests --filter "ErrorBodies_CarryMachineReadableCode|NonJsonContentType_WithBody_Returns415|RoutedSiteUnreachable"` → **FAIL**.
|
||||
3. Implement:
|
||||
- `RouteHelper.cs`: add `public sealed class SiteUnreachableException : InvalidOperationException` and throw it (instead of bare `InvalidOperationException`) when `RouteToCallAsync` reports failure whose message indicates unreachable/no-contact (and from the site-resolution failure path).
|
||||
- `InboundScriptResult` gains `string? ErrorCode = null` (additive positional-with-default). Catch blocks: timeout → `"TIMEOUT"`; add `catch (SiteUnreachableException ex)` before the generic catch → `new InboundScriptResult(false, null, "Site unreachable", "SITE_UNREACHABLE")`; generic → `"SCRIPT_ERROR"`; compile failure → `"SCRIPT_COMPILE_FAILED"`.
|
||||
- `EndpointExtensions.cs`: add a local helper `static IResult Error(string code, string message, int status) => Results.Json(new { error = message, code }, statusCode: status);` and route every error return through it. Content-type branch: if `ContentLength > 0` and `ContentType` is non-empty and does not contain `"json"` (ordinal-ignore-case) → `Error("UNSUPPORTED_MEDIA_TYPE", "Content-Type must be application/json", StatusCodes.Status415UnsupportedMediaType)`. Script-failure branch passes `scriptResult.ErrorCode ?? "SCRIPT_ERROR"` through.
|
||||
- `InboundApiEndpointFilter.cs`: add `code = "STANDBY_NODE"` / `"BODY_TOO_LARGE"` to its two bodies.
|
||||
- Update the doc's failure-body examples/table to the implemented code set.
|
||||
4. Run filters → **PASS**; run the full InboundAPI test project (existing error-shape assertions that check only `error` still pass — the field is additive).
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.InboundAPI tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests docs/requirements/Component-InboundAPI.md && git commit -m "feat(inbound-api): machine-readable error codes per spec (incl. SITE_UNREACHABLE) + 415 for non-JSON bodies"`
|
||||
|
||||
### Task 6: `InboundApiEndpointFilter` hot-reads options via `IOptionsMonitor`
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** 2 min
|
||||
**Parallelizable with:** 1, 2, 7, 8, 17, 18, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundApiEndpointFilter.cs` (lines 26-38, 66)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/EndpointGatingTests.cs` (append)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test: `BodySizeCap_HotReloads_WithoutRestart` — construct the filter with a test `IOptionsMonitor<InboundApiOptions>` whose `CurrentValue` is swapped between two invocations (cap 10 bytes → reject; cap 1 MiB → accept); assert the second request passes. (A minimal `TestOptionsMonitor<T>` stub: implements `CurrentValue` from a mutable field, `Get`→CurrentValue, `OnChange`→null.)
|
||||
2. Run filter test → **FAIL** (ctor takes `IOptions`).
|
||||
3. Change the ctor parameter to `IOptionsMonitor<InboundApiOptions> options`, store the monitor, and read `_options.CurrentValue.MaxRequestBodyBytes` per request (matches the sibling `AuditWriteMiddleware` convention).
|
||||
4. Run → **PASS**; `dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests`.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundApiEndpointFilter.cs tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/EndpointGatingTests.cs && git commit -m "fix(inbound-api): endpoint filter hot-reads MaxRequestBodyBytes via IOptionsMonitor"`
|
||||
|
||||
### Task 7: Parallelize startup compile of inbound methods
|
||||
|
||||
**Classification:** trivial
|
||||
**Estimated implement time:** 2 min
|
||||
**Parallelizable with:** 1, 2, 6, 8, 17, 18, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/Program.cs` (lines 383-387)
|
||||
|
||||
`CompileAndRegister` is safe for concurrent callers (ConcurrentDictionary caches, no shared mutable compile state). Replace the foreach with:
|
||||
|
||||
```csharp
|
||||
Parallel.ForEach(methods, method => executor.CompileAndRegister(method));
|
||||
```
|
||||
|
||||
**Steps:** No practical unit test (Host composition root; behavior identical, latency only). 1. Make the edit with a comment citing the review finding (serial Roslyn compiles stretch the failover-recovery window at hundreds of methods). 2. `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → green. 3. `dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests` → green. 4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Host/Program.cs && git commit -m "perf(host): compile inbound API methods in parallel at startup"`
|
||||
|
||||
### Task 8: `ExternalSystemDefinition.TimeoutSeconds` — entity + EF mapping + migration
|
||||
|
||||
**Classification:** high-risk (schema migration)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 1, 2, 6, 7, 17, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Entities/ExternalSystems/ExternalSystemDefinition.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/ExternalSystemConfiguration.cs`
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Migrations/<timestamp>_AddExternalSystemTimeoutSeconds.cs` (+ Designer, + snapshot update — generated)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests` (entity default), `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests` (mapping round-trip, follow the existing external-system repository test pattern)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (Commons.Tests): `ExternalSystemDefinition_TimeoutSeconds_DefaultsToZeroMeaningUseGatewayDefault` — construct the entity, assert `TimeoutSeconds == 0`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter ExternalSystemDefinition_TimeoutSeconds` → **FAIL** (no property).
|
||||
2. Add to the entity:
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Per-system HTTP round-trip timeout in seconds for all method calls
|
||||
/// (spec: Component-ExternalSystemGateway "Timeout"). 0 = unset — the gateway's
|
||||
/// configured DefaultHttpTimeout applies.
|
||||
/// </summary>
|
||||
public int TimeoutSeconds { get; set; }
|
||||
```
|
||||
|
||||
3. EF mapping: in `ExternalSystemConfiguration`, add `builder.Property(e => e.TimeoutSeconds).HasDefaultValue(0);`.
|
||||
4. Generate the migration — **build first**: `dotnet build ZB.MOM.WW.ScadaBridge.slnx` then `dotnet ef migrations add AddExternalSystemTimeoutSeconds --project src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase --startup-project src/ZB.MOM.WW.ScadaBridge.Host`. Open the generated file and verify it is NOT empty (one `AddColumn<int>` on the external-systems table, default 0); if empty, delete the generated files, rebuild, regenerate.
|
||||
5. Run tests: the Commons filter → **PASS**; `dotnet test tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests` → green.
|
||||
6. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Commons/Entities/ExternalSystems/ExternalSystemDefinition.cs src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase && git add tests/ZB.MOM.WW.ScadaBridge.Commons.Tests && git commit -m "feat(esg): per-system TimeoutSeconds column on ExternalSystemDefinition (spec Component-ExternalSystemGateway Timeout) — entity + EF + migration"`
|
||||
|
||||
### Task 9: Honor the per-system timeout in `ExternalSystemClient.InvokeHttpAsync`
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 10, 11, 12 (Files disjoint)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs` (lines 300-307, 322-323, 345-346)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs` (append)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (use the project's existing stub-handler `HttpClient` pattern):
|
||||
|
||||
```csharp
|
||||
[Fact]
|
||||
public async Task PerSystemTimeout_OverridesGatewayDefault()
|
||||
{
|
||||
// System timeout 1s; handler black-holes for 5s; gateway default is 30s.
|
||||
var system = new ExternalSystemDefinition("slow", "http://x", "none") { TimeoutSeconds = 1 };
|
||||
var method = new ExternalSystemMethod("m", "GET", "/p");
|
||||
SetupRepository(system, method); // existing helper pattern
|
||||
var handler = new DelegateHandler(async (_, ct) =>
|
||||
{
|
||||
await Task.Delay(TimeSpan.FromSeconds(5), ct);
|
||||
return new HttpResponseMessage(HttpStatusCode.OK);
|
||||
});
|
||||
_httpClientFactory.CreateClient(Arg.Any<string>()).Returns(new HttpClient(handler));
|
||||
var client = new ExternalSystemClient(_httpClientFactory, _repository, NullLogger<ExternalSystemClient>.Instance);
|
||||
|
||||
var sw = System.Diagnostics.Stopwatch.StartNew();
|
||||
var result = await client.CallAsync("slow", "m");
|
||||
sw.Stop();
|
||||
|
||||
Assert.False(result.Success);
|
||||
Assert.Contains("Timeout", result.ErrorMessage);
|
||||
Assert.True(sw.Elapsed < TimeSpan.FromSeconds(4), "per-system 1s timeout must apply, not the 30s default");
|
||||
}
|
||||
```
|
||||
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter PerSystemTimeout_OverridesGatewayDefault` → **FAIL** (~30s default would apply; the test waits 5s then sees OK → success).
|
||||
3. Implement: before the CTS creation, `var effectiveTimeout = system.TimeoutSeconds > 0 ? TimeSpan.FromSeconds(system.TimeoutSeconds) : _options.DefaultHttpTimeout;` — use `effectiveTimeout` in `new CancellationTokenSource(effectiveTimeout)` and in both timeout error messages (replacing `_options.DefaultHttpTimeout.TotalSeconds`). Update the stale comment block at lines 300-305 ("has no per-system Timeout field yet") to describe the new resolution order.
|
||||
4. Run → **PASS**; full project run.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs && git commit -m "feat(esg): honor per-system TimeoutSeconds in InvokeHttpAsync (falls back to DefaultHttpTimeout)"`
|
||||
|
||||
### Task 10: Carry TimeoutSeconds through the artifact pipeline to sites
|
||||
|
||||
**Classification:** high-risk (message contract + site SQLite schema)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 9, 11, 12
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/ExternalSystemArtifact.cs` (additive trailing param `int TimeoutSeconds = 0`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs` (~line 194: pass `es.TimeoutSeconds`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs` (`MigrateSchemaAsync`: `TryAddColumnAsync(connection, "external_systems", "timeout_seconds", "INTEGER NOT NULL DEFAULT 0")`; `StoreExternalSystemAsync`: new `int timeoutSeconds` param + column in the upsert)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/DeploymentManagerActor.cs` (~line 1533: pass `es.TimeoutSeconds`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Repositories/SiteExternalSystemRepository.cs` (`MapExternalSystem` + its SELECTs: read `timeout_seconds` into `TimeoutSeconds`)
|
||||
- Check/Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Actors/SiteReplicationActor.cs` (it re-applies `ReplicateArtifacts` — verify its `StoreExternalSystemAsync` call site compiles with the new parameter)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests` (site storage round-trip test — follow the existing `StoreExternalSystemAsync` test if present, else add `SiteStorage_ExternalSystem_TimeoutSeconds_RoundTrips`)
|
||||
|
||||
The message contract change is **additive** (trailing positional with default), per the repo's additive-only evolution rule — an old central sending to a new site yields 0 (= use default), and vice versa deserializes cleanly.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (SiteRuntime.Tests, in-memory/temp-file SQLite): store an external system with `timeoutSeconds: 7` via `SiteStorageService.StoreExternalSystemAsync`, read it back via `SiteExternalSystemRepository.GetExternalSystemByNameAsync`, assert `TimeoutSeconds == 7`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter TimeoutSeconds_RoundTrips` → **FAIL** (compile error / missing column).
|
||||
2. Make the six edits above: add the artifact record's trailing `int TimeoutSeconds = 0`; thread it central-build → site-store → site-read. The SQLite upsert gains `timeout_seconds = excluded.timeout_seconds`.
|
||||
3. Run → **PASS**; then `dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests` and `dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests` (artifact-build assertions may enumerate record fields — update).
|
||||
4. `dotnet build ZB.MOM.WW.ScadaBridge.slnx` → green (catches any other `ExternalSystemArtifact` construction site).
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Artifacts/ExternalSystemArtifact.cs src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ArtifactDeploymentService.cs src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ && git commit -m "feat(esg): TimeoutSeconds travels the artifact pipeline (additive contract) into site SQLite so site-side calls honor it"`
|
||||
|
||||
### Task 11: TimeoutSeconds management commands + CLI
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 9, 10, 12
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/ExternalSystemCommands.cs` (Create/Update records: trailing `int? TimeoutSeconds = null`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (`HandleCreateExternalSystem` ~1599, `HandleUpdateExternalSystem` ~1612: apply `cmd.TimeoutSeconds ?? 0` / preserve-on-null semantics for update)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/ExternalSystemCommands.cs` (create/update: `--timeout-seconds` `Option<int?>`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests` (append to the existing external-system handler tests); `tests/ZB.MOM.WW.ScadaBridge.CLI.Tests` (option parse test — **note: CLI.Tests is not in the slnx; test the csproj directly**)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (ManagementService.Tests): `CreateExternalSystem_PersistsTimeoutSeconds` — send `CreateExternalSystemCommand(..., TimeoutSeconds: 15)`, assert the persisted definition has `TimeoutSeconds == 15`; and `UpdateExternalSystem_NullTimeout_PreservesExisting`. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests --filter TimeoutSeconds` → **FAIL**.
|
||||
2. Implement: additive record params; create handler sets `TimeoutSeconds = cmd.TimeoutSeconds ?? 0`; update handler `if (cmd.TimeoutSeconds is { } ts) def.TimeoutSeconds = ts;`. CLI: add the option to both `create` and `update`, pass through, add `--timeout-seconds` to the table projection if the list output enumerates fields.
|
||||
3. Run → **PASS**; run `dotnet test tests/ZB.MOM.WW.ScadaBridge.CLI.Tests` **directly** (not via slnx).
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/ExternalSystemCommands.cs src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs src/ZB.MOM.WW.ScadaBridge.CLI tests/ && git commit -m "feat(esg): --timeout-seconds on external-system create/update (management command + CLI, additive contract)"`
|
||||
|
||||
### Task 12: TimeoutSeconds in Central UI form + spec doc sync
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 9, 10, 11
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Design/ExternalSystemForm.razor` (numeric input next to the existing `_maxRetries`/`_retryDelaySeconds` fields; load in `OnInitializedAsync`, save in both create/update branches)
|
||||
- Modify: `docs/requirements/Component-ExternalSystemGateway.md` (line 38 area + §"Call Timeout & Error Handling" ~line 99: document per-system `TimeoutSeconds` with 0 = gateway `DefaultHttpTimeout` fallback and that it now ships to sites in the artifact)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests` (bUnit — append to the existing ExternalSystemForm test if one exists; otherwise assert the rendered form contains the `Timeout (seconds)` field)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing bUnit test: render `ExternalSystemForm` (existing test scaffolding), assert a labeled `Timeout (seconds)` input exists and that saving with value 20 passes `TimeoutSeconds == 20` to the repository/command. Run → **FAIL**.
|
||||
2. Implement: `private int _timeoutSeconds;` bound field with helper text "0 = use gateway default"; wire load + both save paths (the form writes the entity directly per the existing pattern). Update the doc.
|
||||
3. Run → **PASS**; `dotnet test tests/ZB.MOM.WW.ScadaBridge.CentralUI.Tests`.
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.CentralUI docs/requirements/Component-ExternalSystemGateway.md tests/ && git commit -m "feat(esg): TimeoutSeconds in ExternalSystem UI form + Component-ExternalSystemGateway doc sync — closes the spec-drift gap"`
|
||||
|
||||
### Task 13: Path-template substitution (`/recipes/{id}`) in `BuildUrl`
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 10, 11, 12 (NOT 9/14/15/16 — same file; run after 9)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs` (`BuildUrl` lines 431-463, `InvokeHttpAsync` body-serialization branch lines 286-298)
|
||||
- Modify: `docs/requirements/Component-ExternalSystemGateway.md` (path/method-definition section: document `{param}` substitution + consumed-parameter removal + missing-parameter error)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs` (append)
|
||||
|
||||
Design: `BuildUrl` becomes `BuildUrl(baseUrl, path, parameters, httpMethod, out ISet<string> consumedParams)` (or returns a tuple). Regex `\{([A-Za-z_][A-Za-z0-9_]*)\}` over the trimmed path: each placeholder is replaced with `Uri.EscapeDataString(value?.ToString())` from `parameters` and its name recorded as consumed; a placeholder with no matching parameter (or a null value) throws `ArgumentException("Path template parameter '{id}' has no value …")` — a clear authoring error, mirroring `ValidateHttpMethod`. Consumed names are excluded from the GET/DELETE query string AND from the POST/PUT/PATCH JSON body.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing tests:
|
||||
- `PathTemplate_SubstitutesAndConsumesParameter`: method path `/recipes/{id}`, GET, parameters `{ id = "R 1", filter = "x" }` → captured request URI is `.../recipes/R%201?filter=x` (no `id` in the query).
|
||||
- `PathTemplate_Post_ConsumedParamExcludedFromBody`: POST `/orders/{orderId}/lines`, parameters `{ orderId = 5, qty = 2 }` → body JSON contains `qty` but not `orderId`.
|
||||
- `PathTemplate_MissingParameter_ThrowsClearError`: GET `/recipes/{id}` with no `id` → `ArgumentException` naming `id`.
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter PathTemplate` → **FAIL** (URL contains literal `{id}`).
|
||||
3. Implement as designed (a `private static readonly Regex PathParamRegex` + substitution loop; thread `consumedParams` into the query-string LINQ filter and into a `parameters.Where(p => !consumed.Contains(p.Key))` projection before body serialization).
|
||||
4. Run → **PASS**; full project run.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs docs/requirements/Component-ExternalSystemGateway.md tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs && git commit -m "feat(esg): {param} path-template substitution with consumed-param removal — implements the spec'd /recipes/{id} form"`
|
||||
|
||||
### Task 14: Bound outbound HTTP response buffering
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 10, 11, 12 (same-file chain: after 13)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs` (line 312 `SendAsync`, lines 334-347 body read)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemGatewayOptions.cs` (add `MaxResponseBodyBytes`, default `10 * 1024 * 1024`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs` (append)
|
||||
|
||||
Design: `SendAsync(request, HttpCompletionOption.ResponseHeadersRead, linkedCts.Token)`; pre-check `Content-Length` header when present (fail fast); then a bounded read (cap+1 pattern): stream into a `MemoryStream` in 16 KiB chunks, aborting once `MaxResponseBodyBytes` is exceeded. Oversize → `PermanentExternalSystemException` ("response exceeded N bytes" — retrying won't shrink it; buffering it into S&F would defeat the cap).
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test: `OversizedResponseBody_FailsPermanently_WithoutBufferingWholeBody` — options with `MaxResponseBodyBytes = 1024`; stub handler returns 2 MiB of `'x'`; `CallAsync` → `result.Success == false`, error contains `"exceeded"`; and a same-shape `CachedCallAsync` does NOT enqueue to S&F (assert via the existing S&F test double if the suite has one, else assert `WasBuffered == false`).
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter OversizedResponseBody` → **FAIL** (full body read succeeds).
|
||||
3. Implement: add the option; replace `ReadAsStringAsync` with a private `ReadBodyBoundedAsync(HttpContent content, long maxBytes, CancellationToken token)` returning `string` or throwing `PermanentExternalSystemException`; keep the existing ordered OCE catch filters around the read.
|
||||
4. Run → **PASS**; full project run.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs && git commit -m "perf(esg): bound outbound response bodies (ResponseHeadersRead + MaxResponseBodyBytes cap; oversize = permanent failure)"`
|
||||
|
||||
### Task 15: Structured JSON `AuthConfiguration` with legacy colon fallback
|
||||
|
||||
**Classification:** high-risk (auth parsing; existing rows must keep working)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 10, 11, 12 (same-file chain: after 14)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs` (`ApplyAuth`, lines 465-533)
|
||||
- Modify: `docs/requirements/Component-ExternalSystemGateway.md` (auth-configuration format section)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs` (append)
|
||||
|
||||
The entity doc-comment ("JSON-serialized authentication configuration") and the UI placeholders (`{"key":"xyz"}` / `{"username":"u","password":"p"}`) already promise JSON — but `ApplyAuth` colon-splits, so a JSON-shaped config is silently misparsed **today** (a `{"key":"xyz"}` apikey config produces header name `{"key"`), and any API key containing `:` breaks. Parse order per branch: trimmed value starting with `{` → JSON (`apikey`: `{"header"?: string, "key": string}`, default header `X-API-Key`; `basic`: `{"username": string, "password": string}`); otherwise the legacy colon-split/bare-key behavior, unchanged. Malformed JSON → the existing "malformed config" warning path (send without header), never an exception.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing tests:
|
||||
- `ApiKeyAuth_JsonConfig_SetsConfiguredHeaderAndKeepsColons`: `AuthConfiguration = "{\"header\":\"X-Auth\",\"key\":\"ab:cd:ef\"}"` → captured request has header `X-Auth: ab:cd:ef`.
|
||||
- `ApiKeyAuth_JsonConfig_DefaultHeader`: `{"key":"xyz"}` → `X-API-Key: xyz` (this is the UI-placeholder shape that is broken today).
|
||||
- `BasicAuth_JsonConfig_EncodesUsernamePassword`: `{"username":"u","password":"p:w"}` → `Authorization: Basic` of `u:p:w`.
|
||||
- `ApiKeyAuth_LegacyColonConfig_StillWorks`: `"X-Custom:secret"` → `X-Custom: secret` (backward compatibility pinned).
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter "JsonConfig|LegacyColonConfig"` → **FAIL**.
|
||||
3. Implement a private `static bool TryParseJsonAuth(string config, out Dictionary<string, string?> fields)` (System.Text.Json, case-insensitive property names, catch `JsonException` → false) and branch `apikey`/`basic` on it before the legacy split. Log a warning (values never logged) when JSON parsing yields no usable `key`/`username`. Update the doc section with both accepted formats and the deprecation note on the colon form.
|
||||
4. Run → **PASS**; full project run.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs docs/requirements/Component-ExternalSystemGateway.md tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemClientTests.cs && git commit -m "fix(esg): structured JSON AuthConfiguration (matches entity doc + UI placeholders) with legacy colon fallback — keys containing ':' no longer misparse"`
|
||||
|
||||
### Task 16: Cancellation-aware `ErrorClassifier.IsTransient` overload
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 10, 11, 12, 17-24 (same-file chain: after 15)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ErrorClassifier.cs` (lines 27-38)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemClient.cs` (line 325 call site → pass the caller token)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ErrorClassifierTests.cs` (append)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test: `IsTransient_CallerCancelledOce_IsNotTransient` — `var cts = new CancellationTokenSource(); cts.Cancel(); Assert.False(ErrorClassifier.IsTransient(new OperationCanceledException(), cts.Token)); Assert.True(ErrorClassifier.IsTransient(new OperationCanceledException(), CancellationToken.None));` → **FAIL** (no overload).
|
||||
2. Implement (mirrors `SmtpErrorClassifier`/`SmsErrorClassifier`):
|
||||
|
||||
```csharp
|
||||
/// <summary>
|
||||
/// Cancellation-aware overload: a caller-requested cancellation is NOT transient
|
||||
/// (buffering caller-abandoned work into S&F would be wrong). Prefer this overload
|
||||
/// at call sites that hold the caller's token.
|
||||
/// </summary>
|
||||
public static bool IsTransient(Exception exception, CancellationToken cancellationToken)
|
||||
{
|
||||
if (exception is OperationCanceledException && cancellationToken.IsCancellationRequested)
|
||||
return false;
|
||||
return IsTransient(exception);
|
||||
}
|
||||
```
|
||||
|
||||
Switch the `catch (Exception ex) when (ErrorClassifier.IsTransient(ex))` filter in `InvokeHttpAsync` to `ErrorClassifier.IsTransient(ex, cancellationToken)` (behavior identical — the ordered filters already peel caller-cancel — but the predicate is now misuse-resistant); add a doc-comment on the old overload pointing to the new one.
|
||||
3. Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter IsTransient_CallerCancelledOce` → **PASS**; full project run.
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ErrorClassifierTests.cs && git commit -m "fix(esg): cancellation-aware ErrorClassifier.IsTransient overload — caller-cancelled work can never classify transient"`
|
||||
|
||||
### Task 17: SMS first-transient short-circuit (duplicate amplification + sweep bound)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 1-16, 19, 22, 23, 24
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/SmsNotificationDeliveryAdapter.cs` (recipient loop, lines 148-172; `RollUp` doc-comment)
|
||||
- Modify: `docs/requirements/Component-NotificationService.md` (the re-send-to-all note ~line 102: add the short-circuit)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/Delivery/` (append to the existing SMS adapter test file)
|
||||
|
||||
Any transient rolls the whole notification up to retry-all, so every recipient accepted *after* the first transient is a guaranteed duplicate text on retry. Breaking the loop on the first transient costs nothing (retry re-attempts everyone) and — because a black-holed endpoint's first request eats the full per-request timeout — also bounds the worst-case sweep occupancy to ~one timeout instead of recipients × timeout, which is the substance of the P2 performance finding at current scale. Bounded parallelism is deferred (see coverage table).
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test: `MidListTransient_ShortCircuits_RemainingRecipientsNotAttempted` — list of 3 numbers; stub handler: first number → 500 (transient), and a request counter. Assert `DeliveryOutcome.Result == Transient` and **exactly 1** HTTP request was made (current code makes 3).
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests --filter MidListTransient_ShortCircuits` → **FAIL** (3 requests).
|
||||
3. Implement: in the `switch`, the `Transient` case sets `firstTransientError ??= attempt.Detail;` then `break`s out of the `foreach` (convert to `goto` -free form: set a flag and `break` the loop after the switch, or restructure to `if/else if`). Add the rationale comment: *"every recipient accepted after the first transient is a guaranteed duplicate on retry (the whole notification re-sends); stopping here also bounds a black-holed endpoint to one timeout per sweep."* Update `RollUp`'s doc-comment and the component doc line.
|
||||
4. Run → **PASS**; full `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests`.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/SmsNotificationDeliveryAdapter.cs docs/requirements/Component-NotificationService.md tests/ && git commit -m "fix(notifications): SMS adapter short-circuits on first transient — kills duplicate amplification and bounds the dispatch sweep"`
|
||||
|
||||
### Task 18: Honor `SmsConfiguration.MaxRetries`/`RetryDelay` — per-type retry policy
|
||||
|
||||
**Classification:** high-risk (behavioral change to dispatch/parking)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 1-16, 23, 24 (NOT 17/21 — shared test dirs are fine, but 21 edits the same actor file: run 18 before 21)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs` (`RunDispatchSweepAsync` ~line 319, `DeliverOneAsync` signature, `ResolveRetryPolicyAsync` lines 370-401)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Entities/Notifications/SmsConfiguration.cs` (lines 28-45: delete the RESERVED doc-comments, document the live behavior)
|
||||
- Modify: `docs/requirements/Component-NotificationService.md` (~line 65 deferred note), `CLAUDE.md` (the "Notification Outbox retry reuses central SMTP max-retry-count and fixed interval" decision line → "…for Email; Sms uses SmsConfiguration.MaxRetries/RetryDelay, falling back to the SMTP policy when unset")
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/NotificationOutboxActorDispatchTests.cs` (append)
|
||||
|
||||
Design: `ResolveRetryPolicyAsync` becomes `ResolveRetryPoliciesAsync` returning a small `RetryPolicies(SmtpPolicy, SmsPolicy)` record, resolved once per sweep: the SMTP policy exactly as today; the SMS policy from `INotificationRepository.GetSmsConfigurationAsync()` with the same non-positive-clamp logic, falling back to the SMTP policy when no SMS config row exists. `DeliverOneAsync` selects by `notification.Type` (`Sms` → SMS policy, everything else → SMTP policy). This is plan 08's flagged dead-field finding, owned here.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test (follow the existing dispatch-test scaffolding, which drives the actor with substitute repositories): `SmsNotification_TransientFailure_UsesSmsRetryPolicy` — SMTP config `MaxRetries = 10`; SMS config `MaxRetries = 1, RetryDelay = 5 min`; an SMS notification with `RetryCount = 1` and a transient adapter outcome → row is **Parked** (SMS policy exhausted) not Retrying; a parallel `EmailNotification_StillUsesSmtpPolicy` asserts an email row with `RetryCount = 1` goes to Retrying with the SMTP delay.
|
||||
2. `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests --filter "UsesSmsRetryPolicy|StillUsesSmtpPolicy"` → **FAIL** (SMS row retries under SMTP policy).
|
||||
3. Implement as designed; keep the clamp warnings (parameterized by which config produced them). Update the entity doc-comments, component doc, and the CLAUDE.md decision line in the same commit (docs travel with code).
|
||||
4. Run → **PASS**; full NotificationOutbox.Tests run.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs src/ZB.MOM.WW.ScadaBridge.Commons/Entities/Notifications/SmsConfiguration.cs docs/requirements/Component-NotificationService.md CLAUDE.md tests/ && git commit -m "feat(notifications): SMS notifications retry under SmsConfiguration.MaxRetries/RetryDelay (SMTP policy remains the Email + fallback policy) — wires the dead fields"`
|
||||
|
||||
### Task 19: Configurable OAuth2 authority/scope — entity + EF + migration + token service
|
||||
|
||||
**Classification:** high-risk (schema migration + auth flow)
|
||||
**Estimated implement time:** 5 min
|
||||
**Parallelizable with:** 1-7, 17, 23, 24 (**migration serialization: run after Task 8** — both regenerate the EF model snapshot)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Entities/Notifications/SmtpConfiguration.cs` (add `string? OAuth2Authority`, `string? OAuth2Scope`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase/Configurations/NotificationConfiguration.cs` (map both nullable columns)
|
||||
- Create: migration `AddSmtpOAuth2AuthorityScope` (same command pattern as Task 8; **build first**, verify non-empty)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationService/OAuth2TokenService.cs` (`GetTokenAsync` gains optional `string? authority = null, string? scope = null`; defaults preserve today's M365 URL/scope; cache key incorporates authority+scope so two configs with the same credentials but different authorities never share a token)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/EmailNotificationDeliveryAdapter.cs` (line 188: pass `smtpConfig.OAuth2Authority`, `smtpConfig.OAuth2Scope`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests` (append to the OAuth2 token service tests)
|
||||
|
||||
Authority template: the stored value is the full token-endpoint URL with an optional `{tenant}` placeholder (e.g. `https://login.microsoftonline.com/{tenant}/oauth2/v2.0/token`); null → today's hardcoded M365 endpoint. Scope: null → `https://outlook.office365.com/.default`.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing tests: `GetTokenAsync_CustomAuthorityAndScope_PostsToConfiguredEndpoint` — stub HTTP handler captures the request; call with `authority: "https://idp.example/{tenant}/token", scope: "smtp.send"`; assert the POST went to `https://idp.example/T1/token` with form `scope=smtp.send`. And `GetTokenAsync_DefaultsPreserved_M365` (no authority/scope → today's URL + scope). Run filter → **FAIL** (no parameters).
|
||||
2. Implement entity + EF + migration (verify non-empty), then the token-service parameters and the adapter call site; cache key: `CredentialKey($"{credentials}|{authority}|{scope}")`.
|
||||
3. Run → **PASS**; `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests` and `tests/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase.Tests`.
|
||||
4. Commit: `git add src/ZB.MOM.WW.ScadaBridge.Commons/Entities/Notifications/SmtpConfiguration.cs src/ZB.MOM.WW.ScadaBridge.ConfigurationDatabase src/ZB.MOM.WW.ScadaBridge.NotificationService/OAuth2TokenService.cs src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/EmailNotificationDeliveryAdapter.cs tests/ && git commit -m "feat(notifications): configurable OAuth2 authority + scope per SMTP configuration (defaults preserve M365) — entity + EF + migration + token service"`
|
||||
|
||||
### Task 20: OAuth2 authority/scope surfaces — management command + CLI + UI + doc
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 9-16, 21, 22
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Commons/Messages/Management/NotificationCommands.cs` (`UpdateSmtpConfigCommand`: trailing `string? OAuth2Authority = null, string? OAuth2Scope = null`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (the SMTP update handler: apply when non-null)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/NotificationCommands.cs` (`smtp update`: `--oauth2-authority`, `--oauth2-scope` options, ~line 127+ group)
|
||||
- Modify: the Central UI SMTP page (`src/ZB.MOM.WW.ScadaBridge.CentralUI/Components/Pages/Notifications/` — the SMTP form component; two text inputs shown only when AuthType is OAuth2)
|
||||
- Modify: `docs/requirements/Component-NotificationService.md` (~line 48: the "and other modern SMTP providers" sentence now true — document the two fields and their M365 defaults)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests` (handler applies fields), `tests/ZB.MOM.WW.ScadaBridge.CLI.Tests` (option parse — test csproj directly)
|
||||
|
||||
**Steps:** 1. Failing handler test `UpdateSmtpConfig_AppliesOAuth2AuthorityAndScope` → run filter → **FAIL**. 2. Implement the additive command params, handler application (preserve-on-null), CLI options, UI inputs, doc paragraph. 3. Run filters → **PASS**; run ManagementService.Tests + CLI.Tests (direct). 4. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.Commons/Messages src/ZB.MOM.WW.ScadaBridge.ManagementService src/ZB.MOM.WW.ScadaBridge.CLI src/ZB.MOM.WW.ScadaBridge.CentralUI docs/requirements/Component-NotificationService.md tests/ && git commit -m "feat(notifications): OAuth2 authority/scope management + CLI + UI surfaces"`
|
||||
|
||||
### Task 21: Notification low-severity cleanups — deterministic SMTP selection + OAuth2 cache visibility
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 9-16, 20, 22 (after 18 and 19 — shared files)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/Delivery/EmailNotificationDeliveryAdapter.cs` (lines 75-76)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationOutbox/NotificationOutboxActor.cs` (retry-policy resolution — post-Task-18 shape)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.NotificationService/OAuth2TokenService.cs` (`CacheEntry`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests/Delivery/` (append)
|
||||
|
||||
Concrete edits (S7, S8):
|
||||
1. **Deterministic SMTP pick**: in both call sites replace `FirstOrDefault()` / `configurations[0]` with `configurations.OrderBy(c => c.Id).FirstOrDefault()` and, when `configurations.Count > 1`, log a warning: `"Multiple SMTP configurations exist; using the lowest-Id row ({Id}). Enforce a single row or delete extras."` Failing test first: `MultipleSmtpConfigs_LowestIdWins` — repository returns rows Id 7 then Id 3 (that order); assert the adapter connects with row 3's server. Run → **FAIL** (row 7 picked) → implement → **PASS**.
|
||||
2. **OAuth2 cache visibility**: replace `CacheEntry`'s mutable `Token`/`Expiry` fields with a single `volatile TokenValue? Current;` where `private sealed record TokenValue(string Token, DateTimeOffset Expiry);` — reads outside the lock see a consistent pair; writes inside the lock assign one reference. (Growth stays bounded by distinct credential triples — accepted per report; no eviction added.) Existing OAuth2 tests must stay green.
|
||||
|
||||
Run `dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationOutbox.Tests` and `tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests`. Commit: `git add -A src/ZB.MOM.WW.ScadaBridge.NotificationOutbox src/ZB.MOM.WW.ScadaBridge.NotificationService tests/ && git commit -m "fix(notifications): deterministic lowest-Id SMTP config selection (+ multi-row warning); atomic OAuth2 token cache entry"`
|
||||
|
||||
### Task 22: Validate Twilio `AccountSid` format at config save
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 9, 10, 12, 13-17, 20, 21, 23, 24 (after 11 — same `ManagementActor.cs`)
|
||||
**Files:**
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs` (the `UpdateSmsConfig` handler: reject a non-matching SID)
|
||||
- Modify: the Central UI SMS form (`/notifications/sms` page component: same client-side validation message)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests` (append)
|
||||
|
||||
`AccountSid` is interpolated un-escaped into the Twilio request URI (`SmsNotificationDeliveryAdapter.cs:131`); a validation regex at save time closes the door on URI-injection via a malformed SID.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Failing test: `UpdateSmsConfig_MalformedAccountSid_Rejected` — send the SMS-config update command with `AccountSid = "../evil?x="`; assert `ManagementCommandException` containing `"Account SID"`; and `UpdateSmsConfig_ValidSid_Accepted` with `"AC" + new string('a', 32)`.
|
||||
2. Run filter → **FAIL**.
|
||||
3. Implement in the handler, before persisting: `if (!System.Text.RegularExpressions.Regex.IsMatch(cmd.AccountSid, "^AC[0-9a-fA-F]{32}$")) throw new ManagementCommandException("Account SID must match Twilio's format: 'AC' followed by 32 hex characters.");` Mirror the message in the UI form's validation.
|
||||
4. Run → **PASS**; run ManagementService.Tests.
|
||||
5. Commit: `git add src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementActor.cs src/ZB.MOM.WW.ScadaBridge.CentralUI tests/ && git commit -m "fix(notifications): validate Twilio AccountSid format at save — closes the un-escaped URI interpolation door"`
|
||||
|
||||
### Task 23: Document DelmiaNotifier at-least-once semantics + idempotency requirement
|
||||
|
||||
**Classification:** small (documentation)
|
||||
**Estimated implement time:** 4 min
|
||||
**Parallelizable with:** 1-22
|
||||
**Files:**
|
||||
- Modify: `docs/plans/2026-06-26-delmia-recipe-notifier-design.md` (after the failover table, ~line 100)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.DelmiaNotifier/README.md` (operator-facing note)
|
||||
- Modify: `docs/requirements/Component-InboundAPI.md` (authoring note for the `DelmiaRecipeDownload` method)
|
||||
|
||||
The behavior is by design (timeout → failover), so no code change — but the consequence (at-least-once: a connection reset *after* the server processed the POST, or a slow response exceeding `TimeoutSeconds`, both roll over to the next URL and re-POST) is nowhere stated, and the deployed inbound method carries the burden.
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Add a "Delivery semantics: at-least-once" section to the design doc: enumerate the two duplicate-producing cases; state the requirement — *the `DelmiaRecipeDownload` inbound method MUST be idempotent on `(MachineCode, DownloadPath, WorkOrderNumber)`; a duplicate must return the same `Result`/`ResultText` as the original, not double-apply side effects.* Cross-link from the failover table row for timeout.
|
||||
2. Mirror one paragraph in the DelmiaNotifier README (`## Delivery semantics`) and add the authoring note to `Component-InboundAPI.md` where the method is referenced.
|
||||
3. Manual verification step (record the outcome in the commit message): fetch the deployed `DelmiaRecipeDownload` script (`scadabridge api-method get` via the CLI against the target environment, or read the row) and confirm it is idempotent on the natural key; if it is not, file/fix in the environment — this plan only mandates and documents the contract.
|
||||
4. `grep -n "at-least-once" docs/plans/2026-06-26-delmia-recipe-notifier-design.md src/ZB.MOM.WW.ScadaBridge.DelmiaNotifier/README.md` → both hit.
|
||||
5. Commit: `git add docs/plans/2026-06-26-delmia-recipe-notifier-design.md src/ZB.MOM.WW.ScadaBridge.DelmiaNotifier/README.md docs/requirements/Component-InboundAPI.md && git commit -m "docs(delmia): document at-least-once failover semantics + DelmiaRecipeDownload idempotency requirement"`
|
||||
|
||||
### Task 24: Document the dual-key API-key rotation procedure
|
||||
|
||||
**Classification:** trivial (documentation)
|
||||
**Estimated implement time:** 3 min
|
||||
**Parallelizable with:** 1, 3-22 (after 2 and 5 — same doc file)
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/Component-InboundAPI.md` (new "### Key rotation" subsection under the API-key auth section, ~line 62+)
|
||||
|
||||
**Steps:**
|
||||
|
||||
1. Add the procedure the `sbk_<keyId>_<secret>` design already supports: (1) create a new key with the same scope set (`apikey create`), (2) deliver the new token to the caller and confirm cutover (the audit log's Actor field shows which key serves traffic), (3) disable — don't delete — the old key (`apikey update --enabled false`; reversible), (4) delete after a quarantine period. Note that scopes are copied at creation (scope drift between the pair is an operator error) and that both keys verifying simultaneously is the intended overlap window.
|
||||
2. Verify cross-references: the CLI command names used must match `src/ZB.MOM.WW.ScadaBridge.CLI/Commands/SecurityCommands.cs` (adjust wording to the real verbs).
|
||||
3. Commit: `git add docs/requirements/Component-InboundAPI.md && git commit -m "docs(inbound-api): dual-key API-key rotation procedure (create -> migrate -> disable -> delete)"`
|
||||
|
||||
---
|
||||
|
||||
## Dependencies on other plans
|
||||
|
||||
- **Plan 05 (Templates/Deployment/Transport)** designs the general "script artifact changed → invalidate everywhere" contract and owns the **bundle-import** side of cache invalidation. This plan's Task 3 provides the Inbound API consumer entry point (`InboundScriptExecutor.InvalidateMethod`) that plan 05's import path must call for every overwritten `ApiMethod`. **Deliberate resilience:** Task 1's per-request revision check is the correctness fallback — inbound methods heal even if an invalidation call is missed or plan 05 lands later, so there is no hard sequencing requirement in either direction.
|
||||
- **Plan 04 (Data/Audit backbone)** owns NotificationOutbox dispatcher/audit-row gaps. Tasks 17/18/21 touch `NotificationOutboxActor.cs` and the delivery adapters — if plan 04 is executed concurrently, coordinate on that file (this plan's edits are confined to `RunDispatchSweepAsync`'s policy resolution, `DeliverOneAsync`'s policy selection, and the two adapters).
|
||||
- **Plan 08** flagged `SmsConfiguration.MaxRetryCount` unused — ownership of the wiring is **here** (Task 18); plan 08 should not duplicate it.
|
||||
- Tasks 8 and 19 both add EF migrations against the shared model snapshot — they are sequenced (19 after 8) within this plan; if any other plan adds migrations in the same window, serialize migration-generation across plans.
|
||||
|
||||
## Execution order
|
||||
|
||||
**P0 (start immediately, independent):** Task 1 (the High-severity stale-handler fix) and Task 8 (the High-severity spec-gap migration).
|
||||
|
||||
- **Inbound chain:** 1 → 2 → 3 → 4 → 6 → 5 → 24 (2/3 also gate 11's ManagementActor edits ordering; 24 last because it shares `Component-InboundAPI.md` with 2 and 5).
|
||||
- **ESG chain:** 8 → {9, 10, 11, 12 in parallel} → 13 → 14 → 15 → 16 (9/13/14/15/16 serialize on `ExternalSystemClient.cs`; 10/11/12 are parallel with all of them).
|
||||
- **Notifications:** 17 (anytime) ; 18 → 21 ; 19 (after 8, migration serialization) → 20 → 21 ; 22 after 11 (shared `ManagementActor.cs`).
|
||||
- **Docs:** 23 anytime; 24 after 2/5.
|
||||
|
||||
Intra-plan critical path: **1 → 2 → 3 → 4 → 5** (inbound executor file chain) and **8 → 9 → 13 → 14 → 15 → 16** (ESG client file chain) — roughly equal length; run them in parallel lanes. Everything else hangs off those two spines or is free-floating.
|
||||
@@ -0,0 +1,30 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-06-edge-integrations.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: Revision-checked inbound handler cache (self-healing across failover / direct SQL / known-bad)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: DB-authoritative compile semantics — management warning text + design doc", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 3, "subject": "Task 3: InvalidateMethod seam — the Inbound API consumer of plan 05's invalidation contract", "status": "pending", "blockedBy": [1, 2] },
|
||||
{ "id": 4, "subject": "Task 4: Abandoned inbound executions — defer DI-scope disposal + orphan telemetry", "status": "pending", "blockedBy": [3] },
|
||||
{ "id": 5, "subject": "Task 5: Inbound error contract — code field + SITE_UNREACHABLE + 415 for non-JSON bodies", "status": "pending", "blockedBy": [4, 6] },
|
||||
{ "id": 6, "subject": "Task 6: InboundApiEndpointFilter hot-reads options via IOptionsMonitor", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 7, "subject": "Task 7: Parallelize startup compile of inbound methods", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 8, "subject": "Task 8: ExternalSystemDefinition.TimeoutSeconds — entity + EF mapping + migration", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 9, "subject": "Task 9: Honor the per-system timeout in ExternalSystemClient.InvokeHttpAsync", "status": "pending", "blockedBy": [8] },
|
||||
{ "id": 10, "subject": "Task 10: Carry TimeoutSeconds through the artifact pipeline to sites", "status": "pending", "blockedBy": [8] },
|
||||
{ "id": 11, "subject": "Task 11: TimeoutSeconds management commands + CLI", "status": "pending", "blockedBy": [8, 3] },
|
||||
{ "id": 12, "subject": "Task 12: TimeoutSeconds in Central UI form + spec doc sync", "status": "pending", "blockedBy": [8] },
|
||||
{ "id": 13, "subject": "Task 13: Path-template substitution (/recipes/{id}) in BuildUrl", "status": "pending", "blockedBy": [9] },
|
||||
{ "id": 14, "subject": "Task 14: Bound outbound HTTP response buffering", "status": "pending", "blockedBy": [13] },
|
||||
{ "id": 15, "subject": "Task 15: Structured JSON AuthConfiguration with legacy colon fallback", "status": "pending", "blockedBy": [14] },
|
||||
{ "id": 16, "subject": "Task 16: Cancellation-aware ErrorClassifier.IsTransient overload", "status": "pending", "blockedBy": [15] },
|
||||
{ "id": 17, "subject": "Task 17: SMS first-transient short-circuit (duplicate amplification + sweep bound)", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 18, "subject": "Task 18: Honor SmsConfiguration.MaxRetries/RetryDelay — per-type retry policy", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 19, "subject": "Task 19: Configurable OAuth2 authority/scope — entity + EF + migration + token service", "status": "pending", "blockedBy": [8] },
|
||||
{ "id": 20, "subject": "Task 20: OAuth2 authority/scope surfaces — management command + CLI + UI + doc", "status": "pending", "blockedBy": [19] },
|
||||
{ "id": 21, "subject": "Task 21: Notification low-severity cleanups — deterministic SMTP selection + OAuth2 cache visibility", "status": "pending", "blockedBy": [18, 19] },
|
||||
{ "id": 22, "subject": "Task 22: Validate Twilio AccountSid format at config save", "status": "pending", "blockedBy": [11] },
|
||||
{ "id": 23, "subject": "Task 23: Document DelmiaNotifier at-least-once semantics + idempotency requirement", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 24, "subject": "Task 24: Document the dual-key API-key rotation procedure", "status": "pending", "blockedBy": [2, 5] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
File diff suppressed because it is too large
Load Diff
@@ -0,0 +1,39 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-07-ui-management-security.md",
|
||||
"tasks": [
|
||||
{ "id": 1, "subject": "Task 1: DisableLoginGuard — refuse the flag outside Development", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 2, "subject": "Task 2: Wire the guard at the Host composition root + doc", "status": "pending", "blockedBy": [1] },
|
||||
{ "id": 3, "subject": "Task 3: Scrub deploy/wonder-app-vd03 — remove DisableLogin: true", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 4, "subject": "Task 4: ArtifactDeploymentService.DeployToSiteAsync", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 5, "subject": "Task 5: HandleDeployArtifacts honors SiteId + site-scope enforcement + docs", "status": "pending", "blockedBy": [4] },
|
||||
{ "id": 6, "subject": "Task 6: ConfigSecretScrubber — JSON secret elision + sentinel merge", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 7, "subject": "Task 7: DataConnection List/Get/audit secret elision + sentinel-preserving update", "status": "pending", "blockedBy": [6] },
|
||||
{ "id": 8, "subject": "Task 8: ExternalSystem AuthConfiguration elision + preserve-if-null update", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 9, "subject": "Task 9: DatabaseConnection ConnectionString elision + preserve-if-null update", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 10, "subject": "Task 10: GetRequiredRoles any-of refactor + gate ListSecuredWrites", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 11, "subject": "Task 11: Reconcile the area-management role gate (code + three docs)", "status": "pending", "blockedBy": [10] },
|
||||
{ "id": 12, "subject": "Task 12: Canonicalize role-mapping casing at the write path", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 13, "subject": "Task 13: GetRequiredRoles matrix test — freeze the authorization table", "status": "pending", "blockedBy": [10, 11, 22, 23] },
|
||||
{ "id": 14, "subject": "Task 14: SecuredWrite repository — TryMarkExpiredAsync + CountAsync", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 15, "subject": "Task 15: Server-side secured-write TTL — enforce at approve/reject", "status": "pending", "blockedBy": [14] },
|
||||
{ "id": 16, "subject": "Task 16: Opportunistic expiry sweep in HandleListSecuredWrites", "status": "pending", "blockedBy": [15] },
|
||||
{ "id": 17, "subject": "Task 17: Secured-write list paging — command, handler, CLI", "status": "pending", "blockedBy": [14] },
|
||||
{ "id": 18, "subject": "Task 18: SecuredWrites page — history pager + age in approve dialog + docs", "status": "pending", "blockedBy": [17] },
|
||||
{ "id": 19, "subject": "Task 19: Per-command long-running Ask timeout", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 20, "subject": "Task 20: Ship ManagementService timeout config + doc", "status": "pending", "blockedBy": [19] },
|
||||
{ "id": 21, "subject": "Task 21: Browse/Verify/Cert commands gain optional SiteIdentifier", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 22, "subject": "Task 22: Actor handlers — BrowseNode / SearchAddressSpace / VerifyEndpoint", "status": "pending", "blockedBy": [10, 21] },
|
||||
{ "id": 23, "subject": "Task 23: Actor handlers — TrustServerCert / RemoveServerCert / ListServerCerts (Admin)", "status": "pending", "blockedBy": [10, 21] },
|
||||
{ "id": 24, "subject": "Task 24: Dispatch-coverage test — every registered command reaches a handler", "status": "pending", "blockedBy": [22, 23] },
|
||||
{ "id": 25, "subject": "Task 25: CLI parity — browse / search / verify-endpoint / certs commands", "status": "pending", "blockedBy": [22, 23] },
|
||||
{ "id": 26, "subject": "Task 26: LoginThrottle — fixed-window LDAP-bind failure lockout", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 27, "subject": "Task 27: Wire the throttle — ManagementAuthenticator + auth endpoints + doc", "status": "pending", "blockedBy": [26] },
|
||||
{ "id": 28, "subject": "Task 28: Additive paging for ListTemplates / ListInstances", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 29, "subject": "Task 29: PollGate — non-reentrant poll timers on Health + Alarm Summary", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 30, "subject": "Task 30: Transport single-flight + base64 copy elimination", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 31, "subject": "Task 31: Low cleanup A — hub send fault logging, event-log filter comment, stale CLAUDE.md note", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 32, "subject": "Task 32: Low cleanup B — AlarmSummary memoization + sortable-header a11y", "status": "pending", "blockedBy": [] },
|
||||
{ "id": 33, "subject": "Task 33: Add CLI.Tests to the slnx + retire the gotcha", "status": "pending", "blockedBy": [] }
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
@@ -0,0 +1,739 @@
|
||||
# Conventions, Tests & Underdeveloped Areas Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Close the cross-cutting gaps from architecture review 08 — the CLI.Tests slnx blind spot, the ~12 components binding options without startup validation, the vestigial site-side notification surface, the misnamed-thin PerformanceTests project, the docs-code drift owned by no other plan, and a triaged register for the 23-item deferred-work inventory.
|
||||
|
||||
**Architecture:** All changes are convention-hardening around existing seams: eager options validation copies the shipped `OptionsValidatorBase<T>` pattern (`ZB.MOM.WW.Configuration` package) already used by HealthMonitoring/ClusterInfrastructure/AuditLog/KpiHistory; the vestigial notification removal relies on the existing `PurgeCentralOnlyNotificationConfigAsync` guarantee (which stays); new tests slot into existing test projects (PerformanceTests, Commons.Tests). No new runtime components, no schema changes, no wire-contract changes.
|
||||
|
||||
**Tech Stack:** C# / .NET 10, xUnit, Akka.NET TestKit/Streams, `Microsoft.Extensions.Options` (`IValidateOptions<>` + `ValidateOnStart`), Newtonsoft.Json (contract-lock tests only), SQLite (site storage).
|
||||
|
||||
**Build:** `dotnet build ZB.MOM.WW.ScadaBridge.slnx`. Tests: `dotnet test ZB.MOM.WW.ScadaBridge.slnx` — after Task 1 this picks up CLI.Tests too.
|
||||
|
||||
## Findings Coverage
|
||||
|
||||
| Report finding (08-conventions-tests-underdeveloped.md) | Severity | Coverage |
|
||||
|---|---|---|
|
||||
| §1.1 Commons folders beyond the documented four | Low | Task 10 — **narrowed on re-verify**: `Serialization/` and `Validators/` are already in `Component-Commons.md`'s layout tree (lines 293–294); only `Observability/` is missing |
|
||||
| §1.2 POCO persistence-ignorance | Pass | No action |
|
||||
| §1.3 Site-side repository-implementation exception undocumented | Low | Task 10 |
|
||||
| §1.3 / §3 item 23 Vestigial `SiteNotificationRepository` + `StoreNotificationListAsync` write path | Low/Medium | Task 6 |
|
||||
| §1.4 `Communication` → `HealthMonitoring` layering inversion | Low | **Deferred** (Task 11 register entry): moving `ICentralHealthAggregator` + `SiteHealthState` to Commons ripples across HealthMonitoring, Communication, ManagementService, CentralUI, AuditLog + their tests — disproportionate to a cosmetic note; documented as accepted with revisit trigger |
|
||||
| §1.4 `ManagementService.csproj` `..\` path-separator inconsistency | Low | Task 10 |
|
||||
| §1.5 Options validation on only ~6 of ~20 option classes | Medium | Tasks 2, 3, 4, 5 |
|
||||
| §1.6 UTC / §1.7 correlation IDs / §1.9 naming | Pass | No action |
|
||||
| §1.8 No contract-lock tests for ClusterClient record contracts (also Rec 10) | Low | Task 9 |
|
||||
| §2.1 Thin test spots (SiteCallAudit.Tests, DeploymentManager.Tests) | Info | **Deferred** (Task 11 register entry): no defect identified; backfill tracked with revisit trigger |
|
||||
| §2.3 PerformanceTests is a correctness-at-small-scale suite | Medium | Tasks 7 (streaming throughput) + 8 (failover-timing harness placeholder, delegating the rig to Plan 01) |
|
||||
| §2.4 CLI.Tests excluded from slnx | High | Task 1 |
|
||||
| §3 item 1 / §4 Transport `AreaName: null` vs doc claim | High | **Owned by plan 05** |
|
||||
| §3 item 3 AuditLog reconciliation dials NodeA only | Medium | **Owned by plan 04** |
|
||||
| §3 item 5 `SmsConfiguration.MaxRetryCount` stored but unused | Medium | **Owned by plan 06** |
|
||||
| §3 item 6 Role-check case-sensitivity asymmetry | Medium | **Owned by plan 07** |
|
||||
| §3 items 7–22 (hash-chain, Parquet, live alarm stream, cert-trust central persistence, CSV parity, WaitForAttribute, BrowseNext, StubOpcUaClient, unified outbox page, folder drag-drop, bundle signing, EXPIRED purge, backlog-reporter options, KPI downsampling) | Low | Task 11 — each gets an explicit register row with defer rationale + revisit trigger (none needs code in this plan) |
|
||||
| §4 `docs/components/` lags 3 components vs README claim | Low | Task 10 (scope the claim; backfill tracked in Task 11 register) |
|
||||
| §4 DelmiaNotifier breaks `Component-<Name>.md` convention | Low | Task 10 (document the exception) |
|
||||
| §4 Component-KpiHistory / StoreAndForward / ClusterInfrastructure doc sync | Pass | No action |
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add CLI.Tests to the slnx and retire the gotcha documentation
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 2, 3, 4, 5, 6, 7, 8, 9, 11
|
||||
**Files:**
|
||||
- Modify: `ZB.MOM.WW.ScadaBridge.slnx` (insert one line after line 51, the ManagementService.Tests entry, mirroring src ordering)
|
||||
- Modify: `CLAUDE.md` (line 69, the Editing Rules test-command bullet)
|
||||
- Modify: `/Users/dohertj2/.claude/projects/-Users-dohertj2-Desktop-ScadaBridge/memory/MEMORY.md` (index entry "CLI.Tests is not in the slnx") and `/Users/dohertj2/.claude/projects/-Users-dohertj2-Desktop-ScadaBridge/memory/cli-tests-not-in-slnx.md`
|
||||
|
||||
1. Verify current state (expect CLI.Tests absent):
|
||||
```bash
|
||||
grep -c "CLI.Tests" ZB.MOM.WW.ScadaBridge.slnx # expect: 0
|
||||
```
|
||||
2. Edit `ZB.MOM.WW.ScadaBridge.slnx`: after the line
|
||||
` <Project Path="tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests.csproj" />`
|
||||
insert:
|
||||
```xml
|
||||
<Project Path="tests/ZB.MOM.WW.ScadaBridge.CLI.Tests/ZB.MOM.WW.ScadaBridge.CLI.Tests.csproj" />
|
||||
```
|
||||
3. Verify the solution builds and the CLI tests now run at the solution level:
|
||||
```bash
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
dotnet test ZB.MOM.WW.ScadaBridge.slnx --no-build --filter "FullyQualifiedName~ScadaBridge.CLI.Tests"
|
||||
```
|
||||
Expected: build succeeds; the test run lists `ZB.MOM.WW.ScadaBridge.CLI.Tests` and reports ~279 passing tests (0 failed). If any CLI test fails to compile, fix forward — that is exactly the rot this task exists to expose.
|
||||
4. Update `CLAUDE.md:69`. Replace the bullet:
|
||||
> `- Run tests with \`dotnet test ZB.MOM.WW.ScadaBridge.slnx\`, **but \`ZB.MOM.WW.ScadaBridge.CLI.Tests\` is not in the slnx** and is silently skipped — when touching the CLI, test that project directly (\`dotnet test tests/ZB.MOM.WW.ScadaBridge.CLI.Tests\`), or a green solution run can hide CLI tests that never compiled.`
|
||||
|
||||
with:
|
||||
> `- Run tests with \`dotnet test ZB.MOM.WW.ScadaBridge.slnx\`. All 30 test projects, including \`ZB.MOM.WW.ScadaBridge.CLI.Tests\`, are members of the slnx (CLI.Tests was added 2026-07; the old "silently skipped" gotcha no longer applies).`
|
||||
5. Update the auto-memory: in `MEMORY.md`, change the index line for `cli-tests-not-in-slnx.md` to note the gotcha is fixed (e.g. "FIXED 2026-07: CLI.Tests is now in the slnx; entry kept as history"), and add a first-line status note to `cli-tests-not-in-slnx.md` itself.
|
||||
6. Commit:
|
||||
```bash
|
||||
git add ZB.MOM.WW.ScadaBridge.slnx CLAUDE.md
|
||||
git commit -m "build(slnx): add CLI.Tests to the solution — retire the silently-skipped-279-tests gotcha (arch-review 08 §2.4)"
|
||||
```
|
||||
(The memory files live outside the repo and are not committed.)
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Options validation — Communication + DataConnectionLayer
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 3, 4, 5, 6, 7, 8, 9, 10, 11
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Communication/CommunicationOptionsValidator.cs`
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.DataConnectionLayer/DataConnectionOptionsValidator.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/ServiceCollectionExtensions.cs` (lines 13–14), `src/ZB.MOM.WW.ScadaBridge.DataConnectionLayer/ServiceCollectionExtensions.cs` (lines 15–22)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Communication/ZB.MOM.WW.ScadaBridge.Communication.csproj`, `src/ZB.MOM.WW.ScadaBridge.DataConnectionLayer/ZB.MOM.WW.ScadaBridge.DataConnectionLayer.csproj` (add `<PackageReference Include="ZB.MOM.WW.Configuration" />` — no Version attribute; central package management already has it, see `HealthMonitoring.csproj:15`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/CommunicationOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.DataConnectionLayer.Tests/DataConnectionOptionsValidatorTests.cs`
|
||||
|
||||
The pattern to copy is `src/ZB.MOM.WW.ScadaBridge.HealthMonitoring/HealthMonitoringOptionsValidator.cs` (an `OptionsValidatorBase<T>` from the `ZB.MOM.WW.Configuration` package with `builder.RequireThat(...)` per field) registered via `TryAddEnumerable(ServiceDescriptor.Singleton<IValidateOptions<T>, TValidator>())` + `.ValidateOnStart()` on the `AddOptions` chain (`HealthMonitoring/ServiceCollectionExtensions.cs:157-162`). Tests copy `tests/ZB.MOM.WW.ScadaBridge.HealthMonitoring.Tests/HealthMonitoringOptionsValidatorTests.cs`.
|
||||
|
||||
1. Write failing tests first — `tests/ZB.MOM.WW.ScadaBridge.Communication.Tests/CommunicationOptionsValidatorTests.cs`:
|
||||
```csharp
|
||||
using Microsoft.Extensions.Options;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.Communication.Tests;
|
||||
|
||||
public class CommunicationOptionsValidatorTests
|
||||
{
|
||||
private static ValidateOptionsResult Validate(CommunicationOptions options) =>
|
||||
new CommunicationOptionsValidator().Validate(Options.DefaultName, options);
|
||||
|
||||
[Fact]
|
||||
public void DefaultOptions_AreValid()
|
||||
{
|
||||
var result = Validate(new CommunicationOptions());
|
||||
Assert.True(result.Succeeded, result.FailureMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ZeroDeploymentTimeout_IsRejected()
|
||||
{
|
||||
var result = Validate(new CommunicationOptions { DeploymentTimeout = TimeSpan.Zero });
|
||||
Assert.True(result.Failed);
|
||||
Assert.Contains("DeploymentTimeout", result.FailureMessage);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void NonPositiveGrpcMaxConcurrentStreams_IsRejected()
|
||||
{
|
||||
var result = Validate(new CommunicationOptions { GrpcMaxConcurrentStreams = 0 });
|
||||
Assert.True(result.Failed);
|
||||
Assert.Contains("GrpcMaxConcurrentStreams", result.FailureMessage);
|
||||
}
|
||||
}
|
||||
```
|
||||
Mirror the same three-test shape for `DataConnectionOptionsValidator` (`ReconnectInterval`, `SeedReadMaxAttempts`).
|
||||
2. Run, expect FAIL (validator types don't exist → compile error, which counts):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter "FullyQualifiedName~CommunicationOptionsValidator"
|
||||
```
|
||||
3. Implement `CommunicationOptionsValidator` (positive-duration checks for every `TimeSpan` on `CommunicationOptions` — `DeploymentTimeout`, `LifecycleTimeout`, `ArtifactDeploymentTimeout`, `QueryTimeout`, `IntegrationTimeout`, `DebugViewTimeout`, `HealthReportTimeout`, `NotificationForwardTimeout`, `GrpcKeepAlivePingDelay`, `GrpcKeepAlivePingTimeout`, `GrpcMaxStreamLifetime`, `TransportHeartbeatInterval`, `TransportFailureThreshold` — plus `GrpcMaxConcurrentStreams > 0`). Error messages must name the config key (`Communication:<Field>`), same style as `HealthMonitoringOptionsValidator`:
|
||||
```csharp
|
||||
using ZB.MOM.WW.Configuration;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.Communication;
|
||||
|
||||
/// <summary>
|
||||
/// Validates <see cref="CommunicationOptions"/> at startup (ValidateOnStart) so a
|
||||
/// malformed "Communication" appsettings section fails fast at boot with a
|
||||
/// key-naming message instead of surfacing at first Ask/gRPC use.
|
||||
/// </summary>
|
||||
public sealed class CommunicationOptionsValidator : OptionsValidatorBase<CommunicationOptions>
|
||||
{
|
||||
protected override void Validate(ValidationBuilder builder, CommunicationOptions options)
|
||||
{
|
||||
builder.RequireThat(options.DeploymentTimeout > TimeSpan.Zero,
|
||||
$"Communication:DeploymentTimeout must be a positive duration (was {options.DeploymentTimeout}).");
|
||||
// ... one RequireThat per field listed above ...
|
||||
builder.RequireThat(options.GrpcMaxConcurrentStreams > 0,
|
||||
$"Communication:GrpcMaxConcurrentStreams must be positive (was {options.GrpcMaxConcurrentStreams}).");
|
||||
}
|
||||
}
|
||||
```
|
||||
`DataConnectionOptionsValidator` likewise: `ReconnectInterval`, `TagResolutionRetryInterval`, `WriteTimeout`, `SeedReadTimeout`, `StableConnectionThreshold`, `SeedReadRetryDelay` positive durations; `SeedReadMaxAttempts > 0`. Read `OpcUaGlobalOptions`/`MxGatewayGlobalOptions` (`src/.../DataConnectionLayer/`) and add `RequireThat` checks for any numeric/duration fields they carry (add validators for them only if they have such fields).
|
||||
4. Wire registration. `Communication/ServiceCollectionExtensions.cs:13-14` becomes:
|
||||
```csharp
|
||||
services.AddOptions<CommunicationOptions>()
|
||||
.BindConfiguration("Communication")
|
||||
.ValidateOnStart();
|
||||
services.TryAddEnumerable(
|
||||
ServiceDescriptor.Singleton<IValidateOptions<CommunicationOptions>, CommunicationOptionsValidator>());
|
||||
```
|
||||
Same shape for the three `AddOptions` calls in `DataConnectionLayer/ServiceCollectionExtensions.cs:15-22`. Add the `ZB.MOM.WW.Configuration` PackageReference to both csprojs.
|
||||
5. Run, expect PASS:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Communication.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.DataConnectionLayer.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
```
|
||||
6. Commit:
|
||||
```bash
|
||||
git add src/ZB.MOM.WW.ScadaBridge.Communication src/ZB.MOM.WW.ScadaBridge.DataConnectionLayer tests/ZB.MOM.WW.ScadaBridge.Communication.Tests tests/ZB.MOM.WW.ScadaBridge.DataConnectionLayer.Tests
|
||||
git commit -m "feat(options): eager startup validation for Communication + DataConnectionLayer options (arch-review 08 §1.5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Options validation — site pipeline (SiteRuntime, StoreAndForward, SiteEventLogging)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 4, 5, 7, 8, 9, 10, 11 (NOT 6 — both touch the SiteRuntime project; run sequentially to keep csproj/test churn separate)
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/SiteRuntimeOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.StoreAndForward/StoreAndForwardOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.SiteEventLogging/SiteEventLogOptionsValidator.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/SiteServiceRegistration.cs` (lines 124–127 — the `services.Configure<T>(config.GetSection(...))` calls for these three options)
|
||||
- Modify: the three component csprojs (add `<PackageReference Include="ZB.MOM.WW.Configuration" />`)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/SiteRuntimeOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.StoreAndForward.Tests/StoreAndForwardOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.SiteEventLogging.Tests/SiteEventLogOptionsValidatorTests.cs`
|
||||
|
||||
These three options classes are bound in the Host, not in the component SCEs (`Host/SiteServiceRegistration.cs:124-127`), so binding conversion happens there; validator classes live with their owners per the "options classes owned by component projects" convention.
|
||||
|
||||
1. Write failing tests (same shape as Task 2 step 1; one `DefaultOptions_AreValid` + 2–3 rejection cases per validator). Key rejection cases:
|
||||
- `SiteRuntimeOptions`: `StartupBatchSize = 0`, `ScriptExecutionThreadCount = 0`, `StreamBufferSize = 0` (all feed thread-pool/stream sizing; also validate `StartupBatchDelayMs >= 0`, `MaxScriptCallDepth > 0`, `ScriptExecutionTimeoutSeconds > 0`, `MirroredAlarmCapPerSource > 0`, `NativeAlarmRetryIntervalMs > 0`, `ConfigFetchTimeoutSeconds > 0`, `ConfigFetchRetryCount >= 0`).
|
||||
- `StoreAndForwardOptions`: `DefaultRetryInterval = TimeSpan.Zero`, `RetryTimerInterval = TimeSpan.Zero`, `SqliteDbPath = ""` (also `DefaultMaxRetries >= 0`).
|
||||
- `SiteEventLogOptions`: `RetentionDays = 0`, `PurgeInterval = TimeSpan.Zero`, `MaxQueryPageSize < QueryPageSize` cross-check (also `MaxStorageMb > 0`, `WriteQueueCapacity > 0`, `DatabasePath` non-empty).
|
||||
2. Run, expect FAIL (compile error on missing validator types):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.StoreAndForward.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
```
|
||||
3. Implement the three validators (`OptionsValidatorBase<T>`, key-naming messages using the `ScadaBridge:SiteRuntime` / `ScadaBridge:StoreAndForward` / `ScadaBridge:SiteEventLog` prefixes). Convert the Host bindings — `Host/SiteServiceRegistration.cs:124-127` pattern per option:
|
||||
```csharp
|
||||
services.AddOptions<SiteRuntimeOptions>()
|
||||
.Bind(config.GetSection("ScadaBridge:SiteRuntime"))
|
||||
.ValidateOnStart();
|
||||
services.TryAddEnumerable(
|
||||
ServiceDescriptor.Singleton<IValidateOptions<SiteRuntimeOptions>, SiteRuntimeOptionsValidator>());
|
||||
```
|
||||
(Keep any other `Configure<>` calls in that block untouched.)
|
||||
4. Run, expect PASS; then guard against startup regressions (Host composition-root tests boot both roles):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.StoreAndForward.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteEventLogging.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~CompositionRoot"
|
||||
```
|
||||
5. Commit:
|
||||
```bash
|
||||
git add src/ZB.MOM.WW.ScadaBridge.SiteRuntime src/ZB.MOM.WW.ScadaBridge.StoreAndForward src/ZB.MOM.WW.ScadaBridge.SiteEventLogging src/ZB.MOM.WW.ScadaBridge.Host/SiteServiceRegistration.cs tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests tests/ZB.MOM.WW.ScadaBridge.StoreAndForward.Tests tests/ZB.MOM.WW.ScadaBridge.SiteEventLogging.Tests
|
||||
git commit -m "feat(options): eager startup validation for site-pipeline options (SiteRuntime, S&F, SiteEventLog) (arch-review 08 §1.5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Options validation — edge + management (InboundAPI, ExternalSystemGateway, NotificationService, ManagementService)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 5, 6, 7, 8, 9, 10, 11
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.InboundAPI/InboundApiOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ExternalSystemGatewayOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.NotificationService/NotificationOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.ManagementService/ManagementServiceOptionsValidator.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Host/Program.cs` (line 273 — `Configure<InboundApiOptions>` → `AddOptions().Bind().ValidateOnStart()` + validator registration), `src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway/ServiceCollectionExtensions.cs` (lines 23–24), `src/ZB.MOM.WW.ScadaBridge.NotificationService/ServiceCollectionExtensions.cs` (lines 20–21), `src/ZB.MOM.WW.ScadaBridge.ManagementService/ServiceCollectionExtensions.cs` (lines 15–16)
|
||||
- Modify: the four component csprojs (add `ZB.MOM.WW.Configuration` PackageReference)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests/InboundApiOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests/ExternalSystemGatewayOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests/NotificationOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests/ManagementServiceOptionsValidatorTests.cs`
|
||||
|
||||
1. Write failing tests (Task 2 shape). Rejection cases:
|
||||
- `InboundApiOptions`: `DefaultMethodTimeout = TimeSpan.Zero`, `MaxRequestBodyBytes = 0`. Do **not** require `ApiKeyPepper` non-empty (empty is a legitimate dev default — check `Host/Program.cs` usage before deciding; if a non-empty pepper is required in production the `StartupValidator` owns that, not this validator).
|
||||
- `ExternalSystemGatewayOptions`: `DefaultHttpTimeout = TimeSpan.Zero`, `MaxConcurrentConnectionsPerSystem = 0`.
|
||||
- `NotificationOptions`: `ConnectionTimeoutSeconds = 0`, `MaxConcurrentConnections = 0`.
|
||||
- `ManagementServiceOptions`: `CommandTimeout = TimeSpan.Zero`.
|
||||
2. Run one project, expect FAIL:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
```
|
||||
3. Implement the four validators (key prefixes `ScadaBridge:InboundApi`, `ScadaBridge:ExternalSystemGateway`, `ScadaBridge:Notification`, `ScadaBridge:ManagementService`) and convert each binding site to `AddOptions<T>().BindConfiguration(...)/.Bind(...).ValidateOnStart()` + `TryAddEnumerable` validator registration (exactly the Task 2 step 4 shape).
|
||||
4. Run, expect PASS:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~CompositionRoot"
|
||||
```
|
||||
5. Commit:
|
||||
```bash
|
||||
git add src/ZB.MOM.WW.ScadaBridge.InboundAPI src/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway src/ZB.MOM.WW.ScadaBridge.NotificationService src/ZB.MOM.WW.ScadaBridge.ManagementService src/ZB.MOM.WW.ScadaBridge.Host/Program.cs tests/ZB.MOM.WW.ScadaBridge.InboundAPI.Tests tests/ZB.MOM.WW.ScadaBridge.ExternalSystemGateway.Tests tests/ZB.MOM.WW.ScadaBridge.NotificationService.Tests tests/ZB.MOM.WW.ScadaBridge.ManagementService.Tests
|
||||
git commit -m "feat(options): eager startup validation for edge/management options (InboundAPI, ESG, Notification, ManagementService) (arch-review 08 §1.5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Options validation — central components (Transport, SiteCallAudit, DeploymentManager)
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 6, 7, 8, 9, 10, 11
|
||||
**Files:**
|
||||
- Create: `src/ZB.MOM.WW.ScadaBridge.Transport/TransportOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/SiteCallAuditOptionsValidator.cs`, `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/DeploymentManagerOptionsValidator.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.Transport/ServiceCollectionExtensions.cs` (line 25), `src/ZB.MOM.WW.ScadaBridge.SiteCallAudit/ServiceCollectionExtensions.cs` (lines 40–41), `src/ZB.MOM.WW.ScadaBridge.DeploymentManager/ServiceCollectionExtensions.cs` (line 31)
|
||||
- Modify: the three component csprojs (add `ZB.MOM.WW.Configuration` PackageReference)
|
||||
- Test: `tests/ZB.MOM.WW.ScadaBridge.Transport.Tests/TransportOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests/SiteCallAuditOptionsValidatorTests.cs`, `tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests/DeploymentManagerOptionsValidatorTests.cs`
|
||||
|
||||
1. Write failing tests (Task 2 shape). Rejection cases:
|
||||
- `TransportOptions` (`ScadaBridge:Transport`): `BundleSessionTtlMinutes = 0`, `MaxBundleSizeMb = 0`, `Pbkdf2Iterations = 0` (security-sensitive: also require `Pbkdf2Iterations >= 100_000` so a config typo cannot silently weaken bundle key derivation), `MaxUnlockAttemptsPerSession = 0`, plus positivity for `MaxBundleEntryDecompressedMb`, `MaxBundleEntryCount`, `MaxBundleEntryCompressionRatio`, `MaxUnlockAttemptsPerIpPerHour`, `SchemaVersionMajor`, and `SourceEnvironment` non-empty.
|
||||
- `SiteCallAuditOptions` (`ScadaBridge:SiteCallAudit`): `StuckAgeThreshold = TimeSpan.Zero`, `KpiInterval = TimeSpan.Zero`, `ReconciliationBatchSize = 0`, `RetentionDays = 0`, plus positivity for `RelayTimeout`, `ReconciliationInterval`, `PurgeInterval`. Note: validate the base properties, not the `Resolved*` computed properties.
|
||||
- `DeploymentManagerOptions` (`ScadaBridge:DeploymentManager` — confirm the section name at the binding site; `ServiceCollectionExtensions.cs:31` uses bare `AddOptions<DeploymentManagerOptions>()`, i.e. code-configured defaults, so ValidateOnStart still applies): `LifecycleCommandTimeout = TimeSpan.Zero`, `ArtifactDeploymentTimeoutPerSite = TimeSpan.Zero`, `OperationLockTimeout = TimeSpan.Zero`.
|
||||
2. Run one project, expect FAIL:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
```
|
||||
3. Implement the three validators + registration conversion (Task 2 step 4 shape). Transport becomes:
|
||||
```csharp
|
||||
services.AddOptions<TransportOptions>().BindConfiguration(OptionsSection).ValidateOnStart();
|
||||
services.TryAddEnumerable(
|
||||
ServiceDescriptor.Singleton<IValidateOptions<TransportOptions>, TransportOptionsValidator>());
|
||||
```
|
||||
4. Run, expect PASS:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Transport.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests --filter "FullyQualifiedName~OptionsValidator"
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
```
|
||||
5. Commit:
|
||||
```bash
|
||||
git add src/ZB.MOM.WW.ScadaBridge.Transport src/ZB.MOM.WW.ScadaBridge.SiteCallAudit src/ZB.MOM.WW.ScadaBridge.DeploymentManager tests/ZB.MOM.WW.ScadaBridge.Transport.Tests tests/ZB.MOM.WW.ScadaBridge.SiteCallAudit.Tests tests/ZB.MOM.WW.ScadaBridge.DeploymentManager.Tests
|
||||
git commit -m "feat(options): eager startup validation for central-component options (Transport, SiteCallAudit, DeploymentManager) (arch-review 08 §1.5)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Remove the vestigial site-side notification-list surface
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 4, 5, 7, 8, 9, 11 (NOT 3 — same project; NOT 10 — both touch `docs/components/SiteRuntime.md`? No: Task 10 does not touch that file. Parallelizable with 10 too, but keep 3 sequential.)
|
||||
**Files:**
|
||||
- Delete: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Repositories/SiteNotificationRepository.cs`
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/ServiceCollectionExtensions.cs` (line 65 — remove `services.AddScoped<INotificationRepository, SiteNotificationRepository>();`)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Persistence/SiteStorageService.cs` — remove `StoreNotificationListAsync` (lines 715–739) and `StoreSmtpConfigurationAsync` (lines ~741–785). **Keep** `PurgeCentralOnlyNotificationConfigAsync` (lines 691–713) and the table schemas — the purge is the security cleanup for DBs written by older builds.
|
||||
- Modify tests: `tests/ZB.MOM.WW.ScadaBridge.Host.Tests/CompositionRootTests.cs` (lines 444–451), `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Repositories/SiteRepositoryTests.cs` (~lines 185–200), `tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests/Persistence/ArtifactStorageTests.cs` (all `StoreNotificationListAsync`/`StoreSmtpConfigurationAsync`/`SiteNotificationRepository` usages: ~lines 125–195)
|
||||
- Modify: `docs/components/SiteRuntime.md` (line 15 — drop `SiteNotificationRepository` from the `Repositories/` listing and note why)
|
||||
|
||||
Context (verified): the central-only notification design means `PurgeCentralOnlyNotificationConfigAsync` empties `notification_lists` + `smtp_configurations` on every artifact apply (`DeploymentManagerActor.cs:1556`, `SiteReplicationActor.cs:267`), so the registered `SiteNotificationRepository` can only ever read empty tables, and the two `Store*` methods have **zero production callers** (grep: only tests). `SiteExternalSystemRepository` is NOT vestigial — external system definitions do ship to sites — leave it alone.
|
||||
|
||||
1. Write the failing test first — replace `Site_NotificationRepository_IsSiteImplementation` in `CompositionRootTests.cs:444-451` with a design-invariant lock:
|
||||
```csharp
|
||||
[Fact]
|
||||
public void Site_NotificationRepository_IsNotRegistered()
|
||||
{
|
||||
// Notification delivery is central-only (CLAUDE.md "Notification delivery is
|
||||
// central-only"); the site-local notification_lists/smtp_configurations tables
|
||||
// are purged on every artifact apply, so a site-side INotificationRepository
|
||||
// could only ever serve empty results. It must not be registered at all.
|
||||
using var scope = _host.Services.CreateScope();
|
||||
var repo = scope.ServiceProvider.GetService<INotificationRepository>();
|
||||
Assert.Null(repo);
|
||||
}
|
||||
```
|
||||
2. Run, expect FAIL (repo is still registered):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~Site_NotificationRepository"
|
||||
```
|
||||
3. Implement: delete `SiteNotificationRepository.cs`; remove the DI registration at `SiteRuntime/ServiceCollectionExtensions.cs:65`; delete `StoreNotificationListAsync` + `StoreSmtpConfigurationAsync` from `SiteStorageService.cs`; extend the doc comment on `PurgeCentralOnlyNotificationConfigAsync` with one sentence noting the write paths and site repository were removed (with date + review pointer) so a future maintainer doesn't reintroduce them. Check whether `SiteRuntime/Repositories/SyntheticId.cs` is still used by `SiteExternalSystemRepository` — if yes keep it, if the notification repo was its last consumer delete it too.
|
||||
4. Delete/rewrite the orphaned tests: the notification-list blocks in `SiteRepositoryTests.cs` and `ArtifactStorageTests.cs` (they exercise deleted API). Where a test asserted persistence-across-restart semantics generically, keep the external-system variant only.
|
||||
5. Run, expect PASS:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Host.Tests --filter "FullyQualifiedName~CompositionRoot"
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests
|
||||
dotnet build ZB.MOM.WW.ScadaBridge.slnx
|
||||
```
|
||||
6. Update `docs/components/SiteRuntime.md:15` (`Repositories/` bullet → `SiteExternalSystemRepository` only, with a one-line note that the notification variant was removed because notification config never lives on sites).
|
||||
7. Commit:
|
||||
```bash
|
||||
git add -A src/ZB.MOM.WW.ScadaBridge.SiteRuntime tests/ZB.MOM.WW.ScadaBridge.SiteRuntime.Tests tests/ZB.MOM.WW.ScadaBridge.Host.Tests docs/components/SiteRuntime.md
|
||||
git commit -m "refactor(site-runtime): excise vestigial site-side notification-list surface — repo, DI registration, dead write paths (arch-review 08 §1.3/#23)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 7: PerformanceTests — site-stream throughput test
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 5, 6, 9, 10, 11 (and 8 — different new files)
|
||||
**Files:**
|
||||
- Create: `tests/ZB.MOM.WW.ScadaBridge.PerformanceTests/Streaming/SiteStreamThroughputTests.cs`
|
||||
- (No csproj change needed: `PerformanceTests.csproj` already references SiteRuntime and Commons.)
|
||||
|
||||
This adds the first of the two highest-value real measurements the review called for (§2.3): sustained throughput of the site-wide Akka stream (`SiteStreamManager`, `Source.ActorRef` + `BroadcastHub`, per-subscriber buffering) that carries every attribute value and alarm state change on a site. Follows the existing `HotPathLatencyTests` style: `Stopwatch`-based, `[Trait("Category", "Performance")]`, conservative thresholds so CI stays green on slow machines.
|
||||
|
||||
1. Write the test (this is the implementation — a perf test is its own assertion):
|
||||
```csharp
|
||||
using System.Diagnostics;
|
||||
using Akka.Actor;
|
||||
using Akka.TestKit.Xunit2;
|
||||
using Microsoft.Extensions.Logging.Abstractions;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming;
|
||||
using ZB.MOM.WW.ScadaBridge.SiteRuntime;
|
||||
using ZB.MOM.WW.ScadaBridge.SiteRuntime.Streaming;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.PerformanceTests.Streaming;
|
||||
|
||||
/// <summary>
|
||||
/// Throughput measurement for the site-wide attribute/alarm stream
|
||||
/// (SiteStreamManager: Source.ActorRef → BroadcastHub with per-subscriber
|
||||
/// buffering, DropHead overflow). Design envelope: this stream carries every
|
||||
/// attribute value change on a site; it must comfortably sustain tens of
|
||||
/// thousands of events/sec. Thresholds are deliberately conservative
|
||||
/// (CI-machine safe) — this test exists to catch order-of-magnitude
|
||||
/// regressions, not to benchmark.
|
||||
/// </summary>
|
||||
public class SiteStreamThroughputTests : TestKit
|
||||
{
|
||||
private sealed class CountingActor : ReceiveActor
|
||||
{
|
||||
public static long Received;
|
||||
public CountingActor()
|
||||
{
|
||||
Receive<AttributeValueChanged>(_ => Interlocked.Increment(ref Received));
|
||||
ReceiveAny(_ => { });
|
||||
}
|
||||
}
|
||||
|
||||
[Trait("Category", "Performance")]
|
||||
[Fact]
|
||||
public async Task SiteStream_SustainsAtLeast10kEventsPerSecond_ToOneSubscriber()
|
||||
{
|
||||
const int eventCount = 100_000;
|
||||
CountingActor.Received = 0;
|
||||
|
||||
var manager = new SiteStreamManager(
|
||||
new SiteRuntimeOptions { StreamBufferSize = eventCount },
|
||||
NullLogger<SiteStreamManager>.Instance);
|
||||
manager.Initialize(Sys);
|
||||
|
||||
var counter = Sys.ActorOf(Props.Create<CountingActor>());
|
||||
manager.Subscribe("PerfInstance", counter);
|
||||
|
||||
var sw = Stopwatch.StartNew();
|
||||
for (var i = 0; i < eventCount; i++)
|
||||
{
|
||||
manager.PublishAttributeValueChanged(new AttributeValueChanged(
|
||||
"PerfInstance", "Attr", "Attr", i, "Good", DateTimeOffset.UtcNow));
|
||||
}
|
||||
|
||||
// Drain: wait until the counter goes quiet (no growth across a 200ms window).
|
||||
long last = -1;
|
||||
var deadline = DateTime.UtcNow.AddSeconds(30);
|
||||
while (DateTime.UtcNow < deadline)
|
||||
{
|
||||
var now = Interlocked.Read(ref CountingActor.Received);
|
||||
if (now == last && now > 0) break;
|
||||
last = now;
|
||||
await Task.Delay(200);
|
||||
}
|
||||
sw.Stop();
|
||||
|
||||
var received = Interlocked.Read(ref CountingActor.Received);
|
||||
var eventsPerSecond = received / sw.Elapsed.TotalSeconds;
|
||||
|
||||
// DropHead means under extreme pressure some events may drop; with the
|
||||
// buffer sized to the burst, deliver ratio should be ~1.0.
|
||||
Assert.True(received >= eventCount * 0.9,
|
||||
$"Expected >=90% delivery, got {received}/{eventCount}");
|
||||
Assert.True(eventsPerSecond >= 10_000,
|
||||
$"Expected >=10k events/s sustained, got {eventsPerSecond:F0}/s over {sw.Elapsed.TotalSeconds:F1}s");
|
||||
}
|
||||
}
|
||||
```
|
||||
Adjust to actual constructor/subscribe signatures on read (`SiteStreamManager.Subscribe(string instanceName, IActorRef subscriber)` — verified; subscription filters by `InstanceUniqueName`). If subscriber delivery turns out to route a wrapper message rather than the raw record, match on the wrapper.
|
||||
2. Run, expect PASS (and eyeball the reported rate in the failure-message format by temporarily inverting the assertion if you want the number):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.PerformanceTests --filter "FullyQualifiedName~SiteStreamThroughput"
|
||||
```
|
||||
3. Commit:
|
||||
```bash
|
||||
git add tests/ZB.MOM.WW.ScadaBridge.PerformanceTests/Streaming
|
||||
git commit -m "test(perf): site-wide stream throughput measurement — first real perf-envelope test (arch-review 08 §2.3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 8: PerformanceTests — failover-timing benchmark harness placeholder
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~3 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 5, 6, 7, 9, 10, 11
|
||||
**Files:**
|
||||
- Create: `tests/ZB.MOM.WW.ScadaBridge.PerformanceTests/Failover/FailoverTimingTests.cs`
|
||||
|
||||
The design claims ~25s total failover (CLAUDE.md "Cluster & Failover"). Measuring it needs a real two-node cluster with a hard kill — **that rig is owned by Plan 01** (the two-node failover test rig from overall recommendation P2-10). This task ships the measurement protocol as an explicitly-skipped harness so the perf suite documents the envelope and Plan 01's rig has a ready seat to plug into; it must NOT duplicate the rig.
|
||||
|
||||
1. Create the placeholder:
|
||||
```csharp
|
||||
namespace ZB.MOM.WW.ScadaBridge.PerformanceTests.Failover;
|
||||
|
||||
/// <summary>
|
||||
/// Failover-timing benchmark harness (placeholder).
|
||||
///
|
||||
/// Design envelope under test (CLAUDE.md "Cluster & Failover"): failure
|
||||
/// detection 2s heartbeat / 10s threshold, SBR stable-after 15s, total
|
||||
/// failover ~25s for a hard node loss.
|
||||
///
|
||||
/// Measurement protocol (to be wired to the two-node cluster rig delivered
|
||||
/// by PLAN-01 — see archreview/plans/PLAN-01-*.md; do not duplicate that rig
|
||||
/// here):
|
||||
/// 1. Form a real 2-node cluster (docker/ topology or Akka.Remote in-proc
|
||||
/// multi-ActorSystem rig from PLAN-01).
|
||||
/// 2. Confirm a cluster singleton (e.g. site DeploymentManager) is hosted
|
||||
/// on node A; record T0.
|
||||
/// 3. Hard-kill node A (process kill / ActorSystem.Abort — not
|
||||
/// CoordinatedShutdown; graceful stop exercises a different path).
|
||||
/// 4. Poll node B for singleton re-host; record T1 at first successful
|
||||
/// response from the migrated singleton.
|
||||
/// 5. Assert T1 - T0 <= 40s (25s design + margin) and report the number.
|
||||
/// </summary>
|
||||
public class FailoverTimingTests
|
||||
{
|
||||
[Trait("Category", "Performance")]
|
||||
[Fact(Skip = "Requires the real two-node failover rig delivered by PLAN-01 " +
|
||||
"(overall review P2-10). This class documents the measurement " +
|
||||
"protocol; wire it to the rig when PLAN-01 lands.")]
|
||||
public void HardKillFailover_SingletonRehostedWithinDesignEnvelope()
|
||||
{
|
||||
// Intentionally empty until the PLAN-01 rig exists.
|
||||
}
|
||||
}
|
||||
```
|
||||
2. Verify it compiles and shows as skipped:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.PerformanceTests --filter "FullyQualifiedName~FailoverTiming"
|
||||
```
|
||||
Expected: 1 skipped, 0 failed.
|
||||
3. Commit:
|
||||
```bash
|
||||
git add tests/ZB.MOM.WW.ScadaBridge.PerformanceTests/Failover
|
||||
git commit -m "test(perf): failover-timing harness placeholder documenting the 25s envelope protocol, pending PLAN-01 two-node rig (arch-review 08 §2.3)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 9: Contract-lock tests for top ClusterClient message records
|
||||
|
||||
**Classification:** standard
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 5, 6, 7, 8, 10, 11
|
||||
**Files:**
|
||||
- Create: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/Messages/ClusterClientContractLockTests.cs`
|
||||
- Modify: `Directory.Packages.props` (add `<PackageVersion Include="Newtonsoft.Json" Version="13.0.4" />` — first check `dotnet list tests/ZB.MOM.WW.ScadaBridge.Commons.Tests package --include-transitive | grep -i newtonsoft` and pin to the version Akka already pulls transitively, to avoid a version bump ride-along)
|
||||
- Modify: `tests/ZB.MOM.WW.ScadaBridge.Commons.Tests/ZB.MOM.WW.ScadaBridge.Commons.Tests.csproj` (add `<PackageReference Include="Newtonsoft.Json" />`)
|
||||
|
||||
Rationale (§1.8, Rec 10): ClusterClient traffic rides Akka.NET's default Newtonsoft-JSON serialization; "additive-only evolution" is currently discipline, not enforcement — the gRPC side has proto contract-lock tests (`Communication.Tests/Protos/*ProtoTests.cs`, `Grpc/ProtoContractTests.cs`) but the record contracts have none. These tests lock the version-skew guarantees: (a) an old-shape payload (missing newer trailing optional fields) still deserializes with defaults, (b) unknown fields from a *newer* peer are ignored, (c) round-trip preserves values.
|
||||
|
||||
Records to lock (highest-traffic cross-cluster surface): `ManagementEnvelope` (`Commons/Messages/Management/ManagementEnvelope.cs`), `DeployInstanceCommand` (`Messages/Deployment/DeployInstanceCommand.cs` — has documented optional trailing fields `CentralFetchBaseUrl`/`FetchToken`), `GetAttributeRequest` + `SetStaticAttributeCommand` (`Messages/Instance/`), `AttributeValueChanged` (`Messages/Streaming/`), one notification-forward record from `Messages/Notification/NotificationMessages.cs`, and `CreateDataConnectionCommand` (`Messages/Management/DataConnectionCommands.cs` — the report's own example of defaulted trailing parameters).
|
||||
|
||||
1. Write the failing test file (fails to compile until the package reference lands — that is the FAIL step):
|
||||
```csharp
|
||||
using Newtonsoft.Json;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Management;
|
||||
using ZB.MOM.WW.ScadaBridge.Commons.Messages.Streaming;
|
||||
|
||||
namespace ZB.MOM.WW.ScadaBridge.Commons.Tests.Messages;
|
||||
|
||||
/// <summary>
|
||||
/// Contract-lock tests for the highest-traffic ClusterClient record contracts.
|
||||
/// ClusterClient traffic uses Akka.NET's default Newtonsoft-JSON serializer and
|
||||
/// the design supports system-wide artifact version skew across sites, so these
|
||||
/// records must obey additive-only evolution: old-shape payloads deserialize
|
||||
/// (missing new fields → defaults) and unknown fields are ignored. If a change
|
||||
/// breaks one of these tests, it breaks rolling cross-cluster compatibility —
|
||||
/// do not "fix the test"; make the change additive.
|
||||
/// </summary>
|
||||
public class ClusterClientContractLockTests
|
||||
{
|
||||
private static readonly JsonSerializerSettings AkkaLikeSettings = new()
|
||||
{
|
||||
MissingMemberHandling = MissingMemberHandling.Ignore,
|
||||
};
|
||||
|
||||
[Fact]
|
||||
public void CreateDataConnectionCommand_OldShape_WithoutNewerTrailingFields_Deserializes()
|
||||
{
|
||||
// Shape as emitted before BackupConfiguration/FailoverRetryCount existed.
|
||||
// (Adjust property list to the record's actual required leading fields.)
|
||||
const string oldShape =
|
||||
"""{"Name":"conn1","SiteId":3,"Protocol":"OpcUa","Configuration":"{}"}""";
|
||||
|
||||
var cmd = JsonConvert.DeserializeObject<CreateDataConnectionCommand>(oldShape, AkkaLikeSettings);
|
||||
|
||||
Assert.NotNull(cmd);
|
||||
Assert.Null(cmd!.BackupConfiguration); // default applied
|
||||
Assert.Equal(3, cmd.FailoverRetryCount); // default applied
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void AttributeValueChanged_UnknownFieldFromNewerPeer_IsIgnored()
|
||||
{
|
||||
const string newerShape =
|
||||
"""{"InstanceUniqueName":"i","AttributePath":"a","AttributeName":"a","Value":1,"Quality":"Good","Timestamp":"2026-07-08T00:00:00+00:00","SomeFutureField":"x"}""";
|
||||
|
||||
var evt = JsonConvert.DeserializeObject<AttributeValueChanged>(newerShape, AkkaLikeSettings);
|
||||
|
||||
Assert.NotNull(evt);
|
||||
Assert.Equal("Good", evt!.Quality);
|
||||
}
|
||||
|
||||
[Fact]
|
||||
public void ManagementEnvelope_RoundTrip_PreservesCorrelationId()
|
||||
{
|
||||
// Construct with the real AuthenticatedUser shape (read Commons/…/AuthenticatedUser).
|
||||
var envelope = /* new ManagementEnvelope(user, command: new object(), "corr-1") */ default(ManagementEnvelope);
|
||||
// serialize → deserialize → assert CorrelationId round-trips.
|
||||
}
|
||||
}
|
||||
```
|
||||
Flesh out one old-shape + one unknown-field + one round-trip case per listed record (read each record first; the literals above must match real constructor signatures — that is part of the task, and any mismatch found IS the drift this suite exists to surface).
|
||||
2. Run, expect FAIL (missing Newtonsoft reference → compile error):
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter "FullyQualifiedName~ClusterClientContractLock"
|
||||
```
|
||||
3. Add the `PackageVersion` to `Directory.Packages.props` and the `PackageReference` to the Commons.Tests csproj; finish the test bodies.
|
||||
4. Run, expect PASS:
|
||||
```bash
|
||||
dotnet test tests/ZB.MOM.WW.ScadaBridge.Commons.Tests --filter "FullyQualifiedName~ClusterClientContractLock"
|
||||
```
|
||||
5. Commit:
|
||||
```bash
|
||||
git add Directory.Packages.props tests/ZB.MOM.WW.ScadaBridge.Commons.Tests
|
||||
git commit -m "test(commons): contract-lock tests for top ClusterClient records — enforce additive-only evolution under version skew (arch-review 08 §1.8)"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 10: Docs-code drift + conventions cleanup batch (Low findings, consolidated)
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 2, 3, 4, 5, 6, 7, 8, 9, 11 (NOT 1 — both edit CLAUDE.md)
|
||||
**Files:**
|
||||
- Modify: `docs/requirements/Component-Commons.md` (layout tree, lines ~293–294)
|
||||
- Modify: `docs/requirements/Component-ConfigurationDatabase.md` (repositories section)
|
||||
- Modify: `src/ZB.MOM.WW.ScadaBridge.ManagementService/ZB.MOM.WW.ScadaBridge.ManagementService.csproj` (line 22)
|
||||
- Modify: `README.md` (line ~111, the "One doc per component" claim)
|
||||
- Modify: `docs/components/README.md` (index)
|
||||
- Modify: `CLAUDE.md` (Document Conventions section)
|
||||
|
||||
Concrete edits (each verified against current file state):
|
||||
|
||||
1. **Component-Commons.md — add `Observability/`** (§1.1, narrowed: `Serialization/` and `Validators/` are already documented at lines 293–294; only `Observability/` — which contains `ScadaBridgeTelemetry.cs` — is missing). In the layout tree, before the `├── Serialization/` line add:
|
||||
```
|
||||
├── Observability/ # ScadaBridgeTelemetry (ActivitySource/metrics names)
|
||||
```
|
||||
(Adjust the comment after reading `src/ZB.MOM.WW.ScadaBridge.Commons/Observability/ScadaBridgeTelemetry.cs` — describe what it actually holds.)
|
||||
2. **Component-ConfigurationDatabase.md — document the site-side repository exception** (§1.3). In the section stating repository implementations live in Configuration Database, add:
|
||||
> Site-side exception: read-only SQLite-backed implementations of shared repository interfaces live in `src/ZB.MOM.WW.ScadaBridge.SiteRuntime/Repositories/` (currently `SiteExternalSystemRepository`) — same Commons interface, site-local store, writes throw `NotSupportedException` because site artifacts are managed exclusively by deployment from Central.
|
||||
|
||||
(If Task 6 has not run yet, mention `SiteNotificationRepository` too and let Task 6's doc step remove it; if running after Task 6, list only the external-system repository.)
|
||||
3. **ManagementService.csproj line 22** — change `..\ZB.MOM.WW.ScadaBridge.InboundAPI\ZB.MOM.WW.ScadaBridge.InboundAPI.csproj` to `../ZB.MOM.WW.ScadaBridge.InboundAPI/ZB.MOM.WW.ScadaBridge.InboundAPI.csproj` (§1.4 cosmetic; every other reference in the repo uses `/`).
|
||||
4. **README.md ~line 111** (§4): replace the claim "One doc per component (plus the shared TreeView)" with an accurate scope, e.g.:
|
||||
> Reference docs exist for 24 of the 27 components (plus the shared TreeView); ScriptAnalysis (#25), KpiHistory (#26), and DelmiaNotifier (#27) are pending — tracked in `docs/plans/2026-07-08-deferred-work-register.md`.
|
||||
5. **docs/components/README.md**: add the same three-pending note to the index so the two lists can't drift independently.
|
||||
6. **CLAUDE.md Document Conventions** (§4 DelmiaNotifier): add one bullet:
|
||||
> - Exception: DelmiaNotifier (#27) is an external client tool, not a cluster component; its spec is the project README (`src/ZB.MOM.WW.ScadaBridge.DelmiaNotifier/README.md`) plus `docs/plans/2026-06-26-delmia-recipe-notifier-design.md` — there is intentionally no `Component-DelmiaNotifier.md`.
|
||||
7. Verify (config/docs task — verification replaces the failing test):
|
||||
```bash
|
||||
dotnet build src/ZB.MOM.WW.ScadaBridge.ManagementService # csproj separator change still builds
|
||||
grep -n "Observability" docs/requirements/Component-Commons.md # expect: 1 hit in the tree
|
||||
grep -rn '\\\\ZB.MOM' src --include="*.csproj" # expect: no hits
|
||||
grep -n "24 of the 27\|pending" README.md docs/components/README.md # expect: scoped claim present
|
||||
grep -n "Component-DelmiaNotifier" CLAUDE.md # expect: the exception bullet
|
||||
```
|
||||
8. Commit:
|
||||
```bash
|
||||
git add docs/requirements/Component-Commons.md docs/requirements/Component-ConfigurationDatabase.md src/ZB.MOM.WW.ScadaBridge.ManagementService/ZB.MOM.WW.ScadaBridge.ManagementService.csproj README.md docs/components/README.md CLAUDE.md
|
||||
git commit -m "docs: close arch-review 08 drift batch — Commons Observability folder, site-repo exception, csproj separator, docs/components scope, DelmiaNotifier convention exception"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 11: Deferred-work register — triage the 23-item inventory into a tracked list
|
||||
|
||||
**Classification:** small
|
||||
**Estimated implement time:** ~5 min
|
||||
**Parallelizable with:** 1, 2, 3, 4, 5, 6, 7, 8, 9 (write after 10 only if you want the README pointer target to exist first — create this file before or simultaneously; the README in Task 10 points here)
|
||||
**Files:**
|
||||
- Create: `docs/plans/2026-07-08-deferred-work-register.md`
|
||||
|
||||
Turn the review's §3 inventory (23 items) into the explicit tracked register the review asked for: every item gets a row with **status** (`fix-now → plan NN` / `deferred`), **owner/where-noted**, **rationale**, and a **revisit trigger** (the concrete condition that reopens it). This is the single place the "consciously logged" deferrals live from now on; new deferrals get appended here instead of scattering.
|
||||
|
||||
1. Create `docs/plans/2026-07-08-deferred-work-register.md` with this exact structure and content (source: `archreview/08-conventions-tests-underdeveloped.md` §3; carry each item's file/line citation over):
|
||||
|
||||
```markdown
|
||||
# Deferred-Work Register (established 2026-07-08, from architecture review 08)
|
||||
|
||||
Single tracked list of consciously-deferred work. Rules: every deferral gets a row
|
||||
(rationale + revisit trigger); fix-now items reference the archreview plan that owns
|
||||
them and are removed from this table when that plan's task lands.
|
||||
|
||||
## Fix-now (owned by archreview plans)
|
||||
| # | Item | Owner |
|
||||
|---|------|-------|
|
||||
| 1 | Transport: Area membership doesn't travel in bundles (EntitySerializer.cs:251,518) | PLAN-05 |
|
||||
| 2 | CLI.Tests not in slnx | PLAN-08 Task 1 |
|
||||
| 3 | AuditLog reconciliation dials site NodeA only (SiteEnumerator.cs:32,64) | PLAN-04 |
|
||||
| 4 | Options validation missing on ~12 components | PLAN-08 Tasks 2–5 |
|
||||
| 5 | SmsConfiguration.MaxRetryCount stored but not honored | PLAN-06 |
|
||||
| 6 | Role-check case-sensitivity asymmetry (ManagementActor.cs:1910) | PLAN-07 |
|
||||
| 23 | Vestigial site-side notification-list surface | PLAN-08 Task 6 |
|
||||
|
||||
## Deferred (with rationale + revisit trigger)
|
||||
| # | Item | Where noted | Rationale for deferral | Revisit trigger |
|
||||
|---|------|-------------|------------------------|-----------------|
|
||||
| 7 | SecuredWrite audit rows leave SourceNode NULL | CLAUDE.md Security | Logged follow-up; low-volume channel | First per-node forensic query against secured writes |
|
||||
| 8 | Hash-chain tamper evidence (T1); CLI verify-chain is a no-op stub | audit-log roadmap :12 | v1.x by locked decision; append-only DB roles are the control | Compliance requirement for cryptographic tamper evidence |
|
||||
| 9 | Parquet audit archival (T2); endpoint returns 501 | AuditEndpoints.cs:204 | v1.x; 501 + CLI messaging are honest | AuditLog partition volume nears retention ceiling |
|
||||
| 10 | Aggregated live alarm stream for Alarm Summary | m7 design :252 | Snapshot fan-out acceptable at current instance counts | Alarm Summary latency complaints or >~50 instances/site |
|
||||
| 11 | Central-persisted OPC UA cert-trust audit | m7 follow-ups | Broadcast-to-both-nodes covers HA | Governance/audit requirement for trust decisions |
|
||||
| 12 | Native-alarm-source-override CSV import | m7 follow-ups | Parity gap only; attribute CSV shipped | First bulk native-alarm rollout request |
|
||||
| 13 | WaitForAttribute quality-gated mode | ScriptRuntimeContext.cs:405 | Planned enhancement per spec §4.2 | First script needing Good-only waits |
|
||||
| 14 | WaitForAttribute in Test-Run sandbox | waitfor-deferred :222 | Test-Run parity gap; production unaffected | Author feedback on Test-Run fidelity |
|
||||
| 15 | BrowseNext final-page signal not surfaced | BrowseCommands.cs:34 | One wasted round-trip | UX complaint on browse paging |
|
||||
| 16 | StubOpcUaClient throws on browse | m7 design :245 | Limits offline UI coverage only | Next browse/search UI regression |
|
||||
| 17 | Unified notifications+site-calls outbox page | stillpending :118 | Explicit M9 decision to keep two pages | Operator confusion reports |
|
||||
| 18 | Folder drag-drop | same, [PERM] | Permanently closed; menu reorder shipped | — (closed) |
|
||||
| 19 | Bundle signing / cluster-to-cluster pull / differential bundles | transport-design :402 | v1 manifest hash + AES-GCM held sufficient | Non-repudiation requirement across orgs |
|
||||
| 20 | Deployment EXPIRED-row purge | DeploymentManagerRepository.cs:310 | Read path already compensates | EXPIRED rows visible in ops queries |
|
||||
| 21 | SiteAuditBacklogReporter threshold consolidation | SiteAuditBacklogReporter.cs:28 | Config-shape cleanup only | Next SqliteAuditWriterOptions change |
|
||||
| 22 | KPI history hourly rollups | Component-KpiHistory.md | YAGNI; 90-day retention bounds table | KpiSample query latency on dashboards |
|
||||
|
||||
## New deferrals from review 08 (this plan)
|
||||
| Item | Rationale | Revisit trigger |
|
||||
|------|-----------|-----------------|
|
||||
| Communication → HealthMonitoring layering (ICentralHealthAggregator consumed by CentralCommunicationActor.cs:351) | Moving the interface + SiteHealthState to Commons ripples across 5 projects for a cosmetic inversion | Next breaking change to ICentralHealthAggregator |
|
||||
| docs/components reference docs for ScriptAnalysis, KpiHistory, DelmiaNotifier | Reference docs are substantial (StyleGuide-conformant); README claim scoped instead (PLAN-08 Task 10) | Next doc-writing session touching those components |
|
||||
| Test-coverage backfill: SiteCallAudit.Tests (31 tests/1.6k LOC), DeploymentManager.Tests | No defect identified; coverage partly lives in ManagementService/Host/Integration suites | First regression escaping either component |
|
||||
| Failover-timing + broader perf envelope (S&F drain rate, per-subscriber backpressure) | Needs PLAN-01 two-node rig; placeholder harness shipped (PLAN-08 Task 8) | PLAN-01 rig landing |
|
||||
```
|
||||
2. Verify (docs task):
|
||||
```bash
|
||||
grep -c "^| " docs/plans/2026-07-08-deferred-work-register.md
|
||||
```
|
||||
Expected: ≥ 30 table rows (7 fix-now + 16 deferred + 4 new + headers).
|
||||
3. Commit:
|
||||
```bash
|
||||
git add docs/plans/2026-07-08-deferred-work-register.md
|
||||
git commit -m "docs(plans): deferred-work register — triage the 23-item arch-review inventory into fix-now (plan-owned) vs deferred-with-rationale"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
## Dependencies on other plans
|
||||
|
||||
- **Plan 01 (Cluster/Host/Failover):** Task 8 is a placeholder that documents the failover-timing measurement protocol; the actual two-node kill rig is Plan 01's (overall review P2-10). When Plan 01's rig lands, un-skip `FailoverTimingTests` and wire it in. No file overlap expected, but Plan 01 may touch `Host/Program.cs` (Task 4 also edits it — coordinate merge order).
|
||||
- **Plan 05 (Templates/Deploy/Transport):** owns the Transport `AreaName: null` contradiction (§3 item 1 / §4). Plan 05 also touches `Transport/`; Task 5 here only touches `Transport/ServiceCollectionExtensions.cs` + a new validator file — small merge-conflict surface.
|
||||
- **Plan 04 (Data/Audit):** owns AuditLog NodeA-only reconciliation (§3 item 3).
|
||||
- **Plan 06 (Edge integrations):** owns `SmsConfiguration.MaxRetryCount` (§3 item 5).
|
||||
- **Plan 07 (UI/Management/Security):** owns role-casing asymmetry (§3 item 6). Plan 07 may also touch `ManagementActor.cs`; Task 4 here touches only `ManagementService/ServiceCollectionExtensions.cs` + csproj.
|
||||
- The register (Task 11) cites plan numbers 04/05/06/07 — content-only references, no execution dependency.
|
||||
|
||||
## Execution order
|
||||
|
||||
1. **Task 1 first** (slnx) — it changes what `dotnet test ZB.MOM.WW.ScadaBridge.slnx` covers, so every later task's solution-level verification includes CLI.Tests.
|
||||
2. **Tasks 2, 4, 5, 7, 8, 9, 11 in parallel** (disjoint files).
|
||||
3. **Task 3 then Task 6 sequentially** (both live in the SiteRuntime project; either order works, but running 6 after 3 keeps the CompositionRoot test churn in one direction).
|
||||
4. **Task 10 last among docs** (it edits CLAUDE.md — serialize with Task 1's CLAUDE.md edit — and its README pointer targets the Task 11 register, so run after 11).
|
||||
5. Finish with a full `dotnet build ZB.MOM.WW.ScadaBridge.slnx && dotnet test ZB.MOM.WW.ScadaBridge.slnx` — expect all 30 test projects (now including CLI.Tests' ~279) green.
|
||||
@@ -0,0 +1,72 @@
|
||||
{
|
||||
"planPath": "archreview/plans/PLAN-08-conventions-tests.md",
|
||||
"tasks": [
|
||||
{
|
||||
"id": 1,
|
||||
"subject": "Task 1: Add CLI.Tests to the slnx and retire the gotcha documentation",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 2,
|
||||
"subject": "Task 2: Options validation — Communication + DataConnectionLayer",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 3,
|
||||
"subject": "Task 3: Options validation — site pipeline (SiteRuntime, StoreAndForward, SiteEventLogging)",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 4,
|
||||
"subject": "Task 4: Options validation — edge + management (InboundAPI, ExternalSystemGateway, NotificationService, ManagementService)",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 5,
|
||||
"subject": "Task 5: Options validation — central components (Transport, SiteCallAudit, DeploymentManager)",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 6,
|
||||
"subject": "Task 6: Remove the vestigial site-side notification-list surface",
|
||||
"status": "pending",
|
||||
"blockedBy": [3]
|
||||
},
|
||||
{
|
||||
"id": 7,
|
||||
"subject": "Task 7: PerformanceTests — site-stream throughput test",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 8,
|
||||
"subject": "Task 8: PerformanceTests — failover-timing benchmark harness placeholder",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 9,
|
||||
"subject": "Task 9: Contract-lock tests for top ClusterClient message records",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
},
|
||||
{
|
||||
"id": 10,
|
||||
"subject": "Task 10: Docs-code drift + conventions cleanup batch (Low findings, consolidated)",
|
||||
"status": "pending",
|
||||
"blockedBy": [1, 11]
|
||||
},
|
||||
{
|
||||
"id": 11,
|
||||
"subject": "Task 11: Deferred-work register — triage the 23-item inventory into a tracked list",
|
||||
"status": "pending",
|
||||
"blockedBy": []
|
||||
}
|
||||
],
|
||||
"lastUpdated": "2026-07-08"
|
||||
}
|
||||
Reference in New Issue
Block a user