Commit Graph

27 Commits

Author SHA1 Message Date
Joseph Doherty 7804fe7958 docs: arch-review remediation — component docs sweep, execution log, residuals register
Final consistency sweep per plan §6: verified component docs against shipped
WP1-WP3 + adversarial-review-fix state, corrected drift found in SiteRuntime
(recursion-exempt run cap, stale ScriptExecutionActor/AlarmExecutionActor
references), TemplateEngine (BundleImporter watermark path), DeploymentManager
(phase-2 PendingDeployment staging), CentralUI (shared KPI cache, dedup'd alarm
poll, render coalescing), StoreAndForward (rate-limited drop logging), and
ConfigurationDatabase (documented DbContext-pooling non-adoption). Updated the
docs/components/ developer-reference set (SiteRuntime, SiteEventLogging,
InboundAPI) to drop the deleted per-run actor classes. Amended one known-issue
for the superseding MaxBatchSize:64 read-page pin. Added CLAUDE.md bullets for
stream graceful-completion reconnect, the required site audit DB path, honest
CLI HTTP timeouts, bulk DeploySiteAsync, and LocalDb 0.2.1. New execution log
records the phase→commit map, gate results, adversarial-review tally, the
three test-flake root causes, and the nine-item residuals register.
2026-08-15 01:30:59 -04:00
Joseph Doherty 5d075f1374 fix(central): review findings — no client-side audit truncation, insert-first upsert, QI-safe scripts, honest operator replies
Six adversarial-review findings in the central SQL/ingest layer.

F1 (AuditLogRepository.InsertChunkAsync) — the set-based ingest declared each
string parameter at its COLUMN width (Actor/Target 256, Action 64, Outcome 16,
Category 32, SourceNode 64), so SqlClient truncated an over-long value at bind
time and committed the mutilated row — silent, in an append-only store, with no
PayloadTruncated flag — while the per-row and reconciliation paths sent the same
value in full and let the server reject it with 2628. Bind at the value's own
length instead; explicit SqlDbType is kept (it fixes the VALUES constructor's
derived column types and datetime2 precision). Design: reject everywhere,
truncate nowhere — matching today's per-row behaviour.

F2 (SiteCallAuditRepository.UpsertAsync) — the single-statement upsert ran the
monotonic UPDATE first and INSERTed only if nothing matched. Two writers racing
the first packet of one TrackedOperationId (the cached dual-write and the
reconciliation pull carry DIFFERENT lifecycle states) both matched nothing, and
the loser then skipped its INSERT or swallowed a 2627 — dropping its
Status/RetryCount/HttpStatus/TerminalAtUtc. Legs swapped to
`IF NOT EXISTS … INSERT; UPDATE <monotonic>` — still one round trip, and the
loser's UPDATE now lands on the winner's row. The duplicate-key catch re-runs
the monotonic UPDATE for the same reason. Moved to raw SQL with explicitly-typed
parameters so the intricate rank predicate exists in exactly one place (an
untyped DateTime would bind as `datetime` and round the freshness tiebreaker).

F3 (docs/plans/sql/*.sql) — filtered-index DDL failed with error 1934 under the
documented `docker exec … sqlcmd` path, which defaults QUOTED_IDENTIFIER OFF;
once IX_Notifications_Delivered exists, QI-OFF DML on Notifications fails too.
All four scripts now open with `SET QUOTED_IDENTIFIER ON; SET ANSI_NULLS ON; GO`
(own batch, so it is in force when the next batch parses), and the migration
convention in Component-ConfigurationDatabase.md documents `sqlcmd -I`. Verified
live: the pre-fix script fails 1934 without -I, the fixed one applies.

F4 (SiteCallAuditActor) — the off-mailbox reconciliation/purge passes reuse the
injected repository, so tests drove one DbContext from the pass and a mailbox
handler concurrently. Serialized at the CALL via a private SerializedRepository
wrapper applied only by the test constructors, rather than running the pass
on-mailbox: production keeps its PipeTo shape untouched, and the existing
"a blocked drain does not stall ingest/query/KPI" regression tests stay
meaningful (they would have been invalidated by suspending the mailbox).

F5 (AuditLogIngestActor) — when the batch failed because the 20 s IngestBudget
expired, the per-row fallback reused the same expired token: N instant failures,
N counter bumps, zero accepted. The fallback now gets a fresh 5 s budget (inside
the 30 s outer Ask), and a blown budget bumps the failure counter ONCE for the
batch instead of once per row.

F6 (NotificationOutboxRepository.UpdateAsync) — ExecuteUpdate's row count was
discarded, so an operator Retry/Discard of a notification the retention purge had
already deleted reported success (the pre-ExecuteUpdate code threw
DbUpdateConcurrencyException). UpdateAsync now returns whether a row matched; the
operator one-shots answer "notification not found" and emit no audit row for the
action that did not happen, while the dispatcher logs a warning (its delivery
already happened; nothing to retry). GetByIdAsync switched to AsNoTracking since
the write is out-of-band.

Tests: 5 new SQL-backed regressions (over-long Target rejected on both paths +
boundary round-trip; concurrent first-write and already-created-by-another-writer
upserts; vanished-row UpdateAsync), a token-identity pin on the ingest fallback,
a repository-concurrency detector for the SiteCallAudit passes, and vanished-row
operator-path tests. The F1/F2/F4 regressions were each confirmed failing against
the pre-fix code. Suites: ConfigurationDatabase 369, AuditLog 378, SiteCallAudit
66, NotificationOutbox 152 — all green, solution builds with 0 warnings.
2026-08-14 23:46:28 -04:00
Joseph Doherty f25b75a64d Merge branch 'worktree-agent-a7084b23177344196' into arch-review-remediation 2026-08-14 21:15:31 -04:00
Joseph Doherty 2ce0ad7ed1 perf(comms): alarms-only seed, capped buffers, at-least-once audit pull 2026-08-14 21:10:19 -04:00
Joseph Doherty 5db2a810c0 perf(central): set-based ingest, aligned partition purge, KPI query shapes, EF hygiene 2026-08-14 21:07:12 -04:00
Joseph Doherty d8487eab2d docs: close arch-review 08 drift batch — Commons Observability folder, site-repo exception, csproj separator, docs/components scope, DelmiaNotifier convention exception
Claude-Session: https://claude.ai/code/session_01MtdgwpEeCUn6cUA5f1LMPj
2026-07-10 07:00:09 -04:00
Joseph Doherty 7f1b59ff84 docs: reconcile design docs with shipped code (enums, Sms, per-channel retention, SiteCalls schema, conventions)
PLAN-04 Task 24. Component-Commons: AuditChannel gains SecuredWrite (5), AuditKind full
15-value list (+SecuredWrite*/ReconciliationAbandoned), NotificationType Email/Sms (Teams
evaluated+dropped), AuditLogRow persistence-only-projection deviation note. ConfigDB:
Type discriminator Email/Sms, data-layer conventions (UTC DateTime + Utc converter, MSSQL-
only repos/fixtures, AuditLog clustered-key + P3/P4/P6 tracked follow-ups). SiteCallAudit:
schema table rewritten to shipped columns (Channel/Target/AuditStatus-Status/SourceNode/
HttpStatus/IngestedAtUtc; no Kind/TargetSummary/provenance) + stored-vs-tracking-view note.
AuditLog: per-channel retention config lists all 5 channels incl SecuredWrite. Also fixed
two stale 'Teams planned' refs surfaced by the verification grep (HighLevelReqs, CentralUI).
2026-07-09 10:03:13 -04:00
Joseph Doherty eb69b93947 docs(audit-log): document append-only enforcement honestly (CI guard default; DB roles are optional DBA hardening; purger now has CREATE TABLE + scoped DELETE) 2026-07-09 07:03:10 -04:00
Joseph Doherty ba335519f4 docs(m9): mark M9 delivered + sync TemplateEngine/TreeView/DataConnection/schema-library/CLI docs
- 2026-06-15-stillpending-completion-design.md: M9 section marked DELIVERED with per-feature
  summary and deferrals (folder drag-drop, unified outbox page).
- stillpending.md: T22–T26/T28/T30–T32 + CLI cached-call marked [DELIVERED M9]; permanent
  deferrals (folder drag-drop, unified outbox page) retained as [PERM].
- Component-TemplateEngine.md: TemplateFolder SortOrder + ReorderTemplateFolderCommand;
  Expression-trigger analysisKind (Advisory/Strict) on Alarm + Script; Script parameter
  JSON Schema / lib: ref note; Inheritance Resolve authoring section (GetResolvedTemplateMembersCommand /
  TemplateInheritanceResolver / staleness banner); updated Responsibilities.
- Component-TreeView.md: T22 search box wired note; T23 folder sibling reorder + root context
  menu note; drag-drop permanently deferred clarified in V7 worked example.
- Component-CentralUI.md: template tree search + inherited-members panel (T26 staleness banner)
  added to Template Authoring; drag-drop permanently deferred note; Schema Library page (T32)
  added as new subsection; ParameterValueForm + Monaco hover (T30/T31) noted; connection
  live-status (T25) + move-connection (T24) added to Site & Data Connection Management.
- Component-ConfigurationDatabase.md: SharedSchema entity + ISharedSchemaRepository row added.
- Component-CLI.md: --trigger-kind option added to template alarm add/update and script add/update.
- src/ZB.MOM.WW.ScadaBridge.CLI/README.md: --trigger-kind option added to template alarm
  add/update and template script add/update command tables (already had cached-call group).
2026-06-18 13:39:33 -04:00
Joseph Doherty 13605d3dfd docs(m4.1): reconcile Config-DB AuditLog schema + Commons (AuditEvent/ApiKey/SiteCall/NotificationType) to shipped code 2026-06-16 20:29:16 -04:00
Joseph Doherty 43228185b4 docs: convert standard diagrams from draw.io PNGs to inline Mermaid
Gitea renders mermaid inline, so the flow/state/hierarchy/DAG diagrams
move to text-in-markdown: auto-layout (removes the manual overlap-prone
draw.io step), diffable source, no committed binaries, and a dark-text
theme so labels stay legible. Keep draw.io PNGs only for the two complex
bespoke diagrams (logical architecture, env2 topology) where pixel
control still wins. All 24 mermaid blocks validated by rendering.
2026-06-01 00:23:00 -04:00
Joseph Doherty bdee12f4e9 docs: render architecture & flow diagrams as draw.io charts
Replace ASCII-art diagrams across the README and docs/ with editable
.drawio sources plus exported PNGs, so the diagrams render clearly in
rendered markdown and can be maintained/regenerated instead of being
hand-edited as fragile text art. Non-diagram blocks (code, folder
trees, UI wireframes) were left as text.
2026-05-31 23:32:53 -04:00
Joseph Doherty 003e54c1fb docs: native alarm ingestion across component docs + CLAUDE.md 2026-05-31 02:55:00 -04:00
Joseph Doherty 7b0b9c7365 refactor: rename ScadaLink → ZB.MOM.WW.ScadaBridge (code + projects + namespaces)
Solution + 23 src projects + 26 test projects renamed; folders, csproj,
namespaces, and ScadaLinkDbContext/ScadaBridgeDbContext class updated.
ActorSystem "scadalink" → "scadabridge", Akka seed-node URLs migrated.
SQL roles/logins, LDAP domains, CLI command name, and CLI config dir
(~/.scadalink → ~/.scadabridge) also renamed.

Build green; 5 Host.Tests fail awaiting SQL login rename in next commit.
Pre-existing StaleTagMonitor timing flakes unchanged.

Rename script committed at tools/rename-to-scadabridge.sh.
2026-05-28 09:37:45 -04:00
Joseph Doherty 487859bff0 docs+code: close Theme 1 — 24 design-doc / XML-doc drift findings
Doc/XML-comment drift + small adherence fixes across 17 modules. Highlights:
- Host-017: site CoordinatedShutdown ordering — SiteStreamGrpcServer gains
  CancelAllStreams() (refuse new streams, cancel active), wired into
  Program.cs site branch via ApplicationStopping.
- InboundAPI-021: ParentExecutionId now travels on RouteToGet/SetAttributes
  symmetric with RouteToCallRequest; RouteHelper stamps from _parentExecutionId.
- ClusterInfra-012: ClusterOptionsValidator now requires both seed nodes.
- Comm-018: SiteCommunicationActor.HeartbeatMessage.IsActive derived from
  cluster leader check (was hardcoded true).
- DM-020: reconciliation audit row attributes the current user, not prior deployer.
- SEL-019: EventLogPurgeService early-exits on standby via active-node check.
- Plus comment/XML-doc accuracy fixes across AuditLog, ConfigurationDatabase,
  NotificationOutbox, SiteRuntime, SiteCallAudit; doc refreshes for Component-
  Commons / -ManagementService / -CLI / -ExternalSystemGateway / -HealthMonitoring
  / -Transport / -ConfigurationDatabase; CD-023 index-name doc alignment.

11 new regression tests (RouteHelper x4, SiteStreamGrpcServer x2,
ClusterOptionsValidator x1, SiteCommunicationActor x1, DeploymentService x1,
EventLogPurgeService x3). Build clean (0 warnings); InboundAPI/Communication/
Host suites all green. README regenerated: 112 open (was 136).
2026-05-28 06:28:31 -04:00
Joseph Doherty b1daf9abb8 docs: README + component cross-references for Transport (#24) 2026-05-24 04:52:55 -04:00
Joseph Doherty f205746894 docs(audit): disambiguate Config DB refs to 'audit log viewer' post-rename
Task 10's reviewer noted that Component-CentralUI.md renamed the
IAuditService page from 'Audit Log Viewer' to 'Configuration Audit Log
Viewer' to avoid collision with the new operational Audit Log page (#23).
Two stale lowercased refs in Component-ConfigurationDatabase.md needed
the same disambiguation.
2026-05-20 08:39:13 -04:00
Joseph Doherty 36a598840f docs(audit): add AuditLog table, partitioning, and DB roles to Config DB 2026-05-20 07:58:27 -04:00
Joseph Doherty 77cc4865c5 docs(requirements): add SiteCalls table and purge to Configuration Database 2026-05-19 11:53:54 -04:00
Joseph Doherty bb35453d58 docs(notification-outbox): fix stale cross-references after outbox addition 2026-05-18 23:45:43 -04:00
Joseph Doherty 097bb3361e docs(notification-outbox): polish Task 8 docs — purge basis, message-contract naming, density 2026-05-18 23:34:40 -04:00
Joseph Doherty ef4614d710 docs(notification-outbox): clarify purge is a repository operation, not job ownership 2026-05-18 23:32:07 -04:00
Joseph Doherty bb5519f6f2 docs(notification-outbox): add Notifications table, entity, and message contracts 2026-05-18 23:30:16 -04:00
Joseph Doherty 7d1cc5cbb4 fix(configuration-database): resolve ConfigurationDatabase-005,006,008,009,010,011 — bounded gRPC columns, split queries, CSV-parse logging, null guards, coverage 2026-05-16 22:14:23 -04:00
Joseph Doherty 8155dbc411 docs(templates): describe folder hierarchy and management commands 2026-05-11 11:28:09 -04:00
Joseph Doherty 416a03b782 feat: complete gRPC streaming channel — site host, docker config, docs, integration tests
Switch site host to WebApplicationBuilder with Kestrel HTTP/2 gRPC server,
add GrpcPort/keepalive config, wire SiteStreamManager as ISiteStreamSubscriber,
expose gRPC ports in docker-compose, add site seed script, update all 10
requirement docs + CLAUDE.md + README.md for the new dual-transport architecture.
2026-03-21 12:38:33 -04:00
Joseph Doherty d91aa83665 refactor(docs): move requirements and test infra docs into docs/ subdirectories
Organize documentation by moving requirements (HighLevelReqs, Component-*,
lmxproxy_protocol) to docs/requirements/ and test infrastructure docs to
docs/test_infra/. Updates all cross-references in README, CLAUDE.md,
infra/README, component docs, and 23 plan files.
2026-03-21 01:11:35 -04:00