Secret-store outage (final review M5): SiteSecretsCollector returns a store
read failure as SiteSecretsCollection.StoreError instead of throwing. A deploy
now fails only the sites that reference a secret (nothing sent, not stamped,
DeploySiteSecrets row with the error), and ComputeCurrentArtifactHashesAsync
omits only those sites (they read Out of date) instead of the exception
aborting the whole sweep and the fleet-wide pending-updates check.
Removal-only deploys: the site reply gains SecretsRemoved (proto field 7,
names only) - the deployed_secrets rows the deploy soft-removed. Central
carries it on SiteArtifactResult and writes the DeploySiteSecrets row with a
removed list whenever it is non-empty, so dropping a site's last secret:
reference is audited centrally rather than only in site logs.
I2: topology guide + Component-SiteRuntime + register row 127 state that a site pair using
deployed secrets must run LocalDb replication. I3: row 127 + topology guide state the upgrade
impact (whole artifact deploy fails and reads Out of date until referenced secrets exist
centrally). M6: audit row carries secrets/missing, no removed field; removal-only deploys write
no central row. M2/M3/I1 as-built notes; CLAUDE.md secrets bullet corrected.
The site applies DeployArtifactsCommand.Secrets as the full set (rows absent from it are
soft-removed and DeployedSecretsMirror deletes them on both nodes), while null means
'central shipped no secrets' and touches nothing. The command builder shipped null for a
site with no references, so dropping the LAST secret: reference never removed the secret
from either site node. Found by the Task 13 live gate (check 5: site-a-a and site-a-b
both still held sb-gate-mxgw r1, deployed_secrets.is_removed=0 after the removal deploy).
Central now always ships a list. Old sites ignore proto field 9; the artifact hash is
unchanged (empty and null hash identically); SecretsDeployed stays null and no
DeploySiteSecrets audit row is written when nothing shipped. Regression test
NoSecretReferences_ShipsEmptySecretsList_SoTheSiteRemovesPreviouslyDeployedOnes (red
before, green after). Spec notes in Component-DeploymentManager/SiteRuntime + the design
doc's as-built section.
Script file access (design docs/plans/2026-09-28-script-file-access-design.md): Files.ReadTextAsync /
ReadLinesAsync / ExistsAsync on site ScriptGlobals (+ Instance.Files), backed by the site-singleton
ScriptFileReader. The one host-mediated exception to the System.IO ban - scripts still never name a
System.IO type. OFF by default: ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots (empty = every call
throws Disabled) + MaxFileBytes (1 MiB, validated 1..64 MiB, roots must be absolute). Guard: fully
qualified, no device paths / ADS, canonical path strictly inside a root (directory-boundary match), no
reparse point on the file or any directory below the root, size cap before and during the read.
Errors are one Commons type, ScriptFileAccessException(+ScriptFileAccessFailure); Commons.Types.Scripts
joins the default script imports on every surface so it binds by short name. Mirrored on
ScriptCompileSurface (CompileFiles, parity-pinned) and the Test Run sandbox (binds, refuses Disabled).
Inbound (central) scripts do not get it.
Rig: docker/recipes (sample RecipeValueName,RecipeValue files incl. an invalid one) bind-mounted
read-only at /app/recipes on both site-a nodes with AllowedRoots [/app/recipes]. The TestMesMachine
SimDelmiaRecipe simulator now reads + parses the downloaded file, converts each value to its recipe tag's
type and loads it all-or-nothing (14 new TestDelmiaReceiver attributes); verify-mes-test-machine.sh
group A covers accept / re-download / invalid lines / NotFound / OutsideAllowedRoots, VERIFY_GROUPS
selects groups (rig: AD).
vd03 (DEV) upgraded 2b9a7c3c -> fe89363b via the deployment-only winrm channel: COPY_ONLY backup,
55->57 delta script (AddAlarmMesReportable, AddSiteArtifactRevision), ~53 s window, snapshots
identical to baseline, artifacts deployed. Register row 126 now covers the QA pair only.
docker/seed-mes-test-machine.sh (idempotent) builds a standalone TestMesMachine base template -
Left/Right TestMESReceiver + TestDelmiaReceiver (prod attribute names, STATIC), computed alarms
with IsMesReportable, CvdReactor-compatible MesMoveIn/MesMoveOut/ProcessRecipeDownload/AlarmStatus
scripts ported to SetAsync+WaitAsync, and Galaxy-side simulators with reject/no-response knobs -
plus instances SBTEST01/02 and the SimpleAlarmStatusRequest (was a stub) / AlarmStatusRequest
inbound methods in the legacy /mes/(simple)alarmstatus shapes, resolving machines through BTDB.
BTDB rows SBTEST01=990001 / SBTEST02=990002 added on vd03.
docker/verify-mes-test-machine.sh drives the real inbound methods via flow-debug: PASS 25/25.
Task 13/14. seed-mes-alarm-demo.sh builds a template whose three computed alarms differ in
how they acquire the flag (template --mes-reportable, nothing, instance override over a
template false), a site Call script projecting them through Alarms.CurrentAsync() /
MesReportableAsync(), and two inbound methods + a scoped API key. Idempotent by name lookup.
verify-mes-alarm-demo.sh drives the real inbound HTTP surface: PASS 14/14 on the docker rig
(migration 20260911135501_AddAlarmMesReportable applied at boot). Playwright
AlarmOverride_MesReportable_RoundTrips passed. Results + the facts the gate established are
in design doc section 8.
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
Moves the legacy BTDB MachineAlarm.FlaggedForMES lookup onto TemplateAlarm.IsMesReportable
(instance-overridable), surfaces it as ScriptAlarm.IsMesReportable / Alarms.MesReportableAsync(),
and adds an AlarmStatus inbound method demo on the docker rig. Native alarms stay on the
severity band (no per-alarm row).
Claude-Session: https://claude.ai/code/session_01Y5ef34iUUMjmwnMpL9xgmB
A null resetValue encoded to null, which wrote an empty string and then handed the
waiter a null target (= any change), fast-pathing on the stale cache and reinstating
the race the overload exists to close. The context parameter is now non-nullable and
the accessor throws ArgumentException before any write. Also: additive-timeout note on
all surfaces, batch-failure message no longer names the bool overload, and the
error-message/batch-failure branches are pinned.
Claude-Session: https://claude.ai/code/session_01XPsuy2ZZ5Gbyfpt8PrK5BV
The MES/Delmia handshake fast-paths on the previous run's cached CompleteFlag
because the reset write has not echoed back before the wait registers. An
edge-triggered wait is rejected (a subscription can swallow the reset when the
device flips inside one sampling interval); the fix encodes the reset leg inside
a new WriteBatchAndWaitAsync overload that observes the reset echo before writing
the trigger and returns a phase-aware HandshakeResult.
Claude-Session: https://claude.ai/code/session_01XPsuy2ZZ5Gbyfpt8PrK5BV
Second slice of the 2026-09-09 whole-system review remediation
(docs/plans/2026-09-09-arch-review.md §10), on top of the High slice 146a1323.
Runtime/devices: SR-2 alarm on-trigger integration scope; SR-3 correlated NativeAlarmActor
subscribe retry; SR-4 immutable audited DB target; SR-5 ordered InstanceActor stop/flush;
SR-12 rehydration fence; SR-13 S&F gate recheck per delivery; DC-4 no OPC UA security
downgrade; DC-5 alarm snapshot for every new subscriber; ORCH-1 sync-dispose joins the pump.
Management/UI: MG-2 site scope on audit tree + Flow Debugger Replay; MG-3 stale-probe memo
removed; MG-4 server-owned bulk-deploy deadline; UI-5 sandbox runs in a killable child
process (new CentralUI.SandboxWorker project); UI-14 bounded sandbox output/IPC.
Edge/audit: EI-1 relay audit ownership; EI-2 SkipBodyCapture suppresses error text; EI-3/EI-11
error-field redaction + caps; EI-4 single relay deadline, pre-execution-only retry; EI-5
authenticated relay provenance; EI-10 body cap before audit read.
Persistence/contracts: DL-1 global Instance.UniqueName unique index + conflict translation;
DL-2 DataConnection protocol config encrypted at rest (migration
20260909203046_EnforceGlobalInstanceNamesAndProtectProtocolConfiguration — coordinated
central upgrade, old binaries cannot read protected rows); DL-3 bounded Replay paging;
DL-6 cross-site Overwrite of a deployed instance rejected; XC-4 proto descriptor contract test.
Config/hygiene: XC-1 central listener isolation by local port; XC-2/XC-3 inert SMTP + cron
keys removed from rig settings; CM-2 env2 bootstrap guard enabled; DC-6 deploy site exit code;
DC-7 notifier retries only explicit standby rejection; XC-5 ClusterClient purged from docs.
Gates recorded in the report: build 0/0, 9 397 passed / 0 failed, rig redeployed on all
eight nodes with the migration applied. 46 Low findings remain open.
Joseph Doherty