061432c8c4
Claude-Session: https://claude.ai/code/session_01GT67AFKC45MBz1wPZnVbX8
97 lines
8.3 KiB
Markdown
97 lines
8.3 KiB
Markdown
# 2026-11-15: #52 `CvdReactor.ProcessRecipeDownload` child-name fix, DEV vd03 + QA vq03/vq04 (Z28061 excluded)
|
||
|
||
**Owner go:** 2026-11-15, "yeah" to applying #52 (DEV first, then QA). **Z28061 is excluded on both tiers**: another team uses it and no redeploy slot has been agreed.
|
||
|
||
**Change:** one line in template `CvdReactor` (id 1), script `ProcessRecipeDownload` (id 4): `Children["DelmiaRecv"]` becomes `Children["DelmiaReceiver"]`. The body is `deploy-prep/2026-11-12-gates-52-131/fixed/CvdReactor.ProcessRecipeDownload.cs`. Every other script field is re-sent from `original/CvdReactor.ProcessRecipeDownload.meta.json` and read back unchanged.
|
||
|
||
**Scope:** management API only, through the ScadaBridge CLI (built from `main`) run from the Mac.
|
||
- No binary, no migration, no service restart, no WinRM. Only the redeployed instance actors restarted.
|
||
- Not touched: Z28061 (template change only; its deployment was left alone), every other template, script and instance, OtOpcUa, mxaccessgw, HistorianGateway and Overview.
|
||
|
||
**Tooling change (`apply.sh`, same commit):** every CLI call now passes `--url`, `--username` and `--password` explicitly. Earlier on 2026-11-15, an env-only `SCADABRIDGE_URL` pointing at QA had silently reached **DEV**, because the CLI fell back to its configured default central. Two optional guards were added:
|
||
- `SB_EXPECT=<names>` refuses to run unless the listed instances exist on the central, which proves the tier. The full instance list is printed first.
|
||
- `SB_SKIP=<names>` never deploys the listed instances. They are left stale.
|
||
|
||
## Functional check (read-only)
|
||
|
||
The call is `flow-debug run-script --instance-id 2 --script ProcessRecipeDownload --params '{}'` on **Z28062**, made only after confirming `DelmiaReceiver.ReadyFlag` is **False / Good**. In that state, the script returns before its `try` block, so nothing is written to the PLC.
|
||
- The returned text is the same before and after: `Receiver ready flag not set to true`. That is the correct answer when the real flag is False, so the text alone does not prove the fix.
|
||
- The evidence is the trace instead. **Before** the fix, the run logged `WARN runtime GetAttribute: attribute 'DelmiaRecv.ReadyFlag' not found on instance 'Z28062'`. This is the #52 defect: a null child, so the hard-coded text came back regardless of the real flag. **After** the fix, the warning is gone, and the script read the existing `DelmiaReceiver.ReadyFlag`.
|
||
- Not exercised: the download path with `ReadyFlag` True. It writes `RecipeDownloadFlag` and the recipe fields to the receiver and waits on `RecipeProcessedFlag`. Running it needs a planned recipe download.
|
||
- On QA, **Z28063 has `DelmiaReceiver.ReadyFlag` True**. A call there would take the write path, so it was **not** called. Its first real `DelmiaRecipeDownload` will now take the download path, which it did not before.
|
||
|
||
## DEV vd03 (`http://wonder-app-vd03.zmr.zimmer.com:8085`, DisableLogin)
|
||
|
||
| Step | Result |
|
||
|---|---|
|
||
| Tier proof | Instances: SBTEST01, SBTEST02, Z28061, Z28061Sim, Z28062, ZIPSEN. This is DEV's set |
|
||
| `check 52` | Live body **ORIGINAL**. Z28061, Z28061Sim and Z28062 not stale (deployed 2026-10-28) |
|
||
| Negative control | The trace logs `DelmiaRecv.ReadyFlag' not found` |
|
||
| `apply 52` (`SB_SKIP=Z28061`) | Script 4 updated; metadata read back unchanged. Deployed **Z28061Sim** (deployment 97) and **Z28062** (98). Z28061 skipped |
|
||
| `check 52` after | Body **FIXED**. Z28061Sim and Z28062 not stale (22:23:20–21Z). **Z28061 `stale=True`**, as intended |
|
||
| Trace after | RouteCall and ScriptRun Delivered, with the same ResultText and **no `not found` warning** |
|
||
|
||
## QA vq03/vq04 (active central **vq04** `:8085`, login `multi-role`)
|
||
|
||
**The active central is vq04.** `/health/active` returns vq03 503 and vq04 200. On 2026-11-13 it was vq03. Live flow-debug runs are refused on the standby (`STANDBY_NODE`), so every QA call went to vq04. Reads and writes go through the active node either way.
|
||
|
||
| Step | Result |
|
||
|---|---|
|
||
| Safety re-check (22:24Z) | The 8 instances to redeploy: `CycleRunning` False/False on both sides, FurnaceTemp 23–25 °C. All SAFE. Open batches: Z28063 L 29049, Z28065 R 29020, Z28068 L 29081 |
|
||
| Tier proof | Instances: WarmCanary, Z28061–Z28069, ZIPSEN. This is QA's set |
|
||
| `check 52` | Live body **ORIGINAL**. All 9 CvdReactor instances not stale (deployed 2026-11-06) |
|
||
| Negative control | Z28062 trace logs `DelmiaRecv.ReadyFlag' not found` |
|
||
| `apply 52` (`SB_SKIP=Z28061`) | Script 4 updated; metadata read back unchanged. Deployed **Z28062–Z28069** (deployments 233–240). Z28061 skipped |
|
||
| `check 52` after | Body **FIXED**. Z28062–Z28069 not stale (22:24:33–37Z). **Z28061 `stale=True`** |
|
||
| Trace after | Z28062: RouteCall and ScriptRun Delivered, with **no `not found` warning** |
|
||
| Post-redeploy | All 8 instances 244/244 attributes Good. `CycleRunning` False. The open batches and current steps are unchanged (Z28063 L 29049/step 2, Z28065 R 29020/step 2, Z28068 L 29081/step 6). `ReadyFlag` values unchanged |
|
||
|
||
## Follow-up
|
||
|
||
**Z28061 still runs the old script on both tiers.** It shows `stale=True` and still carries the #52 defect. Redeploy it once the other team agrees a slot. The tracking issue is in Gitea (#52 follow-up). The command is either `instance deploy --id 1` (explicit `--url`), or `apply.sh apply 52` without `SB_SKIP`, which is a no-op for the script and deploys only the stale instances.
|
||
|
||
## Rollback
|
||
|
||
Per tier: `SCADABRIDGE_URL=<central> SCADABRIDGE_USERNAME=… SCADABRIDGE_PASSWORD=… SB_SKIP=Z28061 bash apply.sh rollback 52`. It refuses unless the live body is FIXED. It writes the original body back, and the instances redeploy onto it.
|
||
|
||
## Z28061 (2026-11-15, #142)
|
||
|
||
The owner said: "if its idle you can do rest of #52". So on each tier Z28061 was redeployed **only if it was provably idle at that moment**, which stands in for the slot with its other team. Only the management CLI was used, with an explicit `--url`. Only Z28061 was deployed. `apply.sh apply` was not used, because it deploys every CvdReactor instance.
|
||
|
||
### DEV vd03: SKIPPED, idle can't be proven
|
||
|
||
- `apply.sh check 52` at 23:06Z: the body is FIXED, Z28061 (id 1) is `stale=True` (deployed 2026-10-28), and Z28061Sim and Z28062 are not stale.
|
||
- `debug snapshot` at 23:06:03Z: 270 attributes, **16 Good**.
|
||
- Every idle-check attribute is **Bad, value null**: `CycleRunning`, `MachineCycleID`, `FurnaceTemp` and `MachineBatchID` on both sides, plus `Left/RightDelmiaReceiver.RecipeDownloadFlag`.
|
||
- Only `DelmiaReceiver.RecipeDownloadFlag` and `ReadyFlag` are Good, both False, with the value last changed 2026-08-26.
|
||
- The rule says Bad quality means not proven idle, so it was **not deployed**. DEV Z28061 still runs the old body (`Children["DelmiaRecv"]`).
|
||
- **To unblock DEV:** get Z28061's DEV data source back to Good quality, or have its team confirm it's idle.
|
||
|
||
### QA vq03/vq04: DONE
|
||
|
||
- **Active central:** vq03 (`/health/active` vq03 200, vq04 503); the #116 stop-and-hold drill had moved it there. Every call went to vq03.
|
||
- **`apply.sh check 52`:** body FIXED, `SB_EXPECT=Z28063,Z28069,WarmCanary` passed. Z28061 (id 1) was the only stale instance; Z28062–Z28069 were already current.
|
||
|
||
**Idle check** (snapshots at 23:06:18Z and 23:07:25Z, 67 s apart; 244 attributes, all Good):
|
||
|
||
| Attribute | Left | Right |
|
||
|---|---|---|
|
||
| `CycleRunning` | False | False |
|
||
| `MachineCycleID` | 0 | 0 |
|
||
| `FurnaceTemp` | 25 | 25 |
|
||
| `MachineBatchID` | 0 | 0 |
|
||
|
||
- `DelmiaReceiver.RecipeDownloadFlag` and `ReadyFlag`: both False. QA's Z28061 has no Left/Right Delmia receivers.
|
||
- Between the two snapshots, the only change was `TrunkPressure` 0.18 → 0.17, which is noise.
|
||
|
||
**Deploy and verification:**
|
||
- **Deploy:** `instance deploy --id 1` gave deployment 241, status 2, done at 23:07:36.81Z in 0.26 s; revision `sha256:2727a432…`.
|
||
- **`instance diff`:** `isStale=False`.
|
||
- **Snapshot after (23:08:02Z):** 244 attributes, 244 Good, no value change except `TrunkPressure` noise, and no batch, so nothing to preserve.
|
||
- **Trace:** `ReadyFlag` re-read as False / Good first, so the script returns before any PLC write. `flow-debug run-script --instance-id 1 --script ProcessRecipeDownload --params '{}'` at 23:08:13Z gave RouteCall and ScriptRun both Delivered, with **no `not found` warning**.
|
||
|
||
### Result
|
||
|
||
- **QA:** all ten CvdReactor instances (Z28061–Z28069) run the fixed body.
|
||
- **DEV:** Z28062 and Z28061Sim are fixed. **Z28061 is still pending** on #142.
|