From 9f7d36e7dc52b27be132a059f70cb92593361d24 Mon Sep 17 00:00:00 2001 From: Joseph Doherty Date: Thu, 12 Nov 2026 07:41:12 -0500 Subject: [PATCH] docs(index): mxaccessgw Contracts/Client 0.4.0 published; OtOpcUa adopts the stream-attached ack (lmxopcua#877) Claude-Session: https://claude.ai/code/session_01GT67ACmpMR7b9LGRPLijMi --- CLAUDE.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CLAUDE.md b/CLAUDE.md index 6c2d13e..13d411a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -37,7 +37,7 @@ own `CLAUDE.md` for the full picture. See [Refreshing this index](#refreshing-th | Project | Location | Stack | Repo | Summary | |---|---|---|---|---| | **OtOpcUa** | `~/Desktop/OtOpcUa` | .NET 10, OPC UA, gRPC | `gitea.dohertylan.com/dohertj2/lmxopcua` | OPC UA server that exposes industrial data sources under a **two-subtree address space (v3.0, `master` since 2026-07-16)** — `https://zb.com/otopcua/raw` (device tree Folder→Driver→Device→TagGroup→Tag, NodeId `s=`) and `https://zb.com/otopcua/uns` (equipment tree Area→Line→Equipment→signal); every value has one source fanned to both NodeIds. **This is the wire contract ScadaBridge binds against** (ScadaBridge consumes v3 end-to-end; its #14 is closed). Drivers: Modbus, S7, AB CIP/Legacy, TwinCAT, FOCAS, OpcUaClient, `Sql` (SQL Server / PostgreSQL / Oracle, read-only poll), `Mqtt` (Plain + Sparkplug B, writable where a tag declares a destination), `MTConnect` (read-only), `Heidenhain` (LSV2, read-only, #803), Calculation, and AVEVA Galaxy as a standard Equipment-kind driver via in-process `GalaxyDriver` → gRPC → **mxaccessgw**. Historian backend is pluggable (`ServerHistorian:Backend`): **HistorianGateway** (default), **SqlServer** or **QuestDb** — the latter two add typed values with real status and OPC UA HistoryUpdate (QuestDB: soft delete + TTL retention). Address-space availability guarantee: an unreadable deployment artifact is *no answer*, never an empty configuration — nodes hold last-known-good, and a driver-only node boots and serves from its LocalDb cache with central SQL down (ServiceLevel stays 240/250). Redundancy: one independent 2-node Akka mesh per application `Cluster`; Primary = oldest Up `driver` member (ServiceLevel 250) + one Secondary (240) **per `Cluster`**; cross-mesh commands ride gRPC with per-cluster PSK auth (Akka `ClusterClient` fully deleted 2026-08-06). Data plane is machine-to-machine: OPC UA UserName `apikey` + `otua__` minted on AdminUI `/api-keys`; `OpcUa:AllowAnonymous` defaults `true` (anonymous is a normal grantable principal); LDAP is AdminUI-login only. Node ACLs are enforced **only** behind `Security:Acl:Enabled` (default `false`); with it on, scoped `NodeAcl` grants decide **every** data-plane operation incl. browse/read/subscribe, fail closed, and reach open sessions. Every driver's authoring path now consults the driver's own runtime parse, so what authoring accepts is what the driver serves (tracker zeroed 2026-08-10, origin `d06e92c6`). **Deployed on wonder-app-vd03 (DEV) 2026-11-12: master `5fb4b4db` (was `b8b4c18f`; #793 — snapshot-isolation migration, new `Security__ApiKeys__Pepper`, first Windows host on OPC UA SDK 2.0.1; post-deploy deploy `e4b4824a` Applied, then `9adc5590` with the FOCAS `MaxRpm` tag re-pointed/renamed `MotorSpeedPct` (Good); ACL off; vd03 ScadaBridge has no OPC UA link to OtOpcUa (MxGateway only); owner actions all closed; record `docs/deploy/vd03-upgrade-b8b4c18f-to-master.md` §h). QA vq03/vq04 still `b8b4c18f`.** **`master` @ `31be927f` (2026-11-11; out-of-process script runner Windows pipe I/O off the completion port — host CPU about halved on windev, p99 now within budget (#789); earlier `f109e5f2`: Galaxy initialize budget 20 s (#875); #717 Warm-standby spec+plan drafted (awaiting owner); #761 gate F (no Galaxy session leak) PASS; earlier `43f8b8e8`: SDK server-side refusals now logged (#874); hosted v2-integration CI green for the first time (#860/#818); vd03 upgrade pack + preflight `docs/deploy/vd03-upgrade-b8b4c18f-to-master.md` (#793) — executed 2026-11-12, see above; earlier `363f0d81`: Galaxy status: failure rows map by the MxStatusDetail vocabulary and OK rows are graded by item quality — an off-scan/Bad/Uncertain attribute is no longer published Good (#866/#872); folder event queues are a ring buffer (#870); earlier `7a27ef1f`: Galaxy native-alarm ack no longer reports success on an unconfirmed gateway OK (#869), per-platform host status works (#864/#865), MTConnect blank string value now Good "" (#824), live PercentGood/DurationGood/TimeAverage/Total match HistoryRead (#809/#821), folder event subscriptions get the ConditionRefresh cut (#855); earlier `e857135e`: Galaxy first-value-after-subscribe race fixed + reconnects fenced per gateway session (#859/#861); credential lockout replaces the SDK lockout — anonymous never locked, parallel keyed sessions wait instead of failing (#833/#844); credential references (`secret:` and `env:`) scoped per driver instance (#815/#835/#849); ConditionRefresh on `i=2253` no longer overtaken by stale deferred events (#854); earlier `5bbe1e3c` added narrow-grant UNS access + scripted-alarm ack for Area/Line/Equipment-scoped keys with ACLs on, #808):** opt-in out-of-process script runner (`Scripting:Runner`, #780 — sends `BadResourceUnavailable`, ScadaBridge#112 handles it), VT→VT dependencies now flow + `VirtualTagDependencyCycle` deploy error (#791), disabled VT keeps its node as `BadOutOfService` (#787), Sql write block Insert/Upsert/Procedure (#781), OPC SDK 1.5.378.182 (stricter CA KeyUsage — check DEV/QA certs, #782). **Full change history, live-gate records and the per-issue consumer-impact record for ScadaBridge: `.claude/skills/sister-project-history/otopcua.md` (skill `sister-project-history`).** | -| **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** Alarm ack (main `488649c`, 2026-10-20) authenticates as the operator and writes `.AckMsg` so the Historian records the real operator; `AcknowledgeAlarmRequest` gained `operator_password`/`operator_domain` (proto change, published as Contracts/Client **`0.3.0`** on the feed 2026-11-06, mxaccessgw#214; OtOpcUa's `GatewayGalaxyAlarmAcknowledger` must send the password before it deploys against this — lmxopcua#749). **Deployed on wonder-app-vd03 (DEV) 2026-11-11: main `2b13564e` (was `4f931e4`)** — vd03's OtOpcUa has no Galaxy driver (one FOCAS driver), so the lmxopcua#749 ack-credential gap never bit there; OtOpcUa master (deployed on vd03 2026-11-12) is required before any OtOpcUa that *does* ack Galaxy alarms points at this gateway. **Main `afd504d5` (2026-11-12, unreleased, not deployed):** additive proto — opt-in `StreamEventsRequest.emit_stream_attached` → first-message `MxEvent.stream_attached` once the stream is registered, capability `stream-events-attached-ack` (#227; OtOpcUa adoption lmxopcua#877); and `892bdc1a` fixes wnwrap alarm times reported 4–5 h late (DATE+TIME are already UTC; #226) — **all alarm timestamps shift to correct UTC on the next worker deploy**. Main `e5e7f18` (2026-11-06) also locks the worker pipe to the gateway's account and redacts secured bulk writes in logs. | +| **MxAccessGateway** (`mxaccessgw`) | `~/Desktop/MxAccessGateway` | .NET 10 gateway (x64) + .NET 4.8 worker (**x86**), gRPC | `gitea.dohertylan.com/dohertj2/mxaccessgw` | gRPC gateway giving modern clients full MXAccess parity without loading 32-bit COM. Two-process: gateway (ASP.NET Core gRPC + Blazor dashboard) + per-session x86 worker that owns the MXAccess COM STA. **OtOpcUa depends on this.** Alarm ack (main `488649c`, 2026-10-20) authenticates as the operator and writes `.AckMsg` so the Historian records the real operator; `AcknowledgeAlarmRequest` gained `operator_password`/`operator_domain` (proto change, published as Contracts/Client **`0.3.0`** on the feed 2026-11-06, mxaccessgw#214; OtOpcUa's `GatewayGalaxyAlarmAcknowledger` must send the password before it deploys against this — lmxopcua#749). **Deployed on wonder-app-vd03 (DEV) 2026-11-11: main `2b13564e` (was `4f931e4`)** — vd03's OtOpcUa has no Galaxy driver (one FOCAS driver), so the lmxopcua#749 ack-credential gap never bit there; OtOpcUa master (deployed on vd03 2026-11-12) is required before any OtOpcUa that *does* ack Galaxy alarms points at this gateway. **Main `afd504d5` (2026-11-12, not deployed):** additive proto — opt-in `StreamEventsRequest.emit_stream_attached` → first-message `MxEvent.stream_attached` once the stream is registered, capability `stream-events-attached-ack` (#227); **published as Contracts/Client `0.4.0`** (`447070a5`), consumed by OtOpcUa master `f264977f` (lmxopcua#877; ScadaBridge stays on 0.2.1 and doesn't opt in); and `892bdc1a` fixes wnwrap alarm times reported 4–5 h late (DATE+TIME are already UTC; #226) — **all alarm timestamps shift to correct UTC on the next worker deploy**. Main `e5e7f18` (2026-11-06) also locks the worker pipe to the gateway's account and redacts secured bulk writes in logs. | | **ScadaBridge** | `~/Desktop/ScadaBridge` | .NET 10, Akka.NET, Docker | `gitea.dohertylan.com/dohertj2/ScadaBridge` | **Deployed on wonder-app-vd03 (DEV, single central + site) 2026-11-11: `424db155` (was `0e540291`; migrations 58→64 incl. the AuditLog clustered-key flip, #111 DEV data point: 2,394 rows / 8 MB, ~1.5 s). QA vq03/vq04 not deployed.** **`main` `c810bd0f` (2026-11-10 issue waves: AuditLog clustered-key flip + off-box chain-head export + maintenance login + archive retention — four migrations ship together with both centrals stopped (#107); opt-in `leader-auto-down` downing provider (#106, drill #116); derived templates survive bundles (#118); bundle-signing policy + runbook (#110); Flow Debugger Node log + Akka→Serilog (#56/#120); OPC UA Good-subcode + BadResourceUnavailable handling (#117/#112)). Previous tip `3808b49f` (2026-10-22: 2026-10-21 eight-slice system review remediated — ~107 findings, bearer-token CLI auth, `MachineDataDb` removed, config-audit summaries + retention, migration `ReviewRemediationIndexes` (QA/prod not applied); Topology area rename fix, first-layer preload and area-subtree **Deploy all** / CLI `deploy area` (Gitea #38–#40); record `docs/plans/2026-10-21-system-review-remediation.md`). Previous tip `3a217b94` (2026-09-29: stale per-node secret-mirror status rows are now pruned; 2026-09-28: site-secrets follow-ups — a central secret-store outage fails only the sites that reference a secret, removal-only deploys are audited, each site node's mirror failures (standby included) ride the health report via a 12th replicated table `secret_mirror_status`, artifact-deploy CLI commands exit 1 on a failed site; rig gate 9/9). Previous tip `23a139a4` (2026-09-28: site secrets ride the artifact deploy and the gRPC secrets hub is retired — see below). Previous tip `5cc09b08` (2026-09-28: site scripts gain a read-only, allow-listed `Files` global — `ScadaBridge:SiteRuntime:ScriptFileAccess:AllowedRoots`, off by default — used by the in-memory MES/Delmia/alarm-API test machines SBTEST01/02 (`docker/seed-mes-test-machine.sh`) to parse downloaded recipe files; DEV vd03 and QA vq03/vq04 on `b3021e7c` since 2026-09-28, migrations 57; register row 126 closed) — adds site artifact staleness (`Site.ArtifactRevisionHash` stamped on every successful artifact deploy, migration `AddSiteArtifactRevision`; Sites page *Artifacts current / out of date / never deployed* badge), a rail-footer **Pending updates** indicator over a process-wide `PendingUpdatesCache` (stale instances + stale-artifact sites, Deployer/Admin), and a scrollable rail nav; design `docs/plans/2026-09-14-pending-updates-design.md`. Previous tip `9d529b82` (2026-09-11) — adds the Gitea #36 reset-aware handshake (`WriteBatchAndWaitAsync` reset-echo overload), the computed-alarm `IsMesReportable` flag (template checkbox + nullable instance override, `Alarms.MesReportableAsync()`, migration `AddAlarmMesReportable`; native alarms stay on the 900–999 band) and a nav fix nesting Flow Debugger in the Deployment section. DEPLOYED: `b3021e7c` on DEV vd03 and the QA pair vq03/vq04 (2026-09-28, migrations 55→57; records `docs/deployment/deployment-records/2026-09-28-wonder-app-{vd03-fe89363b,vq03-vq04-b3021e7c}.md`).** **Merged `23a139a4` (2026-09-28): site secrets now ride the artifact deploy and the pull-only gRPC secrets hub is retired in ScadaBridge** (design `docs/plans/2026-09-28-site-secrets-artifact-deploy-design.md`; central shares one secret store via `Secrets:Store:Provider=SqlServer`; rig live gate PASS 8/8, vd03/QA NOT deployed — rollout tracked as ScadaBridge register row 127 — see `ZB.MOM.WW.Secrets/CLAUDE.md` for the unpublished `0.6.3` hub-logging work this superseded for ScadaBridge). Management/API/CLI parity complete (`docs/plans/2026-09-10-management-api-cli-parity-review.md`): all 149 eligible contracts exposed in 20 CLI groups, 19 missing operations added, deadline/registry/outcome gaps fixed; CLI documentation checked against all 159 leaf commands; 9554 tests verified and primary 8-node rig rebuilt with read-only smoke checks. Previously completed 2026-09-09 whole-system arch review remediation (`docs/plans/2026-09-09-arch-review.md`: 92 findings; all 13 High + 33 Medium fixed; all 46 Low findings also committed and pushed (§11); 9478 solution tests and deployed browser smoke pass, primary 8-node rig rebuilt). Adds the `ZB.MOM.WW.ScadaBridge.CentralUI.SandboxWorker` child-process project and a coordinated central-pair migration that encrypts DataConnection protocol config at rest (old binaries cannot read protected rows). Full implementation of the distributed SCADA platform — hub-and-spoke (1 central cluster + N site clusters). Projects prefixed `ZB.MOM.WW.ScadaBridge.*`; solution `ZB.MOM.WW.ScadaBridge.slnx`. Ships `src/`, `tests/`, `docker/` topology, and the design docs that are the spec. **Inbound API keys live on the shared SQL Server store on a central pair** (design 2026-09-02, ScadaBridge register row 108): ScadaBridge is Auth's first consumer of the new `ZB.MOM.WW.Auth.ApiKeys.SqlServer` package at `0.3.0`, pointing both central nodes at one `zbauth` schema in the configuration database instead of each node's own per-node SQLite file. **Flow Debugger (including the instance picker and v1.4 step logs, 2026-09-09):** Central UI + CLI trace of one inbound-API method or value-change flow across Central / Site / Tags / External swimlanes, Live or Replay, built on `ScriptTrace` audit rows and the outbound-call audit rows; arch-reviewed and remediated the same day (F1–F13 — redacted + byte-capped inline trace copy, route-time site scope on Live runs, Replay open to `OperationalAudit` roles under `RequireFlowDebugger`; `docs/plans/2026-09-06-flow-debugger-arch-review.md` + `-remediation.md`); script entry (site → instance → script, 2026-09-08). | | **HistorianGateway** | `~/Desktop/HistorianGateway` | .NET 10 x64, gRPC, Blazor | `gitea.dohertylan.com/dohertj2/historiangw` | Single-process gRPC sidecar exposing (1) a full read/write API to the AVEVA Historian (5 gRPC services; 15 retrieval modes; historical/backfill writes; tag-config lifecycle; SQL live-value path; store-forward + redundancy resilience; all default-disabled) and (2) read-only Galaxy object-hierarchy browse via the shared `ZB.MOM.WW.GalaxyRepository` lib. No COM, no x86 worker. **Deployed on wonder-app-vd03 (DEV) 2026-11-11: `ea9f21c` (Contracts/Client 1.0.0; was `d8d890c`).** **`main` @ `860bf99` (2026-11-10): redundancy + store-forward compose (F1b, #72 — keep both off on boxes until live gate #115), deterministic server-op faults (#112), versioned outbox frames + size limits + expiry cap (#113), stream keepalive guard on by default (#59), ReadEvents EventCount pass-down (#81). Earlier: `99b4a7d` (2026-11-06, historiangw#55): gRPC-only Historian client at `src/Historian.Client`** — `histsdk/` and the whole SQL path (RuntimeDb, SQL live-value writer, SQL `ReadEvents`) are removed; `Transport` must be `RemoteGrpc`, ReadBlocks removed from the contract (#62, 2026-11-10); RenameTags restored over StartJob (#56); UInt1 tags created as UInt2, Int8/UInt8/FileTime/Guid/Structure/Event rejected (lossless-cast-or-reject); empty event ids get a gateway-minted GUID (#28); AddHistoricalValues backfills string tags via `string_value` (#19). merged to main 2026-11-10 (PRs #116, #117 → `733eb25`); released as **Contracts/Client 0.4.0** (2026-11-10, PR #118 → `dde0c28`, tag `v0.4.0`, #79 closed — breaking: ReadBlocks removed; OtOpcUa consumes 0.4.0 on master `16111a5e` (historiangw#119 closed; no deployment)), then **1.0.0** (2026-11-10, PR #132 → `ea9f21c`, tag `v1.0.0` — breaking: secure-by-default client TLS, `AllowUntrustedServerCertificate` replaces `RequireCertificateValidation` (#58); also #25 WriteLiveValues hardening + #126; wire unchanged `historian_gateway.v1`; OtOpcUa master `230cef4e` consumes1.0.0 with the adapter rename verified; historiangw#131 closed). Deployed boxes still run `d8d890c` with the old config (redeploy needs a config change). Dev: dashboard `:5220` (HTTP/1.1) + gRPC h2c `:5221`; production: one `Kestrel:Endpoints:Https` with `Protocols: Http1AndHttp2` (ALPN). Default-on leased session pools amortize the historian handshake (v6 pool + a separate v8 event-session pool). Client lib `ZB.MOM.WW.HistorianGateway.Client` + Contracts are on the Gitea feed at `1.0.0` (OtOpcUa pinned to `1.0.0`, master `230cef4e`; historiangw#131 closed, no deployment); **OtOpcUa is the first consumer** (its sole historian backend since PR #423); ScadaBridge has no coupling. Native event reads now return rows through the gRPC-only client (2026-10-16 live run; the old 0-row C2 gate and its SQL `ReadEvents` workaround are gone — historiangw#24 closed 2026-11-09; the deployed `d8d890c` boxes still use the SQL reader until #78). Known 2023 R2 server-side gates: SysTimeSec raw reads still time out (#29, blocks lmxopcua#694/#700), `Int1`/`UInt1` analog-tag creation is degenerate (OtOpcUa maps Boolean → `Int2`), string historical writes non-convergent (follow-ups #18–#25). Only remaining archreview item: the secure-by-default TLS flip, deferred to 1.0.0. A Java client, `com.zb.mom.ww.historiangw:zb-mom-ww-historiangw-client:0.2.0` (Gitea Maven feed, `clients/java/`; 0.2.0 = the 0.4.0 contract, 2026-11-10, historiangw#120 — the consumer is still on 0.1.0), is consumed by the ignitionoee repo's `modules/historian` Ignition module. **Full change history (ArchReview cycles 1–2, pending.md items, PR list): `.claude/skills/sister-project-history/historiangateway.md`.** `HistorianStatus.GetOperationalDiagnostics` (22 RPCs) and the historical Int1 serializer are **merged to `main`** (Int1 persisted readback still unverified, #22/#29). Offline suite on `main`: 1311 passed / 54 skipped / 0 failed. New gap work targets native Historian gRPC only. | | **BTDB** (Batch Tracker) | `~/Desktop/btdb` — local `master` @ `a81a480` (2026-10-01), **~211 commits ahead of `origin/master` (`d147c10`, 2023) and unpushed**; no branch in flight (`feat/transfer-machine-batch` and `fix/transfer-lot-walk` — DB-6 reopened: forward loop guard, whole-lot chain walks incl. merges — both merged 2026-09-11; `feat/manager-transfers` — the Manager edits, shows and audits transfers — merged `695192d` the same day). **Claude artifacts are gitignored + untracked there** (`CLAUDE.md`, `.claude/`, `docs/plans/`, `docs/reviews/`, `*.tasks.json`) — local only, never to the corporate remote. | SQL Server SSDT project + SQL CLR (net472; **Windows-only build**, the docker rig rebuilds it cross-platform from the same sources); **BTDB Manager = Blazor Web App net10.0** on the family libs (Auth 0.4.1 — **sign-in by user bind, `BindMode=User`, no LDAP service account**, Theme 0.4.1, Health 0.3.0, Telemetry + Configuration 0.1.0, Secrets 0.6.2), Dapper + embedded queries, self-contained win-x64 NSSM service `ZB.MOM.WW.BTDB.Manager` on `:9877` (→ `:9876` after cut-over) | **`ZimmerBiomet/BTDB`** on the corporate Gitea `wonder-app-vd03:3000` (not on `gitea.dohertylan.com`) | The `BT` SQL Server database that is the plant's **batch / cycle / work-order system of record for data collection**. Galaxy machine objects write through `StartMachineBatch` / `StartMachineCycle` / `End*` / `AddMachineBatchWorkOrder` / `AddOtherData*` / `AddMachineBatchTransfer` / **`TransferMachineBatch`** (new 2026-09-11: end source if open + start destination by `Code` + copy not-moved-out work orders + link, one owned transaction) (own-or-join transaction template, `THROW 50000+`, all `GETUTCDATE()`); reports pull per-batch time-series **out of the AVEVA Historian through CLR sprocs** (`OPENQUERY(INSQL, …)` + `Runtime.dbo.*`), so **BT must be co-tenant with the Historian's `Runtime` DB** (`wonder-sql-*`). MESAPI and **ScadaBridge's inbound API** resolve `SAPID → Machine.Code`; no checked-in ScadaBridge script calls a BT sproc. **State 2026-09-11 (all merged on local `master`):** the Blazor conversion phases 1–3 (legacy net6.0 MVC app, Telerik reports and the mis-named `install.bat` deleted), the 2026-09-04 full review remediation (12 work packages, 60 findings fixed, 0 open; static invariant tests over the SQL + CLR sources, opt-in rig integration tests behind `BTDB_TEST_CONNECTION`, Gitea Actions gate `./build.sh`), and the DARS sprint-1 schema (`MachineBatchTransfer`, `ContainerID`/`MoveOutTimestamp` on `MachineBatchWO`, nestable `MachineGroup.ParentMachineGroupID`, Gen1 reactor load script) — the old `DARS/WW_ARCH/BatchTracker_DB/` fork and the `feat/manager-blazor` worktree are superseded. **Phase-4 cut-over on the DEV box** (`docs/operations/cut-over.md`) is the open step and is **blocked by the 2026-09-11 remote-access hold**. Repo orientation lives in btdb's own (gitignored) `CLAUDE.md`. **Full schema/contract notes, family edges and the gap analysis: `.claude/skills/sister-project-history/btdb.md`.** **Local Docker rig:** `btdb/docker/` — runtime-only image on `:9877` joined to `scadabridge-net` + the redundant glauth pair, registered in Overview as **BTDB Manager**; `docker/deploy-db.sh` publishes `BT` (dacpac + CLR via two side projects under `docker/db/`, plus a `Runtime` stand-in) into `scadabridge-mssql`; `smoke-queries.sh`, `validate-config-import.sh`, `db/validate-clr.sql` and `Database/Scripts/Validation/*.sql` are the rig gates. |