Commit Graph

22 Commits

Author SHA1 Message Date
Joseph Doherty
eb61d07ffd Add 5 component data-flow diagrams (Mermaid source + PNG)
- otopcua-dataflow: equipment → drivers (8, with stability tiers) → namespaces → ACL → consumers
- redpanda-eventhub: site ScadaBridge → store-and-forward → central cluster (topics + retention tiers + schema registry) → enterprise consumers
- snowbridge-dataflow: source adapters (Historian/Redpanda) → governed selection + approval workflow → Snowflake landing
- scadabridge-dataflow: OtOpcUa inputs → scripts/templates → multiple outputs (Redpanda, Web APIs, DB, email, equipment writes, Camstar)
- snowflake-dbt-dataflow: landing → staging → curated layer (dim_equipment, fact_state_transitions, mart_oee) → Power BI / AI/ML / ad-hoc
2026-04-17 14:24:45 -04:00
Joseph Doherty
e394c35020 Add regeneration quick-reference to CLAUDE.md 2026-04-17 14:06:24 -04:00
Joseph Doherty
658de96849 Generate first PPTX: 18-slide mixed-stakeholder deck
First run of the regeneration pipeline. 18 slides per presentation-spec.md:
title, exec summary, today's reality, vision, three pillars, enterprise
layout, systems, layered architecture (colored boxes), end-to-end data flow,
OtOpcUa, analytics stack, Redpanda EventHub, 3-year roadmap (table),
Year 1 focus, legacy retirement, open coordination items, non-goals,
asks & next steps.

Design: dark navy (#1B2838) for title/vision/closing slides, white for
content, blue (#2980B9) headers, orange (#E67E22) accents.
2026-04-17 13:49:55 -04:00
Joseph Doherty
42af4fd976 Mark corrections-doc E2 (Aveva System Platform IO pattern verification) as RESOLVED with GREEN-YELLOW verdict — the OtOpcUa team completed the research, published findings at lmxopcua/docs/v2/aveva-system-platform-io-research.md, and added a Phase 1 acceptance test (Task E.10, decision #142) to catch AppServer-specific quirks well before the Year 3 tier-3 cutover schedule. AVEVA's OI Gateway is the documented path; multiple non-AVEVA upstream-server integrations exist in published partner walkthroughs; no re-architecting of OtOpcUa needed. Two integrator-burden risks the plan team should track: validation/GxP paperwork (no AVEVA Part 11 blueprint for non-AVEVA upstream servers — engage QA/regulatory in Year 1) and unpublished scale benchmarks (in-house benchmark required in Year 2 before tier-3 cutover scheduling).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 13:27:28 -04:00
Joseph Doherty
78a58b3a31 Resolve enterprise shortname = zb (closes corrections-doc D4) and propagate through all UNS path examples and schema seeds.
Updated goal-state.md UNS hierarchy table (level 1 example with rationale: matches existing ZB.MOM.WW.* namespace prefix, short by design for a segment that appears in every equipment path, operators already say "ZB" colloquially), all worked-example paths in text + OPC UA browse forms, small-site placeholder example. Removed enterprise-shortname from the §UNS-hierarchy TBD list.

Updated schemas/uns/example-warsaw-west.json `enterprise: "zb"`.

Updated corrections-doc D4 entry to RESOLVED with full propagation list, and updated summary table accordingly.

Production deployments use `zb` from cluster-create. The hardcoded `_default` reserved-segment rule is unchanged (still the placeholder for unused Area/Line levels at single-cluster sites).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 13:12:59 -04:00
Joseph Doherty
8704f9e455 Integrate Round 3 OtOpcUa corrections into the plan files (goal-state.md, roadmap.md) and append a Round 3 addendum to the corrections doc for audit trail.
goal-state.md: schemas-repo seed paragraph (line 574) now reflects the `_base` equipment-class template (universal cross-machine baseline that every other class extends), explicit alignment to OPC UA Companion Spec OPC 40010 (Machinery) for the Identification component + MachineryOperationMode enum, OPC UA Part 9 for alarm-summary fields (HasActiveAlarms, ActiveAlarmCount, HighestActiveAlarmSeverity), ISO 22400 for lifetime counters (TotalRunSeconds, TotalCycles) that feed Availability + Performance KPIs, the canonical state vocabulary declared in `_base.stateModel`, and the OtOpcUa central config DB extension with 9 nullable OPC 40010 identity columns (Manufacturer, Model, SerialNumber, HardwareRevision, SoftwareRevision, YearOfConstruction, AssetLocation, ManufacturerUri, DeviceManualUri). Updated format-decisions count from 8 to 10 (added D9 _base+extends inheritance, D10 category→folder mapping). Multi-identifier section (line 156) gains a paragraph describing the OPC 40010 fields as additional first-class metadata beyond the five identifiers, with the operator-set / driver-dynamic-override pattern documented.

roadmap.md: OtOpcUa Year 1 cell (line 66) gains the universal `_base` equipment-class template seeded by the OtOpcUa team, with explicit OPC 40010 / OPC UA Part 9 / ISO 22400 references and the rationale ("avoids per-class drift in identity / state / alarm field naming and ensures every machine in the estate exposes the same baseline metadata regardless of vendor").

handoffs/otopcua-corrections-2026-04-17.md: appended a Round 3 addendum capturing the four follow-on additions (ACL design closing B1, dev-environment two-tier model, cutover scope removal closing C5, `_base` template + OPC 40010 columns building on B2). Updated summary table marks B1 / C1 / C5 as CLOSED, B2 as PARTIALLY CLOSED. Round 3 additions are committed in lmxopcua at `4903a19` and `d8fa3a0`, and in 3yearplan at `5953685` and `cd85159`.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 13:04:18 -04:00
Joseph Doherty
cd85159951 Add _base equipment-class template for universal cross-machine metadata that every machine in the OtOpcUa estate exposes regardless of vendor, protocol, or machine type. References OPC UA Companion Spec OPC 40010 (Machinery) for the Identification component (Manufacturer, Model, ProductInstanceUri, SerialNumber, HardwareRevision, SoftwareRevision, YearOfConstruction, ManufacturerUri, DeviceManual, AssetLocation) plus the MachineryOperationMode enum (Auto, Manual, Maintenance, Service, Setup, Other), OPC UA Part 9 for the alarm summary fields (HasActiveAlarms, ActiveAlarmCount, HighestActiveAlarmSeverity), ISO 22400 for the lifetime counter fields (TotalRunSeconds, TotalCycles) that feed Availability + Performance KPIs at Layer 3, and the 3-year-plan handoff §"Canonical Model Integration" for the canonical state vocabulary (Running / Idle / Faulted / Starved / Blocked) declared in _base.stateModel. Includes the OtOpcUa five-identifier set (EquipmentUuid, MachineCode, ZTag, SAPID, plus DeviceClass = EquipmentClassRef) so every machine surfaces the join keys downstream consumers need; ConnectionState + LastDataTimestamp + DriverType for driver-side observability that does not require any particular equipment-protocol feature; optional production context (CurrentWorkOrder, CurrentPartNumber, CurrentRecipe, CurrentOperator, CurrentShift) marked isRequired: false since not every machine type surfaces these. Plus two universal alarm definitions (communication-loss, data-stale) that apply to every equipment regardless of class.
Equipment-class.schema.json gains an `extends` field for class inheritance — child classes inherit signals, alarms, and stateModel from the parent and can add new ones or override individual entries by name. Convention: `_` prefix on classId marks an abstract base class (e.g. `_base`) intended only to be extended, not assigned directly to equipment via Equipment.EquipmentClassRef.

FANUC CNC class updated to extends: "_base"; redundant identity signals (Version, ActiveAlarmCount) removed since they're now in the base; remaining FANUC-specific signals updated with cross-references showing how they feed into the base signals at Layer 3 (RunState → canonical Running/Idle/Faulted derivation; AlarmActive → HasActiveAlarms / HighestActiveAlarmSeverity; PartsCount → TotalCycles; MainProgramNumber → CurrentRecipe).

Format-decisions.md adds D9 (rationale for `_base` + `extends` inheritance, with references to OPC 40010 / Part 9 / ISO 22400 / handoff) and D10 (signal `category` drives OPC UA folder placement, per OPC 40010 Identification + Status pattern, with a category-to-folder mapping table).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 12:54:17 -04:00
Joseph Doherty
6b0883ff95 Integrate v2 corrections addendum — ACL committed, schemas seed, cutover ownership
B1 resolved: ACL model designed and committed (decisions #129-132).
6-level scope hierarchy, NodePermissions bitmask, generation-versioned
NodeAcl table, Phase 1 ships before any driver phase. Updated goal-state
and roadmap.

B2 partially resolved: schemas repo seed exists at schemas/ (temporary).
FANUC CNC pilot class, JSON Schema format definitions, UNS subtree
example, docs. Still needs: owner team, dedicated repo, format ratification,
CI gate, consumer integration plumbing.

C5 resolved: consumer cutover OUT of OtOpcUa v2 scope (decision #136).
Integration/operations team owns cutover, not yet named. Plan updated
to explicitly assign ownership outside OtOpcUa.

CLAUDE.md updated with schemas/ in the file index.
2026-04-17 12:40:14 -04:00
Joseph Doherty
5953685ffb Seed the canonical OT schemas content under 3yearplan/schemas/ as a temporary location until a dedicated schemas repo is created (Gitea push-to-create is disabled, the dedicated repo needs a manual UI step). Initial seed contributed by the OtOpcUa team to unblock the EquipmentClassRef integration timeline (lmxopcua decision #112) and to provide the future cross-team owner with a concrete starting point rather than a blank slate. Marked DRAFT throughout with prominent "ownership TBD" framing in README and CONTRIBUTING — the future owner team should treat this seed as a starting point and revise format / structure / naming as the open questions in README "Open Questions" get resolved.
Includes: README explaining purpose / scope / temporary-location framing / format decision, CONTRIBUTING.md with proposed workflow + per-class semver versioning policy + validation commands, format/equipment-class.schema.json defining the shape of a class template (classId, version, displayName, applicability, signals, alarms, optional stateModel), format/tag-definition.schema.json defining the shape of a single canonical signal (name, dataType, category, unit, isArray, accessLevel, writeIdempotent, isHistorized, scaling), format/uns-subtree.schema.json defining the shape of a per-site UNS subtree (enterprise + site + areas + lines), classes/fanuc-cnc.json as the worked pilot class with 16 signals + 3 alarms + suggested state-derivation notes (per OtOpcUa corrections doc D1), uns/example-warsaw-west.json as a worked UNS subtree example, docs/overview.md (what / why / lifecycle / what's NOT in this repo), docs/format-decisions.md (8 numbered decisions covering JSON Schema choice per corrections D2, per-class semver, additive-only minor bumps, _default placeholder reservation, signal-name vs UNS-segment regex distinction, stateModel-as-informational, no per-equipment overrides at this layer, applicability.drivers as OtOpcUa driver enumeration), docs/consumer-integration.md (how OtOpcUa / Redpanda / dbt each integrate). $id URLs in the JSON schemas resolve at the actual current path so validators don't 404.

Top-level README adds a row to the Component Detail Files table pointing to schemas/. Corrections doc B2 (schemas-repo dependencies) marked partially RESOLVED with the seed location and a list of what still needs the plan team or cross-team owner to decide (owner team naming, dedicated repo migration, format-decision ratification, FANUC CNC pilot confirmation, CI gate setup, Redpanda + dbt consumer integration plumbing).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 12:35:27 -04:00
Joseph Doherty
dee56a6846 Mark corrections-doc B1 (data-path ACLs) and C5 (consumer cutover scope) as RESOLVED. B1: OtOpcUa team has designed and committed the OPC UA client data-path authorization model in lmxopcua/docs/v2/acl-design.md (decisions #129–132) covering NodePermissions bitmask flags for Browse/Read/Subscribe/HistoryRead/WriteOperate/WriteTune/WriteConfigure/AlarmRead/AlarmAck/AlarmConfirm/AlarmShelve/MethodCall plus common bundles, 6-level scope hierarchy with default-deny + additive grants, NodeAcl table generation-versioned alongside the rest of the content, cluster-create workflow seeding the v1 LDAP-role-to-permission map for v1 → v2 consumer migration parity, Admin UI ACL tab with bulk grant + permission simulator, denied-only audit logging; the "must work from day one of Tier 1 cutover" timing constraint is satisfied because Phase 1 (Configuration + Admin scaffold) completes before any driver phase. C5: consumer cutover (ScadaBridge / Ignition / System Platform IO) is OUT of v2 scope per lmxopcua decision #136 — OtOpcUa team's scope ends at Phase 5 (all drivers built, all stability protections in place, full Admin UI shipped including ACL editor); cutover sequencing per site, validation methodology, rollback procedures, and Aveva-pattern validation for tier 3 are deliverables of a separate integration / operations team that has yet to be named. Plan should explicitly assign ownership of the cutover plan to that team and link to their forthcoming doc.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 11:59:01 -04:00
Joseph Doherty
bed8c8e12b Remove equipment protocol survey — driver list confirmed by v2 team
The OtOpcUa v2 implementation team committed all 8 core drivers from
internal knowledge of the estate, making the formal protocol survey
unnecessary for driver scoping. Removed current-state/equipment-protocol-
survey.md and cleaned up all references across 7 files.

The UNS hierarchy snapshot (per-site equipment-instance walk for site/area/
line/equipment assignments + UUIDs) is now a standalone Year 1 deliverable,
decoupled from protocol discovery. Tracked in status.md and goal-state.md
UNS naming hierarchy section.

Eliminates ~52 TBDs (all placeholder data in the pre-seeded survey rows).
2026-04-17 11:54:46 -04:00
Joseph Doherty
f53a775968 Mark corrections-doc C1 (driver list pre-survey) as RESOLVED — the OtOpcUa team has confirmed all seven v2 drivers (Modbus TCP including DL205, AB CIP, AB Legacy, S7, TwinCAT, FOCAS, OPC UA Client) plus Galaxy/MXAccess by direct knowledge of the equipment estate; the survey is no longer a v2 prerequisite. TwinCAT and AB Legacy specifically called out as committed by known Beckhoff and SLC/MicroLogix legacy installations. Survey may still inform long-tail driver scoping and per-site capacity planning per the handoff's Long-tail drivers section, but the v2 driver list is fixed. Recommends the handoff's "Core library scope is driven by the survey" wording be updated to reflect that the v2.0 core library is pre-committed by direct equipment-estate knowledge, with the survey informing only long-tail driver scoping. Captured as lmxopcua decision #128 (2026-04-17).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 11:35:34 -04:00
Joseph Doherty
c3587b2efa Update equipment identifier model per v2 hardening addendum
- EquipmentId is now system-generated ('EQ-' + 12 hex from UUID), never
  operator-supplied — eliminates duplicate-identity corruption from typos
  and bulk-import renames (lmxopcua decision #125)
- ZTag and SAPID fleet-wide uniqueness enforced via ExternalIdReservation
  table outside generation versioning — rollback-safe (decision #124)
- Identifier table now shows who-sets-it column (3 operator, 2 system)
- Note added: ExternalIdReservation pattern is a precedent for non-versioned
  cross-generation invariants; check for similar hazard when scoping ACLs
2026-04-17 11:12:40 -04:00
Joseph Doherty
8a6c227dbc Add same-day addendum to OtOpcUa corrections doc noting four v2 design defects an adversarial review surfaced after the corrections doc was filed (one critical: cross-cluster namespace binding, three high: namespace state bypassing publish boundary, ZTag/SAPID rollback-reuse hazard, operator-supplied EquipmentId minting duplicate identities) — all four closed in lmxopcua v2 branch at commit a59ad2e (decisions #122–125). Two of the fixes refine claims this corrections doc made (C4 multi-identifier model: EquipmentId is now system-generated not operator-supplied; D3 ACL location: ExternalIdReservation precedent shows some cross-generation invariants need non-versioned tables) so plan-team awareness matters; the other two (same-cluster namespace invariant, Namespace generation-versioning) are purely internal correctness with no handoff relevance, included for audit trail.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 11:10:05 -04:00
Joseph Doherty
68dbc014da Integrate OtOpcUa v2 implementation corrections into plan
19 corrections from handoffs/otopcua-corrections-2026-04-17.md:

Inaccuracies fixed:
- A1: OPC UA-native equipment requires OpcUaClient gateway driver (~hours
  config), not "no driver build"
- A2: "single endpoint" is per-node (non-transparent redundancy), not
  per-cluster; no VIP planned

Missing constraints added:
- B1: ACL surface (EquipmentAcl table, Admin UI, NodeManager enforcement)
  as Year 1 deliverable before Tier 1 cutover
- B2: schemas-repo creation on OtOpcUa critical path with FANUC CNC pilot
- B3: Certificate-distribution as pre-cutover step (per-node ApplicationUri
  trust-pinning)

Architectural decisions incorporated:
- C1: 8 committed core drivers (added TwinCAT/Beckhoff, split AB Legacy)
- C2: Three-tier driver stability model (A/B/C with out-of-process for
  Galaxy and FOCAS)
- C3: Polly v8+ resilience with default-no-retry on writes
- C4: Multi-identifier equipment model (5 IDs: UUID, EquipmentId,
  MachineCode, ZTag, SAPID)
- C5: Consumer cutover plan needs an owner (flagged)
- C6: Per-building cluster implications at Warsaw clarified

TBDs resolved:
- D1: Pilot equipment class = FANUC CNC
- D2: Schemas repo format = JSON Schema (.json), Protobuf derived
- D3: ACL definitions in central config DB alongside driver/topology
- D4: Enterprise shortname still unresolved (flagged as pre-cutover blocker)

New TBDs added:
- E1: UUID generation authority (OtOpcUa vs external system)
- E2: Aveva System Platform IO pattern validation (Year 1/2 research)
- E3: Site-wide vs per-cluster consumer addressing at Warsaw
- E4: Cluster endpoint wording (resolved via A2)
2026-04-17 10:05:07 -04:00
Joseph Doherty
9b2acfe699 Add OtOpcUa implementation corrections (2026-04-17) capturing mismatches between the otopcua-handoff and the v2 design work in lmxopcua/docs/v2/: 2 framing inaccuracies (native-OPC-UA-needs-no-driver, single-endpoint-per-cluster), 3 missing constraints (namespace ACLs not yet planned in the data path, schemas-repo dependencies blocking equipment-class templates, per-node ApplicationUri trust-pinning as a pre-cutover certificate-distribution step), 6 architectural decisions to revisit (driver list committed pre-survey, Tier A/B/C process-isolation model with Galaxy + FOCAS out-of-process, Polly v8+ resilience, 5-identifier equipment model with MachineCode/ZTag/SAPID alongside UUID, missing tier 1/2/3 consumer cutover plan, per-building cluster pattern interactions at Warsaw), 4 resolved TBDs (pilot class = FANUC CNC, schemas-repo format = JSON Schema, ACL location = central config DB co-located with topology, enterprise shortname still unresolved), and 4 new TBDs (UUID-generation authority, System Platform IO Aveva-pattern validation as Year 1/2 research, multi-cluster site addressing at Warsaw, cluster-endpoint mental model). Format follows the handoff's Sending-Corrections-Back protocol (what plan says / what was found / what plan should say).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
2026-04-17 09:54:36 -04:00
Joseph Doherty
8428b7c186 Fix ScadaBridge accuracy per design repo review
Corrections:
- Notifications: email only, not Teams. Design repo documents SMTP/OAuth2
  email only; Teams was incorrectly claimed. Corrected in current-state.md
  and legacy-integrations.md (LEG-003).
- EventHub/Kafka forwarding: committed but not yet implemented. Clarified
  as a Year 1 ScadaBridge Extensions deliverable, not an existing capability.

Additions from design repo (previously undocumented):
- Dual transport (Akka.NET ClusterClient + gRPC server-streaming)
- Split-brain resolver (keep-oldest, 15s stability, ~25s failover)
- Staggered batch startup (20 instances at a time)
- Central UI: Blazor Server with LDAP/AD, JWT sessions, SignalR debug
- Comprehensive synchronous audit logging (JSON after-state)
- Three-phase deployment process with rollback
- Site-level SQLite (flattened config, not full SQL Server)
- Supervision detail: OneForOneStrategy, Resume/Stop per actor type
2026-04-17 09:30:22 -04:00
Joseph Doherty
fc3e19fde1 Add OtOpcUa implementation handoff document
Self-contained extract of all OtOpcUa design material from the plan:
architecture context, LmxOpcUa starting point, two namespaces, driver
strategy, deployment, auth, rollout tiers, UNS hierarchy, canonical
model integration, digital twin touchpoints, sites, roadmap, and all
open TBDs. Includes correction-submission protocol for the implementing
agent.
2026-04-17 09:21:25 -04:00
Joseph Doherty
d89c23a659 Add ScadaBridge design repo link (repo name: scadalink-design) 2026-04-17 09:15:33 -04:00
Joseph Doherty
f46a9da0d8 Add links document with LmxOpcUa repo reference 2026-04-17 09:14:59 -04:00
Joseph Doherty
fcd8d24d60 Add README with plan overview, architecture, and document index 2026-04-17 09:13:50 -04:00
Joseph Doherty
ec1dfe59e4 Initial commit: 3-year shopfloor IT/OT transformation plan
Core plan: current-state, goal-state (layered architecture, OtOpcUa,
Redpanda EventHub, SnowBridge, canonical model, UNS posture + naming
hierarchy, digital twin use cases absorbed), roadmap (7 workstreams x 3
years), and status bookmark.

Component detail files: legacy integrations inventory (3 integrations,
pillar 3 denominator closed), equipment protocol survey template (dual
mandate with UNS hierarchy snapshot), digital twin management brief
(conversation complete, outcome recorded).

Output generation pipeline: specs for 18-slide mixed-stakeholder PPTX
and faithful-typeset PDF, with README, design doc, and implementation
plan. No generated outputs yet — deferred until source data is stable.
2026-04-17 09:12:35 -04:00