Files
3yearplan/STATUS.md
Joseph Doherty bed8c8e12b Remove equipment protocol survey — driver list confirmed by v2 team
The OtOpcUa v2 implementation team committed all 8 core drivers from
internal knowledge of the estate, making the formal protocol survey
unnecessary for driver scoping. Removed current-state/equipment-protocol-
survey.md and cleaned up all references across 7 files.

The UNS hierarchy snapshot (per-site equipment-instance walk for site/area/
line/equipment assignments + UUIDs) is now a standalone Year 1 deliverable,
decoupled from protocol discovery. Tracked in status.md and goal-state.md
UNS naming hierarchy section.

Eliminates ~52 TBDs (all placeholder data in the pre-seeded survey rows).
2026-04-17 11:54:46 -04:00

120 lines
21 KiB
Markdown
Raw Blame History

This file contains ambiguous Unicode characters
This file contains Unicode characters that might be confused with other characters. If you think that this is intentional, you can safely ignore this warning. Use the Escape button to reveal them.
# Plan — Working Session Status
**Saved:** 2026-04-15
**Claude Code session ID:** `4851af09-c6b2-41fd-a419-a43d8cf80721`
**Resume with:** `claude --resume 4851af09-c6b2-41fd-a419-a43d8cf80721` (or pick from `/resume` inside Claude Code)
> This file is a **bookmark**, not a replacement for the plan. The authoritative content lives in `CLAUDE.md`, `current-state.md`, `goal-state.md`, `roadmap.md`, and the component detail files under `current-state/`, `goal-state/`, and `outputs/`. Read this file only to find out where we left off.
## Where we are
The plan is substantively fleshed out. All three core documents are populated, the canonical model + UNS framing has been declared, the digital twin management conversation has happened and been absorbed, and most open items are either design-time details the build team can close or items deliberately marked out of scope. An output generation pipeline (PPTX + PDF) has been scaffolded with spec files but no outputs generated yet.
### Files
**Core plan content:**
- [`CLAUDE.md`](CLAUDE.md) — plan purpose, document index (now including the component detail files and outputs pipeline), markdown-first conventions, component breakout rules.
- [`current-state.md`](current-state.md) — snapshot of today's estate (enterprise layout, clusters, systems, integrations, equipment access patterns).
- [`goal-state.md`](goal-state.md) — target end-state with Vision, layered architecture, **Unified Namespace posture + naming hierarchy standard**, component designs (OtOpcUa, SnowBridge, Redpanda EventHub with **Canonical Equipment/Production/Event Model + canonical state vocabulary**, dbt layer, ScadaBridge extensions), success criteria, observability, Strategic Considerations (Digital Twin with use cases received + Power BI), and Non-Goals.
- [`roadmap.md`](roadmap.md) — 3-year workstreams × years grid with 7 workstreams and cross-workstream dependencies; Year 1 Redpanda and dbt cells updated for canonical model delivery.
**Component detail files:**
- [`current-state/legacy-integrations.md`](current-state/legacy-integrations.md) — authoritative inventory for pillar 3 retirement. **Closed as denominator = 3**: LEG-001 Delmia DNC, LEG-002 Camstar MES, LEG-003 custom email notification service. Historian MSSQL reporting surface explicitly carved out as *not* legacy.
- ~~`current-state/equipment-protocol-survey.md`~~ — **Removed.** Protocol survey no longer needed; the OtOpcUa v2 implementation team committed the 8-driver core library from internal knowledge. The UNS hierarchy snapshot (equipment-instance walk) is now a standalone Year 1 deliverable tracked separately.
- [`goal-state/digital-twin-management-brief.md`](goal-state/digital-twin-management-brief.md) — meeting-prep artifact for the (now completed) digital twin management conversation; "Outcome" section at top captures the resolution.
**Input / reference files:**
- [`digital_twin_usecases.md.txt`](digital_twin_usecases.md.txt) — management's delivered requirements for digital twin (three use cases: standardized state model, virtual testing/simulation, cross-system canonical model). Source for the plan's digital twin response.
**Output generation pipeline (specs only — no outputs generated yet):**
- [`outputs/README.md`](outputs/README.md) — trigger phrases (`regenerate outputs` / `regenerate presentation` / `regenerate longform`), regeneration procedure, edit-this-not-that rules.
- [`outputs/DESIGN.md`](outputs/DESIGN.md) — design for the generation pipeline.
- [`outputs/IMPLEMENTATION-PLAN.md`](outputs/IMPLEMENTATION-PLAN.md) — scaffolding plan (partially executed — specs written, generation not yet run).
- [`outputs/presentation-spec.md`](outputs/presentation-spec.md) — 18-slide mixed-stakeholder deck structure anchor.
- [`outputs/longform-spec.md`](outputs/longform-spec.md) — faithful-typeset PDF structure anchor.
- `outputs/diagrams/` and `outputs/generated/` — empty, waiting for first regeneration run.
### Major decisions captured (pointers, not restatements)
- **Vision theme:** *stable, single point of integration between shopfloor OT and enterprise IT* — used as the tiebreaker for ambiguous decisions.
- **Three in-scope pillars:** unification (100% of sites on standardized stack), analytics/AI enablement (≤15m analytics SLO, one "not possible before" use case in production), legacy middleware retirement (inventory to zero). Binary at end of plan.
- **UX split:** Ignition owns KPI UX long-term; Aveva System Platform HMI owns validated-data UX long-term. Not a primary goal of this plan.
- **IT↔OT boundary:** single crossing at ScadaBridge central. OT = machine data (System Platform, equipment OPC UA, OtOpcUa, ScadaBridge, Aveva Historian, Ignition). IT = enterprise apps (Camstar, Delmia, Snowflake, SnowBridge, Power BI/BOBJ).
- **Layered architecture:** Layer 1 Equipment → Layer 2 OtOpcUa → Layer 3 SCADA (System Platform + Ignition) → Layer 4 ScadaBridge → Enterprise IT.
- **OtOpcUa** (layer 2): custom-built, clustered, co-located on System Platform nodes, hybrid driver strategy (proactive core library + on-demand long-tail), OPC UA-native auth, **absorbs LmxOpcUa** as its System Platform namespace. Tiered cutover: ScadaBridge first, Ignition second, System Platform IO last. **Namespace architecture supports a future `simulated` namespace** for integration testing (digital twin use case 2) — architecturally supported, not committed for build.
- **Redpanda EventHub:** self-hosted, central cluster in South Bend (single-cluster HA, VM-level DR out of scope), per-topic tiered retention (operational 7d / analytics 30d / compliance 90d), bundled Schema Registry, Protobuf via central `schemas` repo with `buf` CI, `BACKWARD_TRANSITIVE` compatibility, `TopicNameStrategy` subjects, `{domain}.{entity}.{event-type}` naming, site identity in message (not topic), SASL/OAUTHBEARER + prefix ACLs. Store-and-forward at ScadaBridge handles site resilience. **Analytics-tier retention is also a replay surface** for integration testing / simulation-lite (digital twin use case 2).
- **Canonical Equipment, Production, and Event Model** (added via digital twin use cases 1 and 3): the plan commits to declaring the composition of OtOpcUa equipment namespace + Redpanda canonical topics + `schemas` repo + dbt curated layer as **the** canonical model. Three surfaces, one source of truth (`schemas` repo). Includes a **canonical machine state vocabulary** (`Running / Idle / Faulted / Starved / Blocked` + TBD additions like `Changeover`, `Maintenance`, `Setup`). Year 1 Redpanda and dbt cells are updated to deliver v1.
- **Unified Namespace (UNS) posture:** the canonical model above is also declared as the plan's UNS, framed for stakeholders using UNS vocabulary. **Deliberate deviations from classic MQTT/Sparkplug UNS:** Kafka instead of MQTT (for analytics/replay), flat `{domain}.{entity}.{event-type}` topics with site in message (for bounded topic count), stateless events instead of Sparkplug state machine. Optional future **UNS projection service** (MQTT/Sparkplug and/or enterprise OPC UA aggregator) is architecturally supported but not committed for build; decision trigger documented.
- **UNS naming hierarchy standard:** 5 levels always present — Enterprise → Site → Area → Line → Equipment, with `_default` placeholder where a level doesn't apply. Text form `ent.warsaw-west.bldg-3.line-2.cnc-mill-05` / OPC UA form `ent/warsaw-west/bldg-3/line-2/cnc-mill-05`. Stable **equipment UUIDv4** alongside the path (path is navigation, UUID is lineage). Authority lives in `schemas` repo; OtOpcUa / Redpanda / dbt consume the authoritative definition. **Enterprise shortname is currently `ent` placeholder — needs assignment.**
- **SnowBridge:** custom-built machine-data-to-Snowflake upload service; Year 1 starting source is Aveva Historian SQL; UI + API with blast-radius-based approval workflow; selection state in internal datastore (not git).
- **Snowflake transform tooling:** dbt only, run by a self-hosted orchestrator (specific orchestrator out of scope).
- **Aggregation boundary:** aggregation lives in Snowflake (dbt). ScadaBridge does deadband/exception-based filtering (global default ~1% of span) plus tag opt-in via SnowBridge — not source-side summarization.
- **Observability:** commit to signals (Redpanda, ScadaBridge, SnowBridge, dbt), tool is out of scope.
- **Digital Twin (management-delivered use cases, 2026-04-15):** three use cases received — (1) standardized equipment state model, (2) virtual testing / simulation, (3) cross-system canonical model. **Use cases 1 and 3 absorbed into the plan** as the canonical state vocabulary + canonical model declaration (see above). **Use case 2 served minimally** by Redpanda historical replay + future OtOpcUa `simulated` namespace; full commissioning-grade simulation stays out of scope pending a separately funded initiative.
- **Enterprise reporting coordination (BOBJ → Power BI migration, in-flight adjacent initiative):** three consumption paths analyzed (Snowflake dbt / Historian direct / both). Recommended position: **Path C with Path A as strategic direction** — most machine-data and cross-domain reports move to Snowflake over Years 23, compliance reports stay on Historian indefinitely. Conversation with reporting team still to be scheduled.
- **Output generation pipeline:** PPTX + PDF generation from plan markdown, repeatability anchored by spec files (`presentation-spec.md`, `longform-spec.md`) rather than prompts. Spec files written; diagrams and generation run deferred until the source plan is stable.
## Top pending items (from most recent status check)
All four items from the previous status check have been **advanced to the point where the next move is a real-world action** (management meeting, reporting-team conversation, field survey, or — for legacy — closed outright). The in-room plan work that could be done without external input has been done. The remaining open items are **external dependencies**, not plan-authoring gaps.
### External-dependency items — waiting on real-world action
1. **BOBJ → Power BI coordination with reporting team.** Plan position documented in `goal-state.md` → Strategic Considerations → **Enterprise reporting: BOBJ → Power BI migration (adjacent initiative)** — three consumption paths analyzed, recommended position stated (Path C with Path A as strategic direction), eight questions and a four-bucket decision rubric included. **Action needed:** schedule the coordination conversation with the reporting team; bring back a bucket assignment. Once a bucket is assigned, update `goal-state.md` → Enterprise reporting and, if the outcome is Bucket A or B, update `roadmap.md` → Snowflake dbt Transform Layer to include reporting-shaped views.
2. **UNS hierarchy snapshot walk.** The protocol survey has been **removed** — the OtOpcUa v2 implementation team committed the core driver list (8 drivers) based on internal knowledge, making a formal protocol survey unnecessary for driver scoping. What remains is the **UNS hierarchy snapshot**: a per-site equipment-instance walk capturing site / area / line / equipment assignments and stable UUIDs, which feeds the initial `schemas` repo hierarchy definition and canonical model. See `goal-state.md`**Unified Namespace (UNS) posture → UNS naming hierarchy standard**. **Action needed:** assign a walk owner; walk System Platform IO config, Ignition OPC UA connections, and ScadaBridge templates across integrated sites within Q1Q2 of Year 1; capture equipment instances at site/area/line/equipment granularity (not protocol — that's already resolved). The canonical model v1 cannot be published without the initial hierarchy snapshot. **Sub-blocker:** the UNS hierarchy's enterprise-level shortname is currently a placeholder (`ent` in goal-state.md); the real shortname needs to be assigned before the initial hierarchy snapshot can be committed to the `schemas` repo.
3. **Digital twin use case 2 — funded simulation initiative (exploratory).** The digital twin management conversation is complete; management provided three use cases and the plan absorbs two of them (canonical state vocabulary + canonical model declaration — see closed items). **Use case 2 (Virtual Testing / Simulation)** is served minimally by Redpanda historical replay + OtOpcUa's architectural support for a future `simulated` namespace, but **full commissioning-grade simulation stays out of scope** for this plan. **No action needed** unless and until a funded simulation initiative materializes with a sponsor, scope, and timeline; at that point the meeting-prep brief at [`goal-state/digital-twin-management-brief.md`](goal-state/digital-twin-management-brief.md) can be reused for a use-case-2-specific scoping conversation. Keep on the radar, don't actively work on it.
### Closed since last status check
All closed items below were worked through the same 2026-04-15 session. Grouped roughly chronologically.
**Denominators and discovery templates:**
- ~~**Legacy integration inventory population.**~~ **Closed 2026-04-15.** The inventory in `current-state/legacy-integrations.md` is complete as the pillar 3 denominator: **3 rows** — LEG-001 Delmia DNC, LEG-002 Camstar MES (Camstar-initiated, confirmed this session), LEG-003 custom email notification service (added this session). Historian's MSSQL reporting surface (BOBJ / Power BI) was explicitly carved out as **not legacy** and documented under "Deliberately not tracked" in the inventory file — the rationale is that Historian's SQL interface is its native consumption surface, not a bespoke integration. Detail fields on the three rows (sites, owners, volumes, exact transports) remain `_TBD_` and will get filled in during migration planning.
- ~~**Equipment protocol survey template.**~~ **Advanced 2026-04-15.** The survey was listed as a Year 1 prerequisite but had no template; now a full template with schema, classification rule, rollup views, and discovery approach lives at `current-state/equipment-protocol-survey.md`. **Then further advanced** to carry a dual mandate (see below). Still open: actually running the survey (tracked above as item #2).
**Digital twin — full lifecycle in one session:**
- ~~**Digital twin conversation preparation.**~~ **Advanced 2026-04-15.** The 8 clarification questions existed but lacked framing; now wrapped with a full meeting-prep brief at `goal-state/digital-twin-management-brief.md`. Superseded by the conversation outcome below.
- ~~**Digital twin management conversation.**~~ **Closed 2026-04-15.** Conversation happened. Management delivered three use cases (source: `digital_twin_usecases.md.txt`): (1) standardized equipment state model, (2) virtual testing / simulation, (3) cross-system canonical model. Plan response: (a) use cases 1 and 3 absorbed as plan additions (see Canonical Model + UNS work below); (b) use case 2 served minimally by Redpanda historical replay + OtOpcUa architectural support for a future `simulated` namespace — full commissioning simulation stays out of scope. The `goal-state.md` Digital Twin section, the `digital-twin-management-brief.md` outcome, and the Year 1 Redpanda/dbt roadmap cells are all updated. Narrower open item carried forward as external-dependency item #3 above.
**Canonical model and UNS work (follows from digital twin use cases 1 and 3):**
- ~~**Canonical Equipment, Production, and Event Model declaration.**~~ **Closed 2026-04-15.** New subsection under `goal-state.md` → Async Event Backbone declares the canonical model: three surfaces (OtOpcUa equipment namespace, Redpanda topics + Protobuf schemas, dbt curated layer) with `schemas` repo as single source of truth. Committed **canonical machine state vocabulary** (`Running / Idle / Faulted / Starved / Blocked` + TBD additions) with explicit semantics, rules, and governance. OEE computed on the canonical state stream named as a candidate for pillar 2's "not possible before" use case. Year 1 Redpanda cell in `roadmap.md` commits to publishing v1.
- ~~**Unified Namespace (UNS) posture declaration.**~~ **Closed 2026-04-15.** New subsection under `goal-state.md` → Target IT/OT Integration declares the canonical model as **the plan's UNS**, with three deliberate deviations from classic MQTT/Sparkplug UNS (Kafka instead of MQTT, flat topics with site-in-message, stateless events instead of Sparkplug state). Optional future **UNS projection service** (MQTT/Sparkplug and/or enterprise OPC UA aggregator) documented as architecturally supported but not committed for build. Cross-references added from Canonical Model subsection and Digital Twin section.
- ~~**UNS naming hierarchy standard.**~~ **Closed 2026-04-15.** Five-level hierarchy committed: Enterprise → Site → Area → Line → Equipment, always present, `_default` placeholder where a level doesn't apply. Naming rules align with Redpanda topic convention (`[a-z0-9-]`, dots/slashes for segments, hyphens within). Stable **equipment UUIDv4** alongside the path. Authority in `schemas` repo. Evolution governance, worked examples, out-of-scope list (no product/job hierarchy — that's Camstar MES), and TBDs all captured. `current-state/equipment-protocol-survey.md` updated to note the dual mandate — same discovery walk produces the initial hierarchy snapshot at equipment-instance granularity.
**Adjacent initiatives:**
- ~~**BOBJ → Power BI coordination framing.**~~ **Advanced 2026-04-15.** The coordination question was flagged but no plan position existed; now documented as a new Strategic Considerations subsection in `goal-state.md` with three paths, recommended position, and eight questions for the reporting team. Still open: actually having the coordination conversation (tracked above as item #1).
**Output generation pipeline:**
- ~~**Repeatable PPTX + PDF generation pipeline.**~~ **Advanced 2026-04-15.** Design brainstormed (A+D pattern — Claude in full control, spec-file anchors, no templates yet). Directory scaffold created at `outputs/`. README, DESIGN, IMPLEMENTATION-PLAN, presentation-spec (18 slides, mixed-stakeholder), and longform-spec (3 chapters + 2 appendices, faithful typeset) all written. **Deferred:** Mermaid diagram source files, first PNG rendering, first PPTX and PDF generation, inaugural run-log entry — all wait on source data being stable. Trigger phrases (`regenerate outputs` / `regenerate presentation` / `regenerate longform`) documented in `outputs/README.md` for any future session.
Items that can wait, design details that close during implementation, and deliberately deferred / out-of-scope items are listed in the working conversation — no need to re-enumerate here; they're all captured as `_TBD_` markers in the authoritative files.
## Recommended resume flow
1. Resume the Claude Code session (`claude --resume 4851af09-c6b2-41fd-a419-a43d8cf80721`).
2. Skim this file and `CLAUDE.md` to re-orient (~2 minutes).
3. Pick one of the three external-dependency items above — or whatever has become most pressing — and tell me which.
4. If you've already had the Power BI coordination conversation with the reporting team, bring the answers and I'll fold them into the plan.
5. If a funded simulation initiative has materialized (digital twin use case 2), say so and I'll reuse the meeting brief for a scoping conversation.
6. If the equipment protocol survey walk has been run, bring the data and I'll populate both `current-state/equipment-protocol-survey.md` and the initial UNS hierarchy snapshot in one pass.
7. If the enterprise shortname has been assigned (currently `ent` placeholder in `goal-state.md`), tell me and I'll propagate the find-and-replace across the UNS hierarchy examples.
8. When ready to generate outputs (deck + PDF), say `regenerate outputs`, `regenerate presentation`, or `regenerate longform` and I'll follow the checklist in `outputs/README.md`.
## What not to do on resume
- Don't re-open settled decisions without a reason. The plan's decisions are load-bearing and have explicit rationale captured inline; reversing one should require new information, not re-litigation.
- Don't add new workstreams to `roadmap.md` without a matching commitment to one of the three pillars. That's how plans quietly bloat.
- Don't let Digital Twin reappear as a new committed workstream. Management's three use cases have been resolved: uses 1 and 3 absorbed into the canonical model + UNS work; use 2 stays out of scope unless a separately funded simulation initiative materializes. Full commissioning-grade simulation is not a stealth pillar.
- Don't let Copilot 365 reappear. It was deliberately removed earlier — it's handled implicitly by the Snowflake/dbt + canonical model path.
- Don't build a parallel MQTT UNS broker just because "UNS" means MQTT to many vendors. The plan's UNS posture is deliberate: Redpanda IS the UNS backbone, and a projection service is a small optional addition when a specific consumer requires it — not the default path.
- Don't hand-edit files under `outputs/generated/` — they're disposable, regenerated from the spec files on every run. Edit specs or source plan files instead.