Add 5 component data-flow diagrams (Mermaid source + PNG)

- otopcua-dataflow: equipment → drivers (8, with stability tiers) → namespaces → ACL → consumers
- redpanda-eventhub: site ScadaBridge → store-and-forward → central cluster (topics + retention tiers + schema registry) → enterprise consumers
- snowbridge-dataflow: source adapters (Historian/Redpanda) → governed selection + approval workflow → Snowflake landing
- scadabridge-dataflow: OtOpcUa inputs → scripts/templates → multiple outputs (Redpanda, Web APIs, DB, email, equipment writes, Camstar)
- snowflake-dbt-dataflow: landing → staging → curated layer (dim_equipment, fact_state_transitions, mart_oee) → Power BI / AI/ML / ad-hoc
This commit is contained in:
Joseph Doherty
2026-04-17 14:24:45 -04:00
parent e394c35020
commit eb61d07ffd
10 changed files with 202 additions and 0 deletions

View File

@@ -0,0 +1,50 @@
flowchart LR
subgraph Equipment["Equipment (Layer 1)"]
PLC1["Modbus TCP/RTU<br/>devices"]
PLC2["Siemens S7<br/>PLCs"]
PLC3["OPC UA-native<br/>equipment"]
PLC4["AB CIP/Legacy<br/>PLCs"]
PLC5["Beckhoff TwinCAT<br/>controllers"]
CNC["FANUC CNC<br/>(FOCAS)"]
end
subgraph OtOpcUa["OtOpcUa Cluster — per site (Layer 2)"]
direction TB
subgraph Drivers["Core Drivers (8)"]
D1["Modbus<br/>Tier A"]
D2["S7<br/>Tier B"]
D3["OPC UA Client<br/>Tier A"]
D4["AB CIP + Legacy<br/>Tier B"]
D5["TwinCAT<br/>Tier B"]
D6["FOCAS<br/>Tier C ⚡"]
end
subgraph Namespaces["Two Namespaces"]
EN["Equipment NS<br/>(raw data)"]
SPN["System Platform NS<br/>(processed data)"]
end
ACL["ACL Enforcer<br/>6-level scope"]
end
subgraph Consumers["Downstream Consumers"]
SB["ScadaBridge<br/>(Tier 1 cutover)"]
IG["Ignition SCADA<br/>(Tier 2 cutover)"]
SP["System Platform IO<br/>(Tier 3 cutover)"]
end
PLC1 --> D1
PLC2 --> D2
PLC3 --> D3
PLC4 --> D4
PLC5 --> D5
CNC -.->|"out-of-process<br/>named pipe"| D6
D1 --> EN
D2 --> EN
D3 --> EN
D4 --> EN
D5 --> EN
D6 --> EN
EN --> ACL
SPN --> ACL
ACL --> SB
ACL --> IG
ACL --> SP

Binary file not shown.

After

Width:  |  Height:  |  Size: 122 KiB

View File

@@ -0,0 +1,47 @@
flowchart LR
subgraph Sites["Site ScadaBridge Clusters"]
SB1["ScadaBridge<br/>Warsaw West"]
SB2["ScadaBridge<br/>Shannon"]
SB3["ScadaBridge<br/>Ponce"]
SBN["ScadaBridge<br/>... other sites"]
end
subgraph SAF["Store & Forward<br/>(per site, per call)"]
Q1["Local Queue"]
Q2["Local Queue"]
Q3["Local Queue"]
QN["Local Queue"]
end
subgraph Redpanda["Redpanda Central Cluster (South Bend)"]
direction TB
subgraph Topics["Topics: {domain}.{entity}.{event-type}"]
T1["equipment.tag.value-changed<br/>⏱ analytics 30d"]
T2["equipment.state.transitioned<br/>⏱ analytics 30d"]
T3["mes.workorder.started<br/>⏱ analytics 30d"]
T4["scada.alarm.raised<br/>⏱ operational 7d"]
T5["quality.inspection.completed<br/>⏱ compliance 90d"]
end
SR["Schema Registry<br/>Protobuf + BACKWARD_TRANSITIVE"]
AUTH["SASL/OAUTHBEARER<br/>+ prefix ACLs"]
end
subgraph Consumers["Enterprise Consumers"]
SNB["SnowBridge<br/>→ Snowflake"]
KPI["KPI Processors"]
CAM["Camstar<br/>Integration"]
REPLAY["Historical Replay<br/>(simulation-lite)"]
end
SB1 --> Q1 --> T1
SB2 --> Q2 --> T2
SB3 --> Q3 --> T3
SBN --> QN --> T4
SR -.->|"validates"| Topics
AUTH -.->|"enforces"| Topics
T1 --> SNB
T2 --> SNB
T2 --> KPI
T3 --> CAM
T1 --> REPLAY
T5 --> SNB

Binary file not shown.

After

Width:  |  Height:  |  Size: 168 KiB

View File

@@ -0,0 +1,36 @@
flowchart LR
subgraph Inputs["Data Inputs"]
OT["OtOpcUa<br/>Equipment NS"]
SP["OtOpcUa<br/>System Platform NS"]
end
subgraph ScadaBridge["ScadaBridge Cluster (per site)"]
direction TB
TPL["Templates<br/>(central DB → site push)"]
SCR["Scripts<br/>(C# Roslyn)"]
SAF["Store & Forward<br/>(per-call, optional)"]
subgraph Internals["Akka.NET Runtime"]
SUP["Supervision<br/>(self-healing)"]
CLST["2-node cluster<br/>(~25s failover)"]
end
end
subgraph Outputs["Integration Targets"]
RED["Redpanda EventHub<br/>(committed, Year 1)"]
API["External Web APIs<br/>(pre-configured, generic)"]
DB["SQL Server<br/>(batch tracking)"]
NOT["Email Notifications<br/>(contact-list driven)"]
EQ["Equipment Writes<br/>(OPC UA via OtOpcUa)"]
CAM["Camstar MES<br/>(direct Web API)"]
end
OT --> SCR
SP --> SCR
TPL --> SCR
SCR --> SAF
SAF --> RED
SAF --> API
SAF --> DB
SAF --> NOT
SCR --> EQ
SCR --> CAM

Binary file not shown.

After

Width:  |  Height:  |  Size: 123 KiB

View File

@@ -0,0 +1,30 @@
flowchart LR
subgraph Sources["Data Sources"]
HIS["Aveva Historian<br/>(SQL interface)<br/>Year 1 adapter"]
RED["Redpanda Topics<br/>(ScadaBridge events)<br/>Year 2 adapter"]
FUT["Future Sources<br/>(Ignition, Data Hub, etc.)"]
end
subgraph SnowBridge["SnowBridge (.NET service)"]
direction TB
SA["Source Adapters<br/>(pluggable per source)"]
SEL["Selection Engine<br/>(which tags/topics → Snowflake)"]
GOV["Governance Layer"]
subgraph Approval["Approval Workflow"]
SELF["Self-service<br/>(single tag, non-compliance)"]
FOUR["Four-eyes review<br/>(new topic, compliance tier,<br/>high-cost impact)"]
end
UI["Operator Web UI + API<br/>(RBAC, audit trail,<br/>exportable state)"]
end
subgraph Snowflake["Snowflake"]
LAND["Landing Tables<br/>(Snowpipe Streaming<br/>or COPY)"]
end
HIS --> SA
RED --> SA
FUT -.-> SA
SA --> SEL
UI --> GOV --> SEL
GOV --> Approval
SEL --> LAND

Binary file not shown.

After

Width:  |  Height:  |  Size: 106 KiB

View File

@@ -0,0 +1,39 @@
flowchart TB
subgraph Input["Data Ingestion"]
SNB["SnowBridge<br/>(Historian SQL + Redpanda)"]
end
subgraph Snowflake["Snowflake + dbt"]
direction TB
LAND["Landing Tables<br/>(raw, as-received)"]
STG["Staging Models<br/>(cleaned, typed)"]
subgraph Curated["Curated Layer (canonical model)"]
DIM["dim_equipment<br/>(UUID + 5 identifiers<br/>+ UNS path + class)"]
STATE["fact_state_transitions<br/>(Running/Idle/Faulted/<br/>Starved/Blocked)"]
TAGS["fact_tag_values<br/>(time-series, filtered)"]
OEE["mart_oee<br/>(cross-site OEE<br/>from canonical state)"]
EVENTS["fact_events<br/>(canonical event stream)"]
end
TESTS["dbt tests<br/>(enum divergence checks,<br/>source freshness ≤15min)"]
end
subgraph Consumers["Analytics Consumers"]
PBI["Power BI<br/>(reporting)"]
AIML["AI/ML Models<br/>(predictive)"]
ADHOC["Ad-hoc SQL<br/>(analysts)"]
end
SNB --> LAND
LAND --> STG
STG --> DIM
STG --> STATE
STG --> TAGS
STATE --> OEE
TAGS --> OEE
DIM --> PBI
OEE --> PBI
EVENTS --> AIML
STATE --> AIML
DIM --> ADHOC
TAGS --> ADHOC
TESTS -.->|"validates"| Curated

Binary file not shown.

After

Width:  |  Height:  |  Size: 126 KiB