dohertj2/3yearplan
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Expand SnowBridge to own ingest + in-process transform; drop dbt

Browse Source 
SnowBridge now owns machine-data ingest, in-process .NET transformation,
and direct writes to curated tables in Snowflake. Collapses the previous
ingest/transform split into a single service; no dbt, no external
orchestrator, no Snowflake landing tier. Keeps the in-house .NET pattern
consistent with ScadaBridge and OtOpcUa.

The "Snowflake dbt Transform Layer" roadmap workstream merges into
SnowBridge (7 → 6 workstreams); Year 2 canonical-state-based OEE moves
with it. Canonical model still has three surfaces — the third is
renamed from "dbt curated layer" to "SnowBridge curated layer in
Snowflake"; mechanics unchanged.
This commit is contained in:
Joseph Doherty
2026-04-24 14:53:16 -04:00
parent 22a86974f6
commit 98bf2d0da4
15 changed files with 130 additions and 124 deletions
Download Patch File Download Diff File Expand all files Collapse all files

2
current-state.md
View File

@@ -145,7 +145,7 @@ SCADA responsibilities are split across two platforms by purpose:
- **Retention policy: permanent.** No TTL or rollup is applied — historian data is retained **forever** as a matter of policy. This means the "drill-down to Historian for raw data" pattern in `goal-state.md` works at any historical horizon, and the historian is the authoritative long-term system of record for validated tag data regardless of how much Snowflake chooses to store.
- **Integration role:** serves as the system of record for validated/compliance-grade tag data collected via Aveva System Platform, and exposes a **SQL interface** (OPENQUERY and history views) for read access. Downstream use of that SQL interface for Snowflake ingestion is discussed in `goal-state.md` under Aveva Historian → Snowflake.
- **Current consumers (reporting):** the primary consumer of Historian data today is **enterprise reporting**, currently on **SAP BusinessObjects (BOBJ)**. Reporting is actively **migrating from SAP BOBJ to Power BI** — this is an in-flight transition that this plan should be aware of but does not own.
- **Implication for pillar 2:** the "enterprise analytics/AI enablement" target in `goal-state.md` sits alongside this Power BI migration, not in competition with it. Whether Power BI consumes from Snowflake (via the dbt curated layer), from Historian directly, or from both is a TBD that coordinates between the two initiatives.
- **Implication for pillar 2:** the "enterprise analytics/AI enablement" target in `goal-state.md` sits alongside this Power BI migration, not in competition with it. Whether Power BI consumes from Snowflake (via the SnowBridge curated layer), from Historian directly, or from both is a TBD that coordinates between the two initiatives.
- _TBD — current storage footprint and growth rate, other consumers beyond reporting (e.g., Aveva Historian Client / Insight / Trend tools, ad-hoc analyst SQL, regulatory/audit exports), and how the BOBJ→Power BI migration coordinates with the Snowflake path for machine data._
_TBD — additional shopfloor systems and HMIs not covered above (if any)._