Files

T

Joseph Doherty d4135e8ad3 fix(data-access): correct self-referential SQL in WorkCenter filter

The WHERE clause was comparing Code to itself instead of the aliased
table reference, which would always be true.

2026-01-06 14:12:07 -05:00

19 KiB

Raw Blame History

Data Sync Configuration Report

This document describes all data synchronization imports from the legacy JDE Scoping Tool (LotFinder) application.

Executive Summary

Source System	Database Type	Entity Count	Connection
JDE	Oracle (Oracle.ManagedDataAccess)	21 (16 active + 5 archive)	Config.JDECS
CMS	Oracle (DDTek.Oracle)	1	Config.CMSCS

*Note: StatusCode sync uses GIW connection (Config.GIWCS via DDTek.Oracle) instead of standard JDE connection.

Total Syncs: 22

Active Syncs: 17 (scheduled to run)
Archive Syncs: 5 (configured but ALL DISABLED)

Cache Files: All syncs have corresponding cache files in CACHED_DB_FILES/ (except StatusCode)

Schedule Legend

Schedule	Interval (min)	Frequency	PrepurgeData	ReIndexData	Sync Type
Mass	10080	Weekly	Yes	Yes	Full Reload
Daily	1440	Daily	No	No	Incremental Merge
Hourly	60	Hourly	No	No	Incremental Merge

PrepurgeData=true: Table is truncated before import (full reload)
PrepurgeData=false: Records are merged/upserted (incremental)
Filtered Query: Uses date/time parameters to fetch only changed records

Master Sync Table

Active Syncs (17)

#	Import Name	Source	Dest Table	Mass	Daily	Hourly	Filter	Cache File	Notes
1	WorkOrder	JDE	WorkOrder_Curr	Yes	Yes	Yes	Yes	`workorder_curr.json.zstd`
2	LotUsage	JDE	LotUsage_Curr	Yes	Yes	Yes	Yes	`lotusage_curr.json.zstd`
3	Item	JDE	Item	Yes	Yes	Yes	Yes	`item.json.zstd`
4	Lot	JDE	Lot	Yes	Yes	Yes	Yes	`lot.json.zstd`
5	WorkOrderTime	JDE	WorkOrderTime_Curr	Yes	Yes	Yes	Yes	`workordertime_curr.json.zstd`
6	WorkOrderComponent	JDE	WorkOrderComponent_Curr	Yes	Yes	Yes	Yes	`workordercomponent_curr.json.zstd`
7	WorkOrderStep	JDE	WorkOrderStep_Curr	Yes	Yes	Yes	Yes	`workorderstep_curr.json.zstd`
8	WorkOrderRouting	JDE	WorkOrderRouting	Yes	Yes	Yes	Yes	`workorderrouting.json.zstd`
9	Branch	JDE	Branch	Yes	Yes	Yes	Yes	`branch.json.zstd`	typeCode='BP'
10	ProfitCenter	JDE	ProfitCenter	Yes	Yes	Yes	Yes	`profitcenter.json.zstd`	typeCode='I3'
11	WorkCenter	JDE	WorkCenter	Yes	Yes	Yes	Yes	`workcenter.json.zstd`	typeCode='WC'
12	StatusCode	JDE	StatusCode	Yes	Yes	Yes	Yes	(none)	GIW connection
13	JdeUser	JDE	JdeUser	Yes	Yes	Yes	No	`jdeuser.json.zstd`	Same query both
14	OrgHierarchy	JDE	OrgHierarchy	Yes	Yes	Yes	Yes	`orghierarchy.json.zstd`
15	RouteMaster	JDE	RouteMaster	Yes	Yes	Yes	Yes	`routemaster.json.zstd`
16	FunctionCode	JDE	FunctionCode	Yes	Yes	Yes	No	`functioncode.json.zstd`	Always full reload
17	MisData	CMS	MisData	Yes	Yes	No	Yes	`misdata.json.zstd`	Hourly disabled

Archive Syncs (5) - ALL DISABLED

#	Import Name	Source	Dest Table	Mass	Daily	Hourly	Filter	Cache File	Notes
18	WorkOrder_Archive	JDE	WorkOrder_Hist	No	No	No	No	`workorder_hist.json.zstd`	DISABLED
19	LotUsage_Archive	JDE	LotUsage_Hist	No	No	No	No	`lotusage_hist.json.zstd`	DISABLED
20	WorkOrderTime_Archive	JDE	WorkOrderTime_Hist	No	No	No	No	`workordertime_hist.json.zstd`	DISABLED
21	WorkOrderComponent_Archive	JDE	WorkOrderComponent_Hist	No	No	No	No	`workordercomponent_hist.json.zstd`	DISABLED
22	WorkOrderStep_Archive	JDE	WorkOrderStep_Hist	No	No	No	No	`workorderstep_hist.json.zstd`	DISABLED

Cache File Location: CACHED_DB_FILES/

Detailed Entity Configurations

1. WorkOrder

Property	Value
Source System	JDE
Source Data	WORKORDER
Destination Table	WorkOrder_Curr
Data Fetch Function	Commons.Process.JDE.GetWorkOrders
Post Processing	None
JDE Table	JDESTAGE.F4801_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_WORKORDERS.sql
Filtered: JDE/SQL_GET_WORKORDERS_FILTERED.sql
Archive: JDE_ARCHIVE/SQL_GET_WORKORDERS.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

2. LotUsage

Property	Value
Source System	JDE
Source Data	LOTUSAGE
Destination Table	LotUsage_Curr
Data Fetch Function	Commons.Process.JDE.GetLotUsages
Post Processing	None
JDE Table	JDESTAGE.F4111_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_LOT_USAGES.sql
Filtered: JDE/SQL_GET_LOT_USAGES_FILTERED.sql
Archive: JDE_ARCHIVE/SQL_GET_LOT_USAGES.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

3. Item

Property	Value
Source System	JDE
Source Data	ITEM
Destination Table	Item
Data Fetch Function	Commons.Process.JDE.GetItems
Post Processing	None
JDE Table	JDESTAGE.F4101_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_ITEMS.sql
Filtered: JDE/SQL_GET_ITEMS_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

4. Lot

Property	Value
Source System	JDE
Source Data	LOT
Destination Table	Lot
Data Fetch Function	Commons.Process.JDE.GetLots
Post Processing	None
JDE Table	JDESTAGE.F4108_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_LOTS.sql
Filtered: JDE/SQL_GET_LOTS_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

5. WorkOrderTime

Property	Value
Source System	JDE
Source Data	WORKORDERTIME
Destination Table	WorkOrderTime_Curr
Data Fetch Function	Commons.Process.JDE.GetWorkOrderTimes
Post Processing	None
JDE Table	JDESTAGE.F31122_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_WORKORDER_TIMES.sql
Filtered: JDE/SQL_GET_WORKORDER_TIMES_FILTERED.sql
Archive: JDE_ARCHIVE/SQL_GET_WORKORDER_TIMES.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

6. WorkOrderComponent

Property	Value
Source System	JDE
Source Data	WORKORDERCOMPONENT
Destination Table	WorkOrderComponent_Curr
Data Fetch Function	Commons.Process.JDE.GetWorkOrderComponents
Post Processing	None
JDE Table	JDESTAGE.F3111_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_WORKORDER_COMPONENTS.sql
Filtered: JDE/SQL_GET_WORKORDER_COMPONENTS_FILTERED.sql
Archive: JDE_ARCHIVE/SQL_GET_WORKORDER_COMPONENTS.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

7. WorkOrderStep

Property	Value
Source System	JDE
Source Data	WORKORDERSTEP
Destination Table	WorkOrderStep_Curr
Data Fetch Function	Commons.Process.JDE.GetWorkOrderSteps
Post Processing	None
JDE Table	JDESTAGE.F3112_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_WORKORDER_STEP.sql
Filtered: JDE/SQL_GET_WORKORDER_STEP_FILTERED.sql
Archive: JDE_ARCHIVE/SQL_GET_WORKORDER_STEP.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

8. WorkOrderRouting

Property	Value
Source System	JDE
Source Data	WORKORDERROUTING
Destination Table	WorkOrderRouting
Data Fetch Function	Commons.Process.JDE.GetWorkOrderRoutings
Post Processing	None
JDE Table	JDESTAGE.F3112Z1_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_WORKORDER_ROUTING.sql
Filtered: JDE/SQL_GET_WORKORDER_ROUTING_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

Special Processing: Filters out records with invalid dates (year < 1900 or > 2500)

9. Branch

Property	Value
Source System	JDE
Source Data	BRANCH
Destination Table	Branch
Data Fetch Function	Commons.Process.JDE.GetBranches
Post Processing	None
JDE Table	JDESTAGE.F0006_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_BUSINESS_UNITS.sql
Filtered: JDE/SQL_GET_BUSINESS_UNITS_FILTERED.sql

Filter Parameters: typeCode='BP', dateUpdated, timeUpdated

10. ProfitCenter

Property	Value
Source System	JDE
Source Data	PROFITCENTER
Destination Table	ProfitCenter
Data Fetch Function	Commons.Process.JDE.GetProfitCenters
Post Processing	None
JDE Table	JDESTAGE.F0006_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_BUSINESS_UNITS.sql
Filtered: JDE/SQL_GET_BUSINESS_UNITS_FILTERED.sql

Filter Parameters: typeCode='I3', dateUpdated, timeUpdated

11. WorkCenter

Property	Value
Source System	JDE
Source Data	WORKCENTER
Destination Table	WorkCenter
Data Fetch Function	Commons.Process.JDE.GetWorkCenters
Post Processing	None
JDE Table	JDESTAGE.F0006_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_BUSINESS_UNITS.sql
Filtered: JDE/SQL_GET_BUSINESS_UNITS_FILTERED.sql

Filter Parameters: typeCode='WC', dateUpdated, timeUpdated

12. StatusCode

Property	Value
Source System	JDE
Source Data	STATUSCODE
Destination Table	StatusCode
Data Fetch Function	Commons.Process.JDE.GetStatusCodes
Post Processing	None
JDE Table	JDESTAGE.F0005_VIEW
Connection	GIW (not JDE)

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_STATUS_CODES.sql
Filtered: JDE/SQL_GET_STATUS_CODES_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated

Note: This sync uses the GIW connection (Config.GIWCS) via DDTek.Oracle, not the standard JDE connection.

13. JdeUser

Property	Value
Source System	JDE
Source Data	USER
Destination Table	JdeUser
Data Fetch Function	Commons.Process.JDE.GetUsers
Post Processing	None
JDE Tables	JDESTAGE.F0101_VIEW, JDESTAGE.F0092_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_USERS.sql

Note: No filtered query variant exists. The same query is used for both full and incremental syncs (filter parameters are passed but not used in query).

14. OrgHierarchy

Property	Value
Source System	JDE
Source Data	ORGHIERARCHY
Destination Table	OrgHierarchy
Data Fetch Function	Commons.Process.JDE.GetOrgHierarchy
Post Processing	None
JDE Table	JDESTAGE.F30006_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_ORG_HIERARCHY.sql
Filtered: JDE/SQL_GET_ORG_HIERARCHY_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

15. RouteMaster

Property	Value
Source System	JDE
Source Data	ROUTEMASTER
Destination Table	RouteMaster
Data Fetch Function	Commons.Process.JDE.GetRouteMasters
Post Processing	None
JDE Table	JDESTAGE.F3003_VIEW

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	No	No
Hourly	Yes	60	No	No

Query Files:

Full: JDE/SQL_GET_ROUTE_MASTER.sql
Filtered: JDE/SQL_GET_ROUTE_MASTER_FILTERED.sql

Filter Parameters: dateUpdated, timeUpdated (JDE date/time format)

16. FunctionCode

Property	Value
Source System	JDE
Source Data	FUNCTIONCODE
Destination Table	FunctionCode
Data Fetch Function	Commons.Process.JDE.GetFunctionCodes
Post Processing	None
JDE Table	PRODDTA.F00192 (direct table, not view)

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	10080	Yes	Yes
Daily	Yes	1440	Yes	Yes
Hourly	Yes	60	Yes	Yes

Query Files:

Full: JDE/SQL_GET_FUNCTION_CODES.sql

Note: No filtered query variant. Always performs full reload (PrepurgeData=true for all schedules). Uses aggregation with LISTAGG to combine multi-row descriptions.

17. MisData

Property	Value
Source System	CMS
Source Data	MISDATA
Destination Table	MisData
Data Fetch Function	Commons.Process.CMS.GetMisData
Post Processing	Commons.Process.LotFinderDB.PostProcessMisData
CMS Schema	INFODBA

Schedule Configuration:

Schedule	Enabled	Interval	Prepurge	ReIndex
Mass	Yes	100800	Yes	Yes
Daily	Yes	1440	No	No
Hourly	No	60	No	No

Query Files:

Full: CMS/SQL_GET_MIS_DATA.sql
Filtered: CMS/SQL_GET_MIS_DATA_FILTERED.sql

Filter Parameters: lastUpdateDT (standard DateTime)

Note:

Mass interval is 100800 minutes (~10 weeks), much longer than other syncs
Hourly sync is disabled
Has post-processing action for additional data transformation
Query timeout is extended (1200*50 = 60000 seconds)
ReleaseDate is converted to local time after fetch

Archive Query Pattern

Archive syncs (#18-22) fetch historical data using UNION ALL from both current and archived schemas:

SELECT ... FROM QADTA.F{table}
UNION ALL
SELECT ... FROM ARCDTAQA.F{table}

Query Files: Located in JDE_ARCHIVE/ folder.

Note: Archive syncs have IsEnabled=true in dsconfig but all schedule types have Enabled=false. They are configured for manual/on-demand execution only.

Source Database Reference

JDE Oracle Connection (Config.JDECS)

Driver: Oracle.ManagedDataAccess.Client
Schemas Used:
- JDESTAGE - Views for current production data
- PRODDTA - Direct table access (FunctionCode only)
- QADTA - Current data for archive queries
- ARCDTAQA - Archived historical data

GIW Oracle Connection (Config.GIWCS)

Driver: DDTek.Oracle
Used By: StatusCode sync only
Schema: JDESTAGE

CMS Oracle Connection (Config.CMSCS)

Driver: DDTek.Oracle
Schema: INFODBA
Tables: Complex 11-table join for MIS data

JDE Date/Time Format

JDE uses a special date/time format conversion:

Dates are converted via ToJDEDate() helper
Times are converted via ToJDETime() helper
Filter conditions use: (date > :dateUpdated) OR (date = :dateUpdated AND time >= :timeUpdated)

Source Files Reference

File Type	Location
dsconfig JSON files	OLD/WorkerService/dsconfig/*.json
JDE Source Queries	OLD/WorkerService/bin/UPDATER/JdeQueries/*.sql
JDE Archive Queries	OLD/WorkerService/bin/UPDATER/JdeArchivalQueries/*.sql
CMS Source Queries	OLD/DataModel/CmsQueries/*.sql
C# Data Fetch Code	OLD/DataModel/Process/JDE..cs, CMS..cs
Configuration	OLD/DataModel/Config.cs
Query Repository	OLD/DataModel/Process/QueryRepository.cs

19 KiB Raw Blame History

Data Sync Configuration Report

Executive Summary

Schedule Legend

Master Sync Table

Active Syncs (17)

Archive Syncs (5) - ALL DISABLED

Detailed Entity Configurations

1. WorkOrder

2. LotUsage

3. Item

4. Lot

5. WorkOrderTime

6. WorkOrderComponent

7. WorkOrderStep

8. WorkOrderRouting

9. Branch

10. ProfitCenter

11. WorkCenter

12. StatusCode

13. JdeUser

14. OrgHierarchy

15. RouteMaster

16. FunctionCode

17. MisData

Archive Query Pattern

Source Database Reference

JDE Oracle Connection (Config.JDECS)

GIW Oracle Connection (Config.GIWCS)

CMS Oracle Connection (Config.CMSCS)

JDE Date/Time Format

Source Files Reference

19 KiB

Raw Blame History