dohertj2/lmxopcua
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Replace full address space rebuild with incremental subtree sync

Browse Source 
On Galaxy deploy changes, only the affected gobject subtrees are torn down
and rebuilt instead of destroying the entire address space. Unchanged nodes,
subscriptions, and alarm tracking continue uninterrupted.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-03-26 15:23:11 -04:00
parent bfd360a6db
commit 3c326e2d45
6 changed files with 1047 additions and 46 deletions
Download Patch File Download Diff File Expand all files Collapse all files

15
docs/reqs/OpcUaServerReqs.md
View File

@@ -181,17 +181,20 @@ When a Galaxy deployment change is detected, the server shall rebuild the addres
### Acceptance Criteria ### Acceptance Criteria
- When Galaxy Repository detects a deployment change, the OPC UA address space is rebuilt. - When Galaxy Repository detects a deployment change, the OPC UA address space is updated.
- Only changed gobject subtrees are torn down and rebuilt; unchanged nodes, subscriptions, and alarm tracking remain intact.
- Existing OPC UA client sessions are preserved — clients stay connected. - Existing OPC UA client sessions are preserved — clients stay connected.
- Subscriptions for tags that still exist after rebuild continue to work. - Subscriptions for tags on unchanged objects continue to work without interruption.
- Subscriptions for tags that no longer exist receive a Bad_NodeIdUnknown status notification. - Subscriptions for tags that no longer exist receive a Bad_NodeIdUnknown status notification.
- Rebuild is logged at Information level with timing (duration). - Sync is logged at Information level with the number of changed gobjects.
### Details ### Details
- Rebuild is a full replace, not an incremental diff. Re-query hierarchy and attributes, build new tree, swap atomically. - Uses incremental subtree sync: compares previous hierarchy+attributes with new, identifies changed gobject IDs, expands to include child subtrees, tears down only affected subtrees, and rebuilds them.
- During rebuild, reads/writes against the old address space may fail briefly. This is acceptable. - First build (no cached state) performs a full build.
- New MXAccess subscriptions for new tags are established; removed tags are unsubscribed. - If no changes are detected, the sync is a no-op (logged and skipped).
- Alarm tracking and MXAccess subscriptions for unchanged objects are not disrupted.
- Falls back to full rebuild behavior if the entire hierarchy changes.
--- ---

219
partial_update.md Normal file
View File

@@ -0,0 +1,219 @@
# Partial Address Space Update Plan
## Problem
When the Galaxy detects a new deployment (`time_of_last_deploy` changes), the server performs a **full rebuild**: unsubscribes all MXAccess tags, deletes all OPC UA nodes, reconstructs the entire address space, then re-subscribes. This disrupts all connected clients even if only one object changed.
## Goal
Replace the full rebuild with a **subtree-level sync**: detect which Galaxy objects changed, tear down and rebuild only those subtrees, and leave everything else untouched.
## Current Flow (Full Rebuild)
```
ChangeDetectionService polls galaxy.time_of_last_deploy
→ timestamp changed
→ RebuildAddressSpace(newHierarchy, newAttributes)
1. Unsubscribe ALL MXAccess tags
2. Delete ALL OPC UA nodes
3. Clear all dictionaries
4. BuildAddressSpace() from scratch
5. Re-subscribe surviving tags
```
## Proposed Flow (Subtree Sync)
```
ChangeDetectionService polls galaxy.time_of_last_deploy
→ timestamp changed
→ SyncAddressSpace(newHierarchy, newAttributes)
1. Compare old vs new by GobjectId
2. Identify changed gobjects (added, removed, or any field/attribute difference)
3. Expand changed set to include child gobjects (subtree)
4. Tear down changed subtrees (delete nodes, unsubscribe, remove alarm tracking)
5. Rebuild changed subtrees using existing BuildAddressSpace logic
6. Update cache
```
## Design
### 1. Cache Previous State
```csharp
private List<GalaxyObjectInfo>? _lastHierarchy;
private List<GalaxyAttributeInfo>? _lastAttributes;
```
### 2. Detect Changed GobjectIds
Compare old vs new to find which gobjects have any difference:
```csharp
static HashSet<int> FindChangedGobjectIds(
List<GalaxyObjectInfo> oldH, List<GalaxyAttributeInfo> oldA,
List<GalaxyObjectInfo> newH, List<GalaxyAttributeInfo> newA)
```
A gobject is "changed" if any of these differ:
- **Added**: gobject_id exists in new but not old
- **Removed**: gobject_id exists in old but not new
- **Object modified**: any field differs (TagName, BrowseName, ParentGobjectId, IsArea, ContainedName)
- **Attributes modified**: the set of attributes for that gobject_id differs (count, or any attribute field changed)
### 3. Expand to Subtrees
If a parent object changed, its children must also be rebuilt (they may reference the parent node). Expand the changed set:
```csharp
// Walk children: if gobject X changed, all gobjects with ParentGobjectId == X are also changed
static HashSet<int> ExpandToSubtrees(HashSet<int> changed, List<GalaxyObjectInfo> hierarchy)
```
This is recursive — if TestArea changed, TestMachine_001, DelmiaReceiver_001, and MESReceiver_001 all get rebuilt.
### 4. Tear Down Changed Subtrees
For each changed gobject_id:
- Find all variable nodes owned by this gobject (from `_tagToVariableNode` by matching tag prefix)
- Unsubscribe active MXAccess subscriptions for those tags
- Remove alarm tracking entries for those tags
- Delete the variable nodes from OPC UA
- Delete the object/folder node itself
- Remove from all dictionaries
### 5. Rebuild Changed Subtrees
Reuse existing code:
- Filter `newHierarchy` and `newAttributes` to only the changed gobject_ids
- Run the same node creation logic (topological sort, CreateFolder/CreateObject, CreateAttributeVariable, alarm tracking)
- The parent nodes for changed subtrees already exist (unchanged parents stay in place)
- If a parent was also removed (whole subtree removed), skip — children under root folder
### 6. SyncAddressSpace Method
```csharp
public void SyncAddressSpace(List<GalaxyObjectInfo> newHierarchy, List<GalaxyAttributeInfo> newAttributes)
{
lock (Lock)
{
if (_lastHierarchy == null)
{
BuildAddressSpace(newHierarchy, newAttributes);
_lastHierarchy = newHierarchy;
_lastAttributes = newAttributes;
return;
}
var changedIds = FindChangedGobjectIds(_lastHierarchy, _lastAttributes, newHierarchy, newAttributes);
if (changedIds.Count == 0)
{
Log.Information("No address space changes detected");
_lastHierarchy = newHierarchy;
_lastAttributes = newAttributes;
return;
}
// Expand to include child subtrees
changedIds = ExpandToSubtrees(changedIds, _lastHierarchy);
changedIds = ExpandToSubtrees(changedIds, newHierarchy);
Log.Information("Incremental sync: {Count} gobjects changed", changedIds.Count);
// Tear down changed subtrees
TearDownGobjects(changedIds);
// Rebuild changed subtrees from new data
var changedHierarchy = newHierarchy.Where(h => changedIds.Contains(h.GobjectId)).ToList();
var changedAttributes = newAttributes.Where(a => changedIds.Contains(a.GobjectId)).ToList();
BuildSubtree(changedHierarchy, changedAttributes);
_lastHierarchy = newHierarchy;
_lastAttributes = newAttributes;
Log.Information("Incremental sync complete: {Objects} objects, {Variables} variables, {Alarms} alarms",
ObjectNodeCount, VariableNodeCount, _alarmInAlarmTags.Count);
}
}
```
### 7. TearDownGobjects
```csharp
private void TearDownGobjects(HashSet<int> gobjectIds)
{
// Collect tag references owned by these gobjects
// (match by TagName prefix from hierarchy)
// For each: unsubscribe, remove alarm tracking, delete node, remove from dictionaries
}
```
Key: need a way to map gobject_id → set of tag references. Options:
- Store a `_gobjectIdToTagRefs` dictionary during build
- Or derive from `_tagToVariableNode` keys + hierarchy TagName matching
### 8. BuildSubtree
Reuse the same logic as `BuildAddressSpace` but:
- Only process the filtered hierarchy/attributes
- Parent nodes for the subtree roots already exist in `nodeMap` (they're unchanged)
- Need access to the existing `nodeMap` — either keep it as a field or rebuild from `PredefinedNodes`
This means `nodeMap` (currently local to `BuildAddressSpace`) should become a class field `_nodeMap`:
```csharp
private readonly Dictionary<int, NodeState> _nodeMap = new();
```
## What Stays the Same
- Nodes for unchanged gobjects → untouched
- MXAccess subscriptions for unchanged tags → untouched
- Alarm tracking for unchanged alarms → untouched
- OPC UA client subscriptions on unchanged nodes → uninterrupted
- `_pendingDataChanges` queue → continues processing (DispatchLoop skips missing nodes gracefully)
## Edge Cases
| Case | Handling |
|---|---|
| First build (no cache) | Full `BuildAddressSpace` |
| No changes detected | Log and skip |
| Object removed | Tear down subtree, children become orphaned → also removed |
| Object added | Build new subtree under existing parent |
| Object re-parented | Both old and new parent subtrees detected as changed → both rebuilt |
| All objects changed | Equivalent to full rebuild (acceptable) |
| Root folder | Never torn down — only child subtrees |
## Testing
### Unit Tests (`AddressSpaceDiffTests`)
- `FindChangedGobjectIds` — verify detection of added, removed, modified objects
- `FindChangedGobjectIds` — verify attribute changes trigger gobject as changed
- `ExpandToSubtrees` — verify children are included
### Integration Tests (`IncrementalSyncTests`)
- Add object → appears in browse, existing subscriptions unaffected
- Remove object → disappears, subscriptions on surviving nodes continue
- Modify attribute on one object → only that subtree rebuilds, others untouched
- Verify subscription continuity: subscribe to node on Object A, modify Object B, subscription on A still delivers data
## Files to Create/Modify
| File | Change |
|---|---|
| `src/.../OpcUa/LmxNodeManager.cs` | Add `SyncAddressSpace`, `TearDownGobjects`, `BuildSubtree`; promote `nodeMap` to field; cache `_lastHierarchy`/`_lastAttributes` |
| `src/.../OpcUa/AddressSpaceDiff.cs` | NEW — `FindChangedGobjectIds`, `ExpandToSubtrees` (static helpers) |
| `tests/.../OpcUa/AddressSpaceDiffTests.cs` | NEW — unit tests for diff logic |
| `tests/.../Integration/IncrementalSyncTests.cs` | NEW — integration tests |
## Comparison: Full Rebuild vs Subtree Sync
| Aspect | Full Rebuild | Subtree Sync |
|---|---|---|
| Scope of disruption | All nodes, all clients | Only changed subtrees |
| MXAccess churn | Unsubscribe/resubscribe all | Only changed tags |
| Lock duration | Long (rebuild everything) | Short (rebuild subset) |
| Complexity | Simple (clear + build) | Moderate (diff + selective rebuild) |
| Correctness risk | Low (clean slate) | Medium (must handle orphans, partial state) |
| Fallback | N/A | Fall back to full rebuild on error |

138
src/ZB.MOM.WW.LmxOpcUa.Host/OpcUa/AddressSpaceDiff.cs Normal file
View File

@@ -0,0 +1,138 @@
using System.Collections.Generic;
using System.Linq;
using ZB.MOM.WW.LmxOpcUa.Host.Domain;
namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
{
/// <summary>
/// Computes the set of changed Galaxy object IDs between two snapshots of hierarchy and attributes.
/// </summary>
public static class AddressSpaceDiff
{
/// <summary>
/// Compares old and new hierarchy+attributes and returns the set of gobject IDs that have any difference.
/// </summary>
public static HashSet<int> FindChangedGobjectIds(
List<GalaxyObjectInfo> oldHierarchy, List<GalaxyAttributeInfo> oldAttributes,
List<GalaxyObjectInfo> newHierarchy, List<GalaxyAttributeInfo> newAttributes)
{
var changed = new HashSet<int>();
var oldObjects = oldHierarchy.ToDictionary(h => h.GobjectId);
var newObjects = newHierarchy.ToDictionary(h => h.GobjectId);
// Added objects
foreach (var id in newObjects.Keys)
{
if (!oldObjects.ContainsKey(id))
changed.Add(id);
}
// Removed objects
foreach (var id in oldObjects.Keys)
{
if (!newObjects.ContainsKey(id))
changed.Add(id);
}
// Modified objects
foreach (var kvp in newObjects)
{
if (oldObjects.TryGetValue(kvp.Key, out var oldObj) && !ObjectsEqual(oldObj, kvp.Value))
changed.Add(kvp.Key);
}
// Attribute changes — group by gobject_id and compare
var oldAttrsByObj = oldAttributes.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
var newAttrsByObj = newAttributes.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
// All gobject_ids that have attributes in either old or new
var allAttrGobjectIds = new HashSet<int>(oldAttrsByObj.Keys);
allAttrGobjectIds.UnionWith(newAttrsByObj.Keys);
foreach (var id in allAttrGobjectIds)
{
if (changed.Contains(id))
continue;
oldAttrsByObj.TryGetValue(id, out var oldAttrs);
newAttrsByObj.TryGetValue(id, out var newAttrs);
if (!AttributeSetsEqual(oldAttrs, newAttrs))
changed.Add(id);
}
return changed;
}
/// <summary>
/// Expands a set of changed gobject IDs to include all descendant gobject IDs in the hierarchy.
/// </summary>
public static HashSet<int> ExpandToSubtrees(HashSet<int> changed, List<GalaxyObjectInfo> hierarchy)
{
var childrenByParent = hierarchy.GroupBy(h => h.ParentGobjectId)
.ToDictionary(g => g.Key, g => g.Select(h => h.GobjectId).ToList());
var expanded = new HashSet<int>(changed);
var queue = new Queue<int>(changed);
while (queue.Count > 0)
{
var id = queue.Dequeue();
if (childrenByParent.TryGetValue(id, out var children))
{
foreach (var childId in children)
{
if (expanded.Add(childId))
queue.Enqueue(childId);
}
}
}
return expanded;
}
private static bool ObjectsEqual(GalaxyObjectInfo a, GalaxyObjectInfo b)
{
return a.TagName == b.TagName
&& a.BrowseName == b.BrowseName
&& a.ContainedName == b.ContainedName
&& a.ParentGobjectId == b.ParentGobjectId
&& a.IsArea == b.IsArea;
}
private static bool AttributeSetsEqual(List<GalaxyAttributeInfo>? a, List<GalaxyAttributeInfo>? b)
{
if (a == null && b == null) return true;
if (a == null || b == null) return false;
if (a.Count != b.Count) return false;
// Sort by a stable key and compare pairwise
var sortedA = a.OrderBy(x => x.FullTagReference).ThenBy(x => x.PrimitiveName).ToList();
var sortedB = b.OrderBy(x => x.FullTagReference).ThenBy(x => x.PrimitiveName).ToList();
for (int i = 0; i < sortedA.Count; i++)
{
if (!AttributesEqual(sortedA[i], sortedB[i]))
return false;
}
return true;
}
private static bool AttributesEqual(GalaxyAttributeInfo a, GalaxyAttributeInfo b)
{
return a.AttributeName == b.AttributeName
&& a.FullTagReference == b.FullTagReference
&& a.MxDataType == b.MxDataType
&& a.IsArray == b.IsArray
&& a.ArrayDimension == b.ArrayDimension
&& a.PrimitiveName == b.PrimitiveName
&& a.SecurityClassification == b.SecurityClassification
&& a.IsHistorized == b.IsHistorized
&& a.IsAlarm == b.IsAlarm;
}
}
}

392
src/ZB.MOM.WW.LmxOpcUa.Host/OpcUa/LmxNodeManager.cs
View File

@@ -62,6 +62,12 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
// Alarm tracking: maps InAlarm tag reference → alarm source info // Alarm tracking: maps InAlarm tag reference → alarm source info
private readonly Dictionary<string, AlarmInfo> _alarmInAlarmTags = new Dictionary<string, AlarmInfo>(StringComparer.OrdinalIgnoreCase); private readonly Dictionary<string, AlarmInfo> _alarmInAlarmTags = new Dictionary<string, AlarmInfo>(StringComparer.OrdinalIgnoreCase);
// Incremental sync: persistent node map and reverse lookup
private readonly Dictionary<int, NodeState> _nodeMap = new Dictionary<int, NodeState>();
private readonly Dictionary<int, List<string>> _gobjectToTagRefs = new Dictionary<int, List<string>>();
private List<GalaxyObjectInfo>? _lastHierarchy;
private List<GalaxyAttributeInfo>? _lastAttributes;
private sealed class AlarmInfo private sealed class AlarmInfo
{ {
public string SourceTagReference { get; set; } = ""; public string SourceTagReference { get; set; } = "";
@@ -164,6 +170,8 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
_tagToVariableNode.Clear(); _tagToVariableNode.Clear();
_tagMetadata.Clear(); _tagMetadata.Clear();
_alarmInAlarmTags.Clear(); _alarmInAlarmTags.Clear();
_nodeMap.Clear();
_gobjectToTagRefs.Clear();
VariableNodeCount = 0; VariableNodeCount = 0;
ObjectNodeCount = 0; ObjectNodeCount = 0;
@@ -193,12 +201,10 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
}); });
// Create nodes for each object in hierarchy // Create nodes for each object in hierarchy
var nodeMap = new Dictionary<int, NodeState>();
foreach (var obj in sorted) foreach (var obj in sorted)
{ {
NodeState parentNode; NodeState parentNode;
if (nodeMap.TryGetValue(obj.ParentGobjectId, out var p)) if (_nodeMap.TryGetValue(obj.ParentGobjectId, out var p))
parentNode = p; parentNode = p;
else else
parentNode = rootFolder; parentNode = rootFolder;
@@ -221,7 +227,7 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
} }
AddPredefinedNode(SystemContext, node); AddPredefinedNode(SystemContext, node);
nodeMap[obj.GobjectId] = node; _nodeMap[obj.GobjectId] = node;
// Create variable nodes for this object's attributes // Create variable nodes for this object's attributes
if (attrsByObject.TryGetValue(obj.GobjectId, out var objAttrs)) if (attrsByObject.TryGetValue(obj.GobjectId, out var objAttrs))
@@ -347,7 +353,7 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
} }
// Enable EventNotifier on object nodes that contain alarms // Enable EventNotifier on object nodes that contain alarms
if (hasAlarms && nodeMap.TryGetValue(obj.GobjectId, out var objNode)) if (hasAlarms && _nodeMap.TryGetValue(obj.GobjectId, out var objNode))
{ {
if (objNode is BaseObjectState objState) if (objNode is BaseObjectState objState)
objState.EventNotifier = EventNotifiers.SubscribeToEvents; objState.EventNotifier = EventNotifiers.SubscribeToEvents;
@@ -360,6 +366,9 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
if (_alarmTrackingEnabled) if (_alarmTrackingEnabled)
SubscribeAlarmTags(); SubscribeAlarmTags();
_lastHierarchy = new List<GalaxyObjectInfo>(hierarchy);
_lastAttributes = new List<GalaxyAttributeInfo>(attributes);
Log.Information("Address space built: {Objects} objects, {Variables} variables, {Mappings} tag references, {Alarms} alarm tags", Log.Information("Address space built: {Objects} objects, {Variables} variables, {Mappings} tag references, {Alarms} alarm tags",
ObjectNodeCount, VariableNodeCount, _nodeIdToTagReference.Count, _alarmInAlarmTags.Count); ObjectNodeCount, VariableNodeCount, _nodeIdToTagReference.Count, _alarmInAlarmTags.Count);
} }
@@ -428,54 +437,66 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
/// <param name="hierarchy">The latest Galaxy object hierarchy to publish.</param> /// <param name="hierarchy">The latest Galaxy object hierarchy to publish.</param>
/// <param name="attributes">The latest Galaxy attributes to publish.</param> /// <param name="attributes">The latest Galaxy attributes to publish.</param>
public void RebuildAddressSpace(List<GalaxyObjectInfo> hierarchy, List<GalaxyAttributeInfo> attributes) public void RebuildAddressSpace(List<GalaxyObjectInfo> hierarchy, List<GalaxyAttributeInfo> attributes)
{
SyncAddressSpace(hierarchy, attributes);
}
/// <summary>
/// Incrementally syncs the address space by detecting changed gobjects and rebuilding only those subtrees. (OPC-010)
/// </summary>
public void SyncAddressSpace(List<GalaxyObjectInfo> hierarchy, List<GalaxyAttributeInfo> attributes)
{ {
lock (Lock) lock (Lock)
{ {
Log.Information("Rebuilding address space..."); if (_lastHierarchy == null || _lastAttributes == null)
var activeSubscriptions = new Dictionary<string, int>(_subscriptionRefCounts, StringComparer.OrdinalIgnoreCase);
foreach (var tagRef in activeSubscriptions.Keys)
{ {
try Log.Information("No previous state cached — performing full build");
{ BuildAddressSpace(hierarchy, attributes);
_mxAccessClient.UnsubscribeAsync(tagRef).GetAwaiter().GetResult(); return;
}
catch (Exception ex)
{
Log.Warning(ex, "Failed to unsubscribe {TagRef} during rebuild", tagRef);
}
} }
// Unsubscribe auto-subscribed alarm tags var changedIds = AddressSpaceDiff.FindChangedGobjectIds(
foreach (var kvp in _alarmInAlarmTags) _lastHierarchy, _lastAttributes, hierarchy, attributes);
if (changedIds.Count == 0)
{ {
foreach (var tag in new[] { kvp.Key, kvp.Value.PriorityTagReference, kvp.Value.DescAttrNameTagReference }) Log.Information("No address space changes detected");
_lastHierarchy = hierarchy;
_lastAttributes = attributes;
return;
}
// Expand to include child subtrees in both old and new hierarchy
changedIds = AddressSpaceDiff.ExpandToSubtrees(changedIds, _lastHierarchy);
changedIds = AddressSpaceDiff.ExpandToSubtrees(changedIds, hierarchy);
Log.Information("Incremental sync: {Count} gobjects changed out of {Total}",
changedIds.Count, hierarchy.Count);
// Snapshot subscriptions for changed tags before teardown
var affectedSubscriptions = new Dictionary<string, int>(StringComparer.OrdinalIgnoreCase);
foreach (var id in changedIds)
{
if (_gobjectToTagRefs.TryGetValue(id, out var tagRefs))
{ {
if (!string.IsNullOrEmpty(tag)) foreach (var tagRef in tagRefs)
{ {
try { _mxAccessClient.UnsubscribeAsync(tag).GetAwaiter().GetResult(); } if (_subscriptionRefCounts.TryGetValue(tagRef, out var count))
catch { /* ignore */ } affectedSubscriptions[tagRef] = count;
} }
} }
} }
// Remove all predefined nodes // Tear down changed subtrees
foreach (var nodeId in PredefinedNodes.Keys.ToList()) TearDownGobjects(changedIds);
{
try { DeleteNode(SystemContext, nodeId); }
catch { /* ignore cleanup errors */ }
}
PredefinedNodes.Clear(); // Rebuild changed subtrees from new data
_nodeIdToTagReference.Clear(); var changedHierarchy = hierarchy.Where(h => changedIds.Contains(h.GobjectId)).ToList();
_tagToVariableNode.Clear(); var changedAttributes = attributes.Where(a => changedIds.Contains(a.GobjectId)).ToList();
_tagMetadata.Clear(); BuildSubtree(changedHierarchy, changedAttributes);
_subscriptionRefCounts.Clear();
// Rebuild // Restore subscriptions for surviving tags
BuildAddressSpace(hierarchy, attributes); foreach (var kvp in affectedSubscriptions)
foreach (var kvp in activeSubscriptions)
{ {
if (!_tagToVariableNode.ContainsKey(kvp.Key)) if (!_tagToVariableNode.ContainsKey(kvp.Key))
continue; continue;
@@ -487,11 +508,293 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
} }
catch (Exception ex) catch (Exception ex)
{ {
Log.Warning(ex, "Failed to restore subscription for {TagRef} after rebuild", kvp.Key); Log.Warning(ex, "Failed to restore subscription for {TagRef} after sync", kvp.Key);
} }
} }
Log.Information("Address space rebuild complete"); _lastHierarchy = new List<GalaxyObjectInfo>(hierarchy);
_lastAttributes = new List<GalaxyAttributeInfo>(attributes);
Log.Information("Incremental sync complete: {Objects} objects, {Variables} variables, {Alarms} alarms",
ObjectNodeCount, VariableNodeCount, _alarmInAlarmTags.Count);
}
}
private void TearDownGobjects(HashSet<int> gobjectIds)
{
foreach (var id in gobjectIds)
{
// Remove variable nodes and their tracking data
if (_gobjectToTagRefs.TryGetValue(id, out var tagRefs))
{
foreach (var tagRef in tagRefs.ToList())
{
// Unsubscribe if actively subscribed
if (_subscriptionRefCounts.ContainsKey(tagRef))
{
try { _mxAccessClient.UnsubscribeAsync(tagRef).GetAwaiter().GetResult(); }
catch { /* ignore */ }
_subscriptionRefCounts.Remove(tagRef);
}
// Remove alarm tracking for this tag's InAlarm/Priority/DescAttrName
var alarmKeysToRemove = _alarmInAlarmTags
.Where(kvp => kvp.Value.SourceTagReference == tagRef)
.Select(kvp => kvp.Key)
.ToList();
foreach (var alarmKey in alarmKeysToRemove)
{
var info = _alarmInAlarmTags[alarmKey];
// Unsubscribe alarm auto-subscriptions
foreach (var alarmTag in new[] { alarmKey, info.PriorityTagReference, info.DescAttrNameTagReference })
{
if (!string.IsNullOrEmpty(alarmTag))
{
try { _mxAccessClient.UnsubscribeAsync(alarmTag).GetAwaiter().GetResult(); }
catch { /* ignore */ }
}
}
_alarmInAlarmTags.Remove(alarmKey);
}
// Delete variable node
if (_tagToVariableNode.TryGetValue(tagRef, out var variable))
{
try { DeleteNode(SystemContext, variable.NodeId); }
catch { /* ignore */ }
_tagToVariableNode.Remove(tagRef);
}
// Clean up remaining mappings
var nodeIdStr = _nodeIdToTagReference.FirstOrDefault(kvp => kvp.Value == tagRef).Key;
if (nodeIdStr != null)
_nodeIdToTagReference.Remove(nodeIdStr);
_tagMetadata.Remove(tagRef);
VariableNodeCount--;
}
_gobjectToTagRefs.Remove(id);
}
// Delete the object/folder node itself
if (_nodeMap.TryGetValue(id, out var objNode))
{
try { DeleteNode(SystemContext, objNode.NodeId); }
catch { /* ignore */ }
_nodeMap.Remove(id);
if (!(objNode is FolderState))
ObjectNodeCount--;
}
}
}
private void BuildSubtree(List<GalaxyObjectInfo> hierarchy, List<GalaxyAttributeInfo> attributes)
{
if (hierarchy.Count == 0)
return;
var sorted = TopologicalSort(hierarchy);
var attrsByObject = attributes
.GroupBy(a => a.GobjectId)
.ToDictionary(g => g.Key, g => g.ToList());
// Find root folder for orphaned nodes
NodeState? rootFolder = null;
if (PredefinedNodes.TryGetValue(new NodeId("ZB", NamespaceIndex), out var rootNode))
rootFolder = rootNode;
foreach (var obj in sorted)
{
NodeState parentNode;
if (_nodeMap.TryGetValue(obj.ParentGobjectId, out var p))
parentNode = p;
else if (rootFolder != null)
parentNode = rootFolder;
else
continue; // no parent available
// Create node with final NodeId before adding to parent
NodeState node;
var nodeId = new NodeId(obj.TagName, NamespaceIndex);
if (obj.IsArea)
{
var folder = new FolderState(parentNode)
{
SymbolicName = obj.BrowseName,
ReferenceTypeId = ReferenceTypes.Organizes,
TypeDefinitionId = ObjectTypeIds.FolderType,
NodeId = nodeId,
BrowseName = new QualifiedName(obj.BrowseName, NamespaceIndex),
DisplayName = new LocalizedText("en", obj.BrowseName),
WriteMask = AttributeWriteMask.None,
UserWriteMask = AttributeWriteMask.None,
EventNotifier = EventNotifiers.None
};
parentNode.AddChild(folder);
node = folder;
}
else
{
var objNode = new BaseObjectState(parentNode)
{
SymbolicName = obj.BrowseName,
ReferenceTypeId = ReferenceTypes.HasComponent,
TypeDefinitionId = ObjectTypeIds.BaseObjectType,
NodeId = nodeId,
BrowseName = new QualifiedName(obj.BrowseName, NamespaceIndex),
DisplayName = new LocalizedText("en", obj.BrowseName),
WriteMask = AttributeWriteMask.None,
UserWriteMask = AttributeWriteMask.None,
EventNotifier = EventNotifiers.None
};
parentNode.AddChild(objNode);
node = objNode;
ObjectNodeCount++;
}
AddPredefinedNode(SystemContext, node);
_nodeMap[obj.GobjectId] = node;
parentNode.ClearChangeMasks(SystemContext, false);
// Create variable nodes (same logic as BuildAddressSpace)
if (attrsByObject.TryGetValue(obj.GobjectId, out var objAttrs))
{
var byPrimitive = objAttrs
.GroupBy(a => a.PrimitiveName ?? "")
.OrderBy(g => g.Key);
var primitiveGroupNames = new HashSet<string>(
byPrimitive.Select(g => g.Key).Where(k => !string.IsNullOrEmpty(k)),
StringComparer.OrdinalIgnoreCase);
var variableNodes = new Dictionary<string, BaseDataVariableState>(StringComparer.OrdinalIgnoreCase);
var directGroup = byPrimitive.FirstOrDefault(g => string.IsNullOrEmpty(g.Key));
if (directGroup != null)
{
foreach (var attr in directGroup)
{
var variable = CreateAttributeVariable(node, attr);
if (primitiveGroupNames.Contains(attr.AttributeName))
variableNodes[attr.AttributeName] = variable;
}
}
foreach (var group in byPrimitive)
{
if (string.IsNullOrEmpty(group.Key))
continue;
NodeState parentForAttrs;
if (variableNodes.TryGetValue(group.Key, out var existingVariable))
{
parentForAttrs = existingVariable;
}
else
{
var primNode = CreateObject(node, group.Key, group.Key);
primNode.NodeId = new NodeId(obj.TagName + "." + group.Key, NamespaceIndex);
AddPredefinedNode(SystemContext, primNode);
parentForAttrs = primNode;
}
foreach (var attr in group)
{
CreateAttributeVariable(parentForAttrs, attr);
}
}
}
}
// Alarm tracking for the new subtree
if (_alarmTrackingEnabled)
{
foreach (var obj in sorted)
{
if (obj.IsArea) continue;
if (!attrsByObject.TryGetValue(obj.GobjectId, out var objAttrs)) continue;
var hasAlarms = false;
var alarmAttrs = objAttrs.Where(a => a.IsAlarm && string.IsNullOrEmpty(a.PrimitiveName)).ToList();
foreach (var alarmAttr in alarmAttrs)
{
var inAlarmTagRef = alarmAttr.FullTagReference.TrimEnd('[', ']') + ".InAlarm";
if (!_tagToVariableNode.ContainsKey(inAlarmTagRef))
continue;
var alarmNodeIdStr = alarmAttr.FullTagReference.EndsWith("[]")
? alarmAttr.FullTagReference.Substring(0, alarmAttr.FullTagReference.Length - 2)
: alarmAttr.FullTagReference;
_tagToVariableNode.TryGetValue(alarmAttr.FullTagReference, out var sourceVariable);
var sourceNodeId = new NodeId(alarmNodeIdStr, NamespaceIndex);
var conditionNodeId = new NodeId(alarmNodeIdStr + ".Condition", NamespaceIndex);
var condition = new AlarmConditionState(sourceVariable);
condition.Create(SystemContext, conditionNodeId,
new QualifiedName(alarmAttr.AttributeName + "Alarm", NamespaceIndex),
new LocalizedText("en", alarmAttr.AttributeName + " Alarm"),
true);
condition.SourceNode.Value = sourceNodeId;
condition.SourceName.Value = alarmAttr.AttributeName;
condition.ConditionName.Value = alarmAttr.AttributeName;
condition.AutoReportStateChanges = true;
condition.SetEnableState(SystemContext, true);
condition.SetActiveState(SystemContext, false);
condition.SetAcknowledgedState(SystemContext, true);
condition.SetSeverity(SystemContext, EventSeverity.Medium);
condition.Retain.Value = false;
condition.OnReportEvent = (context, n, e) => Server.ReportEvent(context, e);
if (sourceVariable != null)
{
sourceVariable.AddReference(ReferenceTypeIds.HasCondition, false, conditionNodeId);
condition.AddReference(ReferenceTypeIds.HasCondition, true, sourceNodeId);
}
AddPredefinedNode(SystemContext, condition);
var baseTagRef = alarmAttr.FullTagReference.TrimEnd('[', ']');
_alarmInAlarmTags[inAlarmTagRef] = new AlarmInfo
{
SourceTagReference = alarmAttr.FullTagReference,
SourceNodeId = sourceNodeId,
SourceName = alarmAttr.AttributeName,
ConditionNode = condition,
PriorityTagReference = baseTagRef + ".Priority",
DescAttrNameTagReference = baseTagRef + ".DescAttrName"
};
hasAlarms = true;
}
if (hasAlarms && _nodeMap.TryGetValue(obj.GobjectId, out var objNode))
{
if (objNode is BaseObjectState objState)
objState.EventNotifier = EventNotifiers.SubscribeToEvents;
else if (objNode is FolderState folderState)
folderState.EventNotifier = EventNotifiers.SubscribeToEvents;
}
}
// Subscribe alarm tags for new subtree
foreach (var kvp in _alarmInAlarmTags)
{
// Only subscribe tags that belong to the newly built subtree
var gobjectIds = new HashSet<int>(hierarchy.Select(h => h.GobjectId));
var sourceTagRef = kvp.Value.SourceTagReference;
var ownerAttr = attributes.FirstOrDefault(a => a.FullTagReference == sourceTagRef);
if (ownerAttr == null || !gobjectIds.Contains(ownerAttr.GobjectId))
continue;
foreach (var tag in new[] { kvp.Key, kvp.Value.PriorityTagReference, kvp.Value.DescAttrNameTagReference })
{
if (string.IsNullOrEmpty(tag) || !_tagToVariableNode.ContainsKey(tag))
continue;
try { _mxAccessClient.SubscribeAsync(tag, (_, _) => { }); }
catch { /* ignore */ }
}
}
} }
} }
@@ -559,6 +862,15 @@ namespace ZB.MOM.WW.LmxOpcUa.Host.OpcUa
IsArray = attr.IsArray, IsArray = attr.IsArray,
ArrayDimension = attr.ArrayDimension ArrayDimension = attr.ArrayDimension
}; };
// Track gobject → tag references for incremental sync
if (!_gobjectToTagRefs.TryGetValue(attr.GobjectId, out var tagList))
{
tagList = new List<string>();
_gobjectToTagRefs[attr.GobjectId] = tagList;
}
tagList.Add(attr.FullTagReference);
VariableNodeCount++; VariableNodeCount++;
return variable; return variable;
} }

172
tests/ZB.MOM.WW.LmxOpcUa.Tests/Integration/IncrementalSyncTests.cs Normal file
View File

@@ -0,0 +1,172 @@
using System.Collections.Generic;
using System.Linq;
using System.Threading.Tasks;
using Opc.Ua;
using Shouldly;
using Xunit;
using ZB.MOM.WW.LmxOpcUa.Host.Domain;
using ZB.MOM.WW.LmxOpcUa.Tests.Helpers;
namespace ZB.MOM.WW.LmxOpcUa.Tests.Integration
{
public class IncrementalSyncTests
{
[Fact]
public async Task Sync_AddObject_NewNodeAppears()
{
var fixture = OpcUaServerFixture.WithFakeMxAccessClient();
await fixture.InitializeAsync();
try
{
using var client = new OpcUaTestClient();
await client.ConnectAsync(fixture.EndpointUrl);
// Verify initial state
var children = await client.BrowseAsync(client.MakeNodeId("TestArea"));
children.Select(c => c.Name).ShouldContain("TestMachine_001");
children.Select(c => c.Name).ShouldNotContain("NewObject");
// Add a new object
fixture.GalaxyRepository!.Hierarchy.Add(new GalaxyObjectInfo
{
GobjectId = 100, TagName = "NewObject_001", ContainedName = "NewObject",
BrowseName = "NewObject", ParentGobjectId = 2, IsArea = false
});
fixture.GalaxyRepository.Attributes.Add(new GalaxyAttributeInfo
{
GobjectId = 100, TagName = "NewObject_001", AttributeName = "Status",
FullTagReference = "NewObject_001.Status", MxDataType = 5
});
fixture.Service.TriggerRebuild();
await Task.Delay(1000);
// Reconnect in case session was disrupted during rebuild
using var client2 = new OpcUaTestClient();
await client2.ConnectAsync(fixture.EndpointUrl);
// New object should appear when browsing parent
children = await client2.BrowseAsync(client2.MakeNodeId("TestArea"));
children.Select(c => c.Name).ShouldContain("NewObject",
$"Browse returned: [{string.Join(", ", children.Select(c => c.Name))}]");
// Original object should still be there
children.Select(c => c.Name).ShouldContain("TestMachine_001");
}
finally { await fixture.DisposeAsync(); }
}
[Fact]
public async Task Sync_RemoveObject_NodeDisappears()
{
var fixture = OpcUaServerFixture.WithFakeMxAccessClient();
await fixture.InitializeAsync();
try
{
using var client = new OpcUaTestClient();
await client.ConnectAsync(fixture.EndpointUrl);
// Verify MESReceiver exists
var children = await client.BrowseAsync(client.MakeNodeId("TestMachine_001"));
children.Select(c => c.Name).ShouldContain("MESReceiver");
// Remove MESReceiver (gobject_id 5) and its attributes
fixture.GalaxyRepository!.Hierarchy.RemoveAll(h => h.GobjectId == 5);
fixture.GalaxyRepository.Attributes.RemoveAll(a => a.GobjectId == 5);
fixture.Service.TriggerRebuild();
await Task.Delay(500);
// MESReceiver should be gone
children = await client.BrowseAsync(client.MakeNodeId("TestMachine_001"));
children.Select(c => c.Name).ShouldNotContain("MESReceiver");
// DelmiaReceiver should still be there
children.Select(c => c.Name).ShouldContain("DelmiaReceiver");
}
finally { await fixture.DisposeAsync(); }
}
[Fact]
public async Task Sync_AddAttribute_NewVariableAppears()
{
var fixture = OpcUaServerFixture.WithFakeMxAccessClient();
await fixture.InitializeAsync();
try
{
using var client = new OpcUaTestClient();
await client.ConnectAsync(fixture.EndpointUrl);
// Add a new attribute to TestMachine_001
fixture.GalaxyRepository!.Attributes.Add(new GalaxyAttributeInfo
{
GobjectId = 3, TagName = "TestMachine_001", AttributeName = "NewAttr",
FullTagReference = "TestMachine_001.NewAttr", MxDataType = 5
});
fixture.Service.TriggerRebuild();
await Task.Delay(500);
var children = await client.BrowseAsync(client.MakeNodeId("TestMachine_001"));
children.Select(c => c.Name).ShouldContain("NewAttr");
children.Select(c => c.Name).ShouldContain("MachineID");
}
finally { await fixture.DisposeAsync(); }
}
[Fact]
public async Task Sync_UnchangedObject_SubscriptionSurvives()
{
var mxClient = new FakeMxAccessClient();
var fixture = OpcUaServerFixture.WithFakeMxAccessClient(mxClient: mxClient);
await fixture.InitializeAsync();
try
{
using var client = new OpcUaTestClient();
await client.ConnectAsync(fixture.EndpointUrl);
// Subscribe to MachineID on TestMachine_001
var nodeId = client.MakeNodeId("TestMachine_001.MachineID");
var (sub, item) = await client.SubscribeAsync(nodeId, 250);
await Task.Delay(500);
// Modify a DIFFERENT object (MESReceiver) — TestMachine_001 should be unaffected
var mesAttr = fixture.GalaxyRepository!.Attributes
.First(a => a.GobjectId == 5 && a.AttributeName == "MoveInBatchID");
mesAttr.SecurityClassification = 2; // change something
fixture.Service.TriggerRebuild();
await Task.Delay(500);
// Push a value change through MXAccess — subscription should still deliver
mxClient.SimulateDataChange("TestMachine_001.MachineID", Vtq.Good("UPDATED"));
await Task.Delay(1000);
var lastValue = (item.LastValue as MonitoredItemNotification)?.Value?.Value;
lastValue.ShouldBe("UPDATED");
}
finally { await fixture.DisposeAsync(); }
}
[Fact]
public async Task Sync_NoChanges_NothingHappens()
{
var fixture = OpcUaServerFixture.WithFakeMxAccessClient();
await fixture.InitializeAsync();
try
{
using var client = new OpcUaTestClient();
await client.ConnectAsync(fixture.EndpointUrl);
// Trigger rebuild with no changes
fixture.Service.TriggerRebuild();
await Task.Delay(500);
// Everything should still work
var children = await client.BrowseAsync(client.MakeNodeId("TestMachine_001"));
children.Select(c => c.Name).ShouldContain("MachineID");
}
finally { await fixture.DisposeAsync(); }
}
}
}

157
tests/ZB.MOM.WW.LmxOpcUa.Tests/OpcUa/AddressSpaceDiffTests.cs Normal file
View File

@@ -0,0 +1,157 @@
using System.Collections.Generic;
using Shouldly;
using Xunit;
using ZB.MOM.WW.LmxOpcUa.Host.Domain;
using ZB.MOM.WW.LmxOpcUa.Host.OpcUa;
namespace ZB.MOM.WW.LmxOpcUa.Tests.OpcUa
{
public class AddressSpaceDiffTests
{
private static GalaxyObjectInfo Obj(int id, string tag, int parent = 0, bool isArea = false)
=> new GalaxyObjectInfo { GobjectId = id, TagName = tag, BrowseName = tag, ContainedName = tag, ParentGobjectId = parent, IsArea = isArea };
private static GalaxyAttributeInfo Attr(int gobjectId, string name, string tagName = "Obj", int mxDataType = 5)
=> new GalaxyAttributeInfo { GobjectId = gobjectId, AttributeName = name, FullTagReference = $"{tagName}.{name}", MxDataType = mxDataType, TagName = tagName };
[Fact]
public void NoChanges_ReturnsEmptySet()
{
var h = new List<GalaxyObjectInfo> { Obj(1, "A") };
var a = new List<GalaxyAttributeInfo> { Attr(1, "X") };
var changed = AddressSpaceDiff.FindChangedGobjectIds(h, a, h, a);
changed.ShouldBeEmpty();
}
[Fact]
public void AddedObject_Detected()
{
var oldH = new List<GalaxyObjectInfo> { Obj(1, "A") };
var newH = new List<GalaxyObjectInfo> { Obj(1, "A"), Obj(2, "B") };
var a = new List<GalaxyAttributeInfo>();
var changed = AddressSpaceDiff.FindChangedGobjectIds(oldH, a, newH, a);
changed.ShouldContain(2);
changed.ShouldNotContain(1);
}
[Fact]
public void RemovedObject_Detected()
{
var oldH = new List<GalaxyObjectInfo> { Obj(1, "A"), Obj(2, "B") };
var newH = new List<GalaxyObjectInfo> { Obj(1, "A") };
var a = new List<GalaxyAttributeInfo>();
var changed = AddressSpaceDiff.FindChangedGobjectIds(oldH, a, newH, a);
changed.ShouldContain(2);
changed.ShouldNotContain(1);
}
[Fact]
public void ModifiedObject_BrowseNameChange_Detected()
{
var oldH = new List<GalaxyObjectInfo> { Obj(1, "A") };
var newH = new List<GalaxyObjectInfo> { new GalaxyObjectInfo { GobjectId = 1, TagName = "A", BrowseName = "A_Renamed", ContainedName = "A" } };
var a = new List<GalaxyAttributeInfo>();
var changed = AddressSpaceDiff.FindChangedGobjectIds(oldH, a, newH, a);
changed.ShouldContain(1);
}
[Fact]
public void ModifiedObject_ParentChange_Detected()
{
var oldH = new List<GalaxyObjectInfo> { Obj(1, "A"), Obj(2, "B", parent: 1) };
var newH = new List<GalaxyObjectInfo> { Obj(1, "A"), new GalaxyObjectInfo { GobjectId = 2, TagName = "B", BrowseName = "B", ContainedName = "B", ParentGobjectId = 0 } };
var a = new List<GalaxyAttributeInfo>();
var changed = AddressSpaceDiff.FindChangedGobjectIds(oldH, a, newH, a);
changed.ShouldContain(2);
}
[Fact]
public void AttributeAdded_Detected()
{
var h = new List<GalaxyObjectInfo> { Obj(1, "A") };
var oldA = new List<GalaxyAttributeInfo> { Attr(1, "X") };
var newA = new List<GalaxyAttributeInfo> { Attr(1, "X"), Attr(1, "Y") };
var changed = AddressSpaceDiff.FindChangedGobjectIds(h, oldA, h, newA);
changed.ShouldContain(1);
}
[Fact]
public void AttributeRemoved_Detected()
{
var h = new List<GalaxyObjectInfo> { Obj(1, "A") };
var oldA = new List<GalaxyAttributeInfo> { Attr(1, "X"), Attr(1, "Y") };
var newA = new List<GalaxyAttributeInfo> { Attr(1, "X") };
var changed = AddressSpaceDiff.FindChangedGobjectIds(h, oldA, h, newA);
changed.ShouldContain(1);
}
[Fact]
public void AttributeFieldChange_Detected()
{
var h = new List<GalaxyObjectInfo> { Obj(1, "A") };
var oldA = new List<GalaxyAttributeInfo> { Attr(1, "X", mxDataType: 5) };
var newA = new List<GalaxyAttributeInfo> { Attr(1, "X", mxDataType: 2) };
var changed = AddressSpaceDiff.FindChangedGobjectIds(h, oldA, h, newA);
changed.ShouldContain(1);
}
[Fact]
public void AttributeSecurityChange_Detected()
{
var h = new List<GalaxyObjectInfo> { Obj(1, "A") };
var oldA = new List<GalaxyAttributeInfo> { new GalaxyAttributeInfo { GobjectId = 1, AttributeName = "X", FullTagReference = "A.X", SecurityClassification = 1 } };
var newA = new List<GalaxyAttributeInfo> { new GalaxyAttributeInfo { GobjectId = 1, AttributeName = "X", FullTagReference = "A.X", SecurityClassification = 2 } };
var changed = AddressSpaceDiff.FindChangedGobjectIds(h, oldA, h, newA);
changed.ShouldContain(1);
}
[Fact]
public void ExpandToSubtrees_IncludesChildren()
{
var h = new List<GalaxyObjectInfo>
{
Obj(1, "Root"),
Obj(2, "Child", parent: 1),
Obj(3, "Grandchild", parent: 2),
Obj(4, "Sibling", parent: 1),
Obj(5, "Unrelated")
};
var changed = new HashSet<int> { 1 };
var expanded = AddressSpaceDiff.ExpandToSubtrees(changed, h);
expanded.ShouldContain(1);
expanded.ShouldContain(2);
expanded.ShouldContain(3);
expanded.ShouldContain(4);
expanded.ShouldNotContain(5);
}
[Fact]
public void ExpandToSubtrees_LeafNode_NoExpansion()
{
var h = new List<GalaxyObjectInfo>
{
Obj(1, "Root"),
Obj(2, "Child", parent: 1),
Obj(3, "Sibling", parent: 1)
};
var changed = new HashSet<int> { 2 };
var expanded = AddressSpaceDiff.ExpandToSubtrees(changed, h);
expanded.ShouldContain(2);
expanded.ShouldNotContain(1);
expanded.ShouldNotContain(3);
}
}
}