Joseph Doherty
cbb882293b
The tier system was documented, operator-authorable, and inert. Deleted rather than activated,
because its premise is gone rather than merely unused.
"Tier C" meant a driver running out-of-process behind an IDriverSupervisor that could restart its
Host without tearing down the OPC UA session. No such process exists anywhere: Galaxy reaches
MXAccess over gRPC to the external mxaccessgw sidecar (PR 7.2 retired the in-process
Galaxy.Host/Proxy/Shared projects) and FOCAS has run in-process since its managed wire client
landed 2026-04-24. Consistently, IDriverSupervisor had ZERO implementations and there was nothing
for one to implement against.
The issue understated the inertness. It says the Tier-C-only protections never engaged, implying
the Tier A/B parts did. They did not: nothing constructs MemoryTracking, MemoryRecycle or
ScheduledRecycleScheduler outside their own unit tests, so the whole Core/Stability recycle layer
was dead — meaning option (a), "pass real tiers", was never a flag flip. It would have meant
writing the wiring that never existed AND arming it.
Deleted: MemoryTracking, MemoryRecycle, ScheduledRecycleScheduler, IDriverSupervisor, the
vestigial DriverTypeRegistry (referenced only by its own tests), and the RecycleIntervalSeconds
knob — which the AdminUI let an operator author and the parser validated while it configured
nothing.
DriverTier itself SURVIVES and is load-bearing: DriverResilienceOptions.GetTierDefaults supplies
the real per-capability timeout/retry/breaker policies via DriverFactoryRegistry.GetTier and
DriverCapabilityInvokerFactory. Only the isolation-and-recycle layer above it is gone.
Deliberately NOT deleted:
- WedgeDetector came along in the same directory and is equally dead in production, but it is
tier-agnostic and is not recycle machinery — it only shares the folder. Restored rather than
swept up in a decision that was not about it.
- IDriver.GetMemoryFootprint() and FlushOptionalCachesAsync() lose their only consumer here.
Removing them touches all 12 drivers and every test stub, so they are documented as
consumerless and filed as #525 instead of buried in this diff.
Compatibility: a deployed ResilienceConfig blob still carrying "recycleIntervalSeconds" parses
cleanly (unknown keys are ignored — guarded by a new test, because a blob that suddenly failed to
parse would fall back to tier defaults and silently discard the operator's real overrides), and
the AdminUI's preserve-unknown-keys bag keeps the key rather than rewriting stored config on an
unrelated edit.
The 01/U-6 knob-inertness guard carried an explicit carve-out admitting RecycleIntervalSeconds was
dormant and out of scope; that carve-out is now gone, so the expected set is literally what the
test's name claims.
Note: Host.IntegrationTests has 2 failures (DriverProbeRegistrationTests.is_idempotent,
PrimaryGateFailoverTests) — verified pre-existing by reproducing both on clean master dc9d947b.
Claude-Session: https://claude.ai/code/session_015p7wGqy3YpZNCpDzTpGMKo
120 lines
8.1 KiB
C#
120 lines
8.1 KiB
C#
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
|
||
|
||
namespace ZB.MOM.WW.OtOpcUa.Core.Resilience;
|
||
|
||
/// <summary>
|
||
/// Per-tier × per-capability resilience policy configuration for a driver instance.
|
||
/// Bound from <c>DriverInstance.ResilienceConfig</c> JSON (nullable column; null = tier defaults).
|
||
/// Per <c>docs/v2/plan.md</c>.
|
||
/// </summary>
|
||
public sealed record DriverResilienceOptions
|
||
{
|
||
/// <summary>Tier the owning driver type is registered as; drives the default map.</summary>
|
||
public required DriverTier Tier { get; init; }
|
||
|
||
/// <summary>
|
||
/// Per-capability policy overrides. Capabilities absent from this map fall back to
|
||
/// <see cref="GetTierDefaults(DriverTier)"/> for the configured <see cref="Tier"/>.
|
||
/// </summary>
|
||
public IReadOnlyDictionary<DriverCapability, CapabilityPolicy> CapabilityPolicies { get; init; }
|
||
= new Dictionary<DriverCapability, CapabilityPolicy>();
|
||
|
||
// RecycleIntervalSeconds is GONE (Gitea #522). It configured a scheduled recycle of a Tier C
|
||
// driver's out-of-process Host — a process that no longer exists anywhere: Galaxy reaches MXAccess
|
||
// over gRPC to the external mxaccessgw sidecar (PR 7.2 retired the in-process Host/Proxy/Shared
|
||
// projects) and FOCAS has run in-process since its managed wire client landed 2026-04-24. The
|
||
// scheduler it fed was constructed only in tests, and IDriverSupervisor — the thing that would
|
||
// have performed the recycle — had zero implementations. The parser nevertheless validated the
|
||
// knob, so an operator could author a recycle interval through the AdminUI and get nothing.
|
||
//
|
||
// The JSON key is not an error if present: DriverResilienceOptionsParser ignores unknown keys, so
|
||
// a ResilienceConfig blob still carrying "recycleIntervalSeconds" parses cleanly and the value is
|
||
// simply dropped, which is what it already did in effect.
|
||
|
||
/// <summary>
|
||
/// Look up the effective policy for a capability, falling back to tier defaults when no
|
||
/// override is configured. Never returns null.
|
||
/// </summary>
|
||
/// <param name="capability">The driver capability to resolve the policy for.</param>
|
||
/// <returns>The effective CapabilityPolicy for the specified capability.</returns>
|
||
/// <exception cref="KeyNotFoundException">
|
||
/// Thrown when neither the override map nor the tier defaults carry an entry for the
|
||
/// requested capability. The <c>TierDefaults_Cover_EveryCapability</c> invariant test
|
||
/// in <c>DriverResilienceOptionsTests</c> guarantees every defined enum value is present
|
||
/// in each tier's table, so this only fires when a caller passes an out-of-range value
|
||
/// or someone adds a <see cref="DriverCapability"/> member without updating
|
||
/// <see cref="GetTierDefaults"/>. The message names the missing capability and tier.
|
||
/// </exception>
|
||
public CapabilityPolicy Resolve(DriverCapability capability)
|
||
{
|
||
if (CapabilityPolicies.TryGetValue(capability, out var policy))
|
||
return policy;
|
||
|
||
var defaults = GetTierDefaults(Tier);
|
||
if (defaults.TryGetValue(capability, out var fallback))
|
||
return fallback;
|
||
|
||
throw new KeyNotFoundException(
|
||
$"No policy defined for capability '{capability}' under tier '{Tier}'. " +
|
||
$"This indicates a {nameof(DriverCapability)} enum value missing from {nameof(GetTierDefaults)} — " +
|
||
"add the capability to every tier's default table.");
|
||
}
|
||
|
||
/// <summary>
|
||
/// Per-tier per-capability default policy table, per the Phase 6.1
|
||
/// Stream A.2 specification. The parser enforces no-retry on
|
||
/// <see cref="DriverCapability.Write"/> and <see cref="DriverCapability.AlarmAcknowledge"/>
|
||
/// as an invariant (R2-02/S-8) — a JSON retryCount override on those capabilities is forced back
|
||
/// to 0; per-tag opt-in via <see cref="WriteIdempotentAttribute"/> is the future relaxation.
|
||
/// </summary>
|
||
/// <param name="tier">The driver tier to get defaults for.</param>
|
||
/// <returns>The default policy dictionary for the specified tier.</returns>
|
||
public static IReadOnlyDictionary<DriverCapability, CapabilityPolicy> GetTierDefaults(DriverTier tier) =>
|
||
tier switch
|
||
{
|
||
DriverTier.A => new Dictionary<DriverCapability, CapabilityPolicy>
|
||
{
|
||
[DriverCapability.Read] = new(TimeoutSeconds: 2, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Write] = new(TimeoutSeconds: 2, RetryCount: 0, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Discover] = new(TimeoutSeconds: 30, RetryCount: 2, BreakerFailureThreshold: 3),
|
||
[DriverCapability.Subscribe] = new(TimeoutSeconds: 5, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Probe] = new(TimeoutSeconds: 2, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.AlarmSubscribe] = new(TimeoutSeconds: 5, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.AlarmAcknowledge] = new(TimeoutSeconds: 5, RetryCount: 0, BreakerFailureThreshold: 5),
|
||
[DriverCapability.HistoryRead] = new(TimeoutSeconds: 30, RetryCount: 2, BreakerFailureThreshold: 5),
|
||
},
|
||
DriverTier.B => new Dictionary<DriverCapability, CapabilityPolicy>
|
||
{
|
||
[DriverCapability.Read] = new(TimeoutSeconds: 4, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Write] = new(TimeoutSeconds: 4, RetryCount: 0, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Discover] = new(TimeoutSeconds: 60, RetryCount: 2, BreakerFailureThreshold: 3),
|
||
[DriverCapability.Subscribe] = new(TimeoutSeconds: 8, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.Probe] = new(TimeoutSeconds: 4, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.AlarmSubscribe] = new(TimeoutSeconds: 8, RetryCount: 3, BreakerFailureThreshold: 5),
|
||
[DriverCapability.AlarmAcknowledge] = new(TimeoutSeconds: 8, RetryCount: 0, BreakerFailureThreshold: 5),
|
||
[DriverCapability.HistoryRead] = new(TimeoutSeconds: 60, RetryCount: 2, BreakerFailureThreshold: 5),
|
||
},
|
||
DriverTier.C => new Dictionary<DriverCapability, CapabilityPolicy>
|
||
{
|
||
[DriverCapability.Read] = new(TimeoutSeconds: 10, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
[DriverCapability.Write] = new(TimeoutSeconds: 10, RetryCount: 0, BreakerFailureThreshold: 0),
|
||
[DriverCapability.Discover] = new(TimeoutSeconds: 120, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
[DriverCapability.Subscribe] = new(TimeoutSeconds: 15, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
[DriverCapability.Probe] = new(TimeoutSeconds: 10, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
[DriverCapability.AlarmSubscribe] = new(TimeoutSeconds: 15, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
[DriverCapability.AlarmAcknowledge] = new(TimeoutSeconds: 15, RetryCount: 0, BreakerFailureThreshold: 0),
|
||
[DriverCapability.HistoryRead] = new(TimeoutSeconds: 120, RetryCount: 1, BreakerFailureThreshold: 0),
|
||
},
|
||
_ => throw new ArgumentOutOfRangeException(nameof(tier), tier, $"No default policy table defined for tier {tier}."),
|
||
};
|
||
}
|
||
|
||
/// <summary>Policy for one capability on one driver instance.</summary>
|
||
/// <param name="TimeoutSeconds">Per-call timeout (wraps the inner Polly execution).</param>
|
||
/// <param name="RetryCount">Number of retry attempts after the first failure; zero = no retry.</param>
|
||
/// <param name="BreakerFailureThreshold">
|
||
/// Consecutive-failure count that opens the circuit breaker; zero = no breaker
|
||
/// (Tier C uses the supervisor's process-level breaker instead).
|
||
/// </param>
|
||
public sealed record CapabilityPolicy(int TimeoutSeconds, int RetryCount, int BreakerFailureThreshold);
|