dohertj2/lmxopcua
1
0
Fork 0
Code Issues 14 Pull Requests Actions Packages Projects Releases Wiki Activity

fix(focas): serialize per-device connect so concurrent loops share one session; docs: signed PMC Byte in the #259 override recipe + Makino live notes

Browse Source 
The hw2 read-only Makino gate (Z-34184, 31i-B) logged the fixed-tree bootstrap failing with
ObjectDisposedException: FocasWireClient. EnsureConnectedAsync was unserialized: the alarm poll
loop and the fixed-tree loop both reached it right after init, the second saw the first's
still-connecting client as not connected, disposed it and opened a second session, and the first
caller's catch then disposed the second caller's client (and recorded a spurious backoff failure).
A per-device ConnectGate now serializes the connect with a re-check after acquire, and the failure
path disposes only its own client. FocasConcurrentConnectTests goes red without the gate.

Docs: a Byte PMC tag is signed, so G12 at 100% feed override reads -101 on the Makino and the
recipe's 255 - value gave 356; recipe now masks. Adds the live statinfo/alarm-watch observations.

Claude-Session: https://claude.ai/code/session_01Ef2ozxS7QPanGT8uoATbTe
This commit is contained in:
Joseph Doherty
2026-10-08 21:16:31 -04:00
parent e7dd742541
commit c940349fe8
3 changed files with 172 additions and 40 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/drivers/FOCAS.md
+24 -4
View File
@@ -244,6 +244,13 @@ control's raw codes, published verbatim as Int32 so a client sees exactly what t
| `Edit`, `TmMode` | program-edit state; T/M mode |
Author one the same way as any fixed-tree tag — `FIXED:Status/Run` (Int32), `FIXED:Status/RunText` (String).
Live (hw2 gate, 2026-10-08, Makino Z-34184 31i-B, idle): `cnc_statinfo` answers a 14-byte primary block and read
steadily `Auto=3 Run=0 (RESET) Motion=0 Mstb=0 Emergency=0 Alarm=0 Edit=0 TmMode=0` over 6 minutes, with an empty
`cnc_rdalmmsg2` list — consistent. ⚠️ `Auto=3` (EDIT on the table above) disagrees with `OperationMode/ModeText`,
which read `AUTO` (`cnc_rdopmode` 0x0057 = 1) throughout; ignitionoee independently found 0x0057 toggling 1↔2 with
no explained meaning on the same machine, so treat `OperationMode/` as unconfirmed on the 31i-B and confirm `Auto`
against the panel before relying on either.
Like `Program/`, the subtree is gated on a bootstrap probe: a control that refuses `cnc_statinfo` gets no
`Status/` nodes and the rest of the tree is unaffected. Reads are served from the poll snapshot, so they
issue no wire traffic and go `BadCommunicationError` once the device has stopped answering.
@@ -317,12 +324,19 @@ FANUC signal map:
| Signal | PMC address | Notes |
|---|---|---|
| Feed override `*FV0-*FV7` | `G12` (Byte) | **Inverted**: the byte holds the one's complement. Percent = `255 - value` for a binary-coded switch (`255` = 0%, `155` = 100%); gray-coded/ladder-converted panels differ |
| Feed override `*FV0-*FV7` | `G12` (Byte) | **Inverted**: the byte holds the one's complement. Percent = `~value & 0xFF` for a binary-coded switch (raw `0xFF` = 0%, raw `0x9B` = 100%); gray-coded/ladder-converted panels differ |
| Rapid override `ROV1`, `ROV2` | `G14.0`, `G14.1` (Bit) | Two bits select 100% / 50% / 25% / F0 |
| Spindle override `SOV0-SOV7` | `G30` (Byte) | Percent, usually a direct 0-150 value |
| Spindle override `SOV0-SOV7` | `G30` (Byte) | Percent, usually a direct 0-150 value — mask it (`value & 0xFF`): 128-150 % read negative |
Author them as normal PMC tags (`G12`, `G14.0`, `G30` — G is accepted on the 30i family and the 16i). To publish a
percentage rather than the raw byte, derive it with a virtual tag (`255 - ctx.GetTag("{{equip}}/FeedOverrideRaw")`).
Author them as normal PMC tags (`G12`, `G14.0`, `G30` — G is accepted on the 30i family and the 16i). ⚠️ A `Byte`
tag is **signed** (`sbyte`, published as Int32), so a raw byte above `0x7F` reads negative: on the Makino Z-34184 at
100 % feed override `G12` reads **`-101`** (raw `0x9B` = 155). `255 - value` would therefore give 356, not 100 — mask
before inverting. To publish a percentage rather than the raw byte, derive it with a virtual tag
(`~(int)ctx.GetTag("{{equip}}/FeedOverrideRaw").Value & 0xFF` for `G12`; `(int)… & 0xFF` for `G30`).
Read-only live check (hw2 gate, 2026-10-08, Makino Z-34184 idle at the panel's 100 % positions): `G12 = -101`
(→ 100 %), `G14.0 = G14.1 = false` (→ rapid 100 %), `G30 = 100` (→ 100 %). Plausible against the standard map;
not yet cross-checked against the switches at other positions.
⚠️ **The addresses above are the FANUC *standard* signal assignment; the meaning is defined by the machine
builder's ladder.** Confirm each against the machine's own PMC signal list before trusting the number, and
@@ -489,6 +503,12 @@ an `ALARM:` address without the `alarm` object (not a condition, and no value to
on a non-`ALARM:` address (a condition nothing can raise). Reading an `ALARM:` tag's RawPath through the driver
returns `BadNotSupported`; writing it, `BadNotWritable`.
**Live (passive, read-only) — hw2 gate 2026-10-08, Makino Z-34184.** `ALARM:*`, `ALARM:OTHER`, `ALARM:Servo` and
`ALARM:T6` subscribed for 6 minutes at a 2 s poll: the control answered `cnc_rdalmmsg2` `rc=0` with an **empty**
(0-byte) payload every time, no condition event of any kind was raised (no spurious Raise, no quality churn), and driver
health stayed Healthy. With no alarm active, the 76-vs-80-byte layout and the 30i type codes remain unobserved on
this control.
**Known residuals.** Whether the alarm *list* itself flickers at a CNC reset (the status word does, one poll)
is unobserved; if the live gate shows it, a clear-debounce knob is the follow-up. Real-machine raise/clear is
only observable passively — never induce an alarm on a production control.
src/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS/FocasDriver.cs
+56 -36
View File
@@ -1816,41 +1816,57 @@ public sealed class FocasDriver : IDriver, IReadable, IWritable, ITagDiscovery,
{
if (device.Client is { IsConnected: true } c) return c;
// 05/STAB-8: inside an open backoff window a data-path / per-tick-loop caller fails fast — no
// socket create, no wire connect. The probe loop bypasses this and drives failure/success.
if (!bypassBackoff && !device.Backoff.ShouldAttempt(DateTime.UtcNow))
throw new ConnectionThrottledException(
$"FOCAS connect to {device.Options.HostAddress} throttled — backoff window open after a recent connect failure.");
// Discard the existing client (if any) before creating a new one. A client that is
// non-null but not connected may have been disposed by a HandleRecycle race or a prior
// teardown — retrying ConnectAsync on a disposed FocasWireClient hits ThrowIfDisposed and
// returns a permanent BadCommunicationError with no recovery. Replacing it unconditionally
// ensures EnsureConnectedAsync always works with a fresh, non-disposed instance.
if (device.Client is not null)
{
device.Client.Dispose();
device.Client = null;
}
// Wrap the raw wire client so every operation on the device's single FOCAS/2 socket is
// serialized (request→response on one socket cannot interleave) and time-bounded. Without
// this, the equipment poll, fixed-tree loop, probe, and recycle loop collide on the shared
// socket and a stalled read blocks forever — leaving bound tags at BadWaitingForInitialData.
device.Client = new SynchronizedFocasClient(_clientFactory.Create(), _options.Timeout);
// Serialize the connect (hw2 Makino live gate, 2026-10-08). The alarm poll loop and the fixed-tree loop
// both reach here right after init; unserialized, the second caller saw the first caller's still-
// CONNECTING client as "not connected", disposed it and opened a second session, and the first caller's
// failure path then disposed the second caller's client — two sessions to a CNC with a small handle pool,
// a disposed client handed back, and a spurious backoff failure (observed live as the fixed-tree
// bootstrap failing with ObjectDisposedException: FocasWireClient).
await device.ConnectGate.WaitAsync(ct).ConfigureAwait(false);
try
{
await device.Client.ConnectAsync(device.ParsedAddress, _options.Timeout, ct).ConfigureAwait(false);
// Another caller may have connected while this one waited.
if (device.Client is { IsConnected: true } connected) return connected;
// 05/STAB-8: inside an open backoff window a data-path / per-tick-loop caller fails fast — no
// socket create, no wire connect. The probe loop bypasses this and drives failure/success.
if (!bypassBackoff && !device.Backoff.ShouldAttempt(DateTime.UtcNow))
throw new ConnectionThrottledException(
$"FOCAS connect to {device.Options.HostAddress} throttled — backoff window open after a recent connect failure.");
// Discard the existing client (if any) before creating a new one. A client that is
// non-null but not connected may have been disposed by a HandleRecycle race or a prior
// teardown — retrying ConnectAsync on a disposed FocasWireClient hits ThrowIfDisposed and
// returns a permanent BadCommunicationError with no recovery. Replacing it unconditionally
// ensures EnsureConnectedAsync always works with a fresh, non-disposed instance.
if (device.Client is not null)
{
device.Client.Dispose();
device.Client = null;
}
// Wrap the raw wire client so every operation on the device's single FOCAS/2 socket is
// serialized (request→response on one socket cannot interleave) and time-bounded. Without
// this, the equipment poll, fixed-tree loop, probe, and recycle loop collide on the shared
// socket and a stalled read blocks forever — leaving bound tags at BadWaitingForInitialData.
var client = new SynchronizedFocasClient(_clientFactory.Create(), _options.Timeout);
device.Client = client;
try
{
await client.ConnectAsync(device.ParsedAddress, _options.Timeout, ct).ConfigureAwait(false);
}
catch
{
// Dispose only OUR client — never one another path installed meanwhile (recycle/teardown).
client.Dispose();
if (ReferenceEquals(device.Client, client)) device.Client = null;
device.Backoff.RecordFailure(DateTime.UtcNow); // 05/STAB-8: open the backoff window
throw;
}
device.Backoff.RecordSuccess(); // 05/STAB-8: recovery resets the window immediately
return client;
}
catch
{
device.Client.Dispose();
device.Client = null;
device.Backoff.RecordFailure(DateTime.UtcNow); // 05/STAB-8: open the backoff window
throw;
}
device.Backoff.RecordSuccess(); // 05/STAB-8: recovery resets the window immediately
return device.Client;
finally { device.ConnectGate.Release(); }
}
/// <summary>Disposes the driver and releases all resources synchronously.</summary>
@@ -1919,13 +1935,17 @@ public sealed class FocasDriver : IDriver, IReadable, IWritable, ITagDiscovery,
/// Per-device connect-attempt throttle (05/STAB-8). A dead CNC fails fast inside the
/// backoff window instead of paying a full two-socket reconnect on every tick of each of
/// the read / write / probe / fixed-tree / recycle loops; the probe loop bypasses it and a
/// successful connect resets it. FOCAS's <c>EnsureConnectedAsync</c> is not gate-serialized
/// (the loops already race the client swap by design), so this is best-effort — atomic
/// enough on 64-bit for the int/DateTime state, and a benign extra/skipped attempt is the
/// worst outcome of a race.
/// successful connect resets it. The connect itself is serialized by
/// <see cref="ConnectGate"/>; the backoff is consulted inside it.
/// </summary>
public ConnectionBackoff Backoff { get; } = new(TimeSpan.FromSeconds(1), TimeSpan.FromSeconds(30));
/// <summary>
/// Serializes <c>EnsureConnectedAsync</c> per device, so concurrent loops (alarm poll, fixed tree, data
/// path) share ONE connect instead of racing two sessions and disposing each other's client.
/// </summary>
public SemaphoreSlim ConnectGate { get; } = new(1, 1);
/// <summary>Gets the lock object for probe synchronization.</summary>
public object ProbeLock { get; } = new();
/// <summary>Gets or sets the current host connectivity state.</summary>
tests/Drivers/ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests/FocasConcurrentConnectTests.cs
+92
View File
@@ -0,0 +1,92 @@
using Shouldly;
using Xunit;
using ZB.MOM.WW.OtOpcUa.Core.Abstractions;
using ZB.MOM.WW.OtOpcUa.Driver.FOCAS;
namespace ZB.MOM.WW.OtOpcUa.Driver.FOCAS.Tests;
/// <summary>
/// Two callers reaching <c>EnsureConnectedAsync</c> while the device has no live session (the alarm poll loop
/// and the fixed-tree loop both start right after init) must open ONE session between them. Found live on the
/// Makino Z-34184 (hw2 gate, 2026-10-08): the fixed-tree bootstrap failed with
/// <c>ObjectDisposedException: FocasWireClient</c>, because the second caller saw the first caller's
/// still-connecting client as "not connected", disposed it, and opened a second one — and the first caller's
/// failure path then disposed the SECOND caller's client. Two TCP sessions to a production CNC with a small
/// handle pool, a disposed client handed back to a caller, and a spurious connect-backoff failure.
/// </summary>
[Trait("Category", "Unit")]
public sealed class FocasConcurrentConnectTests
{
/// <summary>A fake whose connect parks until released, and which — like the real wire client — refuses to
/// finish connecting once disposed.</summary>
private sealed class GatedConnectClient(Task release) : FakeFocasClient
{
private bool _disposed;
public override async Task ConnectAsync(FocasHostAddress address, TimeSpan timeout, CancellationToken ct)
{
await release.WaitAsync(ct);
if (_disposed) throw new ObjectDisposedException("FocasWireClient");
await base.ConnectAsync(address, timeout, ct);
}
public override void Dispose()
{
_disposed = true;
base.Dispose();
}
}
/// <summary>Concurrent first reads share one connect; nobody's client is disposed under them.</summary>
[Fact]
public async Task Concurrent_first_reads_open_one_session_and_both_succeed()
{
var release = new TaskCompletionSource(TaskCreationOptions.RunContinuationsAsynchronously);
var factory = new FakeFocasClientFactory { Customise = () => new GatedConnectClient(release.Task) };
await using var drv = new FocasDriver(new FocasDriverOptions
{
Devices = [new FocasDeviceOptions("focas://10.0.0.5:8193")],
RawTags = FocasRawTags.Entries([
new FocasTagDefinition("A", "focas://10.0.0.5:8193", "R100", FocasDataType.Byte),
new FocasTagDefinition("B", "focas://10.0.0.5:8193", "R200", FocasDataType.Byte)]),
Probe = new FocasProbeOptions { Enabled = false },
Timeout = TimeSpan.FromSeconds(5),
}, "drv-1", factory);
await drv.InitializeAsync("{}", CancellationToken.None);
var first = drv.ReadAsync(["A"], CancellationToken.None);
var second = drv.ReadAsync(["B"], CancellationToken.None);
// Give both callers time to reach the connect path before the (single) connect is allowed to finish.
await Task.Delay(200);
release.SetResult();
var results = await Task.WhenAll(first, second);
factory.Clients.Count.ShouldBe(1, "two concurrent callers must share one FOCAS session, not race two");
factory.Clients[0].DisposeCount.ShouldBe(0, "no caller may dispose a client another caller is using");
results[0].Single().StatusCode.ShouldBe(FocasStatusMapper.Good);
results[1].Single().StatusCode.ShouldBe(FocasStatusMapper.Good);
}
/// <summary>A failed connect does not poison a later one — the gate is released on failure.</summary>
[Fact]
public async Task A_failed_connect_releases_the_gate_for_the_next_caller()
{
var factory = new FakeFocasClientFactory();
var attempt = 0;
factory.Customise = () => new FakeFocasClient { ThrowOnConnect = Interlocked.Increment(ref attempt) == 1 };
await using var drv = new FocasDriver(new FocasDriverOptions
{
Devices = [new FocasDeviceOptions("focas://10.0.0.5:8193")],
RawTags = FocasRawTags.Entries([new FocasTagDefinition("A", "focas://10.0.0.5:8193", "R100", FocasDataType.Byte)]),
Probe = new FocasProbeOptions { Enabled = false },
}, "drv-1", factory);
await drv.InitializeAsync("{}", CancellationToken.None);
(await drv.ReadAsync(["A"], CancellationToken.None)).Single().StatusCode
.ShouldNotBe(FocasStatusMapper.Good);
// Past the 1 s initial connect backoff window, the next caller must be able to connect.
await Task.Delay(TimeSpan.FromMilliseconds(1100));
var read = drv.ReadAsync(["A"], CancellationToken.None);
(await read.WaitAsync(TimeSpan.FromSeconds(5))).Single().StatusCode.ShouldBe(FocasStatusMapper.Good);
}
}