9 Commits

Author SHA1 Message Date
Joseph Doherty 54bcf8a5e0 docs(c2): record 2020-vs-2023R2 positive control for the event-read server gate
Adds the positive control that the prior C2 evidence lacked. The SAME native WCF
event-read client returns real events (5) from a local AVEVA Historian 2020 but 0
from the 2023 R2 server over the identical sequence and window, while both boxes
hold tens of thousands of events in SQL — isolating the zero-rows to the 2023 R2
server, not the client, protocol, or serializers.

- wcf-event-read-spike-results.md: new "2026-06-26 positive control" section
  (2020 vs 2023 R2 A/B from one WCF client; stock-2020-client version-self-block
  caveat; stock-2023R2 gRPC cross-check).
- grpc-event-query-capture.md: re-control note — the 2026-06-22 stock 50-row
  capture did NOT reproduce; the stock 2023 R2 client now also returns 0 rows.
- HistorianGrpcIntegrationTests: correct the stale "capture-gated, NOT
  server-gated" comment to the server-gate conclusion backed by the controls.

Sanitized throughout (counts, native return codes, buffer lengths only).

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 11:08:20 -04:00
dohertj2 f0a1b04b34 Merge pull request 'feat(wcf): C2 spike + ConnectViaAddress/connmode — WCF transport viable, rows server-gated' (#1) from feat/c2-wcf-event-spike into main 2026-06-26 06:48:01 -04:00
Joseph Doherty f2297315b9 docs(c2): correct the WCF finding — transport+auth viable, row-retrieval server-gated
Overturns the earlier wrong "WCF not served on 2023 R2" conclusion (that was a
test error: wrong port/transport for the reverse tunnel). Corrected: the cert
(TLS) transport + NegotiateAuthentication auth reach the 2023 R2 historian
cross-platform; the 0x501 event connection mode makes CM_EVENT RegisterTags
succeed; yet StartEventQuery returns a 0-row buffer + long-polls over a window
that has events. Registration and window ruled out -> the same server-side
per-connection row gate as gRPC. Event reads stay server-gated over BOTH
transports; not client-fixable. Evidence doc rewritten; gRPC + WCF orchestrator
gating messages corrected.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:41:21 -04:00
Joseph Doherty de8d5e91ce feat(wcf): EventReadConnectionModeOverride + cross-platform/bounded C2 spike
Live investigation (direct from a VPN host to the 2023 R2 historian's real WCF
port) showed the certificate transport + NegotiateAuthentication auth work
cross-platform, and that the event-read chain needs the 0x501 event connection
mode for CM_EVENT RegisterTags to succeed (0x402/0x401 fail). Even with
registration succeeding over a window that has events, StartEventQuery returns a
0-row header and long-polls — the same server-side per-connection row gate proven
for gRPC. Adds: EventReadConnectionModeOverride (diagnostic), and spike knobs —
cross-platform cert gate, version-check bypass, per-call timeout, overall budget
with phase-diagnostic dump, connection-mode override.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-26 04:38:58 -04:00
Joseph Doherty 8777c0b816 fix(wcf): set Via via CreateChannel(address, via) — ClientViaBehavior absent in .NET WCF
ClientViaBehavior is a .NET Framework type not present in the System.ServiceModel
client libraries. Use the portable ChannelFactory.CreateChannel(EndpointAddress, Uri)
overload instead, via a CreateChannel helper applied at the history-open and
retrieval-query sites (the critical event path). Fixes the build break in 954b9cc.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:37:38 -04:00
Joseph Doherty 954b9cc9cc feat(wcf): add ConnectViaAddress (WCF Via) for tunneled historian access + wire into C2 spike
When the historian is reached through a port-forward whose local port differs
from the server's real service port, WCF's server-side AddressFilter rejects the
message (To = tunnel port != server port). ConnectViaAddress lets the channel
connect to the tunnel while addressing the SOAP To the real Host/Port endpoint.
Applied in HistorianWcfClientCredentialsHelper.Configure (the critical event
factories already call it). The C2 spike reads HISTORIAN_WCF_EVENT_VIA.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:35:46 -04:00
Joseph Doherty 7992e43908 test(c2): make WCF spike transport-selectable (integrated|certificate) + opt-in verbose
The first live run used the wrong port (32568 direct vs the 42568 WCF tunnel) and
hardcoded RemoteTcpIntegrated; via the tunnel the error advanced from socket-RST
to ProtocolException (binding/security mismatch). Add HISTORIAN_WCF_EVENT_TRANSPORT
(certificate), _DNSID, _ALLOW_UNTRUSTED, and an opt-in _VERBOSE for live binding
diagnosis. Default output stays sanitized; still Windows-only, never fails the suite.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 20:18:23 -04:00
Joseph Doherty 64c9793b91 docs(c2): correct event-read gating messages — WCF not served on 2023 R2
The gRPC ReadEvents throws no longer advise "use the WCF transport for event
reads": that path is moot on 2023 R2 (net.tcp is reset at the framing layer,
live-disproven 2026-06-25). Messages now state event reads are auth-solved but
server-gated over gRPC and have no WCF fallback on 2023 R2, citing the two
evidence docs. WCF orchestrator remarks scoped to legacy 2020 historians; row
layout noted as decoded. String/comment only; throw behavior unchanged.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 17:11:19 -04:00
Joseph Doherty f1c57f7149 docs(c2): record WCF event-read spike live result (RED — transport not served on 2023 R2)
WCF net.tcp (RemoteTcpIntegrated) against the live 2023 R2 historian is reset
at the socket-write/framing layer before any auth — both the event spike and a
basic Probe/ReadRaw throw the identical CommunicationException/SocketException
("forcibly closed by the remote host"). The 2023 R2 box does not serve the
legacy WCF transport; C2's "route via WCF" unblock is moot on this server class.
Sanitized: counts + native return codes + buffer lengths only.

Claude-Session: https://claude.ai/code/session_012SDSQ3AcaXqPcBtDESBRii
2026-06-25 16:58:56 -04:00
9 changed files with 278 additions and 43 deletions
@@ -4,6 +4,16 @@ Captured the stock 2023 R2 client performing a **gRPC event read** that returns
the open item "gRPC event ROW retrieval returns zero rows" (handoff §Current Status item 1). This the open item "gRPC event ROW retrieval returns zero rows" (handoff §Current Status item 1). This
closes the capture-gate: the working request shape is now known. closes the capture-gate: the working request shape is now known.
> **Re-control 2026-06-26 — this 50-row result did NOT reproduce.** Re-running the stock 2023 R2
> client (same `capture-event` harness, native Event connection) against the same server returned
> **0 rows** over 30d / 90d / 365d / 3yr, even though `Runtime.dbo.Events` held ~71.5k events in the
> window (newest minutes old). So the native gRPC event read is **server-gated**, not merely
> request-shape/"capture" gated — the gate also stops the stock client now. Positive control the same
> day: the same native **WCF** event-read client returns real events (5) from a local AVEVA Historian
> 2020 but 0 from this 2023 R2 server (see `wcf-event-read-spike-results.md` §"2026-06-26 — positive
> control"). The 50-row capture below remains valid as the *captured-correct request format*; treat its
> "returns rows" framing as a point-in-time observation that the server no longer honors.
## How it was captured ## How it was captured
`tools/AVEVA.Historian.Grpc2023CaptureHarness` gained a `capture-event` scenario. It loads the `tools/AVEVA.Historian.Grpc2023CaptureHarness` gained a `capture-event` scenario. It loads the
@@ -0,0 +1,105 @@
# WCF event-read spike — live result (2026-06-25/26): transport+auth viable, row-retrieval server-gated
Settles the open question behind **C2** ("event reads over gRPC are gated; the only listed unblock is
*route event reads via WCF*"). The gRPC event-read path is a proven server-side dead-end
(`grpc-event-query-capture.md`: auth fully solved, every client-controllable layer byte-matched to the
stock client, yet the server scopes 0 rows to our connection). This spike resolved the **WCF** leg.
> **Correction to an earlier draft of this doc.** A first pass concluded "the 2023 R2 historian does not
> serve the legacy WCF transport (connection reset at framing)." **That was a test error, not a server
> fact.** It connected to the historian's real WCF port `32568` *directly* and used the Windows-integrated
> transport. In this environment the historian is reached through a **reverse SSH tunnel** (local
> `42568` → historian `32568`), and integrated/Kerberos auth does not work through that tunnel. The
> socket-RST was the tunnel/transport mismatch, not an absent listener. Corrected below.
## What was run
A Windows-only-by-default, env-gated diagnostic (`tests/AVEVA.Historian.Client.Tests/WcfEventReadSpikeTests.cs`)
drives `HistorianWcfEventOrchestrator.ReadEventsAsync` directly. The decisive run was **cross-platform,
direct** (no tunnel): from the VPN-holding host straight to the historian's real WCF endpoint
`net.tcp://<historian>:32568/HistCert`, using the **certificate transport** (`RemoteTcpCertificate`,
TLS, `AllowUntrustedServerCertificate`) and `NegotiateAuthentication` (cross-platform, explicit domain
credentials). The SDK's interface-version gate was bypassed (`VerifyServerInterfaceVersion=false`) —
the 2023 R2 WCF **History interface reports version 13** (this SDK's serializers target 11/12).
## Result — transport+auth viable; row-retrieval server-gated (sanitized)
Progression of the live errors as the addressing/transport was corrected:
| attempt | error |
|---|---|
| direct `:32568`, integrated | `SocketException` "forcibly closed" (wrong port + transport for the tunnel) |
| tunnel `:42568`, integrated | `ProtocolException` at the security UpgradeResponse (integrated can't negotiate through the tunnel) |
| tunnel `:42568`, certificate | reached the WCF dispatcher → `AddressFilter` mismatch (tunnel rewrites the port) |
| **direct `:32568`, certificate, cross-platform** | **past auth** → `ProtocolEvidenceMissingException`: History interface version **13** |
| + `VerifyServerInterfaceVersion=false` | **full chain runs**; query returns a 10-byte **0-row** header, then `GetNext` long-polls |
Connection-mode experiment (certificate transport, direct, version-bypassed, a 1-day window that holds
events), comparing the native OpenConnection mode used for the event-read chain:
| connMode | RegisterTags (RTag2) | EnsureTags (EnsT2) | result buffer | events |
|---|---|---|---|---|
| `0x501` (event) | **0 — success** | 1 (benign-false, as in the 2020 flow) | 10 bytes (0-row header) | **0** |
| `0x401` (write) | 1 (fail) | 1 | 10 bytes | 0 |
| `0x402` (read-only, default) | 1 (fail) | 1 | 10 bytes | 0 |
## Conclusion
1. **WCF transport + auth ARE viable on 2023 R2.** The certificate (TLS) transport negotiates and the
`NegotiateAuthentication` app-level handshake authenticates — **cross-platform** (proven from a
non-Windows VPN host). The earlier "WCF not served" conclusion was wrong. (Integrated/Windows
transport security is not usable through the reverse tunnel — `net.tcp` Kerberos does not tunnel.)
2. **The event-read chain needs the `0x501` event connection mode.** With it, CM_EVENT `RegisterTags`
**succeeds** (it fails on `0x402`/`0x401`). `EnsureTags` returns false, but that is documented as
benign in the 2020 flow that *did* return rows.
3. **Row retrieval is server-gated — same as gRPC.** Even with auth solved and `RegisterTags` succeeding,
over a window that holds events, `StartEventQuery` succeeds but `GetNextEventQueryResultBuffer` returns
a **0-row** header (10 bytes) and long-polls. Registration and window are ruled out as the cause; the
server simply does not scope event rows to a managed connection. This is the **identical** server-side
per-connection retrieval working-set gate proven for gRPC in `grpc-event-query-capture.md`.
**Therefore event reads do not return rows on the 2023 R2 historian over either transport** — gRPC
(retrieval-server-gated) and WCF (transport+auth work, but the same server-side row gate). The only
remaining theoretical unblock is server-side (AVEVA exposing event-row retrieval to a managed
connection) — not client-fixable. **C2 stays closed won't-fix**, for this (corrected) reason.
## SDK additions from this investigation (retained, build-clean, golden where applicable)
- `HistorianClientOptions.ConnectViaAddress` — WCF `Via` (connect to a tunnel/proxy while addressing the
SOAP `To` the real endpoint), so a port-forward whose local port differs from the server's real port
satisfies the server-side WCF AddressFilter.
- `HistorianClientOptions.EventReadConnectionModeOverride` — diagnostic override of the event-read
OpenConnection mode (the `0x501` finding above).
- The C2 spike is now transport-selectable (integrated|certificate), cross-platform for the cert
transport, bounded (per-call timeout + overall budget with a phase-diagnostic dump), and version-gate
bypassable. Output stays sanitized (counts, native return codes, buffer lengths, sha256).
## 2026-06-26 — positive control: same WCF client, 2020 historian vs 2023 R2
The earlier evidence triangulated the gate but lacked a clean *positive* control — proof that the
native event-query path returns rows for **some** historian, so that the 0-row 2023 R2 result can be
attributed to the server rather than to the client/protocol. This run supplies it, A/B against two
historians from the **same** WCF event-read client (`HistorianWcfEventOrchestrator`, whose wire
protocol is byte-replayed from stock 2020 captures), same 365-day window:
| target historian | transport | RegisterTags (RTag2) | result buffer | events |
|---|---|---|---|---|
| **local AVEVA Historian 2020** | WCF, integrated | **0 — success** | terminal after rows | **5** |
| **2023 R2** (the C2 server) | WCF, certificate | (gate, as documented) | 10-byte 0-row header → long-poll | **0** |
SQL ground truth (`Runtime.dbo.Events`) for the same two boxes: the 2020 historian holds ~51.6k events
over the window, the 2023 R2 holds ~71.5k — both populated. So the **identical native WCF event-read
client returns real events from a 2020 historian and zero from the 2023 R2 server**. That isolates the
zero-rows to the 2023 R2 server: not the client, not the protocol, not our serializers.
Notes / honesty caveats:
- The genuinely-**stock** 2020 client (`aahClientManaged.dll` v2020.0406.2652.2, driven by reflection)
could **not** be run end-to-end here: against the local 2020 historian (services patched to build
3383.3) it self-blocks at `StartEventQuery` with `Invalid InterfaceVersion` (242) — a client-side
build/version gate, and the stock client has no version-bypass. Our client (which *does* bypass the
version check and byte-replays the same native sequence) is the faithful proxy that reaches the rows.
- Cross-check on the gRPC leg the same day: the **stock 2023 R2** client (native Event connection, its
own correct event query) returned **0 rows** over 30d/90d/365d/3yr against the 2023 R2 server; the
2026-06-22 "50 rows" stock capture did not reproduce. Same server-gate, both transports, both clients.
- Output sanitized throughout (counts, native return codes, buffer lengths only — no event identity,
host, or credentials).
@@ -34,13 +34,16 @@ namespace AVEVA.Historian.Client.Grpc;
/// (the read path proved the front-door session is sufficient over gRPC). /// (the read path proved the front-door session is sufficient over gRPC).
/// </para> /// </para>
/// <para> /// <para>
/// <b>Live status (2026-06-22):</b> the chain runs end-to-end and <c>StartEventQuery</c> succeeds, but /// <b>Live status — server-gated (settled 2026-06-25):</b> the chain runs end-to-end and
/// <c>GetNextEventQueryResultBuffer</c> <b>long-polls</b> when the query has no rows — it blocks to the /// <c>StartEventQuery</c> succeeds, but <c>GetNextEventQueryResultBuffer</c> <b>long-polls</b> to the
/// call deadline instead of returning the synchronous 5-byte code-85 terminal the 2020 WCF op returns. /// no-data terminal (instead of the synchronous 5-byte code-85 terminal the 2020 WCF op returns); a
/// A poll-deadline expiry is therefore treated as the no-data terminal (see the loop). The idle dev box /// poll-deadline expiry is treated as that terminal (see the loop). This is <b>not</b> an empty-box
/// holds no events, so <b>row-level retrieval is not yet live-verified</b>; verifying parsed rows over /// artifact: the live 2023 R2 server holds tens of thousands of events yet scopes <b>0 rows</b> to a
/// gRPC awaits an event-bearing 2023 R2 server. This is tooled + completes cleanly, NOT proven to /// managed connection. Every client-controllable layer was byte-matched to the stock client that returns
/// return rows. /// rows (see <c>docs/reverse-engineering/grpc-event-query-capture.md</c>) — the gate is a server-internal
/// per-connection retrieval working-set, <b>not client-fixable</b>. The legacy WCF transport is not a
/// fallback on 2023 R2 (<c>docs/reverse-engineering/wcf-event-read-spike-results.md</c>). Tooled +
/// completes cleanly, but proven NOT to return rows over a managed connection.
/// </para> /// </para>
/// </summary> /// </summary>
internal sealed class HistorianGrpcEventOrchestrator internal sealed class HistorianGrpcEventOrchestrator
@@ -101,9 +104,12 @@ internal sealed class HistorianGrpcEventOrchestrator
{ {
throw new ProtocolEvidenceMissingException( throw new ProtocolEvidenceMissingException(
$"ReadEvents over gRPC did not return rows within {OverallBudget.TotalSeconds:0}s: StartEventQuery " + $"ReadEvents over gRPC did not return rows within {OverallBudget.TotalSeconds:0}s: StartEventQuery " +
"succeeds but the CM_EVENT registration replay stalls and GetNextEventQueryResultBuffer long-polls " + "succeeds but GetNextEventQueryResultBuffer long-polls to the no-data terminal. Event-row retrieval is " +
"(no synchronous code-85 terminal over gRPC). Row-level retrieval is not yet verified over gRPC " + "auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"(the dev box holds no events) — use the WCF transport for event reads."); "connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md). The WCF transport reaches the " +
"2023 R2 historian (certificate transport + auth work, CM_EVENT registration succeeds on the 0x501 event " +
"connection) but hits the SAME server-side row gate — 0-row buffer + long-poll (see " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
} }
foreach (HistorianEvent evt in events) foreach (HistorianEvent evt in events)
@@ -169,16 +175,21 @@ internal sealed class HistorianGrpcEventOrchestrator
// reaches the no-data terminal with ZERO rows (the gRPC server long-polls GetNext rather than // reaches the no-data terminal with ZERO rows (the gRPC server long-polls GetNext rather than
// returning the WCF code-85 terminal), we cannot distinguish "genuinely no events in range" // returning the WCF code-85 terminal), we cannot distinguish "genuinely no events in range"
// from "the CM_EVENT registration replay didn't fully land over gRPC" — so we refuse to return // from "the CM_EVENT registration replay didn't fully land over gRPC" — so we refuse to return
// a possibly-false empty list and surface the unverified state instead. An event-bearing 2023 R2 // a possibly-false empty list and surface the gated state instead. Proven server-gated: the live
// server will return rows here and exercise the parse path; flip this once that is captured. // 2023 R2 server holds tens of thousands of events yet scopes 0 to a managed connection
// (grpc-event-query-capture.md). WCF reaches the same historian (cert transport + auth work,
// CM_EVENT registers on the 0x501 event connection) but hits the SAME row gate — not a fallback
// (wcf-event-read-spike-results.md).
if (events.Count == 0) if (events.Count == 0)
{ {
throw new ProtocolEvidenceMissingException( throw new ProtocolEvidenceMissingException(
"ReadEvents over gRPC: the chain completes and StartEventQuery succeeds, but " + "ReadEvents over gRPC: the chain completes and StartEventQuery succeeds, but " +
"GetNextEventQueryResultBuffer returns no rows (it long-polls to the no-data terminal " + "GetNextEventQueryResultBuffer returns no rows (it long-polls to the no-data terminal " +
$"after the CM_EVENT registration replay; last={LastErrorBufferDescription}). Row-level " + $"after the CM_EVENT registration replay; last={LastErrorBufferDescription}). Event-row retrieval is " +
"retrieval is not yet verified over gRPC (the dev box holds no events) — use the WCF " + "auth-solved but SERVER-GATED on 2023 R2 over both transports — the server scopes 0 rows to a managed " +
"transport for event reads until a capture against an event-bearing 2023 R2 server confirms it."); "connection (gRPC: docs/reverse-engineering/grpc-event-query-capture.md; WCF reaches the historian and " +
"registers on the 0x501 event connection yet hits the same row gate: " +
"docs/reverse-engineering/wcf-event-read-spike-results.md). Not client-fixable on either transport.");
} }
return events; return events;
@@ -53,6 +53,25 @@ public sealed class HistorianClientOptions
/// </summary> /// </summary>
public string? ServerDnsIdentity { get; init; } public string? ServerDnsIdentity { get; init; }
/// <summary>
/// Optional WCF "Via" address (e.g. <c>net.tcp://host:42568</c>). When set, the SDK's WCF
/// channel factories <b>connect</b> to this address while still addressing the SOAP message
/// <c>To</c> the logical endpoint built from <see cref="Host"/>/<see cref="Port"/>. Use this when
/// the Historian is reached through a port-forwarding tunnel or proxy whose local port differs
/// from the server's real service port: point <see cref="Host"/>/<see cref="Port"/> at the
/// server's real endpoint (so the server's WCF AddressFilter matches) and set this to the tunnel
/// endpoint. Has no effect on the gRPC transport. Default null (connect == address).
/// </summary>
public string? ConnectViaAddress { get; init; }
/// <summary>
/// Diagnostic override for the native OpenConnection mode the WCF event-read chain uses (default
/// <c>0x402</c>, read-only process). Set to e.g. <c>0x501</c> (event) or <c>0x401</c> (write-enabled)
/// to probe whether CM_EVENT registration / event-row retrieval needs a different connection type on a
/// 2023 R2 server. Null = the default read-only process mode. Intended for protocol investigation.
/// </summary>
public uint? EventReadConnectionModeOverride { get; init; }
/// <summary> /// <summary>
/// For <see cref="HistorianTransport.RemoteGrpc"/>: when true the channel uses TLS /// For <see cref="HistorianTransport.RemoteGrpc"/>: when true the channel uses TLS
/// (<c>https://</c>); when false it uses plaintext (<c>http://</c>). Matches the stock /// (<c>https://</c>); when false it uses plaintext (<c>http://</c>). Matches the stock
@@ -41,7 +41,7 @@ internal static class HistorianWcfAuthChainHelper
try try
{ {
IHistoryServiceContract2 historyChannel = historyFactory.CreateChannel(); IHistoryServiceContract2 historyChannel = HistorianWcfClientCredentialsHelper.CreateChannel(historyFactory, options);
ICommunicationObject historyChannelCo = (ICommunicationObject)historyChannel; ICommunicationObject historyChannelCo = (ICommunicationObject)historyChannel;
try try
{ {
@@ -30,6 +30,24 @@ internal static class HistorianWcfClientCredentialsHelper
} }
} }
/// <summary>
/// Creates a channel from <paramref name="factory"/>, honoring
/// <see cref="HistorianClientOptions.ConnectViaAddress"/> when set: the channel <b>connects</b> to
/// the Via address while still addressing the SOAP message <c>To</c> the factory's logical endpoint.
/// This lets a port-forward whose local port differs from the server's real service port satisfy the
/// server-side WCF AddressFilter (which validates the To header). Use this in place of
/// <c>factory.CreateChannel()</c> at every WCF event/read channel-creation site.
/// </summary>
public static TChannel CreateChannel<TChannel>(ChannelFactory<TChannel> factory, HistorianClientOptions options)
{
ArgumentNullException.ThrowIfNull(factory);
ArgumentNullException.ThrowIfNull(options);
return string.IsNullOrWhiteSpace(options.ConnectViaAddress)
? factory.CreateChannel()
: factory.CreateChannel(factory.Endpoint.Address, new Uri(options.ConnectViaAddress));
}
private sealed class AcceptAnyCertificateValidator : X509CertificateValidator private sealed class AcceptAnyCertificateValidator : X509CertificateValidator
{ {
public static readonly AcceptAnyCertificateValidator Instance = new(); public static readonly AcceptAnyCertificateValidator Instance = new();
@@ -8,10 +8,16 @@ using AVEVA.Historian.Client.Wcf.Contracts;
namespace AVEVA.Historian.Client.Wcf; namespace AVEVA.Historian.Client.Wcf;
/// <remarks> /// <remarks>
/// Mirrors HistorianWcfReadOrchestrator but targets IRetrievalServiceContract4 for the event flow. /// Mirrors HistorianWcfReadOrchestrator but targets IRetrievalServiceContract4 for the event flow. The
/// Event row buffer layout is undecoded as of this pass — when StartEventQuery succeeds, this /// event row-buffer layout is decoded (<see cref="HistorianEventRowProtocol"/>; verified against real
/// orchestrator returns an empty enumeration but logs the row-buffer length via the /// captured rows). A <b>2023 R2</b> historian <i>does</i> serve this transport via the <b>certificate</b>
/// <see cref="LastResultBufferLength"/> diagnostic so a follow-up capture can decode the wire shape. /// (TLS) endpoint (the cert transport + <c>NegotiateAuthentication</c> auth work cross-platform; the
/// integrated/Windows transport does not tunnel). With the <c>0x501</c> event connection mode CM_EVENT
/// registration succeeds — but <c>StartEventQuery</c> still returns a 0-row buffer and long-polls: event
/// rows are <b>server-gated</b> per connection on 2023 R2, the same wall as the gRPC path, and not
/// client-fixable (see <c>docs/reverse-engineering/wcf-event-read-spike-results.md</c> and
/// <c>grpc-event-query-capture.md</c>). The native return codes 76/85 noted below were 2020-historian
/// observations.
/// </remarks> /// </remarks>
internal sealed class HistorianWcfEventOrchestrator internal sealed class HistorianWcfEventOrchestrator
{ {
@@ -143,7 +149,7 @@ internal sealed class HistorianWcfEventOrchestrator
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction); EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
uint clientHandle = HistorianWcfAuthChainHelper.OpenAuthenticatedConnection( uint clientHandle = HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, cancellationToken, _options, histBinding, histEndpoint, contextKey, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode, connectionMode: _options.EventReadConnectionModeOverride ?? HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
additionalSetup: (historyChannel, context) => additionalSetup: (historyChannel, context) =>
AddCmEventTagViaAddT(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrBinding, retrEndpoint)); AddCmEventTagViaAddT(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrBinding, retrEndpoint));
return RunEventQuery(retrBinding, retrEndpoint, clientHandle, startUtc, endUtc, filter, cancellationToken); return RunEventQuery(retrBinding, retrEndpoint, clientHandle, startUtc, endUtc, filter, cancellationToken);
@@ -163,7 +169,7 @@ internal sealed class HistorianWcfEventOrchestrator
try try
{ {
IRetrievalServiceContract4 channel = factory.CreateChannel(); IRetrievalServiceContract4 channel = HistorianWcfClientCredentialsHelper.CreateChannel(factory, _options);
ICommunicationObject channelCo = (ICommunicationObject)channel; ICommunicationObject channelCo = (ICommunicationObject)channel;
try try
{ {
@@ -517,19 +517,26 @@ public sealed class HistorianGrpcIntegrationTests
} }
// Plan #2: ReadEvents over gRPC. The chain runs end-to-end and StartEventQuery succeeds // Plan #2: ReadEvents over gRPC. The chain runs end-to-end and StartEventQuery succeeds
// (no InvalidOperationException), but — confirmed live 2026-06-22 — GetNextEventQueryResultBuffer // (no InvalidOperationException), but GetNextEventQueryResultBuffer LONG-POLLS and returns zero
// LONG-POLLS and returns zero rows: the gRPC server blocks to the deadline instead of // rows: the gRPC server blocks to the deadline instead of returning the synchronous 5-byte
// returning the synchronous 5-byte code-85 terminal the 2020 WCF op returns, so the orchestrator // code-85 terminal the 2020 WCF op returns, so the orchestrator reaches its no-data terminal
// reaches its no-data terminal with zero rows and (rather than assert a possibly-false "no events" // with zero rows and (rather than assert a possibly-false "no events" empty) throws
// empty) throws ProtocolEvidenceMissingException. // ProtocolEvidenceMissingException.
// //
// IMPORTANT (2026-06-22): the zero rows are NOT "no events on the server". Verified against the // SERVER-GATED on 2023 R2 (settled 2026-06-26; supersedes a 2026-06-22 draft that read the zero
// live 2023 R2 box, which holds 19,356 events in the last 30 days (SQL ground truth via the INSQL // rows as a client-side empty-filter/request-shape "capture gate"). Three live controls against
// linked server, Runtime.dbo.Events). The EMPTY-FILTER gRPC event query simply does not match // the same event-bearing 2023 R2 server retire the request-shape theory:
// them. So the gate here is the empty-filter request shape (filter / namespace / event-tag // 1. The STOCK AVEVA 2023 R2 client (aahClientManaged, native Event connection, its OWN correct
// registration), NOT data availability — this is capture-gated (needs a native gRPC event-query // event query — not our empty-filter shape) returns 0 rows over 30d / 90d / 365d / 3yr.
// capture), not server-gated. Flip to asserting parsed rows once that capture lands and the // The 2026-06-22 "50 rows" stock capture did NOT reproduce.
// request is corrected. The chain stays BOUNDED (no multi-minute hang) via the short // 2. SQL ground truth (Runtime.dbo.Events) shows the server holds tens of thousands of events
// in those windows (newest minutes old) — so it is emphatically NOT "no events".
// 3. POSITIVE CONTROL: the same native WCF event-read client (protocol byte-replayed from stock
// 2020 captures) returns real events (5) from a local AVEVA Historian 2020 over the identical
// sequence, but 0 (10-byte 0-row header + long-poll) from this 2023 R2 server.
// The protocol works; the 2023 R2 server simply does not scope event rows to a managed connection.
// Not client-fixable (see docs/reverse-engineering/grpc-event-query-capture.md +
// wcf-event-read-spike-results.md). The chain stays BOUNDED (no multi-minute hang) via the short
// registration + poll deadlines. (Set a small HISTORIAN_GRPC_TIMEOUT to keep this snappy.) // registration + poll deadlines. (Set a small HISTORIAN_GRPC_TIMEOUT to keep this snappy.)
HistorianClient client = new(BuildOptions(host)); HistorianClient client = new(BuildOptions(host));
@@ -36,9 +36,14 @@ public sealed class WcfEventReadSpikeTests
public async Task WcfEventRead_DiagnosticDump_AgainstRemoteHistorian() public async Task WcfEventRead_DiagnosticDump_AgainstRemoteHistorian()
{ {
string? host = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_HOST"); string? host = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_HOST");
if (string.IsNullOrWhiteSpace(host) || !OperatingSystem.IsWindows()) bool certificate = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_TRANSPORT"), "certificate", StringComparison.OrdinalIgnoreCase);
// The certificate (TLS) transport and NegotiateAuthentication-based app auth are cross-platform;
// the integrated (Windows SSPI transport-security) transport is Windows-only. Skip when the gate
// is unconfigured, or when an integrated run is requested off Windows.
if (string.IsNullOrWhiteSpace(host) || (!certificate && !OperatingSystem.IsWindows()))
{ {
return; // inert without the dedicated gate / off Windows return;
} }
int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PORT"), out int p) int port = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PORT"), out int p)
@@ -48,41 +53,95 @@ public sealed class WcfEventReadSpikeTests
string? user = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_USER"); string? user = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PASSWORD"); string? password = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_PASSWORD");
string? spn = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_SPN"); string? spn = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_SPN");
bool integrated = string.IsNullOrEmpty(user); string? dnsId = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_DNSID");
string? via = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_VIA");
bool bypassVersion = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_NOVERSIONCHECK"), "1", StringComparison.Ordinal);
int timeoutSec = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_TIMEOUT_SEC"), out int ts) && ts > 0
? ts : 30;
string? connModeRaw = Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_CONNMODE");
uint? connModeOverride = null;
if (!string.IsNullOrWhiteSpace(connModeRaw))
{
string hex = connModeRaw.StartsWith("0x", StringComparison.OrdinalIgnoreCase) ? connModeRaw[2..] : connModeRaw;
if (uint.TryParse(hex, System.Globalization.NumberStyles.HexNumber, null, out uint cm)) { connModeOverride = cm; }
}
bool allowUntrusted = string.Equals(
Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_ALLOW_UNTRUSTED"), "1", StringComparison.Ordinal);
// Certificate transport carries no Windows transport credential (cert-validated TLS channel);
// integrated derives one from the logged-in identity unless an explicit DOMAIN\user is supplied.
// App-level ValidateClientCredential still uses UserName/Password (or the process identity).
bool integrated = !certificate && string.IsNullOrEmpty(user);
HistorianClientOptions options = new() HistorianClientOptions options = new()
{ {
Host = host, Host = host,
Port = port, Port = port,
Transport = HistorianTransport.RemoteTcpIntegrated, Transport = certificate ? HistorianTransport.RemoteTcpCertificate : HistorianTransport.RemoteTcpIntegrated,
IntegratedSecurity = integrated, IntegratedSecurity = integrated,
UserName = user ?? string.Empty, UserName = user ?? string.Empty,
Password = password ?? string.Empty, Password = password ?? string.Empty,
TargetSpn = string.IsNullOrWhiteSpace(spn) ? "NT SERVICE\\aahClientAccessPoint" : spn, TargetSpn = string.IsNullOrWhiteSpace(spn) ? "NT SERVICE\\aahClientAccessPoint" : spn,
ServerDnsIdentity = string.IsNullOrWhiteSpace(dnsId) ? null : dnsId,
AllowUntrustedServerCertificate = allowUntrusted,
ConnectViaAddress = string.IsNullOrWhiteSpace(via) ? null : via,
VerifyServerInterfaceVersion = !bypassVersion,
ConnectTimeout = TimeSpan.FromSeconds(timeoutSec),
RequestTimeout = TimeSpan.FromSeconds(timeoutSec),
EventReadConnectionModeOverride = connModeOverride,
}; };
HistorianWcfEventOrchestrator orchestrator = new(options); HistorianWcfEventOrchestrator orchestrator = new(options);
DateTime endUtc = DateTime.UtcNow; DateTime endUtc = DateTime.UtcNow;
DateTime startUtc = endUtc - TimeSpan.FromDays(days); DateTime startUtc = endUtc - TimeSpan.FromDays(days);
int budgetSec = int.TryParse(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_BUDGET_SEC"), out int bgs) && bgs > 0
? bgs : 60;
int observed = 0; int observed = 0;
bool hasFirstEvent = false; bool hasFirstEvent = false;
string outcome = "completed"; string outcome = "completed";
try try
{ {
await foreach (HistorianEvent evt in orchestrator.ReadEventsAsync(startUtc, endUtc, filter: null, CancellationToken.None)) // Race the read against an overall budget. The chain is synchronous WCF, so a stuck call
// can't be token-cancelled — but the orchestrator's static/instance diagnostics are set as it
// progresses, so on a budget timeout we still dump them to see which phase (auth / CM_EVENT
// registration / query) is hanging. The abandoned task keeps running in the background harness.
using var budget = new CancellationTokenSource(TimeSpan.FromSeconds(budgetSec));
Task<int> readTask = Task.Run(async () =>
{ {
observed++; int n = 0;
hasFirstEvent = true; await foreach (HistorianEvent evt in orchestrator.ReadEventsAsync(startUtc, endUtc, filter: null, budget.Token))
_ = evt; // event identity intentionally NOT logged (sanitized) {
n++;
_ = evt; // event identity intentionally NOT logged (sanitized)
}
return n;
});
Task finished = await Task.WhenAny(readTask, Task.Delay(TimeSpan.FromSeconds(budgetSec + 5)));
if (finished == readTask)
{
observed = await readTask;
hasFirstEvent = observed > 0;
}
else
{
outcome = $"TIMED OUT after {budgetSec}s (chain still running — see return codes for phase reached)";
} }
} }
catch (Exception ex) catch (Exception ex)
{ {
outcome = $"threw {ex.GetType().Name}"; // message omitted — may carry host/credential text // Default: type name only (sanitized). Opt into messages for live binding/protocol debugging
// via HISTORIAN_WCF_EVENT_VERBOSE=1 — binding/protocol errors may carry the endpoint host
// (already known to the operator) but never credentials; still off by default.
outcome = string.Equals(Environment.GetEnvironmentVariable("HISTORIAN_WCF_EVENT_VERBOSE"), "1", StringComparison.Ordinal)
? $"threw {ex.GetType().FullName} :: {ex.Message} | inner={ex.InnerException?.GetType().FullName} :: {ex.InnerException?.Message}"
: $"threw {ex.GetType().Name}";
} }
// Sanitized diagnostic dump — counts, native return codes, buffer lengths, sha256 ONLY. // Sanitized diagnostic dump — counts, native return codes, buffer lengths, sha256 ONLY.
_output.WriteLine($"[C2 WCF spike] transport: {options.Transport} (integratedSec={integrated}, allowUntrusted={allowUntrusted}, connMode={(connModeOverride.HasValue ? "0x" + connModeOverride.Value.ToString("X") : "default-0x402")})");
_output.WriteLine($"[C2 WCF spike] outcome: {outcome}"); _output.WriteLine($"[C2 WCF spike] outcome: {outcome}");
_output.WriteLine($"[C2 WCF spike] events observed: {observed}"); _output.WriteLine($"[C2 WCF spike] events observed: {observed}");
_output.WriteLine($"[C2 WCF spike] hasFirstEvent: {hasFirstEvent}"); _output.WriteLine($"[C2 WCF spike] hasFirstEvent: {hasFirstEvent}");