Merge handoff-refresh-counts: SendEvent over gRPC + 2023 R2 binary-dive verdicts
- SendEvent over gRPC SHIPPED + live-validated (HistorianGrpcEventWriteOrchestrator; reuses the WCF "OS" event buffer verbatim on HistoryService.AddStreamValues). - 2023 R2 stock-client binary dive: sharpened every pending item to evidence-based verdicts; DeleteTagExtendedProperties confirmed walled via native capture. - Event registration (RegisterTags/EnsureTags) golden-tested vs live capture. - Handoff date/test-count refresh; revision-probe scope comment fix. 331 offline tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
@@ -1,6 +1,6 @@
|
|||||||
# AVEVA Historian Managed Driver Handoff
|
# AVEVA Historian Managed Driver Handoff
|
||||||
|
|
||||||
Last updated: 2026-06-22 (roadmap exhausted — no actionable pure-code tasks remain)
|
Last updated: 2026-06-23 (event-row parser fix merged; roadmap still exhausted — no actionable pure-code tasks remain)
|
||||||
|
|
||||||
> **Current status supersedes the historical blocker narrative below.** The
|
> **Current status supersedes the historical blocker narrative below.** The
|
||||||
> sections from "Active Blocker" onward are a preserved reverse-engineering
|
> sections from "Active Blocker" onward are a preserved reverse-engineering
|
||||||
@@ -76,18 +76,33 @@ reuses the proven 2020 WCF byte serializers/parsers unchanged inside protobuf
|
|||||||
`capture-event` harness (native, returns rows).
|
`capture-event` harness (native, returns rows).
|
||||||
2. **R4.3 active-SF magnitude** — needs an **SF-active server** (D2 storage-engine
|
2. **R4.3 active-SF magnitude** — needs an **SF-active server** (D2 storage-engine
|
||||||
console handle).
|
console handle).
|
||||||
3. **SendEvent over gRPC** — **capture-gated**: no distinct RPC, framing uncaptured.
|
3. **SendEvent over gRPC** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23.** `SendEventAsync`
|
||||||
|
now routes over `RemoteGrpc` (`HistorianGrpcEventWriteOrchestrator`). Captured the native
|
||||||
|
client live (`capture-send-event` harness scenario): the send rides
|
||||||
|
`HistoryService.AddStreamValues` with the **same "OS" (0x534F) buffer the WCF path uses**
|
||||||
|
(`HistorianEventWriteProtocol` — "no distinct RPC" confirmed true), on a v8 Event session +
|
||||||
|
CM_EVENT registration. The write-enabled Event open is **byte-identical** to the read-only one
|
||||||
|
(diffed live — only per-session crypto differs), so the existing event-open path is reused
|
||||||
|
unchanged. End-to-end: pure-managed SDK send → `BSuccess=true` → event read back from the live
|
||||||
|
server (markers `SdkSendProbe`/`SdkCaptureProbe` confirmed in returned rows). Golden-tested
|
||||||
|
(`GrpcEventSendProtocolTests`) + gated live test (`SendEventAsync_OverGrpc_AcceptsEvent`,
|
||||||
|
opt-in `HISTORIAN_GRPC_EVENT_SEND=1`).
|
||||||
4. **ExecuteSqlCommand over gRPC** — **server-walled** (`CSrvDbConnection`;
|
4. **ExecuteSqlCommand over gRPC** — **server-walled** (`CSrvDbConnection`;
|
||||||
RegisterTags prime doesn't help). Use WCF for SQL.
|
RegisterTags prime doesn't help). Use WCF for SQL.
|
||||||
5. **R4.2 revision EDITS** — storage-engine-pipe-only on BOTH transports (the D2 wall).
|
5. **R4.2 revision EDITS** — storage-engine-pipe-only on BOTH transports (the D2 wall).
|
||||||
6. **ReadBlocks** (`StartBlockRetrievalQuery`) — never captured on either transport.
|
6. **ReadBlocks** (`StartBlockRetrievalQuery`) — never captured on either transport.
|
||||||
7. **DeleteTagExtendedProperties** — server-blocked on BOTH transports. The gRPC
|
7. **DeleteTagExtendedProperties** — server-blocked; **confirmed walled by capturing the NATIVE
|
||||||
multiplexed-channel hypothesis was **PROBED + DISPROVEN 2026-06-22** (merge
|
client 2026-06-23.** The agent's "use `deleteFromServer=true`" angle is moot: the native
|
||||||
`c88260c`): GetTgByNm + GetTepByNm primes succeed on one shared write-enabled
|
`HistorianAccess.DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven with the
|
||||||
gRPC channel, yet DelTep is still rejected (native code=1) and the property
|
cross-session sync trick that gets it past the client-side err-229 sync gate
|
||||||
survives — the working set is native in-process registration state, not the
|
(`Capture-DeleteTagExtendedProperties.ps1`), returns **`Success=true` / ErrorCode=Success** —
|
||||||
wire session. Pinned by gated negative test
|
yet across repeated fresh sessions the property is **re-fetchable and re-deletable every time**,
|
||||||
`DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel`.
|
i.e. it is **never durably removed**. So the native client itself only performs an optimistic
|
||||||
|
client-side cache delete; the server does not durably honor it (matches the HCAL cache-sync
|
||||||
|
model the decompile shows). Shipping a `DeleteTagExtendedPropertiesAsync` would return a
|
||||||
|
misleading success while the property persists, so it correctly stays **unshipped**. (Earlier
|
||||||
|
gRPC multiplexed-channel hypothesis also PROBED + DISPROVEN 2026-06-22, merge `c88260c`; pinned
|
||||||
|
by `DeleteTagExtendedProperties_OverGrpc_ProbeMultiplexedChannel`.)
|
||||||
8. **Deferred-by-design** items (`write-commands` D1–D3, non-analog tag create,
|
8. **Deferred-by-design** items (`write-commands` D1–D3, non-analog tag create,
|
||||||
etc.) — bounded out until an explicit customer/user demand signal.
|
etc.) — bounded out until an explicit customer/user demand signal.
|
||||||
|
|
||||||
@@ -101,6 +116,73 @@ Live-server gRPC probe recipe: set
|
|||||||
quotes — `reference_wonder_sql_vd03_credentials`) and run the gated
|
quotes — `reference_wonder_sql_vd03_credentials`) and run the gated
|
||||||
`HistorianGrpcIntegrationTests`.
|
`HistorianGrpcIntegrationTests`.
|
||||||
|
|
||||||
|
### 2023 R2 stock-client binary dive (2026-06-23) — sharpened verdicts
|
||||||
|
|
||||||
|
Re-read the full decompiled stock 2023 R2 managed client
|
||||||
|
(`histsdk-2023r2-analysis/decompiled/`: `Archestra.Historian.GrpcClient`,
|
||||||
|
`ArchestrA.HistorianAccess`, `Archestra.Grpc.Contract`, `HistorianEvent`,
|
||||||
|
`HistorianAccessUtil`) as the oracle for every still-pending item. **Governing
|
||||||
|
fact:** `ArchestrA.HistorianAccess.dll` is a C++/CLI mixed assembly — every
|
||||||
|
data/config/write method is a thin shim into native `<Module>.HistorianClient.*`,
|
||||||
|
and the managed `Grpc*Client` wrappers are instantiated by **nothing** in the
|
||||||
|
decompiled set (`new Grpc*Client(` → zero call sites). So the buffer-building and
|
||||||
|
RPC-dispatch sequencing for these items lives in native C++ not present in the
|
||||||
|
binaries. That confirms the "gated" calls were not from missing managed steps —
|
||||||
|
with these refinements:
|
||||||
|
|
||||||
|
- **Item 1 (gRPC event rows)** — **confirmed native/server-side.** Stock event
|
||||||
|
call graph is provably identical to ours (transport, per-service channels,
|
||||||
|
gzip-only metadata, CM_EVENT registration, v8 ECDH Event-open, `StartEventQuery`
|
||||||
|
request bytes). `EventQuery.StartQuery`/`MoveNext` dispatch straight into native
|
||||||
|
`HistorianClient.StartEventQuery`/`GetNextRow`; the query orchestration that
|
||||||
|
would differ is native and not on the wire. One untested low-effort check
|
||||||
|
remains: byte-diff a captured **Event-connection** EnsureTags/RegisterTags
|
||||||
|
against our replay (the 83-vs-86-byte EnsT gap was never actually compared).
|
||||||
|
- **Item 3 (SendEvent over gRPC)** — ✅ **SHIPPED + LIVE-VALIDATED 2026-06-23** (was
|
||||||
|
"capturable"). RPC confirmed = `HistoryService.AddStreamValues` (the "no distinct RPC"
|
||||||
|
note is TRUE). The `btValues` VTQ buffer turned out to be already-owned: our M2
|
||||||
|
`HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer` ("OS" buffer, decoded from
|
||||||
|
the WCF event-send) is the transport-independent `PackToVtq` equivalent and the gRPC send
|
||||||
|
uses it **verbatim** (live capture: sig `OS`/0x534F, CM_EVENT GUID, identical framing — NOT
|
||||||
|
the historical write's "ON" buffer). The write-enabled Event open is byte-identical to the
|
||||||
|
read-only one (live diff). So SendEvent-over-gRPC was pure assembly:
|
||||||
|
`HistorianGrpcEventWriteOrchestrator` = existing v8 Event open + existing CM_EVENT
|
||||||
|
registration + `AddStreamValues`(OS buffer). End-to-end live-validated (send → `BSuccess`
|
||||||
|
→ read back from the live server). Golden-tested + gated live test.
|
||||||
|
- **Item 4 (ExecuteSql over gRPC)** — **confirmed walled + explained.** The stock
|
||||||
|
client gates SQL **out client-side**: `HistorianAccess.ExecuteSqlCommand` returns
|
||||||
|
`OperationNotSupported` when `IsManagedHistorian(node)` or `!IsProcessConnectionRequested()`
|
||||||
|
(decompile ~:6198/:6214) and never sends the RPC. SQL-over-gRPC is unsupported by
|
||||||
|
design on a managed/gRPC historian; our `ProtocolEvidenceMissingException` is correct.
|
||||||
|
- **Item 5 (R4.2 revision edits)** — **confirmed HARD.** There is **no Revision RPC
|
||||||
|
in the gRPC contract** (zero "Revision" message types); the stock client reaches a
|
||||||
|
revision edit only via the native `HistorianClient.AddRevisionValuesBegin/AddRevisionValue/
|
||||||
|
AddRevisionValuesEnd` transaction trio over the storage-engine channel. NOTE: this is
|
||||||
|
a **distinct capability** from `AddNonStreamValues` (non-streamed original insert) —
|
||||||
|
`HistorianGrpcRevisionProbe` probes the latter; its doc comment was corrected to say so.
|
||||||
|
- **Item 6 (ReadBlocks/LoadBlocks)** — `LoadBlocks` request is a trivial
|
||||||
|
handle+sequence cursor but the `historyBlocks` response is a native blob with no
|
||||||
|
managed decoder, and it needs the D2-blocked `OpenStorageConnection` console handle.
|
||||||
|
Walled.
|
||||||
|
- **Item 7 (DeleteTagExtendedProperties)** — **capture done 2026-06-23; CONFIRMED WALLED, don't
|
||||||
|
ship.** RPC + string handle are correct; ADD/DELETE are structurally identical and neither uses
|
||||||
|
`StartJob`. The `deleteFromServer`-flag hypothesis is now tested and moot: the native
|
||||||
|
`DeleteTagExtendedPropertiesByName(...,deleteFromServer:true)`, driven past the err-229 client
|
||||||
|
sync gate with the cross-session trick (`Capture-DeleteTagExtendedProperties.ps1`), returns
|
||||||
|
`Success=true` — yet the property is **re-fetchable + re-deletable across repeated fresh sessions
|
||||||
|
(never durably removed)**. So the native client only does an optimistic client-side cache delete
|
||||||
|
the server doesn't durably honor (the HCAL cache-sync model). Shipping
|
||||||
|
`DeleteTagExtendedPropertiesAsync` would return a misleading success, so it stays unshipped.
|
||||||
|
- **SF/snapshot/shard/ForwardSnapshot ops** — only `Get/SetSFParameter` are managed-built
|
||||||
|
(typed strings); all others carry opaque native buffers and need the storage console
|
||||||
|
handle. Walled / tooling-internal.
|
||||||
|
|
||||||
|
**Net:** 3 items hard-confirmed walled with real explanations (4, 5, 6 + OpenStorageConnection),
|
||||||
|
and **2 moved to a precise, local-box-capturable target**: **SendEvent** (`PackToVtq` output)
|
||||||
|
and **DeleteTEP** (`BtInput` with `deleteFromServer=true`). Both need native instrumentation of
|
||||||
|
`aahClientManaged.dll` (Frida / IL-rewrite — repo tooling exists under
|
||||||
|
`tools/AVEVA.Historian.NativeTraceHarness` + `scripts/frida/`), not a special server.
|
||||||
|
|
||||||
## Project Direction
|
## Project Direction
|
||||||
|
|
||||||
The project goal is still a fully managed .NET 10 C# AVEVA Historian client.
|
The project goal is still a fully managed .NET 10 C# AVEVA Historian client.
|
||||||
@@ -172,12 +254,14 @@ dotnet build .\Histsdk.slnx --no-restore
|
|||||||
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
|
dotnet test .\Histsdk.slnx --no-build --logger "console;verbosity=minimal"
|
||||||
```
|
```
|
||||||
|
|
||||||
Current known-good result (2026-06-22):
|
Current known-good result (2026-06-23):
|
||||||
|
|
||||||
- Build succeeds (0 warnings / 0 errors).
|
- Build succeeds (0 warnings / 0 errors).
|
||||||
- Offline tests pass: 321/321 (live gRPC/integration tests skip cleanly without
|
- Offline tests pass: 328/328 (live gRPC/integration tests skip cleanly without
|
||||||
their env vars). Gated live tests add to this when `HISTORIAN_*` /
|
their env vars). Gated live tests add to this when `HISTORIAN_*` /
|
||||||
`HISTORIAN_GRPC_*` are set.
|
`HISTORIAN_GRPC_*` are set. The +7 over the prior 321 are the event-row parser
|
||||||
|
fix's golden + gated-capture coverage (`HistorianEventRowProtocolTests`:
|
||||||
|
markerless multi-row, the v11 gRPC header, and the 50-event stock-client capture).
|
||||||
|
|
||||||
The workspace is a Git working tree (origin: gitea.dohertylan.com). Use
|
The workspace is a Git working tree (origin: gitea.dohertylan.com). Use
|
||||||
normal git workflow for change tracking; the prior "no working tree, use
|
normal git workflow for change tracking; the prior "no working tree, use
|
||||||
|
|||||||
@@ -0,0 +1,136 @@
|
|||||||
|
using Google.Protobuf;
|
||||||
|
using AVEVA.Historian.Client.Models;
|
||||||
|
using AVEVA.Historian.Client.Wcf;
|
||||||
|
using GrpcHistory = ArchestrA.Grpc.Contract.History;
|
||||||
|
using GrpcStatus = ArchestrA.Grpc.Contract.Status;
|
||||||
|
|
||||||
|
namespace AVEVA.Historian.Client.Grpc;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// 2023 R2 gRPC orchestrator for the event SEND (<see cref="HistorianClient.SendEventAsync"/>).
|
||||||
|
/// Captured live from the native 2023 R2 client (<c>capture-send-event</c> scenario,
|
||||||
|
/// 2026-06-23): an event send rides <c>HistoryService.AddStreamValues</c> with the SAME
|
||||||
|
/// <c>"OS"</c> (0x534F) storage-sample buffer the WCF AddS2 path uses
|
||||||
|
/// (<see cref="HistorianEventWriteProtocol"/>) — NOT a distinct event RPC and NOT the historical
|
||||||
|
/// write's <c>"ON"</c> buffer. The native client's write-enabled Event <c>OpenConnection</c>
|
||||||
|
/// request is byte-identical to the read-only event open (the ReadOnly arg does not change the v8
|
||||||
|
/// open buffer; diffed live — only the per-session client key + credential token differ), so the
|
||||||
|
/// existing <see cref="HistorianGrpcHandshake.OpenSession"/> event path is reused unchanged. The
|
||||||
|
/// chain on a single Event session:
|
||||||
|
/// <list type="number">
|
||||||
|
/// <item>OpenConnection (v8 Event, ExchangeKey ECDH auth) → string storage handle</item>
|
||||||
|
/// <item>CM_EVENT registration: UpdateClientStatus → RegisterTags → EnsureTags (the same
|
||||||
|
/// buffers the gRPC event READ replays — verified byte-identical to the capture)</item>
|
||||||
|
/// <item><c>HistoryService.AddStreamValues</c>(strHandle, "OS" event buffer)</item>
|
||||||
|
/// </list>
|
||||||
|
/// Only original events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued
|
||||||
|
/// properties have a captured encoding; others throw <see cref="ProtocolEvidenceMissingException"/>
|
||||||
|
/// from <see cref="HistorianEventWriteProtocol"/>.
|
||||||
|
/// </summary>
|
||||||
|
internal sealed class HistorianGrpcEventWriteOrchestrator
|
||||||
|
{
|
||||||
|
private readonly HistorianClientOptions _options;
|
||||||
|
|
||||||
|
public HistorianGrpcEventWriteOrchestrator(HistorianClientOptions options)
|
||||||
|
{
|
||||||
|
_options = options ?? throw new ArgumentNullException(nameof(options));
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>Diagnostic: type+code description of the most recent AddStreamValues error buffer.</summary>
|
||||||
|
public string LastSendErrorDescription { get; private set; } = string.Empty;
|
||||||
|
|
||||||
|
/// <summary>Diagnostic: outcomes of the CM_EVENT registration RPCs.</summary>
|
||||||
|
public string RegistrationDiag { get; private set; } = string.Empty;
|
||||||
|
|
||||||
|
public Task<bool> SendEventAsync(HistorianEvent evt, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
ArgumentNullException.ThrowIfNull(evt);
|
||||||
|
if (!_options.IntegratedSecurity && string.IsNullOrEmpty(_options.UserName))
|
||||||
|
{
|
||||||
|
throw new ProtocolEvidenceMissingException(
|
||||||
|
"Managed gRPC event send currently requires IntegratedSecurity or an explicit UserName + Password.");
|
||||||
|
}
|
||||||
|
|
||||||
|
if (evt.RevisionVersion != 0)
|
||||||
|
{
|
||||||
|
throw new ProtocolEvidenceMissingException(
|
||||||
|
"Only original events (RevisionVersion = 0) have a captured send encoding; " +
|
||||||
|
"revision/update/delete event sends are not yet supported.");
|
||||||
|
}
|
||||||
|
|
||||||
|
return Task.Run(() => Run(evt, cancellationToken), cancellationToken);
|
||||||
|
}
|
||||||
|
|
||||||
|
private bool Run(HistorianEvent evt, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
|
||||||
|
|
||||||
|
// The event SEND uses the same v8 Event connection as the event READ. The write-enabled
|
||||||
|
// open buffer is byte-identical to the read-only one (verified live), so OpenSession's
|
||||||
|
// event path is reused unchanged.
|
||||||
|
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
|
||||||
|
connection, _options, cancellationToken, eventConnection: true);
|
||||||
|
|
||||||
|
RegisterCmEventTag(connection, session, cancellationToken);
|
||||||
|
|
||||||
|
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
|
||||||
|
byte[] pBuf = HistorianEventWriteProtocol.SerializeAddStreamValuesBuffer(evt, DateTime.UtcNow);
|
||||||
|
|
||||||
|
GrpcHistory.AddStreamValuesResponse response = historyClient.AddStreamValues(
|
||||||
|
new GrpcHistory.AddStreamValuesRequest
|
||||||
|
{
|
||||||
|
StrHandle = session.StringHandle,
|
||||||
|
BtValues = ByteString.CopyFrom(pBuf),
|
||||||
|
},
|
||||||
|
connection.Metadata,
|
||||||
|
DateTime.UtcNow.Add(_options.RequestTimeout),
|
||||||
|
cancellationToken);
|
||||||
|
|
||||||
|
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
|
||||||
|
LastSendErrorDescription = HistorianEventRegistrationProtocol.DescribeNativeError(error);
|
||||||
|
return response.Status?.BSuccess ?? false;
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Replays the CM_EVENT registration the native event connection performs before a send:
|
||||||
|
/// UpdateClientStatus → RegisterTags(CM_EVENT) → EnsureTags(CM_EVENT). The buffers are shared
|
||||||
|
/// with the gRPC event READ path (<see cref="HistorianEventRegistrationProtocol"/> +
|
||||||
|
/// <see cref="HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc"/>) and were verified
|
||||||
|
/// byte-identical to the live capture. Best-effort: an individual rejection does not abort the
|
||||||
|
/// send (the server may already hold CM_EVENT registered for the session).
|
||||||
|
/// </summary>
|
||||||
|
private void RegisterCmEventTag(HistorianGrpcConnection connection, HistorianGrpcHandshake.Session session, CancellationToken cancellationToken)
|
||||||
|
{
|
||||||
|
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
|
||||||
|
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
|
||||||
|
|
||||||
|
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
historyClient.UpdateClientStatus(
|
||||||
|
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
|
||||||
|
connection.Metadata, Deadline(), cancellationToken);
|
||||||
|
}
|
||||||
|
catch { /* best-effort */ }
|
||||||
|
|
||||||
|
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
|
||||||
|
try
|
||||||
|
{
|
||||||
|
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
|
||||||
|
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
|
||||||
|
connection.Metadata, Deadline(), cancellationToken);
|
||||||
|
RegistrationDiag += $"RTag={rt.Status?.BSuccess}; ";
|
||||||
|
}
|
||||||
|
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
|
||||||
|
|
||||||
|
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
|
||||||
|
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
|
||||||
|
connection.Metadata, Deadline(), cancellationToken);
|
||||||
|
RegistrationDiag += $"EnsT={et.Status?.BSuccess}; ";
|
||||||
|
}
|
||||||
|
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -20,6 +20,18 @@ namespace AVEVA.Historian.Client.Grpc;
|
|||||||
/// write-enabled session and calls <c>AddNonStreamValuesBegin</c>, surfacing whatever the server
|
/// write-enabled session and calls <c>AddNonStreamValuesBegin</c>, surfacing whatever the server
|
||||||
/// returns. It writes NO data — if Begin succeeds it immediately calls <c>AddNonStreamValuesEnd</c>
|
/// returns. It writes NO data — if Begin succeeds it immediately calls <c>AddNonStreamValuesEnd</c>
|
||||||
/// with <c>bCommit=false</c> to discard the transaction.
|
/// with <c>bCommit=false</c> to discard the transaction.
|
||||||
|
///
|
||||||
|
/// <para><b>Scope note (corrected 2026-06-23 after a 2023 R2 binary re-read).</b> Despite the type
|
||||||
|
/// name, this probes the <i>non-streamed ORIGINAL / backfill insert</i> capability
|
||||||
|
/// (<c>AddNonStreamValues</c>), which is a <b>distinct capability from a revision EDIT</b>
|
||||||
|
/// (overwriting an existing historized value with a new revision). The stock high-level client
|
||||||
|
/// reaches a revision edit via a separate native transaction trio
|
||||||
|
/// <c>HistorianClient.AddRevisionValuesBegin/AddRevisionValue/AddRevisionValuesEnd</c>
|
||||||
|
/// (<c>ArchestrA.HistorianAccess.AddRevisionValues</c>, REVISION_MODE ∈ InsertLatest/UpdateSingle/
|
||||||
|
/// UpdateMultiple). That trio has <b>NO corresponding RPC in the gRPC contract</b> (no "Revision"
|
||||||
|
/// message type exists in <c>Archestra.Grpc.Contract</c>) — it rides the native storage-engine
|
||||||
|
/// transaction channel only. So R4.2 revision edits remain unreachable over gRPC regardless of this
|
||||||
|
/// probe's outcome; this probe neither covers nor unblocks them.</para>
|
||||||
/// </summary>
|
/// </summary>
|
||||||
internal sealed class HistorianGrpcRevisionProbe
|
internal sealed class HistorianGrpcRevisionProbe
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -114,18 +114,24 @@ public sealed class HistorianClient : IAsyncDisposable
|
|||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag
|
/// Sends a single <see cref="HistorianEvent"/> to the Historian's built-in CM_EVENT tag.
|
||||||
/// over the WCF event pipeline (Open2 event mode → CM_EVENT registration → AddS2). The
|
/// Over WCF this runs Open2 event mode → CM_EVENT registration → AddS2; over the 2023 R2
|
||||||
/// event is appended to the historian's event history and is readable back via
|
/// <see cref="HistorianTransport.RemoteGrpc"/> transport it runs the captured-equivalent
|
||||||
/// <see cref="ReadEventsAsync"/> / the <c>v_AlarmEventHistory2</c> view. Only original
|
/// v8 Event OpenConnection → CM_EVENT registration → <c>HistoryService.AddStreamValues</c>
|
||||||
/// events (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties
|
/// with the same "OS" event buffer (live-captured 2026-06-23 — the send rides the same RPC
|
||||||
/// are supported; other property value types and revision/update/delete events throw
|
/// and buffer as the WCF path, not a distinct event RPC). The event is appended to the
|
||||||
|
/// historian's event history and is readable back via <see cref="ReadEventsAsync"/> /
|
||||||
|
/// the <c>v_AlarmEventHistory2</c> view. Only original events
|
||||||
|
/// (<see cref="HistorianEvent.RevisionVersion"/> = 0) with string-valued properties are
|
||||||
|
/// supported; other property value types and revision/update/delete events throw
|
||||||
/// <see cref="ProtocolEvidenceMissingException"/> until their wire encoding is captured.
|
/// <see cref="ProtocolEvidenceMissingException"/> until their wire encoding is captured.
|
||||||
/// </summary>
|
/// </summary>
|
||||||
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default)
|
public Task<bool> SendEventAsync(HistorianEvent historianEvent, CancellationToken cancellationToken = default)
|
||||||
{
|
{
|
||||||
ArgumentNullException.ThrowIfNull(historianEvent);
|
ArgumentNullException.ThrowIfNull(historianEvent);
|
||||||
return new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
|
return _options.Transport == HistorianTransport.RemoteGrpc
|
||||||
|
? new Grpc.HistorianGrpcEventWriteOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken)
|
||||||
|
: new HistorianWcfEventOrchestrator(_options).SendEventAsync(historianEvent, cancellationToken);
|
||||||
}
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
|
|||||||
@@ -0,0 +1,53 @@
|
|||||||
|
using System;
|
||||||
|
using System.Buffers.Binary;
|
||||||
|
using AVEVA.Historian.Client.Wcf;
|
||||||
|
using Xunit;
|
||||||
|
|
||||||
|
namespace AVEVA.Historian.Client.Tests;
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Golden-byte coverage for the 2023 R2 gRPC event-SEND registration buffers, pinned against a live
|
||||||
|
/// native capture (<c>capture-send-event</c> scenario, 2026-06-23). The send itself rides
|
||||||
|
/// <c>HistoryService.AddStreamValues</c> with the same "OS" buffer the WCF path uses
|
||||||
|
/// (<see cref="HistorianEventWriteProtocol"/>, already golden-tested in
|
||||||
|
/// <c>WcfEventWriteProtocolTests</c>); what is gRPC-specific is the CM_EVENT registration the event
|
||||||
|
/// connection performs first (RegisterTags + the 86-byte gRPC EnsureTags). These fixtures are the raw
|
||||||
|
/// bytes the native client sent on the wire — they carry no identity (CM_EVENT / "AnE Event" /
|
||||||
|
/// constant tag + event-type GUIDs / a registration FILETIME).
|
||||||
|
/// </summary>
|
||||||
|
public class GrpcEventSendProtocolTests
|
||||||
|
{
|
||||||
|
// GrpcHistoryClient.RegisterTags.tagInfos captured from the native event connection: the packet
|
||||||
|
// header 50 67 02 00 + count(1) + the 16-byte CM_EVENT tag GUID.
|
||||||
|
private const string CapturedRegisterTagsHex =
|
||||||
|
"506702000100000045813b35f05d464da253871aef49b321";
|
||||||
|
|
||||||
|
// GrpcHistoryClient.EnsureTags.tagInfos captured from the native event connection (86 bytes): the
|
||||||
|
// 8-byte EnsureTags header + CM_EVENT CTagMetadata + a registration FILETIME + the …e01f2f27
|
||||||
|
// event-type GUID.
|
||||||
|
private const string CapturedEnsureTagsHex =
|
||||||
|
"4e670300010000000386000545813b35f05d464da253871aef49b321090800434d5f4556454e54090900416e45204576656e7402020100000001000000004e18d6bd4503dd0142ae595fb63b604791a5ab0be01f2f27";
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void BuildRegisterCmEventInputBuffer_MatchesNativeGrpcCapture()
|
||||||
|
{
|
||||||
|
byte[] expected = Convert.FromHexString(CapturedRegisterTagsHex);
|
||||||
|
byte[] actual = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
|
||||||
|
Assert.Equal(expected, actual);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void SerializeCmEventEnsureTagsGrpc_MatchesNativeGrpcCapture()
|
||||||
|
{
|
||||||
|
byte[] expected = Convert.FromHexString(CapturedEnsureTagsHex);
|
||||||
|
Assert.Equal(86, expected.Length);
|
||||||
|
|
||||||
|
// The only run-varying field is the registration FILETIME (the 8 bytes immediately before the
|
||||||
|
// trailing 16-byte event-type GUID). Feed the captured time back so the comparison is exact.
|
||||||
|
long filetime = BinaryPrimitives.ReadInt64LittleEndian(expected.AsSpan(expected.Length - 24, 8));
|
||||||
|
DateTime createdUtc = DateTime.FromFileTimeUtc(filetime);
|
||||||
|
|
||||||
|
byte[] actual = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(createdUtc);
|
||||||
|
Assert.Equal(expected, actual);
|
||||||
|
}
|
||||||
|
}
|
||||||
@@ -233,6 +233,37 @@ public sealed class HistorianGrpcIntegrationTests
|
|||||||
Assert.Contains(samples, s => s.NumericValue is { } v && Math.Abs(v - expected) < 0.01);
|
Assert.Contains(samples, s => s.NumericValue is { } v && Math.Abs(v - expected) < 0.01);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task SendEventAsync_OverGrpc_AcceptsEvent()
|
||||||
|
{
|
||||||
|
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
|
||||||
|
// Gated additionally on a dedicated opt-in so this WRITE test never runs by accident — it
|
||||||
|
// appends a clearly-marked test event to the server's event history. Captured 2026-06-23:
|
||||||
|
// the gRPC event send rides HistoryService.AddStreamValues with the same "OS" buffer the WCF
|
||||||
|
// path uses (HistorianEventWriteProtocol), on a v8 Event session + CM_EVENT registration.
|
||||||
|
if (string.IsNullOrWhiteSpace(host)
|
||||||
|
|| string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER"))
|
||||||
|
|| Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_SEND") is null)
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
|
||||||
|
HistorianClient client = new(BuildOptions(host));
|
||||||
|
|
||||||
|
var evt = new HistorianEvent(
|
||||||
|
Id: Guid.NewGuid(),
|
||||||
|
EventTimeUtc: DateTime.UtcNow,
|
||||||
|
ReceivedTimeUtc: DateTime.UtcNow,
|
||||||
|
Type: "SdkSendProbe",
|
||||||
|
SourceName: "SdkSendProbe",
|
||||||
|
Namespace: "SdkCapture",
|
||||||
|
RevisionVersion: 0,
|
||||||
|
Properties: new Dictionary<string, object?> { ["SdkProbeProp"] = "SdkProbeValue" });
|
||||||
|
|
||||||
|
bool sent = await client.SendEventAsync(evt, CancellationToken.None);
|
||||||
|
Assert.True(sent, "gRPC SendEvent should be accepted by the server (AddStreamValues BSuccess).");
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task ReadAggregateAsync_OverGrpc_ReturnsTimeWeightedAverageRows()
|
public async Task ReadAggregateAsync_OverGrpc_ReturnsTimeWeightedAverageRows()
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -92,8 +92,10 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
|
|||||||
return DeleteTag(managedDll, args);
|
return DeleteTag(managedDll, args);
|
||||||
case "capture-event":
|
case "capture-event":
|
||||||
return CaptureEvent(managedDll, args);
|
return CaptureEvent(managedDll, args);
|
||||||
|
case "capture-send-event":
|
||||||
|
return CaptureSendEvent(managedDll, args);
|
||||||
default:
|
default:
|
||||||
Console.Error.WriteLine($"Unknown scenario '{scenario}'. Supported: load-check, connect, capture-write, delete-tag, capture-event.");
|
Console.Error.WriteLine($"Unknown scenario '{scenario}'. Supported: load-check, connect, capture-write, delete-tag, capture-event, capture-send-event.");
|
||||||
return 1;
|
return 1;
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
@@ -603,6 +605,132 @@ namespace AVEVA.Historian.Grpc2023CaptureHarness
|
|||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Drives the native 2023 R2 client through an event SEND so the IL-rewritten GrpcClient dumps
|
||||||
|
/// the AddStreamValues.btValues (the event VTQ storage-sample buffer — resolves whether a gRPC
|
||||||
|
/// event send uses the "OS" or "ON" outer signature) AND the Event-connection EnsureTags.btTagInfos
|
||||||
|
/// (the 83-vs-86-byte CM_EVENT registration byte-diff). Opens a WRITE-ENABLED Event connection,
|
||||||
|
/// builds a clearly-marked test HistorianEvent, calls AddStreamedValue, then CloseConnection to
|
||||||
|
/// flush the queued event onto the wire. WRITES a real test event into the server's event history.
|
||||||
|
/// Run with --grpc-rewrite pointing at the instrumented copy and AVEVA_HISTORIAN_RE_CAPTURE set.
|
||||||
|
/// Usage: capture-send-event [--server <host>] [--port 32565] [--cert <host>]
|
||||||
|
/// [--event-type SdkCaptureProbe] [--flush-seconds 6]
|
||||||
|
/// </summary>
|
||||||
|
private static int CaptureSendEvent(string managedDll, string[] args)
|
||||||
|
{
|
||||||
|
Assembly asm = Assembly.LoadFrom(managedDll);
|
||||||
|
Type accessType = Req(asm, "ArchestrA.HistorianAccess");
|
||||||
|
Type connArgsType = Req(asm, "ArchestrA.HistorianConnectionArgs");
|
||||||
|
Type connModeType = Req(asm, "ArchestrA.HistorianConnectionMode");
|
||||||
|
Type connTypeType = Req(asm, "ArchestrA.HistorianConnectionType");
|
||||||
|
Type errorType = Req(asm, "ArchestrA.HistorianAccessError");
|
||||||
|
Type eventType = Req(asm, "ArchestrA.HistorianEvent");
|
||||||
|
Type propTypeEnum = Req(asm, "ArchestrA.HistorianEventPropertyType");
|
||||||
|
Type certInfoType = Req(asm, "ArchestrA.CertificateInfo");
|
||||||
|
Type secModeType = Req(asm, "ArchestrA.HistorianSecurityMode");
|
||||||
|
|
||||||
|
string server = GetOption(args, "--server") ?? Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST") ?? "localhost";
|
||||||
|
int port = int.TryParse(GetOption(args, "--port"), out int p) ? p : 32565;
|
||||||
|
string certName = GetOption(args, "--cert") ?? server;
|
||||||
|
string evtTypeName = GetOption(args, "--event-type") ?? "SdkCaptureProbe";
|
||||||
|
int flushSeconds = int.TryParse(GetOption(args, "--flush-seconds"), out int fs) ? fs : 6;
|
||||||
|
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
|
||||||
|
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
|
||||||
|
if (string.IsNullOrEmpty(user))
|
||||||
|
{
|
||||||
|
Console.Error.WriteLine("Set HISTORIAN_USER/HISTORIAN_PASSWORD.");
|
||||||
|
return 1;
|
||||||
|
}
|
||||||
|
|
||||||
|
if (string.IsNullOrWhiteSpace(Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE")))
|
||||||
|
{
|
||||||
|
string defaultCap = Path.GetFullPath(Path.Combine(
|
||||||
|
"artifacts", "reverse-engineering", "grpc-event-capture", "send-event-capture.ndjson"));
|
||||||
|
Environment.SetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE", defaultCap);
|
||||||
|
}
|
||||||
|
Console.WriteLine($"Capture sink: {Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_RE_CAPTURE")}");
|
||||||
|
|
||||||
|
object connArgs = Activator.CreateInstance(connArgsType)!;
|
||||||
|
SetProp(connArgs, "ServerName", server);
|
||||||
|
SetProp(connArgs, "TcpPort", checked((ushort)port));
|
||||||
|
SetProp(connArgs, "ConnectionMode", Enum.Parse(connModeType, "Historian"));
|
||||||
|
SetProp(connArgs, "ConnectionType", Enum.Parse(connTypeType, "Event")); // EVENT connection
|
||||||
|
SetProp(connArgs, "ReadOnly", false); // WRITE-enabled
|
||||||
|
SetProp(connArgs, "IntegratedSecurity", false);
|
||||||
|
SetProp(connArgs, "AllowUnTrustedConnection", true);
|
||||||
|
SetProp(connArgs, "UserName", user);
|
||||||
|
SetProp(connArgs, "Password", password ?? string.Empty);
|
||||||
|
object certInfo = Activator.CreateInstance(certInfoType)!;
|
||||||
|
TrySetProp(certInfo, "CertificateName", certName);
|
||||||
|
TrySetProp(certInfo, "SecurityMode", Enum.Parse(secModeType, "TransportCertificate"));
|
||||||
|
TrySetProp(connArgs, "SecurityInfo", certInfo);
|
||||||
|
|
||||||
|
object access = Activator.CreateInstance(accessType)!;
|
||||||
|
object?[] openArgs = { connArgs, Activator.CreateInstance(errorType) };
|
||||||
|
Console.WriteLine($"OpenConnection: server={server} port={port} type=Event readonly=false");
|
||||||
|
bool opened;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
opened = (bool)accessType.GetMethod("OpenConnection", new[] { connArgsType, errorType.MakeByRefType() })!
|
||||||
|
.Invoke(access, openArgs)!;
|
||||||
|
}
|
||||||
|
catch (TargetInvocationException tie)
|
||||||
|
{
|
||||||
|
Console.Error.WriteLine($"OpenConnection threw: {tie.InnerException?.GetType().Name}: {tie.InnerException?.Message}");
|
||||||
|
return 2;
|
||||||
|
}
|
||||||
|
Console.WriteLine($"OpenConnection returned: {opened} err={DescribeError(openArgs[1])}");
|
||||||
|
if (!opened) { return 2; }
|
||||||
|
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// Build a clearly-marked test event. Required: Type (≤32 chars), Id, EventTime.
|
||||||
|
object evt = Activator.CreateInstance(eventType)!;
|
||||||
|
SetProp(evt, "Type", evtTypeName);
|
||||||
|
TrySetProp(evt, "Id", Guid.NewGuid());
|
||||||
|
TrySetProp(evt, "EventTime", DateTime.SpecifyKind(DateTime.UtcNow, DateTimeKind.Utc));
|
||||||
|
TrySetProp(evt, "Namespace", "SdkCapture");
|
||||||
|
TrySetProp(evt, "Source", "SdkCaptureProbe");
|
||||||
|
|
||||||
|
// One string property to exercise the property-bag framing.
|
||||||
|
MethodInfo? addProp = eventType.GetMethods().FirstOrDefault(m =>
|
||||||
|
m.Name == "AddProperty" && m.GetParameters().Length == 4);
|
||||||
|
if (addProp != null)
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
object strEnum = Enum.Parse(propTypeEnum, "String", true);
|
||||||
|
object?[] apArgs = { "SdkProbeProp", "SdkProbeValue", strEnum, Activator.CreateInstance(errorType) };
|
||||||
|
addProp.Invoke(evt, apArgs);
|
||||||
|
Console.WriteLine($"AddProperty: err={DescribeError(apArgs[3])}");
|
||||||
|
}
|
||||||
|
catch (Exception ex) { Console.WriteLine($"AddProperty skipped: {ex.GetType().Name}"); }
|
||||||
|
}
|
||||||
|
|
||||||
|
MethodInfo addStreamed = accessType.GetMethods().First(m =>
|
||||||
|
m.Name == "AddStreamedValue" && m.GetParameters().Length == 2
|
||||||
|
&& m.GetParameters()[0].ParameterType == eventType);
|
||||||
|
object?[] asArgs = { evt, Activator.CreateInstance(errorType) };
|
||||||
|
bool sent = (bool)addStreamed.Invoke(access, asArgs)!;
|
||||||
|
Console.WriteLine($"AddStreamedValue({evtTypeName}): {sent} err={DescribeError(asArgs[1])}");
|
||||||
|
|
||||||
|
// Let the native delivery queue flush the event onto the wire (AddStreamValues).
|
||||||
|
System.Threading.Thread.Sleep(flushSeconds * 1000);
|
||||||
|
Console.WriteLine(sent ? "CAPTURE-SEND-EVENT: AddStreamedValue accepted (buffer captured on flush)" : "CAPTURE-SEND-EVENT: AddStreamedValue rejected");
|
||||||
|
return sent ? 0 : 3;
|
||||||
|
}
|
||||||
|
finally
|
||||||
|
{
|
||||||
|
try
|
||||||
|
{
|
||||||
|
// CloseConnection flushes any remaining queued values before teardown.
|
||||||
|
MethodInfo? close = accessType.GetMethod("CloseConnection", new[] { errorType.MakeByRefType() });
|
||||||
|
if (close != null) close.Invoke(access, new object?[] { Activator.CreateInstance(errorType) });
|
||||||
|
}
|
||||||
|
catch { /* best-effort */ }
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Read-only gRPC connect probe: opens a 2023 R2 Historian (mode=Historian) connection via the
|
/// Read-only gRPC connect probe: opens a 2023 R2 Historian (mode=Historian) connection via the
|
||||||
/// native client and reports the resulting connection status. Proves the mixed-mode client can
|
/// native client and reports the resulting connection status. Proves the mixed-mode client can
|
||||||
|
|||||||
Reference in New Issue
Block a user