diff --git a/docs/plans/store-forward-cache-reverse-engineering.md b/docs/plans/store-forward-cache-reverse-engineering.md index 8172570..6e3b9f8 100644 --- a/docs/plans/store-forward-cache-reverse-engineering.md +++ b/docs/plans/store-forward-cache-reverse-engineering.md @@ -1,6 +1,18 @@ # Store/Forward Cache Reverse-Engineering Plan -Last updated: 2026-05-04 +Last updated: 2026-06-21 + +> **2026-06-21 R4.3 re-scope — read this first.** The original plan below +> (2026-05-04) was written against the 2020 Net.TCP/WCF transport, before the +> 2023 R2 gRPC transport existed. Its single biggest open risk — *"is SF state +> readable via a one-shot pull, or only via a duplex push contract we'd have to +> add?"* (Q1/Q2 + §3 Step 3 + Risk 4) — is now **answered: pull, no duplex**. +> The recovered gRPC `StorageService` contract exposes SF state as plain +> request/response RPCs. The current R4.3 scope and recommended path are in +> §9 ("2026-06-21 gRPC re-scope"); the 2020-WCF body below is retained as +> background, not the recommended route. + +Original last-updated: 2026-05-04 This document plans the reverse-engineering effort needed to replace the synthesized `GetStoreForwardStatusAsync` in @@ -499,3 +511,202 @@ Explicitly not part of this plan: - Anything in the `aahClientCommon.CSFConnection.StartStoreforward` / `SetStorageStopped` / `SetTagSynchronized` write surface. + +## 9. 2026-06-21 gRPC re-scope (current R4.3 plan) + +This supersedes the recommended *route* in §2/§3/§4. The deliverable +(§1) and success criteria (§6) are unchanged. What changed is the +transport and the resolved architecture risk. + +### 9.1 What the recovered gRPC contract already gives us + +The 2023 R2 contract under `src/AVEVA.Historian.Client/Grpc/Protos/` +exposes SF state through **first-class pull RPCs** on `StorageService` +(`StorageService.proto`) — no duplex/callback contract, no native +`HISTORIAN_STORAGE_STATUS` C-struct decode: + +- `GetSFParameter(uint32 Handle, string ParameterName) + → (Status status, string ParamaterValue)` — the direct analogue of the + already-shipped `GetSystemParameter`/`GetRuntimeParameter` string-keyed + pulls. This is the primary SF-state lever: a name→value read. +- `GetRemainingSnapshotsSize(uint32 Handle) + → (Status status, uint64 SnapshotSize)` — the pending-buffer magnitude + in one call. Non-zero ⇒ data is queued (`Pending`/`DataStored=true`); + zero ⇒ drained. The cleanest single signal for the idle-vs-active split. +- `GetInfo(string Request) → (Status status, bytes info)` — generic + server info blob; a fallback if a named SF key lives here instead of in + `GetSFParameter`. +- `OpenStorageConnectionResponse.ServerStatus` (field 5) and the + `GetSnapshots`/`StartQuerySnapshot` family — secondary signals. + +`SetSFParameter` exists too but is **out of scope** (read-only mission, §8). + +The `TransactionService.ForwardSnapshot{,Begin,End}` RPCs are the SF +cache *replay/transfer* path (write-side), **not** a status read — also +out of scope here; they belong to the deferred bit-faithful SF cache work, +not to `GetStoreForwardStatusAsync`. + +### 9.2 Plumbing that already exists (reuse, don't rebuild) + +- `HistorianGrpcHandshake.OpenSession` — authenticated gRPC session + (`ValidateClientCredential` NTLM loop + Open2) yielding `ClientHandle` + (uint) + storage-session GUID. Live-verified against the 2023 R2 box. +- `HistorianGrpcStorageConnectionProbe` — already constructs a + `StorageService.StorageServiceClient`, primes `GetInterfaceVersion`, and + calls `OpenStorageConnection`/`CloseStorageConnection`. The SF-status + probe is a near-clone that swaps the `OpenStorageConnection` body for + `GetSFParameter`/`GetRemainingSnapshotsSize` calls. +- `HistorianGrpcChannelFactory` / `HistorianGrpcConnection` — channel, + metadata, deadlines. + +### 9.3 The one open risk that survives: which `Handle`? + +`GetSFParameter`/`GetRemainingSnapshotsSize` both take `uint32 Handle`. +Unknown: do they accept the **session `ClientHandle`** (from +`OpenSession`, which is cheap and unblocked), or do they require the +**storage console `Handle`** returned by `OpenStorageConnection` — which +is the D2 wall (`OpenStorageConnection` routes to the +`\\.\pipe\aahStorageEngine\console` session and is the same storage-engine +pipe that blocks revision writes)? See +[[project_roadmap_exhausted_2020wcf]] and `HistorianGrpcStorageConnectionProbe` +header. + +- **Best case:** these read-only status RPCs accept the session + `ClientHandle` (status reads shouldn't need a console writer session). + Then R4.3-over-gRPC is unblocked end-to-end and is a small, shippable + feature. +- **Worst case:** they require the `OpenStorageConnection` `Handle` ⇒ + R4.3 inherits the D2 storage-engine-pipe wall and stays blocked on the + same root cause as R4.2. Either way the probe answers it in one run. + +### 9.4 Discovery steps (execution order) + +1. **Add `grpc-sf-status-probe` to `tools/AVEVA.Historian.ReverseEngineering`** + (mirror `HistorianGrpcStorageConnectionProbe`). Against the live 2023 R2 + server it: + - opens an authenticated session, gets `ClientHandle`; + - calls `GetRemainingSnapshotsSize(ClientHandle)` and reports + `status.bSuccess` + `SnapshotSize` + any error buffer; + - sweeps `GetSFParameter(ClientHandle, name)` over a candidate + name list (`Status`, `Storing`, `Pending`, `DataStored`, + `SF.Status`, `StoreForwardStatus`, `Forward`, `CacheSize`, + `ErrorOccurred`, plus any names surfaced by Workstream A's IL of + `ConvertUnmanagedSFStorageStatusToManagedStorageStatus`); + - records which names the server accepts and the returned values. + - If every call fails with an auth/handle-shaped error, retry once + with the `OpenStorageConnection` `Handle` to disambiguate §9.3. +2. **Idle baseline first** — run against the server with SF *not* active. + Establishes the "no SF / drained" response shape (expected: + `SnapshotSize=0`, parameter reads succeed-with-defaults or + return a "not configured" sentinel). This alone may be enough to ship + an honest idle-state implementation that is strictly better than + today's hardcoded all-false synthesis (it would be *measured* false). +3. **Active-SF capture** — only if step 2 proves the read works and we + need the active-state fixtures. Force SF on the sacrificial Historian + VM (stop Runtime DB writer; let the queue spill to SF), re-run the + probe, capture the non-zero/`Storing=true` response. This is the one + invasive step and the gate on full success criteria §6.1–6.3. +4. **Map + implement** — add `GrpcGetStoreForwardStatus` to the gRPC + read orchestrator, map the probed fields onto + `HistorianStoreForwardStatus`, route `GetStoreForwardStatusAsync` + to it when `Transport == RemoteGrpc` (keep the synthesized fallback + for non-gRPC transports and for the "no SF configured" sentinel). + Add golden-byte fixtures (idle + active) and + `WcfStoreForwardStatusProtocolTests`-style parse tests. Gate the live + integration test on `HISTORIAN_GRPC_HOST`. + +### 9.5 Effort / feasibility summary + +- **Risk collapsed:** pull-vs-push (the old plan's worst risk) is settled + — it's a pull. No duplex WCF/gRPC callback contract. +- **No native struct decode:** `GetSFParameter` returns a *string*; we + skip the `HISTORIAN_STORAGE_STATUS` C-layout RE entirely (Workstream + A.2 / D.1 become "nice-to-have for field names", not blocking). +- **Reuses shipped plumbing:** session open + `StorageServiceClient` + + channel already exist and are live-verified. +- **Remaining unknowns are empirical, one probe-run each:** (a) the + accepted parameter-name vocabulary, (b) which `Handle` the status RPCs + want (§9.3 — the only thing that could re-block it), (c) the + active-SF response shape (needs the invasive force-SF step). +- **Net:** Step 1–2 are low-risk and could land a *measured* idle-state + `GetStoreForwardStatusAsync` over gRPC quickly. Steps 3–4 (full + success criteria) still need the sacrificial-VM force-SF capture and + are gated on §9.3 not landing on the D2 wall. + +### 9.6 Out of scope (unchanged from §8, restated for gRPC) + +`SetSFParameter`, `ForwardSnapshot*` (SF replay/transfer), the on-disk +cache file format, and redundant-partner SF aggregation all remain out of +scope. R4.3 is read-only status, gRPC-first. + +### 9.7 Idle-baseline run — RESULTS (2026-06-21) + +Built `HistorianGrpcStoreForwardStatusProbe` + the `grpc-sf-status-probe` +CLI command and ran it against the **live 2023 R2 server** with the +historian in its **idle / not-actively-storing** state (storage interface +v4, authenticated session opened OK). Tested both read-only (`0x402`) and +write-enabled (`0x401`) sessions. Findings, with the §9.3 handle question +**resolved**: + +1. **Direct `StorageService` SF pull RPCs are D2-gated — confirmed the + §9.3 worst-case branch.** + - `GetRemainingSnapshotsSize(session.ClientHandle)` → + `bSuccess=false`, error buffer `04 84 00 00 00` (= status `0x84` / + **132 `OperationNotEnabled`**). **Identical under `0x401` and + `0x402`** — so it is NOT the read/write connection-mode gate; the + History-session `ClientHandle` is simply not a valid handle for this + op's handle-space. + - `GetSFParameter(session.ClientHandle, )` → server-side + `RpcException(Unknown, "Exception was thrown by handler")` for **all + 16** candidate names, both session modes. + - These two ops need the **`OpenStorageConnection` console handle**, + and `OpenStorageConnection` itself fails with the storage-engine + console error (`84 55 00 00 00 01 02 00 09 15 00` + + ASCII `"OpenStorageConnection"`) — the **D2 storage-engine-pipe + wall**, the same root cause that blocks R4.2 revision writes. We + cannot obtain the console handle, so these two SF RPCs are + unreachable from a pure managed client. See + [[project_roadmap_exhausted_2020wcf]]. + +2. **One reachable session-handle lever found:** + `StatusService.GetHistorianConsoleStatus(strHandle)` **SUCCEEDS** with + the session string handle (uppercase Open2 GUID) — no console handle + needed — and returns `uiConsoleStatus = 3` at idle. This is the only + SF-adjacent signal reachable from the managed client. **Its enum + semantics are unknown** (3 = presumably "running/normal"); whether it + shifts when SF is actively storing is the open question. + +3. `StatusService.GetHistorianInfo(strHandle, btRequest)` → `bSuccess= + false` for every `btRequest` candidate (empty / `u32(0)` / ascii+utf16 + `"StoreForward"`); its request framing is not yet known. Lower-yield + than `GetHistorianConsoleStatus`; revisit only if needed. + +**Net idle-baseline conclusion.** R4.3's clean direct route +(`GetSFParameter` / `GetRemainingSnapshotsSize`) is **blocked behind the +D2 storage-engine console pipe**, exactly like R4.2 — a pure managed +client cannot open the console session those ops require. The *only* +reachable SF-adjacent signal is `GetHistorianConsoleStatus` → a status +uint. Two paths forward: + +- **(a) Ship a measured idle-state only. — SHIPPED + LIVE-VERIFIED 2026-06-21.** + `HistorianGrpcStatusClient.GetStoreForwardStatusAsync` opens a session, + calls `GetHistorianConsoleStatus`, and returns + `HistorianStoreForwardStatus` all-false but *measured*: it actually + contacts the server and reports `ErrorOccurred=true` (with the underlying + error) when the server is unreachable / the console-status call fails — + strictly better than the blind hardcoded synthesis, which never contacts + the server. Routed via `Historian2020ProtocolDialect.GetStoreForwardStatusAsync` + when `Transport == RemoteGrpc` (non-gRPC keeps the synthesized fallback). + Gated live test `HistorianGrpcIntegrationTests.GetStoreForwardStatusAsync_OverGrpc_ReturnsMeasuredIdleState` + passes against the real 2023 R2 server. `Storing`/`Pending`/`DataStored` + magnitude is intentionally NOT surfaced — it lives behind the D2 wall (see + path (b)). +- **(b) Full success criteria (§6) stay blocked** on the D2 console-pipe + wall. Decoding the active-SF `uiConsoleStatus` value and any + `GetSystemParameter` SF keys still needs the invasive force-SF capture + on a sacrificial Historian — and even then `Storing`/`DataStored` + magnitude is only available via the D2-gated `GetRemainingSnapshotsSize`. + +Probe code: `src/AVEVA.Historian.Client/Grpc/HistorianGrpcStoreForwardStatusProbe.cs`, +CLI `grpc-sf-status-probe [port] [--tls] [--dnsid ] [--write-session]`. +Writes nothing; releases any console session immediately. diff --git a/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStoreForwardStatusProbe.cs b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStoreForwardStatusProbe.cs new file mode 100644 index 0000000..9eba9ee --- /dev/null +++ b/src/AVEVA.Historian.Client/Grpc/HistorianGrpcStoreForwardStatusProbe.cs @@ -0,0 +1,364 @@ +using System.Text; +using Google.Protobuf; +using AVEVA.Historian.Client.Wcf; +using GrpcStorage = ArchestrA.Grpc.Contract.Storage; +using GrpcStatus = ArchestrA.Grpc.Contract.Status; + +namespace AVEVA.Historian.Client.Grpc; + +/// +/// R4.3 discovery probe (see docs/plans/store-forward-cache-reverse-engineering.md §9). +/// Reads store-forward (SF) state from the 2023 R2 StorageService via the recovered PULL +/// RPCs — no duplex/callback contract, no native HISTORIAN_STORAGE_STATUS struct decode: +/// +/// GetRemainingSnapshotsSize(Handle) → uint64 SnapshotSize — the pending-buffer +/// magnitude in one call (non-zero ⇒ data queued). +/// GetSFParameter(Handle, ParameterName) → string — the string-keyed SF state read, +/// the analogue of the already-shipped GetSystemParameter. +/// +/// The one surviving unknown (§9.3) is which uint Handle these RPCs want: the cheap session +/// ClientHandle (unblocked) or the OpenStorageConnection console handle (the D2 +/// storage-engine-pipe wall). This probe tries the session handle FIRST and, only if those calls +/// fail handle-shaped, falls back to opening a storage console session to disambiguate — releasing +/// it immediately. It writes NOTHING. +/// +internal sealed class HistorianGrpcStoreForwardStatusProbe +{ + /// Candidate SF parameter names swept through GetSFParameter. Derived from the + /// managed HistorianStoreForwardStatus fields + the native SF getter vocabulary; the + /// server reveals which it accepts. + private static readonly string[] CandidateParameterNames = + [ + "Status", "Storing", "Pending", "DataStored", "ErrorOccurred", "Error", + "SFStatus", "SF.Status", "StoreForward", "StoreForwardStatus", "Forward", + "ForwardingStatus", "CacheSize", "SnapshotSize", "RemainingSize", "Enabled", + ]; + + private readonly HistorianClientOptions _options; + private readonly bool _writeEnabledSession; + + public HistorianGrpcStoreForwardStatusProbe(HistorianClientOptions options, bool writeEnabledSession = false) + { + _options = options ?? throw new ArgumentNullException(nameof(options)); + _writeEnabledSession = writeEnabledSession; + } + + public Task ProbeAsync(CancellationToken cancellationToken) + => Task.Run(() => Probe(cancellationToken), cancellationToken); + + private HistorianGrpcStoreForwardStatusProbeResult Probe(CancellationToken cancellationToken) + { + var result = new HistorianGrpcStoreForwardStatusProbeResult(); + + using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options); + // The idle probe found GetRemainingSnapshotsSize returns err 132 OperationNotEnabled under a + // read-only session — the same 0x402-vs-0x401 gate the write paths flip. So allow opening the + // session write-enabled to confirm the op succeeds when enabled. + uint connectionMode = _writeEnabledSession + ? HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode + : HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode; + HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession( + connection, _options, cancellationToken, connectionMode); + + result.OpenSucceeded = true; + result.WriteEnabledSession = _writeEnabledSession; + result.ClientHandle = session.ClientHandle; + result.StringHandle = session.StringHandle; + + var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel); + DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout); + + // Session-handle StatusService route (the old plan's Q2): GetHistorianConsoleStatus + + // GetHistorianInfo take the STRING handle, so they're NOT gated on the OpenStorageConnection + // console handle (the D2 wall). This is the most promising idle-baseline lever. + ProbeStatusService(result, connection, Deadline, session, cancellationToken); + + // Prime the Storage service's interface-version / session table. + try + { + GrpcStorage.GetInterfaceVersionResponse version = storageClient.GetInterfaceVersion( + new GrpcStorage.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken); + result.StorageInterfaceVersion = version.UiVersion; + result.StorageInterfaceVersionError = version.UiError; + } + catch (Exception ex) + { + result.StorageInterfaceVersionException = $"{ex.GetType().Name}: {ex.Message}"; + } + + // Phase 1: try the cheap session ClientHandle (best case — status reads shouldn't need a + // console writer session). + result.SessionHandleAttempt = QueryWithHandle( + storageClient, connection, Deadline, session.ClientHandle, "session.ClientHandle", cancellationToken); + + // Phase 2 (disambiguation, §9.3): only if every Phase-1 call failed, try the + // OpenStorageConnection console handle to learn whether SF reads are gated on the D2 wall. + if (!result.SessionHandleAttempt.AnySucceeded) + { + TryConsoleHandleFallback(result, storageClient, connection, Deadline, session, cancellationToken); + } + + return result; + } + + private static HistorianGrpcSfHandleAttempt QueryWithHandle( + GrpcStorage.StorageService.StorageServiceClient storageClient, + HistorianGrpcConnection connection, + Func deadline, + uint handle, + string handleLabel, + CancellationToken cancellationToken) + { + var attempt = new HistorianGrpcSfHandleAttempt { HandleLabel = handleLabel, Handle = handle }; + + // GetRemainingSnapshotsSize — the single cleanest pending/idle signal. + try + { + GrpcStorage.GetRemainingSnapshotsSizeResponse resp = storageClient.GetRemainingSnapshotsSize( + new GrpcStorage.GetRemainingSnapshotsSizeRequest { Handle = handle }, + connection.Metadata, deadline(), cancellationToken); + byte[] err = resp.Status?.BtError?.ToByteArray() ?? []; + attempt.RemainingSnapshotsSizeSucceeded = resp.Status?.BSuccess ?? false; + attempt.RemainingSnapshotsSize = resp.SnapshotSize; + attempt.RemainingSnapshotsSizeError = DescribeError(err); + attempt.RemainingSnapshotsSizeErrorHex = err.Length == 0 ? null : Convert.ToHexString(err); + } + catch (Exception ex) + { + attempt.RemainingSnapshotsSizeException = $"{ex.GetType().Name}: {ex.Message}"; + } + + // Sweep GetSFParameter over the candidate name vocabulary. + foreach (string name in CandidateParameterNames) + { + var pr = new HistorianGrpcSfParameterResult { Name = name }; + try + { + GrpcStorage.GetSFParameterResponse resp = storageClient.GetSFParameter( + new GrpcStorage.GetSFParameterRequest { Handle = handle, ParameterName = name }, + connection.Metadata, deadline(), cancellationToken); + pr.Succeeded = resp.Status?.BSuccess ?? false; + pr.Value = resp.ParamaterValue; + pr.Error = DescribeError(resp.Status?.BtError?.ToByteArray() ?? []); + } + catch (Exception ex) + { + pr.Exception = $"{ex.GetType().Name}: {ex.Message}"; + } + attempt.Parameters.Add(pr); + } + + return attempt; + } + + private void TryConsoleHandleFallback( + HistorianGrpcStoreForwardStatusProbeResult result, + GrpcStorage.StorageService.StorageServiceClient storageClient, + HistorianGrpcConnection connection, + Func deadline, + HistorianGrpcHandshake.Session session, + CancellationToken cancellationToken) + { + result.ConsoleHandleFallbackAttempted = true; + try + { + var request = new GrpcStorage.OpenStorageConnectionRequest + { + HostName = Environment.MachineName, + EnginePath = @"\\.\pipe\aahStorageEngine\console", + FreeDiskSpace = 0, + ProcessName = "AVEVA.Historian.Client", + ProcessId = (uint)Environment.ProcessId, + UserName = _options.IntegratedSecurity ? string.Empty : _options.UserName, + Password = ByteString.Empty, + PwdLength = 0, + ClientType = 4, + ClientVersion = 999_999, + ConnectionMode = HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode, + ConnectionTimeout = (uint)Math.Max(1, _options.RequestTimeout.TotalMilliseconds), + StorageSessionId = session.StringHandle, + }; + GrpcStorage.OpenStorageConnectionResponse open = storageClient.OpenStorageConnection( + request, connection.Metadata, deadline(), cancellationToken); + + byte[] openErr = open.Status?.BtError?.ToByteArray() ?? []; + result.OpenStorageConnectionSucceeded = open.Status?.BSuccess ?? false; + result.OpenStorageConnectionError = DescribeError(openErr); + result.OpenStorageConnectionErrorHex = openErr.Length == 0 ? null : Convert.ToHexString(openErr); + + if (result.OpenStorageConnectionSucceeded) + { + result.ConsoleHandleAttempt = QueryWithHandle( + storageClient, connection, deadline, open.Handle, "OpenStorageConnection.Handle", cancellationToken); + + try + { + storageClient.CloseStorageConnection( + new GrpcStorage.CloseStorageConnectionRequest { Handle = open.Handle }, + connection.Metadata, deadline(), cancellationToken); + } + catch (Exception ex) + { + result.CloseStorageConnectionException = $"{ex.GetType().Name}: {ex.Message}"; + } + } + } + catch (Exception ex) + { + result.OpenStorageConnectionException = $"{ex.GetType().Name}: {ex.Message}"; + } + } + + private void ProbeStatusService( + HistorianGrpcStoreForwardStatusProbeResult result, + HistorianGrpcConnection connection, + Func deadline, + HistorianGrpcHandshake.Session session, + CancellationToken cancellationToken) + { + var status = new HistorianGrpcSfStatusServiceProbe(); + var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel); + string strHandle = session.StringHandle; + + // GetHistorianConsoleStatus(strHandle) → uiConsoleStatus. The "console" is the storage-engine + // console where SF lives; this uint status may encode the SF/storing state. + try + { + GrpcStatus.GetHistorianConsoleStatusResponse resp = statusClient.GetHistorianConsoleStatus( + new GrpcStatus.GetHistorianConsoleStatusRequest { StrHandle = strHandle }, + connection.Metadata, deadline(), cancellationToken); + byte[] err = resp.Status?.BtError?.ToByteArray() ?? []; + status.ConsoleStatusSucceeded = resp.Status?.BSuccess ?? false; + status.ConsoleStatusValue = resp.UiConsoleStatus; + status.ConsoleStatusError = DescribeError(err); + status.ConsoleStatusErrorHex = err.Length == 0 ? null : Convert.ToHexString(err); + } + catch (Exception ex) + { + status.ConsoleStatusException = $"{ex.GetType().Name}: {ex.Message}"; + } + + // GetHistorianInfo(strHandle, btRequest) → btHistorianInfo. btRequest framing is unknown; try a + // small set of candidates and report whichever the server accepts + the returned blob (hex). + var infoCandidates = new List<(string Label, byte[] Request)> + { + ("empty", []), + ("u32(0)", [0, 0, 0, 0]), + ("ascii:StoreForward", Encoding.ASCII.GetBytes("StoreForward")), + ("utf16:StoreForward", Encoding.Unicode.GetBytes("StoreForward")), + }; + foreach ((string label, byte[] request) in infoCandidates) + { + var info = new HistorianGrpcSfHistorianInfoResult { Label = label }; + try + { + GrpcStatus.GetHistorianInfoResponse resp = statusClient.GetHistorianInfo( + new GrpcStatus.GetHistorianInfoRequest { StrHandle = strHandle, BtRequest = ByteString.CopyFrom(request) }, + connection.Metadata, deadline(), cancellationToken); + byte[] blob = resp.BtHistorianInfo?.ToByteArray() ?? []; + byte[] err = resp.Status?.BtError?.ToByteArray() ?? []; + info.Succeeded = resp.Status?.BSuccess ?? false; + info.InfoLength = blob.Length; + info.InfoHex = blob.Length == 0 ? null : Convert.ToHexString(blob.AsSpan(0, Math.Min(blob.Length, 256))); + info.InfoText = DescribeError(blob); + info.Error = DescribeError(err); + } + catch (Exception ex) + { + info.Exception = $"{ex.GetType().Name}: {ex.Message}"; + } + status.HistorianInfo.Add(info); + } + + result.StatusService = status; + } + + /// Short printable preview of a server error buffer (status text only, no secrets). + private static string? DescribeError(byte[] error) + { + if (error.Length == 0) + { + return null; + } + + ReadOnlySpan preview = error.AsSpan(0, Math.Min(error.Length, 96)); + var sb = new StringBuilder(preview.Length); + foreach (byte b in preview) + { + sb.Append(b is >= 0x20 and < 0x7F ? (char)b : '.'); + } + return sb.ToString(); + } +} + +internal sealed class HistorianGrpcStoreForwardStatusProbeResult +{ + public bool OpenSucceeded { get; set; } + public bool WriteEnabledSession { get; set; } + public uint ClientHandle { get; set; } + public string? StringHandle { get; set; } + public uint? StorageInterfaceVersion { get; set; } + public uint? StorageInterfaceVersionError { get; set; } + public string? StorageInterfaceVersionException { get; set; } + + public HistorianGrpcSfStatusServiceProbe? StatusService { get; set; } + + public HistorianGrpcSfHandleAttempt? SessionHandleAttempt { get; set; } + + public bool ConsoleHandleFallbackAttempted { get; set; } + public bool OpenStorageConnectionSucceeded { get; set; } + public string? OpenStorageConnectionError { get; set; } + public string? OpenStorageConnectionErrorHex { get; set; } + public string? OpenStorageConnectionException { get; set; } + public HistorianGrpcSfHandleAttempt? ConsoleHandleAttempt { get; set; } + public string? CloseStorageConnectionException { get; set; } +} + +internal sealed class HistorianGrpcSfStatusServiceProbe +{ + public bool ConsoleStatusSucceeded { get; set; } + public uint ConsoleStatusValue { get; set; } + public string? ConsoleStatusError { get; set; } + public string? ConsoleStatusErrorHex { get; set; } + public string? ConsoleStatusException { get; set; } + + public List HistorianInfo { get; } = new(); +} + +internal sealed class HistorianGrpcSfHistorianInfoResult +{ + public string Label { get; set; } = ""; + public bool Succeeded { get; set; } + public int InfoLength { get; set; } + public string? InfoHex { get; set; } + public string? InfoText { get; set; } + public string? Error { get; set; } + public string? Exception { get; set; } +} + +internal sealed class HistorianGrpcSfHandleAttempt +{ + public string HandleLabel { get; set; } = ""; + public uint Handle { get; set; } + + public bool RemainingSnapshotsSizeSucceeded { get; set; } + public ulong RemainingSnapshotsSize { get; set; } + public string? RemainingSnapshotsSizeError { get; set; } + public string? RemainingSnapshotsSizeErrorHex { get; set; } + public string? RemainingSnapshotsSizeException { get; set; } + + public List Parameters { get; } = new(); + + /// True when any pull RPC (size or a parameter) returned bSuccess for this handle. + public bool AnySucceeded => + RemainingSnapshotsSizeSucceeded || Parameters.Exists(static p => p.Succeeded); +} + +internal sealed class HistorianGrpcSfParameterResult +{ + public string Name { get; set; } = ""; + public bool Succeeded { get; set; } + public string? Value { get; set; } + public string? Error { get; set; } + public string? Exception { get; set; } +} diff --git a/tools/AVEVA.Historian.ReverseEngineering/Program.cs b/tools/AVEVA.Historian.ReverseEngineering/Program.cs index f93f3b1..62bbdda 100644 --- a/tools/AVEVA.Historian.ReverseEngineering/Program.cs +++ b/tools/AVEVA.Historian.ReverseEngineering/Program.cs @@ -78,6 +78,7 @@ try "grpc-revision-probe" => ProbeGrpcRevision(args), "grpc-nonstream-decode" => ProbeGrpcNonStreamedDecode(args), "grpc-open-storage-connection" => ProbeGrpcOpenStorageConnection(args), + "grpc-sf-status-probe" => ProbeGrpcStoreForwardStatus(args), _ => UnknownCommand(args[0]) }; } @@ -3386,6 +3387,48 @@ static int ProbeGrpcOpenStorageConnection(string[] args) return result.OpenStorageSucceeded ? 0 : 2; } +static int ProbeGrpcStoreForwardStatus(string[] args) +{ + // Usage: grpc-sf-status-probe [port] [--tls] [--dnsid ] + // R4.3 idle-baseline discovery (see docs/plans/store-forward-cache-reverse-engineering.md §9). + // Reads store-forward state via StorageService PULL RPCs (GetRemainingSnapshotsSize + sweep of + // GetSFParameter). Tries the cheap session ClientHandle first; falls back to an + // OpenStorageConnection console handle only if that fails, to disambiguate §9.3. Writes NOTHING. + // Reads HISTORIAN_USER / HISTORIAN_PASSWORD for explicit creds; else IntegratedSecurity. + string host = args.Length > 1 ? args[1] : "localhost"; + int port = args.Length > 2 && int.TryParse(args[2], out int parsedPort) + ? parsedPort + : HistorianClientOptions.DefaultGrpcPort; + bool tls = HasOption(args, "--tls"); + string? dnsId = GetOption(args, "--dnsid"); + + string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER"); + string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD"); + bool explicitCreds = !string.IsNullOrEmpty(user); + + var options = new HistorianClientOptions + { + Host = host, + Port = port, + Transport = HistorianTransport.RemoteGrpc, + GrpcUseTls = tls, + AllowUntrustedServerCertificate = tls, + ServerDnsIdentity = dnsId, + IntegratedSecurity = !explicitCreds, + UserName = user ?? string.Empty, + Password = password ?? string.Empty, + }; + + bool writeSession = HasOption(args, "--write-session"); + var probe = new HistorianGrpcStoreForwardStatusProbe(options, writeSession); + HistorianGrpcStoreForwardStatusProbeResult result = probe.ProbeAsync(CancellationToken.None).GetAwaiter().GetResult(); + Console.WriteLine(JsonSerializer.Serialize(result, CreateJsonOptions())); + + bool anySucceeded = (result.SessionHandleAttempt?.AnySucceeded ?? false) + || (result.ConsoleHandleAttempt?.AnySucceeded ?? false); + return anySucceeded ? 0 : 2; +} + static int ProbeGrpcRevision(string[] args) { // Usage: grpc-revision-probe [port] [--tls] [--dnsid ] [--insecure-cert]