M3 R3.1: OpenStorageConnection is a dead end (error 85); precondition is front-door RegisterTags

Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.

Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
  HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
  (the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
  drives non-streamed writes through the native C++ HistorianClient, never this op.

So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.

Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.

- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
  persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 18:51:16 -04:00
parent 78cb689bdf
commit 57b9506d01
5 changed files with 345 additions and 5 deletions
+3 -3
View File
@@ -254,8 +254,8 @@ byte-correct `AddS2` (✅). Appears-and-reads-back is environment-gated on event
| ID | Work | gRPC op | Status |
|---|---|---|---|
| R3.1 | Decode non-streamed VTQ packet | `Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End` | 🟡 **gRPC Begin/End LIVE-VERIFIED 2026-06-21; full sequence MAPPED** (WCF still blocked — D2). Live decode showed `AddNonStreamValues` reaches server `StoreNonStreamValues``\\.\pipe\aahStorageEngine\console` and fails for lack of a console session. Remaining (follow-up): `StorageService.OpenStorageConnection` handshake + `RegisterTags`, THEN the `btInput` decode. See [`revision-write-path.md`](revision-write-path.md) §R3.1. |
| R3.2 | `AddHistoricalValuesAsync` | batched begin→values→end | 🟡 unblocked architecturally; needs R3.1's two live decode loops (OpenStorageConnection handshake + `btInput` serializer) then a real `bCommit=true` write/read-back |
| R3.1 | Decode non-streamed VTQ packet | `Transaction.AddNonStreamValuesBegin/AddNonStreamValues/End` | 🟡 **gRPC Begin/End LIVE-VERIFIED 2026-06-21; precondition CORRECTED 2026-06-21** (WCF still blocked — D2). The earlier "missing console session" step was **disproved live**: `StorageService.OpenStorageConnection` returns `type=4 code=85` ("session not registered") for every param — it's the storage engine's SF/snapshot channel (separate `GrpcStorageClient`/service identity), not a front-door client op. The real precondition is **front-door `HistoryService.RegisterTags`** (RTag2-family) for the target tag — the R3.1 batch failed at `AddNonStreamValues` *because the tag wasn't registered*. Remaining: capture the regular-tag `RegisterTags` `btTagInfos` (only CM_EVENT's tag-GUID form is known) + the `AddNonStreamValues` `btInput`. See [`revision-write-path.md`](revision-write-path.md) §R3.1 follow-up. |
| R3.2 | `AddHistoricalValuesAsync` | batched begin→values→end | 🟡 architecturally unblocked; precondition now correctly identified (front-door `RegisterTags`, not `OpenStorageConnection`); needs a native gRPC capture of the regular-tag `RegisterTags` `btTagInfos` + the `AddNonStreamValues` `btInput`, then a real `bCommit=true` write/read-back |
| R3.3 | Ingest-permission validation | confirm the target accepts original-data insert (distinct from `AddS2` cache wall) | ✅ **distinct on gRPC** — Begin succeeded against a real write-enabled session (the WCF/native cache gate does not apply here) |
**Acceptance:** historical points inserted and read back. **WCF path closed (D2).** gRPC path:
@@ -330,5 +330,5 @@ event-send). M3/M4 as demand dictates.
| M0 gRPC parity + capture tooling | foundation | M | unblocks everything, Windows-free | ✅ **done** |
| M1 cheap surface | TRIVIAL/BOUNDED | ML | most remaining read/config | ✅ **done** (reachable surface; rest bounded out) |
| M2 event send | CAPTURE | SM | headline write capability | ✅ **done** |
| M3 historical writes | BOUNDED | M | backfill | 🟡 **gRPC Begin/End live-verified + full sequence mapped (2026-06-21)**; WCF blocked (D2). Follow-up: OpenStorageConnection handshake + `btInput` decode → commit+read-back |
| M3 historical writes | BOUNDED | M | backfill | 🟡 **gRPC Begin/End live-verified (2026-06-21); precondition corrected** — front-door `HistoryService.RegisterTags` (not `OpenStorageConnection`, which is the SF-channel dead end / error 85). WCF blocked (D2). Follow-up: capture regular-tag `RegisterTags` `btTagInfos` + `AddNonStreamValues` `btInput` → commit+read-back |
| M4 SF / revisions / redundancy | HARD | L×N | parity completeness | defer (R4.2 = same pipe wall) |
+57 -2
View File
@@ -115,8 +115,63 @@ ClientType, ClientVersion, ConnectionMode, ConnectionTimeout, StorageSessionId(i
Raw decode artifact: `artifacts/reverse-engineering/grpc-nonstream-decode/batch1-decode.txt`
(gitignored). Probe command: `grpc-nonstream-decode`; driver:
`HistorianGrpcRevisionProbe.ProbeNonStreamedBuffersAsync` (candidate guess-bytes live in the RE tool,
not `src/`). **Status: M3 transaction lifecycle proven; full insert blocked on the
OpenStorageConnection handshake + btInput decode — a focused follow-up, each step a live probe.**
not `src/`).
### R3.1 follow-up (2026-06-21): `OpenStorageConnection` is the WRONG precondition — error 85 = "session not registered"
The mapped sequence above named `StorageService.OpenStorageConnection` as the missing console-session
step. **A live probe (`grpc-open-storage-connection` CLI / `HistorianGrpcStorageConnectionProbe`)
disproved that.** Against the real 2023 R2 server, over a write-enabled (`0x401`) session, every
`OpenStorageConnection` attempt — sweeping `ConnectionMode` (0x401/0x402/0x1), `StorageSessionId`-in
(Open2-GUID-upper / empty), and `FreeDiskSpace` — returned the **identical** error
`84 55 00 00 00 …09 15 00 "OpenStorageConnection"` = **type 4 (CustomError, 0x80 detail flag), code
`0x55` = 85**, independent of all swept values. So it is a *structural* refusal, not a bad field.
**Decoding the refusal (two corroborating facts):**
1. **Error 85 is the generic "session not registered for this op" code.** The event read path hits the
*same* `type=4 code=85` from `GetNextEventQueryResultBuffer` when the session hasn't registered its
tag first (see `HistorianWcfEventOrchestrator` xmldoc) — the fix there is front-door `RegisterTags2`
(RTag2), NOT a storage connection.
2. **`OpenStorageConnection` is not a front-door client op.** In the 2023 R2 decompile it lives on a
**separate `GrpcStorageClient`** (`Archestra.Historian.GrpcClient`, `GrpcClientBase` with its own
`Initialize(target, port, …)` channel) and the managed `HistorianAccess` non-streamed write goes
through the **native C++ `<Module>.HistorianClient.AddNonStreamedValueAsync`**, never this gRPC op.
The `StorageService` proto is almost entirely snapshots / blocks / SF params / `SendSnapshot`
it is the **storage engine's store-and-forward / snapshot interface** (`HistorianAccess`
documents `OpenStorageConnection`/`CloseStorageConnection` as the SF-snapshot *flush*), reached on
a distinct channel under a service identity. A normal Historian client never opens it on 32565.
**Corrected required sequence — the precondition is front-door tag registration, not a storage conn:**
```
HistoryService.OpenConnection (write-enabled 0x401) ✅ have it
→ HistoryService.RegisterTags(strHandle, btTagInfos = TARGET tag) ⛔ the real missing step
(front door, string handle — the RTag2 family; same op that subscribes the event session)
→ TransactionService.AddNonStreamValuesBegin ✅ works
→ TransactionService.AddNonStreamValues(btInput) ⛔ R3.1 batch failed here precisely
BECAUSE no tag was registered for the session (StoreNonStreamValues had no tag→storage route)
→ TransactionService.AddNonStreamValuesEnd(bCommit)
```
This matches the original 2020-WCF D2 hypothesis ("what populates the session's tag working set is
likely a `RegisterTags2` call") — the gRPC front door does expose that op (`HistoryService.RegisterTags`,
in our `HistoryService.proto`).
**Remaining blockers (both need a native gRPC capture — no static shortcut, do NOT guess bytes):**
1. **`HistoryService.RegisterTags` `btTagInfos` for a *regular analog* tag.** The only known RTag2
buffer is CM_EVENT's (a built-in tag identified by a well-known 16-byte *tag*-GUID,
`0x6750` v2 + count + GUID). Regular tags expose only a uint `tagKey` + a *type*-id GUID via
`GetTagInfo` (see `ParseTagInfoRecord`) — **no per-tag GUID**, so the regular-tag registration
framing (tagKey-based vs tag-GUID-based) is uncaptured.
2. **`AddNonStreamValues` `btInput`** — still C++-built and absent from every decompile (unchanged).
Both require capturing the **native 2023 R2 gRPC client** performing a non-streamed write (it would
emit the exact `RegisterTags` `btTagInfos` + `btInput`), or decoding the C++ serializer. Probe:
`grpc-open-storage-connection` (committed, regression-safe — it opens nothing persistent and
CloseStorageConnections on success). **Status: M3 transaction lifecycle proven; the insert precondition
is now correctly identified as front-door `RegisterTags` (NOT `OpenStorageConnection`); shipping
`AddHistoricalValuesAsync` is blocked on capturing the regular-tag `RegisterTags` `btTagInfos` +
the `AddNonStreamValues` `btInput`.**
---
@@ -0,0 +1,222 @@
using System.Diagnostics;
using System.Text;
using Google.Protobuf;
using AVEVA.Historian.Client.Wcf;
using GrpcStorage = ArchestrA.Grpc.Contract.Storage;
namespace AVEVA.Historian.Client.Grpc;
/// <summary>
/// Live probe for the M3 follow-up step that the R3.1 decode pinned as the missing precondition:
/// <c>StorageService.OpenStorageConnection</c>. The R3.1 finding (see
/// <c>docs/plans/revision-write-path.md</c> §R3.1) was that <c>AddNonStreamValues</c> reaches the
/// server-side <c>CHistStorageConnection::StoreNonStreamValues</c>, which routes to the
/// <c>\\.\pipe\aahStorageEngine\console,sid(...)</c> named pipe and fails for lack of a console
/// session. <c>OpenStorageConnection</c> is the op that creates exactly that console <c>sid</c>
/// session (returning its own <c>uint</c> handle + a NEW storage-session GUID, distinct from the
/// Open2 session).
///
/// Unlike <c>AddNonStreamValues</c>, this op has NO opaque <c>btInput</c> buffer — all 12 request
/// fields are typed protobuf fields (see <c>StorageService.proto</c>). So there are no wire bytes to
/// guess; the only unknowns are the VALUES for a handful of inferable fields (ConnectionMode, the
/// in/out StorageSessionId, FreeDiskSpace, credential framing). This probe sweeps a small matrix of
/// those and reports the server's response for each, so one live run reveals which combination the
/// storage engine accepts. It writes NO historical data — on success it immediately calls
/// <c>CloseStorageConnection</c> to release the console session it opened.
/// </summary>
internal sealed class HistorianGrpcStorageConnectionProbe
{
// Native client identity constants, mirrored from HistorianNativeHandshake so the storage
// engine sees the same client fingerprint the Open2 handshake presented.
private const uint NativeClientType = 4;
private const uint NativeClientVersionInt = 999_999;
private const string EngineConsolePath = @"\\.\pipe\aahStorageEngine\console";
private readonly HistorianClientOptions _options;
public HistorianGrpcStorageConnectionProbe(HistorianClientOptions options)
{
_options = options ?? throw new ArgumentNullException(nameof(options));
}
public Task<HistorianGrpcOpenStorageConnectionResult> ProbeAsync(CancellationToken cancellationToken)
=> Task.Run(() => Probe(cancellationToken), cancellationToken);
private HistorianGrpcOpenStorageConnectionResult Probe(CancellationToken cancellationToken)
{
var result = new HistorianGrpcOpenStorageConnectionResult();
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(
connection, _options, cancellationToken,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode);
result.OpenSucceeded = true;
result.ClientHandle = session.ClientHandle;
result.StorageSessionId = session.StorageSessionId;
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
DateTime Deadline() => DateTime.UtcNow.Add(_options.RequestTimeout);
// Prime the Storage service's interface-version / session table (matches the cross-service
// GetV priming the other write paths use).
try
{
GrpcStorage.GetInterfaceVersionResponse version = storageClient.GetInterfaceVersion(
new GrpcStorage.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
result.StorageInterfaceVersion = version.UiVersion;
result.StorageInterfaceVersionError = version.UiError;
}
catch (Exception ex)
{
result.StorageInterfaceVersionException = $"{ex.GetType().Name}: {ex.Message}";
}
Process current = Process.GetCurrentProcess();
string machineName = Environment.MachineName;
string processName = string.IsNullOrEmpty(current.ProcessName) ? "AVEVA.Historian.Client" : current.ProcessName;
uint processId = checked((uint)current.Id);
string upperGuid = session.StringHandle;
// Password framing: the gRPC session is already NTLM-authenticated (ValidateClientCredential),
// so attempt 1 sends no credential (rely on the authenticated channel). If the storage engine
// demands its own credential we'll see an auth-shaped error and add a credential-bearing
// attempt next iteration. For explicit creds we still try UTF-16LE password bytes as a probe.
byte[] emptyPwd = [];
// Sweep the genuinely-uncertain fields. Order = most-likely-correct first; stop at first
// success. ConnectionMode 0x401 = write-enabled (Process|Write|IntegratedSecurity), the same
// mode Open2 used for the write session. StorageSessionId-in: the native client threads the
// Open2 storage GUID through here (in/out); empty-string is the "create fresh" fallback.
var attempts = new List<(string Label, uint ConnectionMode, string SessionIdIn, uint FreeDiskSpace, byte[] Password)>
{
("mode=0x401, sid=open2-upper", 0x401, upperGuid, 0u, emptyPwd),
("mode=0x401, sid=empty", 0x401, string.Empty, 0u, emptyPwd),
("mode=0x402, sid=open2-upper", 0x402, upperGuid, 0u, emptyPwd),
("mode=0x1, sid=open2-upper", 0x1, upperGuid, 0u, emptyPwd),
("mode=0x401, sid=open2, disk=big", 0x401, upperGuid, 0xFFFFFFFFu, emptyPwd),
};
foreach ((string label, uint mode, string sidIn, uint freeDisk, byte[] pwd) in attempts)
{
var attempt = new HistorianGrpcOpenStorageConnectionAttempt
{
Label = label,
ConnectionMode = mode,
SessionIdIn = sidIn,
};
try
{
var request = new GrpcStorage.OpenStorageConnectionRequest
{
HostName = machineName,
EnginePath = EngineConsolePath,
FreeDiskSpace = freeDisk,
ProcessName = processName,
ProcessId = processId,
UserName = _options.IntegratedSecurity ? string.Empty : _options.UserName,
Password = ByteString.CopyFrom(pwd),
PwdLength = (uint)pwd.Length,
ClientType = NativeClientType,
ClientVersion = NativeClientVersionInt,
ConnectionMode = mode,
ConnectionTimeout = (uint)Math.Max(1, _options.RequestTimeout.TotalMilliseconds),
StorageSessionId = sidIn,
};
GrpcStorage.OpenStorageConnectionResponse response = storageClient.OpenStorageConnection(
request, connection.Metadata, Deadline(), cancellationToken);
attempt.Succeeded = response.Status?.BSuccess ?? false;
attempt.NewHandle = response.Handle;
attempt.NewStorageSessionId = response.StorageSessionId;
attempt.ServerStatus = response.ServerStatus;
attempt.ConnectionTime = response.ConnectionTime;
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
attempt.ErrorHex = error.Length == 0 ? null : Convert.ToHexString(error);
attempt.ErrorPreview = DescribeError(error);
result.Attempts.Add(attempt);
if (attempt.Succeeded)
{
result.OpenStorageSucceeded = true;
result.AcceptedAttempt = label;
result.NewStorageHandle = response.Handle;
result.NewStorageSessionId = response.StorageSessionId;
// Release the console session immediately — this probe persists nothing.
try
{
GrpcStorage.CloseStorageConnectionResponse close = storageClient.CloseStorageConnection(
new GrpcStorage.CloseStorageConnectionRequest { Handle = response.Handle },
connection.Metadata, Deadline(), cancellationToken);
result.CloseSucceeded = close.Status?.BSuccess ?? false;
}
catch (Exception ex)
{
result.CloseException = $"{ex.GetType().Name}: {ex.Message}";
}
break;
}
}
catch (Exception ex)
{
attempt.Exception = $"{ex.GetType().Name}: {ex.Message}";
result.Attempts.Add(attempt);
}
}
return result;
}
/// <summary>Short printable preview of a server error buffer (status codes/messages, no secrets).</summary>
private static string? DescribeError(byte[] error)
{
if (error.Length == 0)
{
return null;
}
ReadOnlySpan<byte> preview = error.AsSpan(0, Math.Min(error.Length, 96));
var sb = new StringBuilder(preview.Length);
foreach (byte b in preview)
{
sb.Append(b is >= 0x20 and < 0x7F ? (char)b : '.');
}
return sb.ToString();
}
}
internal sealed class HistorianGrpcOpenStorageConnectionResult
{
public bool OpenSucceeded { get; set; }
public uint ClientHandle { get; set; }
public Guid StorageSessionId { get; set; }
public uint? StorageInterfaceVersion { get; set; }
public uint? StorageInterfaceVersionError { get; set; }
public string? StorageInterfaceVersionException { get; set; }
public bool OpenStorageSucceeded { get; set; }
public string? AcceptedAttempt { get; set; }
public uint NewStorageHandle { get; set; }
public string? NewStorageSessionId { get; set; }
public bool CloseSucceeded { get; set; }
public string? CloseException { get; set; }
public List<HistorianGrpcOpenStorageConnectionAttempt> Attempts { get; } = new();
}
internal sealed class HistorianGrpcOpenStorageConnectionAttempt
{
public string Label { get; set; } = "";
public uint ConnectionMode { get; set; }
public string SessionIdIn { get; set; } = "";
public bool Succeeded { get; set; }
public uint NewHandle { get; set; }
public string? NewStorageSessionId { get; set; }
public uint ServerStatus { get; set; }
public ulong ConnectionTime { get; set; }
public string? ErrorHex { get; set; }
public string? ErrorPreview { get; set; }
public string? Exception { get; set; }
}
@@ -148,6 +148,31 @@ public sealed class HistorianGrpcIntegrationTests
Assert.True(result.EndDiscardSucceeded, "AddNonStreamValuesEnd(bCommit:false) should discard cleanly.");
}
[Fact]
public async Task OpenStorageConnection_OverGrpc_RefusedAsNotRegistered()
{
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
{
return;
}
// M3 R3.1 follow-up finding (2026-06-21): StorageService.OpenStorageConnection is NOT the
// missing non-streamed-write precondition. It's the storage engine's SF/snapshot channel
// (separate GrpcStorageClient / service identity), and on the Historian front door it is
// refused with native type=4 code=85 ("session not registered") for every parameter combo —
// the same code the event read returns before RegisterTags2. The real precondition is the
// front-door HistoryService.RegisterTags (RTag2-family). See docs/plans/revision-write-path.md
// §"R3.1 follow-up". This test pins the refusal so a future server/behaviour change is noticed.
var probe = new HistorianGrpcStorageConnectionProbe(BuildOptions(host));
HistorianGrpcOpenStorageConnectionResult result = await probe.ProbeAsync(CancellationToken.None);
Assert.True(result.OpenSucceeded, "the write-enabled gRPC session itself should still open.");
Assert.False(result.OpenStorageSucceeded, "OpenStorageConnection is not a front-door client op (error 85).");
Assert.NotEmpty(result.Attempts);
Assert.All(result.Attempts, a => Assert.False(a.Succeeded));
}
private static HistorianClientOptions BuildOptions(string host)
{
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
@@ -76,6 +76,7 @@ try
"capture-tag-info" => CaptureTagInfo(args),
"grpc-revision-probe" => ProbeGrpcRevision(args),
"grpc-nonstream-decode" => ProbeGrpcNonStreamedDecode(args),
"grpc-open-storage-connection" => ProbeGrpcOpenStorageConnection(args),
_ => UnknownCommand(args[0])
};
}
@@ -3213,6 +3214,43 @@ static int WriteMarker(string[] args)
return 0;
}
static int ProbeGrpcOpenStorageConnection(string[] args)
{
// Usage: grpc-open-storage-connection <host> [port] [--tls] [--dnsid <name>]
// M3 follow-up step 1: probe StorageService.OpenStorageConnection — the missing storage-engine
// console-session precondition the R3.1 decode pinned. No btInput to guess (all 12 fields are
// typed); the probe sweeps the uncertain VALUES and reports the server response per attempt.
// Opens NOTHING persistent — on success it CloseStorageConnections immediately.
string host = args.Length > 1 ? args[1] : "localhost";
int port = args.Length > 2 && int.TryParse(args[2], out int parsedPort)
? parsedPort
: HistorianClientOptions.DefaultGrpcPort;
bool tls = HasOption(args, "--tls");
string? dnsId = GetOption(args, "--dnsid");
string? user = Environment.GetEnvironmentVariable("HISTORIAN_USER");
string? password = Environment.GetEnvironmentVariable("HISTORIAN_PASSWORD");
bool explicitCreds = !string.IsNullOrEmpty(user);
var options = new HistorianClientOptions
{
Host = host,
Port = port,
Transport = HistorianTransport.RemoteGrpc,
GrpcUseTls = tls,
AllowUntrustedServerCertificate = tls,
ServerDnsIdentity = dnsId,
IntegratedSecurity = !explicitCreds,
UserName = user ?? string.Empty,
Password = password ?? string.Empty,
};
var probe = new HistorianGrpcStorageConnectionProbe(options);
HistorianGrpcOpenStorageConnectionResult result = probe.ProbeAsync(CancellationToken.None).GetAwaiter().GetResult();
Console.WriteLine(JsonSerializer.Serialize(result, CreateJsonOptions()));
return result.OpenStorageSucceeded ? 0 : 2;
}
static int ProbeGrpcRevision(string[] args)
{
// Usage: grpc-revision-probe <host> [port] [--tls] [--dnsid <name>] [--insecure-cert]