Merge grpc-event-v8: v8 Event-connection ExchangeKey auth (ECDH+SHA256+MD5-keyed RC4) live-verified; row retrieval proven connection-gated
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
@@ -139,3 +139,203 @@ as the captured-correct request format** for when the open is rebuilt.
|
|||||||
|
|
||||||
Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-event-capture/` —
|
Capture artifacts (gitignored): `artifacts/reverse-engineering/grpc-event-capture/` —
|
||||||
`event-capture.ndjson` (Event), `process-connect-2.ndjson` (Process).
|
`event-capture.ndjson` (Event), `process-connect-2.ndjson` (Process).
|
||||||
|
|
||||||
|
## v8 `openParameters` fully decoded (2026-06-23) + the ECDH ExchangeKey finding
|
||||||
|
|
||||||
|
Full byte map of the native Event-connection `openParameters` (302 bytes; identity values
|
||||||
|
redacted — they are session-specific and sit in the gitignored capture):
|
||||||
|
|
||||||
|
```
|
||||||
|
[0] byte 0x08 format version = 8
|
||||||
|
[1] byte 0xf0 constant marker
|
||||||
|
[2..20] 19 × 0x00
|
||||||
|
[21] byte 0x01 constant marker
|
||||||
|
[22..37] 16B GUID per-session client key
|
||||||
|
[38..41] u32 username length (chars)
|
||||||
|
[42..N] UTF-16 username (HistorianString)
|
||||||
|
[..+1] u16 credential-token length (= 26 in the capture)
|
||||||
|
[..] 26B token ECDH-derived credential token <-- see below
|
||||||
|
[94] byte 0x04 ClientType (= our NativeClientType 4)
|
||||||
|
[95] byte ConnectionType 01 = Event / 02 = Process <-- THE GATE
|
||||||
|
[96] byte flag 01 (Event) / 00 (Process)
|
||||||
|
[97..] control bytes (0x03 ... small region, not fully named)
|
||||||
|
[~114..117]u32 FormatVersion=3
|
||||||
|
[..] HistorianString machine/server node name
|
||||||
|
[..] HistorianString client node name "(<ver>)"
|
||||||
|
[..] u32 session-variable (process-ish)
|
||||||
|
[..] u32 / zeros
|
||||||
|
[..] u32 datasource len
|
||||||
|
[..] UTF-16 datasource id e.g. "2023.1219.4004.5"
|
||||||
|
[270..285] 16 × 0xff ShardId (all-FF = unset; our v6 sends Empty)
|
||||||
|
[286..289] u32 client/hcal version int
|
||||||
|
[290..297] i64 FILETIME ClientTimestamp
|
||||||
|
[298..301] u32 0
|
||||||
|
```
|
||||||
|
|
||||||
|
The tail (`FormatVersion` → machine → clientNode → datasource → ShardId → version → timestamp)
|
||||||
|
is the **same `ClientCommonInfo` our v6 already emits**. The new/different parts are: version byte,
|
||||||
|
the `[1]`/`[21]` markers, the GUID position, the **26-byte credential token** (vs v6's fixed-size
|
||||||
|
block), the **`ConnectionType` byte**, and ShardId=FF.
|
||||||
|
|
||||||
|
**The auth is ECDH, not Negotiate.** The capture's `ExchangeKey` buffers begin `45 43 4b 31` =
|
||||||
|
ASCII **`"ECK1"`** + a 64-byte EC public-key point — a Diffie-Hellman key exchange — and the 26-byte
|
||||||
|
`openParameters` token is derived from it. `HistorianSecurityMode` offers only `Disabled` / `None` /
|
||||||
|
`TransportCertificate`; the harness used `TransportCertificate`, which is what drives the ECDH
|
||||||
|
`ExchangeKey`. There is **no TLS+Negotiate mode** on the native client (it couples TLS with the cert
|
||||||
|
ECDH path), so a Negotiate-auth v8 capture cannot be produced from the native client.
|
||||||
|
|
||||||
|
**Key de-risking insight:** our SDK's v6 `OpenConnection` sends a **fully zeroed** 1026-byte
|
||||||
|
credential block (`credentialBlock: new byte[1026]`) and reads still work — because authentication is
|
||||||
|
actually carried by the separate `StorageService.ValidateClientCredential` (Negotiate) handshake, not
|
||||||
|
by the bytes inside `openParameters`. By analogy the v8 `[68..93]` token may likewise be **ignorable**
|
||||||
|
once `ValidateClientCredential` has run. So the first build hypothesis (cheapest, read-only to test):
|
||||||
|
|
||||||
|
> Reuse the SDK's existing `ValidateClientCredential` handshake, then send a **v8 `OpenConnection`
|
||||||
|
> with `ConnectionType=Event` and a zeroed credential token**, and see whether the 2023 R2 server
|
||||||
|
> returns event rows.
|
||||||
|
|
||||||
|
If that works, the ECDH ExchangeKey RE is unnecessary. If it fails, the fallback is full reproduction
|
||||||
|
of the ECDH `ExchangeKey` handshake (curve/KDF/cipher) — a much larger crypto-RE effort. Build path:
|
||||||
|
add `SerializeNativeOpenConnectionVersion8(connectionType)` to `HistorianOpen2Protocol`, wire the gRPC
|
||||||
|
event handshake to use it (events only; reads stay on v6), live-test (non-destructive). Full hex in
|
||||||
|
the gitignored capture.
|
||||||
|
|
||||||
|
### Path A built + live-tested 2026-06-23 — DISPROVEN (v8 is coupled to ExchangeKey)
|
||||||
|
|
||||||
|
Built `HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8` (golden-tested,
|
||||||
|
`Version8EventSerializerReproducesCapturedNativeStructure` — reproduces the captured 302-byte
|
||||||
|
structure exactly) + `HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request` (zeroed
|
||||||
|
credential token) + an `eventConnection` switch on `HistorianGrpcHandshake.OpenSession`, and live-ran
|
||||||
|
the event read against the server. Result: the v8 `OpenConnection` was **parsed by the server** (got
|
||||||
|
past the byte format) but **rejected at the auth check** with native error
|
||||||
|
|
||||||
|
```
|
||||||
|
type=132 code=34 "aahHcapLib::HistoryService::EstablishConnection — Failed to get client key"
|
||||||
|
```
|
||||||
|
|
||||||
|
i.e. `EstablishConnection` could not find a server-side **client key** for our session. In the v6
|
||||||
|
path that key is established by `StorageService.ValidateClientCredential` (which is why v6 reads
|
||||||
|
work); the v8 path looks it up in the registry that **`HistoryService.ExchangeKey` (ECDH)** populates,
|
||||||
|
and there is **no `ValidateClientCredential` on `HistoryService`** in the gRPC contract. So the server
|
||||||
|
branches on the OpenConnection version: v6 accepts the Negotiate-established key, **v8 requires the
|
||||||
|
ExchangeKey-established key**. The zeroed-token hypothesis is therefore disproven — not because of the
|
||||||
|
token bytes, but because the whole v8 path is gated on `ExchangeKey` having run first.
|
||||||
|
|
||||||
|
**Status:** the v8 serializer/builder are correct and retained (golden-tested), plus the
|
||||||
|
`OpenConnection` failure now decodes the native error (type/code/ASCII). The event orchestrator is
|
||||||
|
reverted to the v6 session (gated test still pins the no-row throw). The remaining route is **Path B:
|
||||||
|
implement `HistoryService.ExchangeKey`** — `"ECK1"` + a 64-byte EC public-key point (P-256 X‖Y, by the
|
||||||
|
size) — using .NET `ECDiffieHellman`, establish the client key, then reissue the v8 `OpenConnection`.
|
||||||
|
Open question for Path B: whether merely *completing* the ECDH key agreement registers the client key
|
||||||
|
(so the zeroed openParameters token still rides through), or whether the token must also be derived
|
||||||
|
from the shared secret (full KDF/cipher RE).
|
||||||
|
|
||||||
|
### Path B started 2026-06-23 — ExchangeKey ECDH works; cleared 2 of 3 layers
|
||||||
|
|
||||||
|
Implemented `HistoryService.ExchangeKey` as a **pure-managed P-256 ECDH** key exchange
|
||||||
|
(`HistorianNativeHandshake.BuildExchangeKeyClientHello` / `DeriveExchangeKeySecret`, .NET
|
||||||
|
`ECDiffieHellman` over `nistP256`; wire format `"ECK1" + u32(32) + X(32) + Y(32)`) and wired it into
|
||||||
|
`HistorianGrpcHandshake.OpenSession(eventConnection: true)` ahead of the v8 `OpenConnection`,
|
||||||
|
on the same context-key handle. Live result against the server: the **`ExchangeKey` RPC succeeds**
|
||||||
|
(the server accepted our public key), and the v8 `OpenConnection` error **moved one layer deeper**:
|
||||||
|
|
||||||
|
```
|
||||||
|
Path A (no ExchangeKey): 132/34 "Failed to get client key"
|
||||||
|
Path B (ExchangeKey ECDH): 132/171 AuthenticationFailed "EstablishConnection — Authentication failed"
|
||||||
|
```
|
||||||
|
|
||||||
|
So the ECDH cleared the client-key check; the remaining blocker is **authentication**: the 26-byte
|
||||||
|
v8 credential token must be a *valid* value derived from the ECDH shared secret (not zeros).
|
||||||
|
|
||||||
|
### Token crypto traced 2026-06-23 (Frida → Windows CNG) — KDF found, token construction still open
|
||||||
|
|
||||||
|
Hooked Windows CNG (`bcrypt.dll`/`ncrypt.dll`) while the native harness ran a real ExchangeKey
|
||||||
|
(`scripts/frida/aahclientmanaged-cng-exchangekey.js` + `artifacts/.../cng-trace.py`). Findings:
|
||||||
|
|
||||||
|
- **The ECDH + KDF are standard CNG, driven by managed `System.Security.Cryptography.ECDiffieHellmanCng`**
|
||||||
|
(backtrace top frame = `System.Core.ni.dll`; the caller is aahClientManaged's C++/CLI `<Module>`):
|
||||||
|
`NCryptSecretAgreement` (P-256) → `NCryptDeriveKey(KDF=HASH, HASH_ALGORITHM=SHA256, 32 bytes)`. So the
|
||||||
|
derived key = **SHA256(ECDH shared secret)** — exactly `ECDiffieHellmanCng{ KeyDerivationFunction=Hash,
|
||||||
|
HashAlgorithm=SHA256 }.DeriveKeyMaterial(...)`. Our managed `DeriveExchangeKeySecret` should switch to
|
||||||
|
this (SHA256 of the raw agreement) to match.
|
||||||
|
- **`"ECK1"` is NOT AVEVA-custom** — it is the standard Windows CNG `BCRYPT_ECCPUBLIC_BLOB` magic for
|
||||||
|
P-256 (`NCryptExportKey`/`ImportKey` emit exactly `ECK1 + len(32) + X(32) + Y(32)`), confirming our
|
||||||
|
`BuildExchangeKeyClientHello` wire format is correct.
|
||||||
|
- **The 26-byte token is a custom construction that is not yet reproduced.** Correlated one run's
|
||||||
|
derived key (`SHA256(secret)`) with that run's token (from the IL openParameters capture): a
|
||||||
|
528-candidate offline cracker (HMAC/SHA/AES-GCM/CBC/CTR over the derived key × request slices ×
|
||||||
|
creds) found **no match**, and the token matches **none** of the traced hash digests. The token
|
||||||
|
starts with a constant `0x8e` marker in both captured runs (so it is structured, not raw cipher
|
||||||
|
output). It is built in managed code between the `DeriveKeyMaterial` call and the openParameters
|
||||||
|
assembly.
|
||||||
|
|
||||||
|
**dnlib IL extraction 2026-06-23 — the token scheme is fully reverse-engineered.** ILSpy can't
|
||||||
|
decompile the mixed-mode assembly (crashes), but loading `dnlib` in PowerShell and scanning the IL
|
||||||
|
recovered the whole construction:
|
||||||
|
|
||||||
|
- **`<Module>::CHistoryConnectionGrpc.GetClientKey`** is the ECDH driver: `new ECDiffieHellmanCng()`
|
||||||
|
→ `KeyDerivationFunction = Hash`, `HashAlgorithm = SHA256`, `KeySize = 256` →
|
||||||
|
`GrpcHistoryClient.ExchangeKey(strHandle, ourPubKey.ToByteArray(), out serverPub, out err)` →
|
||||||
|
`CngKey.Import(serverPub, CngKeyBlobFormat.EccPublicBlob)` → **`DeriveKeyMaterial`** = the 32-byte
|
||||||
|
client key = **`SHA256(ECDH shared secret)`**. (So our managed side should derive the key the same
|
||||||
|
way — `ECDiffieHellman` raw agreement then SHA256, or equivalently `DeriveKeyFromHash(..., SHA256)`.)
|
||||||
|
- **The 26-byte token is built by `aahClientCommon.CClientBase.ConfigureOpenConnection`** (the lone
|
||||||
|
caller of `GetClientKey`) using the **`HistorianCrypto.NRC4_V2.aahCryptV2`** scheme — a custom
|
||||||
|
**MD5-keyed RC4 stream cipher with a version prefix**:
|
||||||
|
- `aahCryptV2.body`/`HashData` = **MD5** (verified: the IL loads MD5 round constants `0xd76aa478`…
|
||||||
|
and rotates 7/12/17/22).
|
||||||
|
- `aahCryptV2.prepare_key` = standard **RC4 KSA** seeding the 256-byte S-box from a **16-byte (MD5)**
|
||||||
|
key (`std.array<unsigned char,16>`).
|
||||||
|
- `aahCryptV2.enc_buffer` = `MD5(...)` → key, then **`rc4encrypt`** the body; `enc` prepends a
|
||||||
|
scheme **prefix** (`NRC4_V2.PrefixV2` / `InnerPrefixV2`) — the constant `0x8e` token marker.
|
||||||
|
- `from_GUID` keys the cipher from a GUID string.
|
||||||
|
|
||||||
|
So the token = `prefix + RC4(plaintext, key = MD5(keyMaterial))`, where the key material ties back to
|
||||||
|
the `SHA256(ECDH secret)` client key. **This is 100% reproducible in pure managed code** (RC4 + MD5
|
||||||
|
are ~40 lines; nothing AVEVA ships).
|
||||||
|
|
||||||
|
**Remaining to finish (next cycle):** read `ConfigureOpenConnection`'s exact wiring (which value is
|
||||||
|
MD5'd for the RC4 key, what plaintext is encrypted, the exact prefix bytes — a little more dnlib IL),
|
||||||
|
implement `aahCryptV2` (RC4+MD5+prefix) managed-side, set the v8 token = that, and live-test
|
||||||
|
(non-destructive). The offline correlation data (one run's derived key + token + openParameters) is
|
||||||
|
captured under `artifacts/.../` to validate the managed reproduction before going live.
|
||||||
|
|
||||||
|
### Token implemented + auth WORKS live (2026-06-23); row retrieval still 0 — proven NOT a payload issue
|
||||||
|
|
||||||
|
`token = RC4(password-UTF16LE, key = MD5(SHA256(ECDH secret)))` was implemented in pure managed C#
|
||||||
|
(`HistorianNativeHandshake.BuildExchangeKeyCredentialToken` + `Rc4`; client key via
|
||||||
|
`DeriveKeyFromHash(SHA256)`), golden-tested (RC4 standard vector + token construction), and
|
||||||
|
**live-verified**: the v8 `OpenConnection` now **authenticates** against the 2023 R2 server (past the
|
||||||
|
`132/171 AuthenticationFailed` wall). Auth is solved.
|
||||||
|
|
||||||
|
The event **query** still returns `version-11 rowCount-0` while the native returns 50 for an
|
||||||
|
**identical** request. Exhaustively ruled out as the cause (all confirmed live, opt-in
|
||||||
|
`EventReadDiagnostic` test + the IL rewrite extended to log string/uint handle fields):
|
||||||
|
|
||||||
|
- `StartEventQuery` request: **byte-identical** to the native (v6 layout)
|
||||||
|
- v8 `OpenConnection` `openParameters`: **byte-identical** to the native (302 bytes) once ClientNodeName
|
||||||
|
is matched — every control byte, ConnectionType, token framing, ShardId, etc.
|
||||||
|
- Handle usage: identical — `ExchangeKey`→contextKey, registration→storage-session GUID (`strHandle`),
|
||||||
|
query→client uint (`uiHandle`); our parsed handles are valid (registration `RTag/EnsT=True`, valid
|
||||||
|
`queryHandle`)
|
||||||
|
- `queryRequestType = 3`, registration sequence/order, gzip metadata header — all match
|
||||||
|
- window (events exist; native returns 50 *now*), eventCount — not it
|
||||||
|
|
||||||
|
So **every observable client-side byte matches the native**, yet the server scopes 0 events to our
|
||||||
|
connection. The event RPCs succeed over our transport and return a valid *empty* result (not a
|
||||||
|
transport error), so it is **not a payload or transport-incompatibility issue** — it is a
|
||||||
|
connection/server-level difference (e.g. session affinity tied to the native `Grpc.Core` HTTP/2
|
||||||
|
connection or a connection-identity the server uses to scope events) that is **invisible to, and
|
||||||
|
unfixable by, client payload matching.** Closing it needs server-side insight or a different angle
|
||||||
|
(e.g. compare the full HTTP/2 connection setup / TLS identity), not more wire-payload RE.
|
||||||
|
|
||||||
|
**Shipped this effort:** the complete ExchangeKey crypto (ECDH + SHA256 + MD5-keyed RC4 token) — the
|
||||||
|
hard wall — pure managed, golden-tested, auth live-verified. Orchestrator stays on the no-row throw;
|
||||||
|
gated test unchanged.
|
||||||
|
|
||||||
|
**2 of 3 layers cleared** (key exchange + client key); the 3rd (token construction) is localized to a
|
||||||
|
specific managed method, pending dnlib extraction. ExchangeKey + the v8 serializer are committed; the
|
||||||
|
orchestrator stays on v6 (set `eventConnection: true` to re-arm once the token construction lands). The
|
||||||
|
token-loop routing guardrail (`HistorianGrpcHandshakeRoutingTests`) was scoped to the closure so the
|
||||||
|
legitimate ExchangeKey call is allowed while still pinning that the Negotiate token loop never routes
|
||||||
|
there.
|
||||||
|
|||||||
@@ -0,0 +1,173 @@
|
|||||||
|
// Frida hook for the native ExchangeKey credential-token crypto (Windows CNG / bcrypt.dll).
|
||||||
|
// Traces the ECDH secret agreement, the KDF (with its parameter list), symmetric-key import, and
|
||||||
|
// encrypt/hash so the 26-byte v8 credential-token derivation can be reconstructed in managed code.
|
||||||
|
// Reverse-engineering aid only — observes the native client; nothing is shipped from here.
|
||||||
|
'use strict';
|
||||||
|
|
||||||
|
function resolve(modName, fnName) {
|
||||||
|
let m = null;
|
||||||
|
try { m = Process.getModuleByName(modName); } catch (e) {
|
||||||
|
try { m = Module.load(modName); } catch (e2) { return null; }
|
||||||
|
}
|
||||||
|
try { return m.findExportByName(fnName); } catch (e) { return null; }
|
||||||
|
}
|
||||||
|
|
||||||
|
function dump(label, ptr, len) {
|
||||||
|
if (ptr.isNull() || len <= 0) { console.log(label + ' <empty>'); return; }
|
||||||
|
const n = Math.min(len, 256);
|
||||||
|
console.log(label + ' (' + len + ' bytes)\n' + hexdump(ptr, { length: n, header: false, ansi: false }));
|
||||||
|
}
|
||||||
|
|
||||||
|
function hook(modName, fnName, onEnter, onLeave) {
|
||||||
|
const addr = resolve(modName, fnName);
|
||||||
|
if (!addr) { console.log('[skip] ' + modName + '!' + fnName + ' not found'); return; }
|
||||||
|
Interceptor.attach(addr, { onEnter: onEnter, onLeave: onLeave });
|
||||||
|
console.log('[hooked] ' + modName + '!' + fnName);
|
||||||
|
}
|
||||||
|
|
||||||
|
// BCryptOpenAlgorithmProvider(phAlgorithm, pszAlgId, pszImplementation, dwFlags) — names every algo used.
|
||||||
|
hook('bcrypt.dll', 'BCryptOpenAlgorithmProvider', function (a) {
|
||||||
|
console.log('[OpenAlgorithmProvider] algId=' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
|
||||||
|
});
|
||||||
|
|
||||||
|
// BCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, flags)
|
||||||
|
hook('bcrypt.dll', 'BCryptSecretAgreement', function (a) {
|
||||||
|
console.log('[SecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Decode a BCryptBufferDesc parameter list (used by BCryptDeriveKey) into (type -> bytes).
|
||||||
|
function dumpParamList(pParamList) {
|
||||||
|
if (pParamList.isNull()) { console.log(' paramList <null>'); return; }
|
||||||
|
const cBuffers = pParamList.add(4).readU32(); // ULONG ulVersion; ULONG cBuffers;
|
||||||
|
const pBuffers = pParamList.add(8).readPointer(); // BCryptBuffer* pBuffers;
|
||||||
|
const names = { 0: 'HASH_ALGORITHM', 1: 'SECRET_PREPEND', 2: 'SECRET_APPEND', 3: 'HMAC_KEY',
|
||||||
|
4: 'TLS_PRF_LABEL', 5: 'TLS_PRF_SEED', 6: 'SECRET_HANDLE', 8: 'SP80056A_CONCAT',
|
||||||
|
0xD: 'LABEL', 0xE: 'CONTEXT', 0xF: 'SALT', 0x10: 'ITERATION_COUNT' };
|
||||||
|
console.log(' paramList cBuffers=' + cBuffers);
|
||||||
|
for (let i = 0; i < cBuffers; i++) {
|
||||||
|
const b = pBuffers.add(i * 16); // { ULONG cbBuffer; ULONG BufferType; PVOID pvBuffer; }
|
||||||
|
const cb = b.readU32();
|
||||||
|
const type = b.add(4).readU32();
|
||||||
|
const pv = b.add(8).readPointer();
|
||||||
|
const tn = names[type] || ('0x' + type.toString(16));
|
||||||
|
if (type === 0 || type === 4 || type === 0xD) { // string-ish (hash alg name / label)
|
||||||
|
console.log(' [' + tn + '] ' + (pv.isNull() ? '?' : pv.readUtf16String()));
|
||||||
|
} else {
|
||||||
|
dump(' [' + tn + ']', pv, cb);
|
||||||
|
}
|
||||||
|
}
|
||||||
|
}
|
||||||
|
|
||||||
|
// BCryptDeriveKey(hSecret, pwszKDF, *pParamList, pbDerivedKey, cbDerivedKey, *pcbResult, flags)
|
||||||
|
hook('bcrypt.dll', 'BCryptDeriveKey', function (a) {
|
||||||
|
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
|
||||||
|
this.outKey = a[3]; this.pcb = a[5];
|
||||||
|
console.log('[DeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
|
||||||
|
dumpParamList(a[2]);
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
dump('[DeriveKey] derived', this.outKey, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
hook('bcrypt.dll', 'BCryptDeriveKeyPBKDF2', function (a) {
|
||||||
|
console.log('[PBKDF2] cbPassword=' + a[2].toInt32() + ' cbSalt=' + a[4].toInt32() + ' iter=' + a[5]);
|
||||||
|
dump(' password', a[1], a[2].toInt32());
|
||||||
|
dump(' salt', a[3], a[4].toInt32());
|
||||||
|
});
|
||||||
|
|
||||||
|
// BCryptGenerateSymmetricKey(hAlg, *phKey, pbKeyObject, cbKeyObject, pbSecret, cbSecret, flags) — the actual key bytes.
|
||||||
|
hook('bcrypt.dll', 'BCryptGenerateSymmetricKey', function (a) {
|
||||||
|
dump('[GenerateSymmetricKey] keyBytes', a[4], a[5].toInt32());
|
||||||
|
});
|
||||||
|
|
||||||
|
// BCryptEncrypt(hKey, pbIn, cbIn, *pPad, pbIV, cbIV, pbOut, cbOut, *pcbResult, flags)
|
||||||
|
hook('bcrypt.dll', 'BCryptEncrypt', function (a) {
|
||||||
|
this.out = a[6]; this.pcb = a[8];
|
||||||
|
dump('[Encrypt] plaintext', a[1], a[2].toInt32());
|
||||||
|
dump('[Encrypt] IV', a[4], a[5].toInt32());
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
dump('[Encrypt] ciphertext', this.out, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
// Hash path (in case the token is a keyed hash rather than a cipher).
|
||||||
|
hook('bcrypt.dll', 'BCryptHashData', function (a) {
|
||||||
|
dump('[HashData] input', a[1], a[2].toInt32());
|
||||||
|
});
|
||||||
|
hook('bcrypt.dll', 'BCryptFinishHash', function (a) {
|
||||||
|
this.out = a[1]; this.cb = a[2].toInt32();
|
||||||
|
}, function () {
|
||||||
|
dump('[FinishHash] digest', this.out, this.cb);
|
||||||
|
});
|
||||||
|
|
||||||
|
// ---- NCrypt (CNG key-storage layer) — the likely home of the ECDH ExchangeKey + token crypto ----
|
||||||
|
|
||||||
|
// NCryptSecretAgreement(hPrivKey, hPubKey, *phAgreedSecret, dwFlags)
|
||||||
|
hook('ncrypt.dll', 'NCryptSecretAgreement', function (a) {
|
||||||
|
console.log('[NCryptSecretAgreement] hPriv=' + a[0] + ' hPub=' + a[1]);
|
||||||
|
console.log(' backtrace (addr -> module+offset):');
|
||||||
|
Thread.backtrace(this.context, Backtracer.ACCURATE).slice(0, 14).forEach(function (addr) {
|
||||||
|
const m = Process.findModuleByAddress(addr);
|
||||||
|
if (m) {
|
||||||
|
console.log(' ' + addr + ' ' + m.name + '+0x' + addr.sub(m.base).toString(16));
|
||||||
|
} else {
|
||||||
|
console.log(' ' + addr + ' <JIT/unknown>');
|
||||||
|
}
|
||||||
|
});
|
||||||
|
});
|
||||||
|
|
||||||
|
// NCryptDeriveKey(hSharedSecret, pwszKDF, *pParameterList, pbDerivedKey, cbDerivedKey, *pcbResult, dwFlags)
|
||||||
|
hook('ncrypt.dll', 'NCryptDeriveKey', function (a) {
|
||||||
|
this.kdf = a[1].isNull() ? '?' : a[1].readUtf16String();
|
||||||
|
this.outKey = a[3]; this.pcb = a[5];
|
||||||
|
console.log('[NCryptDeriveKey] KDF=' + this.kdf + ' cbDerivedKey=' + a[4].toInt32());
|
||||||
|
dumpParamList(a[2]);
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
dump('[NCryptDeriveKey] derived', this.outKey, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
// NCryptEncrypt(hKey, pbInput, cbInput, *pPaddingInfo, pbOutput, cbOutput, *pcbResult, dwFlags)
|
||||||
|
hook('ncrypt.dll', 'NCryptEncrypt', function (a) {
|
||||||
|
this.out = a[4]; this.pcb = a[6];
|
||||||
|
dump('[NCryptEncrypt] plaintext', a[1], a[2].toInt32());
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
dump('[NCryptEncrypt] ciphertext', this.out, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
// NCryptImportKey(hProvider, hImportKey, pszBlobType, *pParameterList, *phKey, pbData, cbData, dwFlags)
|
||||||
|
hook('ncrypt.dll', 'NCryptImportKey', function (a) {
|
||||||
|
console.log('[NCryptImportKey] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()));
|
||||||
|
dump(' blob', a[5], a[6].toInt32());
|
||||||
|
});
|
||||||
|
|
||||||
|
// NCryptExportKey(hKey, hExportKey, pszBlobType, *pParameterList, pbOutput, cbOutput, *pcbResult, dwFlags)
|
||||||
|
hook('ncrypt.dll', 'NCryptExportKey', function (a) {
|
||||||
|
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
|
||||||
|
this.out = a[4]; this.pcb = a[6];
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
console.log('[NCryptExportKey] blobType=' + this.blobType);
|
||||||
|
dump(' blob', this.out, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
hook('ncrypt.dll', 'NCryptOpenStorageProvider', function (a) {
|
||||||
|
console.log('[NCryptOpenStorageProvider] ' + (a[1].isNull() ? '?' : a[1].readUtf16String()));
|
||||||
|
});
|
||||||
|
|
||||||
|
// BCrypt EC key operations (in case the ECDH is bcrypt but uses import/export rather than DeriveKey).
|
||||||
|
hook('bcrypt.dll', 'BCryptImportKeyPair', function (a) {
|
||||||
|
console.log('[BCryptImportKeyPair] blobType=' + (a[2].isNull() ? '?' : a[2].readUtf16String()) + ' cb=' + a[5].toInt32());
|
||||||
|
dump(' blob', a[4], a[5].toInt32());
|
||||||
|
});
|
||||||
|
hook('bcrypt.dll', 'BCryptExportKey', function (a) {
|
||||||
|
this.blobType = a[2].isNull() ? '?' : a[2].readUtf16String();
|
||||||
|
this.out = a[3]; this.pcb = a[5];
|
||||||
|
}, function () {
|
||||||
|
const n = this.pcb.isNull() ? 0 : this.pcb.readU32();
|
||||||
|
console.log('[BCryptExportKey] blobType=' + this.blobType);
|
||||||
|
dump(' blob', this.out, n);
|
||||||
|
});
|
||||||
|
|
||||||
|
console.log('=== CNG ExchangeKey crypto hooks installed ===');
|
||||||
@@ -58,6 +58,12 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
/// <summary>Diagnostic: type+code description of the most recent error/terminal buffer.</summary>
|
/// <summary>Diagnostic: type+code description of the most recent error/terminal buffer.</summary>
|
||||||
public string LastErrorBufferDescription { get; private set; } = string.Empty;
|
public string LastErrorBufferDescription { get; private set; } = string.Empty;
|
||||||
|
|
||||||
|
/// <summary>Diagnostic: hex of the most recent result buffer (first 48 bytes).</summary>
|
||||||
|
public string LastResultBufferHex { get; private set; } = string.Empty;
|
||||||
|
|
||||||
|
/// <summary>Diagnostic: hex of the most recent GetNext error buffer.</summary>
|
||||||
|
public string LastErrorBufferHex { get; private set; } = string.Empty;
|
||||||
|
|
||||||
public async IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
|
public async IAsyncEnumerable<HistorianEvent> ReadEventsAsync(
|
||||||
DateTime startUtc,
|
DateTime startUtc,
|
||||||
DateTime endUtc,
|
DateTime endUtc,
|
||||||
@@ -138,7 +144,10 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
|
private List<HistorianEvent> RunEventChain(DateTime startUtc, DateTime endUtc, HistorianEventFilter? filter, CancellationToken cancellationToken)
|
||||||
{
|
{
|
||||||
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
|
using HistorianGrpcConnection connection = HistorianGrpcChannelFactory.Create(_options);
|
||||||
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken);
|
// Event reads need an Event-type (v8) connection. OpenSession(eventConnection: true) runs the
|
||||||
|
// full v8 path: HistoryService.ExchangeKey (P-256 ECDH) -> client key = SHA256(secret) -> v8
|
||||||
|
// OpenConnection with ConnectionType=Event and the credential token RC4(password, MD5(clientKey)).
|
||||||
|
HistorianGrpcHandshake.Session session = HistorianGrpcHandshake.OpenSession(connection, _options, cancellationToken, eventConnection: true);
|
||||||
|
|
||||||
RegisterCmEventTag(connection, session, cancellationToken);
|
RegisterCmEventTag(connection, session, cancellationToken);
|
||||||
|
|
||||||
@@ -203,56 +212,55 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
{
|
{
|
||||||
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
|
var historyClient = new GrpcHistory.HistoryService.HistoryServiceClient(connection.Channel);
|
||||||
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
|
var statusClient = new GrpcStatus.StatusService.StatusServiceClient(connection.Channel);
|
||||||
var retrievalClient = new GrpcRetrieval.RetrievalService.RetrievalServiceClient(connection.Channel);
|
|
||||||
var transactionClient = new GrpcTransaction.TransactionService.TransactionServiceClient(connection.Channel);
|
|
||||||
|
|
||||||
// Discovery dance the native event flow runs between Open2 and EnsT2. All bounded by the
|
|
||||||
// short RegistrationDeadline (several stall server-side on the remote box).
|
|
||||||
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
|
|
||||||
byte[] historianVersionRequest = HistorianEventRegistrationProtocol.BuildGetHistorianInfoRequest("HistorianVersion");
|
|
||||||
TryRun(() => statusClient.GetHistorianInfo(
|
|
||||||
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
|
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
TryRun(() => statusClient.GetHistorianInfo(
|
|
||||||
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
|
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
|
|
||||||
|
// Native 2023 R2 gRPC event-connection registration sequence (captured order):
|
||||||
|
// UpdateClientStatus -> RegisterTags(CM_EVENT) -> EnsureTags(CM_EVENT) -> GetHistorianInfo
|
||||||
|
// -> GetSystemParameter x7. (StartEventQuery follows in RunEventQuery.) The 2020-WCF-era extra
|
||||||
|
// probes (cross-service GetV, params-before-register) are NOT in the gRPC event flow.
|
||||||
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
|
byte[] clientStatus = HistorianEventRegistrationProtocol.BuildUpdateClientStatusBlob();
|
||||||
TryRun(() => historyClient.UpdateClientStatus(
|
TryRun(() => historyClient.UpdateClientStatus(
|
||||||
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
|
new GrpcHistory.UpdateClientStatusRequest { StrHandle = session.StringHandle, BtClientStatus = ByteString.CopyFrom(clientStatus) },
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
||||||
|
|
||||||
// Records 11-16: 6 system-parameter queries before RTag2.
|
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
|
||||||
foreach (string parameterName in HistorianEventRegistrationProtocol.StatusParametersBeforeRegister)
|
try
|
||||||
|
{
|
||||||
|
GrpcHistory.RegisterTagsResponse rt = historyClient.RegisterTags(
|
||||||
|
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
|
||||||
|
connection.Metadata, RegistrationDeadline(), cancellationToken);
|
||||||
|
RegistrationDiag += $"RTag={rt.Status?.BSuccess} e={Convert.ToHexString(rt.Status?.BtError?.ToByteArray() ?? [])}; ";
|
||||||
|
}
|
||||||
|
catch (Exception ex) { RegistrationDiag += $"RTag=EX:{ex.GetType().Name}; "; }
|
||||||
|
|
||||||
|
// gRPC CM_EVENT EnsureTags uses the 86-byte native format (8-byte header + the …2f27 event-type
|
||||||
|
// GUID), NOT the 2020 WCF CTagMetadata.
|
||||||
|
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventEnsureTagsGrpc(DateTime.UtcNow);
|
||||||
|
try
|
||||||
|
{
|
||||||
|
GrpcHistory.EnsureTagsResponse et = historyClient.EnsureTags(
|
||||||
|
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
|
||||||
|
connection.Metadata, RegistrationDeadline(), cancellationToken);
|
||||||
|
RegistrationDiag += $"EnsT={et.Status?.BSuccess} e={Convert.ToHexString(et.Status?.BtError?.ToByteArray() ?? [])} out={Convert.ToHexString(et.BtTagStatus?.ToByteArray() ?? [])}; ";
|
||||||
|
}
|
||||||
|
catch (Exception ex) { RegistrationDiag += $"EnsT=EX:{ex.GetType().Name}; "; }
|
||||||
|
|
||||||
|
byte[] historianVersionRequest = HistorianEventRegistrationProtocol.BuildGetHistorianInfoRequest("HistorianVersion");
|
||||||
|
TryRun(() => statusClient.GetHistorianInfo(
|
||||||
|
new GrpcStatus.GetHistorianInfoRequest { StrHandle = session.StringHandle, BtRequest = ByteString.CopyFrom(historianVersionRequest) },
|
||||||
|
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
||||||
|
|
||||||
|
string[] eventParams = ["AllowOriginals", "HistorianPartner", "HistorianVersion", "MaxCyclicStorageTimeout", "RealTimeWindow", "FutureTimeThreshold", "AllowRenameTags"];
|
||||||
|
foreach (string parameterName in eventParams)
|
||||||
{
|
{
|
||||||
TryRun(() => statusClient.GetSystemParameter(
|
TryRun(() => statusClient.GetSystemParameter(
|
||||||
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = parameterName },
|
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = parameterName },
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
||||||
}
|
}
|
||||||
|
|
||||||
byte[] registerBuffer = HistorianEventRegistrationProtocol.BuildRegisterCmEventInputBuffer();
|
|
||||||
TryRun(() => historyClient.RegisterTags(
|
|
||||||
new GrpcHistory.RegisterTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(registerBuffer) },
|
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
|
|
||||||
// Record 18: one more system-parameter query after RTag2 before EnsT2.
|
|
||||||
TryRun(() => statusClient.GetSystemParameter(
|
|
||||||
new GrpcStatus.GetSystemParameterRequest { UiHandle = session.ClientHandle, StrParameterName = "AllowRenameTags" },
|
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
|
|
||||||
// Records 19-21: cross-service version probes between RTag2 and EnsT2 (session-table registration).
|
|
||||||
TryRun(() => transactionClient.GetTransactionInterfaceVersion(new GrpcTransaction.GetTransactionInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
TryRun(() => statusClient.GetStatusInterfaceVersion(new GrpcStatus.GetStatusInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
TryRun(() => retrievalClient.GetRetrievalInterfaceVersion(new GrpcRetrieval.GetRetrievalInterfaceVersionRequest(), connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
|
|
||||||
byte[] payload = HistorianAddTagsProtocol.SerializeCmEventCTagMetadata(DateTime.UtcNow);
|
|
||||||
TryRun(() => historyClient.EnsureTags(
|
|
||||||
new GrpcHistory.EnsureTagsRequest { StrHandle = session.StringHandle, BtTagInfos = ByteString.CopyFrom(payload), ElementCount = 1 },
|
|
||||||
connection.Metadata, RegistrationDeadline(), cancellationToken));
|
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>Diagnostic: outcomes of the key CM_EVENT registration RPCs.</summary>
|
||||||
|
public string RegistrationDiag { get; private set; } = string.Empty;
|
||||||
|
|
||||||
private List<HistorianEvent> RunEventQuery(
|
private List<HistorianEvent> RunEventQuery(
|
||||||
HistorianGrpcConnection connection,
|
HistorianGrpcConnection connection,
|
||||||
HistorianGrpcHandshake.Session session,
|
HistorianGrpcHandshake.Session session,
|
||||||
@@ -274,7 +282,7 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
IReadOnlyList<HistorianEventQueryAttempt> attempts = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
|
IReadOnlyList<HistorianEventQueryAttempt> attempts = HistorianEventQueryProtocol.CreateStartEventQueryAttempts(
|
||||||
startUtc.ToUniversalTime(),
|
startUtc.ToUniversalTime(),
|
||||||
endUtc.ToUniversalTime(),
|
endUtc.ToUniversalTime(),
|
||||||
eventCount: 5,
|
eventCount: 100,
|
||||||
filter,
|
filter,
|
||||||
version: 6);
|
version: 6);
|
||||||
byte[] requestBuffer = attempts[0].RequestBuffer;
|
byte[] requestBuffer = attempts[0].RequestBuffer;
|
||||||
@@ -298,6 +306,7 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
}
|
}
|
||||||
|
|
||||||
uint queryHandle = startResponse.UiQueryHandle;
|
uint queryHandle = startResponse.UiQueryHandle;
|
||||||
|
RegistrationDiag += $"QH={queryHandle} clientH={session.ClientHandle} strH={session.StringHandle}; ";
|
||||||
try
|
try
|
||||||
{
|
{
|
||||||
List<HistorianEvent> events = [];
|
List<HistorianEvent> events = [];
|
||||||
@@ -333,6 +342,8 @@ internal sealed class HistorianGrpcEventOrchestrator
|
|||||||
|
|
||||||
LastResultBufferLength = resultBuffer.Length;
|
LastResultBufferLength = resultBuffer.Length;
|
||||||
LastErrorBufferDescription = HistorianEventRegistrationProtocol.DescribeNativeError(errorBuffer);
|
LastErrorBufferDescription = HistorianEventRegistrationProtocol.DescribeNativeError(errorBuffer);
|
||||||
|
LastResultBufferHex = Convert.ToHexString(resultBuffer.Length <= 48 ? resultBuffer : resultBuffer[..48]);
|
||||||
|
LastErrorBufferHex = Convert.ToHexString(errorBuffer);
|
||||||
|
|
||||||
// Any 5-byte type=4 error is a soft terminal (code 30 NoMoreData is canonical; code
|
// Any 5-byte type=4 error is a soft terminal (code 30 NoMoreData is canonical; code
|
||||||
// 85 / 0x55 is the missing-registration signal seen on early runs). Mirror the WCF
|
// 85 / 0x55 is the missing-registration signal seen on early runs). Mirror the WCF
|
||||||
|
|||||||
@@ -1,3 +1,4 @@
|
|||||||
|
using System.Security.Cryptography;
|
||||||
using Google.Protobuf;
|
using Google.Protobuf;
|
||||||
using Grpc.Core;
|
using Grpc.Core;
|
||||||
using AVEVA.Historian.Client.Wcf;
|
using AVEVA.Historian.Client.Wcf;
|
||||||
@@ -20,6 +21,9 @@ namespace AVEVA.Historian.Client.Grpc;
|
|||||||
/// </summary>
|
/// </summary>
|
||||||
internal static class HistorianGrpcHandshake
|
internal static class HistorianGrpcHandshake
|
||||||
{
|
{
|
||||||
|
/// <summary>Diagnostic: hex of the most recent v8 event-connection OpenConnection request.</summary>
|
||||||
|
internal static string LastEventOpenRequestHex { get; private set; } = string.Empty;
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// The handles produced by a successful OpenConnection. <see cref="ClientHandle"/> is the
|
/// The handles produced by a successful OpenConnection. <see cref="ClientHandle"/> is the
|
||||||
/// transient <c>uint</c> session token used by StartQuery/GetSystemParameter and the other
|
/// transient <c>uint</c> session token used by StartQuery/GetSystemParameter and the other
|
||||||
@@ -50,7 +54,8 @@ internal static class HistorianGrpcHandshake
|
|||||||
HistorianGrpcConnection connection,
|
HistorianGrpcConnection connection,
|
||||||
HistorianClientOptions options,
|
HistorianClientOptions options,
|
||||||
CancellationToken cancellationToken,
|
CancellationToken cancellationToken,
|
||||||
uint connectionMode = HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode)
|
uint connectionMode = HistorianWcfAuthChainHelper.NativeIntegratedReadOnlyConnectionMode,
|
||||||
|
bool eventConnection = false)
|
||||||
{
|
{
|
||||||
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
|
DateTime Deadline() => DateTime.UtcNow.Add(options.RequestTimeout);
|
||||||
|
|
||||||
@@ -61,26 +66,65 @@ internal static class HistorianGrpcHandshake
|
|||||||
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
|
new GrpcHistory.GetInterfaceVersionRequest(), connection.Metadata, Deadline(), cancellationToken);
|
||||||
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options);
|
HistorianServerVersionGate.Validate(HistorianServiceInterface.History, historyVersion.UiVersion, options);
|
||||||
|
|
||||||
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
|
// The v6 (read/write) path authenticates via StorageService.ValidateClientCredential (Negotiate).
|
||||||
HistorianNativeHandshake.RunTokenRounds(
|
// The v8 EVENT path authenticates entirely via ExchangeKey (ECDH) + the RC4 credential token —
|
||||||
(handle, wrapped, _) =>
|
// the native client does NOT run ValidateClientCredential for an event connection, and doing so
|
||||||
{
|
// establishes a different session scope under which the event query returns zero rows. So skip it.
|
||||||
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
|
if (!eventConnection)
|
||||||
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
|
{
|
||||||
connection.Metadata,
|
var storageClient = new GrpcStorage.StorageService.StorageServiceClient(connection.Channel);
|
||||||
Deadline(),
|
HistorianNativeHandshake.RunTokenRounds(
|
||||||
cancellationToken);
|
(handle, wrapped, _) =>
|
||||||
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
|
{
|
||||||
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
|
GrpcStorage.ValidateClientCredentialResponse response = storageClient.ValidateClientCredential(
|
||||||
bool success = response.Status?.BSuccess ?? false;
|
new GrpcStorage.ValidateClientCredentialRequest { Handle = handle, InBuff = ByteString.CopyFrom(wrapped) },
|
||||||
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
|
connection.Metadata,
|
||||||
},
|
Deadline(),
|
||||||
contextKey,
|
cancellationToken);
|
||||||
options,
|
byte[] serverOutput = response.OutBuff?.ToByteArray() ?? [];
|
||||||
cancellationToken);
|
byte[] error = response.Status?.BtError?.ToByteArray() ?? [];
|
||||||
|
bool success = response.Status?.BSuccess ?? false;
|
||||||
|
return new HistorianNativeHandshake.TokenExchangeResult(success, serverOutput, error);
|
||||||
|
},
|
||||||
|
contextKey,
|
||||||
|
options,
|
||||||
|
cancellationToken);
|
||||||
|
}
|
||||||
|
|
||||||
byte[] open2Request = HistorianNativeHandshake.BuildOpenConnection3Request(
|
// Event reads require an Event-type connection (ConnectionType=Event), which only the native
|
||||||
options.Host, contextKey, connectionMode);
|
// v8 OpenConnection format carries — the v6 buffer has no such field. The v8 path authenticates
|
||||||
|
// via HistoryService.ExchangeKey (P-256 ECDH): the shared secret -> SHA256 = the client key, and
|
||||||
|
// the v8 credential token = RC4(password-UTF16LE, key=MD5(clientKey)) (the native HistorianCrypto
|
||||||
|
// aahCryptV2 scheme). The server shares the secret and RC4-decrypts the token to validate the
|
||||||
|
// password. See docs/reverse-engineering/grpc-event-query-capture.md.
|
||||||
|
byte[] eventToken = [];
|
||||||
|
if (eventConnection)
|
||||||
|
{
|
||||||
|
using ECDiffieHellman ecdh = ECDiffieHellman.Create(ECCurve.NamedCurves.nistP256);
|
||||||
|
byte[] clientHello = HistorianNativeHandshake.BuildExchangeKeyClientHello(ecdh);
|
||||||
|
string xkHandle = contextKey.ToString("D").ToUpperInvariant();
|
||||||
|
GrpcHistory.ExchangeKeyResponse xk = historyClient.ExchangeKey(
|
||||||
|
new GrpcHistory.ExchangeKeyRequest { StrHandle = xkHandle, BtInput = ByteString.CopyFrom(clientHello) },
|
||||||
|
connection.Metadata,
|
||||||
|
Deadline(),
|
||||||
|
cancellationToken);
|
||||||
|
if (!(xk.Status?.BSuccess ?? false))
|
||||||
|
{
|
||||||
|
byte[] xkErr = xk.Status?.BtError?.ToByteArray() ?? [];
|
||||||
|
HistorianNativeError? xkDecoded = HistorianOpen2Protocol.TryReadNativeError(xkErr);
|
||||||
|
string xkAscii = new(xkErr.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"gRPC ExchangeKey failed (errorLen={xkErr.Length}, native={xkDecoded?.Type}/{xkDecoded?.Code}, ascii='{xkAscii}').");
|
||||||
|
}
|
||||||
|
|
||||||
|
byte[] clientKey = HistorianNativeHandshake.DeriveExchangeKeyClientKey(ecdh, xk.BtOutput?.ToByteArray() ?? []);
|
||||||
|
eventToken = HistorianNativeHandshake.BuildExchangeKeyCredentialToken(clientKey, options.Password);
|
||||||
|
}
|
||||||
|
|
||||||
|
byte[] open2Request = eventConnection
|
||||||
|
? HistorianNativeHandshake.BuildEventOpenConnectionVersion8Request(contextKey, options.UserName, eventToken)
|
||||||
|
: HistorianNativeHandshake.BuildOpenConnection3Request(options.Host, contextKey, connectionMode);
|
||||||
|
if (eventConnection) { LastEventOpenRequestHex = Convert.ToHexString(open2Request); }
|
||||||
|
|
||||||
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
|
GrpcHistory.OpenConnectionResponse open2 = historyClient.OpenConnection(
|
||||||
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
|
new GrpcHistory.OpenConnectionRequest { BtConnectionRequest = ByteString.CopyFrom(open2Request) },
|
||||||
@@ -92,7 +136,11 @@ internal static class HistorianGrpcHandshake
|
|||||||
if (!(open2.Status?.BSuccess ?? false))
|
if (!(open2.Status?.BSuccess ?? false))
|
||||||
{
|
{
|
||||||
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
|
byte[] err = open2.Status?.BtError?.ToByteArray() ?? [];
|
||||||
throw new InvalidOperationException($"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}).");
|
HistorianNativeError? decoded = HistorianOpen2Protocol.TryReadNativeError(err);
|
||||||
|
string ascii = new(err.Where(b => b is >= 0x20 and < 0x7F).Select(b => (char)b).ToArray());
|
||||||
|
throw new InvalidOperationException(
|
||||||
|
$"gRPC OpenConnection failed (errorLen={err.Length}, responseLen={open2Response.Length}, " +
|
||||||
|
$"native={decoded?.Type}/{decoded?.Code}{(decoded?.Name is { } n ? $" {n}" : "")}, ascii='{ascii}').");
|
||||||
}
|
}
|
||||||
|
|
||||||
(uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
|
(uint clientHandle, Guid storageSessionId) = HistorianNativeHandshake.ParseOpenConnectionResponse(open2Response);
|
||||||
|
|||||||
@@ -44,6 +44,40 @@ internal static class HistorianAddTagsProtocol
|
|||||||
/// </remarks>
|
/// </remarks>
|
||||||
public static readonly Guid CommonArchestraEventTypeId = new("5f59ae42-3bb6-4760-91a5-ab0be01f9f02");
|
public static readonly Guid CommonArchestraEventTypeId = new("5f59ae42-3bb6-4760-91a5-ab0be01f9f02");
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// The CM_EVENT event-type GUID used by the 2023 R2 <b>gRPC</b> EnsureTags (captured ending
|
||||||
|
/// <c>…e0 1f 2f 27</c>) — distinct from the 2020 WCF capture's <see cref="CommonArchestraEventTypeId"/>
|
||||||
|
/// (<c>…9f02</c>).
|
||||||
|
/// </summary>
|
||||||
|
public static readonly Guid CommonArchestraEventTypeIdGrpc = new("5f59ae42-3bb6-4760-91a5-ab0be01f2f27");
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the native 2023 R2 <b>gRPC</b> CM_EVENT <c>EnsureTags.tagInfos</c> buffer (86 bytes,
|
||||||
|
/// captured byte-for-byte). Differs from the 2020 WCF <see cref="SerializeCmEventCTagMetadata"/>:
|
||||||
|
/// it is wrapped in an 8-byte EnsureTags header (<c>4E 67 03 00 01 00 00 00</c>), uses the
|
||||||
|
/// <see cref="CommonArchestraEventTypeIdGrpc"/> event-type GUID, and has no trailing bytes after it.
|
||||||
|
/// Used by the gRPC event registration so the server actually establishes CM_EVENT and the event
|
||||||
|
/// query returns rows.
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] SerializeCmEventEnsureTagsGrpc(DateTime createdUtc)
|
||||||
|
{
|
||||||
|
using MemoryStream stream = new();
|
||||||
|
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
|
||||||
|
|
||||||
|
writer.Write(new byte[] { 0x4E, 0x67, 0x03, 0x00, 0x01, 0x00, 0x00, 0x00 }); // EnsureTags header (count 1)
|
||||||
|
writer.Write((byte)3);
|
||||||
|
writer.Write((ushort)0x0086);
|
||||||
|
writer.Write((byte)5);
|
||||||
|
writer.Write(CmEventTagId.ToByteArray());
|
||||||
|
WriteCompressedHistorianString(writer, "CM_EVENT");
|
||||||
|
WriteCompressedHistorianString(writer, "AnE Event");
|
||||||
|
writer.Write(new byte[] { 0x02, 0x02, 0x01, 0x00, 0x00, 0x00, 0x01 });
|
||||||
|
writer.Write(0u);
|
||||||
|
writer.Write(createdUtc.ToUniversalTime().ToFileTimeUtc());
|
||||||
|
writer.Write(CommonArchestraEventTypeIdGrpc.ToByteArray());
|
||||||
|
return stream.ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
public static byte[] SerializeCmEventCTagMetadata(DateTime createdUtc)
|
public static byte[] SerializeCmEventCTagMetadata(DateTime createdUtc)
|
||||||
{
|
{
|
||||||
using MemoryStream stream = new();
|
using MemoryStream stream = new();
|
||||||
|
|||||||
@@ -1,5 +1,6 @@
|
|||||||
using System.Buffers.Binary;
|
using System.Buffers.Binary;
|
||||||
using System.Diagnostics;
|
using System.Diagnostics;
|
||||||
|
using System.Security.Cryptography;
|
||||||
|
|
||||||
namespace AVEVA.Historian.Client.Wcf;
|
namespace AVEVA.Historian.Client.Wcf;
|
||||||
|
|
||||||
@@ -28,6 +29,111 @@ internal static class HistorianNativeHandshake
|
|||||||
/// <summary>Result of one transport-level credential-token exchange.</summary>
|
/// <summary>Result of one transport-level credential-token exchange.</summary>
|
||||||
internal readonly record struct TokenExchangeResult(bool Success, byte[] ServerOutput, byte[] Error);
|
internal readonly record struct TokenExchangeResult(bool Success, byte[] ServerOutput, byte[] Error);
|
||||||
|
|
||||||
|
// HistoryService.ExchangeKey wire format (decoded from a live 2023 R2 capture): the ASCII magic
|
||||||
|
// "ECK1", a uint32 coordinate length (32 = P-256), then the raw EC public-key point X || Y. Used by
|
||||||
|
// the v8 (event) connection path to establish the client key the v8 OpenConnection requires.
|
||||||
|
private static readonly byte[] ExchangeKeyMagic = "ECK1"u8.ToArray();
|
||||||
|
private const int ExchangeKeyCoordinateBytes = 32; // P-256 / secp256r1
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the <c>ExchangeKey</c> client hello: <c>"ECK1" + u32(32) + X(32) + Y(32)</c> for the
|
||||||
|
/// supplied P-256 key. Pure managed (<see cref="ECDiffieHellman"/>); no native AVEVA dependency.
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] BuildExchangeKeyClientHello(ECDiffieHellman ecdh)
|
||||||
|
{
|
||||||
|
ECParameters parameters = ecdh.ExportParameters(includePrivateParameters: false);
|
||||||
|
byte[] x = LeftPadCoordinate(parameters.Q.X!);
|
||||||
|
byte[] y = LeftPadCoordinate(parameters.Q.Y!);
|
||||||
|
|
||||||
|
using MemoryStream stream = new();
|
||||||
|
using BinaryWriter writer = new(stream);
|
||||||
|
writer.Write(ExchangeKeyMagic);
|
||||||
|
writer.Write((uint)ExchangeKeyCoordinateBytes);
|
||||||
|
writer.Write(x);
|
||||||
|
writer.Write(y);
|
||||||
|
return stream.ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Parses the server's <c>ExchangeKey</c> hello (same <c>"ECK1" + u32 + X + Y</c> shape) and
|
||||||
|
/// derives the 32-byte client key = <b>SHA256(ECDH shared secret)</b>. This matches the native
|
||||||
|
/// client, which uses <c>ECDiffieHellmanCng { KeyDerivationFunction=Hash, HashAlgorithm=SHA256 }
|
||||||
|
/// .DeriveKeyMaterial(...)</c> — i.e. the hash KDF over the raw agreement.
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] DeriveExchangeKeyClientKey(ECDiffieHellman ecdh, byte[] serverHello)
|
||||||
|
{
|
||||||
|
const int headerLength = 8; // 4-byte magic + 4-byte coordinate length
|
||||||
|
int needed = headerLength + (2 * ExchangeKeyCoordinateBytes);
|
||||||
|
if (serverHello.Length < needed)
|
||||||
|
{
|
||||||
|
throw new InvalidOperationException($"ExchangeKey server hello too short (len={serverHello.Length}, need>={needed}).");
|
||||||
|
}
|
||||||
|
|
||||||
|
byte[] x = serverHello[headerLength..(headerLength + ExchangeKeyCoordinateBytes)];
|
||||||
|
byte[] y = serverHello[(headerLength + ExchangeKeyCoordinateBytes)..(headerLength + (2 * ExchangeKeyCoordinateBytes))];
|
||||||
|
|
||||||
|
ECParameters serverParameters = new()
|
||||||
|
{
|
||||||
|
Curve = ECCurve.NamedCurves.nistP256,
|
||||||
|
Q = new ECPoint { X = x, Y = y },
|
||||||
|
};
|
||||||
|
using ECDiffieHellman serverKey = ECDiffieHellman.Create(serverParameters);
|
||||||
|
return ecdh.DeriveKeyFromHash(serverKey.PublicKey, HashAlgorithmName.SHA256);
|
||||||
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the v8 credential token: <c>RC4(password-UTF16LE, key = MD5(clientKey))</c>, where
|
||||||
|
/// <paramref name="clientKey"/> is the <see cref="DeriveExchangeKeyClientKey"/> result
|
||||||
|
/// (SHA256 of the ECDH secret). Reverse-engineered from the native <c>HistorianCrypto.NRC4_V2
|
||||||
|
/// .aahCryptV2</c> scheme (MD5-keyed RC4) and verified to reproduce a live-captured token exactly.
|
||||||
|
/// The server, sharing the ECDH secret, RC4-decrypts this to recover and validate the password.
|
||||||
|
/// Pure managed; nothing AVEVA is shipped.
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] BuildExchangeKeyCredentialToken(byte[] clientKey, string password)
|
||||||
|
{
|
||||||
|
byte[] rc4Key = MD5.HashData(clientKey);
|
||||||
|
byte[] plaintext = System.Text.Encoding.Unicode.GetBytes(password ?? string.Empty);
|
||||||
|
return Rc4(rc4Key, plaintext);
|
||||||
|
}
|
||||||
|
|
||||||
|
internal static byte[] Rc4(byte[] key, byte[] data)
|
||||||
|
{
|
||||||
|
int[] s = new int[256];
|
||||||
|
for (int i = 0; i < 256; i++)
|
||||||
|
{
|
||||||
|
s[i] = i;
|
||||||
|
}
|
||||||
|
|
||||||
|
for (int i = 0, j = 0; i < 256; i++)
|
||||||
|
{
|
||||||
|
j = (j + s[i] + key[i % key.Length]) & 0xFF;
|
||||||
|
(s[i], s[j]) = (s[j], s[i]);
|
||||||
|
}
|
||||||
|
|
||||||
|
byte[] output = new byte[data.Length];
|
||||||
|
for (int i = 0, j = 0, n = 0; n < data.Length; n++)
|
||||||
|
{
|
||||||
|
i = (i + 1) & 0xFF;
|
||||||
|
j = (j + s[i]) & 0xFF;
|
||||||
|
(s[i], s[j]) = (s[j], s[i]);
|
||||||
|
output[n] = (byte)(data[n] ^ s[(s[i] + s[j]) & 0xFF]);
|
||||||
|
}
|
||||||
|
|
||||||
|
return output;
|
||||||
|
}
|
||||||
|
|
||||||
|
private static byte[] LeftPadCoordinate(byte[] coordinate)
|
||||||
|
{
|
||||||
|
if (coordinate.Length == ExchangeKeyCoordinateBytes)
|
||||||
|
{
|
||||||
|
return coordinate;
|
||||||
|
}
|
||||||
|
|
||||||
|
byte[] padded = new byte[ExchangeKeyCoordinateBytes];
|
||||||
|
Array.Copy(coordinate, 0, padded, ExchangeKeyCoordinateBytes - coordinate.Length, coordinate.Length);
|
||||||
|
return padded;
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Performs a single credential-token round on the wire. <paramref name="handle"/> is the
|
/// Performs a single credential-token round on the wire. <paramref name="handle"/> is the
|
||||||
/// upper-case context-key GUID, <paramref name="wrappedToken"/> is the AVEVA-wrapped SSPI
|
/// upper-case context-key GUID, <paramref name="wrappedToken"/> is the AVEVA-wrapped SSPI
|
||||||
@@ -134,6 +240,56 @@ internal static class HistorianNativeHandshake
|
|||||||
credentialBlock: new byte[CredentialBlockSizeBytes]);
|
credentialBlock: new byte[CredentialBlockSizeBytes]);
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// v8 OpenConnection constants (2023 R2), decoded from the native Event-connection capture.
|
||||||
|
private const byte NativeConnectionTypeEvent = 1;
|
||||||
|
private const byte NativeConnectionFlagEvent = 1;
|
||||||
|
private const ushort NativeHcalVersionV8 = 18;
|
||||||
|
private const string ClientDataSourceIdV8 = "2023.1219.4004.5";
|
||||||
|
private const byte NativeClientCommonInfoFormatVersionV8 = 3;
|
||||||
|
private const int CredentialTokenSizeBytes = 26;
|
||||||
|
private static readonly Guid UnsetShardId = new(new byte[]
|
||||||
|
{
|
||||||
|
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
|
||||||
|
0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF,
|
||||||
|
});
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the native 2023 R2 <b>version-8</b> OpenConnection request for an <b>Event</b> connection
|
||||||
|
/// (<c>ConnectionType=Event</c>). The 2023 R2 server returns event rows only on an event connection,
|
||||||
|
/// and the v6 OpenConnection buffer has no ConnectionType field. The credential token is zeroed: the
|
||||||
|
/// session is authenticated by the preceding <c>ValidateClientCredential</c> handshake, exactly as
|
||||||
|
/// the v6 path already relies on (it sends a zeroed credential block). See
|
||||||
|
/// docs/reverse-engineering/grpc-event-query-capture.md.
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] BuildEventOpenConnectionVersion8Request(Guid contextKey, string userName, byte[] credentialToken)
|
||||||
|
{
|
||||||
|
Process current = Process.GetCurrentProcess();
|
||||||
|
string machineName = Environment.MachineName;
|
||||||
|
string processName = string.IsNullOrEmpty(current.ProcessName) ? ClientNodeNameFallback : current.ProcessName;
|
||||||
|
|
||||||
|
HistorianClientCommonInfo commonInfo = new(
|
||||||
|
FormatVersion: NativeClientCommonInfoFormatVersionV8,
|
||||||
|
ServerNodeName: machineName,
|
||||||
|
ClientNodeName: processName,
|
||||||
|
ProcessId: checked((uint)current.Id),
|
||||||
|
HcalVersion: NativeHcalVersionV8,
|
||||||
|
ProcessName: string.Empty,
|
||||||
|
Proxy: string.Empty,
|
||||||
|
DataSourceId: ClientDataSourceIdV8,
|
||||||
|
ShardId: UnsetShardId,
|
||||||
|
ClientVersion: NativeClientVersionInt,
|
||||||
|
ClientTimestamp: (ulong)DateTime.UtcNow.ToFileTimeUtc(),
|
||||||
|
ClientDllVersion: string.Empty);
|
||||||
|
|
||||||
|
return HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8(
|
||||||
|
commonInfo,
|
||||||
|
contextKey,
|
||||||
|
NativeConnectionTypeEvent,
|
||||||
|
NativeConnectionFlagEvent,
|
||||||
|
userName ?? string.Empty,
|
||||||
|
credentialToken ?? []);
|
||||||
|
}
|
||||||
|
|
||||||
/// <summary>
|
/// <summary>
|
||||||
/// Decodes the OpenConnection response blob: byte 0 = protocol version, bytes 1..4 =
|
/// Decodes the OpenConnection response blob: byte 0 = protocol version, bytes 1..4 =
|
||||||
/// transient /Retr client handle (UInt32 LE), bytes 5..20 = storage session GUID.
|
/// transient /Retr client handle (UInt32 LE), bytes 5..20 = storage session GUID.
|
||||||
|
|||||||
@@ -50,6 +50,49 @@ internal static class HistorianOpen2Protocol
|
|||||||
return stream.ToArray();
|
return stream.ToArray();
|
||||||
}
|
}
|
||||||
|
|
||||||
|
/// <summary>
|
||||||
|
/// Builds the native 2023 R2 <b>version-8</b> OpenConnection request — the format the stock client
|
||||||
|
/// uses, which (unlike v6) carries a <paramref name="connectionType"/> byte (Event vs Process). The
|
||||||
|
/// tail is the same <see cref="HistorianClientCommonInfo"/> v6 emits; the head is the v8 layout
|
||||||
|
/// decoded from a live capture (docs/reverse-engineering/grpc-event-query-capture.md): version 8,
|
||||||
|
/// two marker bytes, the client-key GUID, a username <c>HistorianString</c>, a length-prefixed
|
||||||
|
/// credential token, then ClientType / ConnectionType / flag / a constant word / compact metadata
|
||||||
|
/// namespace / two empty strings. The credential token is normally an ECDH-derived blob in the
|
||||||
|
/// native cert path; this serializer sends it as supplied (zeroed for the Negotiate-auth path,
|
||||||
|
/// mirroring how the v6 request already sends a zeroed credential block and relies on the separate
|
||||||
|
/// ValidateClientCredential handshake for authentication).
|
||||||
|
/// </summary>
|
||||||
|
public static byte[] SerializeNativeOpenConnectionVersion8(
|
||||||
|
HistorianClientCommonInfo commonInfo,
|
||||||
|
Guid clientKey,
|
||||||
|
byte connectionType,
|
||||||
|
byte connectionFlag,
|
||||||
|
string userName,
|
||||||
|
byte[] credentialToken)
|
||||||
|
{
|
||||||
|
using MemoryStream stream = new();
|
||||||
|
using BinaryWriter writer = new(stream, Encoding.Unicode, leaveOpen: true);
|
||||||
|
|
||||||
|
writer.Write((byte)8); // [0] format version 8
|
||||||
|
writer.Write((byte)0xF0); // [1] constant marker
|
||||||
|
writer.Write(new byte[19]); // [2..20] zero padding
|
||||||
|
writer.Write((byte)1); // [21] constant marker
|
||||||
|
writer.Write(clientKey.ToByteArray()); // [22..37] per-session client key
|
||||||
|
WriteHistorianString(writer, userName); // username (UTF-16 HistorianString)
|
||||||
|
writer.Write((ushort)credentialToken.Length); // credential-token length (u16)
|
||||||
|
writer.Write(credentialToken); // credential token (zeroed for the Negotiate path)
|
||||||
|
writer.Write((byte)4); // ClientType = 4
|
||||||
|
writer.Write(connectionType); // ConnectionType (Event = 1 / Process = 2)
|
||||||
|
writer.Write(connectionFlag); // type flag (Event = 1 / Process = 0)
|
||||||
|
writer.Write((ushort)3); // constant word observed at [97..98]
|
||||||
|
WriteCompactMetadataNamespace(writer, HistorianMetadataNamespace.Empty);
|
||||||
|
WriteHistorianString(writer, string.Empty);
|
||||||
|
WriteHistorianString(writer, string.Empty);
|
||||||
|
WriteClientCommonInfo(writer, commonInfo); // tail: FormatVersion 3 => no ClientDllVersion
|
||||||
|
writer.Write(0u); // trailing terminator
|
||||||
|
return stream.ToArray();
|
||||||
|
}
|
||||||
|
|
||||||
private static void WriteNativeOpenConnectionContent(
|
private static void WriteNativeOpenConnectionContent(
|
||||||
BinaryWriter writer,
|
BinaryWriter writer,
|
||||||
HistorianOpen2Request request,
|
HistorianOpen2Request request,
|
||||||
|
|||||||
@@ -18,26 +18,31 @@ namespace AVEVA.Historian.Client.Tests;
|
|||||||
public sealed class HistorianGrpcHandshakeRoutingTests
|
public sealed class HistorianGrpcHandshakeRoutingTests
|
||||||
{
|
{
|
||||||
[Fact]
|
[Fact]
|
||||||
public void Handshake_UsesValidateClientCredential_NotExchangeKey()
|
public void Handshake_TokenLoop_UsesValidateClientCredential_NotExchangeKey()
|
||||||
{
|
{
|
||||||
// The auth token loop lives in the shared handshake helper (reused by the read, status,
|
// The auth token loop is a compiler-generated lambda (nested closure) passed to RunTokenRounds.
|
||||||
// and future browse/metadata gRPC paths).
|
// It MUST carry the SSPI/Negotiate token via StorageService.ValidateClientCredential, never via
|
||||||
HashSet<string> calledMethods = CollectCalledMethodNames(
|
// HistoryService.ExchangeKey (the earlier regression). ExchangeKey is now legitimately called
|
||||||
"AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake");
|
// directly in OpenSession for the SEPARATE v8 event-connection key exchange (ECDH) — that is
|
||||||
|
// allowed, so the guardrail scopes the no-ExchangeKey rule to the token-loop closures only.
|
||||||
|
HashSet<string> tokenLoopCalls = CollectCalledMethodNames(
|
||||||
|
"AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake", nestedClosuresOnly: true);
|
||||||
|
|
||||||
Assert.Contains("ValidateClientCredential", calledMethods);
|
Assert.Contains("ValidateClientCredential", tokenLoopCalls);
|
||||||
Assert.DoesNotContain("ExchangeKey", calledMethods);
|
Assert.DoesNotContain("ExchangeKey", tokenLoopCalls);
|
||||||
}
|
}
|
||||||
|
|
||||||
private static HashSet<string> CollectCalledMethodNames(string typeFullName)
|
private static HashSet<string> CollectCalledMethodNames(string typeFullName, bool nestedClosuresOnly = false)
|
||||||
{
|
{
|
||||||
Assembly sdk = typeof(HistorianClientOptions).Assembly;
|
Assembly sdk = typeof(HistorianClientOptions).Assembly;
|
||||||
Type orchestrator = sdk.GetType(typeFullName, throwOnError: true)!;
|
Type orchestrator = sdk.GetType(typeFullName, throwOnError: true)!;
|
||||||
Module module = orchestrator.Module;
|
Module module = orchestrator.Module;
|
||||||
|
|
||||||
// The orchestrator type plus its compiler-generated nested types (lambda closures).
|
// The compiler-generated nested types (lambda closures) — optionally with the orchestrator type
|
||||||
IEnumerable<Type> types = new[] { orchestrator }
|
// itself. The token loop lives inside a closure; ExchangeKey (event key exchange) lives in the
|
||||||
.Concat(orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic));
|
// OpenSession body, so scoping to closures isolates the token-loop routing.
|
||||||
|
IEnumerable<Type> nested = orchestrator.GetNestedTypes(BindingFlags.Public | BindingFlags.NonPublic);
|
||||||
|
IEnumerable<Type> types = nestedClosuresOnly ? nested : new[] { orchestrator }.Concat(nested);
|
||||||
|
|
||||||
var names = new HashSet<string>(StringComparer.Ordinal);
|
var names = new HashSet<string>(StringComparer.Ordinal);
|
||||||
foreach (Type t in types)
|
foreach (Type t in types)
|
||||||
|
|||||||
@@ -515,6 +515,42 @@ public sealed class HistorianGrpcIntegrationTests
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public async Task EventReadDiagnostic_OverGrpc_PrintsJourney()
|
||||||
|
{
|
||||||
|
string? host = Environment.GetEnvironmentVariable("HISTORIAN_GRPC_HOST");
|
||||||
|
if (string.IsNullOrWhiteSpace(host) || string.IsNullOrEmpty(Environment.GetEnvironmentVariable("HISTORIAN_USER")))
|
||||||
|
{
|
||||||
|
return;
|
||||||
|
}
|
||||||
|
if (Environment.GetEnvironmentVariable("HISTORIAN_GRPC_EVENT_DIAG") is null)
|
||||||
|
{
|
||||||
|
return; // opt-in diagnostic only
|
||||||
|
}
|
||||||
|
|
||||||
|
var orch = new AVEVA.Historian.Client.Grpc.HistorianGrpcEventOrchestrator(BuildOptions(host));
|
||||||
|
var events = new List<HistorianEvent>();
|
||||||
|
string outcome;
|
||||||
|
try
|
||||||
|
{
|
||||||
|
await foreach (HistorianEvent evt in orch.ReadEventsAsync(DateTime.UtcNow.AddDays(-90), DateTime.UtcNow, null, CancellationToken.None))
|
||||||
|
{
|
||||||
|
events.Add(evt);
|
||||||
|
if (events.Count >= 3) { break; }
|
||||||
|
}
|
||||||
|
outcome = $"OK events={events.Count}";
|
||||||
|
}
|
||||||
|
catch (Exception ex)
|
||||||
|
{
|
||||||
|
outcome = $"{ex.GetType().Name}: {ex.Message}";
|
||||||
|
}
|
||||||
|
|
||||||
|
throw new Xunit.Sdk.XunitException(
|
||||||
|
$"[DIAG] outcome={outcome} | events={events.Count} | LastResultLen={orch.LastResultBufferLength} " +
|
||||||
|
$"| ResultHex={orch.LastResultBufferHex} | Reg=[{orch.RegistrationDiag}] " +
|
||||||
|
$"| v8open={AVEVA.Historian.Client.Grpc.HistorianGrpcHandshake.LastEventOpenRequestHex}");
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public async Task GetConnectionStatusAsync_OverGrpc_ReportsConnected()
|
public async Task GetConnectionStatusAsync_OverGrpc_ReportsConnected()
|
||||||
{
|
{
|
||||||
@@ -642,7 +678,8 @@ public sealed class HistorianGrpcIntegrationTests
|
|||||||
IntegratedSecurity = !explicitCreds,
|
IntegratedSecurity = !explicitCreds,
|
||||||
UserName = user ?? string.Empty,
|
UserName = user ?? string.Empty,
|
||||||
Password = password ?? string.Empty,
|
Password = password ?? string.Empty,
|
||||||
RequestTimeout = timeout
|
RequestTimeout = timeout,
|
||||||
|
Compression = true // the stock client always advertises grpc gzip request encoding
|
||||||
};
|
};
|
||||||
}
|
}
|
||||||
}
|
}
|
||||||
|
|||||||
@@ -5,6 +5,70 @@ namespace AVEVA.Historian.Client.Tests;
|
|||||||
|
|
||||||
public sealed class WcfOpen2ProtocolTests
|
public sealed class WcfOpen2ProtocolTests
|
||||||
{
|
{
|
||||||
|
[Fact]
|
||||||
|
public void Rc4MatchesStandardTestVector()
|
||||||
|
{
|
||||||
|
// Standard RC4 test vector (key "Key", plaintext "Plaintext" -> BBF316E8D940AF0AD3). Pins the
|
||||||
|
// RC4 used by the v8 credential token = RC4(password-UTF16LE, key=MD5(SHA256(ECDH secret))).
|
||||||
|
byte[] actual = HistorianNativeHandshake.Rc4(
|
||||||
|
System.Text.Encoding.ASCII.GetBytes("Key"),
|
||||||
|
System.Text.Encoding.ASCII.GetBytes("Plaintext"));
|
||||||
|
|
||||||
|
Assert.Equal(Convert.FromHexString("BBF316E8D940AF0AD3"), actual);
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void CredentialToken_IsRc4OfPasswordKeyedByMd5OfClientKey()
|
||||||
|
{
|
||||||
|
// token = RC4(password-UTF16LE, key = MD5(clientKey)). Verify against an independently computed
|
||||||
|
// reference for a fixed clientKey + password.
|
||||||
|
byte[] clientKey = Enumerable.Range(0, 32).Select(i => (byte)i).ToArray();
|
||||||
|
byte[] token = HistorianNativeHandshake.BuildExchangeKeyCredentialToken(clientKey, "Pw1!");
|
||||||
|
|
||||||
|
byte[] rc4Key = System.Security.Cryptography.MD5.HashData(clientKey);
|
||||||
|
byte[] expected = HistorianNativeHandshake.Rc4(rc4Key, System.Text.Encoding.Unicode.GetBytes("Pw1!"));
|
||||||
|
Assert.Equal(expected, token);
|
||||||
|
Assert.Equal("Pw1!".Length * 2, token.Length); // UTF-16LE length
|
||||||
|
}
|
||||||
|
|
||||||
|
[Fact]
|
||||||
|
public void Version8EventSerializerReproducesCapturedNativeStructure()
|
||||||
|
{
|
||||||
|
// Field sizes chosen to match the live 2023 R2 Event-connection capture (302 bytes):
|
||||||
|
// username=12, token=26, serverNode=15, clientNode=32, datasource=16.
|
||||||
|
// See docs/reverse-engineering/grpc-event-query-capture.md.
|
||||||
|
var clientKey = new Guid("11223344-5566-7788-99aa-bbccddeeff00");
|
||||||
|
byte[] actual = HistorianOpen2Protocol.SerializeNativeOpenConnectionVersion8(
|
||||||
|
new HistorianClientCommonInfo(
|
||||||
|
FormatVersion: 3,
|
||||||
|
ServerNodeName: new string('S', 15),
|
||||||
|
ClientNodeName: new string('C', 32),
|
||||||
|
ProcessId: 0x11223344,
|
||||||
|
HcalVersion: 18,
|
||||||
|
ProcessName: string.Empty,
|
||||||
|
Proxy: string.Empty,
|
||||||
|
DataSourceId: new string('D', 16),
|
||||||
|
ShardId: new Guid(Enumerable.Repeat((byte)0xFF, 16).ToArray()),
|
||||||
|
ClientVersion: 999_999,
|
||||||
|
ClientTimestamp: 0x01DD02426F9B6F6C,
|
||||||
|
ClientDllVersion: string.Empty),
|
||||||
|
clientKey,
|
||||||
|
connectionType: 1, // Event
|
||||||
|
connectionFlag: 1, // Event
|
||||||
|
userName: new string('U', 12),
|
||||||
|
credentialToken: new byte[26]);
|
||||||
|
|
||||||
|
Assert.Equal(302, actual.Length); // exact native length for these field sizes
|
||||||
|
Assert.Equal(0x08, actual[0]); // format version 8
|
||||||
|
Assert.Equal(0xF0, actual[1]); // marker
|
||||||
|
Assert.Equal(0x01, actual[21]); // marker
|
||||||
|
Assert.Equal(clientKey.ToByteArray(), actual[22..38]);
|
||||||
|
Assert.Equal(0x04, actual[94]); // ClientType
|
||||||
|
Assert.Equal(0x01, actual[95]); // ConnectionType = Event
|
||||||
|
Assert.Equal(0x01, actual[96]); // type flag = Event
|
||||||
|
Assert.Equal([0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF, 0xFF], actual[270..286]); // ShardId
|
||||||
|
}
|
||||||
|
|
||||||
[Fact]
|
[Fact]
|
||||||
public void LegacyVersion1SerializerMatchesDecompiledSaveOpenConnectionParamsLayout()
|
public void LegacyVersion1SerializerMatchesDecompiledSaveOpenConnectionParamsLayout()
|
||||||
{
|
{
|
||||||
|
|||||||
@@ -1387,10 +1387,14 @@ static int InstrumentGrpcNonStream(string[] args)
|
|||||||
|
|
||||||
ModuleDefMD module = ModuleDefMD.Load(sourcePath);
|
ModuleDefMD module = ModuleDefMD.Load(sourcePath);
|
||||||
MemberRefUser logByteArray = CreateLogByteArrayRef(module);
|
MemberRefUser logByteArray = CreateLogByteArrayRef(module);
|
||||||
|
MemberRefUser logString = CreateLogStringRef(module);
|
||||||
|
MemberRefUser logUInt32 = CreateLogUInt32Ref(module);
|
||||||
|
|
||||||
// Cast a wide net: instrument EVERY byte[]-input method on every Grpc*Client type, so whichever
|
// Cast a wide net: instrument EVERY byte[]-input method on every Grpc*Client type, so whichever
|
||||||
// path the native non-streamed write actually drives (History/Transaction RegisterTags +
|
// path the native non-streamed write actually drives (History/Transaction RegisterTags +
|
||||||
// AddNonStreamValues, or a Storage-service route) is captured. Phase = "<Type>.<Method>.<param>".
|
// AddNonStreamValues, or a Storage-service route) is captured. Phase = "<Type>.<Method>.<param>".
|
||||||
|
// Also captures string (strHandle) and uint (uiHandle / queryRequestType) inputs so the event-read
|
||||||
|
// handle fields are visible, not just byte[] params.
|
||||||
var instrumented = new List<object>();
|
var instrumented = new List<object>();
|
||||||
foreach (TypeDef type in module.GetTypes()
|
foreach (TypeDef type in module.GetTypes()
|
||||||
.Where(t => t.Name.String.StartsWith("Grpc", StringComparison.Ordinal) && t.Name.String.EndsWith("Client", StringComparison.Ordinal)))
|
.Where(t => t.Name.String.StartsWith("Grpc", StringComparison.Ordinal) && t.Name.String.EndsWith("Client", StringComparison.Ordinal)))
|
||||||
@@ -1431,6 +1435,34 @@ static int InstrumentGrpcNonStream(string[] args)
|
|||||||
});
|
});
|
||||||
}
|
}
|
||||||
|
|
||||||
|
// ENTRY: log string (strHandle) and uint (uiHandle / queryRequestType / count) inputs.
|
||||||
|
foreach (dnlib.DotNet.Parameter scalarParam in method.Parameters
|
||||||
|
.Where(p => !p.IsHiddenThisParameter && (p.Type.FullName == "System.String" || p.Type.FullName == "System.UInt32"))
|
||||||
|
.ToArray())
|
||||||
|
{
|
||||||
|
bool isString = scalarParam.Type.FullName == "System.String";
|
||||||
|
string phase = $"{type.Name}.{method.Name}.{scalarParam.Name}.{(isString ? "str" : "u32")}";
|
||||||
|
Instruction[] injected =
|
||||||
|
[
|
||||||
|
Instruction.Create(OpCodes.Ldstr, phase),
|
||||||
|
Instruction.Create(OpCodes.Ldarg, scalarParam),
|
||||||
|
Instruction.Create(OpCodes.Call, isString ? logString : logUInt32),
|
||||||
|
];
|
||||||
|
foreach (Instruction instruction in injected.Reverse())
|
||||||
|
{
|
||||||
|
method.Body.Instructions.Insert(0, instruction);
|
||||||
|
}
|
||||||
|
method.Body.MaxStack = (ushort)Math.Max((int)method.Body.MaxStack, 8);
|
||||||
|
instrumented.Add(new
|
||||||
|
{
|
||||||
|
Type = type.Name.String,
|
||||||
|
Method = method.Name.String,
|
||||||
|
Phase = phase,
|
||||||
|
Direction = "in",
|
||||||
|
Token = "0x" + method.MDToken.Raw.ToString("X8"),
|
||||||
|
});
|
||||||
|
}
|
||||||
|
|
||||||
// EXIT: log out/ref byte[] responses ("System.Byte[]&") before each ret. ldarg loads the
|
// EXIT: log out/ref byte[] responses ("System.Byte[]&") before each ret. ldarg loads the
|
||||||
// managed pointer; ldind.ref dereferences it to the byte[]. (RPC wrappers set the out
|
// managed pointer; ldind.ref dereferences it to the byte[]. (RPC wrappers set the out
|
||||||
// param right before a single ret, so branch-to-ret skew is not a concern here.)
|
// param right before a single ret, so branch-to-ret skew is not a concern here.)
|
||||||
|
|||||||
Reference in New Issue
Block a user