0780cec9a7f2a78e053c519646e606f4e08a8803
35 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
c2d8fb9bc8 |
R4.3: gRPC store-forward status probe + re-scope
Add HistorianGrpcStoreForwardStatusProbe and the `grpc-sf-status-probe` CLI command. The idle-baseline run against the live 2023 R2 server resolves the plan's §9.3 handle question: the direct StorageService SF pull RPCs (GetSFParameter / GetRemainingSnapshotsSize) require the OpenStorageConnection console handle and are D2-gated (err 132, identical under read-only and write-enabled sessions), while StatusService.GetHistorianConsoleStatus IS reachable on the session string handle (=3 at idle). Records the gRPC re-scope and the idle-baseline findings in docs/plans/store-forward-cache-reverse-engineering.md §9. The probe writes nothing and releases any console session immediately. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
d1e96f48de |
M3 R3.2: AddHistoricalValuesAsync supports Double + Int (Int2/Int4/UInt4)
Extended the historical-write serializer from Float-only to all five analog types EnsureTagAsync supports. Captured each type's "ON" buffer live from the native client (sandbox tag per type, written + captured + deleted): - The 4-byte value descriptor (C0 10 01 00) is CONSTANT across types — it does not encode the type. - The value is u32(0) + native-width value, width by the tag's declared type: Float->float32, Double->double64, Int2->int16, Int4->int32, UInt4->uint32. HistorianHistoricalWriteProtocol.SerializeAddStreamValuesBuffer now takes the HistorianDataType and encodes accordingly (unsupported types throw ProtocolEvidenceMissingException). The orchestrator resolves the type from the tag-info NativeDataTypeDescriptor via MapDataType. Harness capture-write gained --data-type. Golden-tested against all five live captures + the gated write/read-back test validated each type end-to-end through the pure-managed SDK; 281 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
d527784def |
M3 capture harness: add delete-tag scenario (sandbox cleanup)
delete-tag drives the native client's DeleteTags (the clean-delete path, unlike the SDK's WCF DelT which can leave the row). Primes the write session with AddTag first (DeleteTags on a fresh connection returns UnknownClient(51) until the client is registered). Used to remove the capture sandbox tag SdkM3CaptureSandbox from the live server (DeleteTags returned success). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
273103f882 |
M3 R3.2: instrument-grpc-nonstream also captures out/ref byte[] responses
Extends the IL-rewrite to log out (byref) byte[] params at method exit (ldarg + ldind.ref), not just byte[] inputs. This captured GetTagInfosFromName's response, which located the per-tag GUID at offset 8 = exactly where ParseTagInfoRecord reads "typeId" — proving the SDK already parses the GUID AddStreamValues needs. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
9bcfffb365 |
M3 R3.1 CAPTURED: native non-streamed write rides HistoryService.AddStreamValues ("ON" buffer)
Drove the native 2023 R2 client through a committed non-streamed (historical backfill) write to a
sandbox tag, with the IL-rewritten managed gRPC client dumping every byte[] payload. Read the value
back over gRPC = end-to-end validated.
Key discovery: the M3 historical write does NOT use AddNonStreamValues/TransactionService (the
roadmap's assumption from the static decompile). The native client routes it over
HistoryService.AddStreamValues with an "ON" storage-sample buffer (structurally the AddS2 "OS"
family), plus EnsureTags for registration:
AddStreamValues.values (56B): "ON"(0x4E4F) + u16 count=1 + u32 totalLen + u16 payloadLen +
16B tag GUID + FILETIME(sample) + u16 quality=192 + u32 type/desc + FILETIME(received) +
8B double value.
EnsureTags.tagInfos (144B): the analog CTagMetadata the SDK's EnsureTagAsync already builds
(0x4E marker ... fe 00 trailer).
Tooling that produced it:
- instrument-grpc-nonstream now instruments EVERY byte[]-input method on every Grpc*Client
(45 methods) so the real wire path surfaces regardless of assumptions.
- harness pre-loads the instrumented GrpcClient by identity (LoadFrom context reuses an
already-loaded assembly before sibling-probing, so the rewrite wins over the bin original);
capture-write sequence fixed to Begin -> Add -> SendValues -> End (End-before-Send = err 160
InvalidBatchId); GetTagInfoByName(cache:false) + resync wait resolves the server key; cache
gate (D2's 129) does NOT block the primed 2023 R2 client.
Buffers captured to gitignored artifacts/. Next: build the "ON" AddStreamValues serializer in
src/ (adapt the existing AddS2 "OS" serializer) + EnsureTags + ship AddHistoricalValuesAsync.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
|
||
|
Joseph Doherty
|
d5c04cd410 |
M3 R3.1 capture: add capture-write scenario (drives non-streamed write, no run yet)
The capture-write harness scenario drives the native 2023 R2 client through a non-streamed (historical backfill) write so the IL-rewritten GrpcHistoryClient dumps RegisterTags.tagInfos + AddNonStreamValues.inBuff to the capture NDJSON. Sequence: open write-enabled gRPC -> (optional --create) AddTag sandbox -> GetTagInfoByName (real TagKey + primes the per-connection cache, the gate mitigation) -> CreateHistorianDataValueList(NonStreamedOriginal) -> NonStreamedValuesBegin -> AddNonStreamedValue -> AddNonStreamedValuesEnd -> SendValues (the wire push; only with --commit). Not yet run — the actual write to the live server awaits explicit confirmation. Built clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
c1f263ef83 |
M3 R3.1 capture: instrument-grpc-nonstream IL-rewrite + harness --grpc-rewrite loading
Adds the dnlib instrument command + harness wiring to capture the two non-streamed-write buffers from the native 2023 R2 client: - `instrument-grpc-nonstream <GrpcClient.dll> [out]` injects CaptureLogger.LogByteArray at the entry of GrpcHistoryClient.RegisterTags (byte[] tagInfos) and AddNonStreamValues (byte[] inBuff), writing the rewrite to docs/reverse-engineering/dnlib-write-copy/grpc2023 (gitignored — derived AVEVA binary). dnlib preserves the AVEVA public-key identity so aahClientManaged still binds the rewritten copy under the LoadFrom context (no SN re-verification). - harness `--grpc-rewrite <dir>` probes that dir first, so the instrumented GrpcClient.dll + ReverseInstrumentation.dll load ahead of the originals. load-check confirms the rewritten strong-named copy binds (HistorianConnectionMode.Historian=2; GrpcHistoryClient RegisterTags + AddNonStreamValues present). Next: capture-write scenario (open write-enabled -> sandbox tag -> read-prime -> AddNonStreamedValue), which dumps tagInfos + inBuff to the capture NDJSON. Prod write — confirm before running. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
ce8576bd6e |
M3 R3.1 capture: read-only gRPC connect scenario — LIVE-VERIFIED
Added the `connect` scenario to the 2023 R2 capture harness and ran it read-only against the
live server. The native mixed-mode client connects end-to-end over gRPC from this box:
OpenConnection -> True (ErrorCode=Success)
ConnectedToServer = True
ConnectedToServerStorage = True <-- native client HAS the storage-engine session
ConnectedToStoreForward = False
Connection args that work (HistorianConnectionArgs): ServerName, TcpPort=32565,
ConnectionMode=Historian (gRPC), ConnectionType=Process, ReadOnly=true, IntegratedSecurity=false,
UserName/Password (explicit), AllowUnTrustedConnection=true, SecurityInfo=CertificateInfo{
SecurityMode=TransportCertificate, CertificateName=WONDER-SQL-VD03 } (the https:// host over the
loopback tunnel). Creds from HISTORIAN_USER/HISTORIAN_PASSWORD.
Significance: ConnectedToServerStorage=True means the native client establishes the storage
session the pure-managed SDK couldn't — so a write driven through it should route
AddNonStreamValues with a live storage session, and the cache-gate mitigation (read-first)
is promising. Next: IL-rewrite Archestra.Historian.GrpcClient.dll + a write-enabled run to
capture the RegisterTags btTagInfos + AddNonStreamValues btInput (prod write; per-action auth).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
|
||
|
Joseph Doherty
|
328748c0ae |
M3 R3.1 capture: scaffold net481 x64 harness + load-check (PASS)
First increment of the native-2023R2-gRPC capture (docs/plans/revision-write-path.md §"R3.1 capture plan"). Loads the mixed-mode aahClientManaged.dll by path (sibling resolver over bin/ + msi-extract/.../Bin/x64) and reflects the connection API — no live contact. load-check result on this box (net481 x64): - aahClientManaged.dll loads cleanly (no missing VC++ runtime / no BadImageFormat) — confirms the self-contained mixed-mode assembly runs without an AVEVA install. - HistorianConnectionMode.Historian = 2 (the 2023 R2 gRPC mode; ClassicHistorian = 1 = legacy) — the value the live-connect step sets on HistorianConnectionArgs.ConnectionMode. - GrpcHistoryClient resolves with RegisterTags + AddNonStreamValues present — the IL-rewrite capture targets are reachable. Standalone net481 project (not in Histsdk.slnx, like NativeTraceHarness). Next: read-only gRPC connect + tag read (first live step, per-action auth), then IL-rewrite + write capture. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
57b9506d01 |
M3 R3.1: OpenStorageConnection is a dead end (error 85); precondition is front-door RegisterTags
Live-probed StorageService.OpenStorageConnection against the 2023 R2 server over a
write-enabled (0x401) session. Every attempt — sweeping ConnectionMode (0x401/0x402/0x1),
StorageSessionId-in (Open2-GUID / empty), and FreeDiskSpace — returns the IDENTICAL native
error type=4 code=85 ("session not registered"), so it's a structural refusal, not a bad
field value.
Decode (two corroborating facts):
- Error 85 is the same code the event read returns before RegisterTags2 (see
HistorianWcfEventOrchestrator) — a generic "session not registered for this op".
- The 2023 R2 decompile shows OpenStorageConnection lives on a SEPARATE GrpcStorageClient
(the storage engine's SF/snapshot channel, own port + service identity); HistorianAccess
drives non-streamed writes through the native C++ HistorianClient, never this op.
So the roadmap's mapped "missing console session" step was wrong. The real non-streamed-write
precondition is the front-door HistoryService.RegisterTags (RTag2-family) for the target tag —
which is exactly why the R3.1 batch failed at AddNonStreamValues (no tag registered ->
StoreNonStreamValues had no route). Matches the original 2020-WCF D2 hypothesis.
Remaining (both need a native gRPC capture; do not guess bytes): the regular-tag RegisterTags
btTagInfos (only CM_EVENT's tag-GUID form is known) and the AddNonStreamValues btInput.
- HistorianGrpcStorageConnectionProbe + grpc-open-storage-connection CLI (opens nothing
persistent; CloseStorageConnection on success)
- corrected revision-write-path.md §R3.1 follow-up + hcal-roadmap R3.1/R3.2 rows
- gated regression test pinning the error-85 refusal
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
|
||
|
Joseph Doherty
|
8fbb868813 |
M3 R3.1 decode: AddNonStreamValues reaches server StoreNonStreamValues (storage-engine console pipe)
Empirically decoded the AddNonStreamValues btInput framing against the live 2023 R2 server (grpc-nonstream-decode command + ProbeNonStreamedBuffersAsync driver). Every transaction rolled back (bCommit=false) — no data written. Finding: the btInput is assembled native-C++-side (not in any decompile), so 6 evidence-based framings (44-54B, packed HISTORIAN_VALUE2 variants) were probed. All 6 returned the IDENTICAL server error while an empty buffer returned a different InvalidParameter — so non-empty buffers pass parameter validation into CHistStorageConnection::StoreNonStreamValues, which routes to the \.\pipe\aahStorageEngine\console pipe server-side. Identical-across-framings => the blocker is NOT the btInput layout but a missing storage-engine console session / tag-registration precondition for the connection. Next step (untested): StorageService.OpenStorageConnection + tag registration (RegisterTags/AddTagidPairs/AddShardTagids) before AddNonStreamValues, then commit + read-back on a sandbox tag. Documented in revision-write-path.md (R3.1 decode section); raw artifact gitignored. 272 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
23798db1ef |
M3 probe: non-streamed write transaction reachable over 2023 R2 gRPC (Begin/End live-verified)
The D2 storage-engine-pipe wall is WCF-transport-specific. On the 2023 R2 gRPC front door, TransactionService is a first-class service AND the gateway to the storage engine, so the Open2 storage-session GUID (uppercase) is accepted directly as strHandle with no legacy pipe. Live-verified against the real 2023 R2 server over a write-enabled (0x401) gRPC session: AddNonStreamValuesBegin returns a real strTransactionId, and AddNonStreamValuesEnd(bCommit=false) discards it cleanly (no data written). On 2020 WCF the same op returns UnknownClient(51) for every handle + priming chain. - HistorianGrpcRevisionProbe + grpc-revision-probe CLI command + gated test NonStreamedWriteTransaction_OverGrpc_BeginsAndDiscards (live pass). - HistorianGrpcHandshake.OpenSession gains an optional connectionMode param (default read-only 0x402; pass 0x401 for write-enabled) — non-breaking. - Docs: revision-write-path.md "the wall is gone" section; roadmap M3 section, R3.1-R3.3 rows, one-glance table, and status note updated honestly. Not yet shipped: the AddNonStreamValues btInput VTQ buffer is uncaptured (never guess wire bytes), so no value-commit is implemented. Scope is non-streamed ORIGINAL backfill; revision EDITS (R4.2) remain pipe-only even on gRPC. 272 unit tests pass; sanitization scan clean. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
d23722ea73 |
Merge re/r1.10-rename-tags: RenameTagsAsync via History StartJob
# Conflicts: # docs/plans/hcal-capability-matrix.md # docs/plans/hcal-roadmap.md # src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs # tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs # tools/AVEVA.Historian.NativeTraceHarness/Program.cs |
||
|
Joseph Doherty
|
4de222c950 |
Merge re/r1.4-gethi-finding: R1.1 ExecuteSqlCommand + R1.4 GetHistorianInfo (bounded)
# Conflicts: # docs/plans/hcal-roadmap.md # src/AVEVA.Historian.Client/HistorianClient.cs # src/AVEVA.Historian.Client/Protocol/Historian2020ProtocolDialect.cs # tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs # tools/AVEVA.Historian.NativeTraceHarness/Program.cs |
||
|
Joseph Doherty
|
c1b1b3d23b |
R1.11 DelTep capture + R1.3/R1.4/R1.12/R1.13 bounded out
DelTep (extended-property delete) — wire format captured + serializer golden-proven, but live delete is server-blocked and NOT exposed publicly: - Captured the DelTep inBuff via a cross-session trick (harness add-tep gains --tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer. - SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer. - A decisive experiment shows SDK-added properties ARE deletable (the native client read-syncs and deletes one), so SDK-add is complete; the SDK's own DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open channel, and 60s retries. Root cause: the native multiplexes services over one connection (per-connection working set); the SDK's per-service WCF channels don't reproduce it. Kept as documented-but-blocked internal orchestrator path; no public HistorianClient delete API. Bounded out with evidence (no code; docs + roadmap + probe): - R1.12 localized-property write — no op on 2020 (mirror of R1.6); no *LocalizedPropert*/TagLocalized* symbol in any current/*.dll. - R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog type client-side (ValidationFailed, before any WCF op): SingleByteString, DoubleByteString, Int1 all fail, Float works. No Discrete type in the native enum, no TagType setter. No wire request to capture. - R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a parameter-op probe (GetSystemParameter + GETRP return null/throw for every candidate; only HistorianVersion works). 238 unit tests pass; full solution builds with 0 warnings. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
08b950caee |
R1.11 AddTagExtendedPropertiesAsync: extended-property write via AddTEx
Adds user-defined extended properties to an existing tag via the 2020 WCF
AddTEx (AddTagExtendedProperties) op. Write-enabled connection + uppercase
storage-session GUID handle; reuses the write orchestrator open/priming chain.
The AddTEx inBuff is the exact inverse of the R1.5 GetTepByNm read-response
framing, so the serializer mirrors the read parser:
uint32 groupCount + 0x01(group) + [0x09+u16+ASCII tag] + uint32 propCount
+ per prop{ 0x02 + [0x09+u16+ASCII name] + 0x43 VT_BSTR + u16 payloadLen
+ u16 charCount + UTF-16 value } + 0x01(group trailer) + 0x00(terminator).
The trailing 0x00 is required — without it inBuff is one byte short and the
server throws SErrorException in CHistStorage::AddTagExtendedProperties. The
golden fixture pins the clean inBuff the live server accepted (dumped via
AVEVA_HISTORIAN_TEP_DUMP); read-back verified via R1.5. String (0x43) values only.
Delete (DelTep) is deferred: the native DeleteTagExtendedPropertiesByName does a
client-side sync check and returns err 229 for a just-added property, so the
DelTep request never reaches the wire and its inBuff can't be captured yet.
Shipped: HistorianClient.AddTagExtendedPropertiesAsync/AddTagExtendedPropertyAsync;
HistorianTagExtendedPropertyProtocol.SerializeAddRequest; orchestrator path;
golden WcfTagExtendedPropertyWriteProtocolTests (4); gated live write/read-back test;
native-harness `add-tep` scenario + Capture-AddTagExtendedProperties.ps1 +
decode-add-tep-capture.py. Doc: wcf-add-tag-extended-properties.md. 233 tests green.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
|
||
|
Joseph Doherty
|
bc353df8c4 |
R1.10 RenameTagsAsync: async tag rename via History StartJob (StJb)
Tag rename has no dedicated WCF op — the (old,new) name batch rides the generic History StartJob (StJb) job buffer; the server returns a job id and applies renames asynchronously. Handle is the uppercase storage-session GUID, Open2 in write mode; reuses the write orchestrator's open+priming chain. jobBuffer layout (decoded + server-validated): byte[7] zero prefix + uint32 pairCount + per pair (uint32 oldCharCount + UTF-16 oldName + uint32 newCharCount + UTF-16 newName), order (old,new). The raw instrument capture mangles the final byte with MDAS chunk markers (the R1.1 lesson), so the golden fixture pins the CLEAN byte[] the SDK handed the channel (dumped via AVEVA_HISTORIAN_RENAME_DUMP) — the exact buffer the live server accepted and renamed with. Gated server-side by the AllowRenameTags system parameter (default 0): when disabled the native client rejects pre-wire (err 132); the managed SDK surfaces it as StartJob=false -> Accepted=false. Enabling needs a Historian config reload, not just a storage-engine restart. Shipped: HistorianClient.RenameTagAsync/RenameTagsAsync -> HistorianTagRenameResult; HistorianTagRenameProtocol; orchestrator RenameTags/SendStartJobRename; golden WcfTagRenameProtocolTests (4, pins server-accepted buffer); gated live test RenameTagsAsync_AgainstLocalHistorian_RenamesSandboxTag (passed end-to-end). Native-harness `rename` scenario + Capture-RenameTags.ps1 + decode-rename-capture.py. Doc: docs/reverse-engineering/wcf-rename-tags.md. 213 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
fbd839077b |
R1.4 GetHistorianInfo: bounded out on 2020 WCF (named-value-only, no struct)
Captured the native HistorianAccess.GetHistorianInfo(out HistorianInfo, out err) and decoded the wire: over 2020 WCF, GETHI is a named-value query whose only working key is "HistorianVersion" (response ~30 bytes = the version string). Probed 7 storage-mode key names -> all ok=False/err. The 518-byte HISTORIAN_INFO struct + EventStorageMode@514 is the 2023R2 HCAL-native/gRPC model (confirmed from the decompiled 2023R2 source); on 2020 the native client derives the mode outside the WCF wire. Version is already exposed (ProbeAsync/GetRuntimeParameterAsync), so no hollow GetHistorianInfoAsync is shipped (same disposition as R1.3 timezone). This completes the reachable 2020-WCF M1 read surface; remaining M1 = config writes (gated on explicit request) or gRPC/2023R2-only items. RE aids kept: harness `historian-info` scenario, Capture-HistorianInfo.ps1, decode-historian-info-capture.py, and StringHandleProbeDiagnosticTests .GETHI_CandidateInfoNames (asserts the named-value-only finding; gated). Docs: wcf-historian-info.md (new) + roadmap/matrix/wall-doc updates. 230 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
108220c36b |
R1.5 GetTagExtendedPropertiesAsync (GetTepByNm) + R1.6 closed (no op)
Ship tag extended-property reads over the 2020 WCF aa/Retr/GetTepByNm op: HistorianClient.GetTagExtendedPropertiesAsync(tag) -> name/value pairs. String-handle op reached with the Open2 storage-session GUID formatted uppercase (same format that unlocked GETRP/GETHI/ExeC). Routed via the name-based native path (GetTagExtendedPropertiesByName, server-fetch flag), not the index-based TagQuery path. Evidence-backed findings from the capture: - GetTepByNm (and GetTgByNm) succeed with the uppercase handle -- further validates the resolved string-handle wall. - QTB (StartTagQuery) does NOT punch through: captured uppercase, it still fails server-side (CMdServer::StartActiveTagnamesQuery over the aahMetadataServer pipe) -- a metadata-server blocker, not handle format. - R1.6 (localized properties) has NO distinct op (only error-message/UI-text localization in the managed client); collapses into R1.5. Closed, not throwing. Wire format (golden-pinned, synthetic bytes -- no dev tag names committed): - request tagNames = uint count + per-name(uint charCount + UTF-16) - response = uint tagCount + per-tag(marker + compact-ASCII name + uint propCount + per-prop(marker + compact-ASCII name + 0x43 VT_BSTR value) + trailer); sequence-paged. Adds: HistorianTagExtendedProperty model, HistorianTagExtendedPropertyProtocol (codec), HistorianWcfTagExtendedPropertyClient (orchestration), dialect + public API; golden WcfTagExtendedPropertyProtocolTests (4) + gated live test (HISTORIAN_TEP_TAG). Tooling: Capture-TagExtendedProperties.ps1, decode-tag-properties-capture.py, harness tag-extended-properties scenario. Docs: wcf-tag-extended-properties.md; roadmap R1.5 DONE / R1.6 collapsed; wall doc + memory updated with the QTB-server-side nuance. 228 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
4da5287d01 |
R1.2 GetRuntimeParameter + string-handle wall RESOLVED (handle-format bug)
Execute HCAL roadmap R1.2 (GetRuntimeParameterAsync) end-to-end, and in doing so
discover that the "string-handle wall" blocking R1.1/R1.4/R1.5/R1.6 was a handle
FORMAT bug, not a missing native session/filter registration.
R1.2 (shipped, live-verified):
- Captured native GetRuntimeParameter -> WCF op aa/Stat/GETRP (string-handle op,
GETHI's shape), via scripts/Capture-RuntimeParam.ps1 + instrument-wcf-{write,read}message.
- HistorianRuntimeParameterProtocol serializes pRequestBuff (54 67 01 00 + uint
nameCount + per-name uint charCount + UTF-16) and parses pResponseBuff (version +
uint resultCount + CRetVariant 0x43 VT_BSTR + uint16 len + uint16 charCount + UTF-16).
- IStatusServiceContract2.GetRuntimeParameter (GETRP) op; HistorianWcfStatusClient
passes the Open2 storage-session GUID as the string handle, UPPERCASE.
- Public HistorianClient.GetRuntimeParameterAsync(name) via the dialect.
- Golden WcfRuntimeParameterProtocolTests + gated live test; returns HistorianVersion.
String-handle wall RESOLVED (proven, public APIs deferred):
- The Open2 storage GUID works as the string handle when sent UPPERCASE
(ToString("D").ToUpperInvariant()); earlier "blocked" probes used lowercase.
- Live-probed GETHI (R1.4) -> returns data; ExeC (R1.1) -> Retr.GetV prime -> ExeC ->
GetR returns a BinaryFormatter-serialized .NET DataTable. Gated
StringHandleProbeDiagnosticTests + scripts/Capture-ExecSql.ps1 + exec-sql harness scenario.
- Docs flipped: wcf-string-handle-wall.md RESOLVED banner; roadmap R1.1/R1.4 reachable,
R1.5/R1.6 likely; wcf-status-localhost.md GETRP section.
- R1.1/R1.4 public APIs NOT shipped: ExeC needs a GetR paging loop + a BinaryFormatter-
stream parser (BinaryFormatter is removed from .NET 10); GETHI full-info struct needs
its own capture.
223 unit tests pass; gated live tests green against the local 2020 Historian.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
|
||
|
Joseph Doherty
|
6d470eab4a |
R1.7: server-side event filters — ReadEventsAsync(HistorianEventFilter), live-honored
Roadmap M1 R1.7. Filters are set on the native EventQuery object via AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs (time/count/order only). Found via a new harness --dump-type-members command. Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 + harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the operator field. Filter block (decoded byte-for-byte): ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0 The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching predicate returns 0 events; Type=Equal=User.Write returns only User.Write events. Verified live via both the native harness and the SDK. - HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload - HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native) - Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged) - Golden-byte tests (block match, op field, empty-filter regression) + gated live test Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via AddEventFilterCondition) framing is partially captured and not shipped. 216 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
f1e23a3a02 |
M2: implement SendEventAsync — event-send rides WCF AddS2, not the storage pipe
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event delivery uses the non-WCF storage-engine pipe (which would block it like revision writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2 (IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag, so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply. - R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two captures diffed to separate constant framing from value-dependent fields. - R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields + CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor + event Id + Namespace + EventType + version 5 + typed property bag. - R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 -> reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync. - R2.5: gated live test; server accepts the AddS2 (success, empty error buffer). Server requires delivered byte[].Length == declared packet length (uint32@0x04); the native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length mismatch"). Original events only (RevisionVersion=0) with string properties; other property types + revision/update/delete throw ProtocolEvidenceMissingException. Caveat (documented): accepted events are not persisted on the local dev box; the native client behaves identically (event ingestion pipeline inactive) — not an SDK gap. 212 unit tests pass; 16/16 event tests pass live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
1a7519c803 |
RE: resolve R1.8/R1.9 analog/state summary via request+response capture
Captured the native StartQuery2 pRequestBuff and the GetNextQueryResultBuffer2 response (instrument-wcf-writemessage + chained instrument-wcf-readmessage) and decoded both against AnalogSummaryHistory SQL ground truth. Conclusion: the rich multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF binary protocol — the response is the ordinary version-9 row buffer the existing aggregate parser already handles, carrying one value per cycle selected by RetrievalMode (QueryType 5-8), not ValueSelector (inert on this path). So "analog summary" == the existing ReadAggregateAsync; no new src/ code warranted. Tooling (tools/ + scripts/ only, nothing in src/): - NativeTraceHarness: drive summary knobs via --value-selector / --aggregation-type / --max-states (uint16) / --filter - Capture-SummaryRequest.ps1: repeatable instrument+stage+matrix capture, -WithResponse chains the ReadMessage hook - decode-summary-capture.py: StartQuery2 request diff vs baseline - decode-summary-response.py: response decode vs SQL ground truth Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
34e352ba28 |
R1.8/R1.9: empirical summary-query probe + enum-dump RE command
Pushed on recovering the summary query params. Findings: - Added `enum-dump` to the RE CLI (dumps a managed enum's literal members). Confirmed INSQL_QUERYTYPE / HISTORIAN_SUMMARYTYPE are value__-only in the managed metadata — their named members are native-side constants, so they can't be recovered statically. Same for CColumnNameMap.LoadColumnNameMap (column->bit built from native string/const data, not IL ldstr/ldc). - Live-probed StartQuery2 against SysTimeSec sweeping QueryType/SummaryType/ ColumnSelectorFlags. The server ACCEPTS SummaryType 1/2/4/5 (returns a valid version-9 buffer) but yields 0 rows; column flags don't change that; QueryType 15/16 don't exist. So a summary query is NOT Full+SummaryType+ flags — the config lives in the AutoSummaryParameters trailer (currently zeroed) and/or a native summary QueryType. Conclusion recorded in the plan: the request shape needs a NATIVE capture (instrument-wcf-writemessage on a real summary query), not managed-metadata recovery or blind probing. Decode targets remain located. No guessed code in src/; probe scaffolding removed. 208 tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
fa9cde3e2f |
CW-1: reusable capture -> sanitize -> golden-fixture pipeline
Adds the highest-leverage reverse-engineering primitive from the roadmap: one path to turn a live operation buffer into a committable golden fixture. Unblocks every capture-tier item (R0.5, R1.x, R2.1). - ProtocolCaptureSanitizer: redacts identity-bearing values (host, tag, user, machine) from a native buffer in BOTH ASCII and UTF-16LE, overwriting in place with an 'X' fill so length and every field offset are preserved (keeps the fixture useful for byte-layout RE). ASCII-letter matching is case-insensitive; secrets < 3 chars are skipped to avoid collision corruption. AssertNoSecretsRemain is a fail-closed safety net that refuses to emit if any value survives. - ProtocolFixtureWriter: serializes a capture to fixtures/protocol/<op>/<name>.json with sanitized hex, length, SHA-256 of the sanitized bytes, and a scrub report. Timestamps are passed in (deterministic / testable). - capture-tag-info CLI command: captures a live GetTagInfoFromName response and writes the fixture. The same native bytes ride inside 2023 R2 gRPC GetTagInfosFromName, so the fixture is transport-agnostic. - 11 unit tests for the sanitizer/writer (test project now references the RE tool). - First real fixture: get-tag-info/analog-*.json — a 98-byte Int4 CTagMetadata buffer captured live from the local Historian 2020 server, tag name redacted, verified to contain no identity (descriptor 03 c3 00 31 = Int4, as documented). 180 non-live unit tests green. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
b5e5f5485b |
D2: gate is in the C++ HistorianClient, not the managed wrapper
Direct HistorianAccess.AddNonStreamedValue (the 4-param overload that bypasses HistorianDataValueList and goes straight to HistorianClient.AddNonStreamedValueAsync) ALSO fails with 129 TagNotFoundInCache against SysTimeSec, even with validate=false. So the cache check is inside the native C++ HistorianClient's per-connection tag list — there's no managed-callable bypass. Critical insight discovered: the SDK doesn't use the C++ HistorianClient at all. It talks WCF directly. The cache gate that blocks the native wrapper may not block a managed WCF client because the gate is enforced by aahClientManaged, not by the WCF server. This shifts the recommendation for any future D2 attempt from "wrap the native API" (which is genuinely blocked) to "implement the wire path directly on top of the existing ITransactionServiceContract methods and test against the live server" (unverified but plausibly viable). The harness can't help with that path — the wrapper itself is the blocker we'd be bypassing. 177/177 tests still pass; harness gains --write-revision-direct flag for further probing of the native-wrapper path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
3af8a13059 |
D2 (revision-write): probe SysTimeSec — same gate, narrower scope
Extended the harness with --write-revision-target-tag <name> (overrides the value's TagKey via SQL lookup) and --write-revision-skip-validate (passes false to AddNonStreamedValue's `validate` boolean). Added --write-revision-commit gate so the harness validates without actually calling SendValues by default — important when targeting system tags. Probed SysTimeSec (wwTagKey=12, server-cache-resident system tag): - AddNonStreamedValue: ErrorCode=TagNotFoundInCache (129) — same failure - With validate=false: same failure (the cache check is intrinsic, not gated by the boolean) Conclusion: the gate is per-(client-session, tag), not per-server-cache. Even tags the SERVER cache knows about are rejected because the LIBRARY maintains a separate per-connection tag list that AddNonStreamedValue checks. That list isn't populated by knowing the wwTagKey alone — it needs whatever mechanism (RegisterTags2 / read flow side effect / IO server registration) that we haven't reverse-engineered. The revision-write path remains architecturally blocked for managed clients. Plan doc updated with the SysTimeSec finding. 177/177 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2feb56d52c |
D2 (revision-write): empirically blocked by same gate as AddS2
Drove the revision-write flow via reflection in the native trace harness (--write-revision-values) to see whether it bypasses the AddS2 architectural blocker. It doesn't. Findings: - HistorianAccess.CreateHistorianDataValueList(NonStreamedOriginal) succeeds - HistorianDataValueList.NonStreamedValuesBegin() succeeds (batchID 0->1) - HistorianDataValueList.AddNonStreamedValue(value, validate=true, out err) FAILS with ErrorCode=TagNotFoundInCache (129) — same client-side validation gate that blocks AddS2 - AddNonStreamedValuesEnd() returns void; SendValues() returns true with Success because the list is empty (no value was ever added) - No AddNonStreamValues* WCF calls reach the wire Conclusion: the revision-write path requires the tag to be in the library's runtime tag cache, which is only populated by configured IO server / Application Server pipelines, not by HistorianAccess.AddTag. This matches the architectural blocker documented for AddS2 and means no public WriteRevisionsAsync / BeginRevisionAsync should be added to the SDK — the path is unreachable for client-created sandbox tags. The Wcf/Contracts/ITransactionServiceContract methods (AddNonStream- ValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd) remain declared for completeness; no orchestrator or public surface is added. The harness extension is preserved as a deterministic reproducer for the blocker: re-run --write-revision-values to verify the gate any time. docs/plans/revision-write-path.md updated with the empirical finding plus the original plan retained as historical context. 177/177 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7e4d713eb3 |
Cross-platform NegotiateAuthentication; StorageType field; docs polish
HistorianSspiClient rewritten on top of System.Net.Security.NegotiateAuthentication
in place of P/Invoke into secur32.dll's InitializeSecurityContextW. The class
keeps the same Next() / Dispose() / two-constructor surface so callers don't
change. RequiredProtectionLevel=EncryptAndSign + AllowedImpersonationLevel=
Identification produces a request-flag set equivalent to the captured native
0x2081C / 0x81C bitmasks (still preserved as constants for the existing unit
tests). Removes the only Windows P/Invoke in the production SDK; the
[SupportedOSPlatform("windows")] gating elsewhere stays in place pending a
separate sweep.
HistorianStorageType (Cyclic = 1, Delta = 2):
Captured 2026-05-04 via --write-storage-type on the harness. Delta differs
from Cyclic in three places — header byte 10 (0x02 -> 0x06), flag-block
byte 1 (0x01 -> 0x02), and 4 zero bytes inserted after StorageRate before
the FILETIME. Server persists Tag.StorageType=1/2 accordingly. Plumbed
through HistorianTagDefinition.StorageType + serializer + orchestrator + 2
new tests (golden bytes + live SQL persistence verification).
Docs polish:
CLAUDE.md no longer claims "no P/Invoke" (HistorianSspiClient is the one
allowed P/Invoke surface); updated test count to 169+; AGENTS.md Required
SDK Surface and Repository Layout brought up to date with the live state
including the write surface; handoff.md "not a git working tree" obsolete
note removed.
171/171 tests pass with the NegotiateAuthentication replacement (was 169;
+2 new tests for StorageType).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
5ce62a5900 |
Wire ApplyScaling, StorageRate; close out write-commands plan
ApplyScaling (HistorianTagDefinition.ApplyScaling): The EnsT2 trailer's second byte controls server-side scaling — `FE 00` mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling set_ApplyScaling on the native harness and capturing the wire bytes for both values with identical inputs. The earlier docs claimed EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has no such operation — toggling that one byte is the whole fix. StorageRate (HistorianTagDefinition.StorageRateMs): Serializer accepts a non-default rate, validated empirically against the live server which only accepts quantized values (1000/5000/10000/60000/300000 ms). EnsureTagAsync upsert semantics: Second call on the same tag name with different fields succeeds and updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place (verified by direct SQL inspection in a live test). Plan + doc closeout: write-commands-reverse-engineering.md rewritten as a current-state plan with three workstreams (A doc closeout / B idempotency / C1 StorageRate) and a parallelism table; prior phase notes preserved as appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md, README.md updated to remove "writes are out of scope" / non-existent UpdateTags references and document the actual EnsT2 wire format including the `FE xx` trailer. Reverse-engineering harness gains --write-apply-scaling and a SQL post-check that prints the persisted AnalogTag bounds so future RE sessions can verify wire→DB causality without leaving the harness. 169/169 tests pass (was 165; +4 new tests covering ApplyScaling, StorageRate golden bytes, StorageRate live persistence, and EnsureTagAsync upsert semantics). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7a3cd9b76e |
Resolve write-path silent fails + expand EnsureTagAsync, RetrievalMode coverage
DelT and EnsT2 had two distinct silent-fail blockers; both now resolved live end-to-end. Read path's RetrievalMode mapping was missing 11 of 15 enum values (plus a latent Cyclic→4 bug). Investigation tooling kept as env-gated helpers. DelT silent fail: Open2 was using NativeIntegratedReadOnlyConnectionMode (0x402); server returned err 132 OperationNotEnabled silently. Added NativeIntegratedWriteEnabledConnectionMode (0x401) per HistorianAccessUtil.SetConnectionMode bit map (Process=1 | IntegratedSecurity=0x400). Write orchestrator now opens with write-enabled mode. EnsT2 silent fail: byte-by-byte comparison via inspector revealed two bugs in SerializeAnalogCTagMetadata. The original "146-byte byte-for-byte match" was misaligned — it omitted the leading 0x4E marker byte and treated WCF's `01 01 01` EndElement closing markers as if they were part of the InBuff payload. Real native InBuff is 144 bytes with 0x4E lead and 2-byte `FE 00` trailer. Golden test bytes corrected. EnsureTagAsync expansion: probed every analog data type via instrument-wcf-writemessage; byte 11 of CTagMetadata is the data-type discriminator (Float=0x01, Double=0x21, UInt2=0x09, UInt4=0x11, Int2=0x29, Int4=0x31). String/Int1/Int8/UInt8 fail at native AddTag — out of scope for this op. Range encoding decoded: defaults emit compact `1A 03`; non-default emit `1F 00` + 4 doubles in order MinEU/MaxEU/MinRaw/MaxRaw. MinRaw/MaxRaw sent on the wire but server mirrors them to MinEU/MaxEU when ApplyScaling=false (verified against native — server quirk, not SDK bug). RetrievalMode mapping: probed all 15 enum values; QueryType is just the native enum ordinal. Replaced the broken switch with `(uint)mode`. Existing SDK mapped Cyclic→4 (BestFit's value); Cyclic is actually 0. CLAUDE.md updated: stale "Active Protocol Blocker" rewritten as resolved-status block; SDK surface now reflects the read-blocker resolution and the new write ops; "Remaining gaps" punch list refreshed. Tools added (both env-gated, no runtime overhead unless flipped on): - HistorianWcfMessageCaptureBehavior — captures all WCF body bytes when AVEVA_HISTORIAN_SDK_WIRE_CAPTURE is set; used for byte-level diff vs native. - HistorianWcfHistAddressingBehavior — explicitly sets wsa:To header on the Hist channel for parity with native bytes (kept though not load-bearing). - WriteDiag in TagWriteOrchestrator — env-gated EnsT2/DelT response logging (AVEVA_HISTORIAN_DELT_DIAG). NativeTraceHarness CLI: added --write-min-eu/--write-max-eu/--write-min-raw/ --write-max-raw for capturing non-default-range EnsT2 payloads. Tests: 130 → 161 passing (+31). Includes 16-mode RetrievalMode mapping table, 4 per-data-type EnsT2 golden tests, NonDefaultRanges golden test, 6 live round-trip integration tests covering Float/Double/Int2/Int4/UInt4/FloatRanges, 3 live tests for previously-unmapped RetrievalMode values. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
dohertj2
|
cfc8d44e3a |
Implement EnsureTagAsync (live-verified) + DeleteTagAsync (DelT semantics partial)
New SDK surface:
HistorianClient.EnsureTagAsync(HistorianTagDefinition)
HistorianClient.DeleteTagAsync(string tagName)
Plumbing:
src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs
Public input model — TagName/Description/EngineeringUnit/DataType/MinEU/MaxEU.
Currently only HistorianDataType.Float is live-verified.
src/AVEVA.Historian.Client/Wcf/HistorianTagWriteProtocol.cs
SerializeAnalogCTagMetadata produces 146-byte payload byte-for-byte
identical to the captured native EnsT2(Float) request.
SerializeDeleteTagNames produces ushort 0x6751 + ushort 1 + uint count
+ per-tag (uint charCount + UTF-16 chars).
src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs
Both EnsT2 and DelT run the full Stat-priming chain captured for the
analog flow (UpdC3 + Stat.GetV ×3 + Stat.GETHI ×2 + 7× GetSystemParameter
+ Trx.GetV + Retr.GetV).
src/AVEVA.Historian.Client/Wcf/HistorianWcfTagClient.cs
MapDataType extended to accept tag-origin marker 0xC7 (SDK-created tags).
Tests:
5 golden-byte tests (HistorianTagWriteProtocolTests):
SerializeAnalogCTagMetadata byte-for-byte match against captured 146-byte fixture
SerializeAnalogCTagMetadata produces different bytes for different inputs
SerializeDeleteTagNames single-tag matches captured shape
SerializeDeleteTagNames multi-tag appends each
SerializeDeleteTagNames empty list throws
1 live integration test (gated by HISTORIAN_WRITE_SANDBOX_TAG):
EnsureTagAsync_AndDeleteTagAsync_RoundTrip_AgainstLocalHistorian
EnsureTagAsync creates the sandbox tag, GetTagMetadataAsync reads it
back. 130/130 tests pass.
Harness improvements:
--write-delete-after now runs DelT independently of AddStreamedValue
outcome.
HistorianTagStatusList constructed correctly for DeleteTags reflection
call (previous StringCollection attempt failed with TypeMismatch).
Known DelT gap: SDK's DeleteTagAsync returns true but server-side
cascading deletion does not always complete (the row remains in
Runtime.dbo.Tag). The captured native flow's DelT removes the tag
cleanly (verified via harness --write-delete-after), so something
around the WCF DelT call is missing from our orchestrator. Documented
as known issue with SMC-based cleanup as workaround.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
b3d22befd0 |
write-commands plan: AddS2 prereq is architectural - not implementable as generic client write
Three follow-up attempts to satisfy the AddS2 server-cache prereq all
failed at the same client-side gate before any AddS2 byte reached the
wire:
1. TagKey synthetic→real override. First attempt used the placeholder
TagKey=10000000 returned by HistorianAccess.AddTag. Native
AddStreamedValue refused with error 168 "Tag not added to server".
Harness now ALWAYS resolves the real wwTagKey from Runtime.dbo.Tag
after AddTag (logged as TagKeyOverride: Synthetic→RealFromSql).
Error code shifted to 129 "Tag not found in cache" — request now
reaches the server but the server's in-memory tag cache doesn't
know about the new tag.
2. Server-cache settle wait. Up to 8s sleep between AddTag and
AddStreamedValue (--write-resync-wait-seconds N). Wait period
contains 2× UpdC3 + 2× Trx/GetV keep-alives but no server-side
cache update — error 129 persists.
3. Fresh process / fresh connection. Skipped AddTag entirely
(--write-skip-add-tag) and ran AddStreamedValue alone against the
already-existing sandbox tag. New native client instance, new
client-side cache, new server session. SAME error 129 — no AddS2
bytes sent on wire. Capture confirms 44 records ending in Close2.
Interpretation: the Historian engine's runtime tag cache only ingests
tags from configured IOServers / Application Server data pipelines,
not from HistorianAccess.AddTag-only client flows. AddTag populates
Runtime.dbo.Tag (wwTagKey=240 was created) but doesn't register the
tag with the live cache that AddStreamedValue checks. That
registration happens server-side when an upstream data producer (an
OPC driver, AnE event subsystem, Application Server attribute store)
claims the tag.
WriteValueAsync therefore CANNOT be implemented as a generic client
API against this server architecture. The SDK's realistic writeable
surface is now narrowed to EnsureTagAsync + DeleteTagAsync only.
Harness changes:
- --write-skip-add-tag skip the AddTag call (for fresh-cache test)
- --write-skip-add-value skip the AddStreamedValue call (capture EnsT2 only)
- --write-resync-wait-seconds N sleep N seconds between AddTag and
AddStreamedValue (default 0)
- TagKey lookup now ALWAYS hits SQL after AddTag, not just when
the synthetic key is 0.
Plan doc updated with full Phase 2 follow-on findings + revised
remaining work (4-item checklist focused on EnsureTagAsync/
DeleteTagAsync, plus a stretch goal of probing AddRevisionValues*
against an existing-tag).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
b5f9a71fe7 |
write-commands plan: Phase 2 partial - capture EnsT2(Float) wire bytes
Per plan §1 in scope: EnsT2 for analog tags, AddS2, DelT.
Per plan §2 safety: localhost only, single sandbox tag
RetestSdkWriteSandbox, harness refuses any name not starting with
RetestSdkWrite, time-bounded writes, ReadOnly=false only when scenario
is "write".
Phase 2 actually executed:
1. tools/AVEVA.Historian.NativeTraceHarness/Program.cs extended with
--scenario write. New args:
--write-sandbox-tag <name> (default RetestSdkWriteSandbox)
--write-value <numeric> (default 42.5)
--write-data-type <name> (default Float)
--write-delete-after (best-effort cleanup)
Toggles ConnectionArgs.ReadOnly=false when scenario is "write" so
the connection accepts the write attempt instead of rejecting at
the harness boundary with error 132 "Operation is not enabled".
2. Sandbox tag RetestSdkWriteSandbox created in Runtime DB
(wwTagKey=240, AcquisitionType=2 Manual, StorageType=1 Cyclic)
via the harness's AddTag call. Single dedicated tag per safety §1.
3. Captured the full write-flow wire sequence at
artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/
bothmessage-write-capture-latest.ndjson (46 records, 23 outgoing +
23 incoming).
The chain is identical to the event flow except:
- EnsT2 payload is the 146-byte analog CTagMetadata instead of
the 83-byte event one
- NO RTag2 between Open2 and EnsT2 (events used RTag2 with
CmEventTagId)
4. The 146-byte analog CTagMetadata layout is dumped in the plan doc
for layout decoding. Visible fields (still being aligned against
CTagUtil.ConvertTagMetadataToHistorianTag IL at token 0x060055CE):
- tag name "RetestSdkWriteSandbox" (compact ASCII, len 21)
- 16 bytes of FF (CommonArchestraEventTypeId placeholder unused
for analog?)
- description "SDK write-RE sandbox tag" (compact ASCII, len 24)
- metadata provider "MDAS" (compact ASCII)
- engineering unit "test" (compact ASCII)
- Int64 FILETIME (date-created, year 2026)
- uint32 0x2710 = 10000 (storage-related, possibly StorageRate)
- double 1.0 (likely IntegralDivisor or scaling factor)
- 5-byte trailer FE 00 01 01 01 (matches event tag's
2F 27 01 01 01 shape)
5. AddS2 BLOCKED CLIENT-SIDE at error 168 "Tag not added to server".
Native AddStreamedValue refuses to send because the tag isn't in
the server's session cache, even though EnsT2 created it in the
Runtime DB. Likely needs RTag2(analog tag GUID) prereq similar
to the event flow's RTag2(CmEventTagId), or one of
aahClientCommon.CHistStorage.AddTagidPairs (token 0x0600202F) or
AddTagsWithServerTagId (token 0x06002026). AddS2 wire bytes NOT
captured this session.
6. scripts/decode-write-capture.py — sanitized decoder for the
capture, walks the 46 records and dumps the EnsT2 InBuff bytes
for layout work. No identity strings; only sandbox-chosen values
appear in output.
Phase 2 remaining work documented in the plan doc as a 5-item
checklist for the next session:
1. Decode the AddS2 prereq (likely RTag2 with analog tag GUID).
2. Capture AddS2 wire bytes once prereq is satisfied.
3. Implement HistorianAddTagsProtocol.SerializeAnalog/Discrete/
String CTagMetadata variants.
4. Implement HistorianAddStreamValuesProtocol.Serialize.
5. Implement public surface: EnsureTagAsync, WriteValueAsync,
DeleteTagAsync (golden-byte + gated live integration tests).
No SDK source changed — implementation deferred until AddS2 wire
bytes are in hand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
c95824a65d |
Initial commit: managed .NET 10 AVEVA Historian SDK + reverse-engineering toolkit
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|