6d470eab4a024216f9688d19fac6fcaf01dd3fec
7 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
6d470eab4a |
R1.7: server-side event filters — ReadEventsAsync(HistorianEventFilter), live-honored
Roadmap M1 R1.7. Filters are set on the native EventQuery object via AddEventFilter(property, HistorianComparisionType, value) — NOT EventQueryArgs (time/count/order only). Found via a new harness --dump-type-members command. Captured the native filtered StartEventQuery pRequestBuff (Capture-EventFilter.ps1 + harness --event-filter knob) and diffed Equal(0) vs Contains(12) to isolate the operator field. Filter block (decoded byte-for-byte): ushort 0 + uint filterCount + uint condCount + uint nameLen + name(UTF-16) + uint 1 + ushort op + uint 1 + value(0x09-LEN-0x00 compact-ASCII) + byte 0 The filter is REAL, not inert (unlike the analog-summary knobs): a non-matching predicate returns 0 events; Type=Equal=User.Write returns only User.Write events. Verified live via both the native harness and the SDK. - HistorianClient.ReadEventsAsync(start, end, HistorianEventFilter, ct) overload - HistorianEventFilter + HistorianEventComparison (18 ops, ordinals = native) - Filter encoding in HistorianEventQueryProtocol (empty-filter path unchanged) - Golden-byte tests (block match, op field, empty-filter regression) + gated live test Single string-valued predicate only; multi-filter (OR) / multi-condition (AND via AddEventFilterCondition) framing is partially captured and not shipped. 216 unit tests pass. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
f1e23a3a02 |
M2: implement SendEventAsync — event-send rides WCF AddS2, not the storage pipe
Roadmap Milestone 2 (event sending). Capture disproved the assumption that event delivery uses the non-WCF storage-engine pipe (which would block it like revision writes): a native AddStreamedValue(HistorianEvent) leaves over WCF as AddS2 (IHistoryServiceContract2.AddStreamValues2). CM_EVENT is a built-in registered tag, so the 129 TagNotFoundInCache gate that blocks AddS2 for user tags does not apply. - R2.1: NativeTraceHarness "event-send" scenario + Capture-EventSend.ps1; two captures diffed to separate constant framing from value-dependent fields. - R2.2: HistorianEventWriteProtocol serializes the AddS2 pBuf (storage sample buffer wrapping the event VTQ) — golden-byte tested. Decoded "OS" sig + length fields + CM_EVENT tag id + EventTime/ReceivedTime FILETIMEs + Opc 192 + 0x118D descriptor + event Id + Namespace + EventType + version 5 + typed property bag. - R2.3/R2.4: HistorianWcfEventOrchestrator.SendEventAsync (Open2 event-mode 0x501 -> reuse CM_EVENT RTag2/EnsT2 -> AddStreamValues2) + HistorianClient.SendEventAsync. - R2.5: gated live test; server accepts the AddS2 (success, empty error buffer). Server requires delivered byte[].Length == declared packet length (uint32@0x04); the native relies on the MDAS encoder adding a pad byte, so the SDK emits an explicit trailing 0x00 (else AddS2 rejects with "CValuStream buffer size vs packet length mismatch"). Original events only (RevisionVersion=0) with string properties; other property types + revision/update/delete throw ProtocolEvidenceMissingException. Caveat (documented): accepted events are not persisted on the local dev box; the native client behaves identically (event ingestion pipeline inactive) — not an SDK gap. 212 unit tests pass; 16/16 event tests pass live. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC |
||
|
Joseph Doherty
|
1a7519c803 |
RE: resolve R1.8/R1.9 analog/state summary via request+response capture
Captured the native StartQuery2 pRequestBuff and the GetNextQueryResultBuffer2 response (instrument-wcf-writemessage + chained instrument-wcf-readmessage) and decoded both against AnalogSummaryHistory SQL ground truth. Conclusion: the rich multi-aggregate analog/state summary struct is NOT delivered over the 2020 WCF binary protocol — the response is the ordinary version-9 row buffer the existing aggregate parser already handles, carrying one value per cycle selected by RetrievalMode (QueryType 5-8), not ValueSelector (inert on this path). So "analog summary" == the existing ReadAggregateAsync; no new src/ code warranted. Tooling (tools/ + scripts/ only, nothing in src/): - NativeTraceHarness: drive summary knobs via --value-selector / --aggregation-type / --max-states (uint16) / --filter - Capture-SummaryRequest.ps1: repeatable instrument+stage+matrix capture, -WithResponse chains the ReadMessage hook - decode-summary-capture.py: StartQuery2 request diff vs baseline - decode-summary-response.py: response decode vs SQL ground truth Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7288f39f5d |
Add Set-HistorianCredentials.ps1 for DPAPI-encrypted credential persistence
Drops a small helper for the gated live-integration tests that need
HISTORIAN_USER + HISTORIAN_PASSWORD set in the process environment
(currently GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian).
Three modes:
- Default: prompts for username (default <COMPUTERNAME>\<USERNAME>) and
password (silent), saves to %USERPROFILE%\.histsdk\credentials.xml via
Export-Clixml. The SecureString inside the PSCredential is DPAPI-encrypted
and decryptable only by the same Windows user account on the same machine.
- -Load: reads the saved credential and exports HISTORIAN_USER +
HISTORIAN_PASSWORD into the current PowerShell session's environment.
- -Clear: deletes the saved credential file.
Also accepts -Path to override the storage location (e.g. for keeping
multiple credential sets side by side) and -UserName to skip the username
prompt for password-only re-saves.
Stored under the user profile, never inside the repo, so it cannot be
committed accidentally. The file format is plain Export-Clixml — no custom
encoding shenanigans.
Live-verified locally: -Load + dotnet test passes the previously-skipped
GetTagMetadataAsync_ExplicitCredentials_AgainstLocalHistorian test against
the local Historian with IntegratedSecurity=false.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
b5f9a71fe7 |
write-commands plan: Phase 2 partial - capture EnsT2(Float) wire bytes
Per plan §1 in scope: EnsT2 for analog tags, AddS2, DelT.
Per plan §2 safety: localhost only, single sandbox tag
RetestSdkWriteSandbox, harness refuses any name not starting with
RetestSdkWrite, time-bounded writes, ReadOnly=false only when scenario
is "write".
Phase 2 actually executed:
1. tools/AVEVA.Historian.NativeTraceHarness/Program.cs extended with
--scenario write. New args:
--write-sandbox-tag <name> (default RetestSdkWriteSandbox)
--write-value <numeric> (default 42.5)
--write-data-type <name> (default Float)
--write-delete-after (best-effort cleanup)
Toggles ConnectionArgs.ReadOnly=false when scenario is "write" so
the connection accepts the write attempt instead of rejecting at
the harness boundary with error 132 "Operation is not enabled".
2. Sandbox tag RetestSdkWriteSandbox created in Runtime DB
(wwTagKey=240, AcquisitionType=2 Manual, StorageType=1 Cyclic)
via the harness's AddTag call. Single dedicated tag per safety §1.
3. Captured the full write-flow wire sequence at
artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/
bothmessage-write-capture-latest.ndjson (46 records, 23 outgoing +
23 incoming).
The chain is identical to the event flow except:
- EnsT2 payload is the 146-byte analog CTagMetadata instead of
the 83-byte event one
- NO RTag2 between Open2 and EnsT2 (events used RTag2 with
CmEventTagId)
4. The 146-byte analog CTagMetadata layout is dumped in the plan doc
for layout decoding. Visible fields (still being aligned against
CTagUtil.ConvertTagMetadataToHistorianTag IL at token 0x060055CE):
- tag name "RetestSdkWriteSandbox" (compact ASCII, len 21)
- 16 bytes of FF (CommonArchestraEventTypeId placeholder unused
for analog?)
- description "SDK write-RE sandbox tag" (compact ASCII, len 24)
- metadata provider "MDAS" (compact ASCII)
- engineering unit "test" (compact ASCII)
- Int64 FILETIME (date-created, year 2026)
- uint32 0x2710 = 10000 (storage-related, possibly StorageRate)
- double 1.0 (likely IntegralDivisor or scaling factor)
- 5-byte trailer FE 00 01 01 01 (matches event tag's
2F 27 01 01 01 shape)
5. AddS2 BLOCKED CLIENT-SIDE at error 168 "Tag not added to server".
Native AddStreamedValue refuses to send because the tag isn't in
the server's session cache, even though EnsT2 created it in the
Runtime DB. Likely needs RTag2(analog tag GUID) prereq similar
to the event flow's RTag2(CmEventTagId), or one of
aahClientCommon.CHistStorage.AddTagidPairs (token 0x0600202F) or
AddTagsWithServerTagId (token 0x06002026). AddS2 wire bytes NOT
captured this session.
6. scripts/decode-write-capture.py — sanitized decoder for the
capture, walks the 46 records and dumps the EnsT2 InBuff bytes
for layout work. No identity strings; only sandbox-chosen values
appear in output.
Phase 2 remaining work documented in the plan doc as a 5-item
checklist for the next session:
1. Decode the AddS2 prereq (likely RTag2 with analog tag GUID).
2. Capture AddS2 wire bytes once prereq is satisfied.
3. Implement HistorianAddTagsProtocol.SerializeAnalog/Discrete/
String CTagMetadata variants.
4. Implement HistorianAddStreamValuesProtocol.Serialize.
5. Implement public surface: EnsureTagAsync, WriteValueAsync,
DeleteTagAsync (golden-byte + gated live integration tests).
No SDK source changed — implementation deferred until AddS2 wire
bytes are in hand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
5310952ab2 |
Extend HistorianTagMetadata with Description, EngineeringUnit, MinEU/MaxEU
Decoded the GetTagInfoFromName response shape across multiple tag types via captured raw bytes (sanitized decoder script in scripts/decode-taginfo-bytes.py): - Compact-ASCII string slot count varies by tag origin: 2 strings for MDAS-routed external tags (TagName + MetadataProvider), 4 strings for local Sys tags (TagName + Description + ItemName + CreatedBy). Parser now walks strings dynamically until the next byte isn't the 0x09 marker. - Trailing region after the 4-byte fixed block holds (for analog tags) two doubles for MinEU/MaxEU plus an optional EngineeringUnit compact ASCII string and other fields whose exact positions vary. Parser uses a tolerant scan: tries each 8-byte alignment 0..7, picks the first sane (Min ≤ Max, finite, not all-zeros, |x| ≤ 1e15) double pair as MinEU/MaxEU, and finds the first plausible compact ASCII string (1..32 ASCII bytes, not numeric) as EngineeringUnit. HistorianTagMetadata.Description / EngineeringUnit / MinRaw / MaxRaw nullable slots already existed; they're now populated. Live verification: SysTimeSec returns Description="System Time : Seconds", MaxRaw=59.0, EngineeringUnit ="Seconds". Tests: 109 → 114 (+4 synthetic-fixture parser tests + 1 live integration test for the populated analog metadata path). Bulk descriptor probe helper (GetTagInfoRawBytesForProbe) added for future layout work; raw bytes never committed because they contain CreatedBy DOMAIN\user identity. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
dohertj2
|
c95824a65d |
Initial commit: managed .NET 10 AVEVA Historian SDK + reverse-engineering toolkit
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|