dohertj2/histsdk
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
8 Commits 1 Branch 0 Tags
b5f9a71fe7c92fb7c7edbb92ee07e6186939b42f
Commit Graph

6 Commits

Author SHA1 Message Date
dohertj2 b5f9a71fe7 write-commands plan: Phase 2 partial - capture EnsT2(Float) wire bytes 
Per plan §1 in scope: EnsT2 for analog tags, AddS2, DelT.
Per plan §2 safety: localhost only, single sandbox tag
RetestSdkWriteSandbox, harness refuses any name not starting with
RetestSdkWrite, time-bounded writes, ReadOnly=false only when scenario
is "write".

Phase 2 actually executed:

1. tools/AVEVA.Historian.NativeTraceHarness/Program.cs extended with
   --scenario write. New args:
     --write-sandbox-tag <name>  (default RetestSdkWriteSandbox)
     --write-value <numeric>     (default 42.5)
     --write-data-type <name>    (default Float)
     --write-delete-after        (best-effort cleanup)
   Toggles ConnectionArgs.ReadOnly=false when scenario is "write" so
   the connection accepts the write attempt instead of rejecting at
   the harness boundary with error 132 "Operation is not enabled".

2. Sandbox tag RetestSdkWriteSandbox created in Runtime DB
   (wwTagKey=240, AcquisitionType=2 Manual, StorageType=1 Cyclic)
   via the harness's AddTag call. Single dedicated tag per safety §1.

3. Captured the full write-flow wire sequence at
   artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/
   bothmessage-write-capture-latest.ndjson (46 records, 23 outgoing +
   23 incoming).

   The chain is identical to the event flow except:
     - EnsT2 payload is the 146-byte analog CTagMetadata instead of
       the 83-byte event one
     - NO RTag2 between Open2 and EnsT2 (events used RTag2 with
       CmEventTagId)

4. The 146-byte analog CTagMetadata layout is dumped in the plan doc
   for layout decoding. Visible fields (still being aligned against
   CTagUtil.ConvertTagMetadataToHistorianTag IL at token 0x060055CE):
     - tag name "RetestSdkWriteSandbox" (compact ASCII, len 21)
     - 16 bytes of FF (CommonArchestraEventTypeId placeholder unused
       for analog?)
     - description "SDK write-RE sandbox tag" (compact ASCII, len 24)
     - metadata provider "MDAS" (compact ASCII)
     - engineering unit "test" (compact ASCII)
     - Int64 FILETIME (date-created, year 2026)
     - uint32 0x2710 = 10000 (storage-related, possibly StorageRate)
     - double 1.0 (likely IntegralDivisor or scaling factor)
     - 5-byte trailer FE 00 01 01 01 (matches event tag's
       2F 27 01 01 01 shape)

5. AddS2 BLOCKED CLIENT-SIDE at error 168 "Tag not added to server".
   Native AddStreamedValue refuses to send because the tag isn't in
   the server's session cache, even though EnsT2 created it in the
   Runtime DB. Likely needs RTag2(analog tag GUID) prereq similar
   to the event flow's RTag2(CmEventTagId), or one of
   aahClientCommon.CHistStorage.AddTagidPairs (token 0x0600202F) or
   AddTagsWithServerTagId (token 0x06002026). AddS2 wire bytes NOT
   captured this session.

6. scripts/decode-write-capture.py — sanitized decoder for the
   capture, walks the 46 records and dumps the EnsT2 InBuff bytes
   for layout work. No identity strings; only sandbox-chosen values
   appear in output.

Phase 2 remaining work documented in the plan doc as a 5-item
checklist for the next session:
  1. Decode the AddS2 prereq (likely RTag2 with analog tag GUID).
  2. Capture AddS2 wire bytes once prereq is satisfied.
  3. Implement HistorianAddTagsProtocol.SerializeAnalog/Discrete/
     String CTagMetadata variants.
  4. Implement HistorianAddStreamValuesProtocol.Serialize.
  5. Implement public surface: EnsureTagAsync, WriteValueAsync,
     DeleteTagAsync (golden-byte + gated live integration tests).

No SDK source changed — implementation deferred until AddS2 wire
bytes are in hand.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 07:55:27 -04:00
dohertj2 e3c003d978 write-commands plan: execute Phase 1 discovery (no DB writes) 
Phase 1 of docs/plans/write-commands-reverse-engineering.md is purely
discovery — static IL inspection, public-API enumeration, scope
elimination — and produces no DB writes. Phase 2 requires extending
NativeTraceHarness with a write scenario AND explicit operator approval
per the plan's safety rule §1, so it is deferred to a separate session.

Phase 1 findings recorded inline in the plan's status header:

1. §3.4 ModifyData/DeleteData eliminated — no managed wrapper exists.
   `methods` returns zero hits for EditValue, ModifyValue, EditData,
   DeleteData, ModifyData, OverwriteData. Per the plan's own §3.4
   disposition rule, this op is REST/SMC-only.

2. §4.a native serializer tokens identified for Phase 2:

   Public managed-wrapper write API in ArchestrA.HistorianAccess:
   - AddTag (0x0600619A)
   - AddStreamedValue × 3 overloads (0x0600618C/D/E)
   - AddNonStreamedValue × 2 overloads (0x0600618F/90)
   - DeleteTags (0x060061A4)
   - AddRevisionValuesBegin / AddRevisionValue / AddRevisionValuesEnd
     / AddRevisionValues (0x06006175-77, 0x0600617F)
     — covers the bulk-modify use case the eliminated ModifyData would
     have served. Worth folding into Phase 2 scope.

   Native serializers:
   - CTagUtil.ConvertTagMetadataToHistorianTag (0x060055CE) — 412 IL
     instructions; builds CTagMetadata for analog/discrete/string.
     Calls every CTagMetadata accessor we'd need to surface
     (Min/MaxRaw, Min/MaxEU, Unit, Message0/1, MaxLength,
     IntegralDivisor, DefaultTagRate, RolloverValue) plus all the
     CDataType predicates already decoded.
   - CHistoryConnectionWCF.AddStreamValuesToHistorian (0x0600404C)
     — confirms the on-wire shape matches our existing
     IHistoryServiceContract2.AddStreamValues2 declaration.

3. Chicken-and-egg resolved: the first §3.1 EnsT2 test creates the
   sandbox tag itself, so no SMC step is needed.

4. Open question §8.6 answered — the wrapper exposes both
   AddStreamedValue and AddNonStreamedValue, so the SDK should
   eventually surface both real-time and backfill write modes.

Phase 2 next steps recorded inline in the plan as a 5-item executable
checklist (extend harness with --scenario write, capture, decode
EnsT2(analog) + AddS2 bytes, implement public surface, gated live
tests).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 07:41:34 -04:00
dohertj2 6888b8c55a Wire SDK for remote-TCP end to end; live-verify RemoteTcpIntegrated 
Executes docs/plans/tcp-connection-validation.md. Full read-only SDK
surface now works against a remote AVEVA Historian over Net.TCP with
Windows transport authentication. 124/124 tests pass; the +10 new live
integration tests in RemoteTcpIntegrationTests.cs are gated by
HISTORIAN_REMOTE_TCP_HOST + HISTORIAN_REMOTE_TCP_TAG.

Two SDK bugs found while executing the plan:

1. Historian2020ProtocolDialect.ReadRawAsync / ReadAggregateAsync /
   ReadAtTimeAsync / ReadEventsAsync had explicit
   `if (_options.Transport != HistorianTransport.LocalPipe) return Missing<T>`
   guards. These were a guardrail from before the orchestrators handled
   TCP; the orchestrators have always used CreateBindingPair(options)
   which dispatches on transport correctly. Gates removed.

2. HistorianWcfStatusClient and HistorianWcfEventOrchestrator hardcoded
   HistorianWcfBindingFactory.CreatePipeEndpointAddress for the auxiliary
   services (Stat, Trx, Retr). Worked for LocalPipe; for TCP it produced
   an EndpointAddress with scheme net.pipe attached to a TCP binding
   (channel factory rejected the URI). Worse, when only the endpoint was
   transport-aware, the binding still requested a Windows-transport-
   security upgrade that the Stat endpoint over TCP doesn't support
   (auxiliaries don't repeat the auth — the Hist session is already
   authenticated). Added two helpers:
   - HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(options, name)
     -> net.pipe for LocalPipe, net.tcp for remote
   - HistorianWcfBindingFactory.CreateAuxiliaryBinding(options)
     -> NamedPipe for LocalPipe, plain MdasNetTcpBinding for remote
   Both call sites updated.

Live verification against the remote (probed previously in prior
sessions; reachability re-confirmed today):
- ProbeAsync over RemoteTcpIntegrated and RemoteTcpCertificate
- ReadRawAsync (8 samples returned for SysTimeSec)
- ReadAggregateAsync (TimeWeightedAverage, 1-min cycle, 10-min window)
- ReadAtTimeAsync (3 timestamps)
- BrowseTagNamesAsync (finds the test tag)
- GetTagMetadataAsync (full metadata populated)
- ReadEventsAsync (chain runs without throwing)
- GetConnectionStatusAsync (ConnectedToServer=true)
- GetSystemParameterAsync (HistorianVersion="20,0,000,000")

The default 'NT SERVICE\aahClientAccessPoint' SPN turned out to work
for the remote too — discovery workstream A (SPN-finding) was not
needed in practice.

README and the TCP plan doc updated to reflect the executed status.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 07:33:50 -04:00
dohertj2 1b31c24c8d Plan TCP connection validation (live verification of the existing remote-TCP plumbing) 
docs/plans/tcp-connection-validation.md (308 lines):
  Plan to live-verify the RemoteTcpIntegrated and RemoteTcpCertificate
  transports against an actual remote AVEVA Historian. The SDK's
  HistorianWcfBindingFactory already builds all three bindings
  (CreateMdasNetTcpBinding, CreateMdasNetTcpWindowsBinding,
  CreateMdasNetTcpCertificateBinding) but only LocalPipe has been
  exercised end-to-end. Wire format is identical across transports;
  only WCF binding shape and credential negotiation differ.

  Discovery workstreams A/B/C run in parallel (SPN discovery via static
  IL + WCF probe; cert binding requirements via wcf-cert-probe; operator
  preconditions checklist). D blocks on A. Verification tracks V1-V5 also
  parallelize once V1 (ProbeAsync) confirms the transport is reachable.
  Includes risks (SPN mismatch, cert chain validation, idle disconnect,
  Open2 response delta, compression negotiation, time skew, false-positive
  empty reads), success criteria, eight open questions, and explicit
  out-of-scope items filed under the existing write-commands and
  store-forward plans.

  No code changes; no preconditions assumed met. Implementer must satisfy
  §2 preconditions (reachable remote Historian, port 32568 open, test
  account, SPN registered, etc.) before §4 discovery starts.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 07:20:39 -04:00
dohertj2 6f01b83313 Plan two reverse-engineering campaigns: write commands + store/forward cache 
docs/plans/write-commands-reverse-engineering.md (425 lines):
  Plan for adding WriteValueAsync (AddS2 stream values), EnsureTags2 for
  analog/discrete/string tags, and DelT for sandbox cleanup. Hard safety
  rules center on a dedicated sandbox tag gated by env var, time-bounded
  writes, SQL ground-truth verification per session, explicit rollback.
  Five-step RE workflow mirrors the read/event decode (static IL discovery
  -> instrument-wcf-writemessage capture -> instrument-wcf-readmessage
  capture -> byte/IL alignment -> managed serializer + golden-byte tests).
  Risks call out auth-chain unknowns, parameter-name-mismatch class,
  silent-success failure modes, History-vs-Storage service question.

docs/plans/store-forward-cache-reverse-engineering.md (501 lines):
  Plan for replacing the synthesized GetStoreForwardStatusAsync with a
  real implementation. Architecture investigation already partially
  answered via IL inspection during planning: ArchestrA.HistorianAccess.
  GetStoreForwardStatus (token 0x06006187) reads an in-process C struct
  via calli to mdas_GetStorageStatus, kept current by server-pushed WCF
  callbacks (IStatusServiceContract2.SetStoreForwardEvent). CSFConnection.
  GetSFPipeName indicates a separate Named Pipe sidecar exists when SF
  is configured. Five parallelizable discovery workstreams, six concrete
  RE steps with cited tokens, eight risks, eight success criteria.

Both plans deliberately produce no code changes and no captures. They
exist so the next implementer can start with full context.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 07:16:32 -04:00
dohertj2 c95824a65d Initial commit: managed .NET 10 AVEVA Historian SDK + reverse-engineering toolkit 
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:

- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass

Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.

Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
 2026-05-04 06:31:48 -04:00