main
10 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Joseph Doherty
|
b5e5f5485b |
D2: gate is in the C++ HistorianClient, not the managed wrapper
Direct HistorianAccess.AddNonStreamedValue (the 4-param overload that bypasses HistorianDataValueList and goes straight to HistorianClient.AddNonStreamedValueAsync) ALSO fails with 129 TagNotFoundInCache against SysTimeSec, even with validate=false. So the cache check is inside the native C++ HistorianClient's per-connection tag list — there's no managed-callable bypass. Critical insight discovered: the SDK doesn't use the C++ HistorianClient at all. It talks WCF directly. The cache gate that blocks the native wrapper may not block a managed WCF client because the gate is enforced by aahClientManaged, not by the WCF server. This shifts the recommendation for any future D2 attempt from "wrap the native API" (which is genuinely blocked) to "implement the wire path directly on top of the existing ITransactionServiceContract methods and test against the live server" (unverified but plausibly viable). The harness can't help with that path — the wrapper itself is the blocker we'd be bypassing. 177/177 tests still pass; harness gains --write-revision-direct flag for further probing of the native-wrapper path. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
3af8a13059 |
D2 (revision-write): probe SysTimeSec — same gate, narrower scope
Extended the harness with --write-revision-target-tag <name> (overrides the value's TagKey via SQL lookup) and --write-revision-skip-validate (passes false to AddNonStreamedValue's `validate` boolean). Added --write-revision-commit gate so the harness validates without actually calling SendValues by default — important when targeting system tags. Probed SysTimeSec (wwTagKey=12, server-cache-resident system tag): - AddNonStreamedValue: ErrorCode=TagNotFoundInCache (129) — same failure - With validate=false: same failure (the cache check is intrinsic, not gated by the boolean) Conclusion: the gate is per-(client-session, tag), not per-server-cache. Even tags the SERVER cache knows about are rejected because the LIBRARY maintains a separate per-connection tag list that AddNonStreamedValue checks. That list isn't populated by knowing the wwTagKey alone — it needs whatever mechanism (RegisterTags2 / read flow side effect / IO server registration) that we haven't reverse-engineered. The revision-write path remains architecturally blocked for managed clients. Plan doc updated with the SysTimeSec finding. 177/177 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
2feb56d52c |
D2 (revision-write): empirically blocked by same gate as AddS2
Drove the revision-write flow via reflection in the native trace harness (--write-revision-values) to see whether it bypasses the AddS2 architectural blocker. It doesn't. Findings: - HistorianAccess.CreateHistorianDataValueList(NonStreamedOriginal) succeeds - HistorianDataValueList.NonStreamedValuesBegin() succeeds (batchID 0->1) - HistorianDataValueList.AddNonStreamedValue(value, validate=true, out err) FAILS with ErrorCode=TagNotFoundInCache (129) — same client-side validation gate that blocks AddS2 - AddNonStreamedValuesEnd() returns void; SendValues() returns true with Success because the list is empty (no value was ever added) - No AddNonStreamValues* WCF calls reach the wire Conclusion: the revision-write path requires the tag to be in the library's runtime tag cache, which is only populated by configured IO server / Application Server pipelines, not by HistorianAccess.AddTag. This matches the architectural blocker documented for AddS2 and means no public WriteRevisionsAsync / BeginRevisionAsync should be added to the SDK — the path is unreachable for client-created sandbox tags. The Wcf/Contracts/ITransactionServiceContract methods (AddNonStream- ValuesBegin/AddNonStreamValues/AddNonStreamValuesEnd) remain declared for completeness; no orchestrator or public surface is added. The harness extension is preserved as a deterministic reproducer for the blocker: re-run --write-revision-values to verify the gate any time. docs/plans/revision-write-path.md updated with the empirical finding plus the original plan retained as historical context. 177/177 tests still pass. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7e4d713eb3 |
Cross-platform NegotiateAuthentication; StorageType field; docs polish
HistorianSspiClient rewritten on top of System.Net.Security.NegotiateAuthentication
in place of P/Invoke into secur32.dll's InitializeSecurityContextW. The class
keeps the same Next() / Dispose() / two-constructor surface so callers don't
change. RequiredProtectionLevel=EncryptAndSign + AllowedImpersonationLevel=
Identification produces a request-flag set equivalent to the captured native
0x2081C / 0x81C bitmasks (still preserved as constants for the existing unit
tests). Removes the only Windows P/Invoke in the production SDK; the
[SupportedOSPlatform("windows")] gating elsewhere stays in place pending a
separate sweep.
HistorianStorageType (Cyclic = 1, Delta = 2):
Captured 2026-05-04 via --write-storage-type on the harness. Delta differs
from Cyclic in three places — header byte 10 (0x02 -> 0x06), flag-block
byte 1 (0x01 -> 0x02), and 4 zero bytes inserted after StorageRate before
the FILETIME. Server persists Tag.StorageType=1/2 accordingly. Plumbed
through HistorianTagDefinition.StorageType + serializer + orchestrator + 2
new tests (golden bytes + live SQL persistence verification).
Docs polish:
CLAUDE.md no longer claims "no P/Invoke" (HistorianSspiClient is the one
allowed P/Invoke surface); updated test count to 169+; AGENTS.md Required
SDK Surface and Repository Layout brought up to date with the live state
including the write surface; handoff.md "not a git working tree" obsolete
note removed.
171/171 tests pass with the NegotiateAuthentication replacement (was 169;
+2 new tests for StorageType).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
Joseph Doherty
|
5ce62a5900 |
Wire ApplyScaling, StorageRate; close out write-commands plan
ApplyScaling (HistorianTagDefinition.ApplyScaling): The EnsT2 trailer's second byte controls server-side scaling — `FE 00` mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling set_ApplyScaling on the native harness and capturing the wire bytes for both values with identical inputs. The earlier docs claimed EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has no such operation — toggling that one byte is the whole fix. StorageRate (HistorianTagDefinition.StorageRateMs): Serializer accepts a non-default rate, validated empirically against the live server which only accepts quantized values (1000/5000/10000/60000/300000 ms). EnsureTagAsync upsert semantics: Second call on the same tag name with different fields succeeds and updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place (verified by direct SQL inspection in a live test). Plan + doc closeout: write-commands-reverse-engineering.md rewritten as a current-state plan with three workstreams (A doc closeout / B idempotency / C1 StorageRate) and a parallelism table; prior phase notes preserved as appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md, README.md updated to remove "writes are out of scope" / non-existent UpdateTags references and document the actual EnsT2 wire format including the `FE xx` trailer. Reverse-engineering harness gains --write-apply-scaling and a SQL post-check that prints the persisted AnalogTag bounds so future RE sessions can verify wire→DB causality without leaving the harness. 169/169 tests pass (was 165; +4 new tests covering ApplyScaling, StorageRate golden bytes, StorageRate live persistence, and EnsureTagAsync upsert semantics). Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
Joseph Doherty
|
7a3cd9b76e |
Resolve write-path silent fails + expand EnsureTagAsync, RetrievalMode coverage
DelT and EnsT2 had two distinct silent-fail blockers; both now resolved live end-to-end. Read path's RetrievalMode mapping was missing 11 of 15 enum values (plus a latent Cyclic→4 bug). Investigation tooling kept as env-gated helpers. DelT silent fail: Open2 was using NativeIntegratedReadOnlyConnectionMode (0x402); server returned err 132 OperationNotEnabled silently. Added NativeIntegratedWriteEnabledConnectionMode (0x401) per HistorianAccessUtil.SetConnectionMode bit map (Process=1 | IntegratedSecurity=0x400). Write orchestrator now opens with write-enabled mode. EnsT2 silent fail: byte-by-byte comparison via inspector revealed two bugs in SerializeAnalogCTagMetadata. The original "146-byte byte-for-byte match" was misaligned — it omitted the leading 0x4E marker byte and treated WCF's `01 01 01` EndElement closing markers as if they were part of the InBuff payload. Real native InBuff is 144 bytes with 0x4E lead and 2-byte `FE 00` trailer. Golden test bytes corrected. EnsureTagAsync expansion: probed every analog data type via instrument-wcf-writemessage; byte 11 of CTagMetadata is the data-type discriminator (Float=0x01, Double=0x21, UInt2=0x09, UInt4=0x11, Int2=0x29, Int4=0x31). String/Int1/Int8/UInt8 fail at native AddTag — out of scope for this op. Range encoding decoded: defaults emit compact `1A 03`; non-default emit `1F 00` + 4 doubles in order MinEU/MaxEU/MinRaw/MaxRaw. MinRaw/MaxRaw sent on the wire but server mirrors them to MinEU/MaxEU when ApplyScaling=false (verified against native — server quirk, not SDK bug). RetrievalMode mapping: probed all 15 enum values; QueryType is just the native enum ordinal. Replaced the broken switch with `(uint)mode`. Existing SDK mapped Cyclic→4 (BestFit's value); Cyclic is actually 0. CLAUDE.md updated: stale "Active Protocol Blocker" rewritten as resolved-status block; SDK surface now reflects the read-blocker resolution and the new write ops; "Remaining gaps" punch list refreshed. Tools added (both env-gated, no runtime overhead unless flipped on): - HistorianWcfMessageCaptureBehavior — captures all WCF body bytes when AVEVA_HISTORIAN_SDK_WIRE_CAPTURE is set; used for byte-level diff vs native. - HistorianWcfHistAddressingBehavior — explicitly sets wsa:To header on the Hist channel for parity with native bytes (kept though not load-bearing). - WriteDiag in TagWriteOrchestrator — env-gated EnsT2/DelT response logging (AVEVA_HISTORIAN_DELT_DIAG). NativeTraceHarness CLI: added --write-min-eu/--write-max-eu/--write-min-raw/ --write-max-raw for capturing non-default-range EnsT2 payloads. Tests: 130 → 161 passing (+31). Includes 16-mode RetrievalMode mapping table, 4 per-data-type EnsT2 golden tests, NonDefaultRanges golden test, 6 live round-trip integration tests covering Float/Double/Int2/Int4/UInt4/FloatRanges, 3 live tests for previously-unmapped RetrievalMode values. Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com> |
||
|
dohertj2
|
cfc8d44e3a |
Implement EnsureTagAsync (live-verified) + DeleteTagAsync (DelT semantics partial)
New SDK surface:
HistorianClient.EnsureTagAsync(HistorianTagDefinition)
HistorianClient.DeleteTagAsync(string tagName)
Plumbing:
src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs
Public input model — TagName/Description/EngineeringUnit/DataType/MinEU/MaxEU.
Currently only HistorianDataType.Float is live-verified.
src/AVEVA.Historian.Client/Wcf/HistorianTagWriteProtocol.cs
SerializeAnalogCTagMetadata produces 146-byte payload byte-for-byte
identical to the captured native EnsT2(Float) request.
SerializeDeleteTagNames produces ushort 0x6751 + ushort 1 + uint count
+ per-tag (uint charCount + UTF-16 chars).
src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs
Both EnsT2 and DelT run the full Stat-priming chain captured for the
analog flow (UpdC3 + Stat.GetV ×3 + Stat.GETHI ×2 + 7× GetSystemParameter
+ Trx.GetV + Retr.GetV).
src/AVEVA.Historian.Client/Wcf/HistorianWcfTagClient.cs
MapDataType extended to accept tag-origin marker 0xC7 (SDK-created tags).
Tests:
5 golden-byte tests (HistorianTagWriteProtocolTests):
SerializeAnalogCTagMetadata byte-for-byte match against captured 146-byte fixture
SerializeAnalogCTagMetadata produces different bytes for different inputs
SerializeDeleteTagNames single-tag matches captured shape
SerializeDeleteTagNames multi-tag appends each
SerializeDeleteTagNames empty list throws
1 live integration test (gated by HISTORIAN_WRITE_SANDBOX_TAG):
EnsureTagAsync_AndDeleteTagAsync_RoundTrip_AgainstLocalHistorian
EnsureTagAsync creates the sandbox tag, GetTagMetadataAsync reads it
back. 130/130 tests pass.
Harness improvements:
--write-delete-after now runs DelT independently of AddStreamedValue
outcome.
HistorianTagStatusList constructed correctly for DeleteTags reflection
call (previous StringCollection attempt failed with TypeMismatch).
Known DelT gap: SDK's DeleteTagAsync returns true but server-side
cascading deletion does not always complete (the row remains in
Runtime.dbo.Tag). The captured native flow's DelT removes the tag
cleanly (verified via harness --write-delete-after), so something
around the WCF DelT call is missing from our orchestrator. Documented
as known issue with SMC-based cleanup as workaround.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
b3d22befd0 |
write-commands plan: AddS2 prereq is architectural - not implementable as generic client write
Three follow-up attempts to satisfy the AddS2 server-cache prereq all
failed at the same client-side gate before any AddS2 byte reached the
wire:
1. TagKey synthetic→real override. First attempt used the placeholder
TagKey=10000000 returned by HistorianAccess.AddTag. Native
AddStreamedValue refused with error 168 "Tag not added to server".
Harness now ALWAYS resolves the real wwTagKey from Runtime.dbo.Tag
after AddTag (logged as TagKeyOverride: Synthetic→RealFromSql).
Error code shifted to 129 "Tag not found in cache" — request now
reaches the server but the server's in-memory tag cache doesn't
know about the new tag.
2. Server-cache settle wait. Up to 8s sleep between AddTag and
AddStreamedValue (--write-resync-wait-seconds N). Wait period
contains 2× UpdC3 + 2× Trx/GetV keep-alives but no server-side
cache update — error 129 persists.
3. Fresh process / fresh connection. Skipped AddTag entirely
(--write-skip-add-tag) and ran AddStreamedValue alone against the
already-existing sandbox tag. New native client instance, new
client-side cache, new server session. SAME error 129 — no AddS2
bytes sent on wire. Capture confirms 44 records ending in Close2.
Interpretation: the Historian engine's runtime tag cache only ingests
tags from configured IOServers / Application Server data pipelines,
not from HistorianAccess.AddTag-only client flows. AddTag populates
Runtime.dbo.Tag (wwTagKey=240 was created) but doesn't register the
tag with the live cache that AddStreamedValue checks. That
registration happens server-side when an upstream data producer (an
OPC driver, AnE event subsystem, Application Server attribute store)
claims the tag.
WriteValueAsync therefore CANNOT be implemented as a generic client
API against this server architecture. The SDK's realistic writeable
surface is now narrowed to EnsureTagAsync + DeleteTagAsync only.
Harness changes:
- --write-skip-add-tag skip the AddTag call (for fresh-cache test)
- --write-skip-add-value skip the AddStreamedValue call (capture EnsT2 only)
- --write-resync-wait-seconds N sleep N seconds between AddTag and
AddStreamedValue (default 0)
- TagKey lookup now ALWAYS hits SQL after AddTag, not just when
the synthetic key is 0.
Plan doc updated with full Phase 2 follow-on findings + revised
remaining work (4-item checklist focused on EnsureTagAsync/
DeleteTagAsync, plus a stretch goal of probing AddRevisionValues*
against an existing-tag).
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
b5f9a71fe7 |
write-commands plan: Phase 2 partial - capture EnsT2(Float) wire bytes
Per plan §1 in scope: EnsT2 for analog tags, AddS2, DelT.
Per plan §2 safety: localhost only, single sandbox tag
RetestSdkWriteSandbox, harness refuses any name not starting with
RetestSdkWrite, time-bounded writes, ReadOnly=false only when scenario
is "write".
Phase 2 actually executed:
1. tools/AVEVA.Historian.NativeTraceHarness/Program.cs extended with
--scenario write. New args:
--write-sandbox-tag <name> (default RetestSdkWriteSandbox)
--write-value <numeric> (default 42.5)
--write-data-type <name> (default Float)
--write-delete-after (best-effort cleanup)
Toggles ConnectionArgs.ReadOnly=false when scenario is "write" so
the connection accepts the write attempt instead of rejecting at
the harness boundary with error 132 "Operation is not enabled".
2. Sandbox tag RetestSdkWriteSandbox created in Runtime DB
(wwTagKey=240, AcquisitionType=2 Manual, StorageType=1 Cyclic)
via the harness's AddTag call. Single dedicated tag per safety §1.
3. Captured the full write-flow wire sequence at
artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/
bothmessage-write-capture-latest.ndjson (46 records, 23 outgoing +
23 incoming).
The chain is identical to the event flow except:
- EnsT2 payload is the 146-byte analog CTagMetadata instead of
the 83-byte event one
- NO RTag2 between Open2 and EnsT2 (events used RTag2 with
CmEventTagId)
4. The 146-byte analog CTagMetadata layout is dumped in the plan doc
for layout decoding. Visible fields (still being aligned against
CTagUtil.ConvertTagMetadataToHistorianTag IL at token 0x060055CE):
- tag name "RetestSdkWriteSandbox" (compact ASCII, len 21)
- 16 bytes of FF (CommonArchestraEventTypeId placeholder unused
for analog?)
- description "SDK write-RE sandbox tag" (compact ASCII, len 24)
- metadata provider "MDAS" (compact ASCII)
- engineering unit "test" (compact ASCII)
- Int64 FILETIME (date-created, year 2026)
- uint32 0x2710 = 10000 (storage-related, possibly StorageRate)
- double 1.0 (likely IntegralDivisor or scaling factor)
- 5-byte trailer FE 00 01 01 01 (matches event tag's
2F 27 01 01 01 shape)
5. AddS2 BLOCKED CLIENT-SIDE at error 168 "Tag not added to server".
Native AddStreamedValue refuses to send because the tag isn't in
the server's session cache, even though EnsT2 created it in the
Runtime DB. Likely needs RTag2(analog tag GUID) prereq similar
to the event flow's RTag2(CmEventTagId), or one of
aahClientCommon.CHistStorage.AddTagidPairs (token 0x0600202F) or
AddTagsWithServerTagId (token 0x06002026). AddS2 wire bytes NOT
captured this session.
6. scripts/decode-write-capture.py — sanitized decoder for the
capture, walks the 46 records and dumps the EnsT2 InBuff bytes
for layout work. No identity strings; only sandbox-chosen values
appear in output.
Phase 2 remaining work documented in the plan doc as a 5-item
checklist for the next session:
1. Decode the AddS2 prereq (likely RTag2 with analog tag GUID).
2. Capture AddS2 wire bytes once prereq is satisfied.
3. Implement HistorianAddTagsProtocol.SerializeAnalog/Discrete/
String CTagMetadata variants.
4. Implement HistorianAddStreamValuesProtocol.Serialize.
5. Implement public surface: EnsureTagAsync, WriteValueAsync,
DeleteTagAsync (golden-byte + gated live integration tests).
No SDK source changed — implementation deferred until AddS2 wire
bytes are in hand.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|
||
|
dohertj2
|
c95824a65d |
Initial commit: managed .NET 10 AVEVA Historian SDK + reverse-engineering toolkit
Full read-only SDK (src/AVEVA.Historian.Client) implementing the CLAUDE.md required
surface against AVEVA Historian's binary WCF protocol — no native AVEVA runtime
dependency. All operations live-verified against a local Historian:
- ProbeAsync, ReadRawAsync, ReadAggregateAsync, ReadAtTimeAsync, ReadEventsAsync
- BrowseTagNamesAsync, GetTagMetadataAsync (17 native data-type codes mapped)
- GetConnectionStatusAsync, GetStoreForwardStatusAsync, GetSystemParameterAsync
- 108/108 unit + integration tests pass
Includes the reverse-engineering toolkit (tools/AVEVA.Historian.ReverseEngineering)
used to decode the protocol: WCF probes, IL inspection via dnlib, and IL-rewrite
instrumentation (instrument-wcf-{write,read}message etc.) plus the .NET Framework
trace harness (tools/AVEVA.Historian.NativeTraceHarness) for parity testing.
Sanitized handoff evidence under docs/reverse-engineering/. Native AVEVA binaries
(current/, aveva-install-x64/, aveva-install-x86/) are gitignored — fetch separately
from the AVEVA installer.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
|