histsdk/docs/plans/write-commands-reverse-engineering.md

Plan: Reverse-Engineering Write Commands

Status (2026-05-04, post-ApplyScaling landing)

Phase 2 is complete. The write surface that the server actually supports for managed clients is implemented and live-verified:

• EnsureTagAsync for analog tags (Float, Double, Int2, Int4, UInt4), with optional ApplyScaling=true for distinct MinRaw/MaxRaw persistence (AnalogTag.Scaling=1).
• DeleteTagAsync for any tag created via the SDK.

Architecturally blocked / out of scope:

• AddS2 (write samples) — server's runtime cache only ingests from configured IOServers / Application Server pipelines, not from client-only AddTag flows. The native wrapper hits the same wall; this is a server architecture decision, not a protocol gap.
• Discrete / String / Int1 / Int8 / UInt8 tag types — fail at native HistorianAccess.AddTag before any wire bytes leave the client. Likely require a different code path (AddTagExtendedProperties or pre-population via SMC); not investigated.

This plan covers the residual workstreams.

Workstreams

A. Documentation closeout

Status docs across the repo still describe write commands as "in progress" or speculate about a non-existent UpdateTags operation. Closing those out so future agents don't re-walk the same dead ends.

Step	File	Action
A1	docs/reverse-engineering/handoff.md	Add a "Write-flow status" note marking EnsT2/DelT live; remove stale write-blocker callouts
A2	docs/reverse-engineering/implementation-status.md	Flip EnsT2 / DelT rows from "out of scope" to "implemented"; add ApplyScaling row
A3	docs/reverse-engineering/wcf-contract-evidence.md	Add evidence rows for EnsT2(analog) and DelT pointing at the captured fixtures
A4	README.md	Operation-status table reflects the two write ops

B. EnsT2 idempotency / update behavior

We don't currently know what happens when EnsureTagAsync is called against a tag name that already exists with different fields. Three plausible outcomes: server errors, server silently updates, server no-ops. This affects how callers should think about the API (create-only vs upsert).

Step	Action
B1	Add a live integration test that calls EnsureTagAsync twice on the same tag name with different MinEU/MaxEU/Description; query SQL after each call to capture observed behavior
B2	Document the observed contract in HistorianTagDefinition doc-comment and (if surprising) in CLAUDE.md

C. Expose currently-hardcoded CTagMetadata fields

The serializer hardcodes StorageRate=1000ms, StorageType=Cyclic, IntegralDivisor=1.0, and a few flag-block bytes. The server accepts those defaults so existing tests pass, but customers building tags with non-default rates can't currently express that.

Step	Field	Effort	Notes
C1	StorageRate (uint32 ms)	small — wire field is already at a known offset, just plumb a parameter through	Default stays 1000ms
C2	StorageType (Cyclic / Delta)	medium — need a comparison capture to find which byte in the flag block encodes it	Deferred unless customer asks
C3	IntegralDivisor (double)	small — wire field already known	Deferred unless customer asks

C1 is the only one I'm executing in this round. C2/C3 are listed for completeness; pick them up when there's a concrete request.

D. Deferred — no current evidence or customer ask

ID	Item	Why deferred
D1	AddTagExtendedProperties / DeleteTagExtendedProperties	No wire captures yet; no customer ask
D2	AddRevisionValuesBegin/Value/End (revision-write path)	Multi-step capture needed against an existing historized tag; complex; no customer ask
D3	Discrete/String/Int1/Int8/UInt8 EnsT2 root cause	Native AddTag fails for these — likely requires an entirely different code path; would need a fresh capture and IL walkthrough

Parallelism

Track	Files touched	Conflicts with
A1	docs/reverse-engineering/handoff.md	none
A2	docs/reverse-engineering/implementation-status.md	none
A3	docs/reverse-engineering/wcf-contract-evidence.md	none
A4	README.md	none
B	tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs, src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs	C1 (same HistorianTagDefinition file)
C1	src/AVEVA.Historian.Client/Models/HistorianTagDefinition.cs, src/AVEVA.Historian.Client/Wcf/HistorianTagWriteProtocol.cs, src/AVEVA.Historian.Client/Wcf/HistorianWcfTagWriteOrchestrator.cs, tests/AVEVA.Historian.Client.Tests/HistorianTagWriteProtocolTests.cs, tests/AVEVA.Historian.Client.Tests/HistorianClientIntegrationTests.cs	B (same HistorianTagDefinition and integration-test file)

Concurrency-safe groupings:

• A1–A4 are pairwise independent and pairwise independent of B and C1 → can be assigned to four agents in parallel.
• B and C1 both touch HistorianTagDefinition.cs. Sequence them: B first (it just adds a doc-comment) then C1 (which adds a field).

In a single-agent execution (this session), the order is: A* batched edits → B → C1 → build/test → commit.

Success Criteria

• A1–A4: documentation reflects the actual current write surface and removes references to non-existent operations (UpdateTags).
• B: a passing live test that asserts the observed double-EnsT2 behavior, plus a doc-comment update on HistorianTagDefinition.
• C1: HistorianTagDefinition.StorageRateMs field exposed, default preserves existing wire output, golden test for a non-default rate, live test that creates a tag with a non-default rate and asserts via SQL that Tag.CurrentEditorUserKey (or the storage-rate column, TBD) reflects the value.
• All 165+ tests still pass; no regression in existing live tests.

---

Appendix: Prior Phase Notes

The historical phase logs that drove the implementation are preserved below for context. They describe the path from "write surface unknown" to "write surface implemented and live-verified" as it unfolded through 2026-05-04. Anything in this appendix that contradicts the current Status section above is superseded.

Phase 1 findings (recorded, not implementing)

§3.4 ModifyData/DeleteData — ELIMINATED FROM SCOPE

methods aahClientManaged.dll returns no managed wrapper for any of: EditValue, ModifyValue, EditData, DeleteData, ModifyData, OverwriteData. Per the plan's §3.4 disposition rule, this op is REST-only / SMC-only and remains out of scope for the SDK.

§4.a Native serializers identified

The wrapper does have managed-public write API:

Public method	Token	Used for
ArchestrA.HistorianAccess.AddTag	0x0600619A	Creates a new tag (drives EnsT2)
ArchestrA.HistorianAccess.AddStreamedValue	0x0600618C/D/E (3 overloads)	Pushes one timestamped value (drives AddS2)
ArchestrA.HistorianAccess.AddNonStreamedValue	0x0600618F/90 (2 overloads)	Pushes one timestamped value, non-stream mode
ArchestrA.HistorianAccess.DeleteTags	0x060061A4	Removes tags (drives DelT)
ArchestrA.HistorianAccess.AddVersionedStreamedValue	0x0600616F	Pushes versioned value (rev edit)
ArchestrA.HistorianAccess.AddRevisionValuesBegin/Value/End/AddRevisionValues	0x06006175-77, 0x0600617F	Multi-row revision write

Native serializer for EnsT2: <Module>.CTagUtil.ConvertTagMetadataToHistorianTag at token 0x060055CE. WCF wrapper for AddS2: <Module>.CHistoryConnectionWCF.AddStreamValuesToHistorian at token 0x0600404C.

Phase 2 follow-on findings (2026-05-04, second pass)

The AddS2 prereq is architectural, not protocol-level. Three follow-up attempts to trigger AddS2 from the sandbox harness all hit a client-side gate before any AddS2 byte reaches the wire:

1. TagKey synthetic→real override: even with the real wwTagKey the server returns error 129 "Tag not found in cache".
2. Server-cache settle wait of 8s: error 129 persists.
3. Fresh process / fresh connection (skip AddTag): error 129; no AddS2 bytes sent on wire.

The Historian engine's runtime tag cache only ingests tags from configured IOServers / Application Server pipelines, not from HistorianAccess.AddTag-only flows. WriteValueAsync cannot be implemented as a generic client API against this server architecture.

Phase 2 results (write captures + EnsureTagAsync/DeleteTagAsync)

EnsT2 + DelT priming chain captured (no RTag2 between Open2 and EnsT2):

Hist.GetV → Hist.GetI ×2 → Hist.ValCl ×2 → Hist.Open2 →
Stat.GetV ×2 → Stat.GETHI ×2 → Hist.UpdC3 →
Stat.GetSystemParameter ×7 → Trx.GetV → Stat.GetV → Retr.GetV →
Hist.EnsT2 → Hist.Close2

Open2 with NativeIntegratedWriteEnabledConnectionMode = 0x401 (Process | Write | IntegratedSecurity) is required — read-mode 0x402 makes the server return err 132 OperationNotEnabled silently. The analog Float CTagMetadata payload is 144 bytes with a leading 0x4E marker byte and a 2-byte trailer FE xx where the second byte is the ApplyScaling flag (00 = false, 01 = true).

ApplyScaling resolution (2026-05-04)

Earlier docs claimed "MinRaw is mirrored to MinEU — server quirk, not SDK bug". That conclusion was based on tests that always set ApplyScaling=false on the native side. Re-running with set_ApplyScaling(true) on the harness and capturing wire bytes for both values revealed:

• ApplyScaling=false → trailer = FE 00 → server mirrors MinRaw→MinEU, sets AnalogTag.Scaling=0
• ApplyScaling=true → trailer = FE 01 → server persists distinct MinRaw/MaxRaw, sets AnalogTag.Scaling=1

The IHistoryServiceContract2 surface has no UpdateTags operation. Distinct MinRaw/MaxRaw persistence is achieved entirely by toggling that one byte in the EnsT2 payload. The SDK now exposes this via HistorianTagDefinition.ApplyScaling.

Capture artifacts: artifacts/reverse-engineering/apply-scaling-experiment/enst2-applyscaling-{false,true}.ndjson.

Original goal section (preserved for historical reference)

"Write commands work" was originally defined as the four ops: EnsT2, AddS2, DelT, and ModifyData/DeleteData. The realized scope is EnsT2 + DelT only. AddS2 is permanently blocked by server architecture; ModifyData/DeleteData were eliminated by static analysis (no managed wrapper exists). The AddRevisionValuesBegin/Value/End chain remains a stretch goal (item D2 in the current plan) — it was never investigated because no SDK consumer has asked for revision writes.

Original safety rules (still applicable)

• Single dedicated sandbox tag per RE session, name must start with RetestSdkWrite.
• Never write to any tag named in HISTORIAN_TEST_TAG, HISTORIAN_TAG_FILTER, the docs, or the captured RE ndjson.
• Time bounds on writes: every test uses DateTime.UtcNow so writes land inside the live RealTimeWindow / FutureTimeThreshold.
• localhost only; no customer / corporate hosts.
• Sanitization scan after every session.
• Write captures live in artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/ (gitignored).