Wire ApplyScaling, StorageRate; close out write-commands plan

ApplyScaling (HistorianTagDefinition.ApplyScaling):
The EnsT2 trailer's second byte controls server-side scaling — `FE 00`
mirrors MinRaw to MinEU and sets AnalogTag.Scaling=0; `FE 01` persists
distinct MinRaw/MaxRaw and sets Scaling=1. Decoded by toggling
set_ApplyScaling on the native harness and capturing the wire bytes for
both values with identical inputs. The earlier docs claimed
EnsureTagAsync needed a follow-up "UpdateTags" call; the WCF surface has
no such operation — toggling that one byte is the whole fix.

StorageRate (HistorianTagDefinition.StorageRateMs):
Serializer accepts a non-default rate, validated empirically against
the live server which only accepts quantized values
(1000/5000/10000/60000/300000 ms).

EnsureTagAsync upsert semantics:
Second call on the same tag name with different fields succeeds and
updates Description, MinEU, MaxEU, MinRaw, MaxRaw, Scaling in place
(verified by direct SQL inspection in a live test).

Plan + doc closeout:
write-commands-reverse-engineering.md rewritten as a current-state
plan with three workstreams (A doc closeout / B idempotency / C1
StorageRate) and a parallelism table; prior phase notes preserved as
appendix. handoff.md, implementation-status.md, wcf-contract-evidence.md,
README.md updated to remove "writes are out of scope" / non-existent
UpdateTags references and document the actual EnsT2 wire format
including the `FE xx` trailer.

Reverse-engineering harness gains --write-apply-scaling and a SQL
post-check that prints the persisted AnalogTag bounds so future RE
sessions can verify wire→DB causality without leaving the harness.

169/169 tests pass (was 165; +4 new tests covering ApplyScaling,
StorageRate golden bytes, StorageRate live persistence, and
EnsureTagAsync upsert semantics).

Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
This commit is contained in:
Joseph Doherty
2026-05-04 22:04:27 -04:00
parent a175c6e5a0
commit 5ce62a5900
13 changed files with 561 additions and 721 deletions
+18 -3
View File
@@ -1,6 +1,6 @@
# AVEVA Historian Managed Driver Handoff
Last updated: 2026-05-04 (event-flow prereqs)
Last updated: 2026-05-04 (write surface live: EnsT2 + DelT + ApplyScaling)
## Project Direction
@@ -12,7 +12,7 @@ Do not pivot to REST or a P/Invoke production shim unless the project
requirements change. Native and P/Invoke tools in this repo are reverse
engineering aids only.
Required production surface remains narrowly scoped:
Required production surface (all live-verified):
- `ProbeAsync`
- `ReadRawAsync`
@@ -21,8 +21,23 @@ Required production surface remains narrowly scoped:
- `ReadEventsAsync`
- `BrowseTagNamesAsync`
- `GetTagMetadataAsync`
- Status helpers: `GetConnectionStatusAsync`, `GetStoreForwardStatusAsync`, `GetSystemParameterAsync`
Writes are out of scope for the current pass.
Write surface (added 2026-05-04 by explicit user request — see
`docs/plans/write-commands-reverse-engineering.md` Status section):
- `EnsureTagAsync` for analog Float / Double / Int2 / Int4 / UInt4
(with optional `ApplyScaling=true` for distinct MinRaw / MaxRaw
persistence — server sets `AnalogTag.Scaling=1` when the EnsT2
trailer's second byte is `0x01` instead of `0x00`).
- `DeleteTagAsync`.
`AddS2` (write samples) is **architecturally blocked** — server
runtime cache only ingests from configured IOServers / Application
Server pipelines. Discrete / String / Int1 / Int8 / UInt8 EnsT2 fail
at native `AddTag` and are unsupported. There is no `UpdateTags`
operation on the WCF surface; the misnomer in earlier write-up
drafts has been removed.
## Repository Map
@@ -3,13 +3,23 @@
## Completed
- Production SDK targets `net10.0` and has no AVEVA binary references.
- Public API now includes the intended parity surface:
- Public API includes the full intended parity surface:
- TCP probe
- raw, aggregate, at-time, and block history reads
- event reads
- tag browse and metadata calls
- connection, store-forward, and system-parameter status calls
- write-back intentionally remains out of scope for this read-only SDK pass
- **`EnsureTagAsync`** for analog Float/Double/Int2/Int4/UInt4 with
optional `ApplyScaling=true` for distinct MinRaw/MaxRaw persistence
(live-verified end-to-end against `localhost`; SQL post-check confirms
`AnalogTag.Scaling=1` and distinct raw bounds when the flag is set)
- **`DeleteTagAsync`** (live-verified)
- **AddS2 (write samples) is architecturally blocked** — server runtime
cache only ingests from configured IOServers / Application Server
pipelines, not from `HistorianAccess.AddTag`-only flows. Three
independent reproduction attempts confirmed the same
`129 "Tag not found in cache"` failure even with the real wwTagKey,
fresh sessions, and 8s settle waits. Not a protocol gap.
- Internal protocol scaffolding exists:
- `HistorianConnection`
- `HistorianFrameReader`
@@ -159,5 +159,33 @@ the earlier speculative raw-frame layer.
handle `0`. See `wcf-status-localhost.md`.
- Query request and response byte-buffer layouts are still proprietary payloads
inside WCF operations such as `StartQuery` and `GetNextQueryResultBuffer`.
- Write payload layouts remain out of scope until read/query payloads are
decoded and fixture-backed.
- Write payload layouts decoded for the two supported ops:
- `Hist.EnsT2(analog)` 144-byte `CTagMetadata` `InBuff` payload —
leading `0x4E` marker, fixed 10-byte signature, 1-byte CDataType
discriminator (`0x01` Float / `0x21` Double / `0x09` UInt2 / `0x11`
UInt4 / `0x29` Int2 / `0x31` Int4), 16 zero placeholder bytes,
compact-ASCII tag name, 16 bytes of `0xFF`, compact-ASCII description,
compact-ASCII `MDAS`, 7-byte flag block, uint32 storage rate,
int64 FILETIME, scaling block (compact `1A 03` for default
0/100/0/100 ranges OR `1F 00` + 4 doubles MinEU/MaxEU/MinRaw/MaxRaw
for explicit), compact-ASCII engineering unit, uint32 `0x2710`
constant, double 1.0 (IntegralDivisor), 2-byte trailer `FE xx`
where `xx` is the ApplyScaling flag (`0x00` false / `0x01` true).
Live-verified: with `0x01` the server persists distinct
MinRaw/MaxRaw and sets `AnalogTag.Scaling=1`; with `0x00` it
mirrors MinRaw to MinEU. Captured fixtures live at
`artifacts/reverse-engineering/instrumented-wcf-writemessage-writes/`
(default ranges) and
`artifacts/reverse-engineering/apply-scaling-experiment/` (both
ApplyScaling values for the same input ranges). Connection mode is
`0x401` (Process | Write | IntegratedSecurity) — the read-mode
`0x402` makes the server return err 132 silently.
- `Hist.DelT` `tagNames` byte buffer — `ushort 0x6751`, `ushort 1`,
`uint32 tagCount`, then per tag `uint32 charCount + UTF-16-LE chars`.
Decoded via wire capture against the sandbox tag.
- `Hist.AddS2` (write samples) is architecturally blocked — server
runtime cache requires IOServer / Application Server pipeline
registration, not just a `Tag` row in `Runtime.dbo`. Three
reproduction attempts (real wwTagKey, fresh session, 8s settle
wait) confirmed `129 "Tag not found in cache"` is the gate. No
AddS2 wire bytes leave the client.