R1.11 DelTep capture + R1.3/R1.4/R1.12/R1.13 bounded out

DelTep (extended-property delete) — wire format captured + serializer
golden-proven, but live delete is server-blocked and NOT exposed publicly:
- Captured the DelTep inBuff via a cross-session trick (harness add-tep gains
  --tep-skip-add + read-for-sync before --tep-delete; Capture-DeleteTagExtended
  Properties.ps1 / decode-del-tep-capture.py). Layout = same group framing as
  AddTEx but property-name-only (no 0x43 value) + 0x00 group trailer.
- SerializeDeleteRequest + 4 golden tests pin the server-accepted buffer.
- A decisive experiment shows SDK-added properties ARE deletable (the native
  client read-syncs and deletes one), so SDK-add is complete; the SDK's own
  DelTep is rejected by CHistStorage::DeleteTagExtendedProperties even with
  byte-identical inBuff, matching mode/handle, GetTgByNm+GetTepByNm prime, open
  channel, and 60s retries. Root cause: the native multiplexes services over one
  connection (per-connection working set); the SDK's per-service WCF channels
  don't reproduce it. Kept as documented-but-blocked internal orchestrator path;
  no public HistorianClient delete API.

Bounded out with evidence (no code; docs + roadmap + probe):
- R1.12 localized-property write — no op on 2020 (mirror of R1.6); no
  *LocalizedPropert*/TagLocalized* symbol in any current/*.dll.
- R1.13 non-analog tag create — GATED; native AddTag rejects every non-analog
  type client-side (ValidationFailed, before any WCF op): SingleByteString,
  DoubleByteString, Int1 all fail, Float works. No Discrete type in the native
  enum, no TagType setter. No wire request to capture.
- R1.3 timezone + R1.4 EventStorageMode — re-confirmed 2023R2/gRPC-only from
  the Runtime DB schema (no timezone param, no EventStorageMode anywhere) and a
  parameter-op probe (GetSystemParameter + GETRP return null/throw for every
  candidate; only HistorianVersion works).

238 unit tests pass; full solution builds with 0 warnings.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01B6mcaT2PjRFKcogzp9UkfC
This commit is contained in:
Joseph Doherty
2026-06-21 11:26:21 -04:00
parent 08b950caee
commit c1b1b3d23b
14 changed files with 732 additions and 32 deletions
@@ -201,6 +201,15 @@ public sealed class HistorianClient : IAsyncDisposable
return AddTagExtendedPropertiesAsync(tag, [new HistorianTagExtendedProperty(name, value ?? string.Empty)], cancellationToken);
}
// Extended-property DELETE (DelTep) is intentionally NOT exposed publicly. Its wire format is
// captured and the serializer (HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest) is
// golden-verified against a server-accepted buffer, but the SDK cannot yet make the 2020 server
// accept the delete: the server's CHistStorage::DeleteTagExtendedProperties consults a
// per-connection working set that the native client populates by multiplexing GetTepByNm and
// DelTep over ONE connection, which the SDK's per-service WCF channels don't reproduce. See the
// documented-but-blocked path in HistorianWcfTagWriteOrchestrator and
// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// <summary>
/// Creates or updates the named tag in the Historian Runtime database via
/// <c>EnsureTags2</c>. Currently only <see cref="HistorianDataType.Float"/> is
@@ -108,6 +108,59 @@ internal static class HistorianTagExtendedPropertyProtocol
return stream.ToArray();
}
/// <summary>
/// Serializes the <c>DelTep</c> (DeleteTagExtendedProperties) inBuff for a single tag's
/// properties identified by name. Same group framing as <see cref="SerializeAddRequest"/> but
/// each property carries its name only (no <c>0x43</c> value variant) and the group trailer byte
/// is <c>0x00</c> (Add uses <c>0x01</c>). Decoded from a native
/// <c>DeleteTagExtendedPropertiesByName</c> capture (see
/// <c>docs/reverse-engineering/wcf-add-tag-extended-properties.md</c> §Delete):
/// <code>
/// uint32 groupCount (= 1)
/// byte 0x01 (group marker)
/// 0x09 + uint16 byteLen + ASCII tagName
/// uint32 propertyCount
/// repeated: byte 0x02 (property marker) + 0x09 + uint16 byteLen + ASCII propName
/// byte 0x00 (group trailer)
/// byte 0x00 (buffer terminator)
/// </code>
/// The native <c>deleteFromServer</c> argument is not in the buffer — it is the client-side flag
/// that decides whether the wire op fires at all (the SDK always sends the server-delete form,
/// the only one that produces a <c>DelTep</c> call).
/// </summary>
public static byte[] SerializeDeleteRequest(string tagName, IReadOnlyList<string> propertyNames)
{
ArgumentException.ThrowIfNullOrEmpty(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
using MemoryStream stream = new();
Span<byte> u32 = stackalloc byte[4];
BinaryPrimitives.WriteUInt32LittleEndian(u32, 1u); // group count
stream.Write(u32);
stream.WriteByte(GroupMarker);
WriteCompactAscii(stream, tagName);
BinaryPrimitives.WriteUInt32LittleEndian(u32, checked((uint)propertyNames.Count));
stream.Write(u32);
foreach (string name in propertyNames)
{
ArgumentException.ThrowIfNullOrEmpty(name, nameof(propertyNames));
stream.WriteByte(PropertyMarker);
WriteCompactAscii(stream, name); // delete is by name only — no value variant
}
stream.WriteByte(0x00); // group trailer (captured: 0x00 for delete, vs 0x01 for add)
stream.WriteByte(0x00); // buffer terminator
return stream.ToArray();
}
private static void WriteCompactAscii(MemoryStream stream, string value)
{
byte[] ascii = Encoding.ASCII.GetBytes(value);
@@ -53,6 +53,29 @@ internal sealed class HistorianWcfTagWriteOrchestrator
return Task.Run(() => AddTagExtendedProperties(tagName, properties), cancellationToken);
}
/// <summary>
/// DelTep (DeleteTagExtendedProperties) — <b>investigated but server-blocked, not wired to the
/// public API.</b> The inBuff serializer is golden-verified against a server-accepted native
/// capture, and an SDK-added property is deletable (the native client read-syncs and deletes it).
/// But the SDK's own delete is rejected by <c>CHistStorage::DeleteTagExtendedProperties</c>
/// (SErrorException) even with matching Open2 mode/handle/inBuff and a GetTgByNm+GetTepByNm prime:
/// the server consults a per-connection working set the native client populates by multiplexing
/// GetTepByNm and DelTep over one connection, which the SDK's per-service WCF channels don't
/// reproduce. Kept for follow-up RE. See
/// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
internal Task<bool> DeleteTagExtendedPropertiesAsync(
string tagName, IReadOnlyList<string> propertyNames, CancellationToken cancellationToken)
{
ArgumentException.ThrowIfNullOrWhiteSpace(tagName);
ArgumentNullException.ThrowIfNull(propertyNames);
if (propertyNames.Count == 0)
{
throw new ArgumentException("At least one property name is required.", nameof(propertyNames));
}
return Task.Run(() => DeleteTagExtendedProperties(tagName, propertyNames), cancellationToken);
}
private bool EnsureTag(HistorianTagDefinition definition)
{
Guid contextKey = Guid.NewGuid();
@@ -126,8 +149,101 @@ internal sealed class HistorianWcfTagWriteOrchestrator
return result;
}
/// <summary>Env-gated dump of the clean AddTEx inBuff (base64) for golden-fixture capture,
/// mirroring the rename/SQL dump hooks — avoids hand-stitching MDAS chunk markers.</summary>
private bool DeleteTagExtendedProperties(string tagName, IReadOnlyList<string> propertyNames)
{
Guid contextKey = Guid.NewGuid();
var (histBinding, histEndpoint, _, _) = HistorianWcfBindingFactory.CreateBindingPair(_options);
Binding auxBinding = HistorianWcfBindingFactory.CreateAuxiliaryBinding(_options);
EndpointAddress statusEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Status);
EndpointAddress transactionEndpoint = HistorianWcfBindingFactory.CreateAuxiliaryEndpointAddress(_options, HistorianWcfServiceNames.Transaction);
EndpointAddress retrievalEndpoint = _options.Transport == HistorianTransport.LocalPipe
? HistorianWcfBindingFactory.CreatePipeEndpointAddress(_options.Host, HistorianWcfServiceNames.Retrieval)
: HistorianWcfBindingFactory.CreateEndpointAddress(_options.Host, _options.Port, HistorianWcfServiceNames.Retrieval);
bool result = false;
HistorianWcfAuthChainHelper.OpenAuthenticatedConnection(
_options, histBinding, histEndpoint, contextKey, CancellationToken.None,
connectionMode: HistorianWcfAuthChainHelper.NativeIntegratedWriteEnabledConnectionMode,
additionalSetup: (historyChannel, context) =>
{
RunWritePriming(historyChannel, context, auxBinding, statusEndpoint, transactionEndpoint, retrievalEndpoint);
string handle = context.StorageSessionId.ToString("D").ToUpperInvariant();
byte[] inBuff = HistorianTagExtendedPropertyProtocol.SerializeDeleteRequest(tagName, propertyNames);
DumpTepIfRequested(inBuff);
// The server's CHistStorage::DeleteTagExtendedProperties resolves each property from
// the storage session's working set — so the tag identity (GetTgByNm) and its
// extended properties (GetTepByNm) must be fetched on THIS session first, exactly as
// the native client does (register → read → delete). The Retrieval prime channel is
// kept OPEN across the DelTep call (the registration is bound to the live channel
// session); without this prime the server throws SErrorException and DelTep returns
// false. See docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
result = PrimeThenDelete(auxBinding, retrievalEndpoint, handle, context.ClientHandle, tagName, () =>
{
bool ok = historyChannel.DeleteTagExtendedProperties(handle, inBuff, out byte[] errorBuffer);
WriteDiag("DelTep", $"Returned={ok} Tag={tagName} PropCount={propertyNames.Count} InLen={inBuff.Length} ErrLen={errorBuffer?.Length ?? -1} ErrHex={(errorBuffer is null ? "<null>" : Convert.ToHexString(errorBuffer))}");
return ok;
});
});
return result;
}
/// <summary>
/// Primes the Retrieval session before a <c>DelTep</c> delete exactly as the native client does:
/// register the tag identity (<c>GetTgByNm</c>) then fetch its extended properties
/// (<c>GetTepByNm</c>) so the server loads both into the storage session's working set, then runs
/// <paramref name="deleteAction"/> (the DelTep call) WHILE the Retrieval channel is still open.
/// Without this two-step prime — or if the channel is closed before DelTep — the server's
/// <c>CHistStorage::DeleteTagExtendedProperties</c> can't resolve the property and throws
/// <c>SErrorException</c> (DelTep returns false). The fetched rows are discarded — the calls are
/// for their server-side registration effect. See
/// docs/reverse-engineering/wcf-add-tag-extended-properties.md §Delete.
/// </summary>
private bool PrimeThenDelete(
Binding retrievalBinding, EndpointAddress retrievalEndpoint, string handle, uint clientHandle,
string tagName, Func<bool> deleteAction)
{
ChannelFactory<IRetrievalServiceContract4> factory = new(retrievalBinding, retrievalEndpoint);
HistorianWcfClientCredentialsHelper.Configure(factory, _options);
IRetrievalServiceContract4 channel = factory.CreateChannel();
try
{
TryRun(() => channel.GetInterfaceVersion(out _));
byte[] tagNames = HistorianTagExtendedPropertyProtocol.SerializeRequest(tagName);
// Register the tag identity on this session (GetTgByNm, uint clientHandle). tagNames uses
// the same count+charCount+UTF-16 framing as GetTepByNm's request buffer.
try
{
uint tgSequence = 0;
uint tgRet = channel.GetTagInfosFromName(clientHandle, (uint)tagNames.Length, tagNames, ref tgSequence, out uint tgInfosSize, out byte[] tgInfos);
WriteDiag("DelTepPrime", $"GetTgByNm ret={tgRet} clientHandle={clientHandle} seq={tgSequence} infosSize={tgInfosSize} infosLen={tgInfos?.Length ?? -1}");
}
catch (Exception ex) { WriteDiag("DelTepPrime", $"GetTgByNm threw: {ex.GetType().Name}: {ex.Message}"); }
uint sequence = 0;
for (int page = 0; page < 64; page++)
{
bool ok;
byte[] responseBuffer;
try { ok = channel.GetTagExtendedPropertiesFromName(handle, tagNames, ref sequence, out responseBuffer, out byte[] errBuf); }
catch (Exception ex) { WriteDiag("DelTepPrime", $"GetTepByNm threw page={page}: {ex.GetType().Name}: {ex.Message}"); break; }
int rows = HistorianTagExtendedPropertyProtocol.ParseResponse(responseBuffer ?? []).Count;
WriteDiag("DelTepPrime", $"GetTepByNm page={page} ok={ok} respLen={responseBuffer?.Length ?? -1} rows={rows}");
if (!ok) break;
if (rows == 0) break;
}
// DelTep runs here, while the Retrieval prime channel is still open.
return deleteAction();
}
finally
{
CloseSafely(channel, factory);
}
}
/// <summary>Env-gated dump of the clean AddTEx / DelTep inBuff (base64) for golden-fixture
/// capture, mirroring the rename/SQL dump hooks — avoids hand-stitching MDAS chunk markers.</summary>
private static void DumpTepIfRequested(byte[] inBuff)
{
string? path = Environment.GetEnvironmentVariable("AVEVA_HISTORIAN_TEP_DUMP");