histsdk/docs/reverse-engineering/capture-workflow.md

Capture Workflow

Use the reverse-engineering CLI to keep captures repeatable:

dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- manifest
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- exports current\aahClient.dll
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- mark history-raw
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-probe 10.100.0.48 32568
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-tag-info 10.100.0.48 32568 OtOpcUaParityTest_001.Counter

To probe the certificate-secured history endpoint with fully managed WCF/MDAS:

dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe localhost 32568
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe 10.100.0.48 32568 localhost

The optional final argument supplies the expected endpoint DNS identity. On the current development Historian, the remote endpoint presents certificate identity localhost, so the explicit identity is required when connecting by IP address.

Windows built-in packet capture may miss local Historian traffic. For local native-wrapper probes, use the Frida harness:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-NativeTraceHarnessWinsockCapture.ps1 -Scenario history -ServerName localhost -RetrievalMode Full -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1

The Frida harness attaches before OpenConnection and hooks:

• Winsock connect, WSAConnect, send, recv, WSASend, and WSARecv
• CreateFileW, ReadFile, WriteFile, and CloseHandle
• NtCreateFile, NtReadFile, and NtWriteFile

For native trace harness captures that must hook before aahClientManaged.dll loads, pass a preload pause:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-NativeTraceHarnessWinsockCapture.ps1 -Scenario event -PreLoadSleepSeconds 8 -AttachDelaySeconds 0 -OutputPath .\docs\reverse-engineering\winsock-event-preload-localhost-latest.ndjson

The latest preload local event pass still produced no Winsock or tracked pipe payloads even though native event open and StartQuery succeeded.

Artifacts should be treated as diagnostic metadata. The script records byte counts and short hex prefixes only; do not add raw credential/session buffers to the repo.

Current local result: localhost, 127.0.0.1, and the machine LAN IP all complete native reads without observed client-process socket or pipe payloads. That suggests local native reads are not exercising the remote transport path.

To force the native client onto a remote TCP path through the Debian test box:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -OutputPath .\docs\reverse-engineering\debian-relay-history-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-latest.json

This starts a temporary Python TCP relay on 10.100.0.35:32568 forwarding to 10.100.0.48:32568, runs the native harness against 10.100.0.35, pulls back the relay log, and cleans up the remote process. The relay logs connection events, byte counts, and 16-byte hex prefixes only.

Current relay result: the native client reaches the remote Net.TCP/WCF preamble and authentication exchange, but the relayed session is rejected before OpenConnection becomes connected. This gives transport evidence but not query request/response buffers yet. Matching ArchestrA logs identify the relayed target as Server(10.100.0.35) and show Transport with Certificate security, so the relay is not transparent at the certificate/identity layer.

For event mode, the rewritten relay shows the same security boundary but a clear endpoint sequence:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -RewriteEndpointHost -Scenario event -OutputPath .\docs\reverse-engineering\debian-relay-rewrite-event-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-rewrite-event-latest.json

Observed event relay sequence:

• /HistCert preamble with application/ssl-tls
• TLS-style records
• repeated /Hist-Integrated preambles with application/negotiate
• NTLMSSP type 1/2/3 messages
• 13-byte server rejection/reset before connected state

Adding --direct-connection for event mode does not bypass the relay; event direct emits the same endpoint sequence and still fails before connected state.

To test whether native connection flags change that security choice, add extra harness arguments:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -OutputPath .\docs\reverse-engineering\debian-relay-history-direct-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-direct-latest.json -HarnessExtraArgs @("--direct-connection")

Observed result: once the reverse harness forces the private directConnection backing field, the native read succeeds and the relay records only its own startup line. This is useful for native parity snapshots, but it bypasses the remote transport evidence needed for the managed driver.

To collect Windows packet metadata for the relay path without storing raw payload bytes:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-PktmonDebianRelayCapture.ps1 -Scenario history -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1 -OutputPrefix .\docs\reverse-engineering\pktmon-debian-relay-history-latest

This script:

• adds a pktmon TCP filter for 10.100.0.35:32568
• starts pktmon with flags 0x00e, intentionally omitting raw packet bytes
• runs the Debian relay harness scenario
• converts the ETL to text/stat metadata
• deletes the ETL file before writing the summary

Current pktmon result: the metadata capture records TCP flows between 10.100.0.48 and 10.100.0.35:32568 with no payload bytes retained. This is useful for timing, directions, ports, and reset behavior, but still not enough to reconstruct query buffers.

To correlate relay TCP ownership on Windows while running Frida system-boundary hooks:

powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-SystemBoundaryViaDebianRelay.ps1 -Scenario history -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1 -OutputPath .\docs\reverse-engineering\system-boundary-via-debian-relay-history-latest.ndjson

Current system-boundary relay result: the Windows TCP owner monitor attributes the established relay connection to AVEVA.Historian.NativeTraceHarness, but Frida hooks on exported Winsock calls, WSAIoctl, mswsock, file APIs, and NtDeviceIoControlFile still record no transport callbacks. Treat this as negative evidence for further export-level Frida work.

For each scenario:

1. Start Wireshark and API Monitor.
2. Emit a mark <scenario-name> line and note the timestamp.
3. Run the same operation through the native SDK/client.
4. Save raw captures outside the repo.
5. Add only sanitized binary frames or decoded notes under fixtures/protocol.

The production SDK must not reference this harness or any AVEVA native binary.

After a capture is sanitized, add parser tests before enabling the corresponding operation in Historian2020ProtocolDialect.