histsdk/docs/reverse-engineering/capture-workflow.md

# Capture Workflow

Use the reverse-engineering CLI to keep captures repeatable:

```powershell
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- manifest
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- exports current\aahClient.dll
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- mark history-raw
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-probe 10.100.0.48 32568
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-tag-info 10.100.0.48 32568 OtOpcUaParityTest_001.Counter
```

To probe the certificate-secured history endpoint with fully managed WCF/MDAS:

```powershell
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe localhost 32568
dotnet run --project tools\AVEVA.Historian.ReverseEngineering -- wcf-cert-probe 10.100.0.48 32568 localhost
```

The optional final argument supplies the expected endpoint DNS identity. On the
current development Historian, the remote endpoint presents certificate identity
`localhost`, so the explicit identity is required when connecting by IP address.

Windows built-in packet capture may miss local Historian traffic. For local
native-wrapper probes, use the Frida harness:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-NativeTraceHarnessWinsockCapture.ps1 -Scenario history -ServerName localhost -RetrievalMode Full -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1
```

The Frida harness attaches before `OpenConnection` and hooks:

- Winsock `connect`, `WSAConnect`, `send`, `recv`, `WSASend`, and `WSARecv`
- `CreateFileW`, `ReadFile`, `WriteFile`, and `CloseHandle`
- `NtCreateFile`, `NtReadFile`, and `NtWriteFile`

For native trace harness captures that must hook before `aahClientManaged.dll`
loads, pass a preload pause:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-NativeTraceHarnessWinsockCapture.ps1 -Scenario event -PreLoadSleepSeconds 8 -AttachDelaySeconds 0 -OutputPath .\docs\reverse-engineering\winsock-event-preload-localhost-latest.ndjson
```

The latest preload local event pass still produced no Winsock or tracked pipe
payloads even though native event open and `StartQuery` succeeded.

Artifacts should be treated as diagnostic metadata. The script records byte
counts and short hex prefixes only; do not add raw credential/session buffers
to the repo.

Current local result: `localhost`, `127.0.0.1`, and the machine LAN IP all
complete native reads without observed client-process socket or pipe payloads.
That suggests local native reads are not exercising the remote transport path.

To force the native client onto a remote TCP path through the Debian test box:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -OutputPath .\docs\reverse-engineering\debian-relay-history-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-latest.json
```

This starts a temporary Python TCP relay on `10.100.0.35:32568` forwarding to
`10.100.0.48:32568`, runs the native harness against `10.100.0.35`, pulls back
the relay log, and cleans up the remote process. The relay logs connection
events, byte counts, and 16-byte hex prefixes only.

Current relay result: the native client reaches the remote Net.TCP/WCF preamble
and authentication exchange, but the relayed session is rejected before
`OpenConnection` becomes connected. This gives transport evidence but not query
request/response buffers yet. Matching ArchestrA logs identify the relayed
target as `Server(10.100.0.35)` and show `Transport with Certificate` security,
so the relay is not transparent at the certificate/identity layer.

For event mode, the rewritten relay shows the same security boundary but a
clear endpoint sequence:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -RewriteEndpointHost -Scenario event -OutputPath .\docs\reverse-engineering\debian-relay-rewrite-event-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-rewrite-event-latest.json
```

Observed event relay sequence:

- `/HistCert` preamble with `application/ssl-tls`
- TLS-style records
- repeated `/Hist-Integrated` preambles with `application/negotiate`
- NTLMSSP type 1/2/3 messages
- 13-byte server rejection/reset before connected state

Adding `--direct-connection` for event mode does not bypass the relay; event
direct emits the same endpoint sequence and still fails before connected state.

To test whether native connection flags change that security choice, add extra
harness arguments:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-DebianHistorianRelayCapture.ps1 -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -OutputPath .\docs\reverse-engineering\debian-relay-history-direct-latest.ndjson -HarnessOutputPath .\docs\reverse-engineering\native-trace-harness-via-debian-relay-direct-latest.json -HarnessExtraArgs @("--direct-connection")
```

Observed result: once the reverse harness forces the private
`directConnection` backing field, the native read succeeds and the relay records
only its own startup line. This is useful for native parity snapshots, but it
bypasses the remote transport evidence needed for the managed driver.

To collect Windows packet metadata for the relay path without storing raw
payload bytes:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Run-PktmonDebianRelayCapture.ps1 -Scenario history -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1 -OutputPrefix .\docs\reverse-engineering\pktmon-debian-relay-history-latest
```

This script:

- adds a pktmon TCP filter for `10.100.0.35:32568`
- starts pktmon with flags `0x00e`, intentionally omitting raw packet bytes
- runs the Debian relay harness scenario
- converts the ETL to text/stat metadata
- deletes the ETL file before writing the summary

Current pktmon result: the metadata capture records TCP flows between
`10.100.0.48` and `10.100.0.35:32568` with no payload bytes retained. This is
useful for timing, directions, ports, and reset behavior, but still not enough
to reconstruct query buffers.

To correlate relay TCP ownership on Windows while running Frida system-boundary
hooks:

```powershell
powershell.exe -NoProfile -ExecutionPolicy Bypass -File .\scripts\Attach-SystemBoundaryViaDebianRelay.ps1 -Scenario history -SshUser dohertj2 -SshHost 10.100.0.35 -TargetHost 10.100.0.48 -TagName OtOpcUaParityTest_001.Counter -LookbackMinutes 1440 -MaxRows 1 -OutputPath .\docs\reverse-engineering\system-boundary-via-debian-relay-history-latest.ndjson
```

Current system-boundary relay result: the Windows TCP owner monitor attributes
the established relay connection to `AVEVA.Historian.NativeTraceHarness`, but
Frida hooks on exported Winsock calls, `WSAIoctl`, `mswsock`, file APIs, and
`NtDeviceIoControlFile` still record no transport callbacks. Treat this as
negative evidence for further export-level Frida work.

For each scenario:

1. Start Wireshark and API Monitor.
2. Emit a `mark <scenario-name>` line and note the timestamp.
3. Run the same operation through the native SDK/client.
4. Save raw captures outside the repo.
5. Add only sanitized binary frames or decoded notes under `fixtures/protocol`.

The production SDK must not reference this harness or any AVEVA native binary.

After a capture is sanitized, add parser tests before enabling the corresponding
operation in `Historian2020ProtocolDialect`.