Write2/WriteSecured2 timestamp is interpreted as box-local time: stored source timestamp is shifted by the UTC offset #228

Closed
opened 2026-11-12 20:35:53 -05:00 by dohertj2 · 1 comment
Owner

Found while proving #212 on the DEV gateway (wonder-app-vd03), 2026-11-12.

Write2 writes the value correctly, but the timestamp lands shifted by the box's UTC offset. On vd03 (US Eastern, UTC-5 in November) the stored source timestamp is 5 hours in the future.

Evidence (vd03 gateway 2b13564e, item MESReceiver_025.MoveInBatchID, Int32, supervisory advise, user id 0)

Run Timestamp on the wire (Timestamp, UTC) Source timestamp afterwards (OnDataChange + ReadBulk)
1 2026-11-13T01:33:00.000Z 2026-11-13T06:33:00.000Z (+5 h)
2 (pre-shifted by -5 h as a control) 2026-11-12T20:33:50.000Z (meant 01:33:50Z) 2026-11-13T01:33:50.000Z (the intended instant)

Both values were applied (212212, then 212213), with OnWriteComplete Ok / RespondingAutomationObject. A plain Write restore in the same session got a correct source timestamp (01:33:42.766Z, the current time). So the read path is right, and only the timestamp argument of the timestamped writes is off.

Cause

VariantConverter.ConvertToComValue turns the timestamp into value.TimestampValue.ToDateTime(), a DateTimeKind.Utc DateTime. The COM marshaler writes its clock fields into the VT_DATE and ignores Kind. MXAccess reads the Write2 time argument as local time and converts it to UTC, which adds the offset. The read direction (ToTimestamp) treats a DATE from MXAccess as UTC, and that matches what MXAccess sends. So the two directions disagree.

Fix

Convert to local time (ToDateTime().ToLocalTime()) for the timestamp argument of Write2, WriteSecured2, Write2Bulk and WriteSecured2Bulk (MxAccessCommandExecutor). Do not change value conversion: whether a Time-typed attribute value write has the same local-time rule is unproven. Add a unit test for the conversion. The opt-in live test Write2_SupervisoryAdvise_KeepsSentTimestamp (Client.Tests, see #212) then passes on a non-UTC box. On a UTC box the bug cannot be seen at all, which is probably why it was never caught.

Impact

No consumer calls Write2 today: ScadaBridge and OtOpcUa have no call site (grep, 2026-11-12). This only changes what MXAccess receives when the timestamp parameter is set. Side effect of the proof run: MESReceiver_025.MoveInBatchID briefly carried a 06:33:00Z source timestamp for about 12 s before the restore. If that attribute is historized, the Historian may hold one future-dated point for it.

**Found while proving #212 on the DEV gateway (wonder-app-vd03), 2026-11-12.** `Write2` writes the value correctly, but the timestamp lands **shifted by the box's UTC offset**. On vd03 (US Eastern, UTC-5 in November) the stored source timestamp is **5 hours in the future**. ## Evidence (vd03 gateway `2b13564e`, item `MESReceiver_025.MoveInBatchID`, Int32, supervisory advise, user id 0) | Run | Timestamp on the wire (`Timestamp`, UTC) | Source timestamp afterwards (OnDataChange + ReadBulk) | |---|---|---| | 1 | `2026-11-13T01:33:00.000Z` | `2026-11-13T06:33:00.000Z` (+5 h) | | 2 (pre-shifted by -5 h as a control) | `2026-11-12T20:33:50.000Z` (meant `01:33:50Z`) | `2026-11-13T01:33:50.000Z` (the intended instant) | Both values were applied (`212212`, then `212213`), with OnWriteComplete `Ok` / `RespondingAutomationObject`. A plain `Write` restore in the same session got a correct source timestamp (`01:33:42.766Z`, the current time). So the read path is right, and only the timestamp *argument* of the timestamped writes is off. ## Cause `VariantConverter.ConvertToComValue` turns the timestamp into `value.TimestampValue.ToDateTime()`, a `DateTimeKind.Utc` `DateTime`. The COM marshaler writes its clock fields into the `VT_DATE` and ignores `Kind`. MXAccess reads the `Write2` time argument as **local** time and converts it to UTC, which adds the offset. The read direction (`ToTimestamp`) treats a `DATE` from MXAccess as UTC, and that matches what MXAccess sends. So the two directions disagree. ## Fix Convert to local time (`ToDateTime().ToLocalTime()`) for the **timestamp argument** of `Write2`, `WriteSecured2`, `Write2Bulk` and `WriteSecured2Bulk` (`MxAccessCommandExecutor`). Do **not** change value conversion: whether a `Time`-typed attribute *value* write has the same local-time rule is unproven. Add a unit test for the conversion. The opt-in live test `Write2_SupervisoryAdvise_KeepsSentTimestamp` (Client.Tests, see #212) then passes on a non-UTC box. On a UTC box the bug cannot be seen at all, which is probably why it was never caught. ## Impact No consumer calls `Write2` today: ScadaBridge and OtOpcUa have no call site (grep, 2026-11-12). This only changes what MXAccess receives when the timestamp parameter is set. Side effect of the proof run: `MESReceiver_025.MoveInBatchID` briefly carried a `06:33:00Z` source timestamp for about 12 s before the restore. If that attribute is historized, the Historian may hold one future-dated point for it.
dohertj2 added the area:workerpriority:p2type:bug labels 2026-11-12 20:35:53 -05:00
Author
Owner

Fixed on main in 9ef6fc84. The deploy note is in 76a71bc8. Not deployed. Deployment is tracked in #229 (DEV first, then QA, shipped together with #226).

Fix

  • New method VariantConverter.ConvertToComTimestamp. It converts a timestamp_value instant to wall-clock time in the worker box's zone before the COM call, using TimeZoneInfo.ConvertTimeFromUtc with TimeZoneInfo.Local and its DST rules. The zone can be injected through a constructor, used only by tests.
  • MxAccessCommandExecutor uses it for the time argument only, in Write2, WriteSecured2, Write2Bulk and WriteSecured2Bulk. The bulk session methods now take a separate convertTimestamp.
  • Not changed: value conversion. ConvertToComValue still renders a timestamp value as UTC, because the Time-typed value-write rule is unproven. A time argument of another kind (string or number) passes through unchanged.
  • No double conversion. Gateway, server and clients only forward the proto Timestamp, and clients build it from UTC or DateTimeOffset. This is the only conversion.
  • Read path checked and left alone. MXAccess hands OnDataChange times over as UTC, and ToTimestamp treats them as UTC, which is correct: a plain Write reports the current UTC time. The #226 fix (892bdc1a) is the alarm path and is consistent with this. wnwrap DATE+TIME are UTC, so no offset is applied there.
  • DST: UTC to local is always defined and never produces a skipped local time. The remaining 1 h loss is on the MXAccess side, in the hour that repeats when DST ends. Two instants share one wall-clock time there, and an offset-less VT_DATE cannot tell them apart. This is documented in docs/WorkerConversion.md and gateway.md. Run the box on UTC if exact timestamps through that hour matter.

Tests

  • New: VariantConverterTimestampTests. Cases: UTC-5 (the live case 01:33Z → 20:33 local), US-Eastern EDT, both sides of spring-forward and fall-back, the fall-back ambiguity, fixed UTC+05:30 including a year rollover, a UTC box (identity), and a check that the value path stays UTC. All zones are built in code, so results don't depend on the machine.
  • The four executor forwarding tests (Write2, WriteSecured2 and both bulk forms) now inject a fixed UTC-5 zone and assert the −5 h shift.
  • windev (Eastern Standard Time, UTC-5) results:
    • Worker x86 Release build: 0 warnings, 0 errors.
    • Worker.Tests x86: 783 passed, 0 failed, 12 skipped.
    • Gateway Tests on the Mac: 1446 passed, 0 failed, 6 skipped.
  • Mutation checks on windev:
    • Removing the conversion in the converter fails 15 tests: every non-UTC row, the default-zone test, the ambiguity test and all 4 executor tests. The UTC row correctly still passes.
    • Routing the executor back to ConvertToComValue fails the 4 executor tests.
    • Both restored: green.
  • Live, windev: WorkerLiveMxAccessSmokeTests.GatewaySession_WithLiveWorker_Write2ValueWithTimestampToAdvisedItem. This test now sends a whole-second timestamp 30 s in the past and asserts that the stored timestamp equals it when the completion is Ok. It runs an in-process gateway with its own scratch worker, on TestChildObject.TestInt:
Worker Sent Stored Result
9ef6fc84 2026-11-13T01:46:31Z 2026-11-13T01:46:31Z Passed (OnWriteComplete Ok, value restored)
same, conversion removed (negative control) 2026-11-13T01:47:32Z 2026-11-13T06:47:32Z Failed by exactly +5 h, the vd03 symptom (value restored)

The windev MxAccessGw service was not touched: worker PIDs 14700 and 22108 were the same before and after, and /health/ready returned 200. The scratch folder is C:\work\mxgw228.

Deploy note (DEV first, then QA; full text in docs/deploy/2026-11-12-pending-worker-time-fixes-226-228.md)

  1. On windev, build Server and Worker from one main SHA at or after 9ef6fc84. Build the Worker with -p:Platform=x86 and run Worker.Tests x86. Ship #226 in the same deploy.
  2. DEV vd03 (now 2b13564e; the only other change is #227, additive and opt-in): run Upgrade-MxGatewayService.ps1 -ExpectedSha <sha> -Label issue226-228, then the runbook section 7 checks. Then, from the Mac:
    • Run LiveGatewayWrite2Tests with MXGATEWAY_LIVE_GATEWAY_ASSERT_WRITE2_TIMESTAMP=1 on MESReceiver_025.MoveInBatchID, using a scoped key that is revoked afterwards. KeepsSentTimestamp must pass; it failed on 2b13564e (sent 01:36:51Z, stored 06:36:51Z).
    • #226 check: CVDAisle_1.TableAlarms_008.RuntimeAlert reads 2026-07-28T20:02:46.830Z.
  3. QA vq03/vq04 (now 4f931e4): a main build carries the operator-authenticated alarm ack, so the #170 prerequisites must be met first. Deploy rolling, vq03 then vq04, and run the same checks with a QA Z28062 test attribute.
  4. Rollback: runbook section 8. Rolling back to the .bak pair has no store or schema impact.

Closing on merge, following the #226 convention. The deploy and its live check are tracked in #229. Reopen this issue if the post-deploy KeepsSentTimestamp check fails.

Fixed on main in `9ef6fc84`. The deploy note is in `76a71bc8`. **Not deployed.** Deployment is tracked in **#229** (DEV first, then QA, shipped together with #226). **Fix** - New method `VariantConverter.ConvertToComTimestamp`. It converts a `timestamp_value` instant to wall-clock time in the worker box's zone before the COM call, using `TimeZoneInfo.ConvertTimeFromUtc` with `TimeZoneInfo.Local` and its DST rules. The zone can be injected through a constructor, used only by tests. - `MxAccessCommandExecutor` uses it for the **time argument only**, in `Write2`, `WriteSecured2`, `Write2Bulk` and `WriteSecured2Bulk`. The bulk session methods now take a separate `convertTimestamp`. - Not changed: value conversion. `ConvertToComValue` still renders a timestamp *value* as UTC, because the `Time`-typed value-write rule is unproven. A time argument of another kind (string or number) passes through unchanged. - No double conversion. Gateway, server and clients only forward the proto `Timestamp`, and clients build it from UTC or `DateTimeOffset`. This is the only conversion. - Read path checked and left alone. MXAccess hands OnDataChange times over as UTC, and `ToTimestamp` treats them as UTC, which is correct: a plain `Write` reports the current UTC time. The #226 fix (`892bdc1a`) is the alarm path and is consistent with this. wnwrap DATE+TIME are UTC, so no offset is applied there. - DST: UTC to local is always defined and never produces a skipped local time. The remaining 1 h loss is on the MXAccess side, in the hour that repeats when DST ends. Two instants share one wall-clock time there, and an offset-less `VT_DATE` cannot tell them apart. This is documented in `docs/WorkerConversion.md` and `gateway.md`. Run the box on UTC if exact timestamps through that hour matter. **Tests** - New: `VariantConverterTimestampTests`. Cases: UTC-5 (the live case 01:33Z → 20:33 local), US-Eastern EDT, both sides of spring-forward and fall-back, the fall-back ambiguity, fixed UTC+05:30 including a year rollover, a UTC box (identity), and a check that the value path stays UTC. All zones are built in code, so results don't depend on the machine. - The four executor forwarding tests (`Write2`, `WriteSecured2` and both bulk forms) now inject a fixed UTC-5 zone and assert the −5 h shift. - windev (`Eastern Standard Time`, UTC-5) results: - Worker x86 Release build: 0 warnings, 0 errors. - `Worker.Tests` x86: **783 passed, 0 failed, 12 skipped**. - Gateway `Tests` on the Mac: **1446 passed, 0 failed, 6 skipped**. - Mutation checks on windev: - Removing the conversion in the converter fails **15** tests: every non-UTC row, the default-zone test, the ambiguity test and all 4 executor tests. The UTC row correctly still passes. - Routing the executor back to `ConvertToComValue` fails the **4** executor tests. - Both restored: green. - **Live, windev:** `WorkerLiveMxAccessSmokeTests.GatewaySession_WithLiveWorker_Write2ValueWithTimestampToAdvisedItem`. This test now sends a whole-second timestamp 30 s in the past and asserts that the stored timestamp equals it when the completion is `Ok`. It runs an in-process gateway with its own scratch worker, on `TestChildObject.TestInt`: | Worker | Sent | Stored | Result | |---|---|---|---| | `9ef6fc84` | `2026-11-13T01:46:31Z` | `2026-11-13T01:46:31Z` | **Passed** (`OnWriteComplete` Ok, value restored) | | same, conversion removed (negative control) | `2026-11-13T01:47:32Z` | `2026-11-13T06:47:32Z` | **Failed by exactly +5 h**, the vd03 symptom (value restored) | The windev `MxAccessGw` service was not touched: worker PIDs 14700 and 22108 were the same before and after, and `/health/ready` returned 200. The scratch folder is `C:\work\mxgw228`. **Deploy note (DEV first, then QA; full text in `docs/deploy/2026-11-12-pending-worker-time-fixes-226-228.md`)** 1. On windev, build Server and Worker from one `main` SHA at or after `9ef6fc84`. Build the Worker with `-p:Platform=x86` and run `Worker.Tests` x86. Ship #226 in the same deploy. 2. **DEV vd03** (now `2b13564e`; the only other change is #227, additive and opt-in): run `Upgrade-MxGatewayService.ps1 -ExpectedSha <sha> -Label issue226-228`, then the runbook section 7 checks. Then, from the Mac: - Run `LiveGatewayWrite2Tests` with `MXGATEWAY_LIVE_GATEWAY_ASSERT_WRITE2_TIMESTAMP=1` on `MESReceiver_025.MoveInBatchID`, using a scoped key that is revoked afterwards. `KeepsSentTimestamp` must pass; it failed on `2b13564e` (sent 01:36:51Z, stored 06:36:51Z). - #226 check: `CVDAisle_1.TableAlarms_008.RuntimeAlert` reads `2026-07-28T20:02:46.830Z`. 3. **QA vq03/vq04** (now `4f931e4`): a `main` build carries the operator-authenticated alarm ack, so the **#170 prerequisites** must be met first. Deploy rolling, vq03 then vq04, and run the same checks with a QA Z28062 test attribute. 4. Rollback: runbook section 8. Rolling back to the `.bak` pair has no store or schema impact. Closing on merge, following the #226 convention. The deploy and its live check are tracked in #229. Reopen this issue if the post-deploy `KeepsSentTimestamp` check fails.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/mxaccessgw#228