IDriver.GetMemoryFootprint / FlushOptionalCachesAsync have no consumer after #522 #525

Closed
opened 2026-07-30 04:56:41 -04:00 by dohertj2 · 2 comments
Owner

Split out of #522, which deleted the driver-tier recycle machinery. Kept separate on purpose: removing these touches all 12 drivers and every test stub, which would have buried the reviewable part of that change in mechanical churn.

What is true today

  • IDriver.GetMemoryFootprint() and IDriver.FlushOptionalCachesAsync() have no production call site. Their only consumer was MemoryTracking, deleted in #522.
  • Outside the driver implementations themselves, the only references are the interface declarations and the stub IDriver inside DriverHostActor (=> 0 / Task.CompletedTask).
  • GetMemoryFootprint's doc-comment previously claimed "Polled every 30s by Core", which was not true in v3 even before #522 — nothing polled it. Both members now carry a doc-comment stating they are unconsumed.
  • All 12 drivers still implement them; several compute a real number (symbol-table and cache sizes), which is now wasted work on some paths, though cheap.

The decision

(a) Delete both from IDriver. Removes ~12 implementations plus test stubs. Purely mechanical, but wide — best done alone, not alongside a behavioural change.

(b) Keep and build a consumer. If per-driver memory pressure is wanted again, build it for the architecture as it now is: a process-wide watchdog that can ask individual drivers to flush caches — not a per-driver tier whose only remedy needed an out-of-process supervisor that cannot exist in-process (see #522 for why that model is gone). FlushOptionalCachesAsync is the genuinely useful half here — it degrades gracefully, where recycle did not — and the drivers' existing implementations would be reusable as-is.

Unlike #522, there is a real argument for (b): a long-running server with symbol-cache-heavy drivers (TwinCAT, AB CIP) does benefit from a flush-under-pressure path, and the driver-side work is already written. What is missing is only the trigger.

Split out of #522, which deleted the driver-tier recycle machinery. Kept separate on purpose: removing these touches **all 12 drivers and every test stub**, which would have buried the reviewable part of that change in mechanical churn. ## What is true today - `IDriver.GetMemoryFootprint()` and `IDriver.FlushOptionalCachesAsync()` have **no production call site**. Their only consumer was `MemoryTracking`, deleted in #522. - Outside the driver implementations themselves, the only references are the interface declarations and the stub `IDriver` inside `DriverHostActor` (`=> 0` / `Task.CompletedTask`). - `GetMemoryFootprint`'s doc-comment previously claimed *"Polled every 30s by Core"*, which was **not true in v3** even before #522 — nothing polled it. Both members now carry a doc-comment stating they are unconsumed. - All 12 drivers still implement them; several compute a real number (symbol-table and cache sizes), which is now wasted work on some paths, though cheap. ## The decision **(a) Delete both from `IDriver`.** Removes ~12 implementations plus test stubs. Purely mechanical, but wide — best done alone, not alongside a behavioural change. **(b) Keep and build a consumer.** If per-driver memory pressure is wanted again, build it for the architecture as it now is: a **process-wide watchdog** that can ask individual drivers to flush caches — *not* a per-driver tier whose only remedy needed an out-of-process supervisor that cannot exist in-process (see #522 for why that model is gone). `FlushOptionalCachesAsync` is the genuinely useful half here — it degrades gracefully, where recycle did not — and the drivers' existing implementations would be reusable as-is. Unlike #522, there is a real argument for (b): a long-running server with symbol-cache-heavy drivers (TwinCAT, AB CIP) does benefit from a flush-under-pressure path, and the driver-side work is already written. What is missing is only the trigger.
Author
Owner

⚠️ Correcting this issue's own body, and withdrawing a recommendation made against it.

When filing this I wrote that FlushOptionalCachesAsync is "the genuinely useful half" and that "the drivers' existing implementations would be reusable as-is". I then argued the opposite when recommending deletion, on the stated evidence that "1 of 12 actually frees memory". That number was wrong. It came from grepping for two specific cache-clear calls and treating their absence elsewhere as proof of absence — not from reading the twelve implementations.

The actual inventory

FlushOptionalCachesAsync (12 implementations):

Behaviour Count Drivers
No-op (=> Task.CompletedTask) 8 Modbus, FOCAS, TwinCAT, AbLegacy, S7, Sql, Galaxy, Mqtt
Frees real memory 3 AbCip (UDT template cache), Calculation (compiled scripts), MTConnect (cached probe model)
Cosmetic 1 OpcUaClient — resets a counter, frees nothing

GetMemoryFootprint (12 implementations): 7 return 0; 5 compute an estimate (TwinCAT, OpcUaClient, Galaxy, MTConnect, Mqtt) from invented per-entry constants that have never been validated against real memory, because nothing ever consumed them.

MTConnect is the one that undermines the delete case: its flush is designed, not incidental — compare-and-swap discipline referenced across ~8 doc-comments, an invariant that a flush can never leave browse permanently disabled, and tests asserting the footprint genuinely decreases afterwards. Deleting it deletes working, tested code.

What survives of the argument

Narrower, but still real: the two halves are coherent in exactly one driver. AbCip and Calculation free real memory while reporting footprint 0; TwinCAT, Galaxy and Mqtt report footprints behind no-op flushes. Only MTConnect has a footprint that means something and a flush that relieves it — which is what a pressure-triggered consumer would actually need.

Status

The deletion sweep had already edited 12 drivers, IDriver and ~15 test files when the recount landed. It was reverted rather than completed, because the justification no longer held. Nothing from #525 is on master.

The three options now, with the corrected evidence:

  1. Delete both — still defensible on "nothing calls either, coherent once", but knowingly deletes MTConnect's tested flush path.
  2. Delete GetMemoryFootprint only, keep FlushOptionalCachesAsync — removes the half that is genuinely useless (7/12 zeros, unvalidated constants) and keeps the half three drivers implement meaningfully. Leaves one caller-less member.
  3. Close as won't-fix, documenting both as a deliberately-kept extension point.

Unresolved pending a decision. The generalisable lesson: a grep for the shapes you expect is not an inventory — enumerate the set and read each member, especially when the count is what your recommendation rests on.

⚠️ **Correcting this issue's own body, and withdrawing a recommendation made against it.** When filing this I wrote that `FlushOptionalCachesAsync` is *"the genuinely useful half"* and that *"the drivers' existing implementations would be reusable as-is"*. I then argued the opposite when recommending deletion, on the stated evidence that **"1 of 12 actually frees memory"**. That number was wrong. It came from grepping for **two specific cache-clear calls** and treating their absence elsewhere as proof of absence — not from reading the twelve implementations. ## The actual inventory **`FlushOptionalCachesAsync` (12 implementations):** | Behaviour | Count | Drivers | |---|---|---| | No-op (`=> Task.CompletedTask`) | **8** | Modbus, FOCAS, TwinCAT, AbLegacy, S7, Sql, Galaxy, Mqtt | | Frees real memory | **3** | AbCip (UDT template cache), Calculation (compiled scripts), **MTConnect (cached probe model)** | | Cosmetic | 1 | OpcUaClient — resets a counter, frees nothing | **`GetMemoryFootprint` (12 implementations):** 7 return `0`; 5 compute an estimate (TwinCAT, OpcUaClient, Galaxy, MTConnect, Mqtt) from invented per-entry constants that have **never been validated against real memory**, because nothing ever consumed them. MTConnect is the one that undermines the delete case: its flush is *designed*, not incidental — compare-and-swap discipline referenced across ~8 doc-comments, an invariant that a flush can never leave browse permanently disabled, and tests asserting the footprint genuinely decreases afterwards. Deleting it deletes working, tested code. ## What survives of the argument Narrower, but still real: **the two halves are coherent in exactly one driver.** AbCip and Calculation free real memory while reporting footprint `0`; TwinCAT, Galaxy and Mqtt report footprints behind no-op flushes. Only MTConnect has a footprint that means something *and* a flush that relieves it — which is what a pressure-triggered consumer would actually need. ## Status The deletion sweep had already edited 12 drivers, `IDriver` and ~15 test files when the recount landed. It was **reverted rather than completed**, because the justification no longer held. Nothing from #525 is on master. The three options now, with the corrected evidence: 1. **Delete both** — still defensible on "nothing calls either, coherent once", but knowingly deletes MTConnect's tested flush path. 2. **Delete `GetMemoryFootprint` only, keep `FlushOptionalCachesAsync`** — removes the half that is genuinely useless (7/12 zeros, unvalidated constants) and keeps the half three drivers implement meaningfully. Leaves one caller-less member. 3. **Close as won't-fix**, documenting both as a deliberately-kept extension point. Unresolved pending a decision. **The generalisable lesson: a grep for the shapes you expect is not an inventory** — enumerate the set and read each member, especially when the count is what your recommendation rests on.
Author
Owner

Done — option (a) delete, owner decision. master b8154469 (merged 9fa701b1).

Removed from IDriver, all 12 driver implementations, and every test stub — plus the support code that existed only to feed them, which is the part a member-only deletion would have left behind as fresh dead weight:

  • MTConnect's three per-entry byte-weight constants, its cached ProbeModelDataItemCount, and CountDataItems — a full device-tree walk run on every cache fill to populate a number with no remaining reader. That is the "wasted work on some paths" this issue anticipated.
  • OpcUaClient's _discoveredNodeCount field and the discovery-pass write to it.
  • MQTT's ApproxBytesPerAuthoredTag.

Two notes, neither a change in what the software does.

AbCipTemplateCache now has no production reset path — its only caller was AB CIP's flush, which itself had no caller, so nothing changes except how visible the fact is. Recorded on the type, along with a correction: that class's comment claimed a device ReinitializeAsync also flushed it, and no such call ever existed. A UDT redefined on a live controller keeps its stale shape until the driver instance is rebuilt.

MTConnect's /probe cache is still dropped, by InvalidateProbeModel() on an Agent restart — now internal rather than private so the tests that used the flush purely to empty the cache drive the real path instead. Those tests now cover a case that actually occurs in production, which they did not before. Several unrelated doc comments cited FlushOptionalCachesAsync as the canonical compare-and-swap example and are repointed.

Tests deleted: 3 OpcUaClient, 3 Galaxy, 2 MQTT, 1 MTConnect, 1 AbCip. One is worth recording as a lesson rather than a line item — FlushOptionalCachesAsync_resets_footprint_counter asserted a footprint of 0 before the flush and 0 after, with no discovery in between, so it would have passed against a flush that did nothing at all.

Full suite green except 7 failures verified identical on master before and after, by name rather than by assumption (4 AbLegacy integration with the compose profile down, 3 Host.IntegrationTests).

Docs corrected where they described this as live: DriverLifecycle.md (which carried this issue's placeholder), drivers/AbCip.md, drivers/AbLegacy.md, and v2/driver-stability.md — whose "Core polls this every 30 s" and per-driver DriverConfig memory budget were never built as written, and are now banner-ed as design intent rather than description. That doc is also where the argument for option (b) is preserved: if a flush-under-pressure watchdog is wanted later, the deleted per-driver implementations are in git history and were mostly reusable. What was always missing is the trigger.

Done — option **(a) delete**, owner decision. master `b8154469` (merged `9fa701b1`). Removed from `IDriver`, all 12 driver implementations, and every test stub — plus **the support code that existed only to feed them**, which is the part a member-only deletion would have left behind as fresh dead weight: - MTConnect's three per-entry byte-weight constants, its cached `ProbeModelDataItemCount`, and `CountDataItems` — a full device-tree walk run on every cache fill to populate a number with no remaining reader. That is the "wasted work on some paths" this issue anticipated. - OpcUaClient's `_discoveredNodeCount` field and the discovery-pass write to it. - MQTT's `ApproxBytesPerAuthoredTag`. **Two notes, neither a change in what the software does.** `AbCipTemplateCache` now has no production reset path — its only caller was AB CIP's flush, which itself had no caller, so nothing changes except how visible the fact is. Recorded on the type, along with a correction: that class's comment claimed a device `ReinitializeAsync` also flushed it, and no such call ever existed. A UDT redefined on a live controller keeps its stale shape until the driver instance is rebuilt. MTConnect's `/probe` cache is still dropped, by `InvalidateProbeModel()` on an Agent restart — now `internal` rather than `private` so the tests that used the flush purely to empty the cache drive the real path instead. Those tests now cover a case that actually occurs in production, which they did not before. Several unrelated doc comments cited `FlushOptionalCachesAsync` as the canonical compare-and-swap example and are repointed. **Tests deleted:** 3 OpcUaClient, 3 Galaxy, 2 MQTT, 1 MTConnect, 1 AbCip. One is worth recording as a lesson rather than a line item — `FlushOptionalCachesAsync_resets_footprint_counter` asserted a footprint of 0 before the flush and 0 after, with no discovery in between, so it would have passed against a flush that did nothing at all. Full suite green except 7 failures verified identical on master **before and after**, by name rather than by assumption (4 AbLegacy integration with the compose profile down, 3 Host.IntegrationTests). Docs corrected where they described this as live: `DriverLifecycle.md` (which carried this issue's placeholder), `drivers/AbCip.md`, `drivers/AbLegacy.md`, and `v2/driver-stability.md` — whose "Core polls this every 30 s" and per-driver `DriverConfig` memory budget were never built as written, and are now banner-ed as design intent rather than description. That doc is also where the argument for option (b) is preserved: if a flush-under-pressure watchdog is wanted later, the deleted per-driver implementations are in git history and were mostly reusable. What was always missing is the trigger.
Sign in to join this conversation.
1 Participants
Notifications
Due Date
No due date set.
Dependencies

No dependencies set.

Reference: dohertj2/lmxopcua#525