Executes the sizing analysis committed in db35cff. The VM has HBA PCI
passthrough, so memoryReservationLockedToMax is true and all of its RAM
is reserved on the host whether ARC uses it or not - this moved the ESXi
host from 83.9% to 58.8% memory usage. The reservation followed the
resize automatically.
govc vm.power -s TrueNAS # graceful via Tools, ~10s
govc vm.change -vm TrueNAS -m 65536
govc vm.power -on TrueNAS # ~3 min to SSH (HBA init + pool import)
Tunables now zfs_arc_max 40 GiB and zfs_dirty_data_max 8 GiB, both
persisted and verified live. The second one matters more at 64 GB: left
at 16 GiB it would have been 25% of RAM and stacked with ARC during the
nightly Veeam window.
Client impact was nil. Both docker-host NFS mounts are hard, so plex,
sonarr and radarr stalled in uninterruptible sleep and resumed without
restarting; the only D-state process during the outage was the NFS
kernel thread. Pre-flight checked SMB write handles rather than sessions
- the two live sessions were idle Explorer directory handles with
WRITE_DATA false, safe to drop. Post-boot: pool imported, all three NFS
exports restored including the Veeam repo, SMB and NFS running, Plex
serving all three libraries.
Two things resolved themselves: mypool's latched "unrecoverable error"
banner cleared on reimport and it now reports healthy true, so the
zpool clear advice is dropped.
Records the procedure and the pre-flight checks, since the pinned-tunable
trap applies to any future resize of this VM.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Assessed whether the TrueNAS VM can drop from 128 GB to 64 GB of RAM.
It can, but zfs_arc_max is pinned by an explicit tunable (id 8, 96 GiB)
rather than derived from RAM, so it does not follow the VM down. Left
alone on a 64 GiB VM, ZFS would target an ARC larger than the whole
machine - ARC grows until the kernel shrinker forces it back, giving
continuous reclaim pressure and real OOM risk for smbd/nfsd/middlewared
under a write burst. zfs_dirty_data_max (id 15) is pinned at 16 GiB and
has the same problem at 25% of a 64 GiB box.
The sizing evidence is the ARC ghost lists, which count blocks evicted
and then re-requested - the direct measure of whether a larger cache
would help. mru_ghost_hits was 4 and mfu_ghost_hits 0 against 70M hits,
so nothing being evicted is being re-read. The working set is ~22 GiB
(MFU 20.7 GiB serving 94% of hits, plus 1.3 GiB metadata); the other
~75 GiB of ARC is MRU holding streaming media and Veeam backup data read
once. No dedup, no L2ARC, no apps or VMs, 1.78 GiB total process RSS.
Also corrects two stale facts found while looking: mypool is ONLINE with
no known data errors at 96.4T/55%, not DEGRADED at 83.5T - the scary
zpool status banner is a latched historical flag clearable with
zpool clear - and /etc/version reports 25.10.16.12.33, not 25.04.2.6.
SSD_Pool and boot-pool rows were not re-verified and are marked as such.
Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
dohertj2