From 5a4da57457b4cc0217069db056c8a9f0cec90d81 Mon Sep 17 00:00:00 2001 From: dohertj2 Date: Thu, 17 Sep 2026 12:04:45 -0400 Subject: [PATCH] Decommission the old Plex VM Plex has run as a container on the Docker host since the 2026-09-16 migration. The old Debian 11 VM (10.100.0.24) is now retired per plex-docker-migration.md section 9, steps 1, 2 and 4-7: - removed from the Veeam "Backup VMs" job - guest shut down gracefully, pre-migration snapshot removed and consolidated back to Plex.vmdk - Guacamole "Plex SSH" connection deleted - .24 / 10.50.0.31 DHCP leases released on the UDM The VMDKs are deliberately kept until ~2026-10-15 so the VM remains a one-command rollback; step 3 stays open until then. Before powering the VM off, every integration was re-verified as pointing at 10.100.0.35: Sonarr, Radarr, Tautulli, seerr, Uptime Kuma, Homepage, Traefik and the UDM PLEX port-forward. Section 9 gains a decommission log recording two behaviours that cost time to work out: Veeam turns a removed job include into an explicit Exclude tombstone rather than deleting it, and a plain sqlite3 integrity_check on a Plex library DB fails with "unknown tokenizer: collating" because the stock binary lacks Plex's FTS collation - that is not corruption. ESXi port-group counts corrected (LAN_100 7->6, DATA_50 4->3), verified by counting NICs on powered-on VMs. Co-Authored-By: Claude Opus 5 (1M context) --- CLAUDE.md | 4 ++-- esxi.md | 20 ++++++++++---------- plex-docker-migration.md | 34 +++++++++++++++++++++++++++++++++- plex.md | 4 +++- veeam.md | 2 +- 5 files changed, 49 insertions(+), 15 deletions(-) diff --git a/CLAUDE.md b/CLAUDE.md index d1daed4..f3dc47a 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -19,9 +19,9 @@ Credentials live inline in each component's `.md` file under its **Access** sect ## Components - [TrueNAS Server](truenas.md) — NAS at 10.100.0.25 (management) / 10.50.0.x (storage), serves SMB and NFS -- [ESXi Host](esxi.md) — HP server at 10.2.0.12, ESXi 8.0.3, 8 VMs (7 powered on), access via `govc` only (not SSH). **Read esxi.md for credentials and connection details.** +- [ESXi Host](esxi.md) — HP server at 10.2.0.12, ESXi 8.0.3, 8 VMs (6 powered on), access via `govc` only (not SSH). **Read esxi.md for credentials and connection details.** - [Docker Server](docker.md) — Debian 13 at 10.100.0.35, Docker host for 21 containers (arr stack, Traefik, Semaphore/Ansible). **This is the local machine.** -- [Plex Server](plex.md) — **container on the Docker host (10.100.0.35)** since 2026-09-16, Plex Media Server 1.43, NFS media from TrueNAS. Old Debian 11 VM at 10.100.0.24 retained powered-on as rollback. +- [Plex Server](plex.md) — **container on the Docker host (10.100.0.35)** since 2026-09-16, Plex Media Server 1.43, NFS media from TrueNAS. Old Debian 11 VM decommissioned 2026-09-17 (powered off, disks retained until ~2026-10-15). - [Home Assistant](ha.md) — HAOS at 10.100.0.40, HA 2026.2.2, 2,411 entities, smart home automation. API access only (no SSH). - [Ignition Gateway](ignition.md) — Debian 11 at 10.100.0.90, Ignition 8.3.3 (Maker), SCADA/IIoT platform with PostgreSQL 13 - [Veeam Server](veeam.md) — Windows Server 2025 at 10.100.0.30, Veeam B&R 12.3, backup server. **Read veeam.md for credentials and connection details.** diff --git a/esxi.md b/esxi.md index 57e7ffc..4ab554b 100644 --- a/esxi.md +++ b/esxi.md @@ -36,7 +36,7 @@ Carries VLAN-tagged traffic for management, LAN, IoT, and guest networks. | DEFAULT_0 | 0 (untagged) | 1 | 10.2.0.0/24 (ESXi mgmt) | | MGMT_1 | 0 (untagged) | 0 | — | | GUEST_3 | 3 | 0 | — | -| LAN_100 | 100 | 7 | 10.100.0.0/24 | +| LAN_100 | 100 | 6 | 10.100.0.0/24 | | IOT_200 | 200 | 1 | 10.200.0.0/24 | ### jumboFrameSwitch (vmnic5 — 25Gbps Mellanox, MTU 9000) @@ -45,7 +45,7 @@ Dedicated storage network with jumbo frames. Switch port 32 is configured as a t | Port Group | VLAN ID | Active Ports | Subnet | |------------|---------|--------------|--------| -| DATA_50 | 50 | 4 | 10.50.0.0/24 | +| DATA_50 | 50 | 3 | 10.50.0.0/24 | All port groups: promiscuous mode=No, forged transmits=No, MAC changes=No. @@ -68,15 +68,15 @@ All port groups: promiscuous mode=No, forged transmits=No, MAC changes=No. | DOCKER | Debian 12 | 22 | 32GB | 10.100.0.35 | 10.50.0.35 | Docker bridges: 172.18.0.1, 172.19.0.1 | | Veeam | Win Server 2025 | 8 | 8GB | 10.100.0.30 | 10.50.0.32 | — | | HA | Oracle Linux 6 | 16 | 16GB | 10.100.0.40 | — | Docker bridge: 172.30.32.1 | -| Plex | Debian 11 | 8 | 8GB | 10.100.0.24 | 10.50.0.31 | — | | Ignition | Debian 11 | 4 | 16GB | 10.100.0.90 | — | IOT_200 NIC connected but no IP | | DevVM | Win 10 | 8 | 16GB | 10.100.0.47 | — | IOT_200: 10.200.0.47 | ### Powered Off -| VM | Guest OS | CPU | RAM | Port Groups | -|----|----------|-----|-----|-------------| -| DevServer | Win Server 2022 | 12 | 32GB | LAN_100 | +| VM | Guest OS | CPU | RAM | Port Groups | Notes | +|----|----------|-----|-----|-------------|-------| +| DevServer | Win Server 2022 | 12 | 32GB | LAN_100 | — | +| Plex | Debian 11 | 8 | 8GB | LAN_100, DATA_50 | **Decommissioned 2026-09-17** — migrated to a container on DOCKER. Disks retained for rollback until ~2026-10-15, then delete. See [plex-docker-migration.md](plex-docker-migration.md) §9. | ## Per-VM Network Detail @@ -103,11 +103,11 @@ All port groups: promiscuous mode=No, forged transmits=No, MAC changes=No. |-----|-----|------------|-----| | Adapter 2 | 00:0c:29:f3:f3:5b | LAN_100 | 10.100.0.40 | -### Plex -| NIC | MAC | Port Group | IPs | +### Plex (decommissioned 2026-09-17, powered off) +| NIC | MAC | Port Group | Former IP | |-----|-----|------------|-----| -| Adapter 1 | 00:0c:29:4a:ef:3d | LAN_100 | 10.100.0.24 | -| Adapter 2 | 00:0c:29:4a:ef:47 | DATA_50 | 10.50.0.31 | +| Adapter 1 | 00:0c:29:4a:ef:3d | LAN_100 | 10.100.0.24 (lease released) | +| Adapter 2 | 00:0c:29:4a:ef:47 | DATA_50 | 10.50.0.31 (lease released) | ### Ignition | NIC | MAC | Port Group | IPs | diff --git a/plex-docker-migration.md b/plex-docker-migration.md index f96bd8e..26579ab 100644 --- a/plex-docker-migration.md +++ b/plex-docker-migration.md @@ -1,6 +1,6 @@ # Plex Migration: ESXi VM → Docker Container -**Status: EXECUTED 2026-09-16.** Decisions 1 & 2 ratified as Option A / Option A. Cutover completed with ~22 min of Plex downtime (transfer itself 5m38s). All verification in §8 passed except the two items needing a real client, noted there. The old VM is powered on with `plexmediaserver` stopped+disabled; §9 decommission is **not** done. +**Status: EXECUTED 2026-09-16. DECOMMISSIONED 2026-09-17.** Decisions 1 & 2 ratified as Option A / Option A. Cutover completed with ~22 min of Plex downtime (transfer itself 5m38s). All verification in §8 passed except the two items needing a real client, noted there. §9 decommission steps 1, 2 and 4-7 are **done** (see the log at the end of §9); step 3 — deleting the VMDKs — is deliberately **deferred to ~2026-10-15**. **Created:** 2026-09-16 **Source:** `plex` VM, 10.100.0.24, Debian 11 (EOL) **Target:** `DOCKER` host, 10.100.0.35, Debian 13, containerised + Ansible-managed @@ -540,6 +540,38 @@ Cost is only the watch history accumulated while the container was live. To avoi 6. Update docs: [plex.md](plex.md) (rewrite for the container; also drop the misleading `plex.lan.dohertylan.com` FQDN line — it is a wildcard, not a host record), [docker.md](docker.md) (add `plex` to Media Management), [CLAUDE.md](CLAUDE.md) line 24, [esxi.md](esxi.md) lines 71/109/110, [veeam.md](veeam.md) line 58. 7. Update memory: the port-forward now targets `.35` (superseding the 2026-09-16 `.24` fix), and `debian-11-eol-2026-08-31` no longer applies to Plex — only Ignition. +### Decommission log — executed 2026-09-17 + +Run one day after cutover rather than the planned week, on the owner's call that the container was behaving. Every step +below is reversible; the VMDKs were **not** touched. + +**Pre-flight** — re-verified that nothing still pointed at `.24` before powering the VM off: Sonarr and Radarr Plex +connections, Tautulli `pms_ip`/`pms_url`, seerr `plex.ip` (with `machineId` intact), Uptime Kuma monitor 17, Homepage, +the Traefik file-provider service and the UDM `PLEX` port-forward all resolve to `10.100.0.35`. A `grep` of `/opt` found +`.24` only in rotated logs, a stale `seerr/settings.old.json`, and the previously-rendered +`uptime_kuma/configure_monitors.py` (the role default is already `.35`, so the next role run overwrites it). + +| # | Step | Result | +|---|---|---| +| 1 | Veeam `Backup VMs` | Plex removed via `Remove-VBRJobObject`. **Veeam converts a removed include into an explicit `Exclude` rather than deleting the entry** — re-running the removal leaves it as `Exclude`. That is the same shape as the pre-existing `Deployarr` exclusion and is the expected end state, not a failure. | +| 2 | Power off + snapshot | Guest shut down gracefully via `govc vm.power -s Plex` (VMware Tools `toolsOk`); confirmed `plexmediaserver` was still `inactive`/`disabled` first. Snapshot `pre-docker-migration 2026-09-16` then removed **while powered off** — safer consolidation than the plan's on-line ordering. Disk collapsed `Plex-000001.vmdk` → `Plex.vmdk`, `runtime.consolidationNeeded = false`. | +| 3 | Delete VMDKs | **Deferred to ~2026-10-15**, per plan. | +| 4 | Guacamole "Plex SSH" | Deleted via the Guacamole API (connection id 3, `HTTP 204`). **The `guacamole` role is create-only** (`when: item.name not in guac_existing_names`), so it cannot remove a connection — the entry was also deleted from `roles/guacamole/defaults/main.yml` so a rebuild does not recreate it. | +| 5 | Addresses | Neither MAC had a Unifi fixed reservation, so `.24`/`10.50.0.31` were plain leases; both clients (`00:0c:29:4a:ef:3d`, `00:0c:29:4a:ef:47`) were forgotten via `cmd/stamgr`. | +| 6 | Docs | [plex.md](plex.md), [CLAUDE.md](CLAUDE.md), [esxi.md](esxi.md), [veeam.md](veeam.md). `docker.md` already listed `plex`. ESXi port-group counts corrected LAN_100 7→6 and DATA_50 4→3, verified by counting NICs on powered-on VMs. | +| 7 | Memory | `plex-docker-migration` and `debian-11-eol-2026-08-31` updated. | + +**Safety net still in place:** the powered-off VM with intact disks, plus the verified off-box library DB at +`/mnt/share/backups/plex-migration-2026-09-16/` (429 MB, WAL checkpointed to 0 bytes; reads clean — 70,761 `media_parts`, +48,395 `metadata_items`, 3 `section_locations`). + +> Note: a plain `sqlite3` `integrity_check`/`quick_check` on a Plex library DB fails with +> `unknown tokenizer: collating`. That is the stock binary lacking Plex's custom collation on the FTS virtual tables, **not** +> corruption — use `Plex SQLite` for a real whole-DB check. Ordinary table queries work fine with stock `sqlite3`. + +**Before deleting the VMDKs**, confirm no Veeam restore point is still needed, then: +`govc vm.destroy Plex` (removes the VM and its `[NVME_1] Plex/` folder). + --- ## 10. Risks diff --git a/plex.md b/plex.md index 3980ddf..59f303f 100644 --- a/plex.md +++ b/plex.md @@ -1,6 +1,8 @@ # Plex Media Server -> **Migrated to Docker 2026-09-16.** Plex ran on a dedicated Debian 11 VM at 10.100.0.24 until that OS reached LTS EOL (2026-08-31). It now runs as a container on the [Docker host](docker.md). The old VM still exists, powered on but with `plexmediaserver` stopped and disabled, as rollback. Full procedure: [plex-docker-migration.md](plex-docker-migration.md). +> **Migrated to Docker 2026-09-16; old VM decommissioned 2026-09-17.** Plex ran on a dedicated Debian 11 VM at 10.100.0.24 until that OS reached LTS EOL (2026-08-31). It now runs as a container on the [Docker host](docker.md). +> +> The old VM is **powered off** with its disks intact (`[NVME_1] Plex/Plex.vmdk`, 128 GB) and is scheduled for deletion ~2026-10-15. Until then, rollback is: power the VM on, `sudo systemctl enable --now plexmediaserver`, and repoint the UDM `PLEX` port-forward to 10.100.0.24. Its `.24`/`10.50.0.31` leases have been released and it is no longer in the Veeam `Backup VMs` job. A verified copy of the pre-migration library DB is at `/mnt/share/backups/plex-migration-2026-09-16/`. Full procedure: [plex-docker-migration.md](plex-docker-migration.md). ## Access diff --git a/veeam.md b/veeam.md index 4557e4d..380990c 100644 --- a/veeam.md +++ b/veeam.md @@ -55,7 +55,7 @@ | Backup workstation | Endpoint Agent | Yes | Warning | | Backup Joes Macbook | Endpoint Agent | Yes | Warning | -**Backup VMs** includes: Plex, Ignition, HA, DOCKER. Deployarr is explicitly excluded. +**Backup VMs** includes: Ignition, HA, DOCKER. Deployarr and Plex are explicitly excluded (Plex removed 2026-09-17 on decommission; Veeam converts a removed include into an explicit exclude rather than dropping the entry, so the tombstone is expected). `/opt/plex` is now covered by the DOCKER VM backup. **Backup WW_DEV_VM** includes: WW_DEV_VM (Wonderware dev VM, ~256 GB increments).