wwtools/mbproxy/docs/Features/BcdRewriting.md

# BCD Rewriting

The BCD rewriter is the inline codec that translates DirectLOGIC's native Binary-Coded Decimal register values to and from plain binary integers on every relevant Modbus TCP PDU. It is the one place in the proxy that knows which registers are BCD, so upstream consumers can treat the wire as plain `Int16` / `Int32`.

## Why BCD Rewriting Exists

The DL205 / DL260 family stores numeric V-memory register values in native BCD, not binary. The decimal integer `1234` in `V2000` lands on the Modbus wire as `0x1234` (nibbles `1`, `2`, `3`, `4`) — not as the binary `0x04D2`. See [`../Reference/dl205.md`](../Reference/dl205.md) for the device-side rationale and the V-memory ↔ Modbus translation rules.

Upstream consumers (Wonderware, Historian, OPC UA gateways, generic Modbus clients written against the standard) expect plain binary integers. Asking every consumer to BCD-decode the wire is brittle: each consumer would carry the same tag list, the same word-order quirks, and the same risk of drift. The rewriter centralises that translation so the rest of the world sees plain `Int16` / `Int32` and the proxy is the single source of truth for "which addresses are BCD."

The rewriter touches only the BCD slots declared in configuration. Every other byte of the PDU — non-BCD registers, coils, discrete inputs, diagnostic function codes, exception responses — passes through unchanged. MBAP transaction IDs, unit IDs, and the MBAP length field are preserved end-to-end; the rewriter only re-encodes payload bytes whose width does not change.

## CDAB Word Order for 32-Bit Values

A 32-bit BCD value spans a register pair at `Address` and `Address+1` in CDAB (low-word-first) order:

- The register at `Address` holds the **low 4 BCD digits**.
- The register at `Address+1` holds the **high 4 BCD digits**.
- Decoded decimal = `Decode16(high) * 10_000 + Decode16(low)`.

This follows directly from DirectLOGIC's CDAB word convention (see [`../Reference/dl205.md`](../Reference/dl205.md) → Word Order).

Worked example — the register pair `[0x1234][0x5678]` reads on the wire as the low word `0x1234` first and the high word `0x5678` second:

```text
Address:    raw 0x1234 → low  4 digits = 1234
Address+1:  raw 0x5678 → high 4 digits = 5678

Decoded decimal = 5678 * 10_000 + 1234 = 56_781_234
```

`BcdCodec.Encode32` and `BcdCodec.Decode32` in [`../../src/Mbproxy/Bcd/BcdCodec.cs`](../../src/Mbproxy/Bcd/BcdCodec.cs) implement this in both directions. `Encode32(12_345_678)` returns `(low: 0x5678, high: 0x1234)`.

The 16-bit codec is a straight nibble pack / unpack:

```csharp
// From BcdCodec.cs — Encode16 packs four decimal digits into four BCD nibbles.
int d3 = value / 1000;
int d2 = (value / 100) % 10;
int d1 = (value / 10)  % 10;
int d0 = value         % 10;
return (ushort)((d3 << 12) | (d2 << 8) | (d1 << 4) | d0);
```

`Decode16` is the reverse, with a `HasBadNibble` guard that throws `FormatException` if any nibble is `>= 0xA`. The Phase-04 rewrite pipeline catches the exception and surfaces it as a `mbproxy.rewrite.invalid_bcd` warning event instead of corrupting the payload.

## BCD Tag Configuration Shape

Every BCD register the rewriter handles is described by a `BcdTag` record from [`../../src/Mbproxy/Bcd/BcdTag.cs`](../../src/Mbproxy/Bcd/BcdTag.cs):

```csharp
public sealed record BcdTag(ushort Address, byte Width, int CacheTtlMs = 0)
{
    public bool IsThirtyTwoBit => Width == 32;
    public ushort HighRegister => /* Address + 1 for 32-bit tags */;
}
```

- `Address` is the **Modbus PDU register address** (zero-based, decimal). Configuration must translate from octal V-memory to PDU-decimal before reaching this struct — `V2000` octal = decimal 1024 = `0x0400`. The proxy does not perform that translation itself.
- `Width` is `16` (single register) or `32` (CDAB register pair at `Address` and `Address+1`). `BcdTag.Create` rejects any other width.
- `CacheTtlMs` is the Phase-11 response-cache opt-in (covered separately in [`../Architecture/ResponseCache.md`](../Architecture/ResponseCache.md)); it has no effect on rewriter behaviour.

The wire-format options shape lives in [`../../src/Mbproxy/Options/BcdTagOptions.cs`](../../src/Mbproxy/Options/BcdTagOptions.cs) and [`../../src/Mbproxy/Options/BcdTagListOptions.cs`](../../src/Mbproxy/Options/BcdTagListOptions.cs). Configured tags resolve through `BcdTagMapBuilder.Build` (see [`../../src/Mbproxy/Bcd/BcdTagMapBuilder.cs`](../../src/Mbproxy/Bcd/BcdTagMapBuilder.cs)) into an immutable `BcdTagMap` ([`../../src/Mbproxy/Bcd/BcdTagMap.cs`](../../src/Mbproxy/Bcd/BcdTagMap.cs)) per PLC.

Holding-register (FC03) and input-register (FC04) addresses share the **same** configured tag space. The DL205 / DL260 surfaces V-memory through both tables, so the rewriter applies the configured tag list against both FC03 and FC04 responses.

## Function-Code Scope Table

The rewriter touches payloads only for the function codes below. Every other FC — coils (FC01, FC05, FC15), discrete inputs (FC02), diagnostics, exception responses — passes through byte-for-byte.

| FC | Direction | Action |
|----|-----------|--------|
| 03 | Request | Pass through (read; no payload rewrite needed) |
| 03 | Response | Re-encode covered BCD slots from raw nibbles → binary integer |
| 04 | Request | Pass through |
| 04 | Response | Same as FC03 response |
| 06 | Request | Re-encode binary integer → BCD nibbles before forwarding |
| 06 | Response | Decode BCD nibbles → binary integer on the echo (NModbus-style clients validate the echo and would throw otherwise) |
| 16 | Request | Per-register over the configured slots |
| 16 | Response | Pass through (the response carries only start+qty, not values) |

The FC06 response decode is non-obvious: the PLC echoes back the value it actually wrote, which is now BCD-encoded because the proxy rewrote the request on the way in. Clients that validate the echo equals the value they sent (NModbus and similar libraries do this) would throw on the round-trip if the proxy did not decode the echo back.

`BcdPduPipeline.Process` dispatches on direction first, then on FC:

```csharp
public void Process(MbapDirection direction, ReadOnlySpan<byte> mbapHeader,
                    Span<byte> pdu, PduContext context)
{
    if (context is not PerPlcContext ctx) return;
    if (pdu.Length < 1) return;

    byte fc = pdu[0];
    ctx.Counters.IncrementPdusForwarded();
    ctx.Counters.IncrementFcCount(fc);

    if (direction == MbapDirection.RequestToBackend)
        ProcessRequest(fc, pdu, ctx);
    else
        ProcessResponse(fc, pdu, ctx);
}
```

`PerPlcContext` carries the `BcdTagMap`, the per-PLC `ProxyCounters`, the logger, and the matched `InFlightRequest` from the multiplexer's correlation map. If a caller passes a plain `PduContext` (e.g. a test harness using `NoopPduPipeline` alongside the BCD pipeline), the rewriter returns without touching the PDU.

## Partial-Overlap Policy

A request that touches only **one** register of a configured 32-bit BCD pair cannot be re-encoded correctly. There are two shapes:

1. An FC03 / FC04 read whose range covers the low address but not the high address (`qty=1` at the low address) or vice versa.
2. An FC06 write to either the low or high address of a 32-bit pair, or an FC16 write whose range covers only one of the two registers.

In every case the rewriter **passes the PDU through raw** and emits a `mbproxy.rewrite.partial_bcd` warning. The `PartialBcdWarnings` counter increments per occurrence.

The proxy never synthesises a Modbus exception for a partial-overlap. Exception response codes are reserved for transport failure (the per-request watchdog manufactures `0x0B` Gateway Target Device Failed To Respond; the PLC itself produces `0x01`–`0x04`). Using an exception code to signal a configuration / client mismatch would conflate "the device or the path failed" with "the client straddled a 32-bit boundary," and operators chasing the exception would look at the wrong layer.

The rationale for warn-plus-passthrough rather than silent rewrite: silently rewriting only the half the client touched would corrupt the value (a 16-bit BCD encode of a 32-bit binary integer is meaningless). A warning-plus-raw passthrough surfaces the misconfiguration loudly while leaving the client to discover the mismatch in its own data path.

The FC16 request path makes the partial-overlap decision per-tag inside its loop over `TryGetForRange` hits:

```csharp
if (tag.IsThirtyTwoBit)
{
    bool lowInRange  = offsetWords >= 0 && offsetWords < qty;
    bool highInRange = (offsetWords + 1) >= 0 && (offsetWords + 1) < qty;

    if (!lowInRange || !highInRange)
    {
        RewriterLogEvents.PartialBcd(ctx.Logger, ctx.PlcName,
            tag.Address, startAddress, qty);
        ctx.Counters.IncrementPartialBcd();
        continue;
    }
    // ...both registers in range — reconstruct, encode, write back...
}
```

For a 32-bit FC16 write where both registers are in range, the rewriter reconstructs the client's 32-bit binary value from the CDAB pair (`clientHigh * 10_000 + clientLow`), runs `BcdCodec.Encode32` to produce the BCD register pair, and writes both registers back to the PDU buffer in place.

## Unsigned Only

DL205 / DL260 BCD is non-negative in the default ladder pattern. `BcdCodec.Encode16` rejects values outside `[0, 9999]`; `BcdCodec.Encode32` rejects values outside `[0, 99_999_999]`. The rewriter does not implement signed BCD; signed conventions vary by site and any value out of range surfaces as `mbproxy.rewrite.invalid_bcd` rather than being silently coerced.

## Exception Pass-Through

Modbus exception responses pass through unchanged. The rewriter detects an exception response by the high bit of the function code (`fc & 0x80 != 0`), emits a `mbproxy.rewrite.exception_passthrough` event, increments the per-FC exception counter, and returns without touching the payload.

Covered exception codes:

- `0x01` Illegal Function
- `0x02` Illegal Data Address
- `0x03` Illegal Data Value
- `0x04` Server Device Failure
- `0x0B` Gateway Target Device Failed To Respond — manufactured by the per-request watchdog when a correlation entry ages past `Connection.BackendRequestTimeoutMs`. The rewriter does not distinguish proxy-manufactured from PLC-originated exception codes; both pass through identically.

The rewriter increments `Counters.IncrementBackendException(exceptionCode)` per exception so the four common codes surface on the status page through `ExceptionCounts` (`Code01`, `Code02`, `Code03`, `Code04`). The Gateway-Target `0x0B` is also recorded but is more usefully traced through the watchdog log events rather than the per-code counter slot.

## Where the Rewriter Runs in the Pipeline

The rewriter is implemented as `BcdPduPipeline` in [`../../src/Mbproxy/Proxy/BcdPduPipeline.cs`](../../src/Mbproxy/Proxy/BcdPduPipeline.cs), registered as the singleton `IPduPipeline` in production. The class is stateless; per-call state arrives via the `PerPlcContext` passed into `Process`, which carries the `BcdTagMap`, the per-PLC counters, the logger, and (on the response path) the matched `InFlightRequest` from the multiplexer's correlation map.

Per-PLC pipeline ordering:

```text
Upstream request →
    [cache lookup (Phase 11)] →
    [coalesce check (Phase 10)] →
    [BCD rewriter — request path] →
    backend send

Backend response →
    [BCD rewriter — response path] →
    [response-cache populate (Phase 11)] →
    [fanout to all coalesced parties]
```

The rewriter runs **once per request** on the multiplexer's outbound path and **once per response** on the inbound path. Per-party MBAP TxId restoration happens after the rewriter on fanout, so the rewriter only ever sees the canonical (shared) PDU buffer.

For Phase-11 cache hits, the response cache stores **POST-rewriter bytes** — the rewriter is bypassed on hits, both as a CPU optimisation and as a correctness guarantee (a future rewriter change does not retroactively re-transform an entry that was decoded against an earlier rewriter version). See [`../Architecture/ResponseCache.md`](../Architecture/ResponseCache.md).

On the response path, the rewriter cannot infer the original `(StartAddress, Qty)` of an FC03 / FC04 read from the response alone — the response carries only `[fc][byteCount][reg0Hi][reg0Lo]...`. The multiplexer's `CorrelationMap` keys the matched `InFlightRequest` to the response and attaches it to `PerPlcContext.CurrentRequest` before invoking the rewriter, so concurrent responses from different upstream clients each decode against their own request range without cross-talk. If `CurrentRequest` is null (e.g. a unit-test fixture invoking the pipeline directly) the rewriter passes the response bytes through unchanged.

## Hybrid Tag Resolution

For each PLC, the effective BCD tag list is `Global ∪ Add − Remove`, resolved by `BcdTagMapBuilder.Build` in this order:

1. Seed the working set from `BcdTagListOptions.Global`.
2. Apply `PlcBcdOverrides.Remove` — drop every address listed. `Remove` matches by address only; width is irrelevant.
3. Apply `PlcBcdOverrides.Add` — insert each entry into the working set. If an address already exists from `Global`, the `Add` entry **wins** (this is how a per-PLC width override is expressed: list the same address in `Add` with a different `Width`).

The shapes are declared in [`../../src/Mbproxy/Options/BcdTagListOptions.cs`](../../src/Mbproxy/Options/BcdTagListOptions.cs):

```csharp
public sealed class BcdTagListOptions
{
    public IReadOnlyList<BcdTagOptions> Global { get; init; } = [];
}

public sealed class PlcBcdOverrides
{
    public IReadOnlyList<BcdTagOptions> Add { get; init; } = [];
    public IReadOnlyList<ushort> Remove { get; init; } = [];
}
```

Resolution produces a `ValidationResult` carrying the resolved `BcdTagMap`, a list of `BcdError` entries, and a list of `BcdWarning` entries. Callers treat any non-empty `Errors` list as a fatal configuration problem for that PLC.

The user-facing syntax for `Global` + per-PLC `Add` / `Remove` is documented in [`../Operations/Configuration.md`](../Operations/Configuration.md).

`BcdTagMap.TryGetForRange` is the hot-path range scan used by both the request and response paths. It returns every `BcdTag` whose register footprint intersects `[startAddress, startAddress + qty)`, each carrying its zero-based word `OffsetWords` relative to `startAddress`. A 32-bit tag whose low word starts **before** the range but whose high word lies inside the range returns with a **negative** `OffsetWords` — that is the partial-overlap signal the rewriter consumes when deciding whether to re-encode or warn. The no-hit path returns the empty-list singleton without allocating.

## Validation at Startup and Hot-Reload

`BcdTagMapBuilder.Build` runs the same validation pipeline at process start and on every hot-reload of `appsettings.json`. The validation results fall into three buckets, defined in [`../../src/Mbproxy/Bcd/BcdValidationError.cs`](../../src/Mbproxy/Bcd/BcdValidationError.cs):

- `BcdValidationError.DuplicateAddress` — the same address appears more than once in the **resolved** list (after `Remove` and `Add` have been applied). Fatal error; the entry is excluded from the map.
- `BcdValidationError.OverlappingHighRegister` — a 32-bit entry's high register (`Address+1`) collides with the `Address` of a separate entry in the resolved list. Fatal error.
- `BcdValidationError.InvalidWidth` — an entry's `Width` is not `16` or `32`. Fatal error; the entry is excluded.
- `BcdWarning` — a `Remove` entry whose address does not appear in `Global`. Non-fatal, but typically indicates stale configuration (the global entry was removed without cleaning up the per-PLC override).

A successful hot-reload that changes the resolved tag list reseats the per-PLC `BcdTagMap` and, for Phase 11, flushes the entire PLC response cache (see [`./HotReload.md`](./HotReload.md)). In-flight requests already past the rewriter are not retroactively re-rewritten; the next PDU sees the new map. A failed validation rejects the reload as a whole and the previous map stays in effect.

## Counter Accounting

The rewriter feeds two counters that surface on the status page:

- `pdus.rewrittenSlots` — `RewrittenSlots` on `PlcPdusStatus`, incremented per re-encoded register. A 32-bit BCD pair counts as 2 slots; a 16-bit tag counts as 1. The FC06 echo decode is **not** counted to avoid double-counting the FC06 request that already incremented the slot on the way out.
- `pdus.partialBcdWarnings` — `PartialBcdWarnings` on `PlcPdusStatus`, incremented once per partial-overlap event (request or response path).

An out-of-range value (`< 0` or `> 9999` for 16-bit; `< 0` or `> 99_999_999` for 32-bit) on a write, or a bad nibble (`>= 0xA`) on a read, increments an internal invalid-BCD counter and emits `mbproxy.rewrite.invalid_bcd` at warning. The PDU passes through raw in that case; the rewriter never substitutes a value the client did not send (writes) or the PLC did not return (reads).

Both counters are exposed on the status page; see [`../Operations/StatusPage.md`](../Operations/StatusPage.md). The corresponding log events (`mbproxy.rewrite.partial_bcd`, `mbproxy.rewrite.invalid_bcd`, `mbproxy.rewrite.exception_passthrough`) are catalogued in [`../Reference/LogEvents.md`](../Reference/LogEvents.md). Partial-overlap troubleshooting is covered in [`../Operations/Troubleshooting.md`](../Operations/Troubleshooting.md).

The `dl205.json` pymodbus simulator profile encodes BCD test fixtures used by the integration test suite; see [`../Testing/Simulator.md`](../Testing/Simulator.md).

A few invariants the rewriter relies on and the test suite enforces:

- The MBAP length field is **never** modified. Every re-encoded slot is the same byte width as the original (16-bit register in, 16-bit register out), so the PDU length is byte-stable.
- The rewriter is **stateless** at the class level. `BcdPduPipeline` holds no fields; everything per-call arrives via `PerPlcContext`. The same instance is safe to call concurrently from multiple upstream-read tasks and the single backend reader task on a given multiplexer.
- The rewriter operates on the canonical (shared) PDU buffer. Per-party MBAP TxId restoration on coalesced fanout happens **after** the rewriter, so any per-party byte copy only happens when fanout has more than one party.

## Related Documentation

- [`../Architecture/Overview.md`](../Architecture/Overview.md) — service-wide architecture and per-PLC pipeline shape
- [`../Architecture/ResponseCache.md`](../Architecture/ResponseCache.md) — Phase-11 response cache; the cache stores post-rewriter bytes and bypasses the rewriter on hits
- [`./HotReload.md`](./HotReload.md) — hot-reload semantics for BCD tag-list changes
- [`../Operations/Configuration.md`](../Operations/Configuration.md) — `BcdTags.Global` and per-PLC `Add` / `Remove` syntax
- [`../Operations/StatusPage.md`](../Operations/StatusPage.md) — `pdus.rewrittenSlots` and `pdus.partialBcdWarnings` exposure
- [`../Operations/Troubleshooting.md`](../Operations/Troubleshooting.md) — diagnosing partial-overlap warnings
- [`../Reference/LogEvents.md`](../Reference/LogEvents.md) — `mbproxy.rewrite.*` event catalogue
- [`../Testing/Simulator.md`](../Testing/Simulator.md) — the `dl205.json` simulator profile that encodes BCD test fixtures
- [`../Reference/dl205.md`](../Reference/dl205.md) — DL205 / DL260 BCD encoding, CDAB word order, and V-memory ↔ Modbus translation