docs(code-reviews): review Secrets, LocalDb, GalaxyRepository, Overview; re-review changed libs (scadaproj#15)

First reviews of Secrets, LocalDb, GalaxyRepository and Overview, and
2026-11-09 re-reviews of Auth, Health, Theme, Telemetry, Audit and
Configuration, all at 91fb2b4. 49 new open findings, each with a P1/P2/P3
priority and a failure scenario: 1 P1, 13 P2, 35 P3. README regenerated;
REVIEW-PROCESS, template and regen script now cover all ten libraries and
the Priority field.

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 14:33:14 -05:00
parent 91fb2b48dd
commit 477245f5db
14 changed files with 2602 additions and 52 deletions
+178 -5
View File
@@ -7,14 +7,40 @@
| Component spec | `components/observability/spec/SPEC.md` |
| Shared contract | `components/observability/shared-contract/ZB.MOM.WW.Telemetry.md` |
| Status | Reviewed |
| Last reviewed | 2026-06-01 |
| Reviewer | Claude (automated baseline) |
| Commit reviewed | `5f75cd4` |
| Open findings | 0 |
| Last reviewed | 2026-11-09 (re-review; baseline 2026-06-01 at `5f75cd4`) |
| Reviewer | Claude (2026-11-09 re-review, scadaproj#15; baseline: Claude automated) |
| Commit reviewed | `91fb2b4` |
| Open findings | 3 |
## Summary
The library is small, focused, and well-structured: two packages with a clean Serilog/OTel
**2026-11-09 re-review (`91fb2b4`, scadaproj#15).** Re-reviewed at `91fb2b4` (2026-11-09) against the 2026-06-01 baseline `5f75cd4`. Source is at `0.2.0`,
which is built but not published; the feed still carries `0.1.0`. Changes since the baseline:
- nested redaction (`05cc62a`): `RedactionEnricher` projects `StructureValue`/`SequenceValue`/
`DictionaryValue` into mutable views, honours removals, and writes back only what changed;
- one canonical Resource-attribute map (`ZbResource.BuildAttributes`) shared by the OTel SDK and the
Serilog OTLP sink;
- an eager OTLP options validator;
- the 0.2.0 `ZbSerilogOptions` hooks (`MinimumLevelGovernor`, `ConfigureLogger`).
The baseline redaction fixes (Telemetry-001/002/003) hold: removal and nested masking work, and are
tested. The Resource map is genuinely single-sourced. The DI extensions have container-building tests
(`AddZbTelemetryTests`, `MultiHostTests`, `MinimumLevelGovernanceTests`). All 42 tests pass (Serilog 30,
core 12).
There are three new Low findings, each verified with a scratchpad probe:
- **Telemetry-009:** enrichers added through the new `ConfigureLogger` hook run after
`RedactionEnricher`, so properties they add are never redacted.
- **Telemetry-010:** the redaction projection keys logged dictionaries by `ToString()`. Distinct keys
that stringify the same, such as `DateTime` keys differing only by milliseconds, collapse, and an
entry is silently dropped from the log even when the redactor changes nothing.
- **Telemetry-011:** the Telemetry-006 fail-fast validator accepts a scheme-less `host:port` endpoint
as an "absolute URI", and the pre-built-options `AddZbTelemetry(IServiceCollection, ZbTelemetryOptions)`
overload skips validation entirely.
**2026-06-01 baseline (`5f75cd4`).** The library is small, focused, and well-structured: two packages with a clean Serilog/OTel
boundary (the `Serilog.*` stack appears only in the `.Serilog` package; the core package is
pure OTel + ASP.NET Core framework reference), correct argument validation, deliberate
`sealed` types, thorough XML docs, and a deliberate no-process-global-state design for
@@ -34,6 +60,23 @@ happy paths but have no coverage for redactor removal or structured-value redact
## Checklist coverage
2026-11-09 re-review at `91fb2b4`:
| # | Category | Examined | Notes |
|---|----------|----------|-------|
| 1 | Correctness & logic bugs | ☑ | Nested projection/rebuild, removal reconciliation and change detection (`ValueEquals`) are correct, except that dictionary keys collide on `ToString()` → Telemetry-010. Governor ordering (after `ReadFrom.Configuration`) works as documented. |
| 2 | Public API surface & compatibility | ☑ | Additive: `ZbResource.BuildAttributes` (public), the `ZbSerilogOptions` overload, and `InternalsVisibleTo` for the Serilog package. The options-object `AddZbTelemetry` overload skips the new validation → Telemetry-011. |
| 3 | Concurrency & thread safety | ☑ | Enricher state is a `Lazy` with `ExecutionAndPublication`, and projection is per event. No issues found. |
| 4 | Error handling & resilience | ☑ | The validator gives named `ArgumentException`s, but its absolute-URI test is too permissive and is bypassed by one overload → Telemetry-011. |
| 5 | Security & secret handling | ☑ | Redaction reach is now top-level plus nested, with removal honoured. Properties added by `ConfigureLogger` enrichers bypass redaction → Telemetry-009. `Exception` text remains outside the redactor seam, by design and unchanged. |
| 6 | Performance & resource management | ☑ | The per-event projection allocates only when a redactor is registered and the event has properties. Untouched properties are not reallocated. No issues found. |
| 7 | Spec & shared-contract adherence | ☑ | Resource attributes (`service.*`, `host.name`, `site.id`, `node.role`) are identical across all three signals via `BuildAttributes`. No issues found. |
| 8 | Packaging, dependencies & project layout | ☑ | Version 0.2.0; repository URLs corrected to `scadaproj`; the Serilog stack is still confined to `.Serilog`. No issues found. |
| 9 | Testing coverage | ☑ | 42 tests pass, including removal and nested redaction and governor precedence. Untested: enricher ordering against `ConfigureLogger` (Telemetry-009), colliding dictionary keys (Telemetry-010), a scheme-less OTLP endpoint (Telemetry-011). |
| 10 | Documentation & XML docs | ☑ | The `ILogRedactor`/README redaction-reach text is accurate. `ConfigureLogger` docs advertise "extra enrichers" without saying they run after redaction (Telemetry-009). |
<details><summary>2026-06-01 baseline checklist (<code>5f75cd4</code>)</summary>
| # | Category | Examined | Notes |
|---|----------|----------|-------|
| 1 | Correctness & logic bugs | ☑ | Redactor "remove" path is a no-op (Telemetry-001); structured values opaque to redactor (Telemetry-002). |
@@ -47,6 +90,8 @@ happy paths but have no coverage for redactor removal or structured-value redact
| 9 | Testing coverage | ☑ | No tests for redactor removal or structured-value redaction (Telemetry-003). |
| 10 | Documentation & XML docs | ☑ | `MapZbMetrics` doc-comment is stale: claims "only valid when Exporter = Prometheus" (Telemetry-008). |
</details>
## Findings
### Telemetry-001 — `RedactionEnricher` ignores property removal, leaving secrets in the event
@@ -342,3 +387,131 @@ overlay). Align the shared-contract summary for `MapZbMetrics` to match.
Resolved in `544a6dd`, 2026-06-01 — rewrote the `MapZbMetrics` XML doc to state it is valid
under any `Exporter` value (Prometheus always-on; OTLP additive overlay) and aligned the matching
shared-contract summary.
<!-- 2026-11-09 re-review (scadaproj#15) findings below. -->
### Telemetry-009 — Enrichers added via `ZbSerilogOptions.ConfigureLogger` run after `RedactionEnricher`, so their properties are never redacted
| | |
|--|--|
| Severity | Low |
| Priority | P3 |
| Category | Security & secret handling |
| Status | Open |
| Location | `ZB.MOM.WW.Telemetry/src/ZB.MOM.WW.Telemetry.Serilog/ZbSerilogExtensions.cs:118-120`, `ZB.MOM.WW.Telemetry/src/ZB.MOM.WW.Telemetry.Serilog/ZbSerilogOptions.cs:41-46` |
**Description**
`AddZbSerilogCore` calls `ZbSerilogConfig.Apply(…)`, which registers `RedactionEnricher`, and only then
invokes `serilogOptions.ConfigureLogger`. Serilog runs enrichers in registration order, so any
`lc.Enrich.With(...)` or `Enrich.WithProperty(...)` added in the hook stamps its properties after
redaction has already run. They reach every sink unredacted. The `ConfigureLogger` XML doc and the
README advertise this hook as the place for "extra enrichers" and say it runs "after … the shared
enricher/redaction/export set". They never say this means its enrichers are outside the redaction
guarantee that `ILogRedactor` documents ("Called on every log event before it reaches any sink").
Enrichers that come from `ReadFrom.Configuration` (for example `FromLogContext`) are unaffected,
because they are registered before redaction.
**Failure scenario**
MxGateway, which registers an `ILogRedactor` that masks `apiKey`, moves to the 0.2.0 overload and adds
`s.ConfigureLogger = (lc, sp) => lc.Enrich.With(new RequestHeadersEnricher(sp))`, which stamps a
`ClientApiKey` or `apiKey` property. Every log event now carries the raw key to the file and OTLP sinks.
A scratchpad probe reproduced it: a `LoggerConfiguration` with `RedactionEnricher` (redactor masks
`apiKey`) followed by `Enrich.WithProperty("apiKey","s3cret")` emitted `apiKey="s3cret"`. Today this
is latent: no family app uses the `ConfigureLogger` hook yet.
**Recommendation**
Register `RedactionEnricher` as the final stage. One way is to invoke `ConfigureLogger` before
`ZbSerilogConfig.Apply`'s redaction step. Another is to apply redaction through a wrapping sink or the
last enricher after the hook. A third is to keep the order but document plainly that hook-added
enrichers bypass `ILogRedactor`. Add a test that enriches a sensitive property in `ConfigureLogger` and
asserts it is redacted.
**Resolution**
_Unresolved._
### Telemetry-010 — Redaction projection keys logged dictionaries by `ToString()`, silently dropping entries whose keys stringify identically
| | |
|--|--|
| Severity | Low |
| Priority | P3 |
| Category | Correctness & logic bugs |
| Status | Open |
| Location | `ZB.MOM.WW.Telemetry/src/ZB.MOM.WW.Telemetry.Serilog/RedactionEnricher.cs:134-143` (projection), `:163-173` (rebuild) |
**Description**
`Project` turns a `DictionaryValue` into a `ProjectedDictionary` keyed by
`pair.Key.Value?.ToString() ?? ""`. Two distinct keys with the same string form overwrite each other in
both the projected values and `OriginalKeys`. Examples are `DateTime`/`DateTimeOffset` keys that differ
below one second (the default `ToString()` has no fractional seconds), and a null key alongside `""`.
`Rebuild` then emits one entry where there were two. `ValueEquals` sees the element count differ and
writes the shrunken dictionary back onto the event. This happens for every event once any
`ILogRedactor` is registered, even one that does nothing.
**Failure scenario**
A scratchpad probe registered a no-op `ILogRedactor` and logged
`log.Information("{Map}", new Dictionary<DateTime,string>{ [12:00:00.100]="first", [12:00:00.200]="second" })`.
With the enricher the result was `[(11/9/2026 12:00:00 PM: "second")]`. Without it the result was
`[(…12:00:00: "first"), (…12:00:00: "second")]`. Diagnostic log data (for example per-timestamp
samples) is silently lost in MxGateway, the redaction-enabled consumer. Nothing about it is secret.
**Recommendation**
Keep the original `ScalarValue` keys alongside the string view, for example by projecting to a list of
key/value pairs or disambiguating colliding string keys. Alternatively, skip projection and pass a
dictionary through untouched whenever its stringified keys are not unique. Add a regression test with
colliding keys.
**Resolution**
_Unresolved._
### Telemetry-011 — OTLP fail-fast validation accepts scheme-less `host:port` and is skipped by the pre-built-options `AddZbTelemetry` overload
| | |
|--|--|
| Severity | Low |
| Priority | P3 |
| Category | Error handling & resilience |
| Status | Open |
| Location | `ZB.MOM.WW.Telemetry/src/ZB.MOM.WW.Telemetry/ZbTelemetryOptionsValidator.cs:36`, `ZB.MOM.WW.Telemetry/src/ZB.MOM.WW.Telemetry/ZbTelemetryExtensions.cs:40-46` |
**Description**
The Telemetry-006 resolution added `ZbTelemetryOptionsValidator.Validate`, which checks
`Uri.TryCreate(OtlpEndpoint, UriKind.Absolute, out _)`. A scheme-less `host:port`, which is the typo the
original finding named ("missing scheme"), parses as an absolute URI whose scheme is the host name, so
it passes. The checks it ran:
- `"localhost:4317"` → `True`, scheme `localhost`;
- `"collector:4317"` → `True`, scheme `collector`;
- only a literal IP such as `10.0.0.5:4317` is rejected.
Separately, the validator is called only from `BuildOptions` (the two lambda overloads) and
`AddZbSerilog`. The public `AddZbTelemetry(IServiceCollection, ZbTelemetryOptions)` overload validates
neither the OTLP endpoint nor the required `ServiceName`.
**Failure scenario**
An operator sets `Exporter=Otlp, OtlpEndpoint="otel-collector:4317"`, missing `http://`. Startup passes
validation, the exporter gets a URI with scheme `otel-collector`, and OTLP traces, metrics and logs fail
later inside the exporter rather than as the named startup error Telemetry-006 promised. Prometheus
`/metrics` keeps working, so the loss is easy to miss. Any caller that builds `ZbTelemetryOptions`
itself and uses the options overload gets no validation at all. No family app uses that overload today.
**Recommendation**
Require `uri.Scheme` to be `http` or `https` (the OTLP gRPC/HTTP schemes) and a non-empty
`uri.Host`. Call `BuildOptions`-equivalent validation, including `ServiceName`, from the
`AddZbTelemetry(IServiceCollection, ZbTelemetryOptions)` overload as well. Add a test for
`"localhost:4317"`.
**Resolution**
_Unresolved._