fix(secrets): ${secret:} expansion survives configuration reload (scadaproj#32)

ExpandConfigurationAsync wrote through config[key] = ..., which only sets the
providers' in-memory Data; a file provider's reload replaces it, so editing
appsettings.json on a running service reverted every expanded key to its
${secret:...} placeholder.

- ConfigurationManager: expanded values are pinned in an internal overlay
  source appended last; its provider's Load() is a no-op and Build always
  returns the same provider, so reloads (and source rebuilds) keep the pins
- any other IConfigurationRoot: in-place write as before, re-applied from the
  root's reload token
- all references resolve before anything is written (a fail-closed miss no
  longer leaves the config half expanded)
- tests: SecretExpansionReloadTests (real JSON file + Reload(), options
  monitor, plain root); three fail without the fix
- docs: README reload note, CHANGELOG
- code-reviews: Secrets-003 Resolved; README regenerated

Claude-Session: https://claude.ai/code/session_01GT67AEHd2Acv1YPwGfXEoV
This commit is contained in:
Joseph Doherty
2026-11-09 14:52:45 -05:00
parent c3b07d6766
commit f60f641069
7 changed files with 277 additions and 15 deletions
@@ -0,0 +1,51 @@
using Microsoft.Extensions.Configuration;
namespace ZB.MOM.WW.Secrets.Configuration;
/// <summary>
/// The in-memory overlay <see cref="SecretReferenceExpander"/> appends to a
/// <see cref="ConfigurationManager"/> to hold expanded <c>${secret:}</c> values (scadaproj#32).
/// </summary>
/// <remarks>
/// <see cref="Build"/> always returns the SAME provider: <see cref="ConfigurationManager"/> rebuilds
/// every provider from its sources when its source list changes, and a fresh, empty provider would
/// silently drop the pins and expose the placeholders again.
/// </remarks>
internal sealed class SecretExpansionConfigurationSource : IConfigurationSource
{
/// <summary>The one provider this source ever builds.</summary>
public SecretExpansionConfigurationProvider Provider { get; } = new();
/// <inheritdoc />
public IConfigurationProvider Build(IConfigurationBuilder builder) => Provider;
}
/// <summary>
/// Holds the pinned expansions. <see cref="Load"/> is deliberately a no-op, so a configuration
/// reload — which calls <c>Load</c> on every provider — leaves the pins in place while the file
/// providers beneath it reload their placeholder text.
/// </summary>
internal sealed class SecretExpansionConfigurationProvider : ConfigurationProvider
{
/// <inheritdoc />
public override void Load()
{
// Intentionally empty: the pinned values are a startup snapshot and never reload.
}
/// <summary>Pins <paramref name="values"/>, optionally raising this provider's reload token.</summary>
/// <param name="values">Configuration key to expanded value.</param>
/// <param name="notify">Whether to signal listeners that the overlay changed.</param>
public void Pin(IReadOnlyDictionary<string, string> values, bool notify)
{
foreach (KeyValuePair<string, string> entry in values)
{
Data[entry.Key] = entry.Value;
}
if (notify)
{
OnReload();
}
}
}
@@ -1,5 +1,6 @@
using System.Text.RegularExpressions;
using Microsoft.Extensions.Configuration;
using Microsoft.Extensions.Primitives;
using ZB.MOM.WW.Secrets.Abstractions;
namespace ZB.MOM.WW.Secrets.Configuration;
@@ -73,10 +74,9 @@ public sealed class SecretReferenceExpander
}
/// <summary>
/// Walks every entry in <paramref name="config"/> and rewrites, in place, any value that
/// contains a <c>${secret:...}</c> token to its expanded plaintext. Writes go through the
/// <see cref="IConfigurationRoot"/> indexer so subsequent reads — and any options binding done
/// afterward — observe the expanded values.
/// Walks every entry in <paramref name="config"/> and expands, to its plaintext, any value that
/// contains a <c>${secret:...}</c> token, so subsequent reads — and any options binding done
/// afterward — observe the expanded values, <b>including after a configuration reload</b>.
/// </summary>
/// <remarks>
/// This MUST run <em>after</em> configuration is loaded and <em>before</em> options validation
@@ -84,6 +84,34 @@ public sealed class SecretReferenceExpander
/// Fail-closed: a missing referenced secret propagates <see cref="SecretNotFoundException"/> out
/// of startup.
/// <para>
/// <b>Reload (scadaproj#32).</b> Expansion is a startup snapshot: each key that held a reference
/// is pinned to the value it expanded to, for the life of the process, and a reload of the
/// underlying files never reverts it to the placeholder text. How the pin is held depends on
/// <paramref name="config"/>:
/// </para>
/// <list type="bullet">
/// <item><description>
/// A <see cref="ConfigurationManager"/> (what <c>WebApplication.CreateBuilder</c> and
/// <c>Host.CreateApplicationBuilder</c> expose as <c>builder.Configuration</c>, whose
/// <c>appsettings*.json</c> sources reload on change by default) gets a dedicated in-memory
/// overlay source appended LAST. It never reloads, so it wins over every provider beneath it
/// through any number of file reloads, and every <c>IOptionsMonitor</c> rebind reads the pinned
/// value. This is the robust form — pass <c>builder.Configuration</c> here.
/// </description></item>
/// <item><description>
/// Any other <see cref="IConfigurationRoot"/> (e.g. from <c>ConfigurationBuilder.Build()</c>)
/// cannot take a new provider, so the values are written into its providers as before and
/// re-applied from the root's reload token. Direct <see cref="IConfiguration"/> readers see the
/// pinned value after a reload; an <c>IOptionsMonitor</c> bound to that root may rebind from the
/// placeholder first, because change callbacks run in an order this library does not control.
/// Build such a root without <c>reloadOnChange</c>, or pass a <see cref="ConfigurationManager"/>.
/// </description></item>
/// </list>
/// <para>
/// Consequently, editing a referencing key (or rotating the secret it names) takes effect on the
/// next restart, never on reload. Keys that did not hold a reference at startup reload normally.
/// </para>
/// <para>
/// Documentation/comment keys are skipped: any entry whose leaf key segment (the part after the
/// last <c>:</c>) begins with an underscore — the widely-used <c>"_comment"</c> JSON convention —
/// is left untouched. This lets an appsettings file document the <c>${secret:...}</c> syntax in a
@@ -91,7 +119,7 @@ public sealed class SecretReferenceExpander
/// a real reference to resolve. Functional keys are never named with a leading underscore.
/// </para>
/// </remarks>
/// <param name="config">The loaded configuration root to mutate.</param>
/// <param name="config">The loaded configuration root to expand.</param>
/// <param name="ct">A token to cancel the operation.</param>
/// <exception cref="SecretNotFoundException">A referenced secret does not exist (fail-closed).</exception>
public async Task ExpandConfigurationAsync(IConfigurationRoot config, CancellationToken ct)
@@ -110,9 +138,64 @@ public sealed class SecretReferenceExpander
}
}
// Resolve EVERYTHING before writing anything, so a fail-closed miss part-way through leaves
// the configuration untouched rather than half expanded.
var expanded = new Dictionary<string, string>(StringComparer.OrdinalIgnoreCase);
foreach (KeyValuePair<string, string> entry in pending)
{
config[entry.Key] = await ExpandAsync(entry.Value, ct).ConfigureAwait(false);
expanded[entry.Key] = await ExpandAsync(entry.Value, ct).ConfigureAwait(false);
}
if (expanded.Count == 0)
{
return;
}
if (config is IConfigurationBuilder builder)
{
PinInOverlay(builder, expanded);
}
else
{
PinInPlace(config, expanded);
}
}
// ConfigurationManager: append (or reuse) the overlay source. It is never reloaded, so it keeps
// winning over the file providers beneath it whatever they reload to.
private static void PinInOverlay(IConfigurationBuilder builder, Dictionary<string, string> expanded)
{
SecretExpansionConfigurationSource? overlay =
builder.Sources.OfType<SecretExpansionConfigurationSource>().FirstOrDefault();
if (overlay is null)
{
overlay = new SecretExpansionConfigurationSource();
overlay.Provider.Pin(expanded, notify: false);
builder.Add(overlay);
}
else
{
overlay.Provider.Pin(expanded, notify: true);
}
}
// A plain IConfigurationRoot has a fixed provider list: write through the indexer (every
// provider) and re-apply after each reload, which is the best a closed root allows.
private static void PinInPlace(IConfigurationRoot config, Dictionary<string, string> expanded)
{
var pinned = new Dictionary<string, string>(expanded, StringComparer.OrdinalIgnoreCase);
Apply(config, pinned);
ChangeToken.OnChange(config.GetReloadToken, () => Apply(config, pinned));
static void Apply(IConfigurationRoot root, Dictionary<string, string> values)
{
foreach (KeyValuePair<string, string> entry in values)
{
if (!string.Equals(root[entry.Key], entry.Value, StringComparison.Ordinal))
{
root[entry.Key] = entry.Value;
}
}
}
}