jdescopingtool/PLANS/2025-01-22-regex-transformer-implementation.md

Regex Transformer Implementation Plan

For Claude: REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

Goal: Add a RegexTransformer to the DataSync ETL pipeline that transforms string column values using regex, with a custom ConfigManager editor featuring live test/preview.

Architecture: The transformer extends DataTransformerBase and overrides GetValue() to apply regex transformations. Supports two modes: Find & Replace (uses Regex.Replace) and Match & Extract (extracts first capture group). The ConfigManager gets a new RegexTransformerViewModel and RegexEditorView with integrated pattern testing.

Tech Stack: .NET 10, System.Text.RegularExpressions, Avalonia UI, xUnit + NSubstitute

Design Doc: PLANS/2025-01-22-regex-transformer-design.md

---

Task 1: Add NonMatchBehavior Enum to PipelineModel

Files:

• Modify: NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs:181-215

Step 1: Add the enum and new properties

Add after line 215 (after TransformerModel class closing brace), then add properties to TransformerModel:

// Add this using at top of file:
using System.Text.Json.Serialization;

// Add this enum after TransformerModel class (line ~216):
/// <summary>
/// Specifies behavior when a regex pattern does not match the input value.
/// </summary>
[JsonConverter(typeof(JsonStringEnumConverter))]
public enum NonMatchBehavior
{
    /// <summary>Keep the original value unchanged.</summary>
    KeepOriginal,
    /// <summary>Return null/DBNull.</summary>
    ReturnNull,
    /// <summary>Return an empty string.</summary>
    ReturnEmpty
}

// Add these properties inside TransformerModel class (after OutputColumn property, ~line 214):

    /// <summary>
    /// Gets or sets the column name for Regex transformer.
    /// </summary>
    public string? ColumnName { get; set; }

    /// <summary>
    /// Gets or sets the regex pattern for Regex transformer.
    /// </summary>
    public string? Pattern { get; set; }

    /// <summary>
    /// Gets or sets the replacement string for Regex transformer (null = Match &amp; Extract mode).
    /// </summary>
    public string? Replacement { get; set; }

    /// <summary>
    /// Gets or sets whether regex matching is case-insensitive.
    /// </summary>
    public bool IgnoreCase { get; set; }

    /// <summary>
    /// Gets or sets the behavior when regex pattern does not match.
    /// </summary>
    public NonMatchBehavior NonMatchBehavior { get; set; } = NonMatchBehavior.KeepOriginal;

Step 2: Verify it compiles

Run: dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj Expected: Build succeeded

Step 3: Commit

git add NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add NonMatchBehavior enum and regex properties to TransformerModel

Add configuration model support for the new Regex transformer including:
- NonMatchBehavior enum with JSON string serialization
- ColumnName, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
EOF
)"

---

Task 2: Create RegexTransformer with First Test (Find & Replace)

Files:

• Create: NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs
• Create: NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs

Step 1: Write the first failing test

Create test file:

using System.Data;
using JdeScoping.DataSync.Etl.Transformers;
using NSubstitute;

namespace JdeScoping.DataSync.Tests.Etl.Transformers;

public class RegexTransformerTests
{
    [Fact]
    public void FindReplace_RemovesPrefix()
    {
        // Arrange
        var source = CreateMockReader(
            columns: new[] { "BatchID", "Name" },
            values: new object[] { "IIS_12345", "Test" });

        var transformer = new RegexTransformer(
            columnName: "BatchID",
            pattern: "^IIS_",
            replacement: "");

        // Act
        var reader = transformer.Transform(source);
        source.Read().Returns(true);
        reader.Read();

        // Assert
        Assert.Equal("12345", reader.GetValue(0));
        Assert.Equal("Test", reader.GetValue(1)); // Other column unchanged
    }

    private static IDataReader CreateMockReader(string[] columns, object[] values)
    {
        var reader = Substitute.For<IDataReader>();
        reader.FieldCount.Returns(columns.Length);
        for (int i = 0; i < columns.Length; i++)
        {
            var index = i;
            reader.GetName(index).Returns(columns[index]);
            reader.GetOrdinal(columns[index]).Returns(index);
            reader.GetFieldType(index).Returns(values[index]?.GetType() ?? typeof(object));
            reader.GetValue(index).Returns(values[index]);
            reader.IsDBNull(index).Returns(values[index] == null || values[index] == DBNull.Value);
        }
        return reader;
    }
}

Step 2: Run test to verify it fails

Run: dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests.FindReplace_RemovesPrefix" -v n Expected: FAIL - "The type or namespace name 'RegexTransformer' could not be found"

Step 3: Write minimal implementation

Create NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs:

using System.Data;
using System.Text.RegularExpressions;

namespace JdeScoping.DataSync.Etl.Transformers;

/// <summary>
/// Specifies behavior when a regex pattern does not match the input value.
/// </summary>
public enum NonMatchBehavior
{
    /// <summary>Keep the original value unchanged.</summary>
    KeepOriginal,
    /// <summary>Return null/DBNull.</summary>
    ReturnNull,
    /// <summary>Return an empty string.</summary>
    ReturnEmpty
}

/// <summary>
/// A data transformer that applies regex transformations to string values in a column.
/// Supports two modes: Find &amp; Replace (when replacement is provided) and Match &amp; Extract
/// (when replacement is null, extracts first capture group).
/// </summary>
public class RegexTransformer : DataTransformerBase
{
    private readonly string _columnName;
    private readonly string _pattern;
    private readonly string? _replacement;
    private readonly bool _ignoreCase;
    private readonly NonMatchBehavior _nonMatchBehavior;

    private Regex? _regex;
    private int _columnOrdinal = -1;

    /// <inheritdoc />
    public override string TransformerName => $"Regex:{_columnName}";

    /// <summary>
    /// Creates a new RegexTransformer.
    /// </summary>
    /// <param name="columnName">The column to transform.</param>
    /// <param name="pattern">The regex pattern.</param>
    /// <param name="replacement">Replacement string for Find &amp; Replace mode, or null for Match &amp; Extract mode.</param>
    /// <param name="ignoreCase">Whether to use case-insensitive matching.</param>
    /// <param name="nonMatchBehavior">Behavior when pattern does not match.</param>
    public RegexTransformer(
        string columnName,
        string pattern,
        string? replacement = null,
        bool ignoreCase = false,
        NonMatchBehavior nonMatchBehavior = NonMatchBehavior.KeepOriginal)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(columnName);
        ArgumentException.ThrowIfNullOrWhiteSpace(pattern);

        _columnName = columnName;
        _pattern = pattern;
        _replacement = replacement;
        _ignoreCase = ignoreCase;
        _nonMatchBehavior = nonMatchBehavior;
    }

    /// <inheritdoc />
    protected override void OnInitialize(IDataReader source)
    {
        _columnOrdinal = source.GetOrdinal(_columnName);

        var options = RegexOptions.Compiled;
        if (_ignoreCase)
            options |= RegexOptions.IgnoreCase;

        _regex = new Regex(_pattern, options);
    }

    /// <inheritdoc />
    public override object GetValue(int ordinal, IDataReader source)
    {
        var value = source.GetValue(ordinal);

        // Only transform the target column
        if (ordinal != _columnOrdinal)
            return value;

        // Pass through null/DBNull
        if (value == null || value == DBNull.Value)
            return value;

        var stringValue = value.ToString() ?? string.Empty;

        // Find & Replace mode (replacement is not null)
        if (_replacement != null)
        {
            return _regex!.Replace(stringValue, _replacement);
        }

        // Match & Extract mode (replacement is null)
        var match = _regex!.Match(stringValue);
        if (match.Success && match.Groups.Count > 1)
        {
            return match.Groups[1].Value;
        }

        // No match - apply NonMatchBehavior
        return _nonMatchBehavior switch
        {
            NonMatchBehavior.ReturnNull => DBNull.Value,
            NonMatchBehavior.ReturnEmpty => string.Empty,
            _ => value // KeepOriginal
        };
    }

    /// <inheritdoc />
    public override Type GetFieldType(int ordinal, IDataReader source)
    {
        // Target column always returns string
        if (ordinal == _columnOrdinal)
            return typeof(string);
        return source.GetFieldType(ordinal);
    }
}

Step 4: Run test to verify it passes

Run: dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests.FindReplace_RemovesPrefix" -v n Expected: PASS

Step 5: Commit

git add NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
feat(datasync): add RegexTransformer with Find & Replace mode

Initial implementation supporting:
- Find & Replace mode with regex pattern and replacement string
- Case-insensitive option
- NonMatchBehavior enum for handling non-matches
EOF
)"

---

Task 3: Add Match & Extract Mode Tests

Files:

• Modify: NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs

Step 1: Write the failing test for Match & Extract

Add to test class:

[Fact]
public void MatchExtract_ExtractsFirstCaptureGroup()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "ID_12345" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null); // null = Match & Extract mode

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_KeepOriginal()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.KeepOriginal);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("UNKNOWN", reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_ReturnNull()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.ReturnNull);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(DBNull.Value, reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_ReturnEmpty()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.ReturnEmpty);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(string.Empty, reader.GetValue(0));
}

Step 2: Run tests to verify they pass

Run: dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests" -v n Expected: All 5 tests PASS

Step 3: Commit

git add NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
test(datasync): add Match & Extract mode tests for RegexTransformer

Tests cover:
- Extracting first capture group
- NonMatchBehavior: KeepOriginal, ReturnNull, ReturnEmpty
EOF
)"

---

Task 4: Add Edge Case Tests

Files:

• Modify: NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs

Step 1: Add edge case tests

Add to test class:

[Fact]
public void FindReplace_UseCaptureGroups()
{
    // Arrange - swap two numbers
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "123-456" });

    var transformer = new RegexTransformer(
        columnName: "Value",
        pattern: @"(\d+)-(\d+)",
        replacement: "$2-$1");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("456-123", reader.GetValue(0));
}

[Fact]
public void IgnoreCase_MatchesDifferentCase()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID" },
        values: new object[] { "IIS_12345" });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^iis_", // lowercase pattern
        replacement: "",
        ignoreCase: true);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
}

[Fact]
public void NullValue_PassesThrough()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID" },
        values: new object[] { DBNull.Value });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^IIS_",
        replacement: "");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(DBNull.Value, reader.GetValue(0));
}

[Fact]
public void NonTargetColumn_Unchanged()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID", "OtherColumn" },
        values: new object[] { "IIS_12345", "IIS_Should_Not_Change" });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^IIS_",
        replacement: "");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
    Assert.Equal("IIS_Should_Not_Change", reader.GetValue(1));
}

[Fact]
public void InvalidRegex_ThrowsOnTransform()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "test" });

    var transformer = new RegexTransformer(
        columnName: "Value",
        pattern: "[invalid(regex",
        replacement: "");

    // Act & Assert
    var ex = Assert.Throws<RegexParseException>(() => transformer.Transform(source));
    Assert.Contains("Invalid", ex.Message, StringComparison.OrdinalIgnoreCase);
}

[Fact]
public void ColumnNotFound_ThrowsOnTransform()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "test" });
    source.GetOrdinal("NonExistent").Returns(_ => throw new IndexOutOfRangeException("Column not found"));

    var transformer = new RegexTransformer(
        columnName: "NonExistent",
        pattern: "test",
        replacement: "");

    // Act & Assert
    Assert.Throws<IndexOutOfRangeException>(() => transformer.Transform(source));
}

Step 2: Run all transformer tests

Run: dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests" -v n Expected: All 11 tests PASS

Step 3: Commit

git add NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
test(datasync): add edge case tests for RegexTransformer

Tests cover:
- Capture group substitution in replacement
- Case-insensitive matching
- Null/DBNull passthrough
- Non-target columns unchanged
- Invalid regex pattern handling
- Column not found handling
EOF
)"

---

Task 5: Create RegexTransformerViewModel

Files:

• Modify: NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs

Step 1: Add the ViewModel class

Add after JdeDateTransformerViewModel class (before TransformerFactory):

/// <summary>
/// View model for Regex transformer.
/// </summary>
public class RegexTransformerViewModel : TransformerStepViewModelBase
{
    private string _columnName = string.Empty;
    private string _pattern = string.Empty;
    private string? _replacement = string.Empty;
    private bool _isFindReplaceMode = true;
    private bool _ignoreCase;
    private NonMatchBehavior _nonMatchBehavior = NonMatchBehavior.KeepOriginal;

    // Test feature fields
    private string _testInput = string.Empty;
    private string _testResultValue = string.Empty;
    private string _testResultLabel = string.Empty;
    private string _testResultIcon = string.Empty;
    private string _testResultBackground = string.Empty;
    private bool _hasTestResult;
    private bool _hasTestError;
    private string _testErrorMessage = string.Empty;

    public RegexTransformerViewModel(TransformerModel model, Action onChanged) : base(onChanged)
    {
        _columnName = model.ColumnName ?? string.Empty;
        _pattern = model.Pattern ?? string.Empty;
        _replacement = model.Replacement;
        _isFindReplaceMode = model.Replacement != null;
        _ignoreCase = model.IgnoreCase;
        _nonMatchBehavior = model.NonMatchBehavior;
        TestPatternCommand = new RelayCommand(ExecuteTestPattern);
    }

    public RegexTransformerViewModel(Action onChanged) : base(onChanged)
    {
        TestPatternCommand = new RelayCommand(ExecuteTestPattern);
    }

    public override string TransformerType => "Regex";
    public override string DisplayName => "Regex Transform";
    public override string Icon => "󰑑"; // mdi-regex
    public override string Summary => !string.IsNullOrEmpty(_columnName)
        ? $"{_columnName}: {(_isFindReplaceMode ? "Replace" : "Extract")}"
        : "Configure...";

    /// <summary>Gets or sets the column name to transform.</summary>
    public string ColumnName
    {
        get => _columnName;
        set
        {
            if (SetProperty(ref _columnName, value ?? string.Empty))
            {
                OnPropertyChanged(nameof(Summary));
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the regex pattern.</summary>
    public string Pattern
    {
        get => _pattern;
        set
        {
            if (SetProperty(ref _pattern, value ?? string.Empty))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the replacement string (Find &amp; Replace mode).</summary>
    public string? Replacement
    {
        get => _replacement;
        set
        {
            if (SetProperty(ref _replacement, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets whether Find &amp; Replace mode is active.</summary>
    public bool IsFindReplaceMode
    {
        get => _isFindReplaceMode;
        set
        {
            if (SetProperty(ref _isFindReplaceMode, value))
            {
                OnPropertyChanged(nameof(IsMatchExtractMode));
                OnPropertyChanged(nameof(PatternHelpText));
                OnPropertyChanged(nameof(Summary));
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets whether Match &amp; Extract mode is active.</summary>
    public bool IsMatchExtractMode
    {
        get => !_isFindReplaceMode;
        set => IsFindReplaceMode = !value;
    }

    /// <summary>Gets the help text for the pattern field based on current mode.</summary>
    public string PatternHelpText => _isFindReplaceMode
        ? "Pattern to search for in the column value"
        : "Pattern with capture group - first group (parentheses) will be extracted";

    /// <summary>Gets or sets whether matching is case-insensitive.</summary>
    public bool IgnoreCase
    {
        get => _ignoreCase;
        set
        {
            if (SetProperty(ref _ignoreCase, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the behavior when pattern doesn't match.</summary>
    public NonMatchBehavior NonMatchBehavior
    {
        get => _nonMatchBehavior;
        set
        {
            if (SetProperty(ref _nonMatchBehavior, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    // Test feature properties
    public string TestInput
    {
        get => _testInput;
        set => SetProperty(ref _testInput, value ?? string.Empty);
    }

    public string TestResultValue
    {
        get => _testResultValue;
        set => SetProperty(ref _testResultValue, value);
    }

    public string TestResultLabel
    {
        get => _testResultLabel;
        set => SetProperty(ref _testResultLabel, value);
    }

    public string TestResultIcon
    {
        get => _testResultIcon;
        set => SetProperty(ref _testResultIcon, value);
    }

    public string TestResultBackground
    {
        get => _testResultBackground;
        set => SetProperty(ref _testResultBackground, value);
    }

    public bool HasTestResult
    {
        get => _hasTestResult;
        set => SetProperty(ref _hasTestResult, value);
    }

    public bool HasTestError
    {
        get => _hasTestError;
        set => SetProperty(ref _hasTestError, value);
    }

    public string TestErrorMessage
    {
        get => _testErrorMessage;
        set => SetProperty(ref _testErrorMessage, value);
    }

    public ICommand TestPatternCommand { get; }

    private void ExecuteTestPattern()
    {
        ClearTestResult();

        if (string.IsNullOrEmpty(_pattern))
        {
            HasTestError = true;
            TestErrorMessage = "Pattern is required";
            return;
        }

        try
        {
            var options = _ignoreCase ? RegexOptions.IgnoreCase : RegexOptions.None;
            var regex = new Regex(_pattern, options);

            string result;
            bool matched;

            if (_isFindReplaceMode)
            {
                result = regex.Replace(_testInput, _replacement ?? string.Empty);
                matched = regex.IsMatch(_testInput);
            }
            else
            {
                var match = regex.Match(_testInput);
                if (match.Success && match.Groups.Count > 1)
                {
                    result = match.Groups[1].Value;
                    matched = true;
                }
                else
                {
                    matched = false;
                    result = _nonMatchBehavior switch
                    {
                        NonMatchBehavior.ReturnNull => "(null)",
                        NonMatchBehavior.ReturnEmpty => "(empty)",
                        _ => _testInput
                    };
                }
            }

            HasTestResult = true;
            TestResultValue = result;
            TestResultLabel = matched ? "Output" : "No Match";
            TestResultIcon = matched ? "✓" : "—";
            TestResultBackground = matched ? "#22C55E" : "#F59E0B";
        }
        catch (RegexParseException ex)
        {
            HasTestError = true;
            TestErrorMessage = ex.Message;
        }
    }

    private void ClearTestResult()
    {
        HasTestResult = false;
        HasTestError = false;
        TestResultValue = string.Empty;
        TestResultLabel = string.Empty;
        TestErrorMessage = string.Empty;
    }

    public override TransformerModel ToModel() => new()
    {
        Type = TransformerType,
        ColumnName = _columnName,
        Pattern = _pattern,
        Replacement = _isFindReplaceMode ? _replacement : null,
        IgnoreCase = _ignoreCase,
        NonMatchBehavior = _nonMatchBehavior
    };
}

Step 2: Add using statement

Add at top of file:

using System.Text.RegularExpressions;
using JdeScoping.ConfigManager.Models;

Step 3: Verify it compiles

Run: dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj Expected: Build succeeded

Step 4: Commit

git add NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add RegexTransformerViewModel

Implements ViewModel for Regex transformer editor with:
- Column, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
- Mode toggle between Find & Replace and Match & Extract
- Live test/preview functionality with error handling
EOF
)"

---

Task 6: Update TransformerFactory

Files:

• Modify: NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs:284-318

Step 1: Update factory switch statements and AvailableTypes

In TransformerFactory.Create() method, add case:

"regex" => new RegexTransformerViewModel(model, onChanged),

In TransformerFactory.CreateNew() method, add case:

"regex" => new RegexTransformerViewModel(onChanged),

Update AvailableTypes:

public static IReadOnlyList<string> AvailableTypes => ["ColumnDrop", "ColumnRename", "JdeDate", "Regex"];

Step 2: Verify it compiles

Run: dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj Expected: Build succeeded

Step 3: Commit

git add NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): register Regex transformer in TransformerFactory

Add Regex to:
- Create() factory method
- CreateNew() factory method
- AvailableTypes list
EOF
)"

---

Task 7: Create RegexEditorView XAML

Files:

• Create: NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml
• Create: NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs

Step 1: Create the XAML file

Create NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml:

<UserControl xmlns="https://github.com/avaloniaui"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:steps="using:JdeScoping.ConfigManager.ViewModels.PipelineSteps"
             xmlns:models="using:JdeScoping.ConfigManager.Models"
             x:Class="JdeScoping.ConfigManager.Views.Editors.RegexEditorView"
             x:DataType="steps:RegexTransformerViewModel">

    <StackPanel Spacing="16">
        <!-- Header -->
        <StackPanel>
            <TextBlock Text="Regex Transformer"
                       Foreground="#E6EDF5" FontSize="14" FontWeight="SemiBold"/>
            <TextBlock Text="Transform column values using regular expressions"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Column Name -->
        <StackPanel Spacing="4">
            <StackPanel Orientation="Horizontal" Spacing="2">
                <TextBlock Text="Column"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
                <TextBlock Text="*" Foreground="#FF6B6B" FontSize="12"/>
            </StackPanel>
            <TextBox Text="{Binding ColumnName}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="BatchID"/>
            <TextBlock Text="Column to apply the regex transformation"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Mode Selection -->
        <StackPanel Spacing="4">
            <TextBlock Text="Mode" Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
            <StackPanel Orientation="Horizontal" Spacing="12">
                <RadioButton GroupName="RegexMode"
                             IsChecked="{Binding IsFindReplaceMode}"
                             Foreground="#E6EDF5">
                    <TextBlock Text="Find &amp; Replace" FontSize="12"/>
                </RadioButton>
                <RadioButton GroupName="RegexMode"
                             IsChecked="{Binding IsMatchExtractMode}"
                             Foreground="#E6EDF5">
                    <TextBlock Text="Match &amp; Extract" FontSize="12"/>
                </RadioButton>
            </StackPanel>
        </StackPanel>

        <!-- Pattern -->
        <StackPanel Spacing="4">
            <StackPanel Orientation="Horizontal" Spacing="2">
                <TextBlock Text="Pattern"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
                <TextBlock Text="*" Foreground="#FF6B6B" FontSize="12"/>
            </StackPanel>
            <TextBox Text="{Binding Pattern}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="^IIS_"/>
            <TextBlock Text="{Binding PatternHelpText}"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Replacement (only visible in Find & Replace mode) -->
        <StackPanel Spacing="4" IsVisible="{Binding IsFindReplaceMode}">
            <TextBlock Text="Replacement"
                       Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
            <TextBox Text="{Binding Replacement}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="(empty to remove)"/>
            <TextBlock Text="Text to replace matches with (use $1, $2 for capture groups)"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Options Row -->
        <StackPanel Spacing="8">
            <TextBlock Text="Options" Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>

            <Grid ColumnDefinitions="*,*" RowDefinitions="Auto">
                <!-- Case Insensitive Toggle -->
                <CheckBox Grid.Column="0"
                          IsChecked="{Binding IgnoreCase}"
                          Foreground="#E6EDF5" FontSize="12">
                    <TextBlock Text="Case Insensitive" FontSize="12"/>
                </CheckBox>

                <!-- Non-Match Behavior -->
                <StackPanel Grid.Column="1" Spacing="4">
                    <TextBlock Text="If No Match"
                               Foreground="#5C6A7A" FontSize="11"/>
                    <ComboBox SelectedItem="{Binding NonMatchBehavior}"
                              Background="#232A35" Foreground="#E6EDF5"
                              BorderBrush="#3D4550" Height="32"
                              FontSize="11" MinWidth="140">
                        <ComboBox.ItemsSource>
                            <x:Array Type="{x:Type models:NonMatchBehavior}">
                                <models:NonMatchBehavior>KeepOriginal</models:NonMatchBehavior>
                                <models:NonMatchBehavior>ReturnNull</models:NonMatchBehavior>
                                <models:NonMatchBehavior>ReturnEmpty</models:NonMatchBehavior>
                            </x:Array>
                        </ComboBox.ItemsSource>
                    </ComboBox>
                </StackPanel>
            </Grid>
        </StackPanel>

        <!-- Test/Preview Section -->
        <Border Background="#151920" BorderBrush="#2D3540" BorderThickness="1"
                CornerRadius="4" Padding="12" Margin="0,8,0,0">
            <StackPanel Spacing="12">
                <TextBlock Text="Test Pattern"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>

                <!-- Test Input Row -->
                <Grid ColumnDefinitions="*,8,Auto">
                    <TextBox Grid.Column="0"
                             Text="{Binding TestInput}"
                             Background="#232A35" Foreground="#E6EDF5"
                             BorderBrush="#3D4550" Height="36"
                             FontFamily="JetBrains Mono"
                             Watermark="IIS_12345"/>
                    <Button Grid.Column="2"
                            Content="Test"
                            Background="#3B82F6" Foreground="#FFFFFF"
                            BorderThickness="0" Height="36"
                            Padding="16,0"
                            Command="{Binding TestPatternCommand}"/>
                </Grid>

                <!-- Result Display -->
                <Border Background="#0D0F12" BorderBrush="#2D3540" BorderThickness="1"
                        CornerRadius="4" Padding="10" MinHeight="44"
                        IsVisible="{Binding HasTestResult}">
                    <Grid ColumnDefinitions="Auto,8,*">
                        <!-- Status Icon -->
                        <Border Grid.Column="0"
                                Width="24" Height="24" CornerRadius="12"
                                Background="{Binding TestResultBackground}"
                                VerticalAlignment="Center">
                            <TextBlock Text="{Binding TestResultIcon}"
                                       Foreground="#FFFFFF"
                                       FontSize="12" FontWeight="Bold"
                                       HorizontalAlignment="Center"
                                       VerticalAlignment="Center"/>
                        </Border>

                        <!-- Result Text -->
                        <StackPanel Grid.Column="2" Spacing="2" VerticalAlignment="Center">
                            <TextBlock Text="{Binding TestResultLabel}"
                                       Foreground="#5C6A7A" FontSize="10"/>
                            <TextBlock Text="{Binding TestResultValue}"
                                       Foreground="#E6EDF5" FontSize="12"
                                       FontFamily="JetBrains Mono"/>
                        </StackPanel>
                    </Grid>
                </Border>

                <!-- Error Display -->
                <Border Background="#2D1F1F" BorderBrush="#5C2626" BorderThickness="1"
                        CornerRadius="4" Padding="10"
                        IsVisible="{Binding HasTestError}">
                    <StackPanel Spacing="2">
                        <TextBlock Text="Invalid Pattern"
                                   Foreground="#FF6B6B" FontSize="11" FontWeight="Medium"/>
                        <TextBlock Text="{Binding TestErrorMessage}"
                                   Foreground="#CC8888" FontSize="11"
                                   TextWrapping="Wrap"/>
                    </StackPanel>
                </Border>
            </StackPanel>
        </Border>

        <!-- Help Info -->
        <Border Background="#151920" BorderBrush="#2D3540" BorderThickness="1"
                CornerRadius="4" Padding="12" Margin="0,4,0,0">
            <StackPanel Spacing="4">
                <TextBlock Text="Pattern Examples" Foreground="#9BA8B8" FontSize="11" FontWeight="Medium"/>
                <TextBlock Text="  ^IIS_        Remove 'IIS_' prefix" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
                <TextBlock Text="  _SUFFIX$     Remove '_SUFFIX' at end" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
                <TextBlock Text="  (\d+)        Extract first number" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
            </StackPanel>
        </Border>
    </StackPanel>
</UserControl>

Step 2: Create the code-behind file

Create NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs:

using Avalonia.Controls;

namespace JdeScoping.ConfigManager.Views.Editors;

public partial class RegexEditorView : UserControl
{
    public RegexEditorView()
    {
        InitializeComponent();
    }
}

Step 3: Verify it compiles

Run: dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj Expected: Build succeeded

Step 4: Commit

git add NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add RegexEditorView Avalonia UI

Implements editor with:
- Column name input
- Mode toggle (Find & Replace / Match & Extract)
- Pattern and Replacement inputs
- Case insensitive checkbox
- NonMatchBehavior dropdown
- Live test/preview section with result display
- Pattern examples help box
EOF
)"

---

Task 8: Register DataTemplate in MainWindow

Files:

• Modify: NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml:58-60

Step 1: Add DataTemplate for RegexTransformerViewModel

Add after line 60 (after JdeDateTransformerViewModel template):

        <DataTemplate DataType="{x:Type steps:RegexTransformerViewModel}">
            <editors:RegexEditorView/>
        </DataTemplate>

Step 2: Verify it compiles

Run: dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj Expected: Build succeeded

Step 3: Commit

git add NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml
git commit -m "$(cat <<'EOF'
feat(configmanager): register RegexEditorView DataTemplate in MainWindow
EOF
)"

---

Task 9: Add ViewModel Unit Tests

Files:

• Create: NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs

Step 1: Create test file

using JdeScoping.ConfigManager.Models;
using JdeScoping.ConfigManager.ViewModels.PipelineSteps;

namespace JdeScoping.ConfigManager.Tests.ViewModels;

public class RegexTransformerViewModelTests
{
    [Fact]
    public void Constructor_FromModel_LoadsAllProperties()
    {
        // Arrange
        var model = new TransformerModel
        {
            Type = "Regex",
            ColumnName = "BatchID",
            Pattern = "^IIS_",
            Replacement = "",
            IgnoreCase = true,
            NonMatchBehavior = NonMatchBehavior.ReturnEmpty
        };

        // Act
        var vm = new RegexTransformerViewModel(model, () => { });

        // Assert
        Assert.Equal("BatchID", vm.ColumnName);
        Assert.Equal("^IIS_", vm.Pattern);
        Assert.Equal("", vm.Replacement);
        Assert.True(vm.IsFindReplaceMode);
        Assert.True(vm.IgnoreCase);
        Assert.Equal(NonMatchBehavior.ReturnEmpty, vm.NonMatchBehavior);
    }

    [Fact]
    public void Constructor_FromModel_MatchExtractMode_WhenReplacementNull()
    {
        // Arrange
        var model = new TransformerModel
        {
            Type = "Regex",
            ColumnName = "Code",
            Pattern = @"(\d+)",
            Replacement = null
        };

        // Act
        var vm = new RegexTransformerViewModel(model, () => { });

        // Assert
        Assert.False(vm.IsFindReplaceMode);
        Assert.True(vm.IsMatchExtractMode);
    }

    [Fact]
    public void ToModel_SerializesCorrectly_FindReplaceMode()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "BatchID",
            Pattern = "^IIS_",
            Replacement = "",
            IsFindReplaceMode = true,
            IgnoreCase = true,
            NonMatchBehavior = NonMatchBehavior.KeepOriginal
        };

        // Act
        var model = vm.ToModel();

        // Assert
        Assert.Equal("Regex", model.Type);
        Assert.Equal("BatchID", model.ColumnName);
        Assert.Equal("^IIS_", model.Pattern);
        Assert.Equal("", model.Replacement);
        Assert.True(model.IgnoreCase);
        Assert.Equal(NonMatchBehavior.KeepOriginal, model.NonMatchBehavior);
    }

    [Fact]
    public void ToModel_SerializesCorrectly_MatchExtractMode()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "Code",
            Pattern = @"(\d+)",
            IsFindReplaceMode = false
        };

        // Act
        var model = vm.ToModel();

        // Assert
        Assert.Null(model.Replacement); // null indicates Match & Extract mode
    }

    [Fact]
    public void TestPatternCommand_ValidPattern_ShowsResult()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = "^IIS_",
            Replacement = "",
            IsFindReplaceMode = true,
            TestInput = "IIS_12345"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.True(vm.HasTestResult);
        Assert.False(vm.HasTestError);
        Assert.Equal("12345", vm.TestResultValue);
        Assert.Equal("Output", vm.TestResultLabel);
        Assert.Equal("✓", vm.TestResultIcon);
    }

    [Fact]
    public void TestPatternCommand_InvalidPattern_ShowsError()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = "[invalid(regex",
            Replacement = "",
            TestInput = "test"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.False(vm.HasTestResult);
        Assert.True(vm.HasTestError);
        Assert.NotEmpty(vm.TestErrorMessage);
    }

    [Fact]
    public void TestPatternCommand_MatchExtract_NoMatch_ShowsNonMatchBehavior()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = @"(\d+)",
            IsFindReplaceMode = false,
            NonMatchBehavior = NonMatchBehavior.ReturnNull,
            TestInput = "NoNumbers"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.True(vm.HasTestResult);
        Assert.Equal("No Match", vm.TestResultLabel);
        Assert.Equal("(null)", vm.TestResultValue);
    }

    [Fact]
    public void ModeSwitch_UpdatesPatternHelpText()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            IsFindReplaceMode = true
        };
        var findReplaceHelp = vm.PatternHelpText;

        // Act
        vm.IsFindReplaceMode = false;
        var matchExtractHelp = vm.PatternHelpText;

        // Assert
        Assert.NotEqual(findReplaceHelp, matchExtractHelp);
        Assert.Contains("capture group", matchExtractHelp, StringComparison.OrdinalIgnoreCase);
    }

    [Fact]
    public void Summary_ShowsColumnAndMode()
    {
        // Arrange & Act
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "BatchID",
            IsFindReplaceMode = true
        };

        // Assert
        Assert.Contains("BatchID", vm.Summary);
        Assert.Contains("Replace", vm.Summary);
    }

    [Fact]
    public void PropertyChange_NotifiesChanged()
    {
        // Arrange
        var changedCalled = false;
        var vm = new RegexTransformerViewModel(() => changedCalled = true);

        // Act
        vm.ColumnName = "NewColumn";

        // Assert
        Assert.True(changedCalled);
    }
}

Step 2: Run tests

Run: dotnet test NEW/tests/JdeScoping.ConfigManager.Tests/JdeScoping.ConfigManager.Tests.csproj --filter "FullyQualifiedName~RegexTransformerViewModelTests" -v n Expected: All 10 tests PASS

Step 3: Commit

git add NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs
git commit -m "$(cat <<'EOF'
test(configmanager): add RegexTransformerViewModel unit tests

Tests cover:
- Loading from model
- Serializing to model
- Test pattern command (success and error cases)
- Mode switching and help text
- Summary display
- Change notification
EOF
)"

---

Task 10: Run Full Test Suite and Final Verification

Files: None (verification only)

Step 1: Run all transformer tests

Run: dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~TransformerTests" -v n Expected: All tests PASS

Step 2: Run all ConfigManager tests

Run: dotnet test NEW/tests/JdeScoping.ConfigManager.Tests/JdeScoping.ConfigManager.Tests.csproj -v n Expected: All tests PASS

Step 3: Build entire solution

Run: dotnet build NEW/JdeScoping.slnx Expected: Build succeeded with no errors

Step 4: Final commit (if any uncommitted changes)

git status
# If clean, no action needed

---

Summary

Tasks completed:

1. Added NonMatchBehavior enum and regex properties to TransformerModel
2. Created RegexTransformer class with Find & Replace mode
3. Added Match & Extract mode tests
4. Added edge case tests (capture groups, case-insensitive, null handling)
5. Created RegexTransformerViewModel with test functionality
6. Updated TransformerFactory to register Regex transformer
7. Created RegexEditorView Avalonia UI
8. Registered DataTemplate in MainWindow.axaml
9. Added ViewModel unit tests
10. Verified full test suite passes

Files created:

• NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs
• NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml
• NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs
• NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
• NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs

Files modified:

• NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs
• NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs
• NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml