jdescopingtool/PLANS/2025-01-22-regex-transformer-implementation.md

# Regex Transformer Implementation Plan

> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.

**Goal:** Add a RegexTransformer to the DataSync ETL pipeline that transforms string column values using regex, with a custom ConfigManager editor featuring live test/preview.

**Architecture:** The transformer extends `DataTransformerBase` and overrides `GetValue()` to apply regex transformations. Supports two modes: Find & Replace (uses `Regex.Replace`) and Match & Extract (extracts first capture group). The ConfigManager gets a new `RegexTransformerViewModel` and `RegexEditorView` with integrated pattern testing.

**Tech Stack:** .NET 10, System.Text.RegularExpressions, Avalonia UI, xUnit + NSubstitute

**Design Doc:** `PLANS/2025-01-22-regex-transformer-design.md`

---

## Task 1: Add NonMatchBehavior Enum to PipelineModel

**Files:**
- Modify: `NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs:181-215`

**Step 1: Add the enum and new properties**

Add after line 215 (after `TransformerModel` class closing brace), then add properties to `TransformerModel`:

```csharp
// Add this using at top of file:
using System.Text.Json.Serialization;

// Add this enum after TransformerModel class (line ~216):
/// <summary>
/// Specifies behavior when a regex pattern does not match the input value.
/// </summary>
[JsonConverter(typeof(JsonStringEnumConverter))]
public enum NonMatchBehavior
{
    /// <summary>Keep the original value unchanged.</summary>
    KeepOriginal,
    /// <summary>Return null/DBNull.</summary>
    ReturnNull,
    /// <summary>Return an empty string.</summary>
    ReturnEmpty
}

// Add these properties inside TransformerModel class (after OutputColumn property, ~line 214):

    /// <summary>
    /// Gets or sets the column name for Regex transformer.
    /// </summary>
    public string? ColumnName { get; set; }

    /// <summary>
    /// Gets or sets the regex pattern for Regex transformer.
    /// </summary>
    public string? Pattern { get; set; }

    /// <summary>
    /// Gets or sets the replacement string for Regex transformer (null = Match &amp; Extract mode).
    /// </summary>
    public string? Replacement { get; set; }

    /// <summary>
    /// Gets or sets whether regex matching is case-insensitive.
    /// </summary>
    public bool IgnoreCase { get; set; }

    /// <summary>
    /// Gets or sets the behavior when regex pattern does not match.
    /// </summary>
    public NonMatchBehavior NonMatchBehavior { get; set; } = NonMatchBehavior.KeepOriginal;
```

**Step 2: Verify it compiles**

Run: `dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj`
Expected: Build succeeded

**Step 3: Commit**

```bash
git add NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add NonMatchBehavior enum and regex properties to TransformerModel

Add configuration model support for the new Regex transformer including:
- NonMatchBehavior enum with JSON string serialization
- ColumnName, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
EOF
)"
```

---

## Task 2: Create RegexTransformer with First Test (Find & Replace)

**Files:**
- Create: `NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs`
- Create: `NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs`

**Step 1: Write the first failing test**

Create test file:

```csharp
using System.Data;
using JdeScoping.DataSync.Etl.Transformers;
using NSubstitute;

namespace JdeScoping.DataSync.Tests.Etl.Transformers;

public class RegexTransformerTests
{
    [Fact]
    public void FindReplace_RemovesPrefix()
    {
        // Arrange
        var source = CreateMockReader(
            columns: new[] { "BatchID", "Name" },
            values: new object[] { "IIS_12345", "Test" });

        var transformer = new RegexTransformer(
            columnName: "BatchID",
            pattern: "^IIS_",
            replacement: "");

        // Act
        var reader = transformer.Transform(source);
        source.Read().Returns(true);
        reader.Read();

        // Assert
        Assert.Equal("12345", reader.GetValue(0));
        Assert.Equal("Test", reader.GetValue(1)); // Other column unchanged
    }

    private static IDataReader CreateMockReader(string[] columns, object[] values)
    {
        var reader = Substitute.For<IDataReader>();
        reader.FieldCount.Returns(columns.Length);
        for (int i = 0; i < columns.Length; i++)
        {
            var index = i;
            reader.GetName(index).Returns(columns[index]);
            reader.GetOrdinal(columns[index]).Returns(index);
            reader.GetFieldType(index).Returns(values[index]?.GetType() ?? typeof(object));
            reader.GetValue(index).Returns(values[index]);
            reader.IsDBNull(index).Returns(values[index] == null || values[index] == DBNull.Value);
        }
        return reader;
    }
}
```

**Step 2: Run test to verify it fails**

Run: `dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests.FindReplace_RemovesPrefix" -v n`
Expected: FAIL - "The type or namespace name 'RegexTransformer' could not be found"

**Step 3: Write minimal implementation**

Create `NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs`:

```csharp
using System.Data;
using System.Text.RegularExpressions;

namespace JdeScoping.DataSync.Etl.Transformers;

/// <summary>
/// Specifies behavior when a regex pattern does not match the input value.
/// </summary>
public enum NonMatchBehavior
{
    /// <summary>Keep the original value unchanged.</summary>
    KeepOriginal,
    /// <summary>Return null/DBNull.</summary>
    ReturnNull,
    /// <summary>Return an empty string.</summary>
    ReturnEmpty
}

/// <summary>
/// A data transformer that applies regex transformations to string values in a column.
/// Supports two modes: Find &amp; Replace (when replacement is provided) and Match &amp; Extract
/// (when replacement is null, extracts first capture group).
/// </summary>
public class RegexTransformer : DataTransformerBase
{
    private readonly string _columnName;
    private readonly string _pattern;
    private readonly string? _replacement;
    private readonly bool _ignoreCase;
    private readonly NonMatchBehavior _nonMatchBehavior;

    private Regex? _regex;
    private int _columnOrdinal = -1;

    /// <inheritdoc />
    public override string TransformerName => $"Regex:{_columnName}";

    /// <summary>
    /// Creates a new RegexTransformer.
    /// </summary>
    /// <param name="columnName">The column to transform.</param>
    /// <param name="pattern">The regex pattern.</param>
    /// <param name="replacement">Replacement string for Find &amp; Replace mode, or null for Match &amp; Extract mode.</param>
    /// <param name="ignoreCase">Whether to use case-insensitive matching.</param>
    /// <param name="nonMatchBehavior">Behavior when pattern does not match.</param>
    public RegexTransformer(
        string columnName,
        string pattern,
        string? replacement = null,
        bool ignoreCase = false,
        NonMatchBehavior nonMatchBehavior = NonMatchBehavior.KeepOriginal)
    {
        ArgumentException.ThrowIfNullOrWhiteSpace(columnName);
        ArgumentException.ThrowIfNullOrWhiteSpace(pattern);

        _columnName = columnName;
        _pattern = pattern;
        _replacement = replacement;
        _ignoreCase = ignoreCase;
        _nonMatchBehavior = nonMatchBehavior;
    }

    /// <inheritdoc />
    protected override void OnInitialize(IDataReader source)
    {
        _columnOrdinal = source.GetOrdinal(_columnName);

        var options = RegexOptions.Compiled;
        if (_ignoreCase)
            options |= RegexOptions.IgnoreCase;

        _regex = new Regex(_pattern, options);
    }

    /// <inheritdoc />
    public override object GetValue(int ordinal, IDataReader source)
    {
        var value = source.GetValue(ordinal);

        // Only transform the target column
        if (ordinal != _columnOrdinal)
            return value;

        // Pass through null/DBNull
        if (value == null || value == DBNull.Value)
            return value;

        var stringValue = value.ToString() ?? string.Empty;

        // Find & Replace mode (replacement is not null)
        if (_replacement != null)
        {
            return _regex!.Replace(stringValue, _replacement);
        }

        // Match & Extract mode (replacement is null)
        var match = _regex!.Match(stringValue);
        if (match.Success && match.Groups.Count > 1)
        {
            return match.Groups[1].Value;
        }

        // No match - apply NonMatchBehavior
        return _nonMatchBehavior switch
        {
            NonMatchBehavior.ReturnNull => DBNull.Value,
            NonMatchBehavior.ReturnEmpty => string.Empty,
            _ => value // KeepOriginal
        };
    }

    /// <inheritdoc />
    public override Type GetFieldType(int ordinal, IDataReader source)
    {
        // Target column always returns string
        if (ordinal == _columnOrdinal)
            return typeof(string);
        return source.GetFieldType(ordinal);
    }
}
```

**Step 4: Run test to verify it passes**

Run: `dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests.FindReplace_RemovesPrefix" -v n`
Expected: PASS

**Step 5: Commit**

```bash
git add NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
feat(datasync): add RegexTransformer with Find & Replace mode

Initial implementation supporting:
- Find & Replace mode with regex pattern and replacement string
- Case-insensitive option
- NonMatchBehavior enum for handling non-matches
EOF
)"
```

---

## Task 3: Add Match & Extract Mode Tests

**Files:**
- Modify: `NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs`

**Step 1: Write the failing test for Match & Extract**

Add to test class:

```csharp
[Fact]
public void MatchExtract_ExtractsFirstCaptureGroup()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "ID_12345" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null); // null = Match & Extract mode

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_KeepOriginal()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.KeepOriginal);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("UNKNOWN", reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_ReturnNull()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.ReturnNull);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(DBNull.Value, reader.GetValue(0));
}

[Fact]
public void MatchExtract_NoMatch_ReturnEmpty()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Code" },
        values: new object[] { "UNKNOWN" });

    var transformer = new RegexTransformer(
        columnName: "Code",
        pattern: @"ID_(\d+)",
        replacement: null,
        nonMatchBehavior: NonMatchBehavior.ReturnEmpty);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(string.Empty, reader.GetValue(0));
}
```

**Step 2: Run tests to verify they pass**

Run: `dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests" -v n`
Expected: All 5 tests PASS

**Step 3: Commit**

```bash
git add NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
test(datasync): add Match & Extract mode tests for RegexTransformer

Tests cover:
- Extracting first capture group
- NonMatchBehavior: KeepOriginal, ReturnNull, ReturnEmpty
EOF
)"
```

---

## Task 4: Add Edge Case Tests

**Files:**
- Modify: `NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs`

**Step 1: Add edge case tests**

Add to test class:

```csharp
[Fact]
public void FindReplace_UseCaptureGroups()
{
    // Arrange - swap two numbers
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "123-456" });

    var transformer = new RegexTransformer(
        columnName: "Value",
        pattern: @"(\d+)-(\d+)",
        replacement: "$2-$1");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("456-123", reader.GetValue(0));
}

[Fact]
public void IgnoreCase_MatchesDifferentCase()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID" },
        values: new object[] { "IIS_12345" });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^iis_", // lowercase pattern
        replacement: "",
        ignoreCase: true);

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
}

[Fact]
public void NullValue_PassesThrough()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID" },
        values: new object[] { DBNull.Value });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^IIS_",
        replacement: "");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal(DBNull.Value, reader.GetValue(0));
}

[Fact]
public void NonTargetColumn_Unchanged()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "BatchID", "OtherColumn" },
        values: new object[] { "IIS_12345", "IIS_Should_Not_Change" });

    var transformer = new RegexTransformer(
        columnName: "BatchID",
        pattern: "^IIS_",
        replacement: "");

    // Act
    var reader = transformer.Transform(source);
    source.Read().Returns(true);
    reader.Read();

    // Assert
    Assert.Equal("12345", reader.GetValue(0));
    Assert.Equal("IIS_Should_Not_Change", reader.GetValue(1));
}

[Fact]
public void InvalidRegex_ThrowsOnTransform()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "test" });

    var transformer = new RegexTransformer(
        columnName: "Value",
        pattern: "[invalid(regex",
        replacement: "");

    // Act & Assert
    var ex = Assert.Throws<RegexParseException>(() => transformer.Transform(source));
    Assert.Contains("Invalid", ex.Message, StringComparison.OrdinalIgnoreCase);
}

[Fact]
public void ColumnNotFound_ThrowsOnTransform()
{
    // Arrange
    var source = CreateMockReader(
        columns: new[] { "Value" },
        values: new object[] { "test" });
    source.GetOrdinal("NonExistent").Returns(_ => throw new IndexOutOfRangeException("Column not found"));

    var transformer = new RegexTransformer(
        columnName: "NonExistent",
        pattern: "test",
        replacement: "");

    // Act & Assert
    Assert.Throws<IndexOutOfRangeException>(() => transformer.Transform(source));
}
```

**Step 2: Run all transformer tests**

Run: `dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~RegexTransformerTests" -v n`
Expected: All 11 tests PASS

**Step 3: Commit**

```bash
git add NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs
git commit -m "$(cat <<'EOF'
test(datasync): add edge case tests for RegexTransformer

Tests cover:
- Capture group substitution in replacement
- Case-insensitive matching
- Null/DBNull passthrough
- Non-target columns unchanged
- Invalid regex pattern handling
- Column not found handling
EOF
)"
```

---

## Task 5: Create RegexTransformerViewModel

**Files:**
- Modify: `NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs`

**Step 1: Add the ViewModel class**

Add after `JdeDateTransformerViewModel` class (before `TransformerFactory`):

```csharp
/// <summary>
/// View model for Regex transformer.
/// </summary>
public class RegexTransformerViewModel : TransformerStepViewModelBase
{
    private string _columnName = string.Empty;
    private string _pattern = string.Empty;
    private string? _replacement = string.Empty;
    private bool _isFindReplaceMode = true;
    private bool _ignoreCase;
    private NonMatchBehavior _nonMatchBehavior = NonMatchBehavior.KeepOriginal;

    // Test feature fields
    private string _testInput = string.Empty;
    private string _testResultValue = string.Empty;
    private string _testResultLabel = string.Empty;
    private string _testResultIcon = string.Empty;
    private string _testResultBackground = string.Empty;
    private bool _hasTestResult;
    private bool _hasTestError;
    private string _testErrorMessage = string.Empty;

    public RegexTransformerViewModel(TransformerModel model, Action onChanged) : base(onChanged)
    {
        _columnName = model.ColumnName ?? string.Empty;
        _pattern = model.Pattern ?? string.Empty;
        _replacement = model.Replacement;
        _isFindReplaceMode = model.Replacement != null;
        _ignoreCase = model.IgnoreCase;
        _nonMatchBehavior = model.NonMatchBehavior;
        TestPatternCommand = new RelayCommand(ExecuteTestPattern);
    }

    public RegexTransformerViewModel(Action onChanged) : base(onChanged)
    {
        TestPatternCommand = new RelayCommand(ExecuteTestPattern);
    }

    public override string TransformerType => "Regex";
    public override string DisplayName => "Regex Transform";
    public override string Icon => "󰑑"; // mdi-regex
    public override string Summary => !string.IsNullOrEmpty(_columnName)
        ? $"{_columnName}: {(_isFindReplaceMode ? "Replace" : "Extract")}"
        : "Configure...";

    /// <summary>Gets or sets the column name to transform.</summary>
    public string ColumnName
    {
        get => _columnName;
        set
        {
            if (SetProperty(ref _columnName, value ?? string.Empty))
            {
                OnPropertyChanged(nameof(Summary));
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the regex pattern.</summary>
    public string Pattern
    {
        get => _pattern;
        set
        {
            if (SetProperty(ref _pattern, value ?? string.Empty))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the replacement string (Find &amp; Replace mode).</summary>
    public string? Replacement
    {
        get => _replacement;
        set
        {
            if (SetProperty(ref _replacement, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets whether Find &amp; Replace mode is active.</summary>
    public bool IsFindReplaceMode
    {
        get => _isFindReplaceMode;
        set
        {
            if (SetProperty(ref _isFindReplaceMode, value))
            {
                OnPropertyChanged(nameof(IsMatchExtractMode));
                OnPropertyChanged(nameof(PatternHelpText));
                OnPropertyChanged(nameof(Summary));
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets whether Match &amp; Extract mode is active.</summary>
    public bool IsMatchExtractMode
    {
        get => !_isFindReplaceMode;
        set => IsFindReplaceMode = !value;
    }

    /// <summary>Gets the help text for the pattern field based on current mode.</summary>
    public string PatternHelpText => _isFindReplaceMode
        ? "Pattern to search for in the column value"
        : "Pattern with capture group - first group (parentheses) will be extracted";

    /// <summary>Gets or sets whether matching is case-insensitive.</summary>
    public bool IgnoreCase
    {
        get => _ignoreCase;
        set
        {
            if (SetProperty(ref _ignoreCase, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    /// <summary>Gets or sets the behavior when pattern doesn't match.</summary>
    public NonMatchBehavior NonMatchBehavior
    {
        get => _nonMatchBehavior;
        set
        {
            if (SetProperty(ref _nonMatchBehavior, value))
            {
                ClearTestResult();
                NotifyChanged();
            }
        }
    }

    // Test feature properties
    public string TestInput
    {
        get => _testInput;
        set => SetProperty(ref _testInput, value ?? string.Empty);
    }

    public string TestResultValue
    {
        get => _testResultValue;
        set => SetProperty(ref _testResultValue, value);
    }

    public string TestResultLabel
    {
        get => _testResultLabel;
        set => SetProperty(ref _testResultLabel, value);
    }

    public string TestResultIcon
    {
        get => _testResultIcon;
        set => SetProperty(ref _testResultIcon, value);
    }

    public string TestResultBackground
    {
        get => _testResultBackground;
        set => SetProperty(ref _testResultBackground, value);
    }

    public bool HasTestResult
    {
        get => _hasTestResult;
        set => SetProperty(ref _hasTestResult, value);
    }

    public bool HasTestError
    {
        get => _hasTestError;
        set => SetProperty(ref _hasTestError, value);
    }

    public string TestErrorMessage
    {
        get => _testErrorMessage;
        set => SetProperty(ref _testErrorMessage, value);
    }

    public ICommand TestPatternCommand { get; }

    private void ExecuteTestPattern()
    {
        ClearTestResult();

        if (string.IsNullOrEmpty(_pattern))
        {
            HasTestError = true;
            TestErrorMessage = "Pattern is required";
            return;
        }

        try
        {
            var options = _ignoreCase ? RegexOptions.IgnoreCase : RegexOptions.None;
            var regex = new Regex(_pattern, options);

            string result;
            bool matched;

            if (_isFindReplaceMode)
            {
                result = regex.Replace(_testInput, _replacement ?? string.Empty);
                matched = regex.IsMatch(_testInput);
            }
            else
            {
                var match = regex.Match(_testInput);
                if (match.Success && match.Groups.Count > 1)
                {
                    result = match.Groups[1].Value;
                    matched = true;
                }
                else
                {
                    matched = false;
                    result = _nonMatchBehavior switch
                    {
                        NonMatchBehavior.ReturnNull => "(null)",
                        NonMatchBehavior.ReturnEmpty => "(empty)",
                        _ => _testInput
                    };
                }
            }

            HasTestResult = true;
            TestResultValue = result;
            TestResultLabel = matched ? "Output" : "No Match";
            TestResultIcon = matched ? "✓" : "—";
            TestResultBackground = matched ? "#22C55E" : "#F59E0B";
        }
        catch (RegexParseException ex)
        {
            HasTestError = true;
            TestErrorMessage = ex.Message;
        }
    }

    private void ClearTestResult()
    {
        HasTestResult = false;
        HasTestError = false;
        TestResultValue = string.Empty;
        TestResultLabel = string.Empty;
        TestErrorMessage = string.Empty;
    }

    public override TransformerModel ToModel() => new()
    {
        Type = TransformerType,
        ColumnName = _columnName,
        Pattern = _pattern,
        Replacement = _isFindReplaceMode ? _replacement : null,
        IgnoreCase = _ignoreCase,
        NonMatchBehavior = _nonMatchBehavior
    };
}
```

**Step 2: Add using statement**

Add at top of file:
```csharp
using System.Text.RegularExpressions;
using JdeScoping.ConfigManager.Models;
```

**Step 3: Verify it compiles**

Run: `dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj`
Expected: Build succeeded

**Step 4: Commit**

```bash
git add NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add RegexTransformerViewModel

Implements ViewModel for Regex transformer editor with:
- Column, Pattern, Replacement, IgnoreCase, NonMatchBehavior properties
- Mode toggle between Find & Replace and Match & Extract
- Live test/preview functionality with error handling
EOF
)"
```

---

## Task 6: Update TransformerFactory

**Files:**
- Modify: `NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs:284-318`

**Step 1: Update factory switch statements and AvailableTypes**

In `TransformerFactory.Create()` method, add case:
```csharp
"regex" => new RegexTransformerViewModel(model, onChanged),
```

In `TransformerFactory.CreateNew()` method, add case:
```csharp
"regex" => new RegexTransformerViewModel(onChanged),
```

Update `AvailableTypes`:
```csharp
public static IReadOnlyList<string> AvailableTypes => ["ColumnDrop", "ColumnRename", "JdeDate", "Regex"];
```

**Step 2: Verify it compiles**

Run: `dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj`
Expected: Build succeeded

**Step 3: Commit**

```bash
git add NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): register Regex transformer in TransformerFactory

Add Regex to:
- Create() factory method
- CreateNew() factory method
- AvailableTypes list
EOF
)"
```

---

## Task 7: Create RegexEditorView XAML

**Files:**
- Create: `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml`
- Create: `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs`

**Step 1: Create the XAML file**

Create `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml`:

```xml
<UserControl xmlns="https://github.com/avaloniaui"
             xmlns:x="http://schemas.microsoft.com/winfx/2006/xaml"
             xmlns:steps="using:JdeScoping.ConfigManager.ViewModels.PipelineSteps"
             xmlns:models="using:JdeScoping.ConfigManager.Models"
             x:Class="JdeScoping.ConfigManager.Views.Editors.RegexEditorView"
             x:DataType="steps:RegexTransformerViewModel">

    <StackPanel Spacing="16">
        <!-- Header -->
        <StackPanel>
            <TextBlock Text="Regex Transformer"
                       Foreground="#E6EDF5" FontSize="14" FontWeight="SemiBold"/>
            <TextBlock Text="Transform column values using regular expressions"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Column Name -->
        <StackPanel Spacing="4">
            <StackPanel Orientation="Horizontal" Spacing="2">
                <TextBlock Text="Column"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
                <TextBlock Text="*" Foreground="#FF6B6B" FontSize="12"/>
            </StackPanel>
            <TextBox Text="{Binding ColumnName}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="BatchID"/>
            <TextBlock Text="Column to apply the regex transformation"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Mode Selection -->
        <StackPanel Spacing="4">
            <TextBlock Text="Mode" Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
            <StackPanel Orientation="Horizontal" Spacing="12">
                <RadioButton GroupName="RegexMode"
                             IsChecked="{Binding IsFindReplaceMode}"
                             Foreground="#E6EDF5">
                    <TextBlock Text="Find &amp; Replace" FontSize="12"/>
                </RadioButton>
                <RadioButton GroupName="RegexMode"
                             IsChecked="{Binding IsMatchExtractMode}"
                             Foreground="#E6EDF5">
                    <TextBlock Text="Match &amp; Extract" FontSize="12"/>
                </RadioButton>
            </StackPanel>
        </StackPanel>

        <!-- Pattern -->
        <StackPanel Spacing="4">
            <StackPanel Orientation="Horizontal" Spacing="2">
                <TextBlock Text="Pattern"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
                <TextBlock Text="*" Foreground="#FF6B6B" FontSize="12"/>
            </StackPanel>
            <TextBox Text="{Binding Pattern}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="^IIS_"/>
            <TextBlock Text="{Binding PatternHelpText}"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Replacement (only visible in Find & Replace mode) -->
        <StackPanel Spacing="4" IsVisible="{Binding IsFindReplaceMode}">
            <TextBlock Text="Replacement"
                       Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>
            <TextBox Text="{Binding Replacement}"
                     Background="#232A35" Foreground="#E6EDF5"
                     BorderBrush="#3D4550" Height="36"
                     FontFamily="JetBrains Mono"
                     Watermark="(empty to remove)"/>
            <TextBlock Text="Text to replace matches with (use $1, $2 for capture groups)"
                       Foreground="#5C6A7A" FontSize="11"/>
        </StackPanel>

        <!-- Options Row -->
        <StackPanel Spacing="8">
            <TextBlock Text="Options" Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>

            <Grid ColumnDefinitions="*,*" RowDefinitions="Auto">
                <!-- Case Insensitive Toggle -->
                <CheckBox Grid.Column="0"
                          IsChecked="{Binding IgnoreCase}"
                          Foreground="#E6EDF5" FontSize="12">
                    <TextBlock Text="Case Insensitive" FontSize="12"/>
                </CheckBox>

                <!-- Non-Match Behavior -->
                <StackPanel Grid.Column="1" Spacing="4">
                    <TextBlock Text="If No Match"
                               Foreground="#5C6A7A" FontSize="11"/>
                    <ComboBox SelectedItem="{Binding NonMatchBehavior}"
                              Background="#232A35" Foreground="#E6EDF5"
                              BorderBrush="#3D4550" Height="32"
                              FontSize="11" MinWidth="140">
                        <ComboBox.ItemsSource>
                            <x:Array Type="{x:Type models:NonMatchBehavior}">
                                <models:NonMatchBehavior>KeepOriginal</models:NonMatchBehavior>
                                <models:NonMatchBehavior>ReturnNull</models:NonMatchBehavior>
                                <models:NonMatchBehavior>ReturnEmpty</models:NonMatchBehavior>
                            </x:Array>
                        </ComboBox.ItemsSource>
                    </ComboBox>
                </StackPanel>
            </Grid>
        </StackPanel>

        <!-- Test/Preview Section -->
        <Border Background="#151920" BorderBrush="#2D3540" BorderThickness="1"
                CornerRadius="4" Padding="12" Margin="0,8,0,0">
            <StackPanel Spacing="12">
                <TextBlock Text="Test Pattern"
                           Foreground="#9BA8B8" FontSize="12" FontWeight="Medium"/>

                <!-- Test Input Row -->
                <Grid ColumnDefinitions="*,8,Auto">
                    <TextBox Grid.Column="0"
                             Text="{Binding TestInput}"
                             Background="#232A35" Foreground="#E6EDF5"
                             BorderBrush="#3D4550" Height="36"
                             FontFamily="JetBrains Mono"
                             Watermark="IIS_12345"/>
                    <Button Grid.Column="2"
                            Content="Test"
                            Background="#3B82F6" Foreground="#FFFFFF"
                            BorderThickness="0" Height="36"
                            Padding="16,0"
                            Command="{Binding TestPatternCommand}"/>
                </Grid>

                <!-- Result Display -->
                <Border Background="#0D0F12" BorderBrush="#2D3540" BorderThickness="1"
                        CornerRadius="4" Padding="10" MinHeight="44"
                        IsVisible="{Binding HasTestResult}">
                    <Grid ColumnDefinitions="Auto,8,*">
                        <!-- Status Icon -->
                        <Border Grid.Column="0"
                                Width="24" Height="24" CornerRadius="12"
                                Background="{Binding TestResultBackground}"
                                VerticalAlignment="Center">
                            <TextBlock Text="{Binding TestResultIcon}"
                                       Foreground="#FFFFFF"
                                       FontSize="12" FontWeight="Bold"
                                       HorizontalAlignment="Center"
                                       VerticalAlignment="Center"/>
                        </Border>

                        <!-- Result Text -->
                        <StackPanel Grid.Column="2" Spacing="2" VerticalAlignment="Center">
                            <TextBlock Text="{Binding TestResultLabel}"
                                       Foreground="#5C6A7A" FontSize="10"/>
                            <TextBlock Text="{Binding TestResultValue}"
                                       Foreground="#E6EDF5" FontSize="12"
                                       FontFamily="JetBrains Mono"/>
                        </StackPanel>
                    </Grid>
                </Border>

                <!-- Error Display -->
                <Border Background="#2D1F1F" BorderBrush="#5C2626" BorderThickness="1"
                        CornerRadius="4" Padding="10"
                        IsVisible="{Binding HasTestError}">
                    <StackPanel Spacing="2">
                        <TextBlock Text="Invalid Pattern"
                                   Foreground="#FF6B6B" FontSize="11" FontWeight="Medium"/>
                        <TextBlock Text="{Binding TestErrorMessage}"
                                   Foreground="#CC8888" FontSize="11"
                                   TextWrapping="Wrap"/>
                    </StackPanel>
                </Border>
            </StackPanel>
        </Border>

        <!-- Help Info -->
        <Border Background="#151920" BorderBrush="#2D3540" BorderThickness="1"
                CornerRadius="4" Padding="12" Margin="0,4,0,0">
            <StackPanel Spacing="4">
                <TextBlock Text="Pattern Examples" Foreground="#9BA8B8" FontSize="11" FontWeight="Medium"/>
                <TextBlock Text="  ^IIS_        Remove 'IIS_' prefix" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
                <TextBlock Text="  _SUFFIX$     Remove '_SUFFIX' at end" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
                <TextBlock Text="  (\d+)        Extract first number" Foreground="#5C6A7A" FontSize="10" FontFamily="JetBrains Mono"/>
            </StackPanel>
        </Border>
    </StackPanel>
</UserControl>
```

**Step 2: Create the code-behind file**

Create `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs`:

```csharp
using Avalonia.Controls;

namespace JdeScoping.ConfigManager.Views.Editors;

public partial class RegexEditorView : UserControl
{
    public RegexEditorView()
    {
        InitializeComponent();
    }
}
```

**Step 3: Verify it compiles**

Run: `dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj`
Expected: Build succeeded

**Step 4: Commit**

```bash
git add NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs
git commit -m "$(cat <<'EOF'
feat(configmanager): add RegexEditorView Avalonia UI

Implements editor with:
- Column name input
- Mode toggle (Find & Replace / Match & Extract)
- Pattern and Replacement inputs
- Case insensitive checkbox
- NonMatchBehavior dropdown
- Live test/preview section with result display
- Pattern examples help box
EOF
)"
```

---

## Task 8: Register DataTemplate in MainWindow

**Files:**
- Modify: `NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml:58-60`

**Step 1: Add DataTemplate for RegexTransformerViewModel**

Add after line 60 (after JdeDateTransformerViewModel template):

```xml
        <DataTemplate DataType="{x:Type steps:RegexTransformerViewModel}">
            <editors:RegexEditorView/>
        </DataTemplate>
```

**Step 2: Verify it compiles**

Run: `dotnet build NEW/src/Utils/JdeScoping.ConfigManager/JdeScoping.ConfigManager.csproj`
Expected: Build succeeded

**Step 3: Commit**

```bash
git add NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml
git commit -m "$(cat <<'EOF'
feat(configmanager): register RegexEditorView DataTemplate in MainWindow
EOF
)"
```

---

## Task 9: Add ViewModel Unit Tests

**Files:**
- Create: `NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs`

**Step 1: Create test file**

```csharp
using JdeScoping.ConfigManager.Models;
using JdeScoping.ConfigManager.ViewModels.PipelineSteps;

namespace JdeScoping.ConfigManager.Tests.ViewModels;

public class RegexTransformerViewModelTests
{
    [Fact]
    public void Constructor_FromModel_LoadsAllProperties()
    {
        // Arrange
        var model = new TransformerModel
        {
            Type = "Regex",
            ColumnName = "BatchID",
            Pattern = "^IIS_",
            Replacement = "",
            IgnoreCase = true,
            NonMatchBehavior = NonMatchBehavior.ReturnEmpty
        };

        // Act
        var vm = new RegexTransformerViewModel(model, () => { });

        // Assert
        Assert.Equal("BatchID", vm.ColumnName);
        Assert.Equal("^IIS_", vm.Pattern);
        Assert.Equal("", vm.Replacement);
        Assert.True(vm.IsFindReplaceMode);
        Assert.True(vm.IgnoreCase);
        Assert.Equal(NonMatchBehavior.ReturnEmpty, vm.NonMatchBehavior);
    }

    [Fact]
    public void Constructor_FromModel_MatchExtractMode_WhenReplacementNull()
    {
        // Arrange
        var model = new TransformerModel
        {
            Type = "Regex",
            ColumnName = "Code",
            Pattern = @"(\d+)",
            Replacement = null
        };

        // Act
        var vm = new RegexTransformerViewModel(model, () => { });

        // Assert
        Assert.False(vm.IsFindReplaceMode);
        Assert.True(vm.IsMatchExtractMode);
    }

    [Fact]
    public void ToModel_SerializesCorrectly_FindReplaceMode()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "BatchID",
            Pattern = "^IIS_",
            Replacement = "",
            IsFindReplaceMode = true,
            IgnoreCase = true,
            NonMatchBehavior = NonMatchBehavior.KeepOriginal
        };

        // Act
        var model = vm.ToModel();

        // Assert
        Assert.Equal("Regex", model.Type);
        Assert.Equal("BatchID", model.ColumnName);
        Assert.Equal("^IIS_", model.Pattern);
        Assert.Equal("", model.Replacement);
        Assert.True(model.IgnoreCase);
        Assert.Equal(NonMatchBehavior.KeepOriginal, model.NonMatchBehavior);
    }

    [Fact]
    public void ToModel_SerializesCorrectly_MatchExtractMode()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "Code",
            Pattern = @"(\d+)",
            IsFindReplaceMode = false
        };

        // Act
        var model = vm.ToModel();

        // Assert
        Assert.Null(model.Replacement); // null indicates Match & Extract mode
    }

    [Fact]
    public void TestPatternCommand_ValidPattern_ShowsResult()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = "^IIS_",
            Replacement = "",
            IsFindReplaceMode = true,
            TestInput = "IIS_12345"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.True(vm.HasTestResult);
        Assert.False(vm.HasTestError);
        Assert.Equal("12345", vm.TestResultValue);
        Assert.Equal("Output", vm.TestResultLabel);
        Assert.Equal("✓", vm.TestResultIcon);
    }

    [Fact]
    public void TestPatternCommand_InvalidPattern_ShowsError()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = "[invalid(regex",
            Replacement = "",
            TestInput = "test"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.False(vm.HasTestResult);
        Assert.True(vm.HasTestError);
        Assert.NotEmpty(vm.TestErrorMessage);
    }

    [Fact]
    public void TestPatternCommand_MatchExtract_NoMatch_ShowsNonMatchBehavior()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            Pattern = @"(\d+)",
            IsFindReplaceMode = false,
            NonMatchBehavior = NonMatchBehavior.ReturnNull,
            TestInput = "NoNumbers"
        };

        // Act
        vm.TestPatternCommand.Execute(null);

        // Assert
        Assert.True(vm.HasTestResult);
        Assert.Equal("No Match", vm.TestResultLabel);
        Assert.Equal("(null)", vm.TestResultValue);
    }

    [Fact]
    public void ModeSwitch_UpdatesPatternHelpText()
    {
        // Arrange
        var vm = new RegexTransformerViewModel(() => { })
        {
            IsFindReplaceMode = true
        };
        var findReplaceHelp = vm.PatternHelpText;

        // Act
        vm.IsFindReplaceMode = false;
        var matchExtractHelp = vm.PatternHelpText;

        // Assert
        Assert.NotEqual(findReplaceHelp, matchExtractHelp);
        Assert.Contains("capture group", matchExtractHelp, StringComparison.OrdinalIgnoreCase);
    }

    [Fact]
    public void Summary_ShowsColumnAndMode()
    {
        // Arrange & Act
        var vm = new RegexTransformerViewModel(() => { })
        {
            ColumnName = "BatchID",
            IsFindReplaceMode = true
        };

        // Assert
        Assert.Contains("BatchID", vm.Summary);
        Assert.Contains("Replace", vm.Summary);
    }

    [Fact]
    public void PropertyChange_NotifiesChanged()
    {
        // Arrange
        var changedCalled = false;
        var vm = new RegexTransformerViewModel(() => changedCalled = true);

        // Act
        vm.ColumnName = "NewColumn";

        // Assert
        Assert.True(changedCalled);
    }
}
```

**Step 2: Run tests**

Run: `dotnet test NEW/tests/JdeScoping.ConfigManager.Tests/JdeScoping.ConfigManager.Tests.csproj --filter "FullyQualifiedName~RegexTransformerViewModelTests" -v n`
Expected: All 10 tests PASS

**Step 3: Commit**

```bash
git add NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs
git commit -m "$(cat <<'EOF'
test(configmanager): add RegexTransformerViewModel unit tests

Tests cover:
- Loading from model
- Serializing to model
- Test pattern command (success and error cases)
- Mode switching and help text
- Summary display
- Change notification
EOF
)"
```

---

## Task 10: Run Full Test Suite and Final Verification

**Files:** None (verification only)

**Step 1: Run all transformer tests**

Run: `dotnet test NEW/tests/JdeScoping.DataSync.Tests/JdeScoping.DataSync.Tests.csproj --filter "FullyQualifiedName~TransformerTests" -v n`
Expected: All tests PASS

**Step 2: Run all ConfigManager tests**

Run: `dotnet test NEW/tests/JdeScoping.ConfigManager.Tests/JdeScoping.ConfigManager.Tests.csproj -v n`
Expected: All tests PASS

**Step 3: Build entire solution**

Run: `dotnet build NEW/JdeScoping.slnx`
Expected: Build succeeded with no errors

**Step 4: Final commit (if any uncommitted changes)**

```bash
git status
# If clean, no action needed
```

---

## Summary

**Tasks completed:**
1. Added `NonMatchBehavior` enum and regex properties to `TransformerModel`
2. Created `RegexTransformer` class with Find & Replace mode
3. Added Match & Extract mode tests
4. Added edge case tests (capture groups, case-insensitive, null handling)
5. Created `RegexTransformerViewModel` with test functionality
6. Updated `TransformerFactory` to register Regex transformer
7. Created `RegexEditorView` Avalonia UI
8. Registered DataTemplate in `MainWindow.axaml`
9. Added ViewModel unit tests
10. Verified full test suite passes

**Files created:**
- `NEW/src/JdeScoping.DataSync/Etl/Transformers/RegexTransformer.cs`
- `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml`
- `NEW/src/Utils/JdeScoping.ConfigManager/Views/Editors/RegexEditorView.axaml.cs`
- `NEW/tests/JdeScoping.DataSync.Tests/Etl/Transformers/RegexTransformerTests.cs`
- `NEW/tests/JdeScoping.ConfigManager.Tests/ViewModels/RegexTransformerViewModelTests.cs`

**Files modified:**
- `NEW/src/Utils/JdeScoping.ConfigManager/Models/PipelineModel.cs`
- `NEW/src/Utils/JdeScoping.ConfigManager/ViewModels/PipelineSteps/TransformerStepViewModels.cs`
- `NEW/src/Utils/JdeScoping.ConfigManager/Views/MainWindow.axaml`