docs: add feature audit script implementation plan
7 tasks: add Roslyn package, create SourceIndexer, FeatureClassifier, AuditCommand, smoke test, execute audit, cleanup.
This commit is contained in:
Joseph Doherty
813
docs/plans/2026-02-27-feature-audit-script-plan.md
Normal file
813
docs/plans/2026-02-27-feature-audit-script-plan.md
Normal file
@@ -0,0 +1,813 @@
|
||||
# Feature Audit Script Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers-extended-cc:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add a `feature audit` command to the PortTracker CLI that uses Roslyn syntax tree analysis to automatically classify 3394 unknown features into verified/stub/n_a/deferred.
|
||||
|
||||
**Architecture:** Three new files — `SourceIndexer` parses all .cs files and builds a method lookup index, `FeatureClassifier` applies classification heuristics, `AuditCommand` wires the CLI and orchestrates the audit. Direct DB updates via the existing `Database` class.
|
||||
|
||||
**Tech Stack:** `Microsoft.CodeAnalysis.CSharp` (Roslyn) for C# parsing, `Microsoft.Data.Sqlite` (existing), `System.CommandLine` (existing)
|
||||
|
||||
**Design doc:** `docs/plans/2026-02-27-feature-audit-script-design.md`
|
||||
|
||||
---
|
||||
|
||||
## Important Rules (Read Before Every Task)
|
||||
|
||||
1. All new files go under `tools/NatsNet.PortTracker/`
|
||||
2. Follow the existing code style — see `FeatureCommands.cs` and `BatchFilters.cs` for patterns
|
||||
3. Use `System.CommandLine` v3 (preview) APIs — `SetAction`, `parseResult.GetValue()`, etc.
|
||||
4. The `Database` class methods: `Query()`, `Execute()`, `ExecuteScalar<T>()`, `ExecuteInTransaction()`
|
||||
5. Run `dotnet build --project tools/NatsNet.PortTracker` after each file creation to verify compilation
|
||||
|
||||
---
|
||||
|
||||
### Task 0: Add Roslyn NuGet package
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/NatsNet.PortTracker/NatsNet.PortTracker.csproj`
|
||||
|
||||
**Step 1: Add the package reference**
|
||||
|
||||
Add `Microsoft.CodeAnalysis.CSharp` to the csproj:
|
||||
|
||||
```xml
|
||||
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.13.0" />
|
||||
```
|
||||
|
||||
The `<ItemGroup>` should look like:
|
||||
|
||||
```xml
|
||||
<ItemGroup>
|
||||
<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.13.0" />
|
||||
<PackageReference Include="Microsoft.Data.Sqlite" Version="10.0.3" />
|
||||
<PackageReference Include="System.CommandLine" Version="3.0.0-preview.1.26104.118" />
|
||||
</ItemGroup>
|
||||
```
|
||||
|
||||
**Step 2: Restore and build**
|
||||
|
||||
Run: `dotnet build --project tools/NatsNet.PortTracker`
|
||||
Expected: Build succeeded. 0 Error(s).
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/NatsNet.PortTracker/NatsNet.PortTracker.csproj
|
||||
git commit -m "chore: add Roslyn package to PortTracker for feature audit"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Create SourceIndexer — data model and file parsing
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/NatsNet.PortTracker/Audit/SourceIndexer.cs`
|
||||
|
||||
**Step 1: Create the SourceIndexer with MethodInfo record and indexing logic**
|
||||
|
||||
Create `tools/NatsNet.PortTracker/Audit/SourceIndexer.cs`:
|
||||
|
||||
```csharp
|
||||
using Microsoft.CodeAnalysis;
|
||||
using Microsoft.CodeAnalysis.CSharp;
|
||||
using Microsoft.CodeAnalysis.CSharp.Syntax;
|
||||
|
||||
namespace NatsNet.PortTracker.Audit;
|
||||
|
||||
/// <summary>
|
||||
/// Parses .cs files using Roslyn syntax trees and builds a lookup index
|
||||
/// of (className, memberName) -> list of MethodInfo.
|
||||
/// </summary>
|
||||
public sealed class SourceIndexer
|
||||
{
|
||||
public record MethodInfo(
|
||||
string FilePath,
|
||||
int LineNumber,
|
||||
int BodyLineCount,
|
||||
bool IsStub,
|
||||
bool IsPartial,
|
||||
int StatementCount);
|
||||
|
||||
// Key: (className lowercase, memberName lowercase)
|
||||
private readonly Dictionary<(string, string), List<MethodInfo>> _index = new();
|
||||
|
||||
public int FilesIndexed { get; private set; }
|
||||
public int MethodsIndexed { get; private set; }
|
||||
|
||||
/// <summary>
|
||||
/// Recursively parses all .cs files under <paramref name="sourceDir"/>
|
||||
/// (skipping obj/ and bin/) and populates the index.
|
||||
/// </summary>
|
||||
public void IndexDirectory(string sourceDir)
|
||||
{
|
||||
var files = Directory.EnumerateFiles(sourceDir, "*.cs", SearchOption.AllDirectories)
|
||||
.Where(f =>
|
||||
{
|
||||
var rel = Path.GetRelativePath(sourceDir, f);
|
||||
return !rel.Contains($"{Path.DirectorySeparatorChar}obj{Path.DirectorySeparatorChar}")
|
||||
&& !rel.Contains($"{Path.DirectorySeparatorChar}bin{Path.DirectorySeparatorChar}")
|
||||
&& !rel.StartsWith($"obj{Path.DirectorySeparatorChar}")
|
||||
&& !rel.StartsWith($"bin{Path.DirectorySeparatorChar}");
|
||||
});
|
||||
|
||||
foreach (var file in files)
|
||||
{
|
||||
IndexFile(file);
|
||||
FilesIndexed++;
|
||||
}
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Looks up all method declarations for a given class and member name.
|
||||
/// Case-insensitive. Returns empty list if not found.
|
||||
/// </summary>
|
||||
public List<MethodInfo> Lookup(string className, string memberName)
|
||||
{
|
||||
var key = (className.ToLowerInvariant(), memberName.ToLowerInvariant());
|
||||
return _index.TryGetValue(key, out var list) ? list : [];
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Returns true if the class exists anywhere in the index (any member).
|
||||
/// </summary>
|
||||
public bool HasClass(string className)
|
||||
{
|
||||
var lower = className.ToLowerInvariant();
|
||||
return _index.Keys.Any(k => k.Item1 == lower);
|
||||
}
|
||||
|
||||
private void IndexFile(string filePath)
|
||||
{
|
||||
var source = File.ReadAllText(filePath);
|
||||
var tree = CSharpSyntaxTree.ParseText(source, path: filePath);
|
||||
var root = tree.GetCompilationUnitRoot();
|
||||
|
||||
foreach (var typeDecl in root.DescendantNodes().OfType<TypeDeclarationSyntax>())
|
||||
{
|
||||
var className = typeDecl.Identifier.Text.ToLowerInvariant();
|
||||
|
||||
// Methods
|
||||
foreach (var method in typeDecl.Members.OfType<MethodDeclarationSyntax>())
|
||||
{
|
||||
var info = AnalyzeMethod(filePath, method.Body, method.ExpressionBody, method.GetLocation());
|
||||
AddToIndex(className, method.Identifier.Text.ToLowerInvariant(), info);
|
||||
}
|
||||
|
||||
// Properties (get/set are like methods)
|
||||
foreach (var prop in typeDecl.Members.OfType<PropertyDeclarationSyntax>())
|
||||
{
|
||||
var info = AnalyzeProperty(filePath, prop);
|
||||
AddToIndex(className, prop.Identifier.Text.ToLowerInvariant(), info);
|
||||
}
|
||||
|
||||
// Constructors — index as class name
|
||||
foreach (var ctor in typeDecl.Members.OfType<ConstructorDeclarationSyntax>())
|
||||
{
|
||||
var info = AnalyzeMethod(filePath, ctor.Body, ctor.ExpressionBody, ctor.GetLocation());
|
||||
AddToIndex(className, ctor.Identifier.Text.ToLowerInvariant(), info);
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
private MethodInfo AnalyzeMethod(string filePath, BlockSyntax? body, ArrowExpressionClauseSyntax? expressionBody, Location location)
|
||||
{
|
||||
var lineSpan = location.GetLineSpan();
|
||||
var lineNumber = lineSpan.StartLinePosition.Line + 1;
|
||||
|
||||
if (expressionBody is not null)
|
||||
{
|
||||
// Expression-bodied: => expr;
|
||||
var isStub = IsNotImplementedExpression(expressionBody.Expression);
|
||||
return new MethodInfo(filePath, lineNumber, 1, IsStub: isStub, IsPartial: false, StatementCount: isStub ? 0 : 1);
|
||||
}
|
||||
|
||||
if (body is null || body.Statements.Count == 0)
|
||||
{
|
||||
// No body or empty body
|
||||
return new MethodInfo(filePath, lineNumber, 0, IsStub: true, IsPartial: false, StatementCount: 0);
|
||||
}
|
||||
|
||||
var bodyLines = body.GetLocation().GetLineSpan();
|
||||
var bodyLineCount = bodyLines.EndLinePosition.Line - bodyLines.StartLinePosition.Line - 1; // exclude braces
|
||||
|
||||
var statements = body.Statements;
|
||||
var hasNotImplemented = statements.Any(s => IsNotImplementedStatement(s));
|
||||
var meaningfulCount = statements.Count(s => !IsNotImplementedStatement(s));
|
||||
|
||||
// Pure stub: single throw NotImplementedException
|
||||
if (statements.Count == 1 && hasNotImplemented)
|
||||
return new MethodInfo(filePath, lineNumber, bodyLineCount, IsStub: true, IsPartial: false, StatementCount: 0);
|
||||
|
||||
// Partial: has some logic AND a NotImplementedException
|
||||
if (hasNotImplemented && meaningfulCount > 0)
|
||||
return new MethodInfo(filePath, lineNumber, bodyLineCount, IsStub: false, IsPartial: true, StatementCount: meaningfulCount);
|
||||
|
||||
// Real logic
|
||||
return new MethodInfo(filePath, lineNumber, bodyLineCount, IsStub: false, IsPartial: false, StatementCount: meaningfulCount);
|
||||
}
|
||||
|
||||
private MethodInfo AnalyzeProperty(string filePath, PropertyDeclarationSyntax prop)
|
||||
{
|
||||
var lineSpan = prop.GetLocation().GetLineSpan();
|
||||
var lineNumber = lineSpan.StartLinePosition.Line + 1;
|
||||
|
||||
// Expression-bodied property: int Foo => expr;
|
||||
if (prop.ExpressionBody is not null)
|
||||
{
|
||||
var isStub = IsNotImplementedExpression(prop.ExpressionBody.Expression);
|
||||
return new MethodInfo(filePath, lineNumber, 1, IsStub: isStub, IsPartial: false, StatementCount: isStub ? 0 : 1);
|
||||
}
|
||||
|
||||
// Auto-property: int Foo { get; set; } — this is valid, not a stub
|
||||
if (prop.AccessorList is not null && prop.AccessorList.Accessors.All(a => a.Body is null && a.ExpressionBody is null))
|
||||
return new MethodInfo(filePath, lineNumber, 0, IsStub: false, IsPartial: false, StatementCount: 1);
|
||||
|
||||
// Property with accessor bodies — check if any are stubs
|
||||
if (prop.AccessorList is not null)
|
||||
{
|
||||
var hasStub = prop.AccessorList.Accessors.Any(a =>
|
||||
(a.ExpressionBody is not null && IsNotImplementedExpression(a.ExpressionBody.Expression)) ||
|
||||
(a.Body is not null && a.Body.Statements.Count == 1 && IsNotImplementedStatement(a.Body.Statements[0])));
|
||||
return new MethodInfo(filePath, lineNumber, 0, IsStub: hasStub, IsPartial: false, StatementCount: hasStub ? 0 : 1);
|
||||
}
|
||||
|
||||
return new MethodInfo(filePath, lineNumber, 0, IsStub: false, IsPartial: false, StatementCount: 1);
|
||||
}
|
||||
|
||||
private static bool IsNotImplementedExpression(ExpressionSyntax expr)
|
||||
{
|
||||
// throw new NotImplementedException(...)
|
||||
if (expr is ThrowExpressionSyntax throwExpr)
|
||||
return throwExpr.Expression is ObjectCreationExpressionSyntax oc
|
||||
&& oc.Type.ToString().Contains("NotImplementedException");
|
||||
// new NotImplementedException() — shouldn't normally be standalone but handle it
|
||||
return expr is ObjectCreationExpressionSyntax oc2
|
||||
&& oc2.Type.ToString().Contains("NotImplementedException");
|
||||
}
|
||||
|
||||
private static bool IsNotImplementedStatement(StatementSyntax stmt)
|
||||
{
|
||||
// throw new NotImplementedException(...);
|
||||
if (stmt is ThrowStatementSyntax throwStmt && throwStmt.Expression is not null)
|
||||
return throwStmt.Expression is ObjectCreationExpressionSyntax oc
|
||||
&& oc.Type.ToString().Contains("NotImplementedException");
|
||||
// Expression statement containing throw expression
|
||||
if (stmt is ExpressionStatementSyntax exprStmt)
|
||||
return IsNotImplementedExpression(exprStmt.Expression);
|
||||
return false;
|
||||
}
|
||||
|
||||
private void AddToIndex(string className, string memberName, MethodInfo info)
|
||||
{
|
||||
var key = (className, memberName);
|
||||
if (!_index.TryGetValue(key, out var list))
|
||||
{
|
||||
list = [];
|
||||
_index[key] = list;
|
||||
}
|
||||
list.Add(info);
|
||||
MethodsIndexed++;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Build to verify compilation**
|
||||
|
||||
Run: `dotnet build --project tools/NatsNet.PortTracker`
|
||||
Expected: Build succeeded. 0 Error(s).
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/NatsNet.PortTracker/Audit/SourceIndexer.cs
|
||||
git commit -m "feat: add SourceIndexer — Roslyn-based .NET source parser for audit"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 2: Create FeatureClassifier — classification heuristics
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/NatsNet.PortTracker/Audit/FeatureClassifier.cs`
|
||||
|
||||
**Step 1: Create the FeatureClassifier with n_a lookup and heuristics**
|
||||
|
||||
Create `tools/NatsNet.PortTracker/Audit/FeatureClassifier.cs`:
|
||||
|
||||
```csharp
|
||||
namespace NatsNet.PortTracker.Audit;
|
||||
|
||||
/// <summary>
|
||||
/// Classifies features by inspecting the SourceIndexer for their .NET implementation status.
|
||||
/// Priority: n_a lookup → method-not-found → stub detection → verified.
|
||||
/// </summary>
|
||||
public sealed class FeatureClassifier
|
||||
{
|
||||
public record ClassificationResult(string Status, string Reason);
|
||||
|
||||
public record FeatureRecord(
|
||||
long Id,
|
||||
string DotnetClass,
|
||||
string DotnetMethod,
|
||||
string GoFile,
|
||||
string GoMethod);
|
||||
|
||||
private readonly SourceIndexer _indexer;
|
||||
|
||||
// N/A lookup: (goMethod pattern) -> reason
|
||||
// Checked case-insensitively against go_method
|
||||
private static readonly Dictionary<string, string> NaByGoMethod = new(StringComparer.OrdinalIgnoreCase)
|
||||
{
|
||||
["Noticef"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
["Debugf"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
["Tracef"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
["Warnf"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
["Errorf"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
["Fatalf"] = ".NET uses Microsoft.Extensions.Logging",
|
||||
};
|
||||
|
||||
// N/A lookup: go_file + go_method patterns
|
||||
private static readonly List<(Func<FeatureRecord, bool> Match, string Reason)> NaPatterns =
|
||||
[
|
||||
// Signal handling — .NET uses IHostApplicationLifetime
|
||||
(f => f.GoMethod.Equals("handleSignals", StringComparison.OrdinalIgnoreCase), ".NET uses IHostApplicationLifetime"),
|
||||
(f => f.GoMethod.Equals("processSignal", StringComparison.OrdinalIgnoreCase), ".NET uses IHostApplicationLifetime"),
|
||||
];
|
||||
|
||||
public FeatureClassifier(SourceIndexer indexer)
|
||||
{
|
||||
_indexer = indexer;
|
||||
}
|
||||
|
||||
/// <summary>
|
||||
/// Classify a single feature. Returns status and reason.
|
||||
/// </summary>
|
||||
public ClassificationResult Classify(FeatureRecord feature)
|
||||
{
|
||||
// 1. N/A lookup — check go_method against known patterns
|
||||
if (NaByGoMethod.TryGetValue(feature.GoMethod, out var naReason))
|
||||
return new ClassificationResult("n_a", naReason);
|
||||
|
||||
foreach (var (match, reason) in NaPatterns)
|
||||
{
|
||||
if (match(feature))
|
||||
return new ClassificationResult("n_a", reason);
|
||||
}
|
||||
|
||||
// 2. Handle comma-separated dotnet_class (e.g. "ClosedRingBuffer,ClosedClient")
|
||||
var classNames = feature.DotnetClass.Split(',', StringSplitOptions.RemoveEmptyEntries | StringSplitOptions.TrimEntries);
|
||||
var methodName = feature.DotnetMethod;
|
||||
|
||||
// Try each class name
|
||||
foreach (var className in classNames)
|
||||
{
|
||||
var methods = _indexer.Lookup(className, methodName);
|
||||
if (methods.Count > 0)
|
||||
{
|
||||
// Found the method — classify based on body analysis
|
||||
// Use the "best" match: prefer non-stub over stub
|
||||
var best = methods.OrderByDescending(m => m.StatementCount).First();
|
||||
|
||||
if (best.IsStub)
|
||||
return new ClassificationResult("stub", $"Body is throw NotImplementedException at {Path.GetFileName(best.FilePath)}:{best.LineNumber}");
|
||||
|
||||
if (best.IsPartial)
|
||||
return new ClassificationResult("stub", $"Partial implementation with NotImplementedException at {Path.GetFileName(best.FilePath)}:{best.LineNumber}");
|
||||
|
||||
return new ClassificationResult("verified", $"Method found with {best.StatementCount} statement(s) at {Path.GetFileName(best.FilePath)}:{best.LineNumber}");
|
||||
}
|
||||
}
|
||||
|
||||
// 3. Method not found — check if any class exists
|
||||
var anyClassFound = classNames.Any(c => _indexer.HasClass(c));
|
||||
if (anyClassFound)
|
||||
return new ClassificationResult("deferred", "Class exists but method not found");
|
||||
|
||||
return new ClassificationResult("deferred", "Class not found in .NET source");
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Build to verify compilation**
|
||||
|
||||
Run: `dotnet build --project tools/NatsNet.PortTracker`
|
||||
Expected: Build succeeded. 0 Error(s).
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/NatsNet.PortTracker/Audit/FeatureClassifier.cs
|
||||
git commit -m "feat: add FeatureClassifier — heuristic-based feature classification"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 3: Create AuditCommand — CLI wiring and orchestration
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/NatsNet.PortTracker/Commands/AuditCommand.cs`
|
||||
- Modify: `tools/NatsNet.PortTracker/Program.cs:36` — add `AuditCommand` to root command
|
||||
|
||||
**Step 1: Create the AuditCommand**
|
||||
|
||||
Create `tools/NatsNet.PortTracker/Commands/AuditCommand.cs`:
|
||||
|
||||
```csharp
|
||||
using System.CommandLine;
|
||||
using System.Text;
|
||||
using NatsNet.PortTracker.Audit;
|
||||
using NatsNet.PortTracker.Data;
|
||||
|
||||
namespace NatsNet.PortTracker.Commands;
|
||||
|
||||
public static class AuditCommand
|
||||
{
|
||||
public static Command Create(Option<string> dbOption)
|
||||
{
|
||||
var sourceOpt = new Option<string>("--source")
|
||||
{
|
||||
Description = "Path to the .NET source directory",
|
||||
DefaultValueFactory = _ => Path.Combine(Directory.GetCurrentDirectory(), "dotnet", "src", "ZB.MOM.NatsNet.Server")
|
||||
};
|
||||
|
||||
var outputOpt = new Option<string>("--output")
|
||||
{
|
||||
Description = "CSV report output path",
|
||||
DefaultValueFactory = _ => Path.Combine(Directory.GetCurrentDirectory(), "reports", "audit-results.csv")
|
||||
};
|
||||
|
||||
var moduleOpt = new Option<int?>("--module")
|
||||
{
|
||||
Description = "Restrict to a specific module ID"
|
||||
};
|
||||
|
||||
var executeOpt = new Option<bool>("--execute")
|
||||
{
|
||||
Description = "Apply DB updates (default: dry-run preview)",
|
||||
DefaultValueFactory = _ => false
|
||||
};
|
||||
|
||||
var cmd = new Command("audit", "Classify unknown features by inspecting .NET source code");
|
||||
cmd.Add(sourceOpt);
|
||||
cmd.Add(outputOpt);
|
||||
cmd.Add(moduleOpt);
|
||||
cmd.Add(executeOpt);
|
||||
|
||||
cmd.SetAction(parseResult =>
|
||||
{
|
||||
var dbPath = parseResult.GetValue(dbOption)!;
|
||||
var sourcePath = parseResult.GetValue(sourceOpt)!;
|
||||
var outputPath = parseResult.GetValue(outputOpt)!;
|
||||
var moduleId = parseResult.GetValue(moduleOpt);
|
||||
var execute = parseResult.GetValue(executeOpt);
|
||||
|
||||
RunAudit(dbPath, sourcePath, outputPath, moduleId, execute);
|
||||
});
|
||||
|
||||
return cmd;
|
||||
}
|
||||
|
||||
private static void RunAudit(string dbPath, string sourcePath, string outputPath, int? moduleId, bool execute)
|
||||
{
|
||||
// Validate source directory
|
||||
if (!Directory.Exists(sourcePath))
|
||||
{
|
||||
Console.WriteLine($"Error: source directory not found: {sourcePath}");
|
||||
return;
|
||||
}
|
||||
|
||||
// 1. Build source index
|
||||
Console.WriteLine($"Parsing .NET source files in {sourcePath}...");
|
||||
var indexer = new SourceIndexer();
|
||||
indexer.IndexDirectory(sourcePath);
|
||||
Console.WriteLine($"Indexed {indexer.FilesIndexed} files, {indexer.MethodsIndexed} methods/properties.");
|
||||
|
||||
// 2. Query unknown features
|
||||
using var db = new Database(dbPath);
|
||||
var sql = "SELECT id, dotnet_class, dotnet_method, go_file, go_method FROM features WHERE status = 'unknown'";
|
||||
var parameters = new List<(string, object?)>();
|
||||
if (moduleId is not null)
|
||||
{
|
||||
sql += " AND module_id = @module";
|
||||
parameters.Add(("@module", moduleId));
|
||||
}
|
||||
sql += " ORDER BY id";
|
||||
|
||||
var rows = db.Query(sql, parameters.ToArray());
|
||||
if (rows.Count == 0)
|
||||
{
|
||||
Console.WriteLine("No unknown features found.");
|
||||
return;
|
||||
}
|
||||
Console.WriteLine($"Found {rows.Count} unknown features to classify.\n");
|
||||
|
||||
// 3. Classify each feature
|
||||
var classifier = new FeatureClassifier(indexer);
|
||||
var results = new List<(FeatureClassifier.FeatureRecord Feature, FeatureClassifier.ClassificationResult Result)>();
|
||||
|
||||
foreach (var row in rows)
|
||||
{
|
||||
var feature = new FeatureClassifier.FeatureRecord(
|
||||
Id: Convert.ToInt64(row["id"]),
|
||||
DotnetClass: row["dotnet_class"]?.ToString() ?? "",
|
||||
DotnetMethod: row["dotnet_method"]?.ToString() ?? "",
|
||||
GoFile: row["go_file"]?.ToString() ?? "",
|
||||
GoMethod: row["go_method"]?.ToString() ?? "");
|
||||
|
||||
var result = classifier.Classify(feature);
|
||||
results.Add((feature, result));
|
||||
}
|
||||
|
||||
// 4. Write CSV report
|
||||
WriteCsvReport(outputPath, results);
|
||||
|
||||
// 5. Print console summary
|
||||
var grouped = results.GroupBy(r => r.Result.Status)
|
||||
.ToDictionary(g => g.Key, g => g.Count());
|
||||
|
||||
Console.WriteLine("Feature Status Audit Results");
|
||||
Console.WriteLine("=============================");
|
||||
Console.WriteLine($"Source: {sourcePath} ({indexer.FilesIndexed} files, {indexer.MethodsIndexed} methods indexed)");
|
||||
Console.WriteLine($"Features audited: {results.Count}");
|
||||
Console.WriteLine();
|
||||
Console.WriteLine($" verified: {grouped.GetValueOrDefault("verified", 0)}");
|
||||
Console.WriteLine($" stub: {grouped.GetValueOrDefault("stub", 0)}");
|
||||
Console.WriteLine($" n_a: {grouped.GetValueOrDefault("n_a", 0)}");
|
||||
Console.WriteLine($" deferred: {grouped.GetValueOrDefault("deferred", 0)}");
|
||||
Console.WriteLine();
|
||||
|
||||
if (!execute)
|
||||
{
|
||||
Console.WriteLine("Dry-run mode. Add --execute to apply changes.");
|
||||
Console.WriteLine($"Report: {outputPath}");
|
||||
return;
|
||||
}
|
||||
|
||||
// 6. Apply DB updates
|
||||
ApplyUpdates(db, results);
|
||||
Console.WriteLine($"Report: {outputPath}");
|
||||
}
|
||||
|
||||
private static void WriteCsvReport(
|
||||
string outputPath,
|
||||
List<(FeatureClassifier.FeatureRecord Feature, FeatureClassifier.ClassificationResult Result)> results)
|
||||
{
|
||||
// Ensure directory exists
|
||||
var dir = Path.GetDirectoryName(outputPath);
|
||||
if (!string.IsNullOrEmpty(dir))
|
||||
Directory.CreateDirectory(dir);
|
||||
|
||||
var sb = new StringBuilder();
|
||||
sb.AppendLine("id,dotnet_class,dotnet_method,go_file,go_method,old_status,new_status,reason");
|
||||
foreach (var (feature, result) in results)
|
||||
{
|
||||
sb.AppendLine($"{feature.Id},{CsvEscape(feature.DotnetClass)},{CsvEscape(feature.DotnetMethod)},{CsvEscape(feature.GoFile)},{CsvEscape(feature.GoMethod)},unknown,{result.Status},{CsvEscape(result.Reason)}");
|
||||
}
|
||||
File.WriteAllText(outputPath, sb.ToString());
|
||||
}
|
||||
|
||||
private static void ApplyUpdates(
|
||||
Database db,
|
||||
List<(FeatureClassifier.FeatureRecord Feature, FeatureClassifier.ClassificationResult Result)> results)
|
||||
{
|
||||
// Group by (status, notes) for efficient batch updates
|
||||
var groups = results
|
||||
.GroupBy(r => (r.Result.Status, Notes: r.Result.Status == "n_a" ? r.Result.Reason : (string?)null))
|
||||
.ToList();
|
||||
|
||||
var totalUpdated = 0;
|
||||
using var transaction = db.Connection.BeginTransaction();
|
||||
try
|
||||
{
|
||||
foreach (var group in groups)
|
||||
{
|
||||
var ids = group.Select(r => r.Feature.Id).ToList();
|
||||
var status = group.Key.Status;
|
||||
var notes = group.Key.Notes;
|
||||
|
||||
// Build parameterized IN clause
|
||||
var placeholders = new List<string>();
|
||||
using var cmd = db.CreateCommand("");
|
||||
for (var i = 0; i < ids.Count; i++)
|
||||
{
|
||||
placeholders.Add($"@id{i}");
|
||||
cmd.Parameters.AddWithValue($"@id{i}", ids[i]);
|
||||
}
|
||||
|
||||
cmd.Parameters.AddWithValue("@status", status);
|
||||
|
||||
if (notes is not null)
|
||||
{
|
||||
cmd.CommandText = $"UPDATE features SET status = @status, notes = @notes WHERE id IN ({string.Join(", ", placeholders)})";
|
||||
cmd.Parameters.AddWithValue("@notes", notes);
|
||||
}
|
||||
else
|
||||
{
|
||||
cmd.CommandText = $"UPDATE features SET status = @status WHERE id IN ({string.Join(", ", placeholders)})";
|
||||
}
|
||||
|
||||
cmd.Transaction = transaction;
|
||||
var affected = cmd.ExecuteNonQuery();
|
||||
totalUpdated += affected;
|
||||
}
|
||||
|
||||
transaction.Commit();
|
||||
Console.WriteLine($"Updated {totalUpdated} features.");
|
||||
}
|
||||
catch
|
||||
{
|
||||
transaction.Rollback();
|
||||
Console.WriteLine("Error: transaction rolled back.");
|
||||
throw;
|
||||
}
|
||||
}
|
||||
|
||||
private static string CsvEscape(string value)
|
||||
{
|
||||
if (value.Contains(',') || value.Contains('"') || value.Contains('\n'))
|
||||
return $"\"{value.Replace("\"", "\"\"")}\"";
|
||||
return value;
|
||||
}
|
||||
}
|
||||
```
|
||||
|
||||
**Step 2: Wire the command into Program.cs**
|
||||
|
||||
In `tools/NatsNet.PortTracker/Program.cs`, add after the existing command registrations (after line 41, before `var parseResult`):
|
||||
|
||||
Find this line:
|
||||
```csharp
|
||||
rootCommand.Add(PhaseCommands.Create(dbOption, schemaOption));
|
||||
```
|
||||
|
||||
Add immediately after it:
|
||||
```csharp
|
||||
rootCommand.Add(AuditCommand.Create(dbOption));
|
||||
```
|
||||
|
||||
Also add the import — but since the file uses top-level statements and already imports `NatsNet.PortTracker.Commands`, no new using is needed (AuditCommand is in the same namespace).
|
||||
|
||||
**Step 3: Build to verify compilation**
|
||||
|
||||
Run: `dotnet build --project tools/NatsNet.PortTracker`
|
||||
Expected: Build succeeded. 0 Error(s).
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/NatsNet.PortTracker/Commands/AuditCommand.cs tools/NatsNet.PortTracker/Program.cs
|
||||
git commit -m "feat: add audit command — orchestrates feature status classification"
|
||||
```
|
||||
|
||||
---
|
||||
|
||||
### Task 4: Smoke test — dry-run on the real database
|
||||
|
||||
**Files:** None — testing only.
|
||||
|
||||
**Step 1: Run the audit in dry-run mode**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --source dotnet/src/ZB.MOM.NatsNet.Server/ --db porting.db --output reports/audit-results.csv
|
||||
```
|
||||
|
||||
Expected output similar to:
|
||||
```
|
||||
Parsing .NET source files in dotnet/src/ZB.MOM.NatsNet.Server/...
|
||||
Indexed ~92 files, ~NNNN methods/properties.
|
||||
Found 3394 unknown features to classify.
|
||||
|
||||
Feature Status Audit Results
|
||||
=============================
|
||||
Source: dotnet/src/ZB.MOM.NatsNet.Server/ (92 files, NNNN methods indexed)
|
||||
Features audited: 3394
|
||||
|
||||
verified: NNNN
|
||||
stub: NNNN
|
||||
n_a: NNNN
|
||||
deferred: NNNN
|
||||
|
||||
Dry-run mode. Add --execute to apply changes.
|
||||
Report: reports/audit-results.csv
|
||||
```
|
||||
|
||||
**Step 2: Inspect the CSV report**
|
||||
|
||||
```bash
|
||||
head -20 reports/audit-results.csv
|
||||
```
|
||||
|
||||
Verify:
|
||||
- Header row matches: `id,dotnet_class,dotnet_method,go_file,go_method,old_status,new_status,reason`
|
||||
- Each row has a classification and reason
|
||||
- The known n_a features (Noticef, Debugf etc.) show as `n_a`
|
||||
|
||||
**Step 3: Spot-check a few classifications**
|
||||
|
||||
Pick 3-5 features from the CSV and manually verify:
|
||||
- A `verified` feature: check the .NET method has real logic
|
||||
- A `stub` feature: check the .NET method is `throw new NotImplementedException`
|
||||
- A `deferred` feature: check the class/method doesn't exist
|
||||
- An `n_a` feature: check it's a Go logging function
|
||||
|
||||
If any classifications are wrong, fix the heuristics before proceeding.
|
||||
|
||||
**Step 4: Check the counts add up**
|
||||
|
||||
```bash
|
||||
wc -l reports/audit-results.csv
|
||||
```
|
||||
|
||||
Expected: 3395 lines (3394 data rows + 1 header).
|
||||
|
||||
---
|
||||
|
||||
### Task 5: Execute the audit and update the database
|
||||
|
||||
**Files:** None — execution only.
|
||||
|
||||
**Step 1: Back up the database**
|
||||
|
||||
```bash
|
||||
cp porting.db porting.db.pre-audit-backup
|
||||
```
|
||||
|
||||
**Step 2: Run with --execute**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- audit --source dotnet/src/ZB.MOM.NatsNet.Server/ --db porting.db --output reports/audit-results.csv --execute
|
||||
```
|
||||
|
||||
Expected: `Updated 3394 features.`
|
||||
|
||||
**Step 3: Verify zero unknown features remain**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature list --status unknown --db porting.db
|
||||
```
|
||||
|
||||
Expected: `Total: 0 features`
|
||||
|
||||
**Step 4: Verify status breakdown**
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
|
||||
```
|
||||
|
||||
Review the numbers match the dry-run output.
|
||||
|
||||
**Step 5: Generate updated porting report**
|
||||
|
||||
```bash
|
||||
./reports/generate-report.sh
|
||||
```
|
||||
|
||||
**Step 6: Commit everything**
|
||||
|
||||
```bash
|
||||
git add porting.db reports/ tools/NatsNet.PortTracker/
|
||||
git commit -m "feat: run feature status audit — classify 3394 unknown features
|
||||
|
||||
Automated classification using Roslyn syntax tree analysis:
|
||||
verified: NNNN (update with actual numbers)
|
||||
stub: NNNN
|
||||
n_a: NNNN
|
||||
deferred: NNNN"
|
||||
```
|
||||
|
||||
(Update the commit message with the actual numbers from the output.)
|
||||
|
||||
---
|
||||
|
||||
### Task 6: Cleanup — remove backup
|
||||
|
||||
**Files:** None.
|
||||
|
||||
**Step 1: Verify everything is committed and the database is correct**
|
||||
|
||||
```bash
|
||||
git status
|
||||
dotnet run --project tools/NatsNet.PortTracker -- feature list --status unknown --db porting.db
|
||||
```
|
||||
|
||||
Expected: clean working tree, 0 unknown features.
|
||||
|
||||
**Step 2: Remove the pre-audit backup**
|
||||
|
||||
```bash
|
||||
rm porting.db.pre-audit-backup
|
||||
```
|
||||
|
||||
**Step 3: Final summary**
|
||||
|
||||
Print:
|
||||
```
|
||||
Feature Status Audit Complete
|
||||
=============================
|
||||
Total features audited: 3394
|
||||
verified: NNNN
|
||||
stub: NNNN
|
||||
n_a: NNNN
|
||||
deferred: NNNN
|
||||
```
|
||||
@@ -0,0 +1,13 @@
|
||||
{
|
||||
"planPath": "docs/plans/2026-02-27-feature-audit-script-plan.md",
|
||||
"tasks": [
|
||||
{"id": 0, "subject": "Task 0: Add Roslyn NuGet package", "status": "pending"},
|
||||
{"id": 1, "subject": "Task 1: Create SourceIndexer", "status": "pending", "blockedBy": [0]},
|
||||
{"id": 2, "subject": "Task 2: Create FeatureClassifier", "status": "pending", "blockedBy": [1]},
|
||||
{"id": 3, "subject": "Task 3: Create AuditCommand + wire CLI", "status": "pending", "blockedBy": [2]},
|
||||
{"id": 4, "subject": "Task 4: Smoke test dry-run", "status": "pending", "blockedBy": [3]},
|
||||
{"id": 5, "subject": "Task 5: Execute audit and update DB", "status": "pending", "blockedBy": [4]},
|
||||
{"id": 6, "subject": "Task 6: Cleanup and final verification", "status": "pending", "blockedBy": [5]}
|
||||
],
|
||||
"lastUpdated": "2026-02-27T00:00:00Z"
|
||||
}
|
||||
Reference in New Issue
Block a user