Files
natsnet/docs/plans/2026-02-27-feature-audit-script-design.md
Joseph Doherty 60dce2dc9a docs: add feature audit script design
Automated PortTracker CLI command (feature audit) using Roslyn syntax
tree analysis to classify 3394 unknown features into verified/stub/n_a/deferred.
2026-02-27 05:09:37 -05:00

5.8 KiB

Feature Audit Script Design

Date: 2026-02-27 Status: Approved

Problem

3394 features in module 8 (server) are marked unknown. The existing plan (2026-02-27-feature-status-audit-plan.md) describes a manual 68-batch process of inspecting .NET source and classifying each feature. This design automates that process.

Solution

A new PortTracker CLI command feature audit that uses Roslyn syntax tree analysis to parse .NET source files, build a method index, and classify all unknown features automatically.

Command Interface

dotnet run --project tools/NatsNet.PortTracker -- feature audit \
  --source dotnet/src/ZB.MOM.NatsNet.Server/ \
  --output reports/audit-results.csv \
  --db porting.db \
  [--module 8] \
  [--execute]
Flag Default Description
--source dotnet/src/ZB.MOM.NatsNet.Server/ .NET source directory to parse
--output reports/audit-results.csv CSV report output path
--db porting.db SQLite database (inherited from root)
--module (all) Restrict to a specific module ID
--execute false Apply DB updates (default: dry-run)

Architecture

Component 1: Source Indexer (Audit/SourceIndexer.cs)

Parses all .cs files under the source directory into Roslyn syntax trees and builds a lookup index.

Process:

  1. Recursively glob **/*.cs (skip obj/, bin/)
  2. Parse each file with CSharpSyntaxTree.ParseText()
  3. Walk syntax trees for ClassDeclarationSyntax and StructDeclarationSyntax
  4. Extract all method, property, and constructor declarations
  5. Build dictionary: Dictionary<(string className, string memberName), List<MethodInfo>>

MethodInfo:

  • FilePath — source file path
  • LineNumber — starting line
  • BodyLineCount — lines in method body (excluding braces)
  • IsStub — body is throw new NotImplementedException(...) or empty
  • IsPartial — body has some logic AND a NotImplementedException throw
  • StatementCount — number of meaningful statements

Partial class handling: Same class name across multiple files produces multiple entries in the index. Lookup checks all of them — a feature is matched if the method exists in ANY file for that class.

Name matching: Case-insensitive comparison for both class and method names. Handles dotnet_class values that contain commas (e.g. ClosedRingBuffer,ClosedClient) by splitting and checking each.

Component 2: Feature Classifier (Audit/FeatureClassifier.cs)

Classifies each feature using the source index. Priority order (first match wins):

1. N/A Lookup Table

Checked first against (go_file, go_method) or dotnet_class patterns:

Pattern Reason
Go logging functions (Noticef, Debugf, Tracef, Warnf, Errorf, Fatalf) .NET uses Microsoft.Extensions.Logging
Go signal handling (HandleSignals, processSignal) .NET uses IHostApplicationLifetime
Go HTTP handler setup (Statz, Varz, Connz, etc.) .NET uses ASP.NET middleware

Table is extensible — add entries as new patterns are identified.

2. Method Not Found -> deferred

  • dotnet_class not found in source index, OR
  • dotnet_method not found within the class

3. Stub Detection -> stub

  • Body is solely throw new NotImplementedException(...) (expression-bodied or block)
  • Body is empty (no statements)
  • Body has logic but also contains NotImplementedException (partial implementation)

4. Verified -> verified

  • Method exists with 1+ meaningful statements that are not NotImplementedException throws

Component 3: Audit Command (Commands/AuditCommand.cs)

Orchestrates the audit:

  1. Query SELECT id, dotnet_class, dotnet_method, go_file, go_method FROM features WHERE status = 'unknown' (optionally filtered by module)
  2. Build source index via SourceIndexer
  3. Classify each feature via FeatureClassifier
  4. Write CSV report
  5. Print console summary
  6. If --execute: update DB in a single transaction per status group

DB Update Strategy

  • Group features by (new_status, notes) tuple
  • One UPDATE features SET status = @s, notes = @n WHERE id IN (...) per group
  • All groups in a single transaction
  • For n_a features: set notes to the reason from the lookup table

Output

CSV Report

id,dotnet_class,dotnet_method,go_file,go_method,old_status,new_status,reason
150,ServiceRespType,String,server/accounts.go,String,unknown,verified,Method found with 3 statements
151,Account,NewAccount,server/accounts.go,NewAccount,unknown,stub,Body is throw NotImplementedException

Console Summary

Feature Status Audit Results
=============================
Source: dotnet/src/ZB.MOM.NatsNet.Server/ (142 files, 4821 methods indexed)
Features audited: 3394

  verified:  NNNN
  stub:      NNNN
  n_a:       NNNN
  deferred:  NNNN

Dry-run mode. Add --execute to apply changes.
Report: reports/audit-results.csv

Dependencies

New NuGet package required:

<PackageReference Include="Microsoft.CodeAnalysis.CSharp" Version="4.13.0" />

Files to Create/Modify

File Action
tools/NatsNet.PortTracker/NatsNet.PortTracker.csproj Add Roslyn package reference
tools/NatsNet.PortTracker/Audit/SourceIndexer.cs New — Roslyn source parsing and indexing
tools/NatsNet.PortTracker/Audit/FeatureClassifier.cs New — classification heuristics
tools/NatsNet.PortTracker/Commands/AuditCommand.cs New — CLI command wiring
tools/NatsNet.PortTracker/Program.cs Add AuditCommand.Create() to root command

Non-Goals

  • No semantic analysis (full compilation) — syntax trees are sufficient
  • No Go source parsing — we only inspect .NET source
  • No unit test reclassification — separate effort
  • No code changes to the server project — classification only