# Porting Tracker Design

A system for tracking the port of NATS server from Go to .NET 10 C#. Consists of an SQLite database, a Go AST analyzer for initial codebase decomposition, a .NET CLI tool for ongoing management, and 7 phase instruction guides.

## Context

- **Source**: `golang/nats-server/` — ~130K LOC across 109 non-test Go files, 85 test files (~217K test LOC)
- **Target**: .NET 10 C# solution following conventions defined in `documentation_rules.md`
- **Dependencies**: 10+ Go external packages (klauspost/compress, nats-io/jwt, nats-io/nkeys, nats-io/nuid, golang.org/x/crypto, etc.)

## Architecture

Two tools + one database:

1. **Go AST Analyzer** (`tools/go-analyzer/`) — Parses Go source using `go/ast`, `go/parser`, `go/token`, and `golang.org/x/tools/go/callgraph` to extract packages, functions, types, call graphs, LOC counts. Populates the SQLite DB. Used during Phases 1-2 only.
2. **.NET PortTracker CLI** (`tools/NatsNet.PortTracker/`) — Manages the DB after initial population: update status, map .NET items, run reports, export markdown. Used during all phases.
3. **SQLite Database** (`porting.db`) — Single source of truth for porting progress.

## Database Schema

### modules

Tracks top-level Go packages/logical groupings.

```sql
CREATE TABLE modules (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    name TEXT NOT NULL,
    description TEXT,
    go_package TEXT,
    go_file TEXT,
    go_line_start INTEGER,
    go_line_count INTEGER,
    status TEXT NOT NULL DEFAULT 'not_started'
        CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')),
    dotnet_project TEXT,
    dotnet_namespace TEXT,
    dotnet_class TEXT,
    notes TEXT,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### features

Tracks methods/functions/features within a module.

```sql
CREATE TABLE features (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    module_id INTEGER NOT NULL REFERENCES modules(id),
    name TEXT NOT NULL,
    description TEXT,
    go_file TEXT,
    go_class TEXT,
    go_method TEXT,
    go_line_number INTEGER,
    go_line_count INTEGER,
    status TEXT NOT NULL DEFAULT 'not_started'
        CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')),
    dotnet_project TEXT,
    dotnet_class TEXT,
    dotnet_method TEXT,
    notes TEXT,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### unit_tests

Tracks test functions.

```sql
CREATE TABLE unit_tests (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    module_id INTEGER NOT NULL REFERENCES modules(id),
    feature_id INTEGER REFERENCES features(id),
    name TEXT NOT NULL,
    description TEXT,
    go_file TEXT,
    go_class TEXT,
    go_method TEXT,
    go_line_number INTEGER,
    go_line_count INTEGER,
    status TEXT NOT NULL DEFAULT 'not_started'
        CHECK (status IN ('not_started', 'stub', 'complete', 'verified', 'n_a')),
    dotnet_project TEXT,
    dotnet_class TEXT,
    dotnet_method TEXT,
    notes TEXT,
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### dependencies

Polymorphic dependency relationships between any tracked items. Used to determine porting order (port dependencies first).

```sql
CREATE TABLE dependencies (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    source_type TEXT NOT NULL CHECK (source_type IN ('module', 'feature', 'unit_test')),
    source_id INTEGER NOT NULL,
    target_type TEXT NOT NULL CHECK (target_type IN ('module', 'feature', 'unit_test')),
    target_id INTEGER NOT NULL,
    dependency_kind TEXT DEFAULT 'calls',
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    UNIQUE (source_type, source_id, target_type, target_id)
);
```

### library_mappings

Maps Go imports to .NET equivalents.

```sql
CREATE TABLE library_mappings (
    id INTEGER PRIMARY KEY AUTOINCREMENT,
    go_import_path TEXT NOT NULL UNIQUE,
    go_library_name TEXT,
    go_usage_description TEXT,
    dotnet_package TEXT,
    dotnet_namespace TEXT,
    dotnet_usage_notes TEXT,
    status TEXT NOT NULL DEFAULT 'not_mapped'
        CHECK (status IN ('not_mapped', 'mapped', 'verified')),
    created_at DATETIME DEFAULT CURRENT_TIMESTAMP,
    updated_at DATETIME DEFAULT CURRENT_TIMESTAMP
);
```

### Indexes

```sql
CREATE INDEX idx_features_module ON features(module_id);
CREATE INDEX idx_features_status ON features(status);
CREATE INDEX idx_unit_tests_module ON unit_tests(module_id);
CREATE INDEX idx_unit_tests_feature ON unit_tests(feature_id);
CREATE INDEX idx_unit_tests_status ON unit_tests(status);
CREATE INDEX idx_deps_source ON dependencies(source_type, source_id);
CREATE INDEX idx_deps_target ON dependencies(target_type, target_id);
CREATE INDEX idx_library_status ON library_mappings(status);
CREATE INDEX idx_modules_status ON modules(status);
```

## Go AST Analyzer

Located at `tools/go-analyzer/`. A Go program that:

1. **Parses all `.go` files** in `golang/nats-server/server/` and subdirectories using `go/ast` and `go/parser`
2. **Extracts per-file**: package name, struct types, functions/methods with receiver types, line numbers, LOC counts
3. **Builds call graph**: using `golang.org/x/tools/go/callgraph/cha` (Class Hierarchy Analysis) for whole-program analysis
4. **Extracts test functions**: parses `*_test.go` files separately, links to tested functions by name convention and call graph
5. **Extracts imports**: all imported packages per file, distinguishing stdlib vs external
6. **Writes to SQLite**: directly populates `modules`, `features`, `unit_tests`, `dependencies`, and partially populates `library_mappings` (go-side fields only)

### Module Grouping Strategy

- Each Go file in `server/` that represents a major subsystem maps to a module (e.g., `jetstream.go` + `jetstream_api.go` + `jetstream_cluster.go` -> "jetstream")
- Subdirectories (`stree/`, `pse/`, `certidp/`, etc.) each become their own module
- Groupings are auto-generated then user-adjustable via the PortTracker CLI

### Usage

```bash
cd tools/go-analyzer
go run . --source ../../golang/nats-server --db ../../porting.db
```

## .NET PortTracker CLI

Located at `tools/NatsNet.PortTracker/`. A .NET 10 console app using `System.CommandLine` and `Microsoft.Data.Sqlite`.

### Command Structure

```
porttracker
├── init                          # Create/reset the DB schema
├── module
│   ├── list [--status <s>]       # List modules with filters
│   ├── show <id>                 # Show module detail + features + deps
│   ├── update <id> --status <s>  # Update status
│   ├── map <id> --project <p> --class <c>  # Map to .NET
│   └── set-na <id> --reason <r>  # Mark as N/A with reason
├── feature
│   ├── list [--module <id>] [--status <s>]
│   ├── show <id>
│   ├── update <id> --status <s>
│   ├── map <id> --project <p> --class <c> --method <m>
│   └── set-na <id> --reason <r>
├── test
│   ├── list [--module <id>] [--status <s>]
│   ├── show <id>
│   ├── update <id> --status <s>
│   └── map <id> --project <p> --class <c> --method <m>
├── library
│   ├── list [--status <s>]
│   ├── map <id> --package <pkg> --namespace <ns> --notes <n>
│   └── suggest                   # Show unmapped libraries
├── dependency
│   ├── show <type> <id>          # Show deps for an item
│   ├── blocked                   # Items with unported dependencies
│   └── ready                     # Items whose deps are all ported
├── report
│   ├── summary                   # Overall progress stats
│   ├── phase <n>                 # Progress for a specific phase
│   ├── export --format md        # Export markdown status report
│   └── blocking                  # Critical path / blocking chains
└── phase
    ├── list                      # Show all phases and status
    └── check <n>                 # Run verification for phase N
```

### Design Decisions

- **Raw SQL over EF Core**: Simpler, fewer dependencies. This is a tools app, not a production service.
- **Status stored as text**: Readable in DB tools and queries.
- **Batch operations**: Commands accept `--all-in-module <id>` for bulk status updates.
- **DB location**: `porting.db` at repo root, overridable with `--db` flag.

## Phase Instructions

Seven markdown guides in `docs/plans/phases/`. Each is a standalone step-by-step document.

### Phase 1: Go Codebase Decomposition

**Objective**: Parse the Go source into the SQLite DB — modules, features, unit tests, and call dependencies.

**Steps**:
1. Run `porttracker init` to create the database
2. Build and run the Go AST analyzer: `cd tools/go-analyzer && go run . --source ../../golang/nats-server --db ../../porting.db`
3. Review auto-generated module groupings: `porttracker module list`
4. Adjust module groupings as needed via manual SQL or future CLI commands
5. Spot-check feature extraction against Go source files
6. Verify test function extraction: `porttracker test list`
7. Review dependency graph: `porttracker dependency show module <id>` for key modules

**Completion criteria**: All Go source files accounted for. All public functions extracted. Test functions linked to features. Call dependencies populated.

### Phase 2: Verification of Captured Items

**Objective**: Ensure nothing was missed and dependencies are accurate.

**Steps**:
1. Run `porttracker report summary` — compare counts against baselines:
   - File count: `find golang/nats-server/server -name "*.go" ! -name "*_test.go" | wc -l`
   - Function count: `grep -r "^func " golang/nats-server/server/ --include="*.go" --exclude="*_test.go" | wc -l`
   - Test count: `grep -r "^func Test" golang/nats-server/server/ --include="*_test.go" | wc -l`
2. Cross-check: every `*_test.go` function appears in `unit_tests`
3. Cross-check: every non-test `.go` file has a corresponding module
4. Check for orphaned nodes: `porttracker dependency blocked` — items with no path to a root
5. Identify and resolve circular dependencies
6. Manual review of the 10 largest modules for completeness

**Completion criteria**: Counts match baselines. No orphaned nodes. No missing files. Dependency graph is valid.

### Phase 3: Library Mapping

**Objective**: Map every Go external dependency to its .NET equivalent.

**Steps**:
1. Run `porttracker library suggest` to list all unmapped Go imports
2. For each Go import, determine the .NET equivalent:
   - **Go stdlib** -> .NET BCL (e.g., `encoding/json` -> `System.Text.Json`, `sync` -> `System.Threading`, `net/http` -> `Microsoft.AspNetCore` or `System.Net.Http`)
   - **External packages** (e.g., `klauspost/compress` -> evaluate `System.IO.Compression` or a NuGet package)
   - **NATS-specific** (e.g., `nats-io/nkeys` -> .NET NKeys library or custom implementation)
3. Record each mapping: `porttracker library map <id> --package <pkg> --namespace <ns> --notes <notes>`
4. Flag libraries with no clear .NET equivalent — these need design decisions

**Completion criteria**: `porttracker library list --status not_mapped` returns empty. All ambiguous cases documented.

### Phase 4: .NET Solution Design

**Objective**: Design the target .NET 10 solution structure and map every Go item to its .NET counterpart.

**Steps**:
1. Define solution structure following .NET conventions:
   - `src/NATS.Server/` — main server library
   - `src/NATS.Server.Host/` — host/entry point
   - Additional projects as needed per module grouping
   - `tests/NATS.Server.Tests/` — unit tests
   - `tests/NATS.Server.IntegrationTests/` — integration tests
2. For each module: `porttracker module map <id> --project <project> --class <class>`
3. For each feature: `porttracker feature map <id> --project <project> --class <class> --method <method>`
4. For each test: `porttracker test map <id> --project <project> --class <class> --method <method>`
5. For items not applicable in .NET (e.g., Go-specific platform code, custom logger replaced by Serilog):
   `porttracker module set-na <id> --reason "Replaced by Serilog"` (or feature/test equivalent)

**Completion criteria**: Every item has a .NET mapping or is set to N/A with a reason.

### Phase 5: Mapping Verification

**Objective**: Verify all Go items are mapped to .NET targets.

**Steps**:
1. Run `porttracker report summary` — confirm 0 unmapped items (excluding N/A)
2. Review all N/A items — each must have a justification:
   `porttracker module list --status n_a`, `porttracker feature list --status n_a`
3. Verify .NET naming follows conventions:
   - PascalCase for classes and methods
   - Proper namespace hierarchy
   - No collisions (two features mapped to the same .NET class+method)
4. Generate full mapping report: `porttracker report export --format md`
5. Review the exported report for completeness

**Completion criteria**: Zero unmapped items. All N/A items justified. Naming conventions validated. No collisions.

### Phase 6: Initial Porting

**Objective**: Port Go code to .NET, working through the dependency graph bottom-up.

**Steps**:
1. Run `porttracker dependency ready` to find items with all dependencies already ported
2. For each ready item:
   a. `porttracker feature update <id> --status stub` — when creating the class/method skeleton
   b. Implement the logic, referencing the Go source at the line numbers stored in the DB
   c. `porttracker feature update <id> --status complete` — when implementation is done
   d. Run the corresponding unit tests for that feature
3. After completing items, re-run `porttracker dependency ready` to find newly unblocked items
4. Periodically run `porttracker report summary` to track overall progress
5. For batch scaffolding: `porttracker feature update --all-in-module <module_id> --status stub`

**DB update workflow**:
```bash
# Starting a feature
porttracker feature update 42 --status stub

# Feature implemented
porttracker feature update 42 --status complete

# Corresponding test implemented
porttracker test update 87 --status complete

# Check what's unblocked now
porttracker dependency ready
```

**Completion criteria**: All non-N/A items at status `complete` or better.

### Phase 7: Verification of Ported Items

**Objective**: Verify all ported code works correctly through targeted testing.

**Steps**:
1. For each module, identify its mapped tests: `porttracker test list --module <id>`
2. Run targeted tests per module using `dotnet test --filter`:
   ```bash
   dotnet test --filter "FullyQualifiedName~NATS.Server.Tests.Protocol"
   ```
3. For each passing module, update status:
   `porttracker module update <id> --status verified`
   `porttracker feature update --all-in-module <id> --status verified`
4. For failures: investigate, fix, and re-run the targeted tests
5. Run integration tests for cross-module behavior
6. Compare behavior: run Go server and .NET server with same workload, verify equivalent behavior
7. Final report: `porttracker report summary` — all items `verified` or `n_a`

**Completion criteria**: All targeted tests pass. All items at `verified` or `n_a`. Behavioral equivalence confirmed for key scenarios.

## Project Layout

```
natsnet/
├── golang/nats-server/              # Existing Go source (reference)
├── tools/
│   ├── go-analyzer/                 # Go AST analyzer (Phases 1-2)
│   │   ├── go.mod
│   │   ├── main.go
│   │   ├── analyzer.go
│   │   ├── grouper.go
│   │   └── sqlite.go
│   └── NatsNet.PortTracker/         # .NET CLI tool (all phases)
│       ├── NatsNet.PortTracker.csproj
│       ├── Program.cs
│       ├── Commands/
│       ├── Data/
│       └── Reporting/
├── docs/
│   └── plans/
│       └── phases/
│           ├── phase-1-decomposition.md
│           ├── phase-2-verification.md
│           ├── phase-3-library-mapping.md
│           ├── phase-4-dotnet-design.md
│           ├── phase-5-mapping-verification.md
│           ├── phase-6-porting.md
│           └── phase-7-porting-verification.md
├── porting.db                       # SQLite database (gitignored)
├── porting-schema.sql               # DDL reference (committed)
└── documentation_rules.md           # Existing
```

### Notes

- `porting.db` is `.gitignore`d (contains local state); `porting-schema.sql` is committed
- `porttracker init` creates the DB from the embedded schema
- Go analyzer writes directly to SQLite — no intermediate format
- Phase docs reference `porttracker` commands by name
- Run the .NET tool from anywhere: `dotnet run --project tools/NatsNet.PortTracker/`
