natsnet/docs/plans/phases/phase-1-decomposition.md

Phase 1: Go Codebase Decomposition

Objective

Parse the Go NATS server source code into a structured SQLite database, extracting modules, features (functions/methods), unit tests, external library imports, and inter-module dependencies. This database becomes the single source of truth that drives all subsequent porting phases.

Prerequisites

Requirement	Version / Notes
Go	1.25+ (required by tools/go-analyzer/go.mod)
.NET SDK	10.0+
SQLite3 CLI	3.x (optional, for manual inspection)
CGO	Must be enabled (CGO_ENABLED=1); the Go analyzer uses github.com/mattn/go-sqlite3
Go source	Cloned at golang/nats-server/ relative to the repo root

Verify prerequisites before starting:

go version          # should print go1.25 or later
dotnet --version    # should print 10.x
sqlite3 --version   # optional, any 3.x
echo $CGO_ENABLED   # should print 1 (or be unset; we set it explicitly below)

Source and Target Locations

Component	Path
Go source code	golang/ (specifically golang/nats-server/)
.NET ported version	dotnet/

Milestone Tracking

This phase corresponds to Milestone 1 in Gitea. When starting this phase, verify the milestone is open. Assign relevant issues to this milestone as work progresses.

Issue Updates

Each completion criterion has a corresponding Gitea issue. Update issues as you work:

• Starting a criterion: Add a comment noting work has begun
• Blocked: Add a comment describing the blocker
• Complete: Close the issue with a comment summarizing the result (e.g., counts, verification output)

Close issues via CLI:

curl -s -X PATCH "https://gitea.dohertylan.com/api/v1/repos/dohertj2/natsnet/issues/{N}" \
  -H "Content-Type: application/json" \
  -H "Authorization: token $GITEA_TOKEN" \
  -d '{"state":"closed"}'

Or close via the Gitea web UI.

Steps

Step 1: Initialize the porting database

Create a fresh SQLite database with the porting tracker schema. This creates porting.db in the repository root with all tables, indexes, and triggers.

dotnet run --project tools/NatsNet.PortTracker -- init --db porting.db --schema porting-schema.sql

Expected output:

Database initialized at porting.db

If the database already exists, this command is idempotent -- it applies CREATE TABLE IF NOT EXISTS statements and will not destroy existing data.

Step 2: Build the Go analyzer

The Go analyzer is a standalone tool that uses go/ast to parse Go source files and writes results directly into the SQLite database.

cd tools/go-analyzer && CGO_ENABLED=1 go build -o go-analyzer . && cd ../..

This produces the binary tools/go-analyzer/go-analyzer. If the build fails, see the Troubleshooting section below.

Step 3: Run the Go analyzer

Point the analyzer at the NATS server source and the porting database:

./tools/go-analyzer/go-analyzer \
  --source golang/nats-server \
  --db porting.db \
  --schema porting-schema.sql

Expected output (counts will vary with the NATS server version):

Analysis complete:
  Modules:       <N>
  Features:      <N>
  Unit Tests:    <N>
  Dependencies:  <N>
  Imports:       <N>

The analyzer does the following:

1. Walks golang/nats-server/server/ for all .go files (skipping configs/ and testdata/ directories).
2. Groups files into logical modules by directory.
3. Parses each non-test file, extracting every func and method as a feature.
4. Parses each _test.go file, extracting Test* and Benchmark* functions.
5. Infers module-level dependencies from cross-package imports.
6. Collects all import paths and classifies them as stdlib or external.
7. Writes modules, features, unit_tests, dependencies, and library_mappings to the database.

Step 4: Review module groupings

List all modules that were created to confirm the grouping makes sense:

dotnet run --project tools/NatsNet.PortTracker -- module list --db porting.db

This shows every module with its ID, name, status, Go package, and line count. Verify that the major areas of the NATS server are represented (e.g., core, jetstream, client, auth, protocol, route, gateway, leafnode, mqtt, websocket, monitoring, logging, errors, subscriptions, tls, events, raft, config, accounts).

Step 5: Spot-check individual modules

Inspect a few modules to verify that features and tests were correctly extracted:

# Check the first module
dotnet run --project tools/NatsNet.PortTracker -- module show 1 --db porting.db

# Check a few more
dotnet run --project tools/NatsNet.PortTracker -- module show 2 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- module show 3 --db porting.db

For each module, confirm that:

• Features list functions and methods from the corresponding Go files.
• Each feature has a go_file, go_method, and go_line_number.
• Tests are listed under the module and have go_file and go_method populated.
• Dependencies point to other valid modules.

Step 6: Verify test extraction

List all extracted tests to confirm test files were parsed:

dotnet run --project tools/NatsNet.PortTracker -- test list --db porting.db

Spot-check a few individual tests for detail:

dotnet run --project tools/NatsNet.PortTracker -- test show 1 --db porting.db

Verify that each test has a module assignment and that the feature_id link is populated where the analyzer could infer the connection from naming conventions (e.g., TestConnect links to a feature named Connect).

Step 7: Review inter-module dependencies

Check the dependency graph for a representative module:

dotnet run --project tools/NatsNet.PortTracker -- dependency show module 1 --db porting.db

Also view the full blocked-items report to see the overall dependency shape:

dotnet run --project tools/NatsNet.PortTracker -- dependency blocked --db porting.db

And check which items have no unported dependencies and are ready to start:

dotnet run --project tools/NatsNet.PortTracker -- dependency ready --db porting.db

Step 8: Review extracted library mappings

List all external Go libraries that were detected:

dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db

All entries should have status not_mapped at this point. Library mapping is handled in Phase 3.

Step 9: Generate a baseline summary report

Create a summary snapshot to use as a reference for Phase 2 verification:

dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db

Optionally export a full markdown report for archival:

dotnet run --project tools/NatsNet.PortTracker -- report export --format md --output docs/reports/phase-1-baseline.md --db porting.db

Completion Criteria

Phase 1 is complete when ALL of the following are true:

• #1 porting.db exists and contains data in all five tables (modules, features, unit_tests, dependencies, library_mappings).
• #2 All Go source files under golang/nats-server/server/ are accounted for (no files silently skipped without a logged warning).
• #3 All public and private functions/methods are extracted as features.
• #4 All Test* and Benchmark* functions are extracted as unit_tests.
• #5 Test-to-feature links are populated where naming conventions allow inference.
• #6 Module-level dependencies are recorded in the dependencies table.
• #7 External import paths are recorded in the library_mappings table.
• #8 phase check 1 shows all checklist items marked [x]:

dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db

Expected:

Phase 1: Analysis & Schema
  Run Go AST analyzer, populate DB schema, map libraries

Phase 1 Checklist:
  [x] Modules populated:       <N>
  [x] Features populated:      <N>
  [x] Unit tests populated:    <N>
  [x] Dependencies mapped:     <N>
  [x] Libraries identified:    <N>
  [ ] All libraries mapped:    0/<N>

Note: The "All libraries mapped" item will be unchecked -- that is expected. Library mapping is the concern of Phase 3.

• Close the Phase 1 milestone in Gitea:

  curl -s -X PATCH "https://gitea.dohertylan.com/api/v1/repos/dohertj2/natsnet/milestones/1" \
    -H "Content-Type: application/json" \
    -H "Authorization: token $GITEA_TOKEN" \
    -d '{"state":"closed"}'
  

  Or close it via the Gitea web UI at https://gitea.dohertylan.com/dohertj2/natsnet/milestone/1

Troubleshooting

CGO_ENABLED not set or build fails with "gcc not found"

The go-sqlite3 driver requires C compilation. Make sure you have a C compiler installed and CGO is enabled:

# macOS -- Xcode command line tools
xcode-select --install

# Then build with explicit CGO
CGO_ENABLED=1 go build -o go-analyzer .

"cannot find package" errors during Go build

The analyzer depends on github.com/mattn/go-sqlite3. Run:

cd tools/go-analyzer
go mod download
go mod verify

Wrong source path

The --source flag must point to the root of the cloned nats-server repository (the directory that contains the server/ subdirectory). If you see "discovering files: no such file or directory", verify:

ls golang/nats-server/server/

Database locked errors

If you run the analyzer while another process has porting.db open, SQLite may report a lock error. Close any other connections (including sqlite3 CLI sessions) and retry. The schema enables WAL mode to reduce lock contention:

PRAGMA journal_mode=WAL;

Analyzer prints warnings but continues

Warnings like "Warning: skipping server/foo.go: " mean an individual file could not be parsed. The analyzer continues with remaining files. Investigate any warnings -- they may indicate a Go version mismatch or syntax not yet supported by the go/ast parser at your Go version.

Empty database after analyzer runs

If the analyzer prints zeros for all counts, verify that:

1. The --source path is correct and contains Go files.
2. The --schema path points to a valid porting-schema.sql.
3. The --db path is writable.

You can inspect the database directly:

sqlite3 porting.db "SELECT COUNT(*) FROM modules;"
sqlite3 porting.db "SELECT COUNT(*) FROM features;"
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests;"