natsnet/docs/plans/phases/phase-1-decomposition.md

# Phase 1: Go Codebase Decomposition

## Objective

Parse the Go NATS server source code into a structured SQLite database, extracting
modules, features (functions/methods), unit tests, external library imports, and
inter-module dependencies. This database becomes the single source of truth that
drives all subsequent porting phases.

## Prerequisites

| Requirement | Version / Notes |
|---|---|
| Go | 1.25+ (required by `tools/go-analyzer/go.mod`) |
| .NET SDK | 10.0+ |
| SQLite3 CLI | 3.x (optional, for manual inspection) |
| CGO | Must be enabled (`CGO_ENABLED=1`); the Go analyzer uses `github.com/mattn/go-sqlite3` |
| Go source | Cloned at `golang/nats-server/` relative to the repo root |

Verify prerequisites before starting:

```bash
go version          # should print go1.25 or later
dotnet --version    # should print 10.x
sqlite3 --version   # optional, any 3.x
echo $CGO_ENABLED   # should print 1 (or be unset; we set it explicitly below)
```

## Source and Target Locations

| Component | Path |
|---|---|
| Go source code | `golang/` (specifically `golang/nats-server/`) |
| .NET ported version | `dotnet/` |

## Milestone Tracking

This phase corresponds to **Milestone 1** in [Gitea](https://gitea.dohertylan.com/dohertj2/natsnet/milestone/1). When starting this phase, verify the milestone is open. Assign relevant issues to this milestone as work progresses.

### Issue Updates

Each completion criterion has a corresponding Gitea issue. Update issues as you work:

- **Starting a criterion**: Add a comment noting work has begun
- **Blocked**: Add a comment describing the blocker
- **Complete**: Close the issue with a comment summarizing the result (e.g., counts, verification output)

Close issues via CLI:
```bash
curl -s -X PATCH "https://gitea.dohertylan.com/api/v1/repos/dohertj2/natsnet/issues/{N}" \
  -H "Content-Type: application/json" \
  -H "Authorization: token $GITEA_TOKEN" \
  -d '{"state":"closed"}'
```
Or close via the Gitea web UI.

## Steps

### Step 1: Initialize the porting database

Create a fresh SQLite database with the porting tracker schema. This creates
`porting.db` in the repository root with all tables, indexes, and triggers.

```bash
dotnet run --project tools/NatsNet.PortTracker -- init --db porting.db --schema porting-schema.sql
```

Expected output:

```
Database initialized at porting.db
```

If the database already exists, this command is idempotent -- it applies
`CREATE TABLE IF NOT EXISTS` statements and will not destroy existing data.

### Step 2: Build the Go analyzer

The Go analyzer is a standalone tool that uses `go/ast` to parse Go source files
and writes results directly into the SQLite database.

```bash
cd tools/go-analyzer && CGO_ENABLED=1 go build -o go-analyzer . && cd ../..
```

This produces the binary `tools/go-analyzer/go-analyzer`. If the build fails,
see the Troubleshooting section below.

### Step 3: Run the Go analyzer

Point the analyzer at the NATS server source and the porting database:

```bash
./tools/go-analyzer/go-analyzer \
  --source golang/nats-server \
  --db porting.db \
  --schema porting-schema.sql
```

Expected output (counts will vary with the NATS server version):

```
Analysis complete:
  Modules:       <N>
  Features:      <N>
  Unit Tests:    <N>
  Dependencies:  <N>
  Imports:       <N>
```

The analyzer does the following:
1. Walks `golang/nats-server/server/` for all `.go` files (skipping `configs/`
   and `testdata/` directories).
2. Groups files into logical modules by directory.
3. Parses each non-test file, extracting every `func` and method as a feature.
4. Parses each `_test.go` file, extracting `Test*` and `Benchmark*` functions.
5. Infers module-level dependencies from cross-package imports.
6. Collects all import paths and classifies them as stdlib or external.
7. Writes modules, features, unit_tests, dependencies, and library_mappings to
   the database.

### Step 4: Review module groupings

List all modules that were created to confirm the grouping makes sense:

```bash
dotnet run --project tools/NatsNet.PortTracker -- module list --db porting.db
```

This shows every module with its ID, name, status, Go package, and line count.
Verify that the major areas of the NATS server are represented (e.g., core,
jetstream, client, auth, protocol, route, gateway, leafnode, mqtt, websocket,
monitoring, logging, errors, subscriptions, tls, events, raft, config, accounts).

### Step 5: Spot-check individual modules

Inspect a few modules to verify that features and tests were correctly extracted:

```bash
# Check the first module
dotnet run --project tools/NatsNet.PortTracker -- module show 1 --db porting.db

# Check a few more
dotnet run --project tools/NatsNet.PortTracker -- module show 2 --db porting.db
dotnet run --project tools/NatsNet.PortTracker -- module show 3 --db porting.db
```

For each module, confirm that:
- Features list functions and methods from the corresponding Go files.
- Each feature has a `go_file`, `go_method`, and `go_line_number`.
- Tests are listed under the module and have `go_file` and `go_method` populated.
- Dependencies point to other valid modules.

### Step 6: Verify test extraction

List all extracted tests to confirm test files were parsed:

```bash
dotnet run --project tools/NatsNet.PortTracker -- test list --db porting.db
```

Spot-check a few individual tests for detail:

```bash
dotnet run --project tools/NatsNet.PortTracker -- test show 1 --db porting.db
```

Verify that each test has a module assignment and that the `feature_id` link is
populated where the analyzer could infer the connection from naming conventions
(e.g., `TestConnect` links to a feature named `Connect`).

### Step 7: Review inter-module dependencies

Check the dependency graph for a representative module:

```bash
dotnet run --project tools/NatsNet.PortTracker -- dependency show module 1 --db porting.db
```

Also view the full blocked-items report to see the overall dependency shape:

```bash
dotnet run --project tools/NatsNet.PortTracker -- dependency blocked --db porting.db
```

And check which items have no unported dependencies and are ready to start:

```bash
dotnet run --project tools/NatsNet.PortTracker -- dependency ready --db porting.db
```

### Step 8: Review extracted library mappings

List all external Go libraries that were detected:

```bash
dotnet run --project tools/NatsNet.PortTracker -- library list --db porting.db
```

All entries should have status `not_mapped` at this point. Library mapping is
handled in Phase 3.

### Step 9: Generate a baseline summary report

Create a summary snapshot to use as a reference for Phase 2 verification:

```bash
dotnet run --project tools/NatsNet.PortTracker -- report summary --db porting.db
```

Optionally export a full markdown report for archival:

```bash
dotnet run --project tools/NatsNet.PortTracker -- report export --format md --output docs/reports/phase-1-baseline.md --db porting.db
```

## Completion Criteria

Phase 1 is complete when ALL of the following are true:

- [ ] [#1](https://gitea.dohertylan.com/dohertj2/natsnet/issues/1) `porting.db` exists and contains data in all five tables (modules, features,
      unit_tests, dependencies, library_mappings).
- [ ] [#2](https://gitea.dohertylan.com/dohertj2/natsnet/issues/2) All Go source files under `golang/nats-server/server/` are accounted for
      (no files silently skipped without a logged warning).
- [ ] [#3](https://gitea.dohertylan.com/dohertj2/natsnet/issues/3) All public and private functions/methods are extracted as features.
- [ ] [#4](https://gitea.dohertylan.com/dohertj2/natsnet/issues/4) All `Test*` and `Benchmark*` functions are extracted as unit_tests.
- [ ] [#5](https://gitea.dohertylan.com/dohertj2/natsnet/issues/5) Test-to-feature links are populated where naming conventions allow inference.
- [ ] [#6](https://gitea.dohertylan.com/dohertj2/natsnet/issues/6) Module-level dependencies are recorded in the dependencies table.
- [ ] [#7](https://gitea.dohertylan.com/dohertj2/natsnet/issues/7) External import paths are recorded in the library_mappings table.
- [ ] [#8](https://gitea.dohertylan.com/dohertj2/natsnet/issues/8) `phase check 1` shows all checklist items marked `[x]`:

```bash
dotnet run --project tools/NatsNet.PortTracker -- phase check 1 --db porting.db
```

Expected:

```
Phase 1: Analysis & Schema
  Run Go AST analyzer, populate DB schema, map libraries

Phase 1 Checklist:
  [x] Modules populated:       <N>
  [x] Features populated:      <N>
  [x] Unit tests populated:    <N>
  [x] Dependencies mapped:     <N>
  [x] Libraries identified:    <N>
  [ ] All libraries mapped:    0/<N>
```

Note: The "All libraries mapped" item will be unchecked -- that is expected.
Library mapping is the concern of Phase 3.

- [ ] Close the Phase 1 milestone in Gitea:
  ```bash
  curl -s -X PATCH "https://gitea.dohertylan.com/api/v1/repos/dohertj2/natsnet/milestones/1" \
    -H "Content-Type: application/json" \
    -H "Authorization: token $GITEA_TOKEN" \
    -d '{"state":"closed"}'
  ```
  Or close it via the Gitea web UI at https://gitea.dohertylan.com/dohertj2/natsnet/milestone/1

## Troubleshooting

### CGO_ENABLED not set or build fails with "gcc not found"

The `go-sqlite3` driver requires C compilation. Make sure you have a C compiler
installed and CGO is enabled:

```bash
# macOS -- Xcode command line tools
xcode-select --install

# Then build with explicit CGO
CGO_ENABLED=1 go build -o go-analyzer .
```

### "cannot find package" errors during Go build

The analyzer depends on `github.com/mattn/go-sqlite3`. Run:

```bash
cd tools/go-analyzer
go mod download
go mod verify
```

### Wrong source path

The `--source` flag must point to the root of the cloned nats-server repository
(the directory that contains the `server/` subdirectory). If you see
"discovering files: no such file or directory", verify:

```bash
ls golang/nats-server/server/
```

### Database locked errors

If you run the analyzer while another process has `porting.db` open, SQLite may
report a lock error. Close any other connections (including `sqlite3` CLI
sessions) and retry. The schema enables WAL mode to reduce lock contention:

```sql
PRAGMA journal_mode=WAL;
```

### Analyzer prints warnings but continues

Warnings like "Warning: skipping server/foo.go: <parse error>" mean an individual
file could not be parsed. The analyzer continues with remaining files. Investigate
any warnings -- they may indicate a Go version mismatch or syntax not yet
supported by the `go/ast` parser at your Go version.

### Empty database after analyzer runs

If the analyzer prints zeros for all counts, verify that:
1. The `--source` path is correct and contains Go files.
2. The `--schema` path points to a valid `porting-schema.sql`.
3. The `--db` path is writable.

You can inspect the database directly:

```bash
sqlite3 porting.db "SELECT COUNT(*) FROM modules;"
sqlite3 porting.db "SELECT COUNT(*) FROM features;"
sqlite3 porting.db "SELECT COUNT(*) FROM unit_tests;"
```