Improve XML documentation coverage across src modules and sync generated analysis artifacts.

This commit is contained in:
Joseph Doherty
2026-03-14 03:56:58 -04:00
parent ba0d65317a
commit 46ead5ea9f
152 changed files with 2821 additions and 11284 deletions

View File

@@ -0,0 +1,75 @@
# dotTrace DTP Parser Design
**Goal:** Build a repository-local tool that starts from a raw dotTrace `.dtp` snapshot family and emits machine-readable JSON call-tree data suitable for LLM-driven hotspot analysis.
**Context**
The target snapshot format is JetBrains dotTrace multi-file storage:
- `snapshot.dtp` is the index/manifest.
- `snapshot.dtp.0000`, `.0001`, and related files hold the storage sections.
- `snapshot.dtp.States` holds UI state and is not sufficient for call-tree analysis.
The internal binary layout is not publicly specified. A direct handwritten decoder would be brittle and expensive to maintain. The machine already has dotTrace installed, and the shipped JetBrains assemblies expose snapshot storage, metadata, and performance call-tree readers. The design therefore uses dotTraces local runtime libraries as the authoritative decoder while still starting from the raw `.dtp` files.
**Architecture**
Two layers:
1. A small .NET helper opens the raw snapshot, reads the performance DFS call-tree and node payload sections, resolves function names through the profiler metadata section, and emits JSON.
2. A Python CLI is the user-facing entrypoint. It validates input, builds or reuses the helper, runs it, and writes JSON to stdout or a file.
This keeps the user workflow Python-first while using the only reliable decoder available for the undocumented snapshot format.
**Output schema**
The JSON should support both direct consumption and downstream summarization:
- `snapshot`: source path, thread count, node count, payload type.
- `thread_roots`: thread root metadata.
- `call_tree`: synthetic root with recursive children.
- `hotspots`: flat top lists for inclusive and exclusive time.
Each node should include:
- `id`: stable offset-based identifier.
- `name`: resolved method or synthetic node name.
- `kind`: `root`, `thread`, `method`, or `special`.
- `inclusive_time`
- `exclusive_time`
- `call_count`
- `thread_name` when relevant
- `children`
**Resolution strategy**
Method names are resolved from the snapshots metadata section:
- Use the snapshots FUID-to-metadata converter.
- Map `FunctionUID` to `FunctionId`.
- Resolve `MetadataId`.
- Read function and class data with `MetadataSectionHelpers`.
Synthetic and special frames fall back to explicit labels instead of opaque numeric values where possible.
**Error handling**
The tool should fail loudly for the cases that matter:
- Missing dotTrace assemblies.
- Unsupported snapshot layout.
- Missing metadata sections.
- Helper build or execution failure.
Errors should name the failing stage so the Python wrapper can surface actionable messages.
**Testing**
Use the checked-in sample snapshot at `snapshots/js-ordered-consume.dtp` for an end-to-end test:
- JSON parses successfully.
- The root contains thread children.
- Hotspot lists are populated.
- At least one non-special method name is resolved.
This is enough to verify the extraction path without freezing the entire output.

View File

@@ -0,0 +1,186 @@
# dotTrace DTP Parser Implementation Plan
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
**Goal:** Add a Python-first tool that reads a raw dotTrace `.dtp` snapshot family and emits JSON call-tree and hotspot data for LLM analysis.
**Architecture:** A small .NET helper uses JetBrains local dotTrace assemblies to decode snapshot storage, performance call-tree nodes, payloads, and metadata. A Python wrapper validates input, builds the helper if needed, runs it, and writes the resulting JSON.
**Tech Stack:** Python 3 standard library, .NET 10 console app, local JetBrains dotTrace assemblies, `unittest`
---
### Task 1: Add the failing end-to-end test
**Files:**
- Create: `tools/tests/test_dtp_parser.py`
**Step 1: Write the failing test**
Write a `unittest` test that runs:
```bash
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --stdout
```
and asserts:
- exit code is `0`
- stdout is valid JSON
- `call_tree.children` is non-empty
- `hotspots.inclusive` is non-empty
- at least one node name is not marked as special
**Step 2: Run test to verify it fails**
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
Expected: FAIL because `tools/dtp_parse.py` does not exist yet.
**Step 3: Commit**
```bash
git add tools/tests/test_dtp_parser.py
git commit -m "test: add dtp parser end-to-end expectation"
```
### Task 2: Implement the .NET snapshot extractor
**Files:**
- Create: `tools/DtpSnapshotExtractor/DtpSnapshotExtractor.csproj`
- Create: `tools/DtpSnapshotExtractor/Program.cs`
**Step 1: Write the minimal implementation**
Implement a console app that:
- accepts snapshot path and optional output path
- opens the snapshot through JetBrains snapshot storage
- constructs performance call-tree and payload readers
- resolves method names via metadata sections
- builds a JSON object with root tree, thread roots, and hotspot lists
- writes JSON to stdout or output file
**Step 2: Run helper directly**
Run:
```bash
dotnet run --project tools/DtpSnapshotExtractor -- snapshots/js-ordered-consume.dtp
```
Expected: JSON is emitted successfully.
**Step 3: Commit**
```bash
git add tools/DtpSnapshotExtractor/DtpSnapshotExtractor.csproj tools/DtpSnapshotExtractor/Program.cs
git commit -m "feat: add dottrace snapshot extractor helper"
```
### Task 3: Implement the Python entrypoint
**Files:**
- Create: `tools/dtp_parse.py`
**Step 1: Write the minimal implementation**
Implement a CLI that:
- accepts snapshot path
- supports `--out` and `--stdout`
- checks that dotTrace assemblies exist in the local install
- runs `dotnet run --project tools/DtpSnapshotExtractor -- <snapshot>`
- forwards JSON output
**Step 2: Run the wrapper**
Run:
```bash
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --stdout
```
Expected: JSON is emitted successfully.
**Step 3: Commit**
```bash
git add tools/dtp_parse.py
git commit -m "feat: add python dtp parsing entrypoint"
```
### Task 4: Make the test pass and tighten output
**Files:**
- Modify: `tools/DtpSnapshotExtractor/Program.cs`
- Modify: `tools/dtp_parse.py`
- Modify: `tools/tests/test_dtp_parser.py`
**Step 1: Run the failing test**
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
Expected: FAIL with an output-schema or execution issue.
**Step 2: Fix the minimal failing behavior**
Adjust:
- special-node labeling
- JSON schema stability
- helper invocation details
- fallback behavior for unresolved metadata
**Step 3: Re-run the test**
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
Expected: PASS
**Step 4: Commit**
```bash
git add tools/DtpSnapshotExtractor/Program.cs tools/dtp_parse.py tools/tests/test_dtp_parser.py
git commit -m "test: verify dtp parser output"
```
### Task 5: Final verification
**Files:**
- Modify: none unless fixes are required
**Step 1: Run end-to-end extraction**
Run:
```bash
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --out /tmp/js-ordered-consume-calltree.json
```
Expected: JSON file is created.
**Step 2: Run test suite**
Run:
```bash
python3 -m unittest tools.tests.test_dtp_parser -v
```
Expected: PASS
**Step 3: Inspect a hotspot sample**
Confirm the JSON contains:
- resolved method names
- inclusive and exclusive hotspot lists
- nested thread call trees
**Step 4: Commit**
```bash
git add docs/plans/2026-03-14-dtp-parser-design.md docs/plans/2026-03-14-dtp-parser.md
git commit -m "docs: add dtp parser design and plan"
```