Improve XML documentation coverage across src modules and sync generated analysis artifacts.
This commit is contained in:
75
docs/plans/2026-03-14-dtp-parser-design.md
Normal file
75
docs/plans/2026-03-14-dtp-parser-design.md
Normal file
@@ -0,0 +1,75 @@
|
||||
# dotTrace DTP Parser Design
|
||||
|
||||
**Goal:** Build a repository-local tool that starts from a raw dotTrace `.dtp` snapshot family and emits machine-readable JSON call-tree data suitable for LLM-driven hotspot analysis.
|
||||
|
||||
**Context**
|
||||
|
||||
The target snapshot format is JetBrains dotTrace multi-file storage:
|
||||
|
||||
- `snapshot.dtp` is the index/manifest.
|
||||
- `snapshot.dtp.0000`, `.0001`, and related files hold the storage sections.
|
||||
- `snapshot.dtp.States` holds UI state and is not sufficient for call-tree analysis.
|
||||
|
||||
The internal binary layout is not publicly specified. A direct handwritten decoder would be brittle and expensive to maintain. The machine already has dotTrace installed, and the shipped JetBrains assemblies expose snapshot storage, metadata, and performance call-tree readers. The design therefore uses dotTrace’s local runtime libraries as the authoritative decoder while still starting from the raw `.dtp` files.
|
||||
|
||||
**Architecture**
|
||||
|
||||
Two layers:
|
||||
|
||||
1. A small .NET helper opens the raw snapshot, reads the performance DFS call-tree and node payload sections, resolves function names through the profiler metadata section, and emits JSON.
|
||||
2. A Python CLI is the user-facing entrypoint. It validates input, builds or reuses the helper, runs it, and writes JSON to stdout or a file.
|
||||
|
||||
This keeps the user workflow Python-first while using the only reliable decoder available for the undocumented snapshot format.
|
||||
|
||||
**Output schema**
|
||||
|
||||
The JSON should support both direct consumption and downstream summarization:
|
||||
|
||||
- `snapshot`: source path, thread count, node count, payload type.
|
||||
- `thread_roots`: thread root metadata.
|
||||
- `call_tree`: synthetic root with recursive children.
|
||||
- `hotspots`: flat top lists for inclusive and exclusive time.
|
||||
|
||||
Each node should include:
|
||||
|
||||
- `id`: stable offset-based identifier.
|
||||
- `name`: resolved method or synthetic node name.
|
||||
- `kind`: `root`, `thread`, `method`, or `special`.
|
||||
- `inclusive_time`
|
||||
- `exclusive_time`
|
||||
- `call_count`
|
||||
- `thread_name` when relevant
|
||||
- `children`
|
||||
|
||||
**Resolution strategy**
|
||||
|
||||
Method names are resolved from the snapshot’s metadata section:
|
||||
|
||||
- Use the snapshot’s FUID-to-metadata converter.
|
||||
- Map `FunctionUID` to `FunctionId`.
|
||||
- Resolve `MetadataId`.
|
||||
- Read function and class data with `MetadataSectionHelpers`.
|
||||
|
||||
Synthetic and special frames fall back to explicit labels instead of opaque numeric values where possible.
|
||||
|
||||
**Error handling**
|
||||
|
||||
The tool should fail loudly for the cases that matter:
|
||||
|
||||
- Missing dotTrace assemblies.
|
||||
- Unsupported snapshot layout.
|
||||
- Missing metadata sections.
|
||||
- Helper build or execution failure.
|
||||
|
||||
Errors should name the failing stage so the Python wrapper can surface actionable messages.
|
||||
|
||||
**Testing**
|
||||
|
||||
Use the checked-in sample snapshot at `snapshots/js-ordered-consume.dtp` for an end-to-end test:
|
||||
|
||||
- JSON parses successfully.
|
||||
- The root contains thread children.
|
||||
- Hotspot lists are populated.
|
||||
- At least one non-special method name is resolved.
|
||||
|
||||
This is enough to verify the extraction path without freezing the entire output.
|
||||
186
docs/plans/2026-03-14-dtp-parser.md
Normal file
186
docs/plans/2026-03-14-dtp-parser.md
Normal file
@@ -0,0 +1,186 @@
|
||||
# dotTrace DTP Parser Implementation Plan
|
||||
|
||||
> **For Claude:** REQUIRED SUB-SKILL: Use superpowers:executing-plans to implement this plan task-by-task.
|
||||
|
||||
**Goal:** Add a Python-first tool that reads a raw dotTrace `.dtp` snapshot family and emits JSON call-tree and hotspot data for LLM analysis.
|
||||
|
||||
**Architecture:** A small .NET helper uses JetBrains’ local dotTrace assemblies to decode snapshot storage, performance call-tree nodes, payloads, and metadata. A Python wrapper validates input, builds the helper if needed, runs it, and writes the resulting JSON.
|
||||
|
||||
**Tech Stack:** Python 3 standard library, .NET 10 console app, local JetBrains dotTrace assemblies, `unittest`
|
||||
|
||||
---
|
||||
|
||||
### Task 1: Add the failing end-to-end test
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/tests/test_dtp_parser.py`
|
||||
|
||||
**Step 1: Write the failing test**
|
||||
|
||||
Write a `unittest` test that runs:
|
||||
|
||||
```bash
|
||||
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --stdout
|
||||
```
|
||||
|
||||
and asserts:
|
||||
|
||||
- exit code is `0`
|
||||
- stdout is valid JSON
|
||||
- `call_tree.children` is non-empty
|
||||
- `hotspots.inclusive` is non-empty
|
||||
- at least one node name is not marked as special
|
||||
|
||||
**Step 2: Run test to verify it fails**
|
||||
|
||||
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
|
||||
|
||||
Expected: FAIL because `tools/dtp_parse.py` does not exist yet.
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/tests/test_dtp_parser.py
|
||||
git commit -m "test: add dtp parser end-to-end expectation"
|
||||
```
|
||||
|
||||
### Task 2: Implement the .NET snapshot extractor
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/DtpSnapshotExtractor/DtpSnapshotExtractor.csproj`
|
||||
- Create: `tools/DtpSnapshotExtractor/Program.cs`
|
||||
|
||||
**Step 1: Write the minimal implementation**
|
||||
|
||||
Implement a console app that:
|
||||
|
||||
- accepts snapshot path and optional output path
|
||||
- opens the snapshot through JetBrains snapshot storage
|
||||
- constructs performance call-tree and payload readers
|
||||
- resolves method names via metadata sections
|
||||
- builds a JSON object with root tree, thread roots, and hotspot lists
|
||||
- writes JSON to stdout or output file
|
||||
|
||||
**Step 2: Run helper directly**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
dotnet run --project tools/DtpSnapshotExtractor -- snapshots/js-ordered-consume.dtp
|
||||
```
|
||||
|
||||
Expected: JSON is emitted successfully.
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/DtpSnapshotExtractor/DtpSnapshotExtractor.csproj tools/DtpSnapshotExtractor/Program.cs
|
||||
git commit -m "feat: add dottrace snapshot extractor helper"
|
||||
```
|
||||
|
||||
### Task 3: Implement the Python entrypoint
|
||||
|
||||
**Files:**
|
||||
- Create: `tools/dtp_parse.py`
|
||||
|
||||
**Step 1: Write the minimal implementation**
|
||||
|
||||
Implement a CLI that:
|
||||
|
||||
- accepts snapshot path
|
||||
- supports `--out` and `--stdout`
|
||||
- checks that dotTrace assemblies exist in the local install
|
||||
- runs `dotnet run --project tools/DtpSnapshotExtractor -- <snapshot>`
|
||||
- forwards JSON output
|
||||
|
||||
**Step 2: Run the wrapper**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --stdout
|
||||
```
|
||||
|
||||
Expected: JSON is emitted successfully.
|
||||
|
||||
**Step 3: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/dtp_parse.py
|
||||
git commit -m "feat: add python dtp parsing entrypoint"
|
||||
```
|
||||
|
||||
### Task 4: Make the test pass and tighten output
|
||||
|
||||
**Files:**
|
||||
- Modify: `tools/DtpSnapshotExtractor/Program.cs`
|
||||
- Modify: `tools/dtp_parse.py`
|
||||
- Modify: `tools/tests/test_dtp_parser.py`
|
||||
|
||||
**Step 1: Run the failing test**
|
||||
|
||||
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
|
||||
|
||||
Expected: FAIL with an output-schema or execution issue.
|
||||
|
||||
**Step 2: Fix the minimal failing behavior**
|
||||
|
||||
Adjust:
|
||||
|
||||
- special-node labeling
|
||||
- JSON schema stability
|
||||
- helper invocation details
|
||||
- fallback behavior for unresolved metadata
|
||||
|
||||
**Step 3: Re-run the test**
|
||||
|
||||
Run: `python3 -m unittest tools.tests.test_dtp_parser -v`
|
||||
|
||||
Expected: PASS
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add tools/DtpSnapshotExtractor/Program.cs tools/dtp_parse.py tools/tests/test_dtp_parser.py
|
||||
git commit -m "test: verify dtp parser output"
|
||||
```
|
||||
|
||||
### Task 5: Final verification
|
||||
|
||||
**Files:**
|
||||
- Modify: none unless fixes are required
|
||||
|
||||
**Step 1: Run end-to-end extraction**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python3 tools/dtp_parse.py snapshots/js-ordered-consume.dtp --out /tmp/js-ordered-consume-calltree.json
|
||||
```
|
||||
|
||||
Expected: JSON file is created.
|
||||
|
||||
**Step 2: Run test suite**
|
||||
|
||||
Run:
|
||||
|
||||
```bash
|
||||
python3 -m unittest tools.tests.test_dtp_parser -v
|
||||
```
|
||||
|
||||
Expected: PASS
|
||||
|
||||
**Step 3: Inspect a hotspot sample**
|
||||
|
||||
Confirm the JSON contains:
|
||||
|
||||
- resolved method names
|
||||
- inclusive and exclusive hotspot lists
|
||||
- nested thread call trees
|
||||
|
||||
**Step 4: Commit**
|
||||
|
||||
```bash
|
||||
git add docs/plans/2026-03-14-dtp-parser-design.md docs/plans/2026-03-14-dtp-parser.md
|
||||
git commit -m "docs: add dtp parser design and plan"
|
||||
```
|
||||
Reference in New Issue
Block a user