You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The analyzer was previously installing its bundled judge subagent inside
the target repo's .claude/ and writing reports + cache there too. This
mixed the analyzer's tooling with the target's project configuration and
left behind artifacts (.ai-dev-effectiveness-cache/, effectiveness-report.html,
the judge agent itself) in any repo it touched.
New flow:
mkdir ~/dev-effectiveness && cd ~/dev-effectiveness
ai-dev-effectiveness init-judge # installs ./.claude/agents/effort-judge.md
ai-dev-effectiveness analyze /some/target-repo --judge claude-cli
# → reports + cache go to ./target-repo/
# → target repo is read-only
Implementation:
- analyze() takes new `workspace` and `out_dir` kwargs. Agent path and
judge cache dir are resolved as absolute paths inside the workspace
(overriding any YAML setting) so the target repo is guaranteed read-only.
- AnalysisResult exposes `.workspace` and `.out_dir`; `.default_out_path(fmt)`
returns the per-target output location.
- Judge subprocess CWD remains the target (so Read/Grep/git show operate
on target content) but the agent definition comes from the analyzer
workspace via absolute --agent path.
- New `ai_dev_effectiveness.ast_index` module: when `ast-index` is on
PATH, runs `ast-index build` against the target before the judge
starts. Best-effort; never fails the analysis. Disable with --no-ast-index
or AI_DEV_EFFECTIVENESS_NO_AST_INDEX=1.
- CLI:
* `analyze TARGET` — TARGET defaults to '.', out goes to $PWD/<basename>/
when target != $PWD, else to $PWD (legacy single-repo mode).
* `init-judge [WORKSPACE]` — WORKSPACE defaults to '.', help text
explicitly says "NOT one of your project repos".
* New flags: --workspace, --out-dir, --no-ast-index.
- Tests: added test_workspace_target_separation (verifies nothing lands
in target) and test_target_equals_workspace_keeps_legacy_layout.
- README + case-study README rewritten around the new flow with a
workspace-layout diagram.
Verified end-to-end on the HockeyBot worktree:
- Workspace at /tmp/ade-workspace/.claude/
- Output at /tmp/ade-workspace/musing-haslett-b9b799/
- Cache at /tmp/ade-workspace/musing-haslett-b9b799/.ai-dev-effectiveness-cache/
- Target /Users/denn/hockeybot_ws/.claude/worktrees/musing-haslett-b9b799/
contains zero analyzer artifacts.
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
# Once: install the bundled judge subagent into THIS workspace.
16
+
ai-dev-effectiveness init-judge
17
+
18
+
# Now analyze any repo you want. Reports go to ./<target_basename>/.
19
+
ai-dev-effectiveness analyze /path/to/your/repo
20
+
open ./your-repo/effectiveness-report.html
14
21
```
15
22
16
-
That's it. Zero config required for the first run — the tool auto-detects domains from your top-level dirs, classifies commits by language, and identifies AI co-authors from a built-in registry covering Claude (Opus/Sonnet/Haiku), GitHub Copilot, Cursor, Codex, Aider, Cody, Continue, Windsurf, Codeium, Tabnine, and Devin.
23
+
The analyzer **never writes inside the target repo** — the bundled subagent, the judgment cache, and all reports live in your local workspace. Each target gets its own subfolder so you can compare repos side by side.
17
24
18
25
## What it measures
19
26
@@ -28,23 +35,48 @@ When all three agree within 2x, the multiplier is defensible. When they diverge,
28
35
## Optional: AI judge for diff-aware estimates
29
36
30
37
```bash
31
-
ai-dev-effectiveness init-judge # installs the bundled subagent into .claude/
32
-
ai-dev-effectiveness analyze . --judge claude-cli
38
+
cd~/dev-effectiveness # the workspace where init-judge ran
The judge uses your existing Claude Code CLI session — no API key, no metered billing, no diffs leaving your machine. If you have [ast-index](https://github.com/asgardiandevs/ast-index) on your `PATH`, the analyzer runs `ast-index build` against the target before the judge starts so the bundled subagent can use `mcp__ast-index__*` tools for symbol-level lookups (much faster than grep-then-read for "find callers of this function").
43
+
44
+
Other providers (opt-in): `--judge anthropic-api` (separate API key), `--judge openai`, `--judge ollama` (local model).
45
+
46
+
Disable ast-index for a run: `--no-ast-index`.
47
+
48
+
## Workspace layout
49
+
50
+
```
51
+
~/dev-effectiveness/ # your analyzer workspace
52
+
├── .claude/
53
+
│ ├── agents/effort-judge.md # ← installed by `init-judge`
The judge uses your existing Claude Code CLI session — no API key, no metered billing, no diffs leaving your machine. If you have [ast-index](https://github.com/defendend/Claude-ast-index-search) MCP configured, it's used for symbol-level lookups.
65
+
Useful flags:
36
66
37
-
Other providers: `--judge anthropic-api` (separate API key), `--judge openai`, `--judge ollama` (local model).
67
+
-`--workspace PATH` — override the analyzer workspace (default: `$PWD`).
68
+
-`--out-dir PATH` — override where this run's reports go.
69
+
-`--judge-dry-run` — show what the judge would do (sample size, cost, paths) without invoking it.
38
70
39
71
## Case study
40
72
41
73
[`examples/robotics-case-study/`](examples/robotics-case-study/) — anonymized analysis of a real 117K-LOC robotics project built by 1 dev + Claude over 10 months. Use it to see what a credible report looks like and to calibrate your own results.
42
74
43
75
## Customize
44
76
45
-
-`ai-dev-effectiveness init-config` — drops `ai_dev.yaml` into your repo
46
-
- Define your specialist roles, effort constants, agent signatures
47
-
- See [`docs/REPRODUCTION_GUIDE.md`](docs/REPRODUCTION_GUIDE.md) and [`docs/config-reference.md`](docs/config-reference.md)
77
+
-`ai-dev-effectiveness init-config` — drops `ai_dev.yaml` into your workspace.
78
+
- Define your specialist roles, effort constants, agent signatures.
79
+
- See [`docs/REPRODUCTION_GUIDE.md`](docs/REPRODUCTION_GUIDE.md) and [`docs/config-reference.md`](docs/config-reference.md).
0 commit comments