docs(prompts): align install / runbook / test-writer with LISA loader and CI lint

.github/prompts/install-lisa.prompt.md

@@ -264,11 +264,139 @@ lisa --help
 
 ---
 
+## Capture the Install Path (so runbooks can find `microsoft.testsuites`)
+
+`microsoft/testsuites/` is loaded dynamically by LISA. The mechanism (see
+`lisa/parameter_parser/runbook.py::_fix_path_for_old_code_layout`) is:
+
+- LISA computes `<lisa_root>` from the **running** install's
+  `lisa/__init__.py` location (i.e., whatever `pip` resolved the `lisa`
+  console-script to).
+- A runbook's `extension:` path is rewritten to `<lisa_root>/lisa/microsoft`
+  and registered as the Python package `microsoft` **only when** that path
+  is already under `<lisa_root>/lisa/microsoft`.
+- If the user has two LISA checkouts on disk and pip picked one but the
+  runbook points at the other, the rewrite silently doesn't fire and the
+  run dies with `ModuleNotFoundError: No module named 'microsoft'` even
+  though the directory exists.
+
+So the install step must guarantee — and **record** — the *single* repo
+that the `lisa` command actually loads from.
+
+When install completes, do **all four** of the following:
+
+1. **Verify which repo the `lisa` command loads from** (this is the
+   authoritative `LISA_HOME`, never trust `pwd`):
+
+   ```bash
+   # Linux / WSL
+   <PYTHON> -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"
+   ```
+
+   ```powershell
+   # Windows
+   & '<PYTHON>' -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"
+   ```
+
+   If the printed path is **not** the repo you just installed, there is a
+   stale install winning the resolution. Fix it before continuing:
+
+   - `<PYTHON> -m pip uninstall -y lisa` until `pip show lisa` reports nothing,
+     then re-run `pip install -e .[azure]` from the desired repo, OR
+   - inside the desired repo, run `<PYTHON> -m pip install -e . --force-reinstall`,
+     OR
+   - simply delete the unused checkout from disk.
+
+2. **Print the absolute install path** at the end, on its own line:
+
+   ```text
+   LISA_HOME=<path printed by step 1, the running LISA's repo root>
+   PYTHON=<absolute path to the python interpreter that has lisa installed>
+   ```
+
+   `PYTHON` matters whenever a venv is involved (`quick-install-dev.ps1`
+   always creates one at `<install_path>\.venv`; `quick-install.sh` does
+   when invoked with `--use-venv true`). Without it, opening a fresh shell
+   that hasn't activated the venv either fails with `lisa: command not
+   found` or silently picks up a system Python that has no LISA.
+
+   Resolve `PYTHON` like this:
+   - Windows venv: `<install_path>\.venv\Scripts\python.exe`
+   - Linux/WSL venv: `<install_path>/.venv/bin/python`
+   - No venv (system / `pipx` / `--user` install): record `PYTHON=python`
+     so downstream tools know no activation is required.
+
+3. **Set the `LISA_HOME` environment variable** for the current shell. If a
+   venv was created, also tell the user how to activate it (or invoke the
+   venv python directly without activation):
+
+   ```powershell
+   # Windows (PowerShell)
+   $env:LISA_HOME = '<absolute path>'
+   [Environment]::SetEnvironmentVariable('LISA_HOME', '<absolute path>', 'User')
+
+   # If a venv exists, activate it in every new shell, OR call the venv python directly:
+   & '<install_path>\.venv\Scripts\Activate.ps1'
+   # — or —
+   & '<install_path>\.venv\Scripts\python.exe' -m lisa --help
+   ```
+
+   ```bash
+   # Linux / macOS / WSL
+   export LISA_HOME='<absolute path>'
+   echo 'export LISA_HOME="<absolute path>"' >> ~/.bashrc
+
+   # If a venv exists:
+   source '<install_path>/.venv/bin/activate'
+   # — or —
+   '<install_path>/.venv/bin/python' -m lisa --help
+   ```
+
+   Do not bake `Activate.ps1` into `$PROFILE` automatically — mention it as
+   an option, let the user opt in.
+
+4. **Record it in session memory** (so other prompts in the same workspace
+   can read it without re-asking):
+
+   - Path: `/memories/session/lisa-install.md`
+   - Content:
+
+     ```text
+     LISA_HOME=<absolute path verified by step 1>
+     PYTHON=<absolute path to venv python, or literally "python" if no venv>
+     VENV=<absolute path to the venv root, or empty if no venv>
+     ```
+
+   The `lisa_runbook_generator` prompt reads this file and trusts step 1's
+   path as authoritative — if you skip step 1 and write `pwd` here, it WILL
+   bite you the moment a runbook needs microsoft testsuites.
+
+The `microsoft/testsuites/` directory then lives at
+`$LISA_HOME/lisa/microsoft/testsuites` (note the doubled `lisa/lisa/` —
+the outer `lisa/` is the repo, the inner `lisa/` is the Python package).
+
+---
+
 ## Troubleshooting
 
 ### "lisa" command not found
 - **Windows**: Restart PowerShell to refresh PATH
 - **Linux**: Ensure `~/.local/bin` is in PATH: `export PATH="$HOME/.local/bin:$PATH"`
+- **venv install (any OS)**: the `lisa` console-script only exists inside the venv.
+  Either activate it (`.venv\Scripts\Activate.ps1` / `source .venv/bin/activate`)
+  or call it via the venv python without activation:
+  `<install_path>/.venv/bin/python -m lisa --help` (Linux) /
+  `<install_path>\.venv\Scripts\python.exe -m lisa --help` (Windows).
+  After activation, `(.venv)` should appear in the prompt and `where.exe lisa`
+  / `which lisa` should resolve into the venv directory.
+
+### Wrong Python is picked up (system Python instead of venv)
+Symptom: `python -c "import lisa; print(lisa.__file__)"` errors out, or points
+to a different LISA install than expected, even though install succeeded.
+Fix: confirm the active interpreter with `python -c "import sys; print(sys.executable)"`.
+It must match the `PYTHON` line recorded in `/memories/session/lisa-install.md`.
+If not, activate the venv first, or invoke the venv python explicitly
+(`<venv>/bin/python` / `<venv>\Scripts\python.exe`).
 
 ### Python version too old
 - Re-run the install script with `--python-version 3.12` (Linux) or `-PythonVersion "3.12"` (Windows)
@@ -282,6 +410,39 @@ lisa --help
   - Ubuntu/Debian: `sudo apt-get install -y python3-dev build-essential libssl-dev libffi-dev`
   - RHEL/CentOS: `sudo dnf install -y python3-devel gcc openssl-devel libffi-devel`
 
+### `ModuleNotFoundError: No module named 'microsoft'` when running a runbook
+
+Symptom: `lisa -r my_runbook.yml` logs
+`loading Python extensions from <some path>` then fails with
+`ModuleNotFoundError: No module named 'microsoft'`.
+
+LISA found the directory — the failure is in the auto-rename step. The path
+you gave to `extension:` must be under the **running LISA's**
+`<lisa_root>/lisa/microsoft`, otherwise the loader keeps a generic name
+like `lisa_ext_0` and absolute imports such as
+`from microsoft.testsuites.xfstests.xfstests import ...` cannot resolve.
+
+Fix in order of preference:
+1. Re-run the **Capture the Install Path** verification step above to
+   discover the running LISA's actual root
+   (`<PYTHON> -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"`).
+2. If the printed path is not the repo you intended, you have **two LISA
+   checkouts** — pip resolved the wrong one. Either `pip uninstall lisa`
+   in the unused checkout, or `pip install -e . --force-reinstall` from
+   the desired repo.
+3. In the runbook, drop `extension:` and use
+
+   ```yaml
+   import_builtin_tests: true
+   ```
+
+   instead. LISA then loads `<running_lisa_root>/lisa/microsoft` itself —
+   the path can never disagree with the install.
+4. Only if you actually need a custom (non-microsoft) extension, keep
+   `extension:` and make sure each entry is an absolute path under the
+   *running* LISA root, or a path relative to the runbook file that
+   resolves there.
+
 ### VS Code F5 hangs / debugpy traceback in `importlib.metadata`
 Symptom: hitting F5 launches `C:\Program Files\Python312\python.exe` (system Python), then debugpy stalls inside `describe_environment` reading broken package METADATA.
 Cause: newer `ms-python` versions ignore `launch.json`'s `"python"` field and use the workspace interpreter, which defaults to system Python (no LISA installed there).

.github/prompts/lisa_runbook_generator.prompt.md

@@ -4,6 +4,73 @@ You generate LISA runbook YAML files for running tests. Ask the user what scenar
 
 ---
 
+## Step 0: Locate the **executing** LISA install (do this BEFORE generating YAML)
+
+LISA's runbook loader has a built-in path: when `extension:` points to
+anything under `<lisa_root>/lisa/microsoft`, LISA rewrites the path to
+`<lisa_root>/lisa/microsoft` and forces the package name to `microsoft`.
+This is what makes cross-imports like
+`from microsoft.testsuites.xfstests.xfstests import ...` work.
+
+`<lisa_root>` is **not** an env var. It is computed at run time from
+`Path(lisa.__file__).parent.parent` of the LISA install that the `lisa`
+command actually loads. If the user has more than one LISA checkout, pip's
+entry-point may point to a different one than the runbook expects, the
+rewrite never fires, and you get `ModuleNotFoundError: No module named 'microsoft'`
+even though the path on disk looks correct.
+
+Resolve **the executing LISA root** in this order — stop at the first hit:
+
+1. **Run the verification command** (preferred — it cannot be wrong):
+
+   ```bash
+   # Linux / WSL
+   <PYTHON> -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"
+   ```
+
+   ```powershell
+   # Windows
+   & '<PYTHON>' -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"
+   ```
+
+   The printed path **is** `LISA_HOME`. Use it. If `<PYTHON>` is unknown,
+   take it from `/memories/session/lisa-install.md` (written by the install
+   prompt) or from the active venv (`which lisa` / `where.exe lisa` then
+   resolve the script's shebang).
+
+2. **Session memory**: `/memories/session/lisa-install.md` lists `LISA_HOME`,
+   `PYTHON`, `VENV` from the install step. Cross-check with step 1 — if they
+   disagree, **trust step 1** and update the memory file.
+3. **Ask the user** (only if both above fail): "Where is the LISA repo that
+   `lisa --version` runs from?".
+
+Use `LISA_HOME` as a literal absolute path inside the generated runbook
+(see Rule 8). Do NOT emit placeholders like `<lisa_repo_path>` in the final
+YAML.
+
+**venv reminder when presenting the run command:** if `VENV` is non-empty,
+always show the user the run command **with the venv activated or invoked
+explicitly**, e.g.:
+
+```powershell
+# Activate then run (Windows)
+& '<VENV>\Scripts\Activate.ps1'; lisa -r path\to\runbook.yml
+# — or, no activation needed —
+& '<VENV>\Scripts\python.exe' -m lisa -r path\to\runbook.yml
+```
+
+```bash
+# Activate then run (Linux/WSL)
+source '<VENV>/bin/activate' && lisa -r path/to/runbook.yml
+# — or —
+'<VENV>/bin/python' -m lisa -r path/to/runbook.yml
+```
+
+Skipping venv activation — or running a `lisa` from a different repo than
+`LISA_HOME` — are the two common causes of `ModuleNotFoundError`.
+
+---
+
 ## Step 1: Clarify the Scenario
 
 Before generating YAML, ask:
@@ -12,6 +79,8 @@ Before generating YAML, ask:
 3. **What images?** (see Image Formats below)
 4. **Special requirements?** (security profile, WSL, purchase plan, disk/NIC)
 5. **Azure auth method?** (only when platform is azure — see Auth Methods below)
+6. **Where will the runbook live?** — needed to compute the correct `extension:`
+   path (absolute is safest; see Rule 8).
 
 ### Image Formats
 
@@ -112,7 +181,8 @@ Top-level sections (only `platform` + `testcase` are required):
 |---------|---------|
 | `name` | Descriptive run name |
 | `include` | Inherit from other YAML files: `- path: ./azure.yml` |
-| `extension` | Extra test module paths: `- "<lisa_repo_path>/lisa/microsoft/testsuites"` |
+| `extension` | Optional. Extra test-module roots. For microsoft testsuites, prefer `import_builtin_tests: true` (Rule 8 option B) instead of listing `extension:` here — it removes a class of path mistakes. If you must use `extension:`, follow Rule 8. |
+| `import_builtin_tests` | Set to `true` to auto-load the microsoft testsuites bundled with the running LISA install. No path needed. |
 | `variable` | Parameters with `$(name)` substitution. Supports `is_secret: true`, `is_case_visible: true`, `file: ./secrets.yml` |
 | `platform` | Where to run (azure, ready, qemu, openvmm) |
 | `testcase` | What to run — filter by priority, name, area |
@@ -373,4 +443,90 @@ Refer to `lisa/microsoft/runbook/` for available base runbooks.
 4. **Use `include`** for existing base runbooks — don't duplicate. Check `lisa/microsoft/runbook/`.
 5. **Security profiles** require Gen2 images and the marketplace **object format** — the 4-part string shorthand does not support them.
 6. **Transformer phases** run in order: `init` → `expanded` → `environment_connected` → `expanded_cleanup` → `cleanup`. Use `phase: environment_connected` for transformers that need a provisioned VM (e.g., installing components on the node). `expanded` runs before environments are created.
-7. Search `@workspace` for existing runbooks before generating — reuse patterns from `runbook/`.
+7. Search `@workspace` for existing runbooks before generating — reuse patterns from `runbook/`.
+8. **Loading microsoft testsuites — pick exactly one of these patterns; never
+   invent a third one:**
+
+   **(A) Preferred when the runbook lives _inside_ the LISA repo at
+   `<LISA_HOME>/<somewhere>/<runbook>.yml` and references microsoft tests:**
+   use the canonical relative form, exactly like the shipped runbooks under
+   `lisa/microsoft/runbook/`:
+
+   ```yaml
+   extension:
+     - "../testsuites"        # when runbook is at lisa/microsoft/runbook/*.yml
+     # or
+     - "../../testsuites"     # when runbook is one level deeper
+   ```
+
+   The relative path must resolve to `<LISA_HOME>/lisa/microsoft/testsuites`
+   so LISA's `_fix_path_for_old_code_layout` rewrites the package to
+   `microsoft` and absolute imports work.
+
+   **(B) Preferred when the runbook lives _outside_ the LISA repo (e.g., a
+   user-managed `runbooks/` folder somewhere else):** drop `extension:`
+   entirely and use the built-in tests flag:
+
+   ```yaml
+   import_builtin_tests: true
+   ```
+
+   This makes LISA call `import_package(<LISA_HOME>/lisa/microsoft, "microsoft")`
+   internally — same effect as (A), but with no path to get wrong. Use this
+   for the `lisa-bug-fix` / install-prompt-generated runbooks that sit in
+   user-chosen directories.
+
+   **(C) Absolute path** — only for non-microsoft custom extensions, or as a
+   last resort when (A) and (B) don't fit. The path **must** point under
+   `<LISA_HOME>/lisa/microsoft` (verified via Step 0), otherwise the
+   auto-rewrite to package name `microsoft` does not fire and absolute
+   imports break:
+
+   ```yaml
+   extension:
+     - "/abs/path/to/<LISA_HOME>/lisa/microsoft/testsuites"
+   ```
+
+   Never emit `<lisa_repo_path>` placeholders, never use a relative path
+   anchored to CWD, and never point at a different LISA checkout than the
+   one `lisa --version` runs from — that's the bug we're trying to avoid.
+
+---
+
+## Troubleshooting
+
+### `ModuleNotFoundError: No module named 'microsoft'` (or `microsoft.testsuites`)
+
+This fails *after* LISA logs `loading Python extensions from ...`, which
+means LISA found the directory. The real cause is one of:
+
+- **Path is not under `<LISA_HOME>/lisa/microsoft`**, so the loader's
+  auto-rewrite to package name `microsoft` does not fire. Confirm with the
+  Step 0 verification command and compare against your runbook's
+  `extension:` value.
+- **Two LISA repos on disk**, pip's `lisa` entry-point points at one and
+  your runbook points at the other. Only `<LISA_HOME>/lisa/microsoft` of
+  the running repo counts. Either:
+  - delete / `pip uninstall lisa` the other checkout, or
+  - re-`pip install -e .` from the repo you actually want, or
+  - use Rule 8 option (B) `import_builtin_tests: true`, which always loads
+    from the running LISA's repo.
+- **Path is on a different filesystem from the running LISA install** (e.g.,
+  WSL `/mnt/wsl/temp/lisa` vs `/root/lisa`). Same fix as above.
+
+Fix recipe:
+
+1. Verify the running LISA's root:
+
+   ```bash
+   <PYTHON> -c "import lisa, pathlib; print(pathlib.Path(lisa.__file__).parent.parent)"
+   ```
+
+2. Compare with `LISA_HOME` from `/memories/session/lisa-install.md`. They
+   **must** be the same path. If not, fix the install (re-run the install
+   from inside the desired repo) before regenerating runbooks.
+3. Switch the runbook to `import_builtin_tests: true` (Rule 8 option B) and
+   remove the `extension:` line — simplest reliable fix.
+4. Re-run with `-d` and confirm the log shows
+   `loading Python extensions from <LISA_HOME>/lisa/microsoft` (the directory
+   above `testsuites`, not `testsuites` itself — that's the rewrite firing).