Reduce tool-denial failures in Daily Compiler Threat Spec Optimizer#40743
Conversation
This comment has been minimized.
This comment has been minimized.
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
Co-authored-by: pelikhan <4175913+pelikhan@users.noreply.github.com>
|
Hey One item to follow up on:
If you would like a hand, here is a prompt you can assign to your coding agent:
|
There was a problem hiding this comment.
Pull request overview
Note
Copilot couldn't run its full agentic review because no GitHub Actions runner was available. Make sure your repository has a runner available to run Copilot's review, or add a copilot-setup-steps.yml file specifying one with the runs-on attribute. See the docs for more details.
Updates the Daily Compiler Threat Spec Optimizer workflow to reduce Copilot SDK tool-denial failures by broadening the bash allowlist and adding prompt guidance aligned with the restricted tool model.
Changes:
- Replaced exact-match bash command allowlist with command “stems” (e.g.,
git,go,cat, etc.) to tolerate argument variations. - Added a “Tooling Constraint” section to the workflow prompt to guide the agent to use allowed shell commands for inspection.
- Recompiled the generated
.lock.ymlso runtime--allow-toolarguments reflect the updated policy.
Show a summary per file
| File | Description |
|---|---|
| .github/workflows/daily-compiler-threat-spec-optimizer.md | Broadens the tool allowlist and adds prompt guidance about tool constraints. |
| .github/workflows/daily-compiler-threat-spec-optimizer.lock.yml | Regenerates compiled workflow runtime args to match the updated allowlist. |
Copilot's findings
Tip
Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
- Files reviewed: 2/2 changed files
- Comments generated: 3
| bash: | ||
| - git ls-files pkg/workflow/*.go | ||
| - git ls-files pkg/parser/*.go | ||
| - cat specs/compiler-threat-detection-spec.md | ||
| - "git log --since=\"2 days ago\" --oneline -- pkg/workflow pkg/parser actions/setup/js" | ||
| - "git diff -- pkg/workflow pkg/parser actions/setup/js" | ||
| - go test -v ./pkg/workflow/... | ||
| - git | ||
| - cat | ||
| - find | ||
| - ls | ||
| - sed | ||
| - awk | ||
| - grep | ||
| - head | ||
| - pwd | ||
| - go |
| # --allow-tool shell(git:*) | ||
| # --allow-tool shell(go:*) |
| ## Tooling Constraint | ||
|
|
||
| This workflow uses a restricted Copilot SDK shell allowlist. For repository inspection, use the approved shell commands above (`git`, `cat`, `find`, `ls`, `sed`, `awk`, `grep`, `head`, `pwd`, `go`) instead of built-in file read/view tools, and avoid requesting commands outside that set. |
The Daily Compiler Threat Spec Optimizer was exhausting the Copilot SDK tool-denial guardrail and terminating early. The failure came from a brittle workflow tool policy: the agent requested normal repository inspection operations that did not match the workflow’s exact shell allowlist.
Workflow tool policy
git,go,find,ls,cat,sed,awk,grep,head,pwd).git log,git diff, andgo testinvocations.Prompt guidance
Compiled workflow
.lock.ymlso the updated allowlist is reflected in the runtime--allow-toolarguments.Example of the effective change: