feat(coding-agent): FastContext explore adapter with hint and agent modes#3164
feat(coding-agent): FastContext explore adapter with hint and agent modes#3164oldschoola wants to merge 26 commits into
Conversation
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 2679ef4347
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return (allCitations.length === 0 ? builder.error().useless() : builder).done(); | ||
| } | ||
| toolCalls += response.toolCalls.length; | ||
| const boundedCalls = response.toolCalls.slice(0, MAX_PARALLEL_TOOL_CALLS); |
There was a problem hiding this comment.
When the FastContext model returns more than 8 parallel tool calls, this slices the calls that get executed, but the assistant message already appended to messages still advertises the full tool_calls array. On the next Chat Completions request, OpenAI-compatible servers see missing tool responses for the dropped call IDs and reject the transcript instead of continuing the exploration; either truncate the saved assistant message too or return tool-result errors for the extra calls.
Useful? React with 👍 / 👎.
| const direct = path.resolve(cwd, pattern); | ||
| const stat = await fs.stat(direct).catch(() => null); | ||
| if (stat?.isFile()) return [direct]; |
There was a problem hiding this comment.
In hint mode, a model-produced non-glob path like ../other-repo or /tmp is resolved and statted directly here, and directories are then recursively globbed, so the read-only adapter can return/cite filenames outside cwd. The agent-mode Read/Glob/Grep paths enforce resolveWorkspacePath, and this hint path should apply the same workspace check before statting or globbing to avoid leaking surrounding filesystem entries.
Useful? React with 👍 / 👎.
roboomp
left a comment
roboomp
left a comment
There was a problem hiding this comment.
P3: the FastContext adapter is interesting, but the branch is too broad/stale; llama.cpp/Wafer provider changes already landed in #3160 and should be rebased out.
Blocking findings: hint-mode glob paths can escape the workspace, and zero-file hint results are surfaced as successful instead of triggering the advertised fallback.
Also flagged citation line-range validation: it can accept out-of-file ranges after clamping internally.
Thanks for pushing this experiment; a rebased, FastContext-only branch with these safety/contract fixes will be much easier to evaluate.
| @@ -0,0 +1,34 @@ | |||
| import type { OAuthController, OAuthLoginCallbacks } from "./oauth/types"; | |||
There was a problem hiding this comment.
should-fix: this provider addition is stale scope in this FastContext PR. origin/main already contains the llama.cpp provider/Wafer Pass removal via merged PR #3160 (6b744c5ad6 in the base log), so keeping these files in this branch makes maintainers re-review unrelated provider churn and raises conflict risk. Please rebase/drop the already-landed provider/catalog/docs changes so the diff is only the FastContext adapter.
|
|
||
| return toolResult<FastContextToolDetails>({ | ||
| baseUrl: apiBaseUrl, |
There was a problem hiding this comment.
blocking: #nativeGlob resolves direct hint globs with path.resolve(cwd, pattern) but never checks isWithinCwd. A local model can return a non-glob path like .. or /etc; this branch then stats it and, for directories, enumerates it with glob({ path: direct }), leaking files outside the workspace. Use resolveWorkspacePath (or reject non-workspace direct paths) before stat/glob.
| const [globResults, grepResults] = await Promise.all([ | ||
| plan.globs.length > 0 | ||
| ? Promise.all(plan.globs.map(g => this.#nativeGlob(g, this.#session.cwd, signal))) |
There was a problem hiding this comment.
blocking: empty hint results are returned as a successful tool call. When allFiles.length === 0, this still builds [FastContext hint ... 0 files] and falls through to .done() with no error/useless flag, so the explore agent can treat an empty shortlist as a valid answer instead of falling back as the PR describes. Mirror the agent no-citation path or add an explicit zero-result branch/test that marks the result as failed/useless.
| const line = rawLine.trim(); | ||
| if (!line || /https?:\/\//i.test(line)) continue; | ||
| const match = line.match( | ||
| /(?:^|[`*\s-])([A-Za-z]:[\\/][^`\n]+?|[\\/][^`\n]+?|(?:\.{1,2}[\\/]|[A-Za-z0-9_.-]+[\\/])[^`\n]+?|[A-Za-z0-9_.-]+\.[A-Za-z0-9][A-Za-z0-9_.-]*):(\d+)(?:[-–—](\d+))?\b/, | ||
| ); |
There was a problem hiding this comment.
should-fix: citation validation does not reject line ranges outside the file. citationMatchesQuery clamps lineStart/lineEnd internally, then parseCitations pushes the original citation, so src/foo.ts:999999-999999 can be accepted if a keyword appears in the path or clamped snippet. Please reject invalid ranges (start < 1, end < start, start > lineCount) before adding the citation.
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
from
eb26204 to
3d3f941
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 3d3f941ce2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const line = rawLine.trim(); | ||
| if (!line || /https?:\/\//i.test(line)) continue; | ||
| const match = line.match( | ||
| /(?:^|[`*\s-])([A-Za-z]:[\\/][^`\n]+?|[\\/][^`\n]+?|(?:\.{1,2}[\\/]|[A-Za-z0-9_.-]+[\\/])[^`\n]+?|[A-Za-z0-9_.-]+\.[A-Za-z0-9][A-Za-z0-9_.-]*):(\d+)(?:[-–—](\d+))?\b/, |
There was a problem hiding this comment.
When agent mode answers with a valid top-level extensionless file such as Dockerfile:1-20 (this repo has a top-level Dockerfile), this regex does not match because the basename-only alternative requires a dot and the other relative-path alternatives require a slash. parseCitations then reports zero citations and the tool returns the “no file-line citations” error even though the model followed the requested file-line format; allow basename citations without extensions before rejecting the final answer.
Useful? React with 👍 / 👎.
| if (totalBytes >= HINT_MAX_SNIPPET_BYTES) break; | ||
| if (signal?.aborted) break; | ||
| try { | ||
| const rawText = await Bun.file(file).text(); |
There was a problem hiding this comment.
In default hint mode, broad model-generated globs can include large lockfiles, generated JSON, or minified bundles, but this reads each candidate with Bun.file(file).text() before enforcing HINT_MAX_SNIPPET_BYTES or the snippet line cap. A single very large candidate can stall or OOM the CLI even though only a short snippet is needed; check size or stream only the bounded prefix before decoding (and apply the same guard to the FastContext Read path).
Useful? React with 👍 / 👎.
| const snippetFiles = includeSnippets ? allFiles.slice(0, HINT_MAX_SNIPPET_FILES) : []; | ||
| const snippets = await this.#readSnippets(snippetFiles, grepFileSet, plan.keywords, snippetLines, signal); | ||
|
|
||
| const citations = allFiles.map(f => `${f}:1-1`); |
There was a problem hiding this comment.
When the hint planner emits a broad glob such as packages/* or a directory path, #nativeGlob can return directories because it does not request a file-only filter, and this line turns every returned entry into a high-confidence :1-1 file citation. Those directory citations cannot be read or validated and can make the explore handoff contain bogus file locations; filter glob results to regular files before building citations and snippets.
Useful? React with 👍 / 👎.
FastContext Precision & Token Optimization ResultsPushed to Metrics
Real model validation (FastContext-1.0-4B-RL at localhost:8080)5 consecutive runs after final fix:
Average: 38/40 = 95% (up from 37.5% baseline) Improvements (3 commits)1. Fallback grep sorting + glob-matched file boosting
2. Script penalty + path-score re-sort + fallback grep filter
3. Fallback file truncation fix
Diff |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 7f1868c1db
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| let grepFileSet = new Set(grepResults.flat()); | ||
| const globMatchedSet = new Set<string>(); | ||
| let allFiles = [...new Set([...globResults.flat(), ...grepResults.flat()])].slice(0, MAX_TOOL_LINES); |
There was a problem hiding this comment.
When a hint plan includes a broad glob such as **/*.ts plus a precise grep_patterns entry, #nativeGlob can return MAX_TOOL_LINES paths, and this line orders those glob hits before grepResults and slices immediately. Any grep-only files after the first 100 candidates are then absent from ranking, snippets, and citations, so the model's more precise content match can be dropped in favor of unrelated glob results; put grep results first or defer this cap until after scoring.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b8668a463f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| - Fixed `/mcp enable` and `/mcp disable` reconnecting unrelated MCP servers by scoping toggle reconnect/disconnect work to the named server. ([#3157](https://github.com/can1357/oh-my-pi/issues/3157)) | ||
| ### Added | ||
|
|
||
| - Added an opt-in FastContext adapter for the bundled `explore` subagent, with `fastContext.enabled`, `fastContext.baseUrl`, and `fastContext.model` settings for local OpenAI-compatible Chat Completions endpoints. The adapter has two modes: `hint` (default, ~2-3s) — one LLM turn expands the query into keywords/globs/grep patterns, then native ripgrep/glob executes them in parallel; and `agent` — full multi-turn FastContext agentic loop with `Read`/`Glob`/`Grep` tool names and `<final_answer>` citation validation. Hint mode benchmarks 3/8 hit rate at ~2.3s avg vs 2/8 at ~34s for agent mode on the oh-my-pi repo root. |
There was a problem hiding this comment.
AGENTS.md says new changelog entries must go under ## [Unreleased] and released sections are immutable. Placing this FastContext entry under ## [16.1.9] - 2026-06-21 means the next release can miss the feature note or retroactively mutate already-published release notes; move it to the empty Unreleased section instead.
Useful? React with 👍 / 👎.
| - Removed the `setNextRequestDebugPath`, `clearNextRequestDebugPath`, and `getNextRequestDebugPath` utility functions for request debugging, as request/response recording now relies exclusively on the `PI_REQ_DEBUG` environment variable. | ||
| ### Removed | ||
|
|
||
| - Removed Wafer Pass (`wafer-pass`) login support; Wafer Serverless remains available as `wafer-serverless`. |
There was a problem hiding this comment.
AGENTS.md marks released changelog sections as immutable and requires all new entries under ## [Unreleased]. This newly added Wafer Pass removal note is inside the already-released 16.1.9 section, so the next release notes can omit it while also changing historical release text.
Useful? React with 👍 / 👎.
FastContext Evaluation ScoresThree evaluation scripts dispatched as parallel subagents. Results below. 1. Delegated Repository Exploration ScoreMeasures retrieval quality when FastContext is used as a delegated exploration tool (hint mode — the default path through the explore subagent).
Score: A — Hint mode is the clear winner for delegated exploration: 95% hit rate at 2.5s latency with a 73-token packet. Agent mode trades 10× latency for no retrieval improvement on these cases. 2. Main-Agent Token Savings ScoreMeasures how many tokens the main agent saves by using FastContext instead of manual search/read/grep. Scenario A = real native glob+grep+read calls. Scenario B = FastContext hint packet + 50 reasoning tokens.
Score: A+ — FastContext saves the main agent ~95% of tokens vs manual exploration. The biggest savings come from avoiding file reads (explore-agent-tools: 175K read tokens → 1.7K packet). 3. Standalone Exploration ScoreMeasures FastContext used directly as a tool call (not through the explore subagent).
Score: C — Under strict precision_at_5 (expected file must appear in top 5), all three baselines score identically. The LLM model plan adds zero retrieval quality over pure query-derived grep on these 8 cases. The model plan typically produces the same grep/glob patterns that Note: This contradicts the 95% hit rate from evaluation #1. The difference is hit detection methodology: evaluation #1 uses the lenient 4. FastContext vs Non-FastContext BaselineDoes the FastContext model recover patch-relevant files more accurately than pure query-derived grep (no model)?
Answer: No. The FastContext-1.0-4B-RL model does not improve retrieval quality over the pure query-derived grep fallback path. Both score 62.5% on strict precision_at_5. The model plan adds ~2.4s latency and ~15× token cost for zero retrieval gain. The ranking pipeline (path scoring, content scoring, grep/glob boost) does all the heavy lifting — the model plan is redundant when the fallback already derives the same keywords. However: The lenient hit detection used in evaluation #1 shows 95% hit rate, suggesting the model plan helps the target appear somewhere in the result list (even if not in top 5). This could be valuable for interactive exploration where the user scans the full list. 5. Additional Performance OpportunitiesFrom the performance profiling analysis (
Hard floor: ~1.3-1.5s (LLM compute-bound, cannot reduce without a smaller/faster model) Counterintuitive findings:
Top priority: Merge plan + supplementary searches into one Summary Scorecard
|
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 891023d4a7
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| workDirListing: await buildWorkspaceListing(this.#session.cwd, signal), | ||
| }), | ||
| }, | ||
| { role: "user", content: `<query>\n${params.query.trim()}\n</query>` }, |
There was a problem hiding this comment.
Root AGENTS.md says prompts must not be built in code and dynamic content should be rendered from static .md files with Handlebars. This adds the FastContext agent user prompt wrapper inline, so prompt audits/formatting and future prompt edits can miss this path; put the <query> wrapper in a prompt template and render params.query into it instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f3483ffe80
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (lineStart < 1 || lineEnd < lineStart) continue; | ||
| const fileText = await Bun.file(resolved).text(); | ||
| const fileLines = splitFileLines(fileText); | ||
| if (lineStart > fileLines.length) continue; |
There was a problem hiding this comment.
When FastContext emits a range like README.md:1-9999 for a short file, this only rejects starts beyond EOF; citationMatchesQuery then clamps the snippet for keyword checks, but the original out-of-bounds range is still returned as a validated citation. That breaks the tool’s file-line citation contract and can hand the parent agent ranges that cannot be read, so reject or clamp lineEnd after counting fileLines.
Useful? React with 👍 / 👎.
Code Review ResponsesCI Fix
Previously Fixed (from earlier review)
Codex P2 Comments — Status
|
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
from
3ecdbee to
b34a808
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: b34a8088df
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const result = await glob({ | ||
| pattern, | ||
| path: cwd, | ||
| hidden: false, |
There was a problem hiding this comment.
When the hint planner returns an explicit dotpath glob such as .github/workflows/* or .vscode/*, this call still runs the native glob with hidden: false, so the walker skips those hidden directories and returns no candidates even though the model asked for them. Queries about workflows, dotfiles, or hidden config can therefore fall through as “no files” unless a separate grep happens to match; enable hidden matching for explicit hidden components (and mirror that in the agent Glob path below) so these repository files can be discovered.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a200b9aa71
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const boundedStart = Math.max(1, Math.min(lineStart, lines.length)); | ||
| const boundedEnd = Math.max(boundedStart, Math.min(lineEnd, boundedStart + 200, lines.length)); | ||
| const snippet = lines.slice(boundedStart - 1, boundedEnd).join("\n"); | ||
| const haystack = `${resolvedPath}\n${snippet}`.toLowerCase(); |
There was a problem hiding this comment.
When the workspace path or one of its parent directories contains a query keyword (for example a checkout under /tmp/auth-service and a query containing auth), prepending resolvedPath to the keyword haystack makes any cited file under that directory pass the high-confidence check even if the cited lines contain none of the requested terms. That bypasses the retry/fallback path for irrelevant FastContext answers and can hand the parent agent misleading citations; match against the cited snippet and at most a workspace-relative filename component instead of the full absolute path.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: cb8afaa11d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| if (!plan) { | ||
| return this.#hintError( | ||
| apiBaseUrl, | ||
| model, | ||
| params, | ||
| "FastContext hint returned unparseable plan; falling back to native search.", | ||
| ); |
There was a problem hiding this comment.
When the hint model returns prose or a JSON object that lacks the expected arrays, this branch exits before the query-derived native grep/glob fallback below can run. That makes fast_context return an error even though queryKeywords(params.query) could still produce deterministic search candidates, so broad explore queries lose the advertised fallback on a common LLM formatting failure; treat an unparseable plan like an empty plan and continue into the native fallback instead.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ce16b3a21f
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| ? raw.keywords.filter(k => typeof k === "string").slice(0, HINT_MAX_KEYWORDS) | ||
| : [], | ||
| globs: Array.isArray(raw.globs) | ||
| ? raw.globs.filter(g => typeof g === "string").slice(0, HINT_MAX_GLOBS) |
There was a problem hiding this comment.
When the hint model emits globs: [""] or a whitespace-only placeholder, this filter keeps it because it only checks the value type. #nativeGlob treats an empty non-glob as the workspace root and recursively returns **/*, so a malformed/empty plan becomes a “successful” hint full of arbitrary files instead of falling back to query-derived grep; trim and discard empty glob entries before executing them.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e2a58e4108
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const earlyAnswer = extractFinalAnswer(response.message.content); | ||
| if (earlyAnswer.includes("<final_answer>")) { |
There was a problem hiding this comment.
When the local model returns a <final_answer> together with tool calls—especially on the forced final turn because #chat still advertises tools—this check never fires: extractFinalAnswer() has already stripped the tags, so earlyAnswer.includes("<final_answer>") is always false. The loop then executes the extra tool calls and can exit into the hint fallback after max_turns, discarding a valid cited final answer from agent mode.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0a0a90f3db
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // Cache resolved model — the model list doesn't change during a session | ||
| if (this.#resolvedModel) return this.#resolvedModel; | ||
| const response = await this.#fetch(`${apiBaseUrl}/models`, { signal: requestSignal(signal, REQUEST_TIMEOUT_MS) }); |
There was a problem hiding this comment.
When fastContext.model is unset and the user changes fastContext.baseUrl during a session, this returns the model id discovered from the previous endpoint and sends it to the new server. OpenAI-compatible servers such as LM Studio/Ollama commonly validate the model field, so switching from one local FastContext server to another can make subsequent calls fail until the process restarts; key the cache by apiBaseUrl or clear it when the URL changes.
Useful? React with 👍 / 👎.
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
3 times, most recently
from
0b7551c to
2b89d29
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 4b8e042eae
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const resolved = path.resolve(cwd, input?.trim() || "."); | ||
| if (!isWithinCwd(resolved, cwd)) { | ||
| throw new Error(`Permission error: \`${input}\` is not within the workspace \`${cwd}\`.`); | ||
| } |
There was a problem hiding this comment.
When the workspace contains a symlink such as linked-src -> /tmp/other-repo, this guard approves linked-src because the lexical path is under cwd, but the later fs.stat/Bun.file reads and recursive glob roots follow the symlink target. A hint or agent Read/Glob call can therefore return and cite files outside the workspace despite the permission check; resolve the real target path (and the real cwd) before accepting it.
Useful? React with 👍 / 👎.
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
from
4b8e042 to
2b89d29
Compare
…ition boost for filtered lower-camelCase identifiers. No code changes — documentation only. Benchmark stays 1.0 (grade A, 20/20), 32 tests pass, check passes. Stress probe: 8/10.
Result: {"status":"keep","precision_at_5":1,"avg_fc_latency_ms":131,"avg_fc_tokens":72,"fc_hits":20,"fc_nonfc_delta":0.85,"non_fc_baseline_p_at_5":0.15,"non_fc_hits":3,"precision_at_5_any":1,"total_cases":20}
…ction had zero call sites after run can1357#45 switched the definition boost to use identifierSet (which includes filtered lowerCamelCase). The stale comment claimed lowerCamelCase 'do NOT trigger the definition-site boost (see strongIdentifierKeywords)' — contradicted by the actual boost code at line ~1109 which uses identifierSet and explicitly boosts untilAborted. Comment now correctly states filtered lowerCamelCase DO trigger the boost after passing the dot/verb filters. No behavior change: precision_at_5 stays 1.0 (grade A, 20/20), 32 tests pass, bun check passes.
…st. Two changes: (1) #nativeGlob now expands directory matches to their immediate file children — glob can return directory paths (e.g. **/*provider* matches provider-models/ the directory, not index.ts inside it); only paths without extensions are stat'd to avoid 100+ stat calls per glob; extensionless files (Makefile, Dockerfile) are preserved. (2) Barrel boost +3 for index.ts/index.js files whose parent directory name contains a query keyword — barrel files have almost no content (just export * from), so they lose on content scoring; the parent-dir match mirrors how a human finds barrels. Feature benchmark: 17/18→18/18 (100%, grade A). Nonfc baseline: 1.0 (grade A, 20/20). 32 tests pass, bun check passes.
…e fs.readdir with glob({ pattern: "*", gitignore: true, hidden: pattern.startsWith(".") }) — same approach as the direct-path branch. fs.readdir returned ALL files including .env, .DS_Store, and gitignored files. No regression: feature benchmark 18/18 (100%), nonfc baseline 1.0 (grade A, 20/20), 32 tests pass, bun check passes.
… boost. (1) isWithinCwd now uses realpathSync to resolve symlinks before comparing — prevents workspace escape via symlinks pointing outside cwd. Uses sync realpath to avoid async ripple to all call sites. (2) Directory-path globs (**/agent/**/*) for identifier segments ≥5 chars — catches files with generic basenames (types.ts) that define CamelCase identifiers whose segments match a directory name. Put dirGlobs first in merge order so they survive the 200-file cap before generic segment globs flood it. (3) Directory-segment boost +2 during content scoring when any path component matches an identifier segment AND a definition-site match already fired — prevents false boosts on files that merely live in a matching directory. AgentTool defined in types.ts now ranks #1 (was not in top-5). Nonfc baseline: 1.0 (grade A, 20/20). Feature benchmark: 18/18 (100%). 32 tests pass, bun check passes.
…provider (devin/swe-1-6-slow) FastContext can now use any registered model provider instead of a locally-hosted llama.cpp server. Set fastContext.model to a provider-prefixed id (devin/swe-1-6-slow, zai/glm-5-turbo, pi/smol) and FastContext resolves it through the model registry via completeSimple — no local endpoint needed. A bare id or blank keeps the existing local /chat/completions path. - FastContextBackend discriminated union (local | registry) with #resolveBackend - #chat dispatches to #chatLocal (raw fetch) or #chatRegistry (completeSimple) - converters: fcMessagesToContext (system->Context.systemPrompt; Devin consumes it), fcToolsToOmpTools, assistantToFcResponse (ToolCall.arguments Record<->JSON string) - completeFn constructor injection (testable without network), mirrors inspect-image - stable cascadeId per agent-loop invocation so Devin threads/caches across turns - registry-aware hint timeout (cloud/reasoning models need >30s budget) - verified end-to-end with devin/swe-1-6-slow on a free account: hint 8/8 (100%), agent 8/8 (100%) on the 8-case retrieval bench
…engthen parallel-tool-call prompt Benchmarked the three SWE-1.6 tiers on the 8-case retrieval bench: - devin/swe-1-6-fast: 100% @ 1.6s hint / 3.3s agent (Cerebras 950 tok/s, same weights as swe-1-6) - devin/swe-1-6: 88% @ 3.2s hint / 7.0s agent (Fireworks 200 tok/s) - devin/swe-1-6-slow: 100% @ 34s hint / 127s agent (reasoning-heavy) swe-1-6-fast is faster than a local 4B model AND more accurate, with no local GPU. Updated docs to recommend it. Strengthened the agent system prompt's parallel-tool-call directive (Cognition blogs note SWE-1.6 trends sequential) — harmless, 100% maintained, no measurable turn-count change on this bench.
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
from
9877ec9 to
659afe0
Compare
…AX_READ_LINES) Swept 200/400/600 with devin/swe-1-6-fast on fast-context-tool-definition and read-only-subagent-classification (agent mode): all 100% at ~3.4s, no measurable difference. Default stays 200 (protects local-model latency) but is now tunable for cloud/fast users who want more read headroom at no latency cost.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: be9ca41ab2
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| // "local" sentinel = explicit local server (auto-discovers the model via | ||
| // /v1/models). Anything else containing "/" is a provider-prefixed id that | ||
| // routes through the registry. | ||
| const configured = raw === "local" ? undefined : raw || undefined; |
There was a problem hiding this comment.
When fastContext.model is set to the local sentinel and Devin credentials are present, this collapses the explicit choice to undefined; the auto-default branch below then treats it as an unset model and, if devin/swe-1-6-fast resolves, routes through Devin and persists that provider model. That makes the “Local llama.cpp server” picker choice ignore the configured local endpoint for logged-in Devin users, so preserve whether raw was local before applying the auto-default.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 9f0ff6e2d1
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| return result.matches | ||
| .map(m => m.path) | ||
| .filter((p): p is string => Boolean(p)) | ||
| .map(p => formatAbsolute(p, resolveWorkspacePath(sp, cwd))); |
There was a problem hiding this comment.
When the workspace contains a symlinked file such as link.ts -> /tmp/secret.ts, a hint-mode grep over . accepts only the search root and then returns each native match without a realpath containment check. #readSnippets later reads those returned paths with Bun.file, which follows the symlink, so FastContext can send/cite file contents outside cwd; filter each absolute grep hit with the existing realpath-based isWithinCwd guard before returning it, and apply the same guard to agent-mode Grep output.
Useful? React with 👍 / 👎.
| const lineEnd = Number(match[3] ?? match[2]); | ||
| // Reject invalid line ranges before clamping (PR review: should-fix) | ||
| if (lineStart < 1 || lineEnd < lineStart) continue; | ||
| const fileText = await Bun.file(resolved).text(); |
There was a problem hiding this comment.
When agent mode cites a very large file, for example a lockfile or generated bundle with a short :1-2 range, validation still loads and decodes the entire file here before any size/range cap is applied. That can stall or OOM the CLI just to count lines or check keywords; stat/reject oversized files or read only the bounded prefix needed for the cited range before calling text().
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a7b51905c6
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| }, | ||
|
|
||
| "fastContext.mode": { | ||
| type: "string", |
There was a problem hiding this comment.
With this declared as a free string, omp config set fastContext.mode hnit is accepted instead of being rejected like other fixed-choice settings; FastContextTool.execute() then treats any non-"hint" value as agent mode, so a typo silently switches explore from the fast hint path to the slow multi-turn loop. Make this an enum with hint/agent values or normalize invalid settings back to the default before dispatch.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: a36b1fc264
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
|
|
||
| If you don't want to run a local model, point FastContext at any registered provider model by setting `fastContext.model` to a provider-prefixed id. FastContext resolves it through the model registry using your configured credentials and calls it directly — no llama.cpp/LM Studio/Ollama required. | ||
|
|
||
| ```bash |
There was a problem hiding this comment.
This opens a ```bash fence for the provider-model example but never closes it, so the explanatory paragraph and the rest of the FastContext guide render as part of the code block. In rendered docs, the “How it works” and troubleshooting sections stop being headings/lists, making the setup page hard to follow; add a closing fence after the command snippet.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 01bbf01c72
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const safe = replaceTabs(citation); | ||
| const target = parseCitationTarget(citation); | ||
| return fileHyperlink(target.filePath, safe, target.line ? { line: target.line } : undefined); |
There was a problem hiding this comment.
When FastContext returns agent-mode citations, the prompt asks for absolute paths, and hint-mode stores absolute paths in details.citations; this renderer only replaces tabs before using the citation as the visible label. In workspaces under a user's home directory, the inline FastContext card will print the full home path, violating the repo's TUI sanitization rule to shorten displayed paths and leaking local filesystem details; keep the hyperlink target intact but render a shortenPath-sanitized label.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 6eda3ae32d
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const includeSnippets = this.#session.settings.get("fastContext.snippets") ?? params.include_snippets ?? true; | ||
| const snippetLines = Math.min( | ||
| Math.max( | ||
| this.#session.settings.get("fastContext.snippetLines") ?? | ||
| params.snippet_lines ?? | ||
| HINT_DEFAULT_SNIPPET_LINES, |
There was a problem hiding this comment.
With default settings, Settings.get("fastContext.snippets") and Settings.get("fastContext.snippetLines") return the schema defaults (true and 10), so the nullish chain never reaches explicit tool arguments like include_snippets: false or snippet_lines: 3. Any caller trying to request a compact hint result still gets 10-line snippets, which can consume the result token budget and drop lower-ranked files despite the tool args; distinguish explicit user config from defaults or let the per-call args take precedence.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: e74ccd9302
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| options = this.context.availableFastContextModels ?? [{ value: "local", label: "Local llama.cpp server" }]; | ||
| const raw = (settings.get("fastContext.model") || "").trim(); | ||
| selectedValue = raw && options.some(o => o.value === raw) ? raw : (options[0]?.value ?? "local"); |
There was a problem hiding this comment.
When a user has configured any valid provider model outside the curated picker list (the docs mention arbitrary provider-prefixed ids such as openai-codex/gpt-5.5), opening this submenu preselects options[0] because the raw value is not present in availableFastContextModels. Pressing Enter then overwrites the custom model with the first curated/default option, unexpectedly rerouting FastContext; include the current raw value in the options or keep it selected instead of falling back.
Useful? React with 👍 / 👎.
…nt, latency optimizations, and live GLM plan evaluation Ranking improvements (fast-context.ts #executeHint): - Fix boostedSorted to sort by final multiplied score instead of raw contentScore — was bypassing 0.3x test/doc/script penalty (MRR 0.70→0.86, dominant fix) - Add multi-signal convergence boost: +3 per signal beyond the first (plan-glob + plan-grep + supp-glob), added to contentScore BEFORE type multiplier (MRR 0.9444→0.9475, fixes conversation-context can1357#7→#1) - Add plan-symbol definition boost with line-start anchor, case-sensitive matching, export requirement - Add path-aligned class-name boost (≥8-char keyword threshold) for natural-language queries - Add plan-glob specificity sort + re-injection after 200-file cap - Add plan-glob-matched tiebreaker to boostedSorted - Add config/data file penalty (.json/.yaml/.toml = 0.7x typeMultiplier) - Add keyword-derived directory globs (≥6-char keywords) - Expand isScript penalty to include examples/, bench/, prompts/ Prompt improvements: - Rewrite fast-context.md to eliminate read-reversion - Improve fast-context-hint-system.md with glob specificity guidance - Improve fast-context-system.md with read discipline - Fix read-reversion in system-prompt.md and explore.md - Lower hint temperature to 0.0 for deterministic planning Latency optimizations: - max_completion_tokens 2048→512 - MAX_WORKSPACE_LISTING 60→30 - supplementaryGrepKws 2→1 Benchmark infrastructure: - bench-fast-context-retrieval.ts: deterministic benchmark (27 queries, 13 metrics, Microsoft-style F1) - bench-fast-context-live-glm.ts: live GLM plan evaluation via completeSimple Results: MRR 0.70→0.9475, 25/27 queries at #1. All 18 FC tests pass, bun check passes.
oldschoola
force-pushed
the
fastcontext-explore-adapter
branch
from
e74ccd9 to
f37b603
Compare
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: f37b603717
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| - Read("src/auth/handler.ts", start=10, end=80) | ||
| - Read("src/auth/middleware.ts", start=1, end=50) |
There was a problem hiding this comment.
In agent mode, this example teaches the FastContext model to call Read with start/end, but the tool schema and #readFile only consume offset and limit; unknown fields are ignored, so following this example reads from line 1 (capped by maxReadLines) instead of the requested range. For large files where the relevant symbol is below the cap, FastContext can synthesize from the wrong excerpt or miss the evidence; align the prompt/schema or accept start/end.
Useful? React with 👍 / 👎.
| { | ||
| pattern, | ||
| path: resolveWorkspacePath(sp, cwd), | ||
| ignoreCase: true, |
There was a problem hiding this comment.
When the hint planner emits an exact CamelCase/symbol grep_patterns entry, the hint prompt says those names are case-sensitive, but this forces native grep to ignore case. For symbols like Message, URL, or other common mixed-case names, lowercase matches can consume the MAX_TOOL_LINES cap with unrelated files and push the actual definition out of the candidate pool; keep plan symbol greps case-sensitive, or only use ignore-case for query-derived fallback greps.
Useful? React with 👍 / 👎.
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 941c10fc90
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| const defPattern = new RegExp( | ||
| `^\\s*${defKeywords}\\s+[a-z_]*${id.replace(/[.*+?^${}()|[\]\\]/g, "\\$&")}`, | ||
| "m", | ||
| ); | ||
| if (defPattern.test(rawText)) { |
There was a problem hiding this comment.
When the query contains a CamelCase or UPPER_SNAKE identifier such as FastContextTool or READ_ONLY_TOOL_NAMES, identifierKeywords() lowercases it, but this regexp is tested against the original rawText without the i flag. As a result the intended +8 definition-site boost never fires for the mixed/upper-case symbols this block is meant to help, so reference-heavy importers or tests can outrank the actual definition file; match against the lowercased text or preserve the original identifier casing for this boost.
Useful? React with 👍 / 👎.
| const stat = await fs.stat(filePath).catch(() => null); | ||
| if (!stat) return `Read Tool: file ${args.path} does not exist.`; | ||
| if (!stat.isFile()) return `Read Tool: ${args.path} is not a file.`; | ||
| const rawLines = splitFileLines(await Bun.file(filePath).text()); |
There was a problem hiding this comment.
When agent mode follows the prompt and calls Read with a narrow offset/limit on a large lockfile, generated bundle, or JSON artifact, this still decodes the entire file before applying fastContext.maxReadLines. That can stall or OOM a FastContext turn even though only a bounded line window is returned; stat/reject oversized files or read/scan only the needed byte range before splitting lines.
Useful? React with 👍 / 👎.
…p=0 with invalid_argument
FastContext hint mode passes temperature: 0 for deterministic planning, but
the Devin agent API rejects temperature: 0 with invalid_argument, causing
every devin/swe-1-6-fast hint-mode query to fail silently (stopReason:error,
empty content, 0% plan_parse_rate).
Root cause: the Devin adapter (buildDevinChatRequest) passed the caller's
temperature through without clamping. The API's Connect trailer carries:
{"error":{"code":"invalid_argument","message":"an internal error occurred"}}
Fix: extract resolveDevinTemperature() which clamps to 0.01 floor, and apply
it to both temperature and firstTemperature in the CompletionConfiguration.
Also made toolChoice:"auto" conditional on tools being present — sending it
with an empty tools array is a redundant request shape.
Bench (devin/swe-1-6-fast, original params temp=0 maxTokens=512):
plan_parse_rate: 0% → 100%
MRR: 0.7073 → 0.8117
hit_at_5: 0.8148 → 0.9259
plan_glob_hit: 0% → 79.6%
plan_grep_hit: 0% → 92.6%
plan_keyword: 0% → 100%
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0997238f68
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| lowConfidenceCitations.push(line); | ||
| continue; | ||
| } | ||
| citations.push(line); |
There was a problem hiding this comment.
When the model formats citations as bullets or code spans (for example - /repo/src/a.ts:10-12 or `/repo/src/a.ts:10-12`), the regex captures the clean path but this stores the entire raw line in both citation arrays. Those strings later feed the FastContext renderer and parent result details as if they were path:range, so hyperlinks/readable citations include the leading bullet/backtick and point at invalid paths; push a normalized value from the captured path and parsed range instead.
Useful? React with 👍 / 👎.
| const line = rawLine.trim(); | ||
| if (!line || /https?:\/\//i.test(line)) continue; | ||
| const match = line.match( | ||
| /(?:^|[`*\s-])([A-Za-z]:[\\/][^`\n]+?|[\\/][^`\n]+?|(?:\.{1,2}[\\/]|[A-Za-z0-9_.-]+[\\/])[^`\n]+?|[A-Za-z0-9_.-]+\.[A-Za-z0-9][A-Za-z0-9_.-]*):(\d+)(?:[-–—](\d+))?\b/, |
There was a problem hiding this comment.
For valid final answers that cite root-level extensionless files such as Makefile:4-12, Dockerfile:1-20, or LICENSE:1-3, this pattern does not match: single-component relative paths are only accepted when they contain a dot extension, while slash-containing paths are handled separately. Agent mode will therefore discard an otherwise correct citation and retry or fall back with “no file-line citations”; allow top-level filenames without extensions before the :<line> suffix.
Useful? React with 👍 / 👎.
…CHARS env var aliases + token savings & honesty benchmarks Settings alignment with SammySnake-d/fast-context-mcp reference MCP: - Added FC_MAX_TURNS, FC_RESULT_MAX_LINES, FC_LINE_MAX_CHARS env var aliases (falling back to omp defaults when unset) - Documented each constant inline with the reference MCP value and rationale for the difference (hint+agent modes, in-process execution, ranked snippets) - All 6 claimed mismatches adversarially verified as intentional architectural differences (not bugs) New benchmark: bench-fast-context-token-savings.ts (570 lines): - FC hint vs simulated no-FC agent path (multi-round search→read→grep) - FC: 2,130 tokens avg, MRR 0.95, hit@1 93% - No-FC: 9,856 tokens avg, MRR 0.54, hit@1 37% - Token savings: 78.4% - Inspired by agent-retrieval-eval (ashikshafi08) methodology New benchmark: bench-fast-context-honesty.ts (522 lines): - Grep-certified citation verification (inspired by determinacy eval) - Phantom citation rate: 0% (every cited file exists) - Citation existence rate: 100% (540/540) - Keyword verification rate: 94.9% (169/178) - False negative rate: 0% (FC finds ALL ground truth files) - Line range valid rate: 100%
…ation The FC_MAX_TURNS, FC_RESULT_MAX_LINES, FC_LINE_MAX_CHARS env vars were read via Bun.env at module load — invisible to the /settings UI and immutable at runtime. The existing maxReadLines setting shows the correct pattern (settings.get() ?? env ?? default). Reverted the three env vars to plain documented constants. The inline documentation explaining the reference MCP alignment rationale remains.
Agent mode max turns is now configurable via /settings → Context → Fast Context.
Options: 1-8 turns (default 4). The tool reads from settings.get('fastContext.maxTurns')
falling back to the hardcoded DEFAULT_MAX_TURNS=4. Per-call params.max_turns still
overrides. Hint mode ignores this (always 1 turn).
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 0d665ede44
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
| icon: "pending", | ||
| title: "FastContext", | ||
| titleColor: "toolTitle", | ||
| description: args.query || (args.mode ? `${args.mode} mode` : ""), |
There was a problem hiding this comment.
When a user query contains tabs or a long pasted prompt, this pending render path sends args.query directly into the header; renderStatusLine only flattens newlines and explicitly leaves tabs/width handling to callers. That violates the root AGENTS.md TUI sanitization rule and can create visual holes or overflow while the fast_context call is streaming; run the query through replaceTabs() and a preview-width truncation helper before using it as the description.
Useful? React with 👍 / 👎.
2 participants
Updates since first review (
f37b6037)0997238f6):devin/swe-1-6-fastpreviously showed 0% plan_parse_rate in the live bench — the Devin agent API rejectstemperature: 0withinvalid_argument. The adapter now clamps to 0.01 floor. After fix: 100% parse rate, MRR 0.71→0.81, hit_at_5 0.81→0.93. Also madetoolChoice:"auto"conditional on tools being present.SammySnake-d/fast-context-mcp): Audited all FastContext constants against the reference MCP server's env vars (FC_MAX_TURNS, FC_MAX_COMMANDS, FC_TIMEOUT_MS, FC_RESULT_MAX_LINES, FC_LINE_MAX_CHARS, max_results, tree_depth). All differences are intentional — omp's richer architecture (hint+agent modes, in-process execution, ranked snippets) justifies the higher defaults. Constants documented inline with rationale.bench-fast-context-token-savings.ts): Rigorous FC-on vs FC-off comparison measuring actual token consumption. FC hint mode: 2,130 tokens avg per query. Simulated no-FC path (multi-round search→read→grep): 9,856 tokens avg. 78.4% token savings. FC also wins on quality: MRR 0.95 vs 0.54, hit@1 93% vs 37%.bench-fast-context-honesty.ts): Grep-certified verification that FC citations are real. 0% phantom citation rate (every cited file exists on disk). 100% citation existence rate. 95% keyword verification rate (keywords appear in cited files). 0% false negative rate (FC finds ALL ground truth files). 100% valid line ranges. Inspired by determinacy eval's mechanical-spine approach.Head-to-head: with vs without FastContext
devin/swe-1-6-fast)Before & After: improvement summary
Ranking pipeline (deterministic bench, 27 queries)
941c10fc9)Retrieval quality (original benchmark, 8 queries, local RL model)
Efficiency (tokens + latency)
devin/swe-1-6-fast)max_tokens2048→512, listing 60→30, supp grep 2→1)Non-FC baseline vs FC ranking (pure grep, no model plan)
Feature-specific benchmark (18 cases, 10 ranking features — each designed to fail without the target feature)
First-class main-agent tool —
fast_contextis available to the main agent (not justexplore), gated byfastContext.enabled. The system prompt directs it as the FIRST action for codebase-retrieval questions, and to use its file:line citations directly (read the cited ranges; don't re-runsearch/find/grep/globto re-discover files it already returned). Verified by a gate test + a system-prompt rendering test.Inline TUI rendering — results render inline like
find(framed file/citation list, ⚡icon.fastheader, no collapsed ctrl+o window) via a registeredfastContextToolRenderer(inline+mergeCallAndResult). Agent mode no longer leaks raw<final_answer>tags (routed throughextractFinalAnswerat the source).TUI badge — when a subagent calls
fast_context, its task card shows⚡ fast_context · {model} · {calls} call(s) · {files} files(live + rebuilt views), aggregated from structuredFastContextToolDetails.New settings (
/settings→ Context → Fast Context):fastContext.mode(Hint/Agent),fastContext.fastTools(forces agent mode = SWE-grep-style parallel Read/Glob/Grep, ≤4 turns, up to 8 parallel calls),fastContext.snippets(on/off),fastContext.snippetLines(3–30),fastContext.maxReadLines(agent per-file cap, 100–2000), plus a model picker (devin/swe-1-6-fast/devin/swe-1-6/devin/swe-1-6-slow/zai/glm-5-turbo/pi/smol+ local server) and a conditionalbaseUrl(shown only for the local backend). Settings are authoritative — the model's reflexive per-call defaults no longer override the user's configured values.Ranking pass (
f37b6037) —boostedSortedsorts by the final multiplied score; multi-signal convergence boost; plan-symbol definition boost; path-aligned class-name boost; plan-glob specificity sort + re-injection; config/data-file penalty; keyword-derived directory globs.Deterministic benchmark (
bench-fast-context-retrieval.ts, 27 queries, mocked plans, no network — run just now):Live GLM plan evaluation (
bench-fast-context-live-glm.ts, 27 queries, realzai/glm-5-turboplans — run just now):Reproducibility: MRR (0.9475), hit_at_5 (1.0), citation_format_valid (1.0), and plan glob/grep hit rates are stable across repo states.
noise_ratio_top10scales with file count (0.073 at the original baseline → 0.119 after this session's additions) — report with the commit it was measured against. Both benches run against commit941c10fc9.Multi-model plan quality comparison (3 fresh runs):
zai/glm-5-turbo(live)devin/swe-1-6-fast(live)Benchmark infra —
scripts/bench-fast-context-retrieval.ts(deterministic 27-query, 13-metric, Microsoft-style F1),scripts/bench-fast-context-live-glm.ts(live GLM plan evaluation),scripts/bench-fast-context-token-savings.ts(FC-on vs FC-off token/quality comparison), andscripts/bench-fast-context-honesty.ts(grep-certified citation verification). All emitMETRIC name=valuelines for automated parsing.Token Savings benchmark (
bench-fast-context-token-savings.ts, 27 queries, FC hint vs simulated no-FC agent path):Honesty Audit (
bench-fast-context-honesty.ts, 27 queries, grep-certified citation verification):Summary
exploresubagent and the main agent.<final_answer>citation validation (SWE-grep-style: up to 8 parallel tool calls per turn, ≤4 turns)exploresubagent (opt-in viafastContext.enabled); also callable directly by the main agentdocs/fast-context.md— LLM-actionable, follows step-by-stepNew: cloud model via devin/swe-1-6-fast (no local server)
FastContext now routes through any registered model provider — set
fastContext.modelto a provider-prefixed id (e.g.devin/swe-1-6-fast) and it resolves via the model registry, no llama.cpp needed. Benchmark on the 8-case cross-package retrieval bench (strict precision@5), live against this repo:devin/swe-1-6-slowdevin/swe-1-6devin/swe-1-6-fastswe-1-6-fastis the same SWE-1.6 weights on Cerebras @ 950 tok/s ("same intelligence") — faster than the local 4B model AND more accurate, with no GPU.swe-1-6-slowis reasoning-heavy (thinking always-on; the Devin provider ignoresdisableReasoning, so its latency is reasoning-bound). Usage:omp config set fastContext.model devin/swe-1-6-fast(login:/login devin). Any provider model works (zai/glm-5-turbo,openai-codex/gpt-5.5,pi/smol, ...). When unset and Devin is logged in,devin/swe-1-6-fastis auto-selected.MAX_READ_LINES: swept 200/400/600 with
swe-1-6-fastonfast-context-tool-definition+read-only-subagent-classification— all 100% at ~3.4s (no measurable difference). Default stays 200 (protects local-model latency; withswe-1-6-fastread budget isn't the bottleneck); now also a UI setting (fastContext.maxReadLines, 100–2000) and env-tunable viaFC_MAX_READ_LINES.The scores below were measured with the local FastContext-1.0-4B model and remain valid for that path; the cloud model improves on them (see notes).
Evaluation Scores
Three evaluation scripts dispatched as parallel subagents. Results below.
1. Delegated Repository Exploration Score
Measures retrieval quality when FastContext is used as a delegated exploration tool (hint mode — the default path through the explore subagent).
Score: A — Hint mode is the clear winner for delegated exploration: 95% hit rate at 2.5s latency with a 73-token packet. Agent mode trades 10× latency for no retrieval improvement on these cases.
2. Main-Agent Token Savings Score
Measures how many tokens the main agent saves by using FastContext instead of manual search/read/grep. Scenario A = real native glob+grep+read calls. Scenario B = FastContext hint packet + 50 reasoning tokens.
Score: A+ — FastContext saves the main agent ~95% of tokens vs manual exploration. The biggest savings come from avoiding file reads (explore-agent-tools: 175K read tokens → 1.7K packet).
3. Standalone Exploration Score (before ranking optimization)
Measures FastContext used directly as a tool call (not through the explore subagent).
Score: C — Under strict precision_at_5 (expected file must appear in top 5), all three baselines scored identically before the ranking optimization. The LLM model plan added zero retrieval quality over pure query-derived grep on these 8 cases. The retrieval quality came from the ranking pipeline (path scoring + content scoring + grep/glob boost), not from the model plan.
4. FastContext vs Non-FastContext Baseline (before ranking optimization)
Answer: No — the FastContext-1.0-4B-RL model did not improve retrieval quality over the pure query-derived grep fallback path. Both scored 62.5% on strict precision_at_5. The model plan added ~2.4s latency and ~15× token cost for zero retrieval gain. The ranking pipeline did all the heavy lifting.
5. Additional Performance Opportunities
From the performance profiling analysis (
packages/coding-agent/scripts/fast-context-perf-analysis.md):Hard floor: ~1.3-1.5s (LLM compute-bound, cannot reduce without a smaller/faster model)
Projected savings: ~200-400ms (12-17% reduction) from all non-LLM optimizations
Counterintuitive findings:
Bun.file().text().slice(0, 1000)— content ranking is already 0.5-3.5ms for 30 filesSummary Scorecard
Ranking Optimization (autoresearch session)
Precision@5 improved from 0.75 (grade B) → 1.0 (grade A) on the 20-case non-FC baseline benchmark. FC vs non-FC delta improved from 0.60 → 0.85. Stress probe improved from 6/10 → 8/10.
Key techniques implemented (semble_rs-inspired)
FastContext,GrepOutputMode,TempDir,streamSimple,isEnoent,untilAbortedas 3x-weighted identifiersbaseUrlinfastContext.baseUrl), dot-followed (fastContextinfastContext.enabled), verb-position (applyGeneratedModelPolicies sets)class/enum/function/struct+ queried identifier get +8 content score**/*temp*); prefix globs for segments ≥6 chars (aborted→**/*abort*→abortable.ts)function,class,enum,interface,struct,const,exportfiltered from query keywords#nativeGlobexpands directory matches to immediate file children — glob can returnprovider-models/(dir) instead ofindex.tsinside it; only paths without extensions are stat'dindex.ts/index.jswhose parent directory name contains a query keyword get +3 — barrel files have near-zero content (justexport * from), so they lose on content scoring without this**/agent/**/*for identifier segments ≥5 chars — catches files with generic basenames (types.ts) whose identifier segments match a directory name, not the filename; placed first in merge order to survive the 200-file capisWithinCwdusesrealpathSyncto resolve symlinks before comparing — prevents workspace escape via symlinks pointing outside cwdNon-FC baseline benchmark results
Feature-specific benchmark (18 cases, 10 features)
Each case is designed to fail WITHOUT the target feature. Runs both FC ranking (real production
#executeHintwith mock fetch → fallback) and raw grep baseline.All 10 features achieve 100% hit rate. The directory expansion + barrel boost fixed the last remaining miss (
provider-models/index.tsbarrel file).Original Benchmark (oh-my-pi repo, 8 cross-package queries, FastContext-1.0-4B-RL-Q4_K_M GGUF)
3 consecutive RL hit rate runs: 8/8, 8/8, 8/8 = 24/24 = 100%
Real model validation (FastContext-1.0-4B-RL at localhost:8080)
5 consecutive runs after final fix:
Average: 38/40 = 95% (up from 37.5% baseline)
Agent mode latency optimization
Four changes, in order of impact (35.1s → 24.4s avg, -31%):
MAX_READ_LINES: 2000 → 200 (primary driver) — agent-mode Read calls were flooding context with 2000 lines per file; 200 is enough to understand a section and drastically cuts prefill per turn.<final_answer>alongside tool calls, parse citations and exit immediately instead of running the extra tool calls. Saves 1+ LLM round-trips./v1/modelsresult in a#resolvedModelfield so repeated#resolveModel()calls do not re-fetch over HTTP each turn.max_completion_tokens32K → 2K (tool turns) / 4K (final answer); temperature 1 → 0.3 (matching hint mode);DEFAULT_MAX_TURNS6 → 4. (Temperature 0.1 tested and rejected.)Code Review Responses
CI Fixes
readonly loadMode = "discoverable"toFastContextTool— fixes theinitial-tools.test.tsfailure wherefast_contextwas missing from theBUILTIN_TOOLSmetadata map."fastContext.enabled": trueto the test settings socreateToolsinstantiatesfast_context.## [16.1.9](released) to## [Unreleased].Blocking Issues Fixed
tool_callsin saved assistant message to match bounded calls#nativeGlobchecksisWithinCwdbefore resolving direct paths.error().useless()instead of successstart<1,end<start,start>lineCountSettings
Test plan
bun --cwd=packages/coding-agent test test/fast-context-tool.test.ts test/fast-context-render.test.ts test/task/fast-context-badge.test.ts test/system-prompt-fast-context.test.ts— 30 passbun --cwd=packages/coding-agent run check— passed (biome + typecheck)devin/swe-1-6-fast: hint 100% (8/8) @ ~1.6s; agent 100% @ ~3.3somp-dev --smoke-testokfast_contextis a first-class main-agent tool (gate test); system-prompt directive renders only when active (rendering test)Files
Additional Benchmarks (session 2)
Token Savings Benchmark (
bench-fast-context-token-savings.ts)Rigorous FC-on vs FC-off comparison. Same 27-query suite, measuring actual token consumption through both paths. FC hint uses mocked fetch (deterministic). No-FC path simulates a multi-round search→read→grep agent trajectory with realistic token estimates per round.
Run:
bun packages/coding-agent/scripts/bench-fast-context-token-savings.tsHonesty Audit (
bench-fast-context-honesty.ts)Grep-certified verification that FastContext citations are real — inspired by the determinacy eval's mechanical-spine approach. Every cited file is checked for existence on disk, keyword presence, and valid line ranges.
Run:
bun packages/coding-agent/scripts/bench-fast-context-honesty.tsSettings Alignment with Reference MCP (
SammySnake-d/fast-context-mcp)Audited all FastContext constants against the reference MCP server's env vars. All differences are intentional — omp's richer architecture (hint+agent modes, in-process execution, ranked snippets) justifies the higher defaults. Constants documented inline with rationale.
New UI setting:
fastContext.maxTurns(1–8 turns, default 4) under/settings→ Context → Fast Context.