/goal auto-compaction still does not trigger after #3147 with strategy: context-full

[luceat-lux-vestra]

Description

After updating to the release that includes #3147 , auto-compaction still does not trigger during /goal mode in my environment.
I am using compaction.strategy: context-full, not handoff.
The session exceeded the configured fixed threshold:
compaction:
enabled: true
strategy: context-full
thresholdTokens: 76384
thresholdPercent: -1
keepRecentTokens: 10000
reserveTokens: 16384
autoContinue: true
supersedeReads: true
dropUseless: true
idleEnabled: false
remoteEnabled: false
The visible context/token usage went over 90k tokens, which is higher than thresholdTokens: 76384, but auto-compaction did not run.
Manual compaction works, but threshold-based auto-compaction in /goal mode does not appear to trigger.
I also tried a local change to make the successful-yield branch always call #checkCompaction(...) before returning, but the issue still reproduced. So this may not be only the early-return path fixed by #3147.

Steps to Reproduce

Update to a release that includes #3147 .
Use the following config:
loop:
mode: compact

compaction:
enabled: true
strategy: context-full
thresholdTokens: 76384
thresholdPercent: -1
keepRecentTokens: 10000
reserveTokens: 16384
autoContinue: true
supersedeReads: true
dropUseless: true
idleEnabled: false
remoteEnabled: false
Start a long-running /goal session.
Let the session continue until the visible context/token usage exceeds 76384 tokens.
In my case, the session exceeded 90k tokens.
Observe whether threshold auto-compaction runs.

Expected Behavior

Once the active /goal session exceeds compaction.thresholdTokens, auto-compaction should run automatically.
With the config above, auto-compaction should become eligible once the session exceeds 76384 tokens.
Since the session reached over 90k tokens, I expected context-full auto-compaction to run before the goal continued further.

Actual Behavior
The /goal session continued past the configured threshold.
No automatic compaction appeared to run, even after the visible context/token usage exceeded 90k tokens.
Manual compaction still works.

Error Output

Platform

macOS

omp version

16.1.9

Bun version

1.3.14

Provider

None

Area

None

Additional context

No response

[roboomp]

Looking into this. Will trace the /goal compaction path beyond the early-return fix in #3147 and report back.

[roboomp]

Opened #3175. Root cause was #checkCompaction subtracting the per-turn supersede/drop-useless tokensSaved from the threshold input — with dropUseless: true + supersedeReads: true, a goal loop with large tool outputs trivially saved enough to drop the input below thresholdTokens while the user-visible context (anchored to the same provider billing) sat well above it, so shouldCompact silently no-op'd. The fix anchors the threshold check on the actual last-turn billed context, keeping the compactionContextTokens floor against the post-prune local estimate so a payload-compression hook still can't deflate the trigger.

[luceat-lux-vestra]

@roboomp

Update after testing #3175 / b74f7bb:

I built and tested the farm/fc89a4e3/goal-auto-compaction-still-not-triggering branch, including commit b74f7bb, but the issue still reproduces in my real /goal session.

So #3175 does not fully fix this issue in my environment.

Config:

compaction:
  enabled: true
  strategy: context-full
  thresholdTokens: 76384
  thresholdPercent: -1
  keepRecentTokens: 10000
  reserveTokens: 16384
  autoContinue: true
  supersedeReads: true
  dropUseless: true
  idleEnabled: false
  remoteEnabled: false

Observed behavior:

• Visible context/token usage still goes above 90k.
• thresholdTokens is 76384.
• Automatic compaction still does not appear to run.
• Manual /compact still works.
• This is strategy: context-full, not handoff.

Could we keep this issue open after #3175? The post-prune token accounting fix may be necessary, but it does not appear to cover the remaining real /goal auto-compaction path.

At this point, I think the next useful step is runtime diagnostics around the actual decision path, specifically whether the real session reaches:

1. #checkCompaction(...)
2. shouldCompact(...)
3. #tryContextPromotion(...)
4. #runAutoCompaction(...)
5. prepareCompaction(...)

If auto_compaction_start is not emitted, then the failure is probably still before #runAutoCompaction() — either the /goal path is not reaching the threshold check, the internal resolved token count is below the visible context count, context promotion is taking over, or a post-turn maintenance guard is returning early.

[roboomp]

Yes — keep this open until the latest #3175 branch is tested against the real /goal session.

The b74f7bb build only had the post-prune token accounting fix. After your result, I pushed a follow-up on the same PR branch:

• 8ab754f636: active /goal text stops now run threshold maintenance before empty/unexpected-stop retry continuations can return from agent_end.
• d444b3c479: formatter-only follow-up from the publish gate.

That update also adds the requested runtime diagnostic log before the threshold decision:

Auto-compaction threshold decision

It includes the strategy, stop reason, context window, resolved threshold, provider-billed context tokens, stored-context estimate, resolved trigger tokens, post-maintenance tokens, prune savings, shouldCompact, and context-promotion enabled state.

If the latest branch still reproduces and there is no auto_compaction_start, the next useful artifact is either that log line or confirmation it never appears. If it appears, it will tell us whether shouldCompact is false, context promotion took over, or the resolved token count differs from the visible status-line count. If it never appears, the remaining bug is before #checkCompaction() is reached.

[luceat-lux-vestra]

@roboomp

Additional test result:

I lowered the threshold significantly to make the issue easier to reproduce:

compaction:
  enabled: true
  strategy: context-full
  thresholdTokens: 32768
  thresholdPercent: -1
  keepRecentTokens: 10000
  reserveTokens: 16384
  autoContinue: true
  supersedeReads: true
  dropUseless: true
  idleEnabled: false
  remoteEnabled: false

Then I started a new /goal run for testing.

The visible context/token usage is now over 70k tokens, which is more than 2x the configured thresholdTokens: 32768, but auto-compaction still did not run.

More importantly, I also do not see the newly added Auto-compaction threshold decision log at all.

So this no longer looks like only a threshold calculation issue. If the diagnostic log is placed inside the threshold decision path, then the real /goal runtime path does not seem to be reaching that decision path at all.

Current observation:

• strategy: context-full
• thresholdTokens: 32768
• visible context/token usage: 70k+
• no auto-compaction
• no auto_compaction_start
• no Auto-compaction threshold decision log

This suggests the remaining issue is likely before shouldCompact(...) / threshold calculation, possibly in the /goal post-turn maintenance path or whatever decides whether #checkCompaction(...) should be called for goal continuations.

[roboomp]

Thanks for the new test. The Auto-compaction threshold decision log lives inside #checkCompaction's threshold branch, so its absence is consistent with either (a) #checkCompaction never being reached or (b) you not looking in the log file (the omp logger is winston-level debug, but it writes to ~/.omp/logs/omp.<DATE>.log, never the TUI).

I pushed e70c71077e to the same PR branch with an agent_end maintenance routing debug log at every branch of the post-turn handler:

• entered, no-assistant-message
• skip-post-turn-maintenance
• successful-yield-active-goal-checkCompaction / successful-yield-no-active-goal
• empty-stop-handled
• active-goal-pre-empt-checkCompaction / active-goal-pre-empt-continuation-scheduled
• unexpected-stop-handled
• bottom-checkCompaction

Each entry carries stopReason, provider, model, content shape, goalModeEnabled, goalStatus, and successfulYield. When #checkCompaction reaches the threshold branch, Auto-compaction threshold decision immediately follows with the billed/stored/resolved/post-maintenance token counts.

To capture both logs in your next run:

tail -n0 -F ~/.omp/logs/omp.$(date +%Y-%m-%d).log \
  | grep -E "agent_end maintenance routing|Auto-compaction threshold decision|auto_compaction_start"

A few lines from each /goal turn that goes over thresholdTokens will tell us exactly which branch is firing (or that no agent_end is firing at all — which would point at a multi-turn tool loop that never yields between turns).