Skip to content
Merged
Show file tree
Hide file tree
Changes from 1 commit
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions packages/agent/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## [Unreleased]

### Fixed

- Allowed configured custom OpenAI-compatible providers to use native remote compaction instead of falling back to local summarization. ([#3104](https://github.com/can1357/oh-my-pi/issues/3104))

## [16.1.5] - 2026-06-19

### Fixed
Expand Down
20 changes: 15 additions & 5 deletions packages/agent/src/compaction/openai.ts
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@
import { ProviderHttpError } from "@oh-my-pi/pi-ai/errors";
import { parseTextSignature } from "@oh-my-pi/pi-ai/providers/openai-shared";
import { transformMessages } from "@oh-my-pi/pi-ai/providers/transform-messages";
import type { AssistantMessage, FetchImpl, Message, Model } from "@oh-my-pi/pi-ai/types";
import type { Api, AssistantMessage, FetchImpl, Message, Model } from "@oh-my-pi/pi-ai/types";
import {
getOpenAIResponsesHistoryItems,
getOpenAIResponsesHistoryPayload,
Expand Down Expand Up @@ -86,12 +86,22 @@ export interface RemoteCompactionResponse {
// OpenAI provider gating + endpoint resolution
// ============================================================================

function isOpenAiRemoteCompactionApi(api: Api | undefined): boolean {
return api === "openai-responses" || api === "azure-openai-responses" || api === "openai-codex-responses";

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Handle Azure compaction with Azure request shape

When a custom/provider model opts in with remoteCompaction.api: "azure-openai-responses", this check sends it through requestOpenAiRemoteCompaction, but that helper still builds an OpenAI-style URL/header set (/v1/responses/compact and Authorization: Bearer). I checked the repo's Azure Responses request builder in packages/ai/src/providers/azure-openai-responses.ts, which uses an api-key header and ?api-version=... on ${baseUrl}/responses; Azure compaction requests using the normal Azure config will therefore fail and fall back to local summarization instead of using the configured native path.

Useful? React with 👍 / 👎.

}

export function shouldUseOpenAiRemoteCompaction(model: Model): boolean {
return model.provider === "openai" || model.provider === "openai-codex";
if (model.remoteCompaction?.enabled === false) return false;
if (model.provider === "openai" || model.provider === "openai-codex") return true;
if (model.remoteCompaction?.enabled !== true) return false;
return isOpenAiRemoteCompactionApi(model.remoteCompaction.api ?? model.api);
}

function resolveOpenAiCompactEndpoint(model: Model): string {
if (model.provider === "openai-codex") {
const configuredEndpoint = model.remoteCompaction?.endpoint;
if (configuredEndpoint && configuredEndpoint.length > 0) return configuredEndpoint;
const compactionApi = model.remoteCompaction?.api ?? model.api;
if (model.provider === "openai-codex" || compactionApi === "openai-codex-responses") {
return resolveOpenAiCodexCompactEndpoint(model.baseUrl);
}

Expand Down Expand Up @@ -444,7 +454,6 @@ export function buildOpenAiNativeHistory(
// ============================================================================
// Endpoint requests
// ============================================================================

export async function requestOpenAiRemoteCompaction(
model: Model,
apiKey: string,
Expand All @@ -454,8 +463,9 @@ export async function requestOpenAiRemoteCompaction(
opts?: { fetch?: FetchImpl; timeoutMs?: number },
): Promise<OpenAiRemoteCompactionResponse> {
const endpoint = resolveOpenAiCompactEndpoint(model);
const requestModel = model.remoteCompaction?.model ?? model.requestModelId ?? model.id;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Honor Azure deployment mappings for compaction

When remoteCompaction.api is azure-openai-responses, this falls back to model.id unless remoteCompaction.model is duplicated, but the normal Azure Responses path resolves AZURE_OPENAI_DEPLOYMENT_NAME_MAP and sends that deployment name as the request model (packages/ai/src/providers/azure-openai-responses.ts). In Azure setups where catalog ids like gpt-5 are mapped to deployment names, remote compaction sends the unmapped id, the Azure request fails, and compaction silently falls back to local summarization even though chat requests work with the same model.

Useful? React with 👍 / 👎.

const request: OpenAiRemoteCompactionRequest = {
model: model.id,
model: requestModel,
input: trimOpenAiCompactInput(compactInput, model.contextWindow ?? Number.POSITIVE_INFINITY, instructions),
instructions,
};
Expand Down
40 changes: 39 additions & 1 deletion packages/agent/test/remote-compaction.test.ts
Original file line number Diff line number Diff line change
Expand Up @@ -5,7 +5,11 @@ import {
createFileOps,
DEFAULT_COMPACTION_SETTINGS,
} from "@oh-my-pi/pi-agent-core/compaction";
import { buildOpenAiNativeHistory, requestOpenAiRemoteCompaction } from "@oh-my-pi/pi-agent-core/compaction/openai";
import {
buildOpenAiNativeHistory,
requestOpenAiRemoteCompaction,
shouldUseOpenAiRemoteCompaction,
} from "@oh-my-pi/pi-agent-core/compaction/openai";
import * as ai from "@oh-my-pi/pi-ai";
import type { AssistantMessage, FetchImpl, Model, ToolResultMessage } from "@oh-my-pi/pi-ai/types";
import { buildModel } from "@oh-my-pi/pi-catalog/build";
Expand Down Expand Up @@ -231,6 +235,40 @@ describe("remote compaction input trimming", () => {
});
});

test("uses configured OpenAI-compatible compaction for custom providers", async () => {
const model = makeOpenAiModel({
provider: "cliproxy-codex",
baseUrl: "http://127.0.0.1:8317/v1",
remoteCompaction: {
enabled: true,
api: "openai-responses",
endpoint: "http://127.0.0.1:8317/v1/responses/compact",
model: "gpt-5.5",
},
});
let requestBody: unknown;
const fetchMock: FetchImpl = async (input, init) => {
expect(String(input)).toBe("http://127.0.0.1:8317/v1/responses/compact");
requestBody = JSON.parse(String(init?.body));
return new Response(
JSON.stringify({
output: [{ type: "compaction_summary", summary: "native compacted" }],
}),
);
};

expect(shouldUseOpenAiRemoteCompaction(model)).toBe(true);
await requestOpenAiRemoteCompaction(
model,
"test-key",
[{ type: "message", role: "user", content: [{ type: "input_text", text: "hi" }] }],
"instructions",
undefined,
{ fetch: fetchMock },
);
expect(requestBody).toMatchObject({ model: "gpt-5.5" });
});

describe("requestOpenAiRemoteCompaction abort", () => {
test("rejects when the abort signal is aborted mid-fetch", async () => {
const controller = new AbortController();
Expand Down
4 changes: 4 additions & 0 deletions packages/catalog/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## [Unreleased]

### Added

- Added model metadata for provider-native remote compaction and compaction-only model selection. ([#3104](https://github.com/can1357/oh-my-pi/issues/3104))

## [16.1.7] - 2026-06-20

### Fixed
Expand Down
16 changes: 16 additions & 0 deletions packages/catalog/src/types.ts
Original file line number Diff line number Diff line change
Expand Up @@ -576,6 +576,18 @@ export type CompatOf<TApi extends Api> = TApi extends "openrouter"
? ResolvedAnthropicCompat
: undefined;

/** Provider-native compaction endpoint configuration for one model. */
export interface RemoteCompactionConfig<TApi extends Api = Api> {
/** Enables provider-native compaction for providers not enabled by built-in policy. */
enabled?: boolean;
/** Adapter family used by the configured compaction endpoint. */
api?: TApi;
/** Absolute compact endpoint URL; when omitted, the adapter derives it from the model base URL. */
endpoint?: string;
/** Model id sent to the compaction endpoint when it differs from the active model id. */
model?: string;
}

// Model interface for the unified model system
export interface Model<TApi extends Api = Api> {
id: string;
Expand Down Expand Up @@ -648,6 +660,10 @@ export interface Model<TApi extends Api = Api> {
preferWebsockets?: boolean;
/** Preferred model to switch to when context promotion is triggered (model id or provider/id). */
contextPromotionTarget?: string;
/** Preferred model to use only for compaction (model id or provider/id); the active session model is unchanged. */
compactionModel?: string;
/** Provider-native compaction endpoint configuration. */
remoteCompaction?: RemoteCompactionConfig<TApi>;
/** Provider-assigned priority value (lower = higher priority). */
priority?: number;
/** Canonical thinking capability metadata for this model. */
Expand Down
4 changes: 4 additions & 0 deletions packages/coding-agent/CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -2,6 +2,10 @@

## [Unreleased]

### Added

- Added `models.yml` `remoteCompaction` and `compactionModel` config so custom providers can opt into provider-native compaction and run compaction on a separate model without changing the active session model. ([#3104](https://github.com/can1357/oh-my-pi/issues/3104))

## [16.1.7] - 2026-06-20

### Fixed
Expand Down
61 changes: 53 additions & 8 deletions packages/coding-agent/src/config/model-registry.ts
Original file line number Diff line number Diff line change
@@ -1,7 +1,15 @@
import { execSync } from "node:child_process";
import * as path from "node:path";
import { registerCustomApi, unregisterCustomApis } from "@oh-my-pi/pi-ai/api-registry";
import type { Api, Context, Model, ModelSpec, SimpleStreamOptions, ThinkingConfig } from "@oh-my-pi/pi-ai/types";
import type {
Api,
Context,
Model,
ModelSpec,
RemoteCompactionConfig,
SimpleStreamOptions,
ThinkingConfig,
} from "@oh-my-pi/pi-ai/types";
import type { AssistantMessageEventStream } from "@oh-my-pi/pi-ai/utils/event-stream";
import { buildModel } from "@oh-my-pi/pi-catalog/build";
import { isVertexExpressOpenAIUrl } from "@oh-my-pi/pi-catalog/hosts";
Expand Down Expand Up @@ -95,6 +103,7 @@ interface ProviderOverride {
apiKey?: string;
authHeader?: boolean;
compat?: ModelSpec<Api>["compat"];
remoteCompaction?: RemoteCompactionConfig<Api>;
transport?: Model<Api>["transport"];
}

Expand Down Expand Up @@ -127,7 +136,7 @@ interface ProviderOverride {
export function mergeDiscoveredModel<TApi extends Api>(
model: Model<TApi>,
existing: Model<Api> | undefined,
providerOverride?: Pick<ProviderOverride, "baseUrl" | "headers" | "transport">,
providerOverride?: Pick<ProviderOverride, "baseUrl" | "headers" | "remoteCompaction" | "transport">,
): Model<TApi> {
if (existing) {
const supportsTools = model.supportsTools ?? existing.supportsTools;
Expand All @@ -136,6 +145,10 @@ export function mergeDiscoveredModel<TApi extends Api>(
baseUrl: providerOverride?.baseUrl ?? model.baseUrl ?? existing.baseUrl,
headers: existing.headers ? { ...existing.headers, ...model.headers } : model.headers,
transport: providerOverride?.transport ?? existing.transport ?? model.transport,
remoteCompaction: mergeRemoteCompactionConfig(
existing.remoteCompaction ?? model.remoteCompaction,
providerOverride?.remoteCompaction,
),
...(supportsTools !== undefined ? { supportsTools } : {}),
compat: model.compatConfig,
} as ModelSpec<TApi>);
Expand All @@ -146,6 +159,7 @@ export function mergeDiscoveredModel<TApi extends Api>(
baseUrl: providerOverride.baseUrl ?? model.baseUrl,
headers: providerOverride.headers ? { ...model.headers, ...providerOverride.headers } : model.headers,
...(providerOverride.transport !== undefined ? { transport: providerOverride.transport } : {}),
remoteCompaction: mergeRemoteCompactionConfig(model.remoteCompaction, providerOverride.remoteCompaction),
compat: model.compatConfig,
} as ModelSpec<TApi>);
}
Expand Down Expand Up @@ -353,6 +367,15 @@ function mergeCompat<TBase extends object, TOverride extends object>(
return merged as TBase & TOverride;
}

function mergeRemoteCompactionConfig(
baseConfig: RemoteCompactionConfig<Api> | undefined,
overrideConfig: RemoteCompactionConfig<Api> | undefined,
): RemoteCompactionConfig<Api> | undefined {
if (!baseConfig) return overrideConfig;
if (!overrideConfig) return baseConfig;
return { ...baseConfig, ...overrideConfig };
}

/**
* Project a built model back to spec shape for the model-manager/cache
* boundary: sparse compat comes from `compatConfig`, never from the resolved
Expand Down Expand Up @@ -380,6 +403,8 @@ interface ModelPatch {
headers?: Record<string, string>;
compat?: ModelSpec<Api>["compat"];
contextPromotionTarget?: string;
compactionModel?: string;
remoteCompaction?: RemoteCompactionConfig<Api>;
premiumMultiplier?: number;
}

Expand All @@ -403,6 +428,10 @@ function applyModelPatch(base: Model<Api>, patch: ModelPatch, transport: ModelTr
if (patch.maxTokens !== undefined) result.maxTokens = patch.maxTokens;
if (patch.omitMaxOutputTokens !== undefined) result.omitMaxOutputTokens = patch.omitMaxOutputTokens;
if (patch.contextPromotionTarget !== undefined) result.contextPromotionTarget = patch.contextPromotionTarget;
if (patch.compactionModel !== undefined) result.compactionModel = patch.compactionModel;
if (patch.remoteCompaction !== undefined) {
result.remoteCompaction = mergeRemoteCompactionConfig(base.remoteCompaction, patch.remoteCompaction);
}
if (patch.premiumMultiplier !== undefined) result.premiumMultiplier = patch.premiumMultiplier;
if (patch.cost) {
result.cost = {
Expand Down Expand Up @@ -475,8 +504,6 @@ function mergeAuthHeader(
/**
* Decide whether a custom-yaml model should force OAuth-style request shaping.
* - Explicit `auth: oauth` → force on.
* - Explicit `auth: apiKey` / `auth: none` → leave unset (auto-detect by key prefix).
* - No `auth` specified and `api: anthropic-messages` → default on. Custom Anthropic
* endpoints are typically Claude-Code-style proxies (e.g. CLIProxyAPI) that expect
* the cloaked request shape regardless of how the proxy itself is authenticated.
* - Otherwise → unset.
Expand All @@ -497,6 +524,7 @@ function buildCustomModelOverlay(
authHeader: boolean | undefined,
providerCompat: ModelSpec<Api>["compat"] | undefined,
providerAuth: ProviderAuthMode | undefined,
providerRemoteCompaction: RemoteCompactionConfig<Api> | undefined,
modelDef: CustomModelDefinitionLike,
): CustomModelOverlay | undefined {
const api = modelDef.api ?? providerApi;
Expand All @@ -518,6 +546,8 @@ function buildCustomModelOverlay(
headers: mergeCustomModelHeaders(providerHeaders, modelDef.headers, authHeader, providerApiKey),
compat: mergeCompat(providerCompat, modelDef.compat),
contextPromotionTarget: modelDef.contextPromotionTarget,
compactionModel: modelDef.compactionModel,
remoteCompaction: mergeRemoteCompactionConfig(providerRemoteCompaction, modelDef.remoteCompaction),
premiumMultiplier: modelDef.premiumMultiplier,
isOAuth: resolveCustomModelIsOAuth(api, providerAuth),
};
Expand Down Expand Up @@ -558,6 +588,8 @@ function finalizeCustomModel(model: CustomModelOverlay, options: CustomModelBuil
omitMaxOutputTokens: resolvedModel.omitMaxOutputTokens ?? reference?.omitMaxOutputTokens,
compat: mergeCompat(reference?.compatConfig, resolvedModel.compat),
contextPromotionTarget: resolvedModel.contextPromotionTarget,
compactionModel: resolvedModel.compactionModel,
remoteCompaction: resolvedModel.remoteCompaction,
premiumMultiplier: resolvedModel.premiumMultiplier,
isOAuth: resolvedModel.isOAuth,
} as ModelSpec<Api>);
Expand Down Expand Up @@ -1127,7 +1159,6 @@ export class ModelRegistry {
const discoverableProviders: DiscoveryProviderConfig[] = [];
const providerEntries = Object.entries(value.providers ?? {});
const configuredProviders = new Set(Object.keys(value.providers ?? {}));

for (const [providerName, providerConfig] of providerEntries) {
const resolvedProviderHeaders = resolveConfigHeaders(providerConfig.headers);
// Always set overrides when baseUrl/headers/apiKey/authHeader/compat/disableStrictTools/transport are present
Expand All @@ -1138,6 +1169,7 @@ export class ModelRegistry {
providerConfig.authHeader !== undefined ||
providerConfig.compat ||
providerConfig.disableStrictTools ||
providerConfig.remoteCompaction ||
providerConfig.transport
) {
const disableStrictCompat = providerConfig.disableStrictTools ? { disableStrictTools: true } : undefined;
Expand All @@ -1147,6 +1179,7 @@ export class ModelRegistry {
apiKey: providerConfig.apiKey,
authHeader: providerConfig.authHeader,
compat: mergeCompat(providerConfig.compat, disableStrictCompat),
remoteCompaction: providerConfig.remoteCompaction,

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply remote compaction overrides to cached discovery models

Adding this provider-level override makes discovery-backed providers look configurable, but warm-start cache loading in #loadCachedDiscoverableModels rebuilds cached rows and only reapplies compat/modelOverrides, while the discovery cache itself is written before this override is merged. With a fresh discovery cache (or before refreshRuntimeDiscoveries runs), models for a custom provider configured only with provider-level remoteCompaction have no remote compaction metadata, so /compact remote falls back locally until a refresh path happens to re-merge the override.

Useful? React with 👍 / 👎.

transport: providerConfig.transport,
});
}
Expand Down Expand Up @@ -1538,12 +1571,18 @@ export class ModelRegistry {
authHeader: override.authHeader ?? baseOverride?.authHeader,
headers: override.headers ? { ...(baseOverride?.headers ?? {}), ...override.headers } : baseOverride?.headers,
compat: override.compat ? mergeCompat(baseOverride?.compat, override.compat) : baseOverride?.compat,
remoteCompaction: mergeRemoteCompactionConfig(baseOverride?.remoteCompaction, override.remoteCompaction),
transport: override.transport ?? baseOverride?.transport,
};
}
#applyProviderTransportOverride<T extends { baseUrl?: string; headers?: Record<string, string> }>(
#applyProviderTransportOverride<
T extends { baseUrl?: string; headers?: Record<string, string>; remoteCompaction?: RemoteCompactionConfig<Api> },
>(
entry: T,
override: Pick<ProviderOverride, "baseUrl" | "headers" | "authHeader" | "apiKey" | "transport">,
override: Pick<
ProviderOverride,
"baseUrl" | "headers" | "authHeader" | "apiKey" | "remoteCompaction" | "transport"
>,
): T {
const headers = mergeAuthHeader(
override.headers ? { ...entry.headers, ...override.headers } : entry.headers,
Expand All @@ -1557,6 +1596,7 @@ export class ModelRegistry {
// Preserve the model's existing transport when the override omits one;
// providers without a `transport` field keep the default per-API dispatch.
...(override.transport !== undefined ? { transport: override.transport } : {}),
remoteCompaction: mergeRemoteCompactionConfig(entry.remoteCompaction, override.remoteCompaction),
};
}
#applyRuntimeProviderOverrides(models: Model<Api>[]): Model<Api>[] {
Expand Down Expand Up @@ -1641,7 +1681,6 @@ export class ModelRegistry {

#parseModels(config: ModelsConfig): CustomModelOverlay[] {
const models: CustomModelOverlay[] = [];

for (const [providerName, providerConfig] of Object.entries(config.providers ?? {})) {
const modelDefs = providerConfig.models ?? [];
if (modelDefs.length === 0) continue; // Override-only, no custom models
Expand All @@ -1662,6 +1701,7 @@ export class ModelRegistry {
providerConfig.authHeader,
providerCompat,
(providerConfig.auth as ProviderAuthMode | undefined) ?? undefined,
providerConfig.remoteCompaction,
modelDef as CustomModelDefinitionLike,
);
if (!model) continue;
Expand Down Expand Up @@ -2069,6 +2109,7 @@ export class ModelRegistry {
config.authHeader,
config.compat,
undefined,
config.remoteCompaction,
modelDef as CustomModelDefinitionLike,
);
if (!overlay) {
Expand Down Expand Up @@ -2136,6 +2177,7 @@ export class ModelRegistry {
providerAuthHeader,
providerCompat,
undefined,
config.remoteCompaction,
modelDef as CustomModelDefinitionLike,
);
if (overlay) results.push(finalizeCustomModel(overlay, { useDefaults: true }));
Expand Down Expand Up @@ -2221,6 +2263,7 @@ export interface ProviderConfigInput {
streamSimple?: (model: Model<Api>, context: Context, options?: SimpleStreamOptions) => AssistantMessageEventStream;
headers?: Record<string, string>;
compat?: ModelSpec<Api>["compat"];
remoteCompaction?: RemoteCompactionConfig<Api>;

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P2 Badge Apply runtime remoteCompaction overrides

Adding remoteCompaction to ProviderConfigInput makes it look usable for extension/runtime provider overrides, but the override-only branch in registerProvider() only runs for baseUrl/headers/apiKey/authHeader/transport and builds a transportOverride without this field. Calls such as registerProvider("openai", { remoteCompaction: { enabled: false } }) or a baseUrl override with compaction metadata therefore leave all existing models unchanged, so runtime providers cannot actually override native compaction unless they also replace every model.

Useful? React with 👍 / 👎.

authHeader?: boolean;
/** Streaming transport override — see {@link Model.transport}. */
transport?: Model<Api>["transport"];
Expand Down Expand Up @@ -2255,6 +2298,8 @@ export interface ProviderConfigInput {
headers?: Record<string, string>;
compat?: ModelSpec<Api>["compat"];
contextPromotionTarget?: string;
compactionModel?: string;
remoteCompaction?: RemoteCompactionConfig<Api>;
premiumMultiplier?: number;
}>;
}
Loading
Loading