Align Assistant provider retry with OpenCode

This commit is contained in:
rcourtman
2026-06-07 17:04:42 +01:00
parent b4a492c4bd
commit 84ec1a7214
24 changed files with 251 additions and 841 deletions
@@ -417,102 +417,55 @@ leave the transcript without exposing hidden provider/tool metadata.
retryable in-memory assistant error may replay the original user turn's
structured mentions, finding id, approval override, handoff resources,
handoff actions, and handoff metadata, but must not reconstruct scoped
context from prompt history or saved transcript prose.
Failed-turn recovery must also expose model-route switching through the
existing drawer model selector so operators can move from a blocked direct
provider route to a configured gateway or alternate model without losing the
draft or creating a parallel picker. When the same model is available through
another configured provider route, the failed turn must offer that route as a
direct one-click route-and-retry action, but route recovery must not loop
between providers that already failed in the same transcript; once equivalent
routes are exhausted, the failed turn must fall back to another configured
notable model/provider before falling back to the general model selector.
Retry remains available, but it must not be the only visible action when a
failed Assistant turn is shown.
Provider recovery before visible output is backend-owned chat-runtime
behavior. When a selected route fails before streaming content, tool
progress, approval, or question events, `internal/ai/chat` may try the next
configured provider/model route in the same user turn and must emit a
`provider_fallback` workflow-state event that identifies the failed and next
provider/model route. This fallback must use chat-suitable model resolution
from `internal/ai/modelresolution`, skipping obvious non-chat endpoints such
as realtime, audio, moderation, embedding, and content-safety catalog
entries. Gateway-equivalent routes, such as a direct provider model exposed
through a configured gateway, must be produced by that model-resolution
policy rather than hardcoded in chat execution or frontend UI logic.
Fallback planning must not block the selected provider's first
attempt on live catalog reads for every other configured provider; the hot
path may queue only explicit provider preferences or stable provider defaults
and defer fallback provider construction until the selected route actually
fails before visible output. Primary interactive chat model resolution is
governed by that same hot-path rule: it must use the explicit configured
context from prompt history or saved transcript prose. Provider/model route
recovery is explicit user-visible recovery, not hidden chat execution. A
failed turn may expose route-and-retry actions through the existing drawer
model selector so operators can move from a blocked direct provider route to a
configured gateway or alternate model without losing the draft or creating a
parallel picker, but `/api/ai/chat` must not automatically switch provider or
model routes inside a single Assistant turn after the selected route fails.
That prohibition includes same-model gateway equivalents and configured
provider defaults: those routes may be offered as explicit recovery actions,
but the selected turn owns its selected route until it completes or fails.
The referenced OpenCode source at fetched `dev` commit
`e82542b8023a8374f29c23b70ec019c8f256354e` models this boundary by keeping
same-route retry visible in `packages/opencode/src/session/retry.ts`,
publishing retry status from `packages/opencode/src/session/processor.ts`,
rendering retry state in `packages/ui/src/components/session-retry.tsx`, and
treating provider/model changes as explicit session events in
`packages/core/src/session/event.ts` and
`packages/opencode/src/session/prompt.ts`. Pulse adapts that behavior for the
drawer instead of implementing automatic cross-provider fallback.
Primary interactive chat model resolution must use the explicit configured
chat route or a stable provider default without calling provider model
catalogs before the selected stream starts. Catalog-backed recommendation
belongs to settings/model-list flows, not `/api/ai/chat` first-response
startup. When the selected direct-provider route fails before visible output,
fallback planning may first try configured gateway-equivalent candidates
returned by `internal/ai/modelresolution` when they preserve the same
provider/model family through a deterministic route string, before falling
back to unrelated configured provider defaults. That same-model gateway
planning must remain catalog-free, capability-driven, and may continue to the
next configured provider if a gateway attempt also fails before visible
output; chat execution and frontend UI must never encode user-specific
network assumptions or named-provider failover rules. Once visible output has
streamed, Pulse must
not silently switch providers for that turn; the error belongs to the visible
attempt and is surfaced through normal failed-turn recovery. Assistant
startup retry behavior must be route-aware: when another provider/model
attempt is already queued for the same user turn, non-final attempts must
fast-fail before visible output instead of spending hidden same-route retry
and backoff time. In that mode, provider-owned startup retries and the
agentic loop's retry of the same provider turn are disabled for the
non-final attempt, and the wait for the first visible provider event is
bounded so `provider_fallback` can reach the browser quickly. The final
queued attempt keeps normal provider retry behavior because there is no
later route to hand off to. The referenced OpenCode source at fetched
`origin/dev` commit `1025540fcc2a69609a0131a7168300205656d728` models retry
as explicit session activity through `RetryError` and `Retried` in
`packages/core/src/session/event.ts` and renders retry status in
`packages/ui/src/components/session-retry.tsx`; Pulse adapts that principle
by surfacing route changes through workflow/model rows and by yielding to
the next configured route quickly instead of hiding repeated startup
retries behind a static waiting label. Provider retry workflow states that
include `retry_after_ms` must render as live countdowns derived from the
workflow `started_at` timestamp while the turn is active, matching
OpenCode's visible retry wait behavior instead of freezing the first retry
label.
belongs to settings/model-list and explicit route-recovery flows, not
`/api/ai/chat` first-response startup. Once the selected provider route
starts, transient pre-output transport failures may retry the same route and
must surface a `provider_retry` workflow state with `attempt`,
`max_attempts`, and `retry_after_ms` before sleeping. If same-route retry is
exhausted before visible output, Pulse must emit a normal provider error and
leave recovery to the failed-turn actions; it must not emit
`provider_fallback`, `failed_provider`, `failed_model`, `next_provider`, or
`next_model` workflow metadata. Once visible output has streamed, Pulse must
also keep the failure on that visible attempt rather than silently changing
route mid-turn. Provider retry workflow states that include `retry_after_ms`
must render as live countdowns derived from the workflow `started_at`
timestamp while the turn is active, matching OpenCode's visible retry wait
behavior instead of freezing the first retry label.
Assistant completion events must carry the effective model route that actually
completed the turn, and the drawer must update the in-flight transcript row
when `provider_fallback` names the next route so message labels, cost
context, retry decisions, and model-route recovery do not continue to point
at the failed provider. The referenced OpenCode source at fetched
`origin/dev` commit `4519a1da329c1a4fc384054e7203ba7d06928205` defines
`session.next.model.switched` in
`packages/core/src/session/event.ts`, appends it as a
`SessionMessage.ModelSwitched` transcript message in
`packages/core/src/session/message-updater.ts`, and renders it as
`ModelSwitchedMessage` in
`packages/opencode/src/cli/cmd/tui/feature-plugins/system/session-v2.tsx`.
Pulse adapts that by rendering model route changes as typed `model_switch`
stream rows on the active assistant turn instead of hiding the route in
transient status text or assistant prose. The initial `provider_start`
workflow state must carry the selected `provider` and concrete `model` route,
and the drawer must render that first route as a selected-model row (`Using
...`) before visible content or tool activity when the event arrives.
Provider-fallback rows must preserve the failed route and next route
together, using the backend `failed_model` and `next_model` payloads, so the
transcript shows the actual recovery path rather than only the successful
replacement model. A successful provider fallback is also a session-model
event, not only transcript decoration: once the live assistant turn completes
with the fallback route as its effective model, the drawer must promote that
route into the active session selection and recent-model list so the next
prompt does not retry the just-failed direct provider before using the route
that worked. This promotion may only apply to fallback rows observed during
the live stream and must not rewrite historical loaded sessions or override
an explicit user model change made while the fallback turn was still running.
completed the selected turn. The drawer must render the initial
`provider_start` workflow state as a selected-model row (`Using ...`) before
visible content or tool activity when the event arrives. Later model-route
changes may still be rendered as typed `model_switch` transcript rows when
they come from explicit route recovery or restored historical data, but the UI
must label them neutrally as selected or switched routes rather than as
automatic provider fallback, and completing a streamed route-switch row must
not promote a fallback route into the active model selection without an
explicit user action.
Interactive Assistant streams must establish the session ID and emit the
`session` event once as soon as the HTTP SSE writer is ready, before finding
handoff recovery, model resolution, provider fallback planning,
handoff recovery, model resolution, selected-provider startup,
handoff/context prefetch, recent-session injection, inventory summary reads,
tool scoping, or provider startup. The chat service then persists/ensures
that same session ID while suppressing duplicate session events. This keeps
@@ -1008,7 +961,7 @@ leave the transcript without exposing hidden provider/tool metadata.
external providers or models. The live fixture must pace its status, tool,
and content events enough for the browser to paint each state in sequence;
unit tests may disable that pace, but the dev fixture must not collapse into
a single final completed row. This fixture must not run as provider fallback
a single final completed row. This fixture must not emit `provider_fallback`
in non-mock mode and must not become Pulse-authored remediation or routing
behavior. The referenced OpenCode source at fetched `origin/dev` commit
`4519a1da329c1a4fc384054e7203ba7d06928205` publishes tool-call and
@@ -123,7 +123,13 @@ to `frontend-modern/src/api/generated/aiChatEvents.ts` by
Provider retry progress must use typed fields (`attempt`, `max_attempts`,
`retry_after_ms`) on that same event instead of frontend-only string parsing or
provider-specific ad hoc events; the Assistant UI may format those fields, but
must not invent retry progress that the stream contract did not carry.
must not invent retry progress that the stream contract did not carry. Assistant
chat stream payloads must not expose automatic provider fallback metadata:
`workflow_state` may carry the selected `provider` and `model` plus same-route
retry fields, but `provider_fallback`, `failed_provider`, `failed_model`,
`next_provider`, and `next_model` are retired from the generated stream event
contract. Cross-route recovery belongs to explicit failed-turn actions, not
hidden `/api/ai/chat` stream mutation.
Assistant local stream fixtures are part of the same frontend API contract:
`frontend-modern/src/api/aiChatDevStreamFixture.ts` may short-circuit only
explicit `/fixture ...` prompts in development or test mode, must emit the same
@@ -134,6 +140,10 @@ obsolete grouped-context wording in fixture answer content. The fixture payload
contract proves the stream reducer and transcript renderer against the same
chronological event order a live provider would produce; UI grouping or footer
summaries are not part of the fixture contract.
The provider-retry fixture must exercise selected-route retry by emitting
`provider_retry` for the selected route and completing with that same model; it
must not simulate an automatic switch to a configured gateway or alternate
provider route.
Queue verification fixtures must cover both the active hold turn and the queued
drain turn so UX proof can exercise queued follow-up ordering and tool rows
without consuming external model quota.
@@ -244,12 +244,7 @@ describe('AIChatAPI', () => {
it('runs the send-hold dev stream fixture without opening a provider request', async () => {
const onEvent = vi.fn();
await AIChatAPI.chat(
'/fixture send-hold',
undefined,
'openrouter:qwen/qwen3.7-plus',
onEvent,
);
await AIChatAPI.chat('/fixture send-hold', undefined, 'openrouter:qwen/qwen3.7-plus', onEvent);
expect(apiFetchMock).not.toHaveBeenCalled();
expect(onEvent.mock.calls.map(([event]) => event.type)).toEqual([
@@ -551,7 +546,6 @@ describe('AIChatAPI', () => {
'workflow_state',
'workflow_state',
'workflow_state',
'workflow_state',
'content',
'done',
]);
@@ -559,28 +553,17 @@ describe('AIChatAPI', () => {
type: 'workflow_state',
data: {
phase: 'provider_retry',
message: 'DeepSeek failed before output; retrying through OpenRouter.',
message: 'Provider connection failed before any output; retrying.',
attempt: 2,
failed_model: 'deepseek:deepseek-chat',
max_attempts: 3,
next_model: 'openrouter:deepseek/deepseek-chat',
retry_after_ms: 3200,
},
});
expect(onEvent.mock.calls[4][0]).toMatchObject({
type: 'workflow_state',
data: {
phase: 'provider_start',
message: 'Retrying through OpenRouter with deepseek/deepseek-chat.',
provider: 'openrouter',
model: 'openrouter:deepseek/deepseek-chat',
},
});
expect(onEvent.mock.calls[6][0]).toMatchObject({
expect(onEvent.mock.calls[5][0]).toMatchObject({
type: 'done',
data: {
session_id: 'dev-fixture-provider-retry',
model: 'openrouter:deepseek/deepseek-chat',
model: 'deepseek:deepseek-chat',
},
});
});
@@ -24,22 +24,12 @@ describe('AI chat stream event contract', () => {
expect(aiChatEventsSource).toContain("type: 'session'");
});
it('exposes provider fallback metadata on workflow state events', () => {
const workflow: WorkflowStateData = {
phase: 'provider_fallback',
message: 'OpenRouter did not start a response; trying DeepSeek.',
state: 'provider_fallback',
failed_provider: 'openrouter',
failed_model: 'openrouter:qwen/qwen3.7-plus',
next_provider: 'deepseek',
next_model: 'deepseek:deepseek-v4-pro',
};
const event: AIChatStreamEvent = { type: 'workflow_state', data: workflow };
expect(event.data.failed_model).toBe('openrouter:qwen/qwen3.7-plus');
expect(event.data.next_model).toBe('deepseek:deepseek-v4-pro');
expect(aiChatEventsSource).toContain('failed_model?: string');
expect(aiChatEventsSource).toContain('next_model?: string');
it('does not expose automatic provider fallback metadata on workflow state events', () => {
expect(aiChatEventsSource).not.toContain('provider_fallback');
expect(aiChatEventsSource).not.toContain('failed_provider?: string');
expect(aiChatEventsSource).not.toContain('failed_model?: string');
expect(aiChatEventsSource).not.toContain('next_provider?: string');
expect(aiChatEventsSource).not.toContain('next_model?: string');
});
it('exposes selected provider and model metadata on workflow state events', () => {
@@ -620,36 +620,25 @@ const buildProviderRetryFixtureEvents = (): AIChatStreamEvent[] => [
type: 'workflow_state',
data: {
phase: 'provider_retry',
message: 'DeepSeek failed before output; retrying through OpenRouter.',
message: 'Provider connection failed before any output; retrying.',
provider: 'deepseek',
model: 'deepseek:deepseek-chat',
failed_model: 'deepseek:deepseek-chat',
next_model: 'openrouter:deepseek/deepseek-chat',
attempt: 2,
max_attempts: 3,
retry_after_ms: 3200,
},
},
{
type: 'workflow_state',
data: {
phase: 'provider_start',
message: 'Retrying through OpenRouter with deepseek/deepseek-chat.',
provider: 'openrouter',
model: 'openrouter:deepseek/deepseek-chat',
},
},
{
type: 'content',
data: {
text: 'The provider retry fixture switched to OpenRouter after the direct provider failed before output.',
text: 'The provider retry fixture retried the selected route after a transient startup failure.',
},
},
{
type: 'done',
data: {
session_id: 'dev-fixture-provider-retry',
model: 'openrouter:deepseek/deepseek-chat',
model: 'deepseek:deepseek-chat',
input_tokens: 73,
output_tokens: 29,
},
@@ -133,10 +133,6 @@ export interface WorkflowStateData {
attempt?: number;
max_attempts?: number;
retry_after_ms?: number;
failed_provider?: string;
failed_model?: string;
next_provider?: string;
next_model?: string;
}
export type AIChatStreamEvent =
@@ -350,7 +350,7 @@ export const MessageItem: Component<MessageItemProps> = (props) => {
if (!model) return '';
return props.getModelRouteLabel?.(model) || formatAIModelRouteLabel(model);
};
const isProviderFallbackEvent = (event: StreamDisplayEvent) => {
const hasPreviousModelRoute = (event: StreamDisplayEvent) => {
const model = event.model?.trim();
const failed = event.failedModel?.trim();
return !!model && !!failed && failed !== model;
@@ -358,7 +358,7 @@ export const MessageItem: Component<MessageItemProps> = (props) => {
const isSelectedModelEvent = (event: StreamDisplayEvent) =>
event.modelEvent === 'selected' && !!event.model?.trim();
const modelSwitchTitle = (event: StreamDisplayEvent) => {
if (!isProviderFallbackEvent(event)) return modelRouteLabel(event.model);
if (!hasPreviousModelRoute(event)) return modelRouteLabel(event.model);
return `${modelRouteLabel(event.failedModel)} -> ${modelRouteLabel(event.model)}`;
};
const messageModelLabel = () => modelRouteLabel(props.message.model);
@@ -673,11 +673,9 @@ export const MessageItem: Component<MessageItemProps> = (props) => {
class="my-2 inline-flex max-w-full items-center gap-2 rounded-md border border-border-subtle bg-surface-alt px-2.5 py-1.5 text-xs text-muted"
role="status"
aria-label={
isProviderFallbackEvent(event)
? 'Assistant provider fallback route changed'
: isSelectedModelEvent(event)
? 'Assistant model route selected'
: 'Assistant model route changed'
isSelectedModelEvent(event)
? 'Assistant model route selected'
: 'Assistant model route changed'
}
title={modelSwitchTitle(event)}
>
@@ -686,7 +684,7 @@ export const MessageItem: Component<MessageItemProps> = (props) => {
aria-hidden="true"
/>
<Show
when={isProviderFallbackEvent(event)}
when={hasPreviousModelRoute(event)}
fallback={
<>
<Show when={isSelectedModelEvent(event)}>
@@ -701,7 +699,7 @@ export const MessageItem: Component<MessageItemProps> = (props) => {
</>
}
>
<span class="shrink-0">Provider fallback</span>
<span class="shrink-0">Switched from</span>
<span class="min-w-0 truncate font-medium text-base-content">
{modelRouteLabel(event.failedModel)}
</span>
@@ -3018,7 +3018,10 @@ describe('AIChat', () => {
expect(mockChat.loadSession).toHaveBeenCalledWith('source-session');
});
expect(mockAIChatAPI.listSessions).toHaveBeenCalledWith({ limit: 30 });
expect(mockNotificationStore.info).toHaveBeenCalledWith('Compacting Assistant session...', 2000);
expect(mockNotificationStore.info).toHaveBeenCalledWith(
'Compacting Assistant session...',
2000,
);
await waitFor(() => {
expect(mockNotificationStore.success).toHaveBeenCalledWith(
'Compacted 8 older messages into a session summary.',
@@ -4552,7 +4555,7 @@ describe('AIChat', () => {
});
});
it('adopts a successful provider fallback route for the active session', async () => {
it('does not adopt a completed route-switch row for the active session automatically', async () => {
const [messages, setMessages] = createSignal<ChatMessage[]>([]);
mockChat.messages.mockImplementation(() => messages());
mockChat.sessionId.mockReturnValue('session-1');
@@ -4569,7 +4572,7 @@ describe('AIChat', () => {
setMessages([
{
id: 'assistant-fallback',
id: 'assistant-route-switch',
role: 'assistant',
content: '',
timestamp: new Date('2026-06-06T12:00:00Z'),
@@ -4589,9 +4592,9 @@ describe('AIChat', () => {
setMessages([
{
id: 'assistant-fallback',
id: 'assistant-route-switch',
role: 'assistant',
content: 'DeepSeek is reachable through OpenRouter.',
content: 'The response completed on another route.',
timestamp: new Date('2026-06-06T12:00:00Z'),
completedAt: new Date('2026-06-06T12:00:03Z'),
model: 'openrouter:deepseek/deepseek-v4-pro',
@@ -4604,34 +4607,27 @@ describe('AIChat', () => {
},
{
type: 'content',
content: 'DeepSeek is reachable through OpenRouter.',
content: 'The response completed on another route.',
},
],
},
]);
await waitFor(() => {
expect(mockChat.setModel).toHaveBeenCalledWith('openrouter:deepseek/deepseek-v4-pro');
});
expect(
screen.getByRole('status', { name: 'Assistant fallback route adopted' }),
).toHaveTextContent('after fallback from');
expect(
screen.getByRole('button', { name: 'Dismiss fallback route notice' }),
).toBeInTheDocument();
expect(mockNotificationStore.success).toHaveBeenCalledWith(
screen.queryByRole('status', { name: 'Assistant fallback route adopted' }),
).not.toBeInTheDocument();
expect(mockChat.setModel).not.toHaveBeenCalledWith('openrouter:deepseek/deepseek-v4-pro');
expect(mockNotificationStore.success).not.toHaveBeenCalledWith(
expect.stringContaining('Assistant model route switched to'),
2500,
expect.any(Number),
);
expect(localStorage.getItem('pulse:ai_chat_models_by_session')).toBe(
JSON.stringify({ 'session-1': 'openrouter:deepseek/deepseek-v4-pro' }),
);
expect(localStorage.getItem('pulse:ai_chat_recent_models')).toBe(
JSON.stringify(['openrouter:deepseek/deepseek-v4-pro']),
JSON.stringify({ 'session-1': 'deepseek:deepseek-v4-pro' }),
);
expect(localStorage.getItem('pulse:ai_chat_recent_models')).toBeNull();
});
it('does not override a user-selected route when a fallback turn completes', async () => {
it('does not override a user-selected route when a route-switch row completes', async () => {
const [messages, setMessages] = createSignal<ChatMessage[]>([]);
const [selectedModel, setSelectedModel] = createSignal('deepseek:deepseek-v4-pro');
mockChat.messages.mockImplementation(() => messages());
@@ -4652,7 +4648,7 @@ describe('AIChat', () => {
setMessages([
{
id: 'assistant-fallback',
id: 'assistant-route-switch',
role: 'assistant',
content: '',
timestamp: new Date('2026-06-06T12:00:00Z'),
@@ -4672,9 +4668,9 @@ describe('AIChat', () => {
setMessages([
{
id: 'assistant-fallback',
id: 'assistant-route-switch',
role: 'assistant',
content: 'Fallback answer.',
content: 'Route switch answer.',
timestamp: new Date('2026-06-06T12:00:00Z'),
completedAt: new Date('2026-06-06T12:00:03Z'),
model: 'openrouter:deepseek/deepseek-v4-pro',
@@ -4687,7 +4683,7 @@ describe('AIChat', () => {
},
{
type: 'content',
content: 'Fallback answer.',
content: 'Route switch answer.',
},
],
},
@@ -5306,9 +5302,7 @@ describe('AIChat', () => {
const status = screen.getByLabelText('Assistant active turn status');
expect(status).toHaveTextContent('Preparing Pulse context.');
expect(status).not.toHaveTextContent(
'OpenRouter is starting the response.',
);
expect(status).not.toHaveTextContent('OpenRouter is starting the response.');
await vi.advanceTimersByTimeAsync(650);
expect(status).toHaveTextContent('Reading current Pulse inventory.');
@@ -5381,15 +5375,11 @@ describe('AIChat', () => {
expect(status).not.toHaveTextContent('OpenRouter is starting the response.');
await vi.advanceTimersByTimeAsync(650);
expect(status).toHaveTextContent(
'Reading current Pulse inventory. · 1 follow-up queued',
);
expect(status).toHaveTextContent('Reading current Pulse inventory. · 1 follow-up queued');
expect(status).not.toHaveTextContent('pulse_query');
await vi.advanceTimersByTimeAsync(650);
expect(status).toHaveTextContent(
'OpenRouter is starting the response. · 1 follow-up queued',
);
expect(status).toHaveTextContent('OpenRouter is starting the response. · 1 follow-up queued');
expect(status).not.toHaveTextContent('Preparing Pulse context.');
});
@@ -5571,7 +5561,9 @@ describe('AIChat', () => {
await waitFor(() => {
const activityDock = screen.getByTestId('assistant-activity-dock');
expect(activityDock).toContainElement(screen.getByLabelText('Assistant active turn status'));
expect(activityDock).toContainElement(
screen.getByLabelText('Assistant active turn status'),
);
expect(activityDock).toContainElement(
screen.getByRole('status', { name: 'Assistant autonomous control warning' }),
);
@@ -1023,7 +1023,7 @@ describe('MessageItem', () => {
expect(screen.queryByText(/Hidden reasoning/i)).not.toBeInTheDocument();
});
it('renders provider fallback model switches as typed transcript status', () => {
it('renders model switches as typed transcript status', () => {
const events: StreamDisplayEvent[] = [
{ type: 'model_switch', model: 'openrouter:deepseek/deepseek-v4-pro' },
];
@@ -1112,7 +1112,7 @@ describe('MessageItem', () => {
expect(screen.getByText(/Preparing Pulse context\./)).toBeInTheDocument();
});
it('renders provider fallback model switches with failed and next routes', () => {
it('renders model route switches with previous and next routes', () => {
const events: StreamDisplayEvent[] = [
{
type: 'model_switch',
@@ -1136,9 +1136,9 @@ describe('MessageItem', () => {
));
const status = screen.getByRole('status', {
name: 'Assistant provider fallback route changed',
name: 'Assistant model route changed',
});
expect(status).toHaveTextContent('Provider fallback');
expect(status).toHaveTextContent('Switched from');
expect(status).toHaveTextContent('OpenAI GPT-4o Mini via OpenRouter');
expect(status).toHaveTextContent('Gemini 3.1 Flash Lite');
expect(status).toHaveAttribute(
@@ -36,11 +36,7 @@ describe('getAssistantActiveTurnStatus', () => {
it('tracks startup timing from the submitted user turn', () => {
expect(
getAssistantActiveTurnStatus(
[userMessage({ timestamp: new Date(1_000) })],
true,
4_000,
),
getAssistantActiveTurnStatus([userMessage({ timestamp: new Date(1_000) })], true, 4_000),
).toEqual({
type: 'thinking',
text: 'Sending prompt',
@@ -183,8 +179,7 @@ describe('getAssistantActiveTurnStatus', () => {
),
).toEqual({
type: 'tool',
text:
'Running $ curl -H "Authorization: Bearer [redacted-secret]" --password [redacted-secret] https://example.local',
text: 'Running $ curl -H "Authorization: Bearer [redacted-secret]" --password [redacted-secret] https://example.local',
});
});
@@ -652,7 +647,7 @@ describe('getAssistantActiveTurnStatus', () => {
});
});
it('shows failed and next model routes for provider fallback status', () => {
it('shows previous and next model routes for explicit route switches', () => {
expect(
getAssistantActiveTurnStatus(
[
@@ -672,7 +667,7 @@ describe('getAssistantActiveTurnStatus', () => {
),
).toEqual({
type: 'thinking',
text: 'Provider fallback: OpenAI: GPT 4o Mini via OpenRouter -> DeepSeek: DeepSeek V4 Pro via OpenRouter',
text: 'Switched from OpenAI: GPT 4o Mini via OpenRouter to DeepSeek: DeepSeek V4 Pro via OpenRouter',
startedAt: 2_000,
});
});
@@ -1279,7 +1279,7 @@ describe('useChat', () => {
dispose();
});
it('updates the in-flight message model when provider fallback starts', async () => {
it('ignores obsolete provider fallback workflow metadata', async () => {
const { getFireEvent } = setupWithEventCapture();
const { value: chat, dispose } = withRoot(() =>
useChat({ sessionId: 's', model: 'openrouter:openai/gpt-4o-mini' }),
@@ -1299,21 +1299,14 @@ describe('useChat', () => {
});
const assistant = chat.messages().find((m) => m.role === 'assistant')!;
expect(assistant.model).toBe('gemini:gemini-3.1-flash-lite');
expect(assistant.streamEvents).toContainEqual(
expect.objectContaining({
type: 'model_switch',
model: 'gemini:gemini-3.1-flash-lite',
failedModel: 'openrouter:openai/gpt-4o-mini',
startedAt: expect.any(Number),
updatedAt: expect.any(Number),
}),
);
expect(assistant.workflowStatus).toEqual(
expect.objectContaining({
phase: 'provider_fallback',
message: 'OpenRouter did not start a response; trying Gemini.',
}),
expect(assistant.model).toBe('openrouter:openai/gpt-4o-mini');
expect(assistant.streamEvents).not.toEqual(
expect.arrayContaining([
expect.objectContaining({
type: 'model_switch',
model: 'gemini:gemini-3.1-flash-lite',
}),
]),
);
dispose();
});
@@ -1369,7 +1362,7 @@ describe('useChat', () => {
dispose();
});
it('does not infer a failed route when workflow only carries a next model', async () => {
it('does not infer a model switch from legacy next-model workflow metadata', async () => {
const { getFireEvent } = setupWithEventCapture();
const { value: chat, dispose } = withRoot(() =>
useChat({ sessionId: 's', model: 'openrouter:openai/gpt-4o-mini' }),
@@ -1388,13 +1381,14 @@ describe('useChat', () => {
});
const assistant = chat.messages().find((m) => m.role === 'assistant')!;
expect(assistant.model).toBe('gemini:gemini-3.1-flash-lite');
expect(assistant.streamEvents).toContainEqual(
expect.objectContaining({
type: 'model_switch',
model: 'gemini:gemini-3.1-flash-lite',
failedModel: undefined,
}),
expect(assistant.model).toBe('openrouter:openai/gpt-4o-mini');
expect(assistant.streamEvents).not.toEqual(
expect.arrayContaining([
expect.objectContaining({
type: 'model_switch',
model: 'gemini:gemini-3.1-flash-lite',
}),
]),
);
dispose();
});
@@ -2146,7 +2140,8 @@ describe('useChat', () => {
await chat.sendMessage('hi');
const fire = getFireEvent();
const input = '{"action":"exec","command":"ls /dev | wc -l","target_host":"current_resource"}';
const input =
'{"action":"exec","command":"ls /dev | wc -l","target_host":"current_resource"}';
fire({
type: 'tool_start',
@@ -522,7 +522,7 @@ const modelSwitchStatusText = (event: StreamDisplayEvent): string => {
const failed = event.failedModel?.trim();
if (event.modelEvent === 'selected') return `Using ${next}`;
if (!failed || failed === model) return `Switched to ${next}`;
return `Provider fallback: ${formatAIModelRouteLabel(failed)} -> ${next}`;
return `Switched from ${formatAIModelRouteLabel(failed)} to ${next}`;
};
const hasVisibleAssistantOutput = (message: ChatMessage): boolean => {
@@ -664,5 +664,8 @@ export const getAssistantActiveTurnStatus = (
);
}
return withAssistantQueuedFollowUpStatus(initialRequestStatus(messages, assistantMessage), messages);
return withAssistantQueuedFollowUpStatus(
initialRequestStatus(messages, assistantMessage),
messages,
);
};
@@ -496,10 +496,8 @@ export function useChat(options: UseChatOptions = {}) {
};
const streamModelEventKind = (event: StreamDisplayEvent): StreamDisplayEvent['modelEvent'] => {
if (event.modelEvent) return event.modelEvent;
const model = event.model?.trim();
const failed = event.failedModel?.trim();
return model && failed && model !== failed ? 'fallback' : 'switch';
if (event.modelEvent && event.modelEvent !== 'fallback') return event.modelEvent;
return 'switch';
};
const isDurableAssistantStreamBoundary = (event: StreamDisplayEvent): boolean =>
@@ -924,20 +922,6 @@ export function useChat(options: UseChatOptions = {}) {
const extractWorkflowModel = (data: unknown): string => extractCompletedModel(data);
const extractWorkflowNextModel = (data: unknown): string => {
if (!data || typeof data !== 'object') return '';
const record = data as Record<string, unknown>;
const nextModel = record.next_model ?? record.nextModel;
return typeof nextModel === 'string' ? nextModel.trim() : '';
};
const extractWorkflowFailedModel = (data: unknown): string => {
if (!data || typeof data !== 'object') return '';
const record = data as Record<string, unknown>;
const failedModel = record.failed_model ?? record.failedModel;
return typeof failedModel === 'string' ? failedModel.trim() : '';
};
const extractErrorMessage = (data: unknown): string => {
if (typeof data === 'string') return data;
if (data && typeof data === 'object') {
@@ -954,6 +938,7 @@ export function useChat(options: UseChatOptions = {}) {
if (!message) return null;
const phase = typeof record.phase === 'string' ? record.phase.trim() : '';
if (phase === 'provider_fallback') return null;
const state = typeof record.state === 'string' ? record.state.trim() : '';
const tool = typeof record.tool === 'string' ? record.tool.trim() : '';
const attempt = positiveNumber(record.attempt);
@@ -1182,25 +1167,14 @@ export function useChat(options: UseChatOptions = {}) {
if (event.type === 'workflow_state') {
const workflowStatus = extractWorkflowStatus(event.data);
const nextModel = extractWorkflowNextModel(event.data);
const failedModel = extractWorkflowFailedModel(event.data);
const startedModel =
workflowStatus?.phase === 'provider_start' ? extractWorkflowModel(event.data) : '';
const routeEvent = nextModel
const routeEvent = startedModel
? {
model: nextModel,
failedModel,
modelEvent: (failedModel ? 'fallback' : 'switch') as NonNullable<
StreamDisplayEvent['modelEvent']
>,
model: startedModel,
modelEvent: 'selected' as const,
}
: startedModel
? {
model: startedModel,
failedModel: '',
modelEvent: 'selected' as const,
}
: null;
: null;
if (!workflowStatus && !routeEvent) return;
if (routeEvent) {
@@ -1208,7 +1182,6 @@ export function useChat(options: UseChatOptions = {}) {
prev.map((msg) => {
if (msg.id !== assistantId) return msg;
return withModelRouteEvent(msg, routeEvent.model, {
failedModel: routeEvent.failedModel || undefined,
modelEvent: routeEvent.modelEvent,
});
}),
@@ -1480,7 +1453,11 @@ export function useChat(options: UseChatOptions = {}) {
) {
const startedAt = evt.pendingTool?.startedAt || evt.startedAt;
const settleUntil = newToolCall.success
? getAssistantFastToolCompletionSettleUntil(startedAt, completedAt, completedAt)
? getAssistantFastToolCompletionSettleUntil(
startedAt,
completedAt,
completedAt,
)
: undefined;
updatedEvents[i] = {
type: 'tool',
@@ -1505,8 +1482,8 @@ export function useChat(options: UseChatOptions = {}) {
}
// Also remove from pendingApprovals if present
const updatedApprovals = (msg.pendingApprovals || []).filter((a) =>
!matchesCompletedTool(a.toolId, a.toolName),
const updatedApprovals = (msg.pendingApprovals || []).filter(
(a) => !matchesCompletedTool(a.toolId, a.toolName),
);
return {
@@ -268,17 +268,6 @@ interface AIChatProps {
let stashedComposerDraft: ComposerDraftStash | null = null;
interface AssistantFallbackRouteAdoptionCandidate {
route: string;
failedModel: string;
}
interface AssistantFallbackRouteNotice extends AssistantFallbackRouteAdoptionCandidate {
failedModelLabel: string;
messageId: string;
routeLabel: string;
}
interface AssistantProviderReadinessRouteNotice {
failedProviderLabel: string;
failedRouteLabel: string;
@@ -307,34 +296,6 @@ const hasProviderRouteFailureEvidence = (message: ChatMessage): boolean => {
);
};
const getAssistantFallbackRouteAdoptionCandidate = (
message: ChatMessage,
): AssistantFallbackRouteAdoptionCandidate | null => {
if (message.role !== 'assistant' || message.error) return null;
const events = [...(message.streamEvents || [])].reverse();
const modelSwitch = events.find(
(event) =>
event.type === 'model_switch' &&
typeof event.model === 'string' &&
event.model.trim() &&
typeof event.failedModel === 'string' &&
event.failedModel.trim(),
);
if (!modelSwitch) return null;
const route = modelSwitch.model?.trim() || '';
const failedModel = modelSwitch.failedModel?.trim() || '';
if (!route || !failedModel) return null;
const completedModel = (message.model || '').trim();
if (message.isStreaming === false && completedModel && completedModel !== route) {
return null;
}
return { route, failedModel };
};
const compactText = (items: Array<string | undefined>): string[] =>
items.filter((item): item is string => typeof item === 'string' && item.trim().length > 0);
@@ -740,10 +701,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
const [modelSelectorOpenRequest, setModelSelectorOpenRequest] = createSignal(0);
const [defaultModel, setDefaultModel] = createSignal('');
const [chatOverrideModel, setChatOverrideModel] = createSignal('');
const pendingFallbackRouteAdoptions = new Map<string, AssistantFallbackRouteAdoptionCandidate>();
const adoptedFallbackRouteMessageIds = new Set<string>();
const [fallbackRouteNotice, setFallbackRouteNotice] =
createSignal<AssistantFallbackRouteNotice | null>(null);
const [providerReadinessRouteNotice, setProviderReadinessRouteNotice] =
createSignal<AssistantProviderReadinessRouteNotice | null>(null);
const [providerReadiness, setProviderReadiness] = createSignal<ChatProviderReadinessState>({
@@ -2137,7 +2094,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
if (options.rememberRecent !== false) {
rememberRecentModel(modelId);
}
setFallbackRouteNotice(null);
setProviderReadinessRouteNotice(null);
};
@@ -2272,59 +2228,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
return alternative;
};
createEffect(() => {
const messages = chat.messages();
const visibleMessageIds = new Set(messages.map((message) => message.id));
for (const messageId of [...pendingFallbackRouteAdoptions.keys()]) {
if (!visibleMessageIds.has(messageId)) {
pendingFallbackRouteAdoptions.delete(messageId);
}
}
for (const messageId of [...adoptedFallbackRouteMessageIds]) {
if (!visibleMessageIds.has(messageId)) {
adoptedFallbackRouteMessageIds.delete(messageId);
}
}
for (const message of messages) {
const candidate = getAssistantFallbackRouteAdoptionCandidate(message);
if (!candidate) continue;
if (message.isStreaming !== false) {
pendingFallbackRouteAdoptions.set(message.id, candidate);
continue;
}
const pending = pendingFallbackRouteAdoptions.get(message.id);
if (
!pending ||
pending.route !== candidate.route ||
adoptedFallbackRouteMessageIds.has(message.id)
) {
continue;
}
const currentRoute = selectedChatModel().trim();
if (currentRoute && currentRoute !== pending.failedModel) {
pendingFallbackRouteAdoptions.delete(message.id);
continue;
}
adoptedFallbackRouteMessageIds.add(message.id);
pendingFallbackRouteAdoptions.delete(message.id);
selectModel(candidate.route);
const routeLabel = formatChatMessageModelRoute(candidate.route);
const failedModelLabel = formatChatMessageModelRoute(pending.failedModel);
setFallbackRouteNotice({
...candidate,
failedModelLabel,
messageId: message.id,
routeLabel,
});
notificationStore.success(`Assistant model route switched to ${routeLabel}`, 2500);
}
});
createEffect(() => {
const sessionId = chat.sessionId();
const storedModel = getStoredModel(sessionId);
@@ -3108,7 +3011,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
const submittedInput = readComposerInputForSubmit();
const prompt = submittedInput.trim();
if (!prompt) return;
setFallbackRouteNotice(null);
setProviderReadinessRouteNotice(null);
composerSubmitDispatchLocked = true;
queueMicrotask(() => {
@@ -3438,7 +3340,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
resetPromptHistoryNavigation();
setEditingQueuedFollowUp(null);
setRestoredPromptDraft(null);
setFallbackRouteNotice(null);
setProviderReadinessRouteNotice(null);
setRedoLastTurnAvailable(false);
aiChatStore.clearContext?.();
@@ -3533,7 +3434,8 @@ export const AIChat: Component<AIChatProps> = (props) => {
focusComposer();
} catch (error) {
logger.error('[AIChat] Failed to compact Assistant session:', error);
const message = error instanceof Error ? error.message : AI_CHAT_COMPACT_SESSION_ERROR_MESSAGE;
const message =
error instanceof Error ? error.message : AI_CHAT_COMPACT_SESSION_ERROR_MESSAGE;
notificationStore.error(message);
} finally {
setCompactingSession(false);
@@ -3703,7 +3605,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
resetPromptHistoryNavigation();
setEditingQueuedFollowUp(null);
setRestoredPromptDraft(null);
setFallbackRouteNotice(null);
setProviderReadinessRouteNotice(null);
setRedoLastTurnAvailable(false);
}
@@ -4482,7 +4383,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
when={
currentStatus() ||
autonomousWarningVisible() ||
fallbackRouteNotice() ||
providerReadinessRouteNotice() ||
chat.queuedFollowUpCount() > 0
}
@@ -4492,9 +4392,7 @@ export const AIChat: Component<AIChatProps> = (props) => {
data-testid="assistant-activity-dock"
>
<Show when={currentStatus()}>
<div
class="flex min-h-8 min-w-0 items-center gap-2 px-2.5 py-1.5 text-xs"
>
<div class="flex min-h-8 min-w-0 items-center gap-2 px-2.5 py-1.5 text-xs">
<div
class="flex min-w-0 flex-1 items-center gap-2"
role="status"
@@ -4509,9 +4407,7 @@ export const AIChat: Component<AIChatProps> = (props) => {
}`}
aria-hidden="true"
/>
<span class="min-w-0 flex-1 truncate font-medium">
{currentStatusText()}
</span>
<span class="min-w-0 flex-1 truncate font-medium">{currentStatusText()}</span>
<span class="flex shrink-0 gap-0.5" aria-hidden="true">
<span
class="h-1 w-1 rounded-full bg-blue-400 animate-bounce"
@@ -4590,45 +4486,11 @@ export const AIChat: Component<AIChatProps> = (props) => {
</button>
</div>
</Show>
<Show when={fallbackRouteNotice()}>
{(notice) => (
<div
class={`flex min-h-8 min-w-0 items-center gap-2 px-2.5 py-1.5 text-xs ${
currentStatus() || autonomousWarningVisible()
? 'border-t border-border/70'
: ''
}`}
role="status"
aria-label="Assistant fallback route adopted"
aria-live="polite"
>
<CheckIcon
class="h-3.5 w-3.5 shrink-0 text-emerald-600 dark:text-emerald-300"
aria-hidden="true"
/>
<span class="min-w-0 flex-1 truncate font-medium">
Using {notice().routeLabel} after fallback from {notice().failedModelLabel}
</span>
<button
type="button"
onClick={() => {
setFallbackRouteNotice(null);
focusComposer();
}}
class="inline-flex h-6 w-6 shrink-0 items-center justify-center rounded-md text-blue-700 transition-colors hover:bg-blue-100 hover:text-blue-900 dark:text-blue-200 dark:hover:bg-blue-900/50"
title="Dismiss fallback route notice"
aria-label="Dismiss fallback route notice"
>
<XIcon class="h-3.5 w-3.5" aria-hidden="true" />
</button>
</div>
)}
</Show>
<Show when={providerReadinessRouteNotice()}>
{(notice) => (
<div
class={`flex min-h-8 min-w-0 items-center gap-2 px-2.5 py-1.5 text-xs ${
currentStatus() || autonomousWarningVisible() || fallbackRouteNotice()
currentStatus() || autonomousWarningVisible()
? 'border-t border-border/70'
: ''
}`}
@@ -4667,7 +4529,6 @@ export const AIChat: Component<AIChatProps> = (props) => {
class={`px-2.5 py-1.5 ${
currentStatus() ||
autonomousWarningVisible() ||
fallbackRouteNotice() ||
providerReadinessRouteNotice()
? 'border-t border-border/70'
: ''
+3 -60
View File
@@ -7,7 +7,6 @@ import (
"runtime/debug"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/google/uuid"
@@ -28,8 +27,6 @@ type parallelToolResult struct {
Err error
}
const providerFallbackStartupTimeout = 4 * time.Second
func isRetryableProviderStreamError(err error) bool {
if err == nil {
return false
@@ -69,15 +66,6 @@ func isRetryableProviderStreamError(err error) bool {
return false
}
func providerStreamEventShowsStartup(event providers.StreamEvent) bool {
switch strings.ToLower(strings.TrimSpace(event.Type)) {
case "content", "done", "thinking", "tool_progress", "tool_start":
return true
default:
return false
}
}
type workflowStateOption func(*WorkflowStateData)
func withWorkflowRetry(nextAttempt, maxAttempts int, retryAfter time.Duration) workflowStateOption {
@@ -565,14 +553,10 @@ type AgenticLoop struct {
requestSanitizer func(providers.ChatRequest) providers.ChatRequest
// When true, provider terminal errors are returned to the caller without
// emitting a stream error event. The chat service uses this to try another
// configured provider before the browser sees a failed first attempt.
// emitting a stream error event. The chat service uses this to centralize
// terminal provider error presentation at the stream boundary.
suppressProviderErrorEvents bool
// When true, this loop is running as one attempt in a service-owned provider
// fallback chain, so hidden startup retries should yield to the next route.
fastFailProviderStartup bool
// Query-only count/overview turns should not let a provider accidentally
// expand a topology request into the full infrastructure tree.
preferSummaryOnlyQueries bool
@@ -619,12 +603,6 @@ func (a *AgenticLoop) SetSuppressProviderErrorEvents(suppress bool) {
a.suppressProviderErrorEvents = suppress
}
func (a *AgenticLoop) SetFastFailProviderStartup(enabled bool) {
a.mu.Lock()
defer a.mu.Unlock()
a.fastFailProviderStartup = enabled
}
func (a *AgenticLoop) SetPreferSummaryOnlyQueries(prefer bool) {
a.mu.Lock()
defer a.mu.Unlock()
@@ -722,7 +700,6 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
modelName := a.modelName
requestSanitizer := a.requestSanitizer
preferSummaryOnlyQueries := a.preferSummaryOnlyQueries
fastFailProviderStartup := a.fastFailProviderStartup
a.mu.Unlock()
// Record telemetry for loop iteration
@@ -755,9 +732,6 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
Tools: tools,
ExecutionID: a.executionID,
}
if fastFailProviderStartup {
req.ProviderStartupMode = providers.ProviderStartupFastFailBeforeVisibleOutput
}
// Tool selection is model-owned. Pulse normally exposes the governed tool
// manifest unchanged. When a run must stop for safety or budget reasons,
@@ -913,36 +887,13 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
}
maxProviderAttempts := 2
if fastFailProviderStartup {
maxProviderAttempts = 1
}
err := error(nil)
for attempt := 1; attempt <= maxProviderAttempts; attempt++ {
attemptSawDone := false
attemptEmittedVisibleEvents := false
var attemptErrorMessages []string
providerCtx := ctx
var cancelProviderStartup context.CancelFunc
var startupTimedOut atomic.Bool
var startupTimer *time.Timer
stopStartupTimer := func() {}
if fastFailProviderStartup {
providerCtx, cancelProviderStartup = context.WithCancel(ctx)
startupTimer = time.AfterFunc(providerFallbackStartupTimeout, func() {
startupTimedOut.Store(true)
cancelProviderStartup()
})
stopStartupTimer = func() {
if startupTimer != nil {
startupTimer.Stop()
}
}
}
err = a.provider.ChatStream(providerCtx, req, func(event providers.StreamEvent) {
if providerStreamEventShowsStartup(event) {
stopStartupTimer()
}
err = a.provider.ChatStream(ctx, req, func(event providers.StreamEvent) {
switch event.Type {
case "content":
if data, ok := event.Data.(providers.ContentEvent); ok {
@@ -1070,14 +1021,6 @@ func (a *AgenticLoop) executeWithTools(ctx context.Context, sessionID string, me
}
}
})
stopStartupTimer()
if cancelProviderStartup != nil {
cancelProviderStartup()
}
if startupTimedOut.Load() && err != nil {
err = fmt.Errorf("provider startup timed out after %s: %w", providerFallbackStartupTimeout, err)
}
effectiveErr := err
if effectiveErr == nil && len(attemptErrorMessages) > 0 {
effectiveErr = fmt.Errorf("stream error: %s", attemptErrorMessages[0])
+18 -157
View File
@@ -805,31 +805,21 @@ func (s *Service) ExecuteStream(ctx context.Context, req ExecuteRequest, callbac
streamCallback(event)
}
attempts, initialProviderErr := s.chatProviderAttempts(ctx, cfgSnapshot, selectedModel, configuredModel, configuredProvider)
if len(attempts) == 0 {
if initialProviderErr != nil {
return initialProviderErr
}
return fmt.Errorf("provider not initialized")
}
initialFallbackModel := ""
if initialProviderErr != nil {
initialFallbackModel = attempts[0].Model
}
if initialFallbackModel != "" {
emitChatProviderFallback(streamCallback, selectedModel, initialFallbackModel)
attempt, providerErr := s.chatProviderAttempt(selectedModel, configuredModel, configuredProvider)
if providerErr != nil {
return providerErr
}
runAttempt := func(attempt chatProviderAttempt, hasFallback bool) ([]Message, *AgenticLoop, bool, error) {
runAttempt := func(attempt chatProviderAttempt) ([]Message, *AgenticLoop, error) {
attemptProvider := attempt.Provider
if attemptProvider == nil {
if strings.TrimSpace(attempt.Model) == "" {
return nil, nil, false, fmt.Errorf("no chat model configured")
return nil, nil, fmt.Errorf("no chat model configured")
}
var providerErr error
attemptProvider, providerErr = s.createProviderForModel(attempt.Model)
if providerErr != nil {
return nil, nil, false, fmt.Errorf("failed to create provider for model %q: %w", attempt.Model, providerErr)
return nil, nil, fmt.Errorf("failed to create provider for model %q: %w", attempt.Model, providerErr)
}
}
@@ -846,14 +836,13 @@ func (s *Service) ExecuteStream(ctx context.Context, req ExecuteRequest, callbac
}
// Create a per-attempt AgenticLoop to ensure complete isolation between
// concurrent sessions and provider fallback attempts. This prevents race
// concurrent sessions and chat attempts. This prevents race
// conditions where ExecuteStream calls overwrite each other's FSM,
// knowledge accumulator, autonomous mode, budget checker, and provider info.
systemPrompt := s.buildSystemPromptForOfferedTools(filteredTools)
loop := NewAgenticLoop(attemptProvider, executor, systemPrompt)
loop.SetOrgID(s.orgID)
loop.SetAutonomousMode(autonomousMode)
loop.SetFastFailProviderStartup(hasFallback)
loop.SetPreferSummaryOnlyQueries(assistantToolScope == assistantTurnToolScopeQueryOnly)
sanitizerOptions := []modelboundary.RequestSanitizerOption{}
if modelBoundaryAllowedInventoryContext != "" {
@@ -891,7 +880,6 @@ func (s *Service) ExecuteStream(ctx context.Context, req ExecuteRequest, callbac
Str("model", attempt.Model).
Msg("[ChatService] Set session FSM on agentic loop")
attemptVisible := false
attemptCallback := func(event StreamEvent) {
if event.Type == "question" {
var data QuestionData
@@ -899,9 +887,6 @@ func (s *Service) ExecuteStream(ctx context.Context, req ExecuteRequest, callbac
s.registerQuestionLoop(data.QuestionID, loop)
}
}
if chatStreamEventBlocksProviderFallback(event) {
attemptVisible = true
}
wrappedCallback(event)
}
@@ -918,36 +903,17 @@ func (s *Service) ExecuteStream(ctx context.Context, req ExecuteRequest, callbac
Err(err).
Msg("[ChatService] Agentic loop returned")
return resultMessages, loop, attemptVisible, err
return resultMessages, loop, err
}
var resultMessages []Message
var loop *AgenticLoop
var streamErr error
lastAttemptModel := selectedModel
for attemptIndex, attempt := range attempts {
var attemptVisible bool
lastAttemptModel = attempt.Model
resultMessages, loop, attemptVisible, streamErr = runAttempt(attempt, attemptIndex < len(attempts)-1)
if streamErr == nil {
selectedModel = attempt.Model
break
}
if attemptVisible || attemptIndex == len(attempts)-1 || ctx.Err() != nil {
emitChatProviderError(streamCallback, streamErr)
break
}
nextAttempt := attempts[attemptIndex+1]
log.Warn().
Err(streamErr).
Str("failed_model", attempt.Model).
Str("fallback_model", nextAttempt.Model).
Str("session_id", session.ID).
Msg("[ChatService] Provider failed before visible output; trying configured fallback provider")
emitChatProviderFallback(streamCallback, attempt.Model, nextAttempt.Model)
}
lastAttemptModel := attempt.Model
resultMessages, loop, streamErr = runAttempt(attempt)
if streamErr != nil {
emitChatProviderError(streamCallback, streamErr)
// Still save any messages we got
for _, msg := range resultMessages {
if msg.Role == "assistant" && strings.TrimSpace(msg.Model) == "" {
@@ -3041,126 +3007,21 @@ type chatProviderAttempt struct {
Provider providers.StreamingProvider
}
func (s *Service) chatProviderAttempts(_ context.Context, cfg *config.AIConfig, primaryModel, configuredModel string, configuredProvider providers.StreamingProvider) ([]chatProviderAttempt, error) {
var attempts []chatProviderAttempt
var primaryErr error
seen := make(map[string]struct{})
seenProviders := make(map[string]struct{})
addAttempt := func(model string, provider providers.StreamingProvider) {
model = strings.TrimSpace(model)
if model == "" && provider == nil {
return
}
key := model
if key == "" {
key = fmt.Sprintf("__provider_%d", len(attempts))
}
if _, ok := seen[key]; ok {
return
}
seen[key] = struct{}{}
if attemptProvider, _ := config.ParseModelString(model); attemptProvider != "" {
seenProviders[attemptProvider] = struct{}{}
}
attempts = append(attempts, chatProviderAttempt{Model: model, Provider: provider})
}
func (s *Service) chatProviderAttempt(primaryModel, configuredModel string, configuredProvider providers.StreamingProvider) (chatProviderAttempt, error) {
primaryModel = strings.TrimSpace(primaryModel)
configuredModel = strings.TrimSpace(configuredModel)
if primaryModel == "" {
if configuredProvider != nil {
addAttempt("", configuredProvider)
} else {
primaryErr = fmt.Errorf("no chat model configured")
return chatProviderAttempt{Provider: configuredProvider}, nil
}
} else {
provider := configuredProvider
if primaryModel != configuredModel {
provider = nil
}
addAttempt(primaryModel, provider)
return chatProviderAttempt{}, fmt.Errorf("no chat model configured")
}
if cfg == nil {
return attempts, primaryErr
provider := configuredProvider
if primaryModel != configuredModel {
provider = nil
}
primaryProvider, _ := config.ParseModelString(primaryModel)
for _, gatewayModel := range modelresolution.GatewayEquivalentChatModels(cfg, primaryModel) {
addAttempt(gatewayModel, nil)
}
for _, providerName := range cfg.GetConfiguredProviders() {
if providerName == primaryProvider {
continue
}
if _, ok := seenProviders[providerName]; ok {
continue
}
model := chatFallbackModelForProvider(cfg, providerName)
if strings.TrimSpace(model) == "" {
log.Debug().
Str("provider", providerName).
Msg("[ChatService] Skipping provider fallback candidate with no stable chat model")
continue
}
addAttempt(model, nil)
}
return attempts, primaryErr
}
func chatFallbackModelForProvider(cfg *config.AIConfig, provider string) string {
if cfg == nil {
return ""
}
if preferred := strings.TrimSpace(cfg.GetPreferredModelForProvider(provider)); preferred != "" &&
modelresolution.IsModelUsableForChatWithConfig(cfg, preferred) {
return preferred
}
if fallback := strings.TrimSpace(config.DefaultModelForProvider(provider)); fallback != "" &&
modelresolution.IsModelUsableForChatWithConfig(cfg, fallback) {
return fallback
}
return ""
}
func chatStreamEventBlocksProviderFallback(event StreamEvent) bool {
switch strings.ToLower(strings.TrimSpace(event.Type)) {
case "approval_needed", "content", "question", "tool_end", "tool_progress", "tool_start":
return true
default:
return false
}
}
func emitChatProviderFallback(callback StreamCallback, failedModel, nextModel string) {
if callback == nil {
return
}
failedProvider := ""
if strings.TrimSpace(failedModel) != "" {
failedProvider, _ = config.ParseModelString(strings.TrimSpace(failedModel))
}
nextProvider := ""
if strings.TrimSpace(nextModel) != "" {
nextProvider, _ = config.ParseModelString(strings.TrimSpace(nextModel))
}
if strings.TrimSpace(failedProvider) == "" {
failedProvider = "selected provider"
}
if strings.TrimSpace(nextProvider) == "" {
nextProvider = "another configured provider"
}
data, _ := json.Marshal(WorkflowStateData{
Phase: "provider_fallback",
Message: fmt.Sprintf("%s did not start a response; trying %s.", providerLabel(failedProvider), providerLabel(nextProvider)),
State: "provider_fallback",
FailedProvider: failedProvider,
FailedModel: strings.TrimSpace(failedModel),
NextProvider: nextProvider,
NextModel: strings.TrimSpace(nextModel),
})
callback(StreamEvent{Type: "workflow_state", Data: data})
return chatProviderAttempt{Model: primaryModel, Provider: provider}, nil
}
func emitChatProviderError(callback StreamCallback, err error) {
@@ -809,35 +809,19 @@ func TestService_ExecuteStream_SuppressesSessionEventWhenCallerAlreadySentIt(t *
}
}
func TestService_ExecuteStream_FallsBackWhenPrimaryProviderFailsBeforeVisibleOutput(t *testing.T) {
func TestService_ExecuteStream_DoesNotFallbackWhenSelectedProviderFailsBeforeVisibleOutput(t *testing.T) {
tmpDir := t.TempDir()
store, err := NewSessionStore(tmpDir)
if err != nil {
t.Fatalf("failed to create session store: %v", err)
}
var primaryStartupMode providers.ProviderStartupMode
var fallbackStartupMode providers.ProviderStartupMode
var fallbackCalled atomic.Bool
primary := &stubServiceProvider{
streamFn: func(ctx context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
primaryStartupMode = req.ProviderStartupMode
return errors.New("API error (401): unauthorized")
},
}
fallback := &stubServiceProvider{
streamFn: func(ctx context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
fallbackStartupMode = req.ProviderStartupMode
callback(providers.StreamEvent{
Type: "content",
Data: providers.ContentEvent{Text: "PULSE_OK"},
})
callback(providers.StreamEvent{
Type: "done",
Data: providers.DoneEvent{InputTokens: 1, OutputTokens: 2},
})
return nil
},
}
service := NewService(Config{
AIConfig: &config.AIConfig{
@@ -851,9 +835,7 @@ func TestService_ExecuteStream_FallsBackWhenPrimaryProviderFailsBeforeVisibleOut
service.sessions = store
service.provider = primary
service.providerFactory = func(model string) (providers.StreamingProvider, error) {
if model == "gemini:gemini-test" {
return fallback, nil
}
fallbackCalled.Store(true)
return nil, errors.New("unexpected model " + model)
}
service.started = true
@@ -863,7 +845,7 @@ func TestService_ExecuteStream_FallsBackWhenPrimaryProviderFailsBeforeVisibleOut
var fallbackEvents int
var doneModel string
err = service.ExecuteStream(context.Background(), ExecuteRequest{
SessionID: "fallback-before-visible-output",
SessionID: "no-fallback-before-visible-output",
Prompt: "reply",
}, func(event StreamEvent) {
switch event.Type {
@@ -891,29 +873,26 @@ func TestService_ExecuteStream_FallsBackWhenPrimaryProviderFailsBeforeVisibleOut
doneModel = data.Model
}
})
if err != nil {
t.Fatalf("ExecuteStream failed: %v", err)
if err == nil {
t.Fatal("expected selected provider error")
}
if got := content.String(); got != "PULSE_OK" {
t.Fatalf("content = %q, want fallback response", got)
if got := content.String(); got != "" {
t.Fatalf("content = %q, want no hidden alternate-route response", got)
}
if errorEvents != 0 {
t.Fatalf("expected hidden primary failure without client error event, got %d", errorEvents)
if errorEvents != 1 {
t.Fatalf("error events = %d, want selected provider failure event", errorEvents)
}
if fallbackEvents != 1 {
t.Fatalf("fallback workflow events = %d, want 1", fallbackEvents)
if fallbackEvents != 0 {
t.Fatalf("fallback workflow events = %d, want 0", fallbackEvents)
}
if doneModel != "gemini:gemini-test" {
t.Fatalf("done model = %q, want fallback model", doneModel)
if doneModel != "" {
t.Fatalf("done model = %q, want no done event", doneModel)
}
if primaryStartupMode != providers.ProviderStartupFastFailBeforeVisibleOutput {
t.Fatalf("primary startup mode = %q, want fast-fail before fallback", primaryStartupMode)
}
if fallbackStartupMode != providers.ProviderStartupDefault {
t.Fatalf("fallback startup mode = %q, want provider default", fallbackStartupMode)
if fallbackCalled.Load() {
t.Fatal("providerFactory should not be called for a hidden alternate route")
}
messages, err := store.GetMessages("fallback-before-visible-output")
messages, err := store.GetMessages("no-fallback-before-visible-output")
if err != nil {
t.Fatalf("GetMessages failed: %v", err)
}
@@ -924,7 +903,7 @@ func TestService_ExecuteStream_FallsBackWhenPrimaryProviderFailsBeforeVisibleOut
}
}
func TestService_ExecuteStream_FallsBackToSameModelGatewayRouteBeforeDefaults(t *testing.T) {
func TestService_ExecuteStream_DoesNotFallbackToSameModelGatewayRoute(t *testing.T) {
tmpDir := t.TempDir()
store, err := NewSessionStore(tmpDir)
if err != nil {
@@ -932,25 +911,13 @@ func TestService_ExecuteStream_FallsBackToSameModelGatewayRouteBeforeDefaults(t
}
var primaryCalls atomic.Int32
var providerFactoryCalls atomic.Int32
primary := &stubServiceProvider{
streamFn: func(ctx context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
primaryCalls.Add(1)
return errors.New("dial tcp: i/o timeout")
},
}
fallback := &stubServiceProvider{
streamFn: func(ctx context.Context, req providers.ChatRequest, callback providers.StreamCallback) error {
callback(providers.StreamEvent{
Type: "content",
Data: providers.ContentEvent{Text: "GATEWAY_OK"},
})
callback(providers.StreamEvent{
Type: "done",
Data: providers.DoneEvent{InputTokens: 1, OutputTokens: 2},
})
return nil
},
}
service := NewService(Config{
AIConfig: &config.AIConfig{
@@ -963,18 +930,16 @@ func TestService_ExecuteStream_FallsBackToSameModelGatewayRouteBeforeDefaults(t
service.sessions = store
service.provider = primary
service.providerFactory = func(model string) (providers.StreamingProvider, error) {
if model == "openrouter:deepseek/deepseek-v4-pro" {
return fallback, nil
}
providerFactoryCalls.Add(1)
return nil, errors.New("unexpected model " + model)
}
service.started = true
var content strings.Builder
var fallbackNextModel string
var fallbackEvents int
var doneModel string
err = service.ExecuteStream(context.Background(), ExecuteRequest{
SessionID: "fallback-to-same-model-gateway",
SessionID: "no-fallback-to-same-model-gateway",
Prompt: "reply",
}, func(event StreamEvent) {
switch event.Type {
@@ -990,7 +955,7 @@ func TestService_ExecuteStream_FallsBackToSameModelGatewayRouteBeforeDefaults(t
t.Fatalf("unmarshal workflow state: %v", err)
}
if data.Phase == "provider_fallback" {
fallbackNextModel = data.NextModel
fallbackEvents++
}
case "done":
var data DoneData
@@ -1000,106 +965,61 @@ func TestService_ExecuteStream_FallsBackToSameModelGatewayRouteBeforeDefaults(t
doneModel = data.Model
}
})
if err != nil {
t.Fatalf("ExecuteStream failed: %v", err)
if err == nil {
t.Fatal("expected selected provider error")
}
if got := content.String(); got != "GATEWAY_OK" {
t.Fatalf("content = %q, want gateway fallback response", got)
if got := content.String(); got != "" {
t.Fatalf("content = %q, want no gateway fallback response", got)
}
if fallbackNextModel != "openrouter:deepseek/deepseek-v4-pro" {
t.Fatalf("fallback next model = %q, want same-model gateway route", fallbackNextModel)
if fallbackEvents != 0 {
t.Fatalf("fallback workflow events = %d, want 0", fallbackEvents)
}
if doneModel != "openrouter:deepseek/deepseek-v4-pro" {
t.Fatalf("done model = %q, want same-model gateway route", doneModel)
if doneModel != "" {
t.Fatalf("done model = %q, want no done event", doneModel)
}
if got := primaryCalls.Load(); got != 1 {
t.Fatalf("primary provider calls = %d, want one fast-fail call before fallback", got)
if got := primaryCalls.Load(); got != 2 {
t.Fatalf("primary provider calls = %d, want two selected-route retry calls", got)
}
if got := providerFactoryCalls.Load(); got != 0 {
t.Fatalf("provider factory calls = %d, want no automatic gateway provider", got)
}
}
func TestService_ChatProviderAttemptsUsesStableFallbackModelsWithoutLiveCatalog(t *testing.T) {
func TestService_ChatProviderAttemptUsesSelectedRouteOnly(t *testing.T) {
service := &Service{}
cfg := &config.AIConfig{
ChatModel: "openrouter:qwen/qwen3.7-plus",
DiscoveryModel: "gemini:gemini-3.1-flash-lite",
OpenRouterAPIKey: "sk-or-test",
GeminiAPIKey: "gemini-test",
DeepSeekAPIKey: "deepseek-test",
}
attempts, err := service.chatProviderAttempts(
context.Background(),
cfg,
attempt, err := service.chatProviderAttempt(
"openrouter:qwen/qwen3.7-plus",
"openrouter:qwen/qwen3.7-plus",
&stubServiceProvider{},
)
if err != nil {
t.Fatalf("chatProviderAttempts failed: %v", err)
t.Fatalf("chatProviderAttempt failed: %v", err)
}
got := make([]string, 0, len(attempts))
for _, attempt := range attempts {
got = append(got, attempt.Model)
if attempt.Model != "openrouter:qwen/qwen3.7-plus" {
t.Fatalf("attempt model = %q, want selected route", attempt.Model)
}
want := []string{
"openrouter:qwen/qwen3.7-plus",
"deepseek:deepseek-v4-flash",
"gemini:gemini-3.1-flash-lite",
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("attempt models = %#v, want %#v", got, want)
}
if attempts[0].Provider == nil {
t.Fatal("primary attempt should reuse the configured provider")
}
for i, attempt := range attempts[1:] {
if attempt.Provider != nil {
t.Fatalf("fallback attempt %d should be lazy provider creation", i+1)
}
if attempt.Provider == nil {
t.Fatal("selected configured route should reuse the configured provider")
}
}
func TestService_ChatProviderAttemptsPrefersSameModelGatewayFallbackBeforeProviderDefaults(t *testing.T) {
func TestService_ChatProviderAttemptDoesNotAddGatewayEquivalentRoute(t *testing.T) {
service := &Service{}
cfg := &config.AIConfig{
ChatModel: "deepseek:deepseek-v4-pro",
DiscoveryModel: "gemini:gemini-3.1-flash-lite",
OpenRouterAPIKey: "sk-or-test",
DeepSeekAPIKey: "deepseek-test",
GeminiAPIKey: "gemini-test",
}
attempts, err := service.chatProviderAttempts(
context.Background(),
cfg,
attempt, err := service.chatProviderAttempt(
"deepseek:deepseek-v4-pro",
"deepseek:deepseek-v4-pro",
&stubServiceProvider{},
)
if err != nil {
t.Fatalf("chatProviderAttempts failed: %v", err)
t.Fatalf("chatProviderAttempt failed: %v", err)
}
got := make([]string, 0, len(attempts))
for _, attempt := range attempts {
got = append(got, attempt.Model)
if attempt.Model != "deepseek:deepseek-v4-pro" {
t.Fatalf("attempt model = %q, want selected route only", attempt.Model)
}
want := []string{
"deepseek:deepseek-v4-pro",
"openrouter:deepseek/deepseek-v4-pro",
"gemini:gemini-3.1-flash-lite",
}
if !reflect.DeepEqual(got, want) {
t.Fatalf("attempt models = %#v, want %#v", got, want)
}
if attempts[0].Provider == nil {
t.Fatal("primary attempt should reuse the configured provider")
}
for i, attempt := range attempts[1:] {
if attempt.Provider != nil {
t.Fatalf("fallback attempt %d should be lazy provider creation", i+1)
}
if attempt.Provider == nil {
t.Fatal("selected configured route should reuse the configured provider")
}
}
+9 -13
View File
@@ -272,19 +272,15 @@ type ThinkingData struct {
// WorkflowStateData is the data for "workflow_state" events.
type WorkflowStateData struct {
Phase string `json:"phase"` // investigate | clarify | plan | approve | execute | verify | complete
Message string `json:"message"` // Human-readable status text for the UI
State string `json:"state,omitempty"` // Backend workflow state, when available
Tool string `json:"tool,omitempty"` // Tool associated with this transition
Provider string `json:"provider,omitempty"` // Provider selected for this workflow step
Model string `json:"model,omitempty"` // Model route selected for this workflow step
Attempt int `json:"attempt,omitempty"` // One-based attempt number for transient provider retries
MaxAttempts int `json:"max_attempts,omitempty"` // Maximum attempts available for the current provider request
RetryAfterMS int64 `json:"retry_after_ms,omitempty"` // Milliseconds until the next provider attempt starts
FailedProvider string `json:"failed_provider,omitempty"` // Provider that failed before visible output
FailedModel string `json:"failed_model,omitempty"` // Model that failed before visible output
NextProvider string `json:"next_provider,omitempty"` // Provider selected for the fallback attempt
NextModel string `json:"next_model,omitempty"` // Model selected for the fallback attempt
Phase string `json:"phase"` // investigate | clarify | plan | approve | execute | verify | complete
Message string `json:"message"` // Human-readable status text for the UI
State string `json:"state,omitempty"` // Backend workflow state, when available
Tool string `json:"tool,omitempty"` // Tool associated with this transition
Provider string `json:"provider,omitempty"` // Provider selected for this workflow step
Model string `json:"model,omitempty"` // Model route selected for this workflow step
Attempt int `json:"attempt,omitempty"` // One-based attempt number for transient provider retries
MaxAttempts int `json:"max_attempts,omitempty"` // Maximum attempts available for the current provider request
RetryAfterMS int64 `json:"retry_after_ms,omitempty"` // Milliseconds until the next provider attempt starts
}
// SessionData is the data for "session" events emitted once the backend has
-8
View File
@@ -426,10 +426,6 @@ func (c *GeminiClient) Chat(ctx context.Context, req ChatRequest) (*ChatResponse
var lastErr error
maxRetries := geminiMaxRetries
if req.FastFailProviderStartup() {
maxRetries = 0
}
for attempt := 0; attempt <= maxRetries; attempt++ {
if attempt > 0 {
// Exponential backoff: 2s, 4s, 8s
@@ -833,10 +829,6 @@ func (c *GeminiClient) ChatStream(ctx context.Context, req ChatRequest, callback
var lastErr error
maxRetries := geminiMaxRetries
if req.FastFailProviderStartup() {
maxRetries = 0
}
for attempt := 0; attempt <= maxRetries; attempt++ {
// Bail out early if the parent context is already cancelled
if ctx.Err() != nil {
-6
View File
@@ -440,9 +440,6 @@ func (c *OpenAIClient) Chat(ctx context.Context, req ChatRequest) (*ChatResponse
var respBody []byte
var lastErr error
maxRetries := openaiMaxRetries
if req.FastFailProviderStartup() {
maxRetries = 0
}
for attempt := 0; attempt <= maxRetries; attempt++ {
if attempt > 0 {
@@ -912,9 +909,6 @@ func (c *OpenAIClient) ChatStream(ctx context.Context, req ChatRequest, callback
streamClient = c.client
}
maxRetries := openaiStreamMaxRetries
if req.FastFailProviderStartup() {
maxRetries = 0
}
for attempt := 0; attempt <= maxRetries; attempt++ {
if attempt > 0 {
-20
View File
@@ -181,26 +181,6 @@ func TestOpenAIClient_ChatStream_RetriesTransientStartupError(t *testing.T) {
assert.Equal(t, "ok", content)
}
func TestOpenAIClient_ChatStream_FastFailSkipsTransientStartupRetry(t *testing.T) {
client := NewOpenAIClient("sk-test", "gpt-4", "https://example.invalid/v1", 0)
attempts := 0
client.streamClient = &http.Client{
Transport: roundTripFunc(func(r *http.Request) (*http.Response, error) {
attempts++
return nil, io.ErrUnexpectedEOF
}),
}
err := client.ChatStream(context.Background(), ChatRequest{
Messages: []Message{{Role: "user", Content: "Hi"}},
ProviderStartupMode: ProviderStartupFastFailBeforeVisibleOutput,
}, func(StreamEvent) {})
require.Error(t, err)
assert.Equal(t, 1, attempts)
assert.Contains(t, err.Error(), "request failed after 0 stream retries")
}
func TestNewOpenAIClient_BoundsStreamResponseHeaderTimeout(t *testing.T) {
client := NewOpenAIClient("sk-test", "gpt-4", "https://api.openai.com/v1", 0)
transport, ok := client.streamClient.Transport.(*http.Transport)
+8 -24
View File
@@ -89,28 +89,16 @@ type ToolChoice struct {
Type ToolChoiceType `json:"type"`
}
type ProviderStartupMode string
const (
// ProviderStartupDefault keeps provider-owned startup retries enabled.
ProviderStartupDefault ProviderStartupMode = ""
// ProviderStartupFastFailBeforeVisibleOutput is used when the caller has a
// separate fallback route ready and wants startup failures to return quickly
// before any user-visible provider output is emitted.
ProviderStartupFastFailBeforeVisibleOutput ProviderStartupMode = "fast_fail_before_visible_output"
)
// ChatRequest represents a request to the AI provider
type ChatRequest struct {
Messages []Message `json:"messages"`
Model string `json:"model"`
ExecutionID string `json:"execution_id,omitempty"` // Stable higher-level run ID shared across related provider turns
MaxTokens int `json:"max_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
System string `json:"system,omitempty"` // System prompt (Anthropic style)
Tools []Tool `json:"tools,omitempty"` // Available tools
ToolChoice *ToolChoice `json:"tool_choice,omitempty"` // nil = model-owned automatic selection; none = text-only safety brake
ProviderStartupMode ProviderStartupMode `json:"-"`
Messages []Message `json:"messages"`
Model string `json:"model"`
ExecutionID string `json:"execution_id,omitempty"` // Stable higher-level run ID shared across related provider turns
MaxTokens int `json:"max_tokens,omitempty"`
Temperature float64 `json:"temperature,omitempty"`
System string `json:"system,omitempty"` // System prompt (Anthropic style)
Tools []Tool `json:"tools,omitempty"` // Available tools
ToolChoice *ToolChoice `json:"tool_choice,omitempty"` // nil = model-owned automatic selection; none = text-only safety brake
}
func (r ChatRequest) NormalizeCollections() ChatRequest {
@@ -129,10 +117,6 @@ func (r ChatRequest) NormalizeCollections() ChatRequest {
return r
}
func (r ChatRequest) FastFailProviderStartup() bool {
return r.ProviderStartupMode == ProviderStartupFastFailBeforeVisibleOutput
}
// ChatResponse represents a response from the AI provider
type ChatResponse struct {
Content string `json:"content"`
+7 -9
View File
@@ -10605,17 +10605,15 @@ func TestContract_ChatStreamEventJSONSnapshots(t *testing.T) {
want: `{"type":"content","data":{"text":"hello"}}`,
},
{
name: "workflow_state",
name: "workflow_state_provider_start",
event: mustStreamEvent(t, "workflow_state", chat.WorkflowStateData{
Phase: "provider_fallback",
Message: "OpenRouter did not start a response; trying DeepSeek.",
State: "provider_fallback",
FailedProvider: "openrouter",
FailedModel: "openrouter:qwen/qwen3.7-plus",
NextProvider: "deepseek",
NextModel: "deepseek:deepseek-v4-pro",
Phase: "provider_start",
Message: "OpenRouter is starting the response.",
State: "investigating",
Provider: "openrouter",
Model: "openrouter:qwen/qwen3.7-plus",
}),
want: `{"type":"workflow_state","data":{"phase":"provider_fallback","message":"OpenRouter did not start a response; trying DeepSeek.","state":"provider_fallback","failed_provider":"openrouter","failed_model":"openrouter:qwen/qwen3.7-plus","next_provider":"deepseek","next_model":"deepseek:deepseek-v4-pro"}}`,
want: `{"type":"workflow_state","data":{"phase":"provider_start","message":"OpenRouter is starting the response.","state":"investigating","provider":"openrouter","model":"openrouter:qwen/qwen3.7-plus"}}`,
},
{
name: "workflow_state_provider_retry",
@@ -2860,7 +2860,7 @@ class SubsystemLookupTest(unittest.TestCase):
{
"heading": "## Shared Boundaries",
"path": "internal/api/access_control_handlers.go",
"line": 417,
"line": 427,
"heading_line": 117,
}
],