feat(ai): surface the Assistant at first AI setup and fix blessed-path readiness

Telemetry shows thousands of installs configure an AI provider but almost
none ever use the interactive Assistant. A live first-session exercise
(fresh install, Ollama qwen3:8b quickstart) found why: after enabling,
nothing changes on screen — the launcher and handoff buttons are gated on
sessionCapabilities.assistantEnabled, which was only read at page load;
the empty transcript was blank; and the blessed Ollama+qwen3:8b path
reported Patrol degraded while telling the user to pull the model they
had just selected.

- Setup-modal success now opens the Assistant drawer, and the AI settings
  save paths refresh the assistantEnabled capability in place
  (aiChatStore.refreshEnabledFromServer) so entry points appear without a
  reload; toasts point at the Assistant instead of back at settings.
- The empty transcript owns a plain-language welcome and three suggested
  prompts that dispatch as real turns (ASSISTANT_SUGGESTED_PROMPTS).
- Patrol static readiness: the blessed Ollama Patrol model is Ready;
  other Ollama models keep the warning, now naming the selected model.
- pulse_summarize fleet argument errors instruct the model to enumerate
  resources itself instead of interrogating the operator (observed live:
  'how is my machine doing?' ended in a resource-ID elicitation).

Contracts: ai-runtime and frontend-primitives Current State updated.
Tests: full internal/ai + internal/api suites green; vitest ChatMessages,
AISettings, aiChat store, and settingsArchitecture suites green; flow
verified live end-to-end.
This commit is contained in:
rcourtman
2026-07-17 23:16:25 +01:00
parent 42c4d7549c
commit 817aaeabc1
14 changed files with 235 additions and 9 deletions
@@ -3870,6 +3870,24 @@ resolve canonical/source IDs and unique aliases before collection, reject
## Current State
First-session assistant discoverability is now a contract concern. Successful
first-time provider setup (the Provider & Models setup modal) must open the
Assistant drawer and refresh the session `assistantEnabled` capability in
place, so the launcher and per-surface handoff buttons appear without a page
reload; the enable/save toasts point at the Assistant rather than back at
settings. The Assistant's empty transcript owns a plain-language welcome plus
static suggested prompts (`ASSISTANT_SUGGESTED_PROMPTS` in `ChatMessages.tsx`)
that dispatch as real chat turns; server-driven workflow starters remain a
separate composer affordance. Patrol static readiness treats the blessed
Ollama Patrol model (`config.OllamaSuggestedPatrolModel`) as Ready — the
recommended free path must not report itself degraded — while other Ollama
models keep the unverified-tool-support warning, which now names the selected
model instead of instructing the operator to pull the model they already
selected. `pulse_summarize` fleet-mode argument errors instruct the model to
enumerate resources itself rather than interrogate the operator for resource
IDs; assistant-facing tool errors must prefer self-recovery guidance over
operator questions.
Automatic Watch detection now carries a declining 7,000-token run allowance:
normal structured turns receive at most 2,048 output tokens, terminal summaries
receive at most 1,024, and the remaining allowance is projected on every
@@ -2304,6 +2304,14 @@ default` instead of fusing provider and badge text such as
## Current State
Assistant availability in the app shell is derived from the
`sessionCapabilities.assistantEnabled` security-status capability, and
`aiChatStore.refreshEnabledFromServer()` is the canonical way to re-derive it
mid-session. AI settings save paths (setup modal, enable toggle, Provider &
Models save) must call it after a successful save so assistant entry points
appear or disappear without a full reload; no surface may flip
`aiChatStore.setEnabled` from settings state directly.
Patrol `fix_rejected` presentation is owned by the Patrol/AI finding surfaces
that render the governed-action loop, while the surrounding badge, button,
loading, and icon composition still uses shared frontend primitives. Shared
@@ -41,8 +41,18 @@ interface ChatMessagesProps {
// Dashboard props
recentSessions?: ChatSession[];
onLoadSession?: (sessionId: string) => void;
// Empty-state affordance: sends the example prompt as a chat turn.
onSuggestedPrompt?: (prompt: string) => void;
}
// Plain-language example prompts for the empty transcript. These teach a
// first-run user what the Assistant is for; each sends as a real turn.
export const ASSISTANT_SUGGESTED_PROMPTS = [
'How is my infrastructure looking right now?',
'Are there any alerts I should look at?',
"What's using the most CPU?",
] as const;
/**
* ChatMessages - Renders the scrollable message list.
*
@@ -265,6 +275,30 @@ export const ChatMessages: Component<ChatMessagesProps> = (props) => {
data-testid="assistant-message-list"
onScroll={updatePinnedToBottom}
>
<Show when={props.messages.length === 0 && props.onSuggestedPrompt}>
<section class="mb-4 w-full" aria-label="Assistant welcome" data-testid="assistant-welcome">
<p class="text-sm text-base-content">
Ask about your infrastructure. The Assistant sees your live inventory, metrics, and
alerts, and can dig into problems with you.
</p>
<div class="mt-2 text-[11px] font-semibold uppercase text-muted">Try one</div>
<div class="mt-1.5 space-y-1.5">
<For each={ASSISTANT_SUGGESTED_PROMPTS}>
{(prompt) => (
<button
type="button"
class="w-full rounded-md border border-border bg-surface px-3 py-2 text-left text-sm text-base-content transition-colors hover:border-blue-300 hover:bg-surface-alt focus:outline-none focus:ring-2 focus:ring-blue-500/30"
onClick={() => props.onSuggestedPrompt?.(prompt)}
data-testid="assistant-suggested-prompt"
>
{prompt}
</button>
)}
</For>
</div>
</section>
</Show>
<Show
when={props.messages.length === 0 && recentSessions().length > 0 && props.onLoadSession}
>
@@ -131,6 +131,28 @@ describe('ChatMessages', () => {
expect(screen.queryByText('Ask about your infrastructure')).not.toBeInTheDocument();
});
it('shows the welcome and suggested prompts in an empty transcript when wired', () => {
const onSuggestedPrompt = vi.fn();
render(() => (
<ChatMessages messages={[]} {...makeHandlers()} onSuggestedPrompt={onSuggestedPrompt} />
));
expect(screen.getByTestId('assistant-welcome')).toBeInTheDocument();
const prompts = screen.getAllByTestId('assistant-suggested-prompt');
expect(prompts).toHaveLength(3);
fireEvent.click(prompts[0]);
expect(onSuggestedPrompt).toHaveBeenCalledWith('How is my infrastructure looking right now?');
});
it('hides the welcome once messages exist', () => {
render(() => (
<ChatMessages messages={[makeMessage()]} {...makeHandlers()} onSuggestedPrompt={vi.fn()} />
));
expect(screen.queryByTestId('assistant-welcome')).not.toBeInTheDocument();
});
it('shows recent sessions as resume actions in an empty transcript', () => {
const onLoadSession = vi.fn();
render(() => (
@@ -4894,6 +4894,9 @@ export const AIChat: Component<AIChatProps> = (props) => {
onUseModelRoute={switchToModelRoute}
queuedFollowUps={chat.queuedFollowUps()}
queuedFollowUpsPaused={chat.queuedFollowUpsPaused()}
onSuggestedPrompt={(prompt) => {
void chat.sendMessage(prompt, undefined, undefined);
}}
onEditQueuedFollowUp={editQueuedFollowUp}
onCancelQueuedFollowUp={(id) => {
chat.cancelQueuedFollowUp(id);
@@ -3,6 +3,7 @@ import { cleanup, fireEvent, render, screen, waitFor, within } from '@solidjs/te
import { Route, Router } from '@solidjs/router';
import { PULSE_MCP_TOKEN_SETUP_PATH } from '@/routing/resourceLinks';
import { aiChatStore } from '@/stores/aiChat';
import { resetAIRuntimeState } from '@/stores/aiRuntimeState';
import type { AISettings as AISettingsType } from '@/types/ai';
import {
@@ -1105,6 +1106,7 @@ describe('AISettings provider setup flow', () => {
afterEach(() => {
cleanup();
aiChatStore.close();
});
it('warns from setup when the saved provider leaves Patrol not ready', async () => {
@@ -1161,10 +1163,52 @@ describe('AISettings provider setup flow', () => {
expect(message).toContain('Model: openrouter:deepseek/deepseek-r1');
expect(message).toContain('reasoning-only model family');
expect(notificationSuccessMock).not.toHaveBeenCalledWith(
'Pulse Intelligence enabled. You can customize settings below.',
expect.stringContaining('Pulse Intelligence enabled. This is the Assistant'),
);
});
it('opens the Assistant and points at it after a successful first-time setup', async () => {
updateSettingsMock.mockResolvedValue({
...baseSettings(),
enabled: true,
configured: true,
anthropic_configured: true,
configured_providers: ['anthropic'],
patrol_readiness: {
status: 'ready',
ready: true,
cause: 'none',
summary: 'Patrol is ready to run tool-backed verification.',
provider: 'anthropic',
model: 'anthropic:claude-sonnet-5',
checks: [],
},
});
renderComponent();
await waitFor(() => {
expect(getSettingsMock).toHaveBeenCalledTimes(1);
});
fireEvent.click(screen.getByRole('button', { name: /enable pulse intelligence/i }));
const setupDialog = await screen.findByRole('dialog', {
name: 'Set up Pulse Intelligence',
});
fireEvent.click(within(setupDialog).getByRole('button', { name: /Anthropic/i }));
fireEvent.input(within(setupDialog).getByPlaceholderText('sk-ant-...'), {
target: { value: 'sk-ant-test' },
});
fireEvent.click(within(setupDialog).getByRole('button', { name: 'Enable Pulse Intelligence' }));
await waitFor(() => {
expect(notificationSuccessMock).toHaveBeenCalledWith(
'Pulse Intelligence enabled. This is the Assistant — ask it anything about your infrastructure.',
);
});
expect(aiChatStore.isOpen).toBe(true);
});
it('names the setup provider when provider setup save fails generically', async () => {
updateSettingsMock.mockRejectedValue(new Error('Unable to save Provider & Models settings.'));
@@ -700,6 +700,21 @@ describe('settings architecture guardrails', () => {
expect(aiSettingsStateSource).not.toContain('OpenRouter returned 401');
});
it('reveals the Assistant after AI settings saves without a page reload', () => {
// Every successful AI settings save path must re-derive the session
// assistantEnabled capability so the launcher and handoff buttons
// appear (or disappear) mid-session, and first-time setup must open
// the Assistant drawer at the moment of maximum intent.
const refreshCalls = aiSettingsStateSource.match(
/aiChatStore\.refreshEnabledFromServer\(\)/g,
);
expect(refreshCalls?.length ?? 0).toBeGreaterThanOrEqual(3);
expect(aiSettingsStateSource).toContain('aiChatStore.open()');
expect(aiSettingsStateSource).not.toContain(
'Pulse Intelligence enabled. You can customize settings below.',
);
});
it('keeps Assistant and Patrol provider-specific fields model-driven', () => {
expect(aiSettingsModelSource).toContain('export const AI_PROVIDERS: AIProvider[] = [');
for (const provider of ['zai', 'groq', 'mistral', 'cerebras', 'together', 'fireworks']) {
@@ -22,6 +22,7 @@ import {
import { hasFeature, loadRuntimeCapabilities } from '@/stores/license';
import { getUpgradeActionDestination } from '@/stores/licenseCommercial';
import { presentationPolicyHidesUpgradePrompts } from '@/stores/sessionPresentationPolicy';
import { aiChatStore } from '@/stores/aiChat';
import { notificationStore } from '@/stores/notifications';
import type { AISettings as AISettingsType, AIProvider, AuthMethod, ModelInfo } from '@/types/ai';
import { normalizeAIControlLevel, type AIControlLevel } from '@/utils/aiControlLevelPresentation';
@@ -839,6 +840,12 @@ export const useAISettingsState = (options: AISettingsStateOptions = {}) => {
hydratePatrolPreflightFromSettings(updated);
void runProviderPreflight(updated);
handleCloseSetupModal();
// First-time configuration is the moment of maximum intent:
// reveal the assistant entry points (the bootstrap only reads
// the capability on page load) and open the Assistant so the
// user meets the feature they just enabled.
void aiChatStore.refreshEnabledFromServer();
aiChatStore.open();
const patrolReadinessMessage = getAISettingsPatrolReadinessSaveMessage(
updated.patrol_readiness,
'Pulse Intelligence enabled',
@@ -846,7 +853,9 @@ export const useAISettingsState = (options: AISettingsStateOptions = {}) => {
if (patrolReadinessMessage) {
notificationStore.warning(patrolReadinessMessage);
} else {
notificationStore.success('Pulse Intelligence enabled. You can customize settings below.');
notificationStore.success(
'Pulse Intelligence enabled. This is the Assistant — ask it anything about your infrastructure.',
);
}
} catch (error) {
logger.error('[AISettings] Setup failed:', error);
@@ -1074,6 +1083,7 @@ export const useAISettingsState = (options: AISettingsStateOptions = {}) => {
syncModelCatalogForSettings(updated);
hydratePatrolPreflightFromSettings(updated);
void runProviderPreflight(updated);
void aiChatStore.refreshEnabledFromServer();
const savedLabel = options.savedLabel ?? 'Provider & Models settings saved';
const patrolReadinessMessage = getAISettingsPatrolReadinessSaveMessage(
updated.patrol_readiness,
@@ -1263,8 +1273,11 @@ export const useAISettingsState = (options: AISettingsStateOptions = {}) => {
setSettings(updated);
syncModelCatalogForSettings(updated);
void runProviderPreflight(updated);
void aiChatStore.refreshEnabledFromServer();
notificationStore.success(
newValue ? 'Pulse Intelligence enabled' : 'Pulse Intelligence disabled',
newValue
? 'Pulse Intelligence enabled. Ask the Assistant anything from the sparkles button on the right edge.'
: 'Pulse Intelligence disabled',
);
} catch (error) {
setForm('enabled', !newValue);
@@ -1,9 +1,17 @@
import { readFileSync } from 'node:fs';
import { resolve } from 'node:path';
import { describe, expect, it, beforeEach } from 'vitest';
import { describe, expect, it, beforeEach, vi } from 'vitest';
import { aiChatStore } from '@/stores/aiChat';
import { eventBus } from '@/stores/events';
const getSecurityStatusMock = vi.fn();
vi.mock('@/api/security', () => ({
SecurityAPI: {
getStatus: (...args: unknown[]) => getSecurityStatusMock(...args),
},
}));
const aiChatSource = readFileSync(
resolve(process.cwd(), 'src/components/AI/Chat/index.tsx'),
'utf8',
@@ -69,6 +77,27 @@ describe('aiChatStore', () => {
expect(aiChatStore.commandRequest).toMatchObject({ action: 'providers' });
});
it('re-derives enabled from the session assistantEnabled capability', async () => {
getSecurityStatusMock.mockResolvedValue({
sessionCapabilities: { assistantEnabled: true },
});
await aiChatStore.refreshEnabledFromServer();
expect(aiChatStore.enabled).toBe(true);
getSecurityStatusMock.mockResolvedValue({
sessionCapabilities: { assistantEnabled: false },
});
await aiChatStore.refreshEnabledFromServer();
expect(aiChatStore.enabled).toBe(false);
});
it('keeps the current enabled state when the capability refresh fails', async () => {
aiChatStore.setEnabled(true);
getSecurityStatusMock.mockRejectedValue(new Error('offline'));
await aiChatStore.refreshEnabledFromServer();
expect(aiChatStore.enabled).toBe(true);
});
it('uses native Assistant terminology for browser-owned chat shell state', () => {
expect(aiChatStoreSource).not.toContain('Pulse AI');
expect(useChatSource).not.toContain('Pulse AI');
+14
View File
@@ -283,6 +283,20 @@ export const aiChatStore = {
setAiEnabled(enabled);
},
// Re-derive assistant availability from the canonical session
// capability. The bootstrap only reads it on page load, so AI
// settings saves call this to reveal (or hide) the assistant entry
// points without a full reload.
async refreshEnabledFromServer() {
try {
const { SecurityAPI } = await import('@/api/security');
const status = await SecurityAPI.getStatus();
setAiEnabled(status?.sessionCapabilities?.assistantEnabled === true);
} catch (error) {
logger.error('[aiChat] Failed to refresh assistant availability', error);
}
},
// Set messages (for persistence from AIChat component)
setMessages(msgs: Message[]) {
setMessages(msgs);
+17 -1
View File
@@ -95,12 +95,28 @@ func PatrolToolReadinessForModel(provider, model string) (string, PatrolFailureC
case providerDefinitionIsGateway(provider):
return PatrolReadinessWarning, PatrolFailureCauseModelToolSupportUnverified, fmt.Sprintf("%s routes vary by model and endpoint. Patrol will fail closed if the routed model rejects tools or tool_choice.", config.AIProviderDisplayName(provider))
case provider == config.AIProviderOllama:
return PatrolReadinessWarning, PatrolFailureCauseModelToolSupportUnverified, fmt.Sprintf("Ollama connectivity alone does not prove tool support. %s passes Patrol's tool check; run ollama pull %s and select it as the Patrol model.", config.OllamaSuggestedPatrolModel, config.OllamaSuggestedPatrolModel)
if patrolOllamaModelIsSuggested(normalizedModel) {
return PatrolReadinessReady, PatrolFailureCauseNone, fmt.Sprintf("%s passes Patrol's tool check on Ollama.", config.OllamaSuggestedPatrolModel)
}
return PatrolReadinessWarning, PatrolFailureCauseModelToolSupportUnverified, fmt.Sprintf("Ollama connectivity alone does not prove tool support, and %s has not passed Patrol's tool check. %s is the verified Patrol model: run ollama pull %s and select it as the Patrol model.", patrolOllamaModelName(normalizedModel), config.OllamaSuggestedPatrolModel, config.OllamaSuggestedPatrolModel)
default:
return PatrolReadinessReady, PatrolFailureCauseNone, "The selected provider path supports Patrol's tool-backed analysis contract."
}
}
// patrolOllamaModelName strips the ollama provider prefix so readiness
// copy names the model the way the operator selected it.
func patrolOllamaModelName(normalizedModel string) string {
return strings.TrimPrefix(normalizedModel, string(config.AIProviderOllama)+":")
}
// patrolOllamaModelIsSuggested reports whether the selected Ollama model
// is the blessed Patrol model, which has passed the tool check and must
// not be reported as unverified.
func patrolOllamaModelIsSuggested(normalizedModel string) bool {
return patrolOllamaModelName(normalizedModel) == strings.ToLower(config.OllamaSuggestedPatrolModel)
}
func providerDefinitionIsGateway(provider string) bool {
def, ok := config.LookupAIProviderDefinition(provider)
return ok && def.Gateway
+10
View File
@@ -71,6 +71,16 @@ func TestEvaluatePatrolConfigReadiness_AssignsStableCause(t *testing.T) {
cfg.PatrolModel = "ollama:llama3.2"
},
},
{
name: "ollama suggested patrol model is ready",
wantCause: PatrolFailureCauseNone,
wantReady: true,
configure: func(cfg *config.AIConfig) {
cfg.Enabled = true
cfg.OllamaBaseURL = "http://127.0.0.1:11434"
cfg.PatrolModel = "ollama:" + config.OllamaSuggestedPatrolModel
},
},
{
name: "deepseek v4 flash ready",
wantCause: PatrolFailureCauseNone,
+2 -2
View File
@@ -243,10 +243,10 @@ func (e *PulseToolExecutor) summarizeFleet(
rawIDs, _ := args["resource_ids"].(string)
rawIDs = strings.TrimSpace(rawIDs)
if rawIDs == "" {
return NewErrorResult(fmt.Errorf("resource_ids (comma-separated) is required for action=fleet")), nil
return NewErrorResult(fmt.Errorf("resource_ids (comma-separated) is required for action=fleet. Do not ask the operator for identifiers: enumerate resources yourself first (e.g. pulse_query) and retry with their ids, or use action=resource for a single resource")), nil
}
if canonicalDefault == "" {
return NewErrorResult(fmt.Errorf("resource_type is required for action=fleet")), nil
return NewErrorResult(fmt.Errorf("resource_type is required for action=fleet. Use the type shared by the listed resources (e.g. vm, node, docker-host); enumerate resources yourself if unsure — do not ask the operator")), nil
}
parts := strings.Split(rawIDs, ",")
if len(parts) > summarizeFleetMaxResources {
+2 -2
View File
@@ -1785,7 +1785,7 @@ func TestContract_AISettingsUpdateProviderResolutionJSONSnapshot(t *testing.T) {
"status":"warning",
"ready":true,
"cause":"model_tool_support_unverified",
"summary":"Ollama connectivity alone does not prove tool support. qwen3:8b passes Patrol's tool check; run ollama pull qwen3:8b and select it as the Patrol model.",
"summary":"Ollama connectivity alone does not prove tool support, and llama3:latest has not passed Patrol's tool check. qwen3:8b is the verified Patrol model: run ollama pull qwen3:8b and select it as the Patrol model.",
"provider":"ollama",
"model":"ollama:llama3:latest",
"checks":[{
@@ -1793,7 +1793,7 @@ func TestContract_AISettingsUpdateProviderResolutionJSONSnapshot(t *testing.T) {
"status":"warning",
"cause":"model_tool_support_unverified",
"label":"Patrol control",
"message":"Ollama connectivity alone does not prove tool support. qwen3:8b passes Patrol's tool check; run ollama pull qwen3:8b and select it as the Patrol model."
"message":"Ollama connectivity alone does not prove tool support, and llama3:latest has not passed Patrol's tool check. qwen3:8b is the verified Patrol model: run ollama pull qwen3:8b and select it as the Patrol model."
}]
}
}`, ollama.URL)