Files
pad/internal/mcp/errors_test.go
T
xarmian 42f6ce96e1 fix(mcp): normalize error envelope shape + extend code taxonomy + actionable hints (TASK-1077/1078/1079) (#388)
Three independent improvements bundled as one PR because they all touch
the same dispatcher error-emission surface; landing them piecemeal
would churn the same lines repeatedly.

## TASK-1077 — uniform envelope shape

Pre-fix some dispatchers emitted plain-string errors via
`mcp.NewToolResultErrorf("%s: %s failed: %s", ...)`. Same underlying
404 surfaced in three different shapes across the surface (item
lookup → structured envelope; note/decide → "item note: prefetch:
404 ..."; bulk-update per-row → bare error string). Inconsistent
shape made it hard for agents to reason about errors uniformly.

Three new helpers in errors.go:

  - validationFailedResult(cmdKey, msg, fixHint) — replaces the
    "X is required" / "invalid Y" chain across every dispatcher.
  - dispatcherErrorResult(cmdKey, op, err) — replaces the internal
    "build request: %s" / "encode body: %s" / "parse current: %s"
    chain. Always emits ErrServerError with a programmer-readable
    Hint.
  - upstreamHTTPErrorResult(...) — wraps every in-handler prefetch /
    sub-call HTTP failure through classifyHTTPStatusKind so the shape
    matches the main pipeline's responses exactly.

Every NewToolResultErrorf call site in internal/mcp/dispatch_http*.go
+ catalog.go retrofitted. bulk-update's per-row `Error string` field
flipped to `Error *ErrorPayload` so every row failure carries the
same {code, message, hint} shape as a top-level failure.

## TASK-1078 — resource-kind-aware error codes

Pre-fix every 4xx 404 collapsed to ErrItemNotFound regardless of
what was being read; pad_workspace list returning 404 (route
missing) reported `code: "item_not_found"` despite the call having
nothing to do with items. Pre-fix every 5xx collapsed to
ErrServerError, indistinguishable from dispatcher internal failures.

Three new codes in errors.go:

  - ErrNotFound — resource-shaped 404s that AREN'T item lookups
    (collection, listing endpoint, link target, attachment).
  - ErrUpstreamError — 5xx with a structured body (transient backend
    failure). Distinct from ErrServerError (catch-all for dispatcher
    internal + un-mapped 4xx).
  - ErrBackendUnreachable — reserved for transport-level failures
    (DNS / connection refused / 5xx with no body); not yet emitted
    by classifyHTTPStatus but available for future transport-aware
    classification.
  - ErrWorkspaceRequired — reserved for the multi-workspace-token
    "ambiguous default" case (TASK-1076's deferred sister error;
    constant available even though dispatcher doesn't emit it yet).

New ResourceKind enum (item/workspace/collection/listing/link/
attachment/unknown) lets callers tell the classifier what they
were reading. classifyHTTPStatusKind is the new entry point;
classifyHTTPStatus preserved as a legacy adapter for callers that
haven't been retrofitted (pass ResourceUnknown → falls back to
pre-TASK-1078 behaviour).

Every retrofit call site passes its known kind + ref/slug, so 404s
now route through the right code with a contextual message
("Item TASK-7 not found.", "Workspace foo not visible.",
"Collection tasks not found.", etc.).

## TASK-1079 — actionable hints

Pre-fix `hint` was usually `"404 page not found"` (chi's default
NotFound body verbatim) or the upstream JSON envelope re-stringified.
Either way: zero diagnostic value, sometimes outright misleading
(double-stringified JSON in a hint field is hostile).

Per-code hint generators in errors.go:

  - itemMissingHint — names the ref + route + suggests pad_item
    search / list as recovery.
  - workspaceMissingHint — names the slug + route + composes with
    the existing available_workspaces enrichment.
  - notFoundHintFor — kind-aware: collection 404 → "use pad_collection
    list to enumerate"; listing 404 → "verify the route matches the
    server's API surface (build version may be stale)"; etc.
  - authHintFor / permissionHintFor — point at re-auth / scope check.
  - upstreamHintFor — flags 5xx as "usually transient — retry once or
    check pad logs."

extractUpstreamMessage parses pad's own structured `{error:{message}}`
envelope when the upstream backend returned one, so hints lift the
inner human-readable message out instead of dumping the literal JSON.
Falls back to the raw body when the JSON shape doesn't match (no
parse failure noise).

## Tests

  - TestDispatcher_AllErrorsUseStructuredEnvelope walks every
    special-case + link dispatcher's missing-required-input error
    path; pins the shape (code, message, hint all set; hint never
    just "404 page not found"). Adding a new dispatcher that uses
    NewToolResultErrorf will fail this test — it's the regression
    gate the DOD wants.
  - TestClassifyHTTPStatus_KindAware pins each ResourceKind →
    expected ErrorCode mapping for 404s.
  - TestClassifyHTTPStatus_HintsAreActionable pins that hints
    reference the actual route + ref + recovery tools, AND forbids
    the bare "404 page not found" passthrough that triggered Bug 17.
  - TestExtractUpstreamMessage covers the 7 input shapes the helper
    can see (structured envelope, empty inner, missing inner field,
    unparseable, wrong shape, empty, with extra fields).
  - Two existing tests updated to reflect the new shapes:
    TestClassifyHTTPStatus 5xx cases now expect ErrUpstreamError;
    TestMakeFanOutHandler_UnknownAction + TestActionEnv_Dispatch_
    UnknownCmdPath substring searches updated for JSON-encoded
    quotes.

## Behavior diff agents will observe

Same underlying 404, three example error envelopes:

  pad_item show TASK-MISSING:
    code: "item_not_found"
    message: "Item not found."
    hint: "Item \"TASK-MISSING\" not found. Route: /api/v1/.../items/TASK-MISSING. Try `pad_item search` or `pad_item list` to find the right ref."

  pad_workspace list (route 404):
    code: "unknown_workspace"
    message: "Workspace not visible to this session."
    hint: "Route: /api/v1/workspaces. Available workspaces: docapp, pad-web."

  pad_project dashboard (workspace doesn't exist):
    code: "unknown_workspace"
    message: "Workspace \"missing\" is not visible to this session."
    hint: "Workspace \"missing\" not visible. Route: /api/v1/workspaces/missing/dashboard. Available workspaces: docapp."

  Backend 500:
    code: "upstream_error"
    message: "pad item show failed: backend returned 500"
    hint: "Backend returned 500. Usually transient — retry once or check pad logs for the underlying error. Route: ..."
2026-05-02 22:10:12 -04:00

313 lines
12 KiB
Go

package mcp
import (
"context"
"encoding/json"
"errors"
"net/http"
"testing"
"github.com/mark3labs/mcp-go/mcp"
)
// TestNewErrorResult_Envelope confirms the wire shape every error
// path produces. The test inspects:
//
// - IsError flag set on the result.
// - StructuredContent contains the envelope as a Go value.
// - Text fallback parses back to the same envelope.
//
// MCP clients pick whichever surface they understand (Claude Desktop
// uses StructuredContent; older clients fall back to text). The
// invariant: both must agree.
func TestNewErrorResult_Envelope(t *testing.T) {
payload := ErrorPayload{
Code: ErrNoWorkspace,
Message: "No workspace context.",
Hint: "Available workspaces: docapp",
AvailableWorkspaces: []WorkspaceHint{
{Slug: "docapp", Name: "Pad", Default: true},
},
}
res := NewErrorResult(payload)
if !res.IsError {
t.Errorf("IsError = false, want true")
}
if res.StructuredContent == nil {
t.Errorf("StructuredContent missing — clients won't see the typed envelope")
}
// Round-trip via the text fallback.
body := textOf(res)
if body == "" {
t.Fatalf("text fallback missing")
}
var got ErrorEnvelope
if err := json.Unmarshal([]byte(body), &got); err != nil {
t.Fatalf("unmarshal envelope: %v\nbody=%s", err, body)
}
if got.Error.Code != ErrNoWorkspace {
t.Errorf("Code = %q, want %q", got.Error.Code, ErrNoWorkspace)
}
if got.Error.Message != payload.Message {
t.Errorf("Message mismatch: got %q, want %q", got.Error.Message, payload.Message)
}
if len(got.Error.AvailableWorkspaces) != 1 {
t.Fatalf("AvailableWorkspaces length = %d, want 1", len(got.Error.AvailableWorkspaces))
}
if got.Error.AvailableWorkspaces[0].Slug != "docapp" {
t.Errorf("workspace slug = %q, want docapp", got.Error.AvailableWorkspaces[0].Slug)
}
}
// fakeWorkspaceLister is a controllable WorkspaceLister for testing.
// Returns the configured slice unless ErrOut is set, in which case it
// errors (best-effort path test).
type fakeWorkspaceLister struct {
hints []WorkspaceHint
errOut error
}
func (f *fakeWorkspaceLister) ListWorkspaces(_ context.Context) ([]WorkspaceHint, error) {
if f.errOut != nil {
return nil, f.errOut
}
return f.hints, nil
}
// TestClassifyExecError covers the full taxonomy via the ExecDispatcher
// classifier. Each subtest pairs a stderr pattern with the expected
// ErrorCode. Hits all 6 patterns the classifier recognizes plus the
// server_error fallback for unmatched output.
func TestClassifyExecError(t *testing.T) {
lookup := &fakeWorkspaceLister{hints: []WorkspaceHint{
{Slug: "docapp"}, {Slug: "pad-web"},
}}
cases := []struct {
name string
stderr string
wantCode ErrorCode
wantHints bool // true if this code populates available_workspaces
}{
{"no_workspace", "Error: no workspace linked. Run 'pad workspace init'.", ErrNoWorkspace, true},
{"unknown_workspace_does_not_exist", "Error: workspace 'foo' does not exist", ErrUnknownWorkspace, true},
{"unknown_workspace_explicit", "Error: unknown workspace bar", ErrUnknownWorkspace, true},
{"auth_required_login", "Error: not authenticated. please run pad auth login.", ErrAuthRequired, false},
{"auth_required_expired", "Error: expired token", ErrAuthRequired, false},
{"permission_denied", "Error: permission denied for this resource", ErrPermissionDenied, false},
{"item_not_found", "Error: item TASK-99 not found", ErrItemNotFound, false},
{"validation_failed_required", "Error: missing required field 'title'", ErrValidationFailed, false},
{"validation_failed_enum", "Error: status must be one of: open, in-progress, done", ErrValidationFailed, false},
{"server_error_unknown", "Error: something went wrong in unexpected ways", ErrServerError, false},
{"server_error_empty", "", ErrServerError, false},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
runErr := errors.New("exit status 1")
res := classifyExecError(context.Background(),
[]string{"item", "list"}, runErr, tc.stderr, lookup)
if !res.IsError {
t.Errorf("expected IsError, got success")
}
env := decodeEnvelope(t, res)
if env.Error.Code != tc.wantCode {
t.Errorf("Code = %q, want %q", env.Error.Code, tc.wantCode)
}
if tc.wantHints {
if len(env.Error.AvailableWorkspaces) == 0 {
t.Errorf("expected available_workspaces populated for %s", tc.wantCode)
}
} else {
if len(env.Error.AvailableWorkspaces) > 0 {
t.Errorf("did not expect available_workspaces for %s; got %v",
tc.wantCode, env.Error.AvailableWorkspaces)
}
}
})
}
}
// TestClassifyExecError_LookupFailureStillReturnsEnvelope ensures
// best-effort listing — when ListWorkspaces fails, we still surface
// the no_workspace error (just with empty available_workspaces).
// Without this guarantee, a transient lookup failure would degrade
// every workspace-context error into a confusing "couldn't even
// produce an error envelope" path.
func TestClassifyExecError_LookupFailureStillReturnsEnvelope(t *testing.T) {
lookup := &fakeWorkspaceLister{errOut: errors.New("listing exploded")}
res := classifyExecError(context.Background(),
[]string{"item", "list"},
errors.New("exit"),
"Error: no workspace linked",
lookup,
)
env := decodeEnvelope(t, res)
if env.Error.Code != ErrNoWorkspace {
t.Errorf("Code = %q, want no_workspace", env.Error.Code)
}
if len(env.Error.AvailableWorkspaces) != 0 {
t.Errorf("expected empty AvailableWorkspaces on lookup failure; got %v",
env.Error.AvailableWorkspaces)
}
}
// TestClassifyHTTPStatus covers the HTTP status → ErrorCode mapping.
// Each documented status maps to the expected code; unmapped statuses
// fall through to server_error.
func TestClassifyHTTPStatus(t *testing.T) {
cases := []struct {
name string
status int
body string
wantCode ErrorCode
}{
{"401_unauthorized", http.StatusUnauthorized, "token invalid", ErrAuthRequired},
{"403_forbidden", http.StatusForbidden, "role insufficient", ErrPermissionDenied},
{"404_item", http.StatusNotFound, "item TASK-99 not found", ErrItemNotFound},
{"404_workspace", http.StatusNotFound, "workspace foo not visible", ErrUnknownWorkspace},
{"409_conflict", http.StatusConflict, "version mismatch", ErrConflict},
{"422_validation", http.StatusUnprocessableEntity, "title required", ErrValidationFailed},
{"400_validation", http.StatusBadRequest, "bad input", ErrValidationFailed},
// TASK-1078: 5xx now maps to upstream_error (distinct from
// server_error, which is reserved for dispatcher internal
// failures + un-mapped 4xx). Pre-fix every 5xx collapsed to
// ErrServerError; the new code lets agents distinguish
// "backend hiccup, retry" from "dispatcher bug, escalate."
{"500_upstream", http.StatusInternalServerError, "boom", ErrUpstreamError},
{"503_upstream", http.StatusServiceUnavailable, "down", ErrUpstreamError},
{"418_other", http.StatusTeapot, "weird", ErrServerError},
}
for _, tc := range cases {
t.Run(tc.name, func(t *testing.T) {
res := classifyHTTPStatus(context.Background(),
"pad item list", tc.status, []byte(tc.body), nil)
if !res.IsError {
t.Errorf("expected IsError")
}
env := decodeEnvelope(t, res)
if env.Error.Code != tc.wantCode {
t.Errorf("Code = %q, want %q", env.Error.Code, tc.wantCode)
}
// Body fragment preservation contract changed in TASK-1077:
// pre-fix the raw body was always copied into Hint, which
// would leak unstructured upstream output (Codex review #387
// round 1). Post-fix only structured-envelope-shaped bodies
// have their inner message lifted; unstructured bodies are
// dropped to avoid leaking tokens / debug dumps.
//
// Each test case above passes a bare string body — those
// hit the safe-fallback path and don't appear in the
// envelope. The unknown_workspace branch can also legitimately
// emit an empty Hint (no upstream message AND no extractable
// slug AND no lister wired). This test asserts the code
// mapping; body preservation specifically is tested in
// TestExtractUpstreamMessage with structured input, and hint
// shape per code is tested in
// TestClassifyHTTPStatus_HintsAreActionable.
_ = env.Error.Hint
})
}
}
// TestParseWorkspaceListJSON locks the JSON shape ExecDispatcher's
// ListWorkspaces consumes. Emits the same fields `pad workspace list
// --format json` would so the parser stays aligned with the CLI as
// the latter evolves.
func TestParseWorkspaceListJSON(t *testing.T) {
t.Run("happy path", func(t *testing.T) {
body := `[{"slug":"docapp","name":"Pad","default":true},{"slug":"pad-web"}]`
got, err := parseWorkspaceListJSON(body)
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(got) != 2 {
t.Fatalf("len = %d, want 2", len(got))
}
if got[0].Slug != "docapp" || !got[0].Default || got[0].Name != "Pad" {
t.Errorf("first hint mismatch: %+v", got[0])
}
if got[1].Slug != "pad-web" {
t.Errorf("second hint slug = %q, want pad-web", got[1].Slug)
}
})
t.Run("empty body", func(t *testing.T) {
got, err := parseWorkspaceListJSON("")
if err != nil || got != nil {
t.Errorf("empty body should return (nil, nil); got (%v, %v)", got, err)
}
})
t.Run("null body", func(t *testing.T) {
got, err := parseWorkspaceListJSON("null")
if err != nil || got != nil {
t.Errorf("null body should return (nil, nil); got (%v, %v)", got, err)
}
})
t.Run("entry without slug skipped", func(t *testing.T) {
body := `[{"name":"orphan"},{"slug":"valid"}]`
got, err := parseWorkspaceListJSON(body)
if err != nil {
t.Fatalf("parse: %v", err)
}
if len(got) != 1 || got[0].Slug != "valid" {
t.Errorf("expected [valid], got %+v", got)
}
})
t.Run("malformed JSON", func(t *testing.T) {
_, err := parseWorkspaceListJSON("not json {")
if err == nil {
t.Errorf("expected error on malformed JSON")
}
})
}
// TestExtractUnknownWorkspaceSlug covers the small regex helper used
// to pull a slug name out of CLI stderr like "workspace 'foo' does
// not exist". Only QUOTED slugs match — Codex review on PR #357
// caught that bare-word matching captured stop-words like "not" out
// of generic "Workspace not found" responses, which would push agents
// toward retrying with a bogus slug. The regex now requires single or
// double quotes around the slug; bare-word phrasings yield empty
// (the caller's message handles that gracefully).
func TestExtractUnknownWorkspaceSlug(t *testing.T) {
cases := map[string]string{
"Error: workspace 'foo' does not exist": "foo",
"Error: workspace \"bar\" not found": "bar",
"workspace 'docapp' is gone": "docapp",
// Bare-word phrasings — slug NOT extractable; empty result.
"unknown workspace baz": "",
"workspace docapp not visible": "",
"Workspace not found": "",
// Unrelated input.
"completely unrelated message": "",
"": "",
}
for stderr, want := range cases {
t.Run(stderr, func(t *testing.T) {
if got := extractUnknownWorkspaceSlug(stderr); got != want {
t.Errorf("got %q, want %q", got, want)
}
})
}
}
// decodeEnvelope helper extracts the ErrorEnvelope from an MCP error
// result. Uses textOf (the shared helper in catalog_test.go) to
// retrieve the JSON body from the result's content blocks.
func decodeEnvelope(t *testing.T, res *mcp.CallToolResult) ErrorEnvelope {
t.Helper()
body := textOf(res)
if body == "" {
t.Fatalf("error result has empty text body")
}
var env ErrorEnvelope
if err := json.Unmarshal([]byte(body), &env); err != nil {
t.Fatalf("decode envelope: %v\nbody=%s", err, body)
}
return env
}