mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-11 13:28:57 +00:00
8d3e389088
* fix(server): a stream ends when the credential that opened it stops being valid (BUG-3007)
Three long-lived connections kept running after the credential that
opened them was destroyed. Measured on a live instance before any code
was written, not inferred from the handlers:
- GET /api/v1/events delivering 165s after a logout, 64s after a PAT revocation
- GET /api/v1/events/stream open 180s after a logout, 120s after a PAT revocation
- GET /api/v1/collab/{id} open 150s after a logout, 100s after a PAT revocation
In every leg `GET /auth/me` on that same credential answered 401 while
the connection was still live, which is what rules out "the credential
was still valid" — the rival explanation a code reading cannot exclude.
The collab leg also carries a control: the same upgrade with no
Authorization header, and with a bogus bearer, both get 401, so the
socket that outlived its credential was genuinely authorized by it.
All three already revalidate SOMETHING on a jittered ~60s tick, and none
of them asked this question. `/events` revalidates the connecting user's
MEMBERSHIP, which a sign-out does not change. `/events/stream` is
stricter — `refreshUser` re-fetches by USER ID and fails closed on a
deleted or disabled user — and still misses it, because a logout
destroys a SESSION and leaves a live enabled user behind. Collab
re-fetches the ITEM and re-runs access for the principal captured at
UPGRADE time. Every one of those is a question about the PRINCIPAL; none
of them changes when the credential dies.
So there was no in-tree reference to copy. `streamCredentialStillValid`
is new, and the three ticks call it before their existing checks.
Scope is deliberately wider than "the session was destroyed": a revoked
PAT still streaming is the same defect for a CLI or MCP caller that a
destroyed session is for a browser, so the invariant is "the credential
that opened this connection is still valid". Stated here rather than
discovered later as a side effect. The PAT legs were REPRODUCED rather
than inferred from the session legs — a PAT authenticates through
`TokenAuth`, a different path with scopes and an allowed-workspace set,
so either endpoint could plausibly have failed closed on one and not the
other. Neither does.
No new plumbing: all three loops already retain the original
*http.Request for their existing checks, verified at each call site
(handlers_events.go:567, handlers_watch_events.go:420,
handlers_collab.go:344), so the credential is re-read from there rather
than threading a session id through a context that carries none today.
On the tick rather than per delivery: all three already have a tick and
a fail-closed branch to reuse, and a store lookup per delivered event
would sit on the hot path of a fan-out. One tick of bounded staleness is
what these loops already promise for membership changes.
A request carrying NO credential returns true. On the fresh-install
window and the legacy no-auth path there is nothing to invalidate, and
closing there would turn a security fix into an availability regression
on exactly the deployments least able to diagnose it. Pinned by its own
test.
Eleven tests. Six assert a stream STOPS (three connections × session
destroyed / PAT revoked); four are counterfactuals asserting it STAYS
OPEN, because all six would pass against a predicate that returned false
unconditionally — which would close every stream on its first tick and
be a worse defect than the one being fixed. The eleventh pins the
`padsess_` branch in both directions: a CLI session bearer is a SESSION,
not an API token, and validating it as a PAT would answer "no such
token" for a live session and close every CLI stream.
Three preconditions caught three false failures while these were being
written — a 503 with no watch bus, a 503 with no collab room manager,
and a 403 for a PAT-authenticated caller who was not a workspace member.
Each would have made a test pass for the wrong reason the moment the fix
landed. That is the fifth, sixth and seventh instance today of the same
shape: an absence asserted with no assertion that the thing producing it
happened.
Mutants, all confirmed to BUILD first:
- predicate always permits -> 7 tests fail (all six STOPS + the bearer)
- predicate always refuses -> 5 tests fail (all four STAYS OPEN + the bearer)
- `padsess_` branch removed -> the bearer test fails, alone
Prose the change falsified, swept and corrected (CONVE-23):
`collabMembershipRevalInterval`'s doc comment and `handleCollab`'s header
both described the tick as covering revocation only; `refreshUser`'s
comment enumerated its fail-closed set without noting what it cannot
see. CLAUDE.md gains the invariant next to the two SSE endpoints.
Gates: make test exit 0, 0 FAIL; make lint 0 issues; make test-pg exit 0,
0 FAIL (run despite no SQL changing, because this is server-side and the
Postgres leg exercises a different store).
* fix(server): three-valued credential liveness, a closed default, and a read-only token door (BUG-3007, codex round 1)
Round 1 found two, and a lead ruling landed on a third — the gap I had
flagged myself.
1. HIGH — a transient store error was read as a revocation. The
predicate was a bool, so `ValidateSession`/`ValidateToken` returning
an error meant "invalid": one database blip would have closed every
affected stream on the next tick, with SSE emitting `unauthorized`,
watch SSE exiting and collab sending a policy-violation close. A
fleet-wide reconnect storm for users whose credentials were fine.
"The lookup says invalid" and "the lookup could not be performed" are
opposite facts and a bool collapses them, so the predicate is
three-valued now: valid / invalid / unknown. Unknown KEEPS the
connection and logs — one tick of extra staleness, bounded by the
store recovering, which is what these loops already accept for
membership changes.
2. RULED — the default branch fails CLOSED. I had flagged that an
unrecognised credential kind fell through to "still valid", and the
ruling is that unknown-kind-keeps-streaming is the same defect in a
different coat.
Proving completeness needed a fact the request did not carry: HOW the
principal was established. `ctxAuthKind` is set once, by the
middleware branch that accepts the credential, and the predicate
switches on it instead of re-sniffing the wire — so there is exactly
one place on this server deciding whether a bearer is a session or an
API token, rather than two that can drift. A future auth path that
forgets to set it now ends long-lived connections loudly on the first
tick and says why.
The no-credential case is its own explicit branch rather than the
default, so it cannot absorb a future kind by accident: the
fresh-install window and the legacy no-auth path have nothing to
revoke, and closing them would turn a security fix into an
availability regression on the deployments least able to diagnose it.
3. MEDIUM — the tick was a database WRITE. `ValidateToken` bumps
`last_used_at` unconditionally, so at the default 1000-connection
admission limit this fix would have added ~16 sustained writes/second
forever. `ValidateTokenForLiveness` is the read-only door, sharing one
implementation with the ordinary one so the two cannot drift on what
"valid" means.
The cost is the smaller half of the argument. `last_used_at` is what
an operator reads to decide whether a token is still in use before
revoking it, and a background liveness probe is not use — letting it
bump the field would make every idle-but-connected token look active
and quietly defeat the audit the column exists for.
Four new tests, each pinning a branch the eleven connection tests cannot
reach on demand: unknown kind closes, no credential stays open, a store
error keeps the connection, and the liveness door leaves `last_used_at`
alone. That last one asserts the ORDINARY door moves it first — without
that precondition it would pass against a column nothing ever writes.
Mutants, all confirmed to BUILD first: default fails open again -> the
unknown-kind test fails; store error treated as revocation -> the
store-error test fails; liveness door touches `last_used_at` -> its test
fails. Each dies to exactly one test, so the three branches are pinned
independently.
Codex also confirmed clean this round: credential-kind coverage for the
three routes (OAuth and MCP tokens never reach them), the no-credential
path, session UA/IP binding, and the collab close/teardown race.
Gates: make test exit 0, 0 FAIL; make lint 0 issues; make test-pg exit 0,
0 FAIL — the Postgres leg matters more this round, since the store now
has a second validation door.
* fix(server): an accept point that resolves a principal without recording its credential kind now fails closed (BUG-3007, codex round 2)
Round 2 found the one hole the round-1 design left: the empty-kind branch
means "no credential on the wire" and KEEPS the connection, and both MCP
accept points landed there. They resolve a user through WithCurrentUser
without going through TokenAuth, so they recorded who but not how — the
exact silent exemption the fail-closed default branch exists to prevent,
reached by a different door.
Not reachable today: no MCP route is long-lived (dispatch_http_routes.go
maps nothing under /api/v1/events or /api/v1/collab), so this is a guard
rather than an incident.
Three changes:
- The empty-kind branch now distinguishes "nobody authenticated" from "an
accept point authenticated somebody and did not say how". The first keeps
the connection (fresh install, legacy no-auth); the second closes it. One
guard covers every present and future accept point without naming any.
- The MCP PAT path records `api_token`. Its credential IS a PAT in the
Authorization header, so the existing liveness door revalidates it
verbatim — no new machinery.
- The MCP OAuth path deliberately records nothing. An opaque fosite token
needs an introspection-backed door that distinguishes ErrInactiveToken
from a storage error, and writing one for a route that does not exist is
speculative; it fails closed and the comment says why.
WithCurrentUser now documents that it cannot record an auth kind and that a
long-lived route dispatched through it will be closed, not exempted.
Three tests. The principal-without-a-kind guard and the MCP PAT kind each
die to exactly one mutant (drop the guard; drop the WithValue). The third
answers round 2's other point: the existing store-error test closes the
whole store, which is permanent, so it proves the classification and not
the promise the classification is made for — that the tick tries again.
This one breaks the seam transiently (rename the table ValidateSession
reads, ask, put it back, ask again) and ends with a revocation, so a
predicate that returned valid unconditionally after an error would fail it.
It is a fence rather than a discriminating test: the mutant only it would
kill is a latching implementation, which this tree does not have.
Gates green including make test-pg; all three new tests run under Postgres.
* test(server): the transient-outage test restores its schema on failure, and gets a Postgres leg (BUG-3007, codex round 3)
Round 3 found no P0/P1 and confirmed the enumeration this fix rests on:
every production path that reaches the three long-lived routes with a
principal now records an auth kind, and nothing sets a user on top of a
different kind. Two P2s, both about the new test and the latent MCP half.
CORRECTION to the round-2 commit message and the PR body, which claimed the
three new tests were "confirmed to run under Postgres". They ran during
`make test-pg` and they ran on SQLite: `testServer` calls
storetest.NewSQLite unconditionally, so no test built on it is ever
Postgres-backed regardless of PAD_TEST_POSTGRES_URL. The claim was false as
written and I did not check the helper before making it.
So the outage test now has a real Postgres leg, via the existing
testServerPostgres helper, sharing one body with the SQLite one so they
cannot drift. The predicate is dialect-independent but the INSTRUMENT is
not — `ALTER TABLE ... RENAME TO` and the error a missing table produces
are both dialect-specific, and "a store error classifies as unknown" would
be worth nothing on the backend it was never asked on. Verified failing-
into-passing on a real container: `make test-pg TEST_PG_PKGS=./internal/server/`
runs the leg rather than skipping it, and it passes.
The restore is now registered with t.Cleanup the moment the table is
renamed, not left to a line after four assertions that can t.Fatalf past
it. Idempotent, so the in-test restore still happens where the test wants
it. Counterfactual run with an injected mid-outage failure: the only
failure reported is the injected one, so the cleanup restored the schema.
`buildHTTPRequest` gains the note round 3 asked for: it forwards the
principal and not the credential, so a synthesized MCP request carries
`api_token` with no bearer to re-check. Unreachable today (routeTable maps
no long-lived route) and closed either way, but the fix when that changes
is to forward the credential, not to widen the predicate.
Gates: make test exit 0, 0 FAIL; make lint 0 issues; make test-pg exit 0,
0 FAIL.
* docs: the BUG-3007 invariant said a no-credential request is unaffected; a resolved principal with no auth kind is closed
Rounds 2 and 3 changed what the empty-kind branch means, and the CLAUDE.md
paragraph written in round 0 still described the old reading. "A request
carrying NO credential is unaffected" is true only when there is also no
resolved principal; a request that names a user without recording how that
user was established is now CLOSED, because an accept point that records
who without recording what leaves a connection nobody can re-check.
Also says what an operator will otherwise learn from an incident: a store
error is not a revocation and keeps the connection to the next tick.
749 lines
30 KiB
Go
749 lines
30 KiB
Go
package server
|
|
|
|
import (
|
|
"errors"
|
|
"log/slog"
|
|
"math/rand"
|
|
"net/http"
|
|
"strconv"
|
|
"time"
|
|
|
|
"github.com/PerpetualSoftware/pad/internal/collab"
|
|
"github.com/PerpetualSoftware/pad/internal/models"
|
|
"github.com/go-chi/chi/v5"
|
|
"github.com/gorilla/websocket"
|
|
)
|
|
|
|
// collabMembershipRevalInterval is how often an active collab WS
|
|
// re-runs authorizeCollabAccess to catch a mid-stream revocation
|
|
// (member removed, role demoted, item-grant revoked, etc.) AND — since
|
|
// BUG-3007 — re-checks that the credential which opened the connection
|
|
// is still valid at all. The two are different questions: the access
|
|
// check asks what the principal captured at upgrade may do, and a
|
|
// logout or a PAT revocation changes none of it. 60s
|
|
// matches the SSE membership-revalidation cadence and trades
|
|
// "promptness of revocation visibility" against "per-conn DB
|
|
// load". Exposed as a package var so tests can shrink it without
|
|
// waiting a real minute.
|
|
var collabMembershipRevalInterval = 60 * time.Second
|
|
|
|
// collabUpgrader is the gorilla/websocket Upgrader used by handleCollab.
|
|
//
|
|
// CheckOrigin defaults: gorilla returns true when the Origin header is
|
|
// absent OR when Origin's host equals Request.Host. The pad web UI is
|
|
// served by this Go binary, so production traffic is always
|
|
// same-origin and the default policy is exactly what we want — no
|
|
// extra CORS-style allow-list to keep in sync with the SSE handler.
|
|
//
|
|
// Buffer sizes left at 4 KiB (gorilla's default) — Yjs binary updates
|
|
// produced by typical keystroke-rate edits fit comfortably; large
|
|
// initial sync messages (full-document state) get fragmented across
|
|
// reads automatically.
|
|
var collabUpgrader = websocket.Upgrader{}
|
|
|
|
// collabMaxMessageBytes caps the size of a single WebSocket message the
|
|
// server will accept on a collab connection. Without a cap, an
|
|
// authenticated client could send an arbitrarily large frame and force
|
|
// the server to buffer it before ReadMessage returns — the auth
|
|
// middleware's HTTP body limit no longer applies once the connection
|
|
// is upgraded.
|
|
//
|
|
// 1 MiB is generous for everyday Yjs ops (keystroke-rate updates are
|
|
// in the tens-of-bytes range) and still big enough to absorb a full
|
|
// initial-sync state for a typical document. If a future workload
|
|
// needs more headroom (e.g. very large Y.Doc snapshots), bump this
|
|
// alongside any matching CLAUDE.md note.
|
|
const collabMaxMessageBytes = 1 << 20 // 1 MiB
|
|
|
|
// handleCollab is the WebSocket entry point for real-time collab on a
|
|
// single item.
|
|
//
|
|
// GET /api/v1/collab/{itemID}
|
|
//
|
|
// Auth + access checks run BEFORE the protocol upgrade (they need to
|
|
// be able to write a JSON error response). Once upgraded, this
|
|
// handler is intentionally bare — the room manager (TASK-1255) is the
|
|
// piece that wires reads + the OpBus together. For now the handler
|
|
// just spins up the connection, logs it, drains incoming frames, and
|
|
// closes cleanly when the client disconnects. That's enough surface
|
|
// area to validate the auth path end-to-end without coupling to
|
|
// in-flight room-manager work.
|
|
//
|
|
// Authorisation re-creates the workspace-access logic from
|
|
// RequireWorkspaceAccess but keyed on the item's workspace ID rather
|
|
// than a {slug} path param — the WebSocket URL takes only itemID. We
|
|
// also re-check freshness of the user (via store.GetUser) so a
|
|
// mid-session admin demotion or member removal closes the upgrade
|
|
// path immediately, mirroring sseSubscriberStillHasAccess. The
|
|
// periodic per-connection revalidation lives in TASK-1256, and since
|
|
// BUG-3007 it also ends the connection when the credential that opened
|
|
// it is destroyed — which the access checks above cannot notice,
|
|
// because they are about the principal and not about the credential.
|
|
func (s *Server) handleCollab(w http.ResponseWriter, r *http.Request) {
|
|
itemID := chi.URLParam(r, "itemID")
|
|
if itemID == "" {
|
|
writeError(w, http.StatusBadRequest, "bad_request", "itemID is required")
|
|
return
|
|
}
|
|
|
|
item, err := s.store.GetItem(itemID)
|
|
if err != nil {
|
|
writeInternalError(w, err)
|
|
return
|
|
}
|
|
if item == nil {
|
|
// 404 — same surface as any other item-not-found path.
|
|
writeError(w, http.StatusNotFound, "not_found", "Item not found")
|
|
return
|
|
}
|
|
|
|
access, err := s.authorizeCollabAccess(r, item)
|
|
if err != nil {
|
|
var sErr *statusError
|
|
if errors.As(err, &sErr) {
|
|
writeError(w, sErr.code, sErr.kind, sErr.message)
|
|
return
|
|
}
|
|
writeInternalError(w, err)
|
|
return
|
|
}
|
|
|
|
// Upgrade. After this returns successfully w/r are hijacked — we
|
|
// MUST NOT touch them; only conn.WriteMessage / conn.Close.
|
|
if s.collab == nil {
|
|
// RoomManager wiring is optional — a self-host build that
|
|
// doesn't enable collab still exposes the route but should
|
|
// fail loud rather than silently accepting the upgrade and
|
|
// dropping every byte. 503 mirrors the SSE handler's
|
|
// "events bus not configured" path.
|
|
writeError(w, http.StatusServiceUnavailable, "unavailable",
|
|
"Collaboration is not available on this server")
|
|
return
|
|
}
|
|
|
|
// Schema-version handshake (TASK-1268, PLAN-1248). The client
|
|
// announces its SCHEMA_VERSION via `?schema_version=...`; if it
|
|
// doesn't match the server's current value we reject the upgrade
|
|
// outright. Admitting a mismatched client and silently letting it
|
|
// stamp old (or future) ops onto the op-log would corrupt the
|
|
// rebuild flow's ability to detect mismatches — the server's
|
|
// stamp is supposed to mark the era of every persisted row.
|
|
//
|
|
// The empty-query path is treated as legacy-compatible "version
|
|
// 1" rather than a hard error so older bundles served from a
|
|
// browser cache during a deploy don't fail with cryptic 400s
|
|
// before the user has a chance to refresh. The compatibility
|
|
// shim only covers v1 — once we ever bump past it, missing
|
|
// schema_version is rejected.
|
|
clientSchemaVersion := r.URL.Query().Get("schema_version")
|
|
if clientSchemaVersion == "" {
|
|
clientSchemaVersion = "1"
|
|
}
|
|
if clientSchemaVersion != s.collab.SchemaVersion() {
|
|
writeError(w, http.StatusBadRequest, "schema_mismatch",
|
|
"This editor is incompatible with the server. Please refresh the page.")
|
|
return
|
|
}
|
|
|
|
// `?since=<id>` is the resume-cursor announce (TASK-1319). The
|
|
// client tells us the highest item_yjs_updates.id its local
|
|
// Y.Doc has applied; if that id is below the current MIN, the
|
|
// expected suffix has been pruned and the room manager sends a
|
|
// `force_refresh` JSON control frame after the upgrade (we need
|
|
// the conn to write the JSON, hence post-upgrade). Empty / blank
|
|
// / unparseable values are tolerated as 0 (treat as fresh
|
|
// client) so older bundles served from a browser cache during a
|
|
// deploy still get a working session.
|
|
var sinceID int64
|
|
if raw := r.URL.Query().Get("since"); raw != "" {
|
|
if v, perr := strconv.ParseInt(raw, 10, 64); perr == nil && v > 0 {
|
|
sinceID = v
|
|
}
|
|
}
|
|
|
|
// `?content_seq=<seq>` is the items.content generation the client's Y.Doc
|
|
// was SEEDED from (BUG-2264). If it predates the item's most recent restore,
|
|
// Join force_refreshes the client before its on-open Y.encodeStateAsUpdate
|
|
// can re-push the stale pre-restore document. Empty / blank / unparseable
|
|
// values are tolerated as 0 (older bundles that don't announce it fall
|
|
// through to the legacy resume-cursor behaviour with no regression).
|
|
var contentSeq int64
|
|
if raw := r.URL.Query().Get("content_seq"); raw != "" {
|
|
if v, perr := strconv.ParseInt(raw, 10, 64); perr == nil && v > 0 {
|
|
contentSeq = v
|
|
}
|
|
}
|
|
|
|
// `?applier_bracket=1` announces that this client sends the applier_apply_start
|
|
// bracket (BUG-2276 residual 2). Only bracket-capable conns can have their applier
|
|
// outcome durably confirmed against a concurrent version restore; the manager
|
|
// prefers them when electing and fails a legacy round-trip SAFE (retryable) rather
|
|
// than risk a clobber. Absent/other value = legacy (false).
|
|
bracketCapable := r.URL.Query().Get("applier_bracket") == "1"
|
|
|
|
conn, err := collabUpgrader.Upgrade(w, r, nil)
|
|
if err != nil {
|
|
// Upgrade itself emits the right HTTP status (e.g. 400 on
|
|
// missing Sec-WebSocket-Key). Just log and bail.
|
|
slog.Warn("collab: websocket upgrade failed",
|
|
"item_id", itemID,
|
|
"error", err,
|
|
)
|
|
return
|
|
}
|
|
defer conn.Close()
|
|
|
|
// Cap incoming message size before any read to bound server-side
|
|
// memory pressure from a misbehaving / malicious peer. ReadMessage
|
|
// returns an error when this is exceeded, which our loop handles
|
|
// like any other read error (close the connection cleanly).
|
|
conn.SetReadLimit(collabMaxMessageBytes)
|
|
|
|
// Identify the connecting principal in logs. currentUser is nil
|
|
// for legacy workspace-scoped API tokens, fresh-install setups,
|
|
// and similar non-user callers — leave the field empty in that
|
|
// case so log readers can tell the connection came in via a
|
|
// non-user path.
|
|
var userID string
|
|
if u := currentUser(r); u != nil {
|
|
userID = u.ID
|
|
}
|
|
|
|
slog.Info("collab: websocket connected",
|
|
"item_id", itemID,
|
|
"workspace_id", item.WorkspaceID,
|
|
"user_id", userID,
|
|
"remote_addr", r.RemoteAddr,
|
|
)
|
|
defer slog.Info("collab: websocket disconnected",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
)
|
|
|
|
// Periodic auth revalidation: catch member-removed /
|
|
// role-demoted / grant-revoked mid-stream and force-close the
|
|
// WS. Mirrors handlers_events.go's sseSubscriberStillHasAccess
|
|
// pattern but routed through the room manager so the close
|
|
// frame goes out under writeMu (no concurrent-write panics
|
|
// against the room's writeLoop / replay path).
|
|
//
|
|
// The loop is gated on `registered` — closed by Join once the conn
|
|
// is in the room's conn map — so the first tick's SetConnWritable
|
|
// can't race Join's setup and no-op against an unregistered conn,
|
|
// which would strand a startup-window demotion (a viewer able to
|
|
// write) until a later tick. If Join bails before registering
|
|
// (schema/force-refresh/closed), revalDone unblocks the wait so the
|
|
// goroutine exits without leaking. Per TASK-265.
|
|
revalDone := make(chan struct{})
|
|
defer close(revalDone)
|
|
registered := make(chan struct{})
|
|
go func() {
|
|
select {
|
|
case <-registered:
|
|
s.collabRevalidationLoop(r, item, conn, itemID, userID, revalDone)
|
|
case <-revalDone:
|
|
}
|
|
}()
|
|
|
|
// Hand the connection to the RoomManager. It owns the
|
|
// op-log replay, fan-out, and lifecycle bookkeeping (lazy create
|
|
// + grace-TTL reclaim). Returns when the WS closes for any reason.
|
|
// access.canWrite gates whether the room persists+rebroadcasts
|
|
// this peer's inbound sync frames — a read-only participant
|
|
// (viewer / view-only guest) still receives broadcasts but its
|
|
// own frames are dropped (TASK-265). The onRegistered callback
|
|
// (closes `registered`) fires once the conn is in the room, so
|
|
// revalidation only starts after SetConnWritable can find it.
|
|
onRegistered := func() { close(registered) }
|
|
if err := s.collab.Join(itemID, conn, sinceID, contentSeq, access.canWrite, bracketCapable, onRegistered); err != nil {
|
|
// ErrForceRefreshSent is the protocol's normal close-after-
|
|
// notify path — the JSON frame is already on the wire and
|
|
// the client knows what to do. Don't warn.
|
|
if errors.Is(err, collab.ErrForceRefreshSent) {
|
|
return
|
|
}
|
|
// ErrStaleSeedFenceUnavailable is an expected, retryable close (the
|
|
// durable stale-seed boundary read failed, so we fail closed without
|
|
// admitting the peer; Join already logged it at warn). The deferred
|
|
// conn.Close closes the WS so the client reconnects with backoff,
|
|
// Y.Doc intact. Don't double-warn. Per BUG-2264 (Codex xhigh).
|
|
if errors.Is(err, collab.ErrStaleSeedFenceUnavailable) {
|
|
return
|
|
}
|
|
// Normal closure paths surface here as websocket.CloseError
|
|
// values that aren't worth logging. Anything unexpected
|
|
// (transport failure, room manager hard error) gets a warn.
|
|
if websocket.IsUnexpectedCloseError(err,
|
|
websocket.CloseNormalClosure,
|
|
websocket.CloseGoingAway,
|
|
websocket.CloseNoStatusReceived,
|
|
) {
|
|
slog.Warn("collab: websocket session ended unexpectedly",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
"error", err,
|
|
)
|
|
}
|
|
}
|
|
}
|
|
|
|
// collabRevalidationLoop ticks every collabMembershipRevalInterval
|
|
// while the WebSocket is open and re-runs authorizeCollabAccess. On
|
|
// access loss it sends a close frame with ClosePolicyViolation +
|
|
// "Your access to this item was revoked." and closes the conn,
|
|
// which propagates through the room manager's read loop and tears
|
|
// the session down cleanly.
|
|
//
|
|
// First fire is jittered across [0, interval) so a fleet of clients
|
|
// that all reconnected after a deploy don't synchronise their reval
|
|
// ticks and storm the auth path together.
|
|
//
|
|
// Stops when stop is closed (handler returning), so a finished
|
|
// session doesn't leak the goroutine + a long-lived ticker.
|
|
func (s *Server) collabRevalidationLoop(
|
|
r *http.Request,
|
|
item *models.Item,
|
|
conn *websocket.Conn,
|
|
itemID string,
|
|
userID string,
|
|
stop <-chan struct{},
|
|
) {
|
|
interval := collabMembershipRevalInterval
|
|
|
|
// First-fire jitter: rand.Int63n is fine for spread purposes —
|
|
// the security argument doesn't depend on unpredictability.
|
|
first := time.Duration(rand.Int63n(int64(interval)))
|
|
timer := time.NewTimer(first)
|
|
defer timer.Stop()
|
|
|
|
for {
|
|
select {
|
|
case <-stop:
|
|
return
|
|
case <-timer.C:
|
|
// The CREDENTIAL first (BUG-3007). Everything below asks what
|
|
// the principal captured at UPGRADE time may do — `currentUser(r)`
|
|
// — and a logout or a PAT revocation changes none of it. This is
|
|
// the connection with WRITE access: measured still open 150s past
|
|
// a logout and 100s past a revocation, with `/auth/me` on that
|
|
// credential answering 401 and a control leg confirming the
|
|
// upgrade really is authenticated (no header -> 401, bogus token
|
|
// -> 401).
|
|
//
|
|
// `CloseConn` with a policy violation, matching the
|
|
// item-disappeared branch below rather than inventing a fourth way
|
|
// to end a collab connection.
|
|
if !s.streamCredentialStillValid(r) {
|
|
slog.Info("collab: credential invalidated mid-stream, closing connection",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
)
|
|
s.collab.CloseConn(
|
|
itemID, conn,
|
|
websocket.ClosePolicyViolation,
|
|
"Your session has ended.",
|
|
)
|
|
return
|
|
}
|
|
// Re-fetch the item every tick so a mid-session move
|
|
// (item collection changed to one the user can't see)
|
|
// or hard-delete is honoured as an access change. The
|
|
// snapshot captured at upgrade time isn't enough.
|
|
fresh, ferr := s.store.GetItem(itemID)
|
|
if ferr != nil {
|
|
slog.Warn("collab: revalidation GetItem failed; keeping connection open",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
"error", ferr,
|
|
)
|
|
timer.Reset(interval)
|
|
continue
|
|
}
|
|
if fresh == nil {
|
|
slog.Info("collab: item disappeared mid-stream, closing connection",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
)
|
|
s.collab.CloseConn(
|
|
itemID, conn,
|
|
websocket.ClosePolicyViolation,
|
|
"This item is no longer available.",
|
|
)
|
|
return
|
|
}
|
|
|
|
access, err := s.authorizeCollabAccess(r, fresh)
|
|
switch {
|
|
case err == nil:
|
|
// Still authorised. Push any write-permission change
|
|
// to the live connection so the room's per-frame gate
|
|
// reflects the current role without a reconnect — e.g.
|
|
// an editor demoted to viewer becomes read-only
|
|
// (TASK-265), or a viewer promoted to editor gains
|
|
// write. This complements the CloseConn path below,
|
|
// which only fires when access is lost entirely.
|
|
s.collab.SetConnWritable(itemID, conn, access.canWrite)
|
|
// Re-arm at the regular cadence; the connect-time
|
|
// jitter has already spread the fleet so subsequent
|
|
// fires can be evenly spaced.
|
|
timer.Reset(interval)
|
|
|
|
case isAccessDenial(err):
|
|
// Real revocation — close the conn.
|
|
slog.Info("collab: access revoked mid-stream, closing connection",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
)
|
|
s.collab.CloseConn(
|
|
itemID, conn,
|
|
websocket.ClosePolicyViolation,
|
|
"Your access to this item was revoked.",
|
|
)
|
|
return
|
|
|
|
default:
|
|
// Transient internal error (DB blip on GetUser /
|
|
// grant lookup, etc.). Logging at warn so an
|
|
// operator notices a sustained pattern, but we
|
|
// MUST NOT close the conn — a single failed
|
|
// query shouldn't punt every active editor.
|
|
// Re-arm and try again on the next tick.
|
|
slog.Warn("collab: revalidation error; keeping connection open",
|
|
"item_id", itemID,
|
|
"user_id", userID,
|
|
"error", err,
|
|
)
|
|
timer.Reset(interval)
|
|
}
|
|
}
|
|
}
|
|
}
|
|
|
|
// isAccessDenial reports whether the given error from
|
|
// authorizeCollabAccess represents a real authorization decision
|
|
// (member removed, role demoted, item-grant revoked) versus an
|
|
// internal / transient error (DB blip on a lookup). Only access
|
|
// denials should close the live WebSocket; transient errors must
|
|
// fall through so a single failed query doesn't punt every active
|
|
// editor in the workspace.
|
|
//
|
|
// authorizeCollabAccess returns *statusError for every "we
|
|
// know they don't have access" branch, and a plain error (without
|
|
// the statusError wrap) for store / internal errors. errors.As is
|
|
// the canonical way to discriminate.
|
|
func isAccessDenial(err error) bool {
|
|
var sErr *statusError
|
|
return errors.As(err, &sErr)
|
|
}
|
|
|
|
// The write-first-apply-second router in handlers_items_content_route.go replaced the
|
|
// helper chain that used to live here (applyContentViaCollab / applyContentViaCollabOnce
|
|
// / directWriteFn / applyContentMaxRetries / isDeterministicWriteFailure), removed in
|
|
// PLAN-2975 unit 2.
|
|
//
|
|
// It is worth saying WHY rather than leaving a gap: that chain retried
|
|
// ErrRoomActiveDuringPrune internally and re-called ApplyExternalContent, which could
|
|
// succeed through a freshly joined applier and return nil — after which the handler's
|
|
// row write ran last, which is BUG-2840 half A. The retry budget now sits in
|
|
// settleContentRoute, above the write, where a re-decision cannot leave content in the
|
|
// document ahead of a refusal. isDeterministicWriteFailure's job — classifying the
|
|
// typed, permanent refusals — is writeTypedItemRefusal's now, and it carries that
|
|
// function's closed-set warning with it.
|
|
|
|
// statusError lets authorizeCollabAccess return a typed error that
|
|
// carries the HTTP status + payload pieces handleCollab should write.
|
|
// Keeping it private to this file — a separate utility might emerge
|
|
// once another WS handler needs the same shape.
|
|
type statusError struct {
|
|
code int
|
|
kind string
|
|
message string
|
|
}
|
|
|
|
func (e *statusError) Error() string { return e.message }
|
|
|
|
func newStatusError(code int, kind, message string) *statusError {
|
|
return &statusError{code: code, kind: kind, message: message}
|
|
}
|
|
|
|
// collabAccess is the positive outcome of authorizeCollabAccess: the
|
|
// caller is admitted to the collab room (read / live-view). canWrite
|
|
// reports whether the caller may additionally PERSIST inbound Yjs
|
|
// frames — false for a workspace viewer or view-only guest, who is
|
|
// admitted as a read-only participant (keeps presence + live view,
|
|
// but the room drops its inbound sync frames). Threaded into
|
|
// RoomManager.Join and refreshed by the periodic revalidation. Per
|
|
// TASK-265.
|
|
type collabAccess struct {
|
|
canWrite bool
|
|
}
|
|
|
|
// collabTokenWriteScopeAllowed reports whether the request's auth
|
|
// principal is permitted to WRITE, mirroring REST's per-method token
|
|
// scope gate (TokenAuth → tokenScopeAllows). The collab upgrade is a
|
|
// GET, so a read-scoped bearer token passes that method check; this
|
|
// re-applies the write-capability half so such a token can't persist
|
|
// Yjs mutations over the socket. Cookie / CLI-session and fresh-install
|
|
// principals carry no token scopes; tokenScopeAllows treats an empty
|
|
// scope string as unrestricted, so they are always allowed. Uses
|
|
// http.MethodPost as the representative mutating verb (tokenScopeAllows
|
|
// only distinguishes read verbs from write verbs). Per TASK-265.
|
|
func (s *Server) collabTokenWriteScopeAllowed(r *http.Request) bool {
|
|
return tokenScopeAllows(TokenScopesFromContext(r.Context()), http.MethodPost, r.URL.Path)
|
|
}
|
|
|
|
// authorizeCollabAccess mirrors RequireWorkspaceAccess but keyed on
|
|
// the item's workspace ID (the WS URL path doesn't carry a workspace
|
|
// slug). It checks:
|
|
//
|
|
// - Fresh install (no users) → grant.
|
|
// - Legacy workspace-scoped API token → grant if the token's
|
|
// workspace matches the item's workspace.
|
|
// - OAuth token allow-list (TASK-953) → reject when the workspace
|
|
// isn't on the consented list, even for valid members.
|
|
// - Authenticated user → admin OR member OR has guest grants.
|
|
// - Anything else → 403 / 401 as appropriate.
|
|
//
|
|
// On success it returns a collabAccess describing the admission —
|
|
// including canWrite, which reports whether the caller may PERSIST
|
|
// inbound Yjs frames (mirrors the REST requireEditPermission
|
|
// predicate). A non-editor (workspace viewer / view-only guest) is
|
|
// admitted read-only (canWrite=false): it keeps live view + presence
|
|
// but its sync frames are dropped by the room. On a known denial it
|
|
// returns a zero collabAccess + *statusError; store errors surface as
|
|
// a non-statusError. TASK-265.
|
|
func (s *Server) authorizeCollabAccess(r *http.Request, item *models.Item) (collabAccess, error) {
|
|
wsID := item.WorkspaceID
|
|
|
|
// Workspace lookup is needed for the OAuth-allow-list slug compare
|
|
// AND so a "vanished workspace" condition surfaces as 404 rather
|
|
// than a confusing 403.
|
|
ws, err := s.store.GetWorkspaceByID(wsID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
if ws == nil {
|
|
return collabAccess{}, newStatusError(http.StatusNotFound, "not_found", "Workspace not found")
|
|
}
|
|
|
|
// OAuth token allow-list gate.
|
|
if !tokenAllowedWorkspaceMatches(r.Context(), ws.Slug) {
|
|
s.recordMCPAuthzDenial(r, "workspace_not_in_allowlist")
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "permission_denied",
|
|
"Token is not authorized for this workspace")
|
|
}
|
|
|
|
// Fresh-install escape hatch. No users yet → no auth at all, so
|
|
// the REST surface treats the caller as owner; grant write too —
|
|
// EXCEPT a legacy workspace token still carries a scope even on a
|
|
// zero-user instance, so a read-scoped token stays read-only here
|
|
// too (collabTokenWriteScopeAllowed returns true for the no-token
|
|
// anonymous setup caller, whose scopes are empty = unrestricted).
|
|
if count, _ := s.store.UserCount(); count == 0 {
|
|
return collabAccess{canWrite: s.collabTokenWriteScopeAllowed(r)}, nil
|
|
}
|
|
|
|
// Legacy API token (workspace-scoped, no user context). The REST
|
|
// middleware maps a matching workspace-scoped token to the editor
|
|
// role, so it may write — but only if the token's SCOPE permits
|
|
// writes (a read-scoped token is admitted read-only, mirroring REST).
|
|
if tokenWsID := tokenWorkspaceID(r); tokenWsID != "" && currentUser(r) == nil {
|
|
if tokenWsID == wsID {
|
|
return collabAccess{canWrite: s.collabTokenWriteScopeAllowed(r)}, nil
|
|
}
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "forbidden",
|
|
"Token not authorized for this workspace")
|
|
}
|
|
|
|
user := currentUser(r)
|
|
if user == nil {
|
|
return collabAccess{}, newStatusError(http.StatusUnauthorized, "unauthorized",
|
|
"Authentication required")
|
|
}
|
|
|
|
// Re-fetch the user fresh so a mid-session role demotion is
|
|
// reflected immediately. Mirrors sseSubscriberStillHasAccess.
|
|
fresh, err := s.store.GetUser(user.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
if fresh == nil {
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "forbidden", "User not found")
|
|
}
|
|
if fresh.IsDisabled() {
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "forbidden", "User is disabled")
|
|
}
|
|
// PLAN-1933 DR-4: the collab upgrade authorizes then persists
|
|
// incoming Yjs frames to item_yjs_updates (room.go) — a content
|
|
// mutation reached over a GET, so the /api/v1 method gate can't
|
|
// catch it. Reject the upgrade for an unverified cloud user before
|
|
// any admin/membership bypass below (the write-lock applies to
|
|
// everyone, including an unverified admin). No-op on self-host and
|
|
// for verified users via emailUnverifiedBlocked. Uses the freshly
|
|
// re-fetched user so an admin force-verify mid-session is honoured.
|
|
if s.emailUnverifiedBlocked(fresh) {
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "email_not_verified",
|
|
"Verify your email address to edit content.")
|
|
}
|
|
// Admin platform-role bypass — cookie session auth only (BUG-1616).
|
|
// Bearer-borne admin (CLI / PAT / MCP) falls through to the
|
|
// membership-only check below. Mirrors RequireWorkspaceAccess,
|
|
// which maps a cookie-session admin to the owner role → write.
|
|
isBearer := isBearerAuth(r)
|
|
if fresh.Role == "admin" && !isBearer {
|
|
return collabAccess{canWrite: true}, nil
|
|
}
|
|
|
|
// Workspace-level gate: any access at all? Membership OR guest grants.
|
|
// Without this, a logged-in user with no relationship to this
|
|
// workspace would silently fall into the item-visibility check below
|
|
// and 404, which would leak whether the item exists. Reject with
|
|
// 403 first so non-members see the same shape they always have.
|
|
member, err := s.store.GetWorkspaceMember(wsID, fresh.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
hasWorkspaceLevelAccess := member != nil
|
|
if !hasWorkspaceLevelAccess {
|
|
// Bearer-admin (BUG-1616): membership-only stance. Skip the
|
|
// guest-grants fallback exactly like RequireWorkspaceAccess.
|
|
if fresh.Role == "admin" && isBearer {
|
|
s.recordMCPAuthzDenial(r, "not_a_member")
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "forbidden",
|
|
"You are not a member of this workspace")
|
|
}
|
|
hasGrants, err := s.store.UserHasGrantsInWorkspace(wsID, fresh.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
hasWorkspaceLevelAccess = hasGrants
|
|
}
|
|
if !hasWorkspaceLevelAccess {
|
|
s.recordMCPAuthzDenial(r, "not_a_member")
|
|
return collabAccess{}, newStatusError(http.StatusForbidden, "forbidden",
|
|
"You are not a member of this workspace")
|
|
}
|
|
|
|
// Compute the write decision ONCE, up front, mirroring the REST
|
|
// edit path (requireEditPermission) EXACTLY. requireEditPermission
|
|
// grants an editor/owner MEMBER by role FIRST — short-circuiting
|
|
// before any grant lookup — and only falls back to
|
|
// ResolveUserPermission for viewers/guests, so that grants can
|
|
// OVERRIDE an insufficient base role. We must preserve that order:
|
|
// ResolveUserPermission resolves item/collection grants BEFORE
|
|
// membership role, so computing canWrite purely from it would let
|
|
// an incidental `view` grant on this item/collection wrongly demote
|
|
// a legitimate editor/owner to read-only. Members are never role
|
|
// "guest" (guests are non-members with grants), so the role check
|
|
// here is the analogue of requireEditPermission's
|
|
// `role != "guest" && requireRole(r, "editor")`. The visibility
|
|
// checks below decide READ admission to the room; canWrite decides
|
|
// whether the admitted conn may persist inbound frames. Per TASK-265.
|
|
var canWrite bool
|
|
if member != nil && roleLevel(member.Role) >= roleLevel("editor") {
|
|
canWrite = true
|
|
} else {
|
|
perm, err := s.store.ResolveUserPermission(wsID, fresh.ID, item.ID, item.CollectionID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
canWrite = permissionLevel(perm) >= permissionLevel("edit")
|
|
}
|
|
// Token write-scope gate (mirror REST): the collab upgrade is a GET,
|
|
// so a read-scoped bearer token (PAT / OAuth) sails past the
|
|
// method-keyed tokenScopeAllows check in TokenAuth — but it must not
|
|
// be able to PERSIST Yjs mutations over the socket. Downgrade to
|
|
// read-only when the caller's token scope doesn't permit writes.
|
|
// Non-token principals (cookie / CLI session) have empty scopes,
|
|
// which map to "unrestricted" → no downgrade. Per TASK-265.
|
|
if canWrite {
|
|
canWrite = s.collabTokenWriteScopeAllowed(r)
|
|
}
|
|
|
|
// Item-level visibility check. Mirrors requireItemVisible +
|
|
// guestResourceFilter without depending on middleware-set request
|
|
// context — the WS path doesn't go through RequireWorkspaceAccess.
|
|
//
|
|
// Two-stage check:
|
|
// 1. Coarse: the item's collection must be in the user's
|
|
// visible set. VisibleCollectionIDs returns nil for "all"
|
|
// access; that's the easy grant. A non-nil slice may include
|
|
// collections "anchored" by an item-level grant (so the
|
|
// collection appears in the nav even though the user only
|
|
// has access to a single item in it) — that's NOT enough to
|
|
// grant collab access to other items in the same collection.
|
|
// 2. Strict (only when the user has item-level grants): require
|
|
// one of (a) full collection grant, (b) member's "specific"
|
|
// access list including this collection, (c) item grant on
|
|
// THIS exact item. Otherwise the visible-IDs hit was
|
|
// anchored by a sibling's grant and we must 404.
|
|
//
|
|
// Without the strict stage, a guest with `item:A` grant could
|
|
// upgrade /api/v1/collab/{B} for a sibling B in the same
|
|
// collection — the bug Codex found in round 3.
|
|
visibleIDs, err := s.store.VisibleCollectionIDs(wsID, fresh.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
if visibleIDs == nil {
|
|
return collabAccess{canWrite: canWrite}, nil // "all" access
|
|
}
|
|
collectionIsVisible := false
|
|
for _, id := range visibleIDs {
|
|
if id == item.CollectionID {
|
|
collectionIsVisible = true
|
|
break
|
|
}
|
|
}
|
|
if !collectionIsVisible {
|
|
return collabAccess{}, newStatusError(http.StatusNotFound, "not_found", "Item not found")
|
|
}
|
|
|
|
// Visible-set hit. If the user has NO item-level grants, the
|
|
// visibility came from full collection access (member's "specific"
|
|
// access list, or a full collection grant) — grant access to any
|
|
// item in the collection.
|
|
collGrants, itemGrants, err := s.store.ListUserGrants(wsID, fresh.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
if len(itemGrants) == 0 {
|
|
return collabAccess{canWrite: canWrite}, nil
|
|
}
|
|
|
|
// User has item grants. The visible-set hit may have been anchored
|
|
// by a sibling item's grant, so we need a strict check.
|
|
|
|
// (a) Full collection grant on this collection.
|
|
for _, g := range collGrants {
|
|
if g.CollectionID == item.CollectionID {
|
|
return collabAccess{canWrite: canWrite}, nil
|
|
}
|
|
}
|
|
|
|
// (b) Member's "specific" access list including this collection.
|
|
// guestResourceFilter only consults this branch for non-guests.
|
|
if member != nil {
|
|
memberColls, err := s.store.GetMemberCollectionAccess(wsID, fresh.ID)
|
|
if err != nil {
|
|
return collabAccess{}, err
|
|
}
|
|
for _, id := range memberColls {
|
|
if id == item.CollectionID {
|
|
return collabAccess{canWrite: canWrite}, nil
|
|
}
|
|
}
|
|
}
|
|
|
|
// (c) Item-level grant on this exact item.
|
|
for _, g := range itemGrants {
|
|
if g.ItemID == item.ID {
|
|
return collabAccess{canWrite: canWrite}, nil
|
|
}
|
|
}
|
|
|
|
// Visibility was anchored by a sibling grant — 404, mirroring
|
|
// requireItemVisible's "don't leak existence" pattern.
|
|
return collabAccess{}, newStatusError(http.StatusNotFound, "not_found", "Item not found")
|
|
}
|