mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-23 02:53:31 +00:00
0fd5d0cdfb
* fix(store): swap plainto_tsquery → websearch_to_tsquery for PG FTS (BUG-842)
`TestListItems_FTS_HyphenatedSearchTerm/task-five` has been failing on
every Go (PostgreSQL) CI run because `plainto_tsquery('english',
'task-five')` doesn't match the asciihword lexeme(s) the english parser
produces for an indexed `task-five-distinctive`. The result is that
every PG full-text search for hyphenated terms returns zero rows.
`websearch_to_tsquery` (Postgres 11+) is purpose-built for arbitrary
user input and tokenizes hyphenated terms the same way `to_tsvector`
does for the indexed document, so the query intersects the index
correctly. Swapped in three spots in the postgres dialect — FTSMatch,
FTSSnippet, FTSRank — and updated the caller-side comments that
referenced plainto_tsquery. SQLite path is unchanged: it goes through
items_fts MATCH with sanitizeFTSQuery, never through these methods.
* fix(server): drain background goroutines on Stop() (BUG-842)
`TestAdminBillingStats_SidecarSidecarError_DegradesToLocalOnly` (and
other server tests) have been flaking on the Go (PostgreSQL) CI runner
with `TempDir RemoveAll cleanup: directory not empty`. Root cause:
several request handlers spawned bare `go func() { ... }()` goroutines
that touched the SQLite WAL DB after the test function returned.
testServer's t.Cleanup closed the store but had no way to drain those
goroutines first, so a fire-and-forget WAL write could re-create the
`-wal`/`-shm` files between Close() and t.TempDir's RemoveAll.
Add a Server.bg sync.WaitGroup, a Server.goAsync helper that wraps a
WaitGroup-tracked goroutine, and a Server.Stop() that blocks until
every goAsync closure has finished. Convert the four known
fire-and-forget sites to goAsync:
- middleware_auth.go (TouchUserActivity)
- handlers_auth.go (password reset email)
- handlers_cloud.go (stripe_processed_events pruning)
- handlers_members.go (workspace invitation email)
Wire `srv.Stop()` into both testServer (server_test.go) and
newMetricsTestServer (metrics_auth_test.go) so cleanup order is
Stop → Close → TempDir RemoveAll. Add
TestServer_Stop_DrainsBackgroundGoroutines to pin the contract: a
goAsync goroutine must block Stop until it returns.
* fix(store): correct PG FTS hyphenation via OR-combined plainto_tsquery (BUG-842)
The previous attempt swapped plainto_tsquery → websearch_to_tsquery,
which was wrong: websearch_to_tsquery treats `-` as a NEGATION operator
(Google-style), so `task-five` becomes `task & !five` and the search
returns 0 rows for the same reason as before. This commit reverts the
swap and applies the actual fix.
PG's english parser indexes `task-five-distinctive` as
`{task-five-distinct, task, five, distinct}` — the asciihword AND its
parts. plainto_tsquery applied to the partial query `task-five`
produces `task-fiv & task & five`: the stemmed asciihword for the
PARTIAL query (`task-fiv`) is NOT in the vector, so the AND fails.
Replacing the hyphen with a space makes plainto emit `task & five`,
which DOES match — but doing that unconditionally breaks `BUG-842`-
style queries: PG indexes the `-842` suffix as a negative-number
lexeme, so `plainto_tsquery('BUG-842')` matches via `-842`, while
`plainto_tsquery('BUG 842')` searches for `842` and misses.
The fix ORs the two query variants together so the search vector is
matched against either the raw user query OR its hyphen-as-space form.
Both `task-five` (against `task-five-distinctive`) and `BUG-842`
(against `BUG-842 fix the cleanup race`) hit. Verified locally against
postgres:17-alpine via PAD_TEST_POSTGRES_URL — both 10x stress and
race-detector runs are green.
Surfaces:
- dialect.go: FTSMatch / FTSSnippet / FTSRank now consume TWO
placeholders each in the PG dialect.
- items.go: listItemsFTS PG branch + SearchItems PG branch update
args to pass (raw, sanitized) for every PG `?` placeholder.
- search.go: SearchItems main / count / facets PG branches updated
likewise. New sanitizePGFTSQuery helper alongside sanitizeFTSQuery.
- documents.go: ListDocuments PG branch updated.
Tests:
- TestListItems_FTS_HyphenatedSearchTerm extended with a `BUG-842`
case to pin the OR-combined logic — naive hyphen-stripping would
silently regress this.
- New TestSanitizePGFTSQuery unit test.
* chore: gofmt 11 files with import-order issues (BUG-842 PR cleanup)
The Go (SQLite) CI job has been failing on `main` (and every PR built
against it) because golangci-lint flags 11 files whose third-party
imports are intermixed with internal imports — the import-grouping
rule that gofmt enforces. None of these were introduced by the
BUG-842 PR; they're pre-existing on main. The PR can't go green
without this cleanup, though, so it's bundled here.
Pure mechanical change — `gofmt -w <files>` only re-orders import
groups; no logic changes. Files touched:
cmd/pad/configure.go
cmd/pad/main.go
internal/cli/format.go
internal/server/handlers_admin_invitations.go
internal/server/handlers_admin_users.go
internal/server/handlers_grants.go
internal/server/handlers_share_links.go
internal/server/handlers_stars.go
internal/server/middleware_auth.go
internal/store/store.go
internal/store/store_test.go
After this commit `gofmt -l ./cmd ./internal` returns clean.
107 lines
3.4 KiB
Go
107 lines
3.4 KiB
Go
package server
|
|
|
|
import (
|
|
"net/http"
|
|
"net/http/httptest"
|
|
"path/filepath"
|
|
"testing"
|
|
|
|
"github.com/PerpetualSoftware/pad/internal/metrics"
|
|
"github.com/PerpetualSoftware/pad/internal/store"
|
|
)
|
|
|
|
// newMetricsTestServer builds a Server with metrics enabled and the given
|
|
// static token. Uses a unique SQLite DB per test so tests never share state.
|
|
func newMetricsTestServer(t *testing.T, token string) *Server {
|
|
t.Helper()
|
|
dir := t.TempDir()
|
|
s, err := store.New(filepath.Join(dir, "test.db"))
|
|
if err != nil {
|
|
t.Fatalf("store.New: %v", err)
|
|
}
|
|
srv := New(s)
|
|
srv.SetMetrics(metrics.New())
|
|
srv.SetMetricsToken(token)
|
|
// Drain background goroutines BEFORE closing the store — see
|
|
// testServer in server_test.go for the BUG-842 race details.
|
|
t.Cleanup(func() {
|
|
srv.Stop()
|
|
s.Close()
|
|
})
|
|
return srv
|
|
}
|
|
|
|
func doMetricsRequest(srv *Server, remoteAddr, authHeader string) *httptest.ResponseRecorder {
|
|
req := httptest.NewRequest("GET", "/metrics", nil)
|
|
req.RemoteAddr = remoteAddr
|
|
if authHeader != "" {
|
|
req.Header.Set("Authorization", authHeader)
|
|
}
|
|
rr := httptest.NewRecorder()
|
|
srv.ServeHTTP(rr, req)
|
|
return rr
|
|
}
|
|
|
|
// TestMetricsAuth_TokenUnset_LoopbackOnly: no PAD_METRICS_TOKEN → loopback
|
|
// callers succeed, everyone else gets 403. Safe default for self-hosters
|
|
// running Prometheus on the same box.
|
|
func TestMetricsAuth_TokenUnset_LoopbackOnly(t *testing.T) {
|
|
srv := newMetricsTestServer(t, "")
|
|
|
|
rr := doMetricsRequest(srv, "127.0.0.1:12345", "")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("loopback /metrics: expected 200, got %d: %s", rr.Code, rr.Body.String())
|
|
}
|
|
|
|
rr = doMetricsRequest(srv, "203.0.113.5:12345", "")
|
|
if rr.Code != http.StatusForbidden {
|
|
t.Fatalf("non-loopback /metrics: expected 403 (no token configured), got %d: %s", rr.Code, rr.Body.String())
|
|
}
|
|
}
|
|
|
|
// TestMetricsAuth_TokenSet_RequiresBearer: with PAD_METRICS_TOKEN set,
|
|
// every scrape — even loopback — must present the correct Bearer token.
|
|
func TestMetricsAuth_TokenSet_RequiresBearer(t *testing.T) {
|
|
srv := newMetricsTestServer(t, "super-secret-token")
|
|
|
|
// Missing header → 401
|
|
rr := doMetricsRequest(srv, "127.0.0.1:1", "")
|
|
if rr.Code != http.StatusUnauthorized {
|
|
t.Fatalf("no auth header: expected 401, got %d", rr.Code)
|
|
}
|
|
|
|
// Wrong token → 401
|
|
rr = doMetricsRequest(srv, "127.0.0.1:1", "Bearer wrong-token")
|
|
if rr.Code != http.StatusUnauthorized {
|
|
t.Fatalf("wrong token: expected 401, got %d", rr.Code)
|
|
}
|
|
|
|
// Non-Bearer scheme → 401
|
|
rr = doMetricsRequest(srv, "127.0.0.1:1", "Basic dXNlcjpwYXNz")
|
|
if rr.Code != http.StatusUnauthorized {
|
|
t.Fatalf("basic auth: expected 401, got %d", rr.Code)
|
|
}
|
|
|
|
// Correct token — from anywhere — succeeds.
|
|
rr = doMetricsRequest(srv, "203.0.113.5:1", "Bearer super-secret-token")
|
|
if rr.Code != http.StatusOK {
|
|
t.Fatalf("valid token: expected 200, got %d: %s", rr.Code, rr.Body.String())
|
|
}
|
|
|
|
// WWW-Authenticate header is set for failed attempts.
|
|
rr = doMetricsRequest(srv, "127.0.0.1:1", "Bearer wrong")
|
|
if got := rr.Header().Get("WWW-Authenticate"); got == "" {
|
|
t.Error("expected WWW-Authenticate header on 401")
|
|
}
|
|
}
|
|
|
|
// TestMetricsAuth_NoMetricsMeans404 sanity-checks that /metrics is
|
|
// simply not routed when metrics are disabled (SetMetrics never called).
|
|
func TestMetricsAuth_NoMetricsMeans404(t *testing.T) {
|
|
srv := testServer(t) // no SetMetrics
|
|
rr := doMetricsRequest(srv, "127.0.0.1:1", "")
|
|
if rr.Code != http.StatusNotFound {
|
|
t.Fatalf("metrics disabled: expected 404, got %d", rr.Code)
|
|
}
|
|
}
|