From 062eef41b2abd664fbfaac8ca5b9be8287d25609 Mon Sep 17 00:00:00 2001 From: xarmian Date: Wed, 22 Apr 2026 20:59:15 -0400 Subject: [PATCH] docs: architecture guide + full .env.example + gitattributes + Makefile note (TASK-687) (#222) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Grouped nice-to-haves called out in the pre-launch audit. 1. docs/architecture.md — new contributor-focused architecture doc. CLAUDE.md covers the same ground but is agent-oriented; this is the human companion. Covers backend layout, request flow, frontend / data model / CLI↔daemon model / agent integration / testing. 2. .env.example — extended to document every PAD_* variable in docs/deployment.md (core, database, real-time events, security, email). Existing Postgres/Redis + encryption secrets kept at the top; new variables grouped by concern with inline comments and safe defaults commented out. 3. .gitattributes — normalize LF line endings repo-wide, mark binary assets, and flag web/build + web/.svelte-kit as generated so they don't pollute GitHub linguist stats or PR diffs. 4. Makefile — CAUTION comment on `make install` noting that the `killall -9 pad` step is system-wide; anyone else's pad daemon on the same machine gets killed too. Designed for single-developer local setups; not for shared hosts. Parent: PLAN-644. --- .env.example | 63 +++++++++++++++ .gitattributes | 49 ++++++++++++ Makefile | 6 +- docs/architecture.md | 177 +++++++++++++++++++++++++++++++++++++++++++ 4 files changed, 294 insertions(+), 1 deletion(-) create mode 100644 .gitattributes create mode 100644 docs/architecture.md diff --git a/.env.example b/.env.example index 43f762c7..608b08a9 100644 --- a/.env.example +++ b/.env.example @@ -57,3 +57,66 @@ PAD_ENCRYPTION_KEY= # mobility (mobile roaming, VPN toggles, carrier NAT); enable only for # high-sensitivity deployments. # PAD_IP_CHANGE_ENFORCE=strict + +# ───────────────────────────────────────────────────────────────────── +# Core server — see docs/deployment.md for full reference. +# ───────────────────────────────────────────────────────────────────── + +# Listen address. 0.0.0.0 for containers, 127.0.0.1 for loopback-only. +# PAD_HOST=0.0.0.0 + +# Listen port. +# PAD_PORT=7777 + +# Public-facing base URL. Used to build invitation + password-reset links. +# PAD_URL=https://pad.example.com + +# Writable data directory — SQLite DB, config, logs. Default: ~/.pad +# PAD_DATA_DIR=/data + +# Log level: debug | info | warn | error +# PAD_LOG_LEVEL=info + +# Operating mode: local | remote | docker | cloud. Controls how the CLI +# discovers the daemon and which credential keying rules apply. +# PAD_MODE=docker + +# ───────────────────────────────────────────────────────────────────── +# Database — SQLite default; set PAD_DB_DRIVER=postgres for Postgres. +# ───────────────────────────────────────────────────────────────────── + +# PAD_DB_DRIVER=postgres +# PAD_DB_PATH=/data/pad.db # SQLite path, ignored on Postgres +# PAD_DATABASE_URL=postgres://pad:secret@postgres:5432/pad?sslmode=disable + +# ───────────────────────────────────────────────────────────────────── +# Real-time events — cross-instance SSE fan-out via Redis pub/sub. +# Without this, SSE stays in-process (fine for single-node). +# ───────────────────────────────────────────────────────────────────── + +# PAD_REDIS_URL=redis://redis:6379 + +# Global cap on concurrent SSE connections (default: 1000). +# PAD_SSE_MAX_CONNECTIONS=1000 + +# Per-workspace cap on concurrent SSE connections (default: 100). +# PAD_SSE_MAX_PER_WORKSPACE=100 + +# ───────────────────────────────────────────────────────────────────── +# Security hardening for production behind TLS. +# ───────────────────────────────────────────────────────────────────── + +# Set Secure flag on session cookies — requires TLS. +# PAD_SECURE_COOKIES=true + +# Comma-separated list of allowed CORS origins. Leave unset to disable CORS. +# PAD_CORS_ORIGINS=https://pad.example.com,https://app.example.com + +# ───────────────────────────────────────────────────────────────────── +# Transactional email via Maileroo. Optional — without it, workspace +# invites fall back to CLI-copyable join codes. +# ───────────────────────────────────────────────────────────────────── + +# PAD_MAILEROO_API_KEY=your-sending-key +# PAD_EMAIL_FROM=noreply@example.com +# PAD_EMAIL_FROM_NAME=Pad diff --git a/.gitattributes b/.gitattributes new file mode 100644 index 00000000..af98f423 --- /dev/null +++ b/.gitattributes @@ -0,0 +1,49 @@ +# Normalize line endings across platforms. +# Without this, Windows contributors can commit CRLF into otherwise-LF files +# and break diffs / hooks / linters. See `man gitattributes`. + +# Default: auto-detect text and normalize to LF on commit. +* text=auto eol=lf + +# Explicit text — safer than relying on auto-detect. +*.go text eol=lf diff=golang +*.ts text eol=lf +*.tsx text eol=lf +*.js text eol=lf +*.jsx text eol=lf +*.svelte text eol=lf +*.html text eol=lf +*.css text eol=lf +*.scss text eol=lf +*.md text eol=lf +*.json text eol=lf +*.yml text eol=lf +*.yaml text eol=lf +*.toml text eol=lf +*.sh text eol=lf +*.py text eol=lf +Makefile text eol=lf +Dockerfile* text eol=lf + +# Vendored / minified — don't rewrite, don't diff as text. +*.min.js binary +*.min.css binary +package-lock.json -diff + +# Binary assets — never touch. +*.png binary +*.jpg binary +*.jpeg binary +*.gif binary +*.ico binary +*.woff binary +*.woff2 binary +*.ttf binary +*.eot binary +*.pdf binary +*.zip binary +*.tar.gz binary + +# Keep generated output out of PR diffs / GitHub linguist stats. +web/build/** linguist-generated=true +web/.svelte-kit/** linguist-generated=true diff --git a/Makefile b/Makefile index 5e441993..d63ae0dd 100644 --- a/Makefile +++ b/Makefile @@ -17,7 +17,11 @@ build-go: go build -ldflags "$(LDFLAGS)" -o $(BINARY) $(BUILD_DIR) install: build - @# Stop running server, install binary, clear stale pid + @# Stop running server, install binary, clear stale pid. + @# CAUTION: `killall -9 pad` is SYSTEM-WIDE. If another user (or another + @# project on the same machine) is also running a `pad` process, it will + @# get killed too. Designed for single-developer local setups; don't run + @# `make install` on a shared host. -killall -9 $(BINARY) 2>/dev/null @sleep 1 @mkdir -p $(INSTALL_DIR) diff --git a/docs/architecture.md b/docs/architecture.md new file mode 100644 index 00000000..6f660ec2 --- /dev/null +++ b/docs/architecture.md @@ -0,0 +1,177 @@ +# Architecture + +High-level map of the Pad codebase for contributors. CLAUDE.md covers the +same ground but is written for AI agents working in the repo — this doc is +the human-readable companion. + +## Shape + +Pad ships as one Go binary. The SvelteKit web UI is built into static +assets and embedded into the binary at compile time via `go:embed`, so a +deployed Pad has exactly one moving piece on the filesystem. SQLite is the +default backing store; PostgreSQL + Redis is the alternate mode for +multi-node production use. + +``` +┌────────────┐ ┌────────────┐ ┌───────────────┐ +│ CLI (pad) │ │ Web UI │ │ AI agents │ +│ │ │ (Svelte 5)│ │ via /pad │ +└─────┬──────┘ └──────┬─────┘ └───────┬───────┘ + │ HTTP │ HTTP/SSE │ HTTP + ▼ ▼ ▼ + ┌─────────────────────────────┐ + │ Pad HTTP server │ + │ (internal/server/*.go) │ + └─┬────────┬───────────────┬──┘ + │ │ │ + ▼ ▼ ▼ + ┌──────────┐ ┌─────────┐ ┌────────────┐ + │ SQLite / │ │ EventBus│ │ Webhooks + │ + │ Postgres │ │ (SSE) │ │ Email │ + └──────────┘ └─────────┘ └────────────┘ +``` + +## Backend (Go) + +- **`cmd/pad/main.go`** — Cobra CLI entry point. Every `pad ` is + registered here. The same binary also serves as the daemon + (`pad server start`) and as the CLI client that talks to it. +- **`internal/server/`** — HTTP router (chi), middleware, handlers, SSE + hub. `server.go` is the main router; `handlers_*.go` group endpoints + by resource. +- **`internal/store/`** — SQL abstractions and migrations. Each resource + (workspaces, items, users, webhooks, sessions, …) has a `.go` + file; `migrations/` is the golang-migrate style migration sources. + Backed by SQLite by default, PostgreSQL when + `PAD_DB_DRIVER=postgres`. +- **`internal/models/`** — shared Go structs that flow through the API + response boundary (`Collection`, `Item`, `User`, `View`, etc.). +- **`internal/items/`** — field-schema validation: collection schemas + declare typed fields (select, text, date, number, …) and this package + validates item fields against them. +- **`internal/collections/`** — default collection definitions per + template (`startup`, `scrum`, `hiring`, `interviewing`, `product`) + and workspace bootstrap logic. +- **`internal/cli/`** — HTTP client used by the CLI to talk to the local + daemon, plus formatting helpers for terminal output. +- **`internal/events/`** — in-process EventBus that fans SSE updates to + connected clients. In Redis mode, a pub/sub bridge replaces the + in-memory bus so multiple Pad replicas stay in sync. +- **`internal/webhooks/`** — outbound webhook dispatcher with HMAC + signing, retries, and delivery log. +- **`internal/email/`** — transactional email via Maileroo. Used for + workspace invitations and password resets; nil-safe if unconfigured. +- **`internal/diff/`** — per-item version history storage + diff + rendering. +- **`internal/links/`** — resolver for `[[wiki-link]]` syntax across + items. +- **`internal/config/`** — workspace detection, `.pad.toml` parsing, + environment-variable loading. + +### Request flow + +1. CLI or web UI sends an HTTP request to `/api/v1/…`. +2. Middleware chain in `internal/server/middleware_*.go` handles auth, + rate limiting, metrics, audit logging. +3. The handler in `internal/server/handlers_*.go` parses the request, + calls one or more `internal/store/*` methods, and writes the JSON + response. +4. If the mutation is observable (item change, comment, etc.), the + handler publishes an event to `internal/events` which fans out to + connected SSE clients at `/api/v1/events`. + +## Frontend (SvelteKit + Svelte 5) + +- **`web/src/routes/`** — page routes. File-based: a folder corresponds + to a URL segment. `[username]/[workspace]/...` is the main + workspace-scoped tree; `console/` is the server-admin UI. +- **`web/src/lib/components/`** — reusable UI (BottomSheet, FieldEditor, + ReactionPicker, NestedChildren, etc.). +- **`web/src/lib/stores/`** — Svelte 5 rune-based stores for cross-route + state (current workspace, page title, current user). +- **`web/src/lib/api/client.ts`** — typed HTTP client. Every REST + endpoint the backend exposes has a method here; adding an endpoint + means adding a client method too. +- **`web/src/lib/types/index.ts`** — mirrors `internal/models/` as + TypeScript types. + +The web UI is built with `npm run build` (static adapter) and the +`build/` output is embedded into the Go binary via `//go:embed` in +`internal/server/embed.go`. `npm run dev` runs a Vite dev server on +`:5173` that proxies API requests to the running Pad daemon on `:7777` +— fast iteration without rebuilding the binary. + +## Data model + +``` +Workspaces + └── Collections (typed by a JSON schema) + └── Items (structured fields + optional markdown content) + ├── parent/child links + ├── blocks / blocked-by dependency links + └── comments, reactions, tags +``` + +- **Collections** have a `fields` JSON schema that declares field keys + (e.g. `status`, `priority`, `due_date`) with types and options. +- **Items** have structured `fields` JSON validated against the + collection's schema, plus optional rich Markdown `content`. +- **Parent/child links** power progress tracking and burndown; any item + type can be a parent of any item type. +- **`[[wiki-link]]` syntax** resolves across all items in a workspace + and renders as clickable links in the UI. + +## CLI ↔ daemon model + +There is only one binary, `pad`. Some commands run purely client-side +(`pad item show REF`), but most go through the daemon: + +- `pad server start` — run the daemon foreground (normal dev mode). +- `pad auth configure` — first-run credential setup, auto-starts the + local daemon on first use. +- All other `pad ` commands are CLI → HTTP → daemon → SQLite. + +The CLI discovers the daemon via `~/.pad/credentials.json` (see +`internal/cli/client.go`). In Docker/Kubernetes mode, a different +client targets the network-served Pad instance. + +## Agent integration + +`skills/pad/SKILL.md` ships inside the binary and gets installed into +an AI agent's configuration by `pad agent install`. The skill is +natural-language — it documents the CLI well enough that any +Claude/Cursor/Copilot-style agent can drive Pad via terminal calls. + +## Testing + +- **Go:** `go test ./...` covers the backend; `internal/store/` tests + run against real SQLite by default and against PostgreSQL when + `PAD_TEST_POSTGRES_URL` is set (see `Makefile` targets `test` and + `test-pg`). +- **Web:** `cd web && npm run build` to catch type / build errors; + `npm run check` runs svelte-check. +- **CI:** `.github/workflows/ci.yml` runs the full matrix — Go + (SQLite + PostgreSQL + race), govulncheck, golangci-lint (new-issues + mode), web build, npm audit, svelte-check. + +## Build and install + +See `CLAUDE.md` for the day-to-day commands (`make install` is the one +you'll run most). The short version: + +- `make build` — build web UI + Go binary to `./pad`. +- `make install` — build, kill running daemon, install to + `$HOME/.local/bin/pad`, restart. **Heads-up:** `make install` runs + `killall -9 pad` system-wide, so any other `pad` process on the host + (including other users' daemons) gets killed. +- `make dev-web` — SvelteKit hot-reload dev server. + +## Further reading + +- [`CLAUDE.md`](../CLAUDE.md) — agent-focused development guide + (identical scope, different audience). +- [`docs/deployment.md`](deployment.md) — full environment-variable + reference, production deployment shapes. +- [`docs/backup.md`](backup.md) — backup and restore procedures. +- [`SECURITY.md`](../SECURITY.md) — reporting a vulnerability, threat + model, hardening tips.