mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-10-26 06:19:49 +00:00
main
4 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 xarmian
|
fa66181b88 |
fix(webhooks): a DNS failure is retried, and a redirect may not turn the POST into a bodyless GET (BUG-3661, BUG-3662) (#2163)
* fix(webhooks): a DNS failure is retried, and a redirect may not turn the POST into a bodyless GET (BUG-3661, BUG-3662) BUG-3661: the owner path's pre-delivery URL check returned the resolver's error like a policy refusal, and attemptDeliver and checkRedirect made every validation error permanent, so a DNS timeout or SERVFAIL ended the endpoint's part in the event for good. The app path's dial turned resolver errors into ErrRefused, which attemptApp maps to permanent. Now ValidateWebhookURLContext returns a typed ResolveError from a lookup bound to the attempt's timeout (a seam, lookupIP), attemptDeliver and checkRedirect treat it as transient, and appfetch's dial returns resolver errors as a network error (ErrNotSent from Poster.Post: transient, unsent). Policy refusals stay permanent. BUG-3662: net/http follows a 301, 302 or 303 after a POST as a bodyless GET and keeps the headers, so the receiver got a GET with a signature for a body it never saw, and a 2xx was recorded as delivered. checkRedirect now refuses a hop that changes the method or drops the body (permanent: the receiver's configuration). 307 and 308 keep both and are followed. Tests: resolver failure transient at delivery, at a redirect and on the app poster, with policy controls staying permanent / refused; 301/302/303 refused without sending the GET; 307/308 deliver the POST with its body and signature. Each fix's mutant turns its test red. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(webhooks): a dial the address screen refuses is permanent, and one deadline covers the whole attempt (BUG-3661, codex r1) Medium: screenDialAddr refusals (a DNS answer that changed between the URL check and the dial) were plain errors, so the Do error counted as transient and a policy refusal was retried. They now wrap errDialBlocked, which attemptDeliver treats as permanent. Low: the pre-delivery lookup had its own deliveryTimeout before the request got another; both now run under one attempt context. TestBUG3661_DialTimeBlockIsPermanent (validation sees a public answer, the dialer resolves localhost) is red with the classification removed. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
be54575954 |
feat(apps): per-endpoint webhook delivery state, a per-install rate cap, and a 24 h drop (TASK-3408, SPEC-6 U10c) (#1783)
* feat(apps): per-endpoint webhook delivery state, a per-install rate cap, and a 24 h drop (TASK-3408, SPEC-6 U10c) webhook_deliveries records each app hook's decision per outbox event. A terminal row (delivered, permanent, refused, skipped, dropped) is never retried, so a retry goes only to app endpoints still owed, and an event an app was refused once (held, disabled, not visible) stays refused, which closes U10b's deliver_from precision and clock-skew residuals for any event refused once. A pending row (transient, deferred, rate_limited) keeps the event owed. Owner hooks record only "delivered", used only to skip re-sending an event an owner endpoint already received while it stays owed to an app (lead ruling, day 87). Owner failure, retry and ack are unchanged; full owner per-endpoint state is TASK-3409. A per-install cap of 600 a minute (burst 60, per instance) defers the excess without charging an attempt. A delivery still owed 24 h after its event is dropped, counted on the hook and shown to the owner as undelivered_dropped. Migrations 124 / pg 098. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): rate cap counts every attempt and only subscribed targets; limiter pruned; bulk skips recorded (codex r1 on U10c, TASK-3408) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): only owed deliveries are dropped; the rate cap is asked after admission; the ledger counts requests sent (codex r2 on U10c, TASK-3408) Admission's checks become one function run under the install lock for admission and as a plain read for AppDeliveryRefusal, so a 24 h-old delivery admission would refuse is recorded refused, not dropped and counted. The rate gate moves after admission, so a refused attempt spends nothing. DeliverAppEvent returns the requests it sent, which the ledger adds. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): a request that failed before sending is not counted (codex r3 on U10c, TASK-3408) appfetch.Poster wraps dial and TLS-handshake failures in ErrNotSent; the dispatcher counts a request as sent only when the failure is neither that nor a policy refusal. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): the drop decision runs under the install lock; a write failing before its first byte is unsent (codex r4 on U10c, TASK-3408) DropOwedDelivery takes the install row and runs admission's checks in the same transaction that records the outcome, so a disable or upgrade cannot land between the check and the drop. The Poster counts bytes written through its connection; a write that fails at zero bytes is ErrNotSent. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): a late delivery replaces a drop and takes it off the count (codex r5 on U10c, TASK-3408) A request admitted before another drainer dropped its event (the first drainer's outbox claim expired mid-request) did reach the app. RecordDelivery now reads the row under its own transaction: delivered replaces dropped and decrements dropped_count; every other terminal decision still stands. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): RecordDelivery inserts before it locks, so a concurrent drop is seen (codex r6 on U10c, TASK-3408) SELECT ... FOR UPDATE locks nothing when the row does not exist, so a drop committing between the read and the upsert was overwritten with the count left raised. RecordDelivery now inserts with ON CONFLICT DO NOTHING (which waits for a concurrent inserter), and only when the row already existed locks and reads it before deciding. A deterministic test drives a drop between the two statements; run against the previous code on Postgres it ends delivered with dropped_count 1. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(apps): the drop/record race test waits for the drop to reach its write (codex r7 on U10c, TASK-3408) Against the previous RecordDelivery on Postgres it fails 3 of 3 (delivered, dropped_count 1); with the fix it passes on both dialects. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
68e306c76c |
feat(apps): app webhook delivery behind an in-flight fence, signature v2 (TASK-3408, SPEC-6 U10b) (#1780)
* feat(apps): app webhook delivery behind an in-flight fence, signature v2 (TASK-3408, SPEC-6 U10b) The outbox drain delivers each app event a second time, to the workspace's app hooks, with a body built from the event-time app-projection block alone (the DOC-3371 §5 table); an event without one is skipped and counted. The envelope's via_app is set on create events only (lead ruling, day 87); actor attribution is U10d. Every attempt is admitted in one transaction that takes the install row FOR SHARE, checks install active, hook released, event subscribed and its collection still the app's, and records the attempt in app_delivery_inflight with a database-time expiry (12 s; the attempt's 10 s deadline starts before admission). Disable, rotate and uninstall now drain: phase 2 waits until no unexpired record remains. https only, never a redirect, dial-time screening with the admin's webhook-flagged private origins (appfetch.Poster). Signature v2: X-Pad-Signature-256 t=,v1= over "t.body", plus X-Pad-Install. Backoff waits are context-aware timers for owner and app hooks alike; the server cancels the dispatcher's context at Stop. Rotate holds the hook again with a fresh secret. Migrations 123 / pg 097. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * test(apps): U10b mutant kills: skipped events are acked, Stop ends a retry, a stopped dispatcher sends nothing, an unsendable admission releases its record; NUL baseline (TASK-3408) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): app webhook fence holds until the body is written; path prefix by segment; deleted workspaces refused (codex r1, TASK-3408) Poster.Post cancels and waits for the transport to close the request body before returning, so a server answering early cannot let the in-flight record end while body bytes still go out. A private origin's path prefix is matched by whole segments, refusing dot segments and percent-encoded paths. Admission refuses an install whose workspace is soft-deleted. The database-clock-step residual is stated where the expiry is defined. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): seal the app webhook connection before Post returns; keep every private-origin path (codex r2, TASK-3408) Waiting for the request body's Close was not enough: the transport closes the body before flushing its buffered writer. Every Poster connection is now wrapped in a write fence: writes go through a lock, and Post seals the connection (unblocking and then waiting out any write in progress) before it returns, so no byte can follow. Keep-alives are off for app hooks, so a connection carries one request. Duplicate private-origin entries keep every path; an entry without one leaves the origin unrestricted. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): app webhooks write the whole request before reading the answer; bulk members never reach apps (codex r3, TASK-3408) The Poster no longer uses net/http's Transport, which may return a response while still writing the request. It dials through the same screen (the screened IP itself), runs crypto/tls with SNI, hostname verification, the system roots and a TLS 1.2 floor, writes the request with (*http.Request).Write, flushes, then parses the answer with http.ReadResponse under a 64 KiB bound, and closes the connection, all on the caller's goroutine. A 2xx therefore means the whole body was sent, and nothing can write after Post returns. This replaces r2's write fence. Events of a bulk operation (batch_id set) are not delivered to apps, whether claimed alone or folded, and are counted skipped_bulk: which members arrive alone depends on claim timing (DOC-3371 §5: bulk edits are invisible to apps in v1). Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): interim answers skipped; private pins stay paired with their path; app comments carry their install; folded bulk counted (codex r4, TASK-3408) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): an event the app skipped is never delivered later because an owner hook kept it pending (codex r5, TASK-3408) webhooks.deliver_from is the first instant an app hook may receive events: set at redeem, at re-enable, and when an upgrade changes the subscriptions. Admission refuses an event that occurred earlier, so an event skipped while the hook was held, the install disabled, or the type unsubscribed is not delivered when a failing owner hook keeps its outbox row pending. Both stamps are whole seconds; an event in the same second as the boundary is admitted (a stated one-second residual). Per-endpoint state (U10c) replaces the shared retry. Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS * fix(apps): hooks released under migration 122 deliver from their release; clock residuals stated (codex r6, TASK-3408) Claude-Session: https://claude.ai/code/session_01Q9WssCjFuHjSfLABoW4tmS |
||
|
xarmian
|
ca158c22d8 |
feat(apps): app install staging: apps/1 manifest, fetch policy, pending reservations, preview (TASK-3397, U8a) (#1770)
SPEC-6 U8a (DOC-3371 §1, §2 steps 1-5, §9). Strict apps/1 parsing; an https/no-redirect fetch policy screened at dial time with an admin private-origin list; pending-install reservations under the owner lock and the instance advisory lock, holding the full charge until the fetch ends; the owner preview with every artifact normalized by the importer's own code (normalizeArtifact, prepareCreateFields, both extracted behaviour-preserving and pinned by a parity golden); apps_enabled and the admin apps settings; a background sweep of expired pending installs; migration 118/092. The preview is advisory: it may refuse what the import would accept, never the reverse; U8b re-checks atomically. Claude-Session: https://claude.ai/code/session_01H63zaWhkb5pAafx2WGjrVC |