mirror of
https://github.com/PerpetualSoftware/pad.git
synced 2026-09-21 10:03:29 +00:00
xarmian
de4d28d576
* feat(attachments): AttachmentStore interface + FSStore (TASK-870) Introduces the storage backend abstraction described in DOC-865 and ships its first concrete implementation. No call sites yet — TASK-871 (upload API) wires it in. internal/attachments/store.go AttachmentStore interface (Put/Get/Stat/Delete) and ErrNotFound sentinel. Put is documented as idempotent — concurrent Puts of the same hash converge — and required to verify that the streamed bytes actually hash to the supplied value. internal/attachments/registry.go Registry routes "<prefix>:<rest>" keys to the store registered for that prefix (Phase 1 = "fs"; Phase 2 will register "s3" alongside). Convenience Get/Stat/Delete helpers resolve + forward in one call so callers don't have to spell out the two-step pattern everywhere. Register panics if the prefix contains ':' since that would make the store unreachable. internal/attachments/fs_store.go FSStore writes to <baseDir>/<aa>/<bb>/<full-hash> with the first 4 hex chars sharding the directory tree two levels deep. Atomic writes: stream + hash to a randomized .tmp in the destination dir, fsync, then intra-directory rename. The streaming sha256 is verified against the supplied hash before the rename, so a mismatch never leaves a visible file. Idempotent fast path: if the canonical file already exists Put short-circuits (and drains the reader so callers don't get a stuck stream). Get returns wrapped ErrNotFound on missing keys; Delete on a missing key is a no-op (matches what the orphan GC needs). Tests cover put/get/stat/delete, hash mismatch, invalid hash format, idempotency, 16-goroutine concurrent Put of the same hash converging to one on-disk file with no orphan tmp files, registry routing, forward-error semantics, and the prefix-with-colon panic. Parent: PLAN-866. * fix(attachments): validate hash on every FSStore key + verify on fast path per Codex review (round 1) Round 1 raised two issues — both real, both fixed. 1. Path traversal in Get/Stat/Delete. extractHash only checked that the key began with "fs:" and the suffix was non-empty before passing it to pathFor(), which used the suffix as a path component. A key like "fs:../../etc/passwd" would escape baseDir for reads/stats/deletes. Fix: extractHash now requires the canonical 64-char lowercase-hex sha256 form via validHash. Same gate that Put already used; now it covers every public method. 2. Idempotent Put fast path skipped hash verification. If the canonical target file already existed, Put returned the key without checking that the supplied reader's bytes hashed to the supplied hash — violating the AttachmentStore.Put contract that implementations MUST verify on every call. A buggy upload path could associate the wrong bytes with an existing hash and silently succeed. Fix: stream r through a hasher when the target exists (no disk I/O), compare against the supplied hash, and reject on mismatch. Also dropped the dead "_short" branch in pathFor — every caller now goes through validHash. Tests added: - TestFSStore_GetStatDeleteRejectBadKeys covers empty/wrong-prefix/empty- hash/non-hex/wrong-length/path-traversal/path-separator/uppercase keys across all three read methods. - TestFSStore_PutFastPathStillVerifiesHash confirms the contract holds on the fast path: a second Put that lies about the hash is rejected with no corruption of the existing file.
60 lines
2.7 KiB
Go
60 lines
2.7 KiB
Go
// Package attachments provides the storage backend abstraction for
|
|
// attachment blobs (images and files uploaded into items). See DOC-865
|
|
// "Attachments — architecture & migration design" for the full design.
|
|
//
|
|
// Storage is content-addressed: every blob is keyed by sha256(content).
|
|
// Identical bytes → identical key → one physical copy. Each AttachmentStore
|
|
// implementation maps a hash to a concrete location (filesystem path, S3
|
|
// object key, …) and namespaces its keys with a backend prefix
|
|
// ("fs:<hash>", "s3:<bucket>/<hash>", …) so a Registry can route a key to
|
|
// the right backend by prefix alone. This decouples item content (which
|
|
// stores opaque "pad-attachment:<uuid>" references) from the backend the
|
|
// blob actually lives in, and enables backend migrations (FS → S3) that
|
|
// touch zero item content.
|
|
package attachments
|
|
|
|
import (
|
|
"context"
|
|
"errors"
|
|
"io"
|
|
)
|
|
|
|
// AttachmentStore is the backend abstraction every storage implementation
|
|
// must satisfy. Methods are safe for concurrent use.
|
|
type AttachmentStore interface {
|
|
// Put writes the blob from r into the backend, addressable by hash.
|
|
// The implementation MUST verify that the bytes streamed from r hash
|
|
// to the supplied hash and return an error if they do not — this is
|
|
// the integrity guarantee callers rely on.
|
|
//
|
|
// Put is idempotent: writing the same hash twice is a no-op on the
|
|
// second call. Concurrent Puts of the same hash converge — both
|
|
// callers receive the same key and the on-disk content is unchanged.
|
|
//
|
|
// The returned key is the "<backend>:<...>" form that Registry uses to
|
|
// route subsequent Get/Stat/Delete calls back to this store.
|
|
//
|
|
// mime is informational (some backends, e.g. S3, store it as object
|
|
// metadata). The FS backend ignores it; MIME for serving comes from
|
|
// the attachments DB row.
|
|
Put(ctx context.Context, hash, mime string, r io.Reader) (key string, err error)
|
|
|
|
// Get returns a ReadCloser positioned at the start of the blob.
|
|
// Callers MUST close the returned reader. If key is unknown to this
|
|
// store, the returned error wraps ErrNotFound.
|
|
Get(ctx context.Context, key string) (io.ReadCloser, error)
|
|
|
|
// Stat returns the size in bytes of the blob identified by key. If
|
|
// key is unknown, the returned error wraps ErrNotFound.
|
|
Stat(ctx context.Context, key string) (size int64, err error)
|
|
|
|
// Delete removes the blob identified by key. Deleting a missing key
|
|
// is NOT an error — callers (e.g. the orphan GC) treat it as the
|
|
// success case.
|
|
Delete(ctx context.Context, key string) error
|
|
}
|
|
|
|
// ErrNotFound is returned (or wrapped) by Get/Stat when the requested key
|
|
// is not present in the backend. Callers can compare with errors.Is.
|
|
var ErrNotFound = errors.New("attachment not found")
|