mirror of
https://github.com/Studio-Saelix/sencho.git
synced 2026-08-19 23:06:49 +00:00
feat(files): copy & duplicate, bulk actions, disk-backed uploads, and an accessible file tree (#1409)
* perf(files): spool uploads to disk instead of buffering in memory
Switch the stack file-explorer upload from multer memoryStorage to
diskStorage and stream the spooled temp file through the file-root
gateway, so an upload is never held fully in RAM. Authorization and
root resolution now run before multer spools, so an unauthorized or
read-only-root request is rejected without writing a temp file, and the
spool is removed on every exit path. The named-volume helper write
verifies the written byte count, since cat cannot report a short write.
* feat(files): copy and duplicate files in the explorer
Add a copy capability to the stack file explorer: a same-folder
Duplicate (auto-suffixed name) and a "Copy to..." destination picker,
on both filesystem and named-volume roots. Copying is within-root,
symlink-leaf-safe, blocks a directory copy into its own subtree, and
refuses to create a protected name (compose/.env) at the stack root
while still allowing a protected file to be duplicated under a new name.
* feat(files): make the file tree keyboard accessible
Bring the stack file explorer tree to the WCAG tree pattern: rows are
treeitems carrying aria-level, aria-selected, and aria-expanded, with a
single roving tabindex and full keyboard navigation (arrow keys,
Home/End, Enter/Space) over a flattened visible-node list that stays in
lockstep with the rendered rows. A polite live region announces the
selected file. No visual change to the tree.
* feat(files): bulk select, delete, move, and download files
Add multi-select to the stack file explorer (checkboxes plus Shift and
Ctrl/Cmd click over the visible order) driving three bulk actions:
delete, move, and download as a streamed .tar.gz. All run within the
active root on both filesystem and named-volume backends, report
per-item results so partial failures surface (with the failed items
kept selected for retry), normalize ancestor/descendant selections
server-side, and cap the archive entry and byte counts before any
bytes are streamed. Protected compose/.env files are excluded from
bulk delete and move but may still be downloaded.
* docs(files): document copy, bulk actions, and keyboard navigation
Add the copy/duplicate and multi-select bulk delete/move/download
sections to the Files & Volumes page, a keyboard-navigation note for the
tree, an updated context-menu reference, and bulk troubleshooting entries.
* fix(files): inline path-injection barriers at the new file-op sinks
CodeQL js/path-injection does not credit the wrapped isPathWithinBase
containment check, so the new copy/bulk/disk-upload flows tripped the
gate. Inline the canonical path.resolve + startsWith barrier at the
realpath sink in resolveSafePathWithin (covers every user-relPath flow)
and confirm the multer spool path resolves within UPLOAD_TMP_DIR before
unlinking it or streaming it onward. Behavior is unchanged; the paths
were already validated.
* fix(files): guard the ancestor-walk realpath sink too
The first barrier covered realpath(target), but the ENOENT ancestor
walk re-derives the path via path.dirname, which static analysis treats
as a fresh tainted value. Add the same inline containment barrier before
that realpath and resolve the root case via the untainted base, so the
only tainted realpath input is one the startsWith check has cleared.
Behavior is unchanged.
* fix(files): resolve the root case off the taint path in the ancestor walk
The compound guard on existing (the same variable as the startsWith
subject) was not credited as a sanitizer. Handle the root case before
the barrier by resolving the untainted base directly, leaving a plain
canonical startsWith guard on the strictly-within ancestor. Behavior is
unchanged.
* fix(files): harden helper-backend bulk download and uploads
Address three issues found in the named-volume (helper) backend:
- Bulk download could send 200 headers before discovering a file the
helper download path refuses, tearing the archive mid-stream. The
prewalk now rejects symlinks, non-regular ("other") entries, and
files over the per-file download cap before any header (400/413).
FileEntry gains an 'other' type so non-regular entries stay distinct
from regular files as they pass through the gateway.
- The helper directory listing was fully buffered before the archive
entry cap could fire. listDir now accepts a limit; the list script
stops after limit+1 rows and the gateway reports truncation.
- A stdin pipeline error during a helper upload masked the container's
real nonzero exit code (and its 4xx mapping) as a generic 500. The
nonzero exit now wins; the masked stream error is logged.
* feat(files): add a New file toolbar button with server-enforced create-only
The stack file explorer could create a folder from a toolbar button but a
new file only from a folder's right-click menu, so a file could not be
created at the stack root at all. Add a New file toolbar button beside New
folder, targeting the current directory.
Creating a file now routes through a new createEmptyStackFile helper that
posts a zero-byte file through the existing upload endpoint with overwrite
off, so the server's exclusive-create path rejects an existing name instead
of clobbering it. A file collision surfaces inline in the dialog; a folder
collision and other failures surface as a toast.
* fix(files): widen the tree row hit area and add horizontal scroll for long names
Right-clicking a file tree row only opened the Sencho context menu when the
click landed on the filename; the rest of the row fell through to the native
browser menu, and long names were truncated with no way to read them.
Make each row span the full pane width (and grow with its content) so the
whole row is the context-menu trigger, and let the tree scroll horizontally
so a long name is reachable instead of clipped. A new opt-in horizontal prop
on ScrollArea adds the styled horizontal scrollbar without clamping content
width.
* docs(files): document the New file button, full-row right-click, and long-name scrolling
* test(files): cover createEmptyStackFile targeting the stack root
Add an API-layer case for the empty-directory (stack root) create path, the
primary reason the New file toolbar button exists, so a regression in the
root-level URL would be caught at unit speed rather than only in e2e.
This commit is contained in:
Anso
@@ -0,0 +1,69 @@
|
||||
/**
|
||||
* Unit tests for the bulk selection helpers (normalizeBulkPaths,
|
||||
* destWithinAnySource) used by the bulk delete/move/download routes.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import { normalizeBulkPaths, destWithinAnySource } from '../utils/bulkPaths';
|
||||
|
||||
describe('normalizeBulkPaths', () => {
|
||||
it('dedupes exact duplicates', () => {
|
||||
expect(normalizeBulkPaths(['a.txt', 'a.txt', 'b.txt'], true)).toEqual(['a.txt', 'b.txt']);
|
||||
});
|
||||
|
||||
it('drops a path whose ancestor is also selected', () => {
|
||||
expect(normalizeBulkPaths(['dir', 'dir/child.txt'], true)).toEqual(['dir']);
|
||||
expect(normalizeBulkPaths(['dir', 'dir/sub/deep.txt'], true)).toEqual(['dir']);
|
||||
});
|
||||
|
||||
it('keeps siblings and unrelated paths', () => {
|
||||
expect(normalizeBulkPaths(['a/x', 'a/y', 'b'], true)).toEqual(['a/x', 'a/y', 'b']);
|
||||
});
|
||||
|
||||
it('does not treat a name-prefix sibling as a descendant', () => {
|
||||
// "dir2" is not inside "dir" even though it shares the prefix.
|
||||
expect(normalizeBulkPaths(['dir', 'dir2/file'], true)).toEqual(['dir', 'dir2/file']);
|
||||
});
|
||||
|
||||
it('preserves the first-seen order', () => {
|
||||
expect(normalizeBulkPaths(['z', 'a', 'm'], true)).toEqual(['z', 'a', 'm']);
|
||||
});
|
||||
|
||||
describe('case sensitivity', () => {
|
||||
it('collapses Foo and foo on a case-insensitive root', () => {
|
||||
expect(normalizeBulkPaths(['Foo', 'foo'], false)).toEqual(['Foo']);
|
||||
});
|
||||
|
||||
it('keeps both Foo and foo on a case-sensitive root (Linux / helper volumes)', () => {
|
||||
expect(normalizeBulkPaths(['Foo', 'foo'], true)).toEqual(['Foo', 'foo']);
|
||||
});
|
||||
|
||||
it('drops a case-differing descendant only when case-insensitive', () => {
|
||||
expect(normalizeBulkPaths(['Foo', 'foo/bar'], false)).toEqual(['Foo']);
|
||||
expect(normalizeBulkPaths(['Foo', 'foo/bar'], true)).toEqual(['Foo', 'foo/bar']);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
||||
describe('destWithinAnySource', () => {
|
||||
it('flags a destination equal to a selected source', () => {
|
||||
expect(destWithinAnySource('dir', ['dir'], true)).toBe(true);
|
||||
});
|
||||
|
||||
it('flags a destination inside a selected source', () => {
|
||||
expect(destWithinAnySource('dir/sub', ['dir'], true)).toBe(true);
|
||||
});
|
||||
|
||||
it('allows a destination outside every source', () => {
|
||||
expect(destWithinAnySource('other', ['dir'], true)).toBe(false);
|
||||
expect(destWithinAnySource('', ['dir'], true)).toBe(false);
|
||||
});
|
||||
|
||||
it('is case-aware', () => {
|
||||
expect(destWithinAnySource('DIR/sub', ['dir'], false)).toBe(true);
|
||||
expect(destWithinAnySource('DIR/sub', ['dir'], true)).toBe(false);
|
||||
});
|
||||
|
||||
it('does not flag a name-prefix sibling destination', () => {
|
||||
expect(destWithinAnySource('dir2', ['dir'], true)).toBe(false);
|
||||
});
|
||||
});
|
||||
@@ -0,0 +1,75 @@
|
||||
import { describe, it, expect, vi, afterEach } from 'vitest';
|
||||
import { FileRootGateway } from '../services/FileRootGateway';
|
||||
import { stackSourceFileRoot, type StackFileRoot } from '../services/StackFileRootsService';
|
||||
import { DOWNLOAD_MAX_BYTES, VolumeBrowserService, type VolumeEntry } from '../services/VolumeBrowserService';
|
||||
import type { FileEntry } from '../services/FileSystemService';
|
||||
|
||||
// assertArchivable is a pure guard (no docker/fs), so it is unit-testable on any
|
||||
// host even though the helper backend itself only runs on Linux.
|
||||
const gateway = FileRootGateway.getInstance(1);
|
||||
|
||||
const fsRoot = stackSourceFileRoot();
|
||||
const helperRoot: StackFileRoot = {
|
||||
...stackSourceFileRoot('myvol'),
|
||||
id: 'vol-1',
|
||||
kind: 'volume',
|
||||
backend: 'helper',
|
||||
};
|
||||
|
||||
const entry = (over: Partial<FileEntry>): FileEntry => ({
|
||||
name: 'f', type: 'file', size: 10, mtime: 0, isProtected: false, ...over,
|
||||
});
|
||||
|
||||
describe('FileRootGateway.assertArchivable', () => {
|
||||
it('never rejects on the fs backend (it streams any in-root file and follows symlinks)', () => {
|
||||
expect(() => gateway.assertArchivable(fsRoot, 'a', entry({ type: 'symlink' }))).not.toThrow();
|
||||
expect(() => gateway.assertArchivable(fsRoot, 'big', entry({ size: DOWNLOAD_MAX_BYTES * 4 }))).not.toThrow();
|
||||
});
|
||||
|
||||
it('allows a normal regular file under the cap on the helper backend', () => {
|
||||
expect(() => gateway.assertArchivable(helperRoot, 'a.txt', entry({ size: DOWNLOAD_MAX_BYTES }))).not.toThrow();
|
||||
});
|
||||
|
||||
it('rejects a helper symlink as unsupported (the helper download refuses to follow it)', () => {
|
||||
expect(() => gateway.assertArchivable(helperRoot, 'link', entry({ type: 'symlink' })))
|
||||
.toThrowError(expect.objectContaining({ code: 'ARCHIVE_UNSUPPORTED' }));
|
||||
});
|
||||
|
||||
it('rejects a helper non-regular (other) entry as unsupported', () => {
|
||||
// fifo/socket/device entries the helper download path refuses as "not a
|
||||
// regular file"; they must be caught here too, not just symlinks.
|
||||
expect(() => gateway.assertArchivable(helperRoot, 'dev', entry({ type: 'other' })))
|
||||
.toThrowError(expect.objectContaining({ code: 'ARCHIVE_UNSUPPORTED' }));
|
||||
});
|
||||
|
||||
it('rejects a helper file above the per-file download cap as too large', () => {
|
||||
expect(() => gateway.assertArchivable(helperRoot, 'big.bin', entry({ size: DOWNLOAD_MAX_BYTES + 1 })))
|
||||
.toThrowError(expect.objectContaining({ code: 'ARCHIVE_TOO_LARGE' }));
|
||||
});
|
||||
});
|
||||
|
||||
describe('FileRootGateway.listDir helper truncation', () => {
|
||||
afterEach(() => vi.restoreAllMocks());
|
||||
|
||||
const volEntry = (name: string): VolumeEntry => ({ name, type: 'file', size: 1, mtime: 0, isProtected: false });
|
||||
|
||||
it('reports truncated when the helper returns the overflow row, trimming to the limit', async () => {
|
||||
// The gateway asks the helper for limit+1; receiving limit+1 means there
|
||||
// were more entries than the limit.
|
||||
vi.spyOn(VolumeBrowserService.prototype, 'listDir').mockResolvedValue(
|
||||
Array.from({ length: 4 }, (_, i) => volEntry(`f${i}`)),
|
||||
);
|
||||
const res = await gateway.listDir(helperRoot, 'stack', '', 3);
|
||||
expect(res.truncated).toBe(true);
|
||||
expect(res.entries).toHaveLength(3);
|
||||
});
|
||||
|
||||
it('reports not truncated when the helper returns at most the limit', async () => {
|
||||
vi.spyOn(VolumeBrowserService.prototype, 'listDir').mockResolvedValue(
|
||||
Array.from({ length: 3 }, (_, i) => volEntry(`f${i}`)),
|
||||
);
|
||||
const res = await gateway.listDir(helperRoot, 'stack', '', 3);
|
||||
expect(res.truncated).toBe(false);
|
||||
expect(res.entries).toHaveLength(3);
|
||||
});
|
||||
});
|
||||
@@ -1,7 +1,7 @@
|
||||
/**
|
||||
* Tests for isValidRelativeStackPath (pure function) and the stack-scoped
|
||||
* file methods on FileSystemService (listStackDirectory, readStackFile,
|
||||
* writeStackFile, writeStackFileBuffer, deleteStackPath, mkdirStackPath).
|
||||
* writeStackFile, writeScopedFileFromTemp, deleteStackPath, mkdirStackPath).
|
||||
*
|
||||
* FileSystemService stack methods are tested against a real temp directory so
|
||||
* that realpath, stat, and fs I/O all run with actual OS semantics.
|
||||
@@ -215,28 +215,6 @@ describe('FileSystemService stack methods', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── writeStackFileBuffer ────────────────────────────────────────────────
|
||||
|
||||
describe('writeStackFileBuffer', () => {
|
||||
it('writes raw bytes correctly', async () => {
|
||||
const data = Buffer.from([0x01, 0x02, 0x03, 0xff]);
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.writeStackFileBuffer(STACK, 'binary.bin', data);
|
||||
|
||||
const read = await fs.readFile(path.join(stackDir, 'binary.bin'));
|
||||
expect(read).toEqual(data);
|
||||
});
|
||||
|
||||
it('creates parent directories when needed', async () => {
|
||||
const payload = Buffer.from([0xde, 0xad]);
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.writeStackFileBuffer(STACK, 'sub/img.bin', payload);
|
||||
|
||||
const read = await fs.readFile(path.join(stackDir, 'sub', 'img.bin'));
|
||||
expect(read).toEqual(payload);
|
||||
});
|
||||
});
|
||||
|
||||
// ── atomic write semantics ──────────────────────────────────────────────
|
||||
|
||||
describe('atomic write semantics', () => {
|
||||
@@ -349,6 +327,123 @@ describe('FileSystemService stack methods', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── writeScopedFileFromTemp (disk-backed upload path) ───────────────────
|
||||
|
||||
describe('writeScopedFileFromTemp', () => {
|
||||
let tempSrcDir: string;
|
||||
beforeEach(async () => {
|
||||
// A separate base dir stands in for the OS upload-spool location, which on
|
||||
// a real host may sit on a different filesystem than the stack dir.
|
||||
tempSrcDir = await fs.mkdtemp(path.join(os.tmpdir(), 'sencho-uptmp-'));
|
||||
});
|
||||
afterEach(async () => {
|
||||
await fs.rm(tempSrcDir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('streams a temp file into the stack and leaves the source temp for the caller', async () => {
|
||||
const src = path.join(tempSrcDir, 'spool-bin');
|
||||
const payload = Buffer.from('streamed upload payload');
|
||||
await fs.writeFile(src, payload);
|
||||
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.writeScopedFileFromTemp(STACK, 'sub/uploaded.bin', src);
|
||||
|
||||
const written = await fs.readFile(path.join(stackDir, 'sub', 'uploaded.bin'));
|
||||
expect(written).toEqual(payload);
|
||||
// The method copies (streams) the source; the caller still owns the temp.
|
||||
await expect(fs.access(src)).resolves.toBeUndefined();
|
||||
const leftovers = (await fs.readdir(path.join(stackDir, 'sub'))).filter(n => n.includes('.sencho-tmp-'));
|
||||
expect(leftovers).toEqual([]);
|
||||
});
|
||||
|
||||
it('exclusive write to an existing target throws FILE_EXISTS, preserves the original, and leaks no tmp', async () => {
|
||||
const target = path.join(stackDir, 'keep.txt');
|
||||
await fs.writeFile(target, 'ORIGINAL');
|
||||
const src = path.join(tempSrcDir, 'incoming');
|
||||
await fs.writeFile(src, 'INCOMING');
|
||||
|
||||
const service = FileSystemService.getInstance();
|
||||
await expect(
|
||||
service.writeScopedFileFromTemp(STACK, 'keep.txt', src, { exclusive: true }),
|
||||
).rejects.toMatchObject({ code: 'FILE_EXISTS' });
|
||||
|
||||
expect(await fs.readFile(target, 'utf-8')).toBe('ORIGINAL');
|
||||
const leftovers = (await fs.readdir(stackDir)).filter(n => n.startsWith('keep.txt.sencho-tmp-'));
|
||||
expect(leftovers).toEqual([]);
|
||||
});
|
||||
|
||||
it('cleans up the staging sibling when the rename step throws', async () => {
|
||||
const src = path.join(tempSrcDir, 'incoming2');
|
||||
await fs.writeFile(src, 'NEW');
|
||||
const fsModule = await import('fs');
|
||||
const renameSpy = vi
|
||||
.spyOn(fsModule.promises, 'rename')
|
||||
.mockRejectedValueOnce(Object.assign(new Error('disk yanked'), { code: 'EIO' }));
|
||||
|
||||
const service = FileSystemService.getInstance();
|
||||
await expect(service.writeScopedFileFromTemp(STACK, 'rfail.txt', src)).rejects.toThrow(/disk yanked/);
|
||||
|
||||
const dirEntries = await fs.readdir(stackDir);
|
||||
expect(dirEntries).not.toContain('rfail.txt');
|
||||
expect(dirEntries.filter(n => n.startsWith('rfail.txt.sencho-tmp-'))).toEqual([]);
|
||||
renameSpy.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
// ── copyScopedPath ──────────────────────────────────────────────────────
|
||||
|
||||
describe('copyScopedPath', () => {
|
||||
it('copies a file and preserves the original', async () => {
|
||||
await fs.writeFile(path.join(stackDir, 'src.txt'), 'hello');
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.copyScopedPath(STACK, 'src.txt', 'dst.txt');
|
||||
expect(await fs.readFile(path.join(stackDir, 'dst.txt'), 'utf-8')).toBe('hello');
|
||||
expect(await fs.readFile(path.join(stackDir, 'src.txt'), 'utf-8')).toBe('hello');
|
||||
});
|
||||
|
||||
it('recursively copies a directory tree', async () => {
|
||||
await fs.mkdir(path.join(stackDir, 'dir/inner'), { recursive: true });
|
||||
await fs.writeFile(path.join(stackDir, 'dir/inner/leaf.txt'), 'leaf');
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.copyScopedPath(STACK, 'dir', 'dir-copy');
|
||||
expect(await fs.readFile(path.join(stackDir, 'dir-copy/inner/leaf.txt'), 'utf-8')).toBe('leaf');
|
||||
});
|
||||
|
||||
it('rejects an existing destination with EEXIST', async () => {
|
||||
await fs.writeFile(path.join(stackDir, 'e-src.txt'), 'a');
|
||||
await fs.writeFile(path.join(stackDir, 'e-dst.txt'), 'b');
|
||||
const service = FileSystemService.getInstance();
|
||||
await expect(service.copyScopedPath(STACK, 'e-src.txt', 'e-dst.txt')).rejects.toMatchObject({ code: 'EEXIST' });
|
||||
expect(await fs.readFile(path.join(stackDir, 'e-dst.txt'), 'utf-8')).toBe('b');
|
||||
});
|
||||
|
||||
it('rejects copying a directory into its own descendant with INVALID_PATH', async () => {
|
||||
await fs.mkdir(path.join(stackDir, 'sc/sub'), { recursive: true });
|
||||
const service = FileSystemService.getInstance();
|
||||
await expect(service.copyScopedPath(STACK, 'sc', 'sc/sub/sc')).rejects.toMatchObject({ code: 'INVALID_PATH' });
|
||||
});
|
||||
|
||||
it('blocks copying onto a protected root name but allows a protected source', async () => {
|
||||
await fs.writeFile(path.join(stackDir, 'compose.yaml'), 'services: {}\n');
|
||||
const service = FileSystemService.getInstance();
|
||||
// Source is protected, destination is not -> allowed.
|
||||
await service.copyScopedPath(STACK, 'compose.yaml', 'compose.yaml.bak');
|
||||
expect(await fs.readFile(path.join(stackDir, 'compose.yaml.bak'), 'utf-8')).toBe('services: {}\n');
|
||||
// Destination is a reserved root name -> blocked.
|
||||
await fs.writeFile(path.join(stackDir, 'plain.yaml'), 'x\n');
|
||||
await expect(service.copyScopedPath(STACK, 'plain.yaml', 'docker-compose.yml')).rejects.toMatchObject({ code: 'PROTECTED_FILE' });
|
||||
});
|
||||
|
||||
it.skipIf(isWindows)('copies a symlink as a link, not its target', async () => {
|
||||
await fs.writeFile(path.join(stackDir, 'target.txt'), 'real');
|
||||
await fs.symlink('target.txt', path.join(stackDir, 'link.txt'));
|
||||
const service = FileSystemService.getInstance();
|
||||
await service.copyScopedPath(STACK, 'link.txt', 'link-copy.txt');
|
||||
const lst = await fs.lstat(path.join(stackDir, 'link-copy.txt'));
|
||||
expect(lst.isSymbolicLink()).toBe(true);
|
||||
});
|
||||
});
|
||||
|
||||
// ── deleteStackPath ─────────────────────────────────────────────────────
|
||||
|
||||
describe('deleteStackPath', () => {
|
||||
|
||||
@@ -20,8 +20,41 @@ import request from 'supertest';
|
||||
import bcrypt from 'bcrypt';
|
||||
import { promises as fs } from 'fs';
|
||||
import path from 'path';
|
||||
import zlib from 'zlib';
|
||||
import { Readable } from 'stream';
|
||||
import * as tar from 'tar-stream';
|
||||
import { setupTestDb, cleanupTestDb, loginAsTestAdmin } from './helpers/setupTestDb';
|
||||
import type { StackFileRoot } from '../services/StackFileRootsService';
|
||||
|
||||
/** Gunzip + untar a bulk-download response body into a { entryName: contents } map. */
|
||||
async function extractTarGz(buf: Buffer): Promise<Record<string, string>> {
|
||||
const out: Record<string, string> = {};
|
||||
const extract = tar.extract();
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
extract.on('entry', (header, stream, next) => {
|
||||
const chunks: Buffer[] = [];
|
||||
stream.on('data', (c: Buffer) => chunks.push(c));
|
||||
stream.on('end', () => { out[header.name] = Buffer.concat(chunks).toString('utf-8'); next(); });
|
||||
stream.on('error', reject);
|
||||
});
|
||||
extract.on('finish', () => resolve());
|
||||
extract.on('error', reject);
|
||||
extract.end(zlib.gunzipSync(buf));
|
||||
});
|
||||
return out;
|
||||
}
|
||||
|
||||
/**
|
||||
* supertest parser that buffers a binary response body. supertest types the
|
||||
* parser's first arg as its Response while passing the raw readable stream at
|
||||
* runtime, so the function is cast to supertest's own parser parameter type.
|
||||
*/
|
||||
const binaryParser = ((res: NodeJS.ReadableStream, cb: (err: Error | null, body: Buffer) => void): void => {
|
||||
const chunks: Buffer[] = [];
|
||||
res.on('data', (c: Buffer) => chunks.push(Buffer.from(c)));
|
||||
res.on('end', () => cb(null, Buffer.concat(chunks)));
|
||||
res.on('error', (err: Error) => cb(err, Buffer.alloc(0)));
|
||||
}) as unknown as Parameters<ReturnType<typeof request>['parse']>[0];
|
||||
|
||||
// On Windows, fs.unlink on a directory returns EPERM instead of EISDIR so the
|
||||
// NOT_EMPTY code path in deleteStackPath is never reached. Skip that test case
|
||||
@@ -35,8 +68,19 @@ let DatabaseService: typeof import('../services/DatabaseService').DatabaseServic
|
||||
let adminCookie: string;
|
||||
let viewerCookie: string;
|
||||
let stacksDir: string;
|
||||
let uploadTmpDir: string;
|
||||
const STACK = 'teststack';
|
||||
|
||||
/** Count files left in the upload spool dir (ENOENT = dir never created = 0). */
|
||||
async function uploadTempCount(): Promise<number> {
|
||||
try {
|
||||
return (await fs.readdir(uploadTmpDir)).length;
|
||||
} catch (err) {
|
||||
if ((err as NodeJS.ErrnoException).code === 'ENOENT') return 0;
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
type FileExplorerMetricEntry = {
|
||||
op: string;
|
||||
count: number;
|
||||
@@ -90,6 +134,10 @@ async function expectDownloadMetricCounts(count: number, successCount: number, e
|
||||
beforeAll(async () => {
|
||||
tmpDir = await setupTestDb();
|
||||
stacksDir = process.env.COMPOSE_DIR!;
|
||||
// Isolate the upload spool dir per worker so temp-leak assertions are
|
||||
// deterministic (the default is a shared os.tmpdir() subdir).
|
||||
uploadTmpDir = path.join(tmpDir, 'uploads');
|
||||
process.env.SENCHO_UPLOAD_DIR = uploadTmpDir;
|
||||
|
||||
// Create stack directory so file operations have something to work with
|
||||
await fs.mkdir(path.join(stacksDir, STACK), { recursive: true });
|
||||
@@ -113,6 +161,7 @@ beforeAll(async () => {
|
||||
});
|
||||
|
||||
afterAll(async () => {
|
||||
delete process.env.SENCHO_UPLOAD_DIR;
|
||||
cleanupTestDb(tmpDir);
|
||||
});
|
||||
|
||||
@@ -947,6 +996,100 @@ describe('POST /api/stacks/:stackName/files/upload', () => {
|
||||
expect(stat.isDirectory()).toBe(true);
|
||||
await fs.rm(dir, { recursive: true, force: true });
|
||||
});
|
||||
|
||||
// The upload spools to a disk temp file (diskStorage); that temp must never be
|
||||
// left behind, on success or on any rejection path.
|
||||
it('leaves no spooled temp file after a successful upload', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', Buffer.from('spool-success'), 'spool-success.txt');
|
||||
expect(res.status).toBe(204);
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'spool-success.txt'));
|
||||
});
|
||||
|
||||
it('leaves no spooled temp file after a 409 conflict', async () => {
|
||||
const target = path.join(stacksDir, STACK, 'spool-conflict.txt');
|
||||
await fs.writeFile(target, 'original');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', Buffer.from('replacement'), 'spool-conflict.txt');
|
||||
expect(res.status).toBe(409);
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
await fs.unlink(target);
|
||||
});
|
||||
|
||||
it('leaves no spooled temp file after exceeding the size limit (413)', async () => {
|
||||
const bigFile = Buffer.alloc(26 * 1024 * 1024, 0x61);
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', bigFile, 'spool-toobig.txt');
|
||||
expect(res.status).toBe(413);
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
}, 20000);
|
||||
|
||||
it('rejects a viewer before any file is spooled (pre-multer auth gate)', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.set('Cookie', viewerCookie)
|
||||
.attach('file', Buffer.from('viewer-data'), 'viewer-spool.txt');
|
||||
expect(res.status).toBe(403);
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
// And nothing was written into the stack dir.
|
||||
await expect(fs.access(path.join(stacksDir, STACK, 'viewer-spool.txt'))).rejects.toThrow();
|
||||
});
|
||||
|
||||
it('rejects an upload to a read-only root before spooling (pre-multer root gate)', async () => {
|
||||
const readonlyRoot: StackFileRoot = {
|
||||
id: 'bind:readonlybind', kind: 'bind', label: '/ro', hostPathOrName: path.join(stacksDir, 'ro-bind'),
|
||||
mounts: [{ service: 'app', containerPath: '/ro', readOnly: true }],
|
||||
readonly: true, accessible: true, browsable: true, writable: false,
|
||||
chmodable: false, dangerous: false, managedSourceOverlap: false,
|
||||
warning: 'This location is read-only.', backend: 'fs',
|
||||
};
|
||||
const { StackFileRootsService } = await import('../services/StackFileRootsService');
|
||||
const spy = vi.spyOn(StackFileRootsService.prototype, 'resolveRoot').mockResolvedValue(readonlyRoot);
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.query({ rootId: 'bind:readonlybind' })
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', Buffer.from('blocked'), 'ro.txt');
|
||||
expect(res.status).toBe(403);
|
||||
expect(res.body.code).toBe('READONLY_ROOT');
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
spy.mockRestore();
|
||||
});
|
||||
|
||||
it('cleans up the spooled temp file when the write throws after spooling', async () => {
|
||||
const { FileSystemService } = await import('../services/FileSystemService');
|
||||
const spy = vi.spyOn(FileSystemService.prototype, 'writeScopedFileFromTemp')
|
||||
.mockRejectedValue(Object.assign(new Error('disk gone'), { code: 'EIO' }));
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', Buffer.from('boom'), 'wfail-upload.txt');
|
||||
expect(res.status).toBe(500);
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
await expect(fs.access(path.join(stacksDir, STACK, 'wfail-upload.txt'))).rejects.toThrow();
|
||||
spy.mockRestore();
|
||||
});
|
||||
|
||||
it('leaves no spooled temp file after an overwrite', async () => {
|
||||
const target = path.join(stacksDir, STACK, 'spool-overwrite.txt');
|
||||
await fs.writeFile(target, 'before');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/upload`)
|
||||
.query({ overwrite: '1' })
|
||||
.set('Cookie', adminCookie)
|
||||
.attach('file', Buffer.from('after'), 'spool-overwrite.txt');
|
||||
expect(res.status).toBe(204);
|
||||
expect(await fs.readFile(target, 'utf-8')).toBe('after');
|
||||
expect(await uploadTempCount()).toBe(0);
|
||||
await fs.unlink(target);
|
||||
});
|
||||
});
|
||||
|
||||
// ── PUT /:stackName/files/content ─────────────────────────────────────────────
|
||||
@@ -1184,6 +1327,364 @@ describe('PATCH /api/stacks/:stackName/files/rename', () => {
|
||||
});
|
||||
});
|
||||
|
||||
// ── POST /:stackName/files/copy ──────────────────────────────────────────────
|
||||
|
||||
describe('POST /api/stacks/:stackName/files/copy', () => {
|
||||
it('rejects unauthenticated requests with 401', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.send({ from: 'a.txt', to: 'b.txt' });
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
|
||||
it('viewer receives 403', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', viewerCookie)
|
||||
.send({ from: 'a.txt', to: 'b.txt' });
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
|
||||
it('copies a file, preserving the original', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'copy-src.txt'), 'payload');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'copy-src.txt', to: 'copy-dst.txt' });
|
||||
expect(res.status).toBe(204);
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'copy-dst.txt'), 'utf-8')).toBe('payload');
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'copy-src.txt'), 'utf-8')).toBe('payload');
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'copy-src.txt'));
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'copy-dst.txt'));
|
||||
});
|
||||
|
||||
it('allows duplicating a protected file under a new name', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'compose.yaml', to: 'compose.yaml.bak' });
|
||||
expect(res.status).toBe(204);
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'compose.yaml.bak'));
|
||||
});
|
||||
|
||||
it('blocks copying onto a protected root name with 409 PROTECTED_FILE', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'rogue.yaml'), 'services: {}\n');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'rogue.yaml', to: 'docker-compose.yml' });
|
||||
expect(res.status).toBe(409);
|
||||
expect(res.body.code).toBe('PROTECTED_FILE');
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'rogue.yaml'));
|
||||
});
|
||||
|
||||
it('returns 409 ALREADY_EXISTS when the destination exists', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'cexist-src.txt'), 'a');
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'cexist-dst.txt'), 'b');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'cexist-src.txt', to: 'cexist-dst.txt' });
|
||||
expect(res.status).toBe(409);
|
||||
expect(res.body.code).toBe('ALREADY_EXISTS');
|
||||
// The existing destination is untouched.
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'cexist-dst.txt'), 'utf-8')).toBe('b');
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'cexist-src.txt'));
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'cexist-dst.txt'));
|
||||
});
|
||||
|
||||
it('recursively copies a directory', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'cdir/nested'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'cdir/nested/deep.txt'), 'deep');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'cdir', to: 'cdir-copy' });
|
||||
expect(res.status).toBe(204);
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'cdir-copy/nested/deep.txt'), 'utf-8')).toBe('deep');
|
||||
await fs.rm(path.join(stacksDir, STACK, 'cdir'), { recursive: true, force: true });
|
||||
await fs.rm(path.join(stacksDir, STACK, 'cdir-copy'), { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rejects copying a directory into its own descendant with 400', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'selfcopy/sub'), { recursive: true });
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: 'selfcopy', to: 'selfcopy/sub/selfcopy' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.code).toBe('INVALID_PATH');
|
||||
await fs.rm(path.join(stacksDir, STACK, 'selfcopy'), { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rejects path traversal in from/to with 400', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/copy`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: '../escape.txt', to: 'x.txt' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.code).toBe('INVALID_PATH');
|
||||
});
|
||||
});
|
||||
|
||||
// ── Bulk operations: delete / move / download ────────────────────────────────
|
||||
|
||||
describe('POST /api/stacks/:stackName/files/bulk-delete', () => {
|
||||
it('rejects unauthenticated requests with 401', async () => {
|
||||
const res = await request(app).post(`/api/stacks/${STACK}/files/bulk-delete`).send({ paths: ['a.txt'] });
|
||||
expect(res.status).toBe(401);
|
||||
});
|
||||
|
||||
it('viewer receives 403', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', viewerCookie)
|
||||
.send({ paths: ['a.txt'] });
|
||||
expect(res.status).toBe(403);
|
||||
});
|
||||
|
||||
it('rejects an empty or oversized selection with 400', async () => {
|
||||
const empty = await request(app).post(`/api/stacks/${STACK}/files/bulk-delete`).set('Cookie', adminCookie).send({ paths: [] });
|
||||
expect(empty.status).toBe(400);
|
||||
const tooMany = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: Array.from({ length: 101 }, (_, i) => `f${i}.txt`) });
|
||||
expect(tooMany.status).toBe(400);
|
||||
expect(tooMany.body.code).toBe('TOO_MANY');
|
||||
});
|
||||
|
||||
it('rejects a selection containing an invalid path with 400', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['ok.txt', '../escape'] });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.code).toBe('INVALID_PATH');
|
||||
});
|
||||
|
||||
it('deletes multiple files and reports per-item results', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bd1.txt'), '1');
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bd2.txt'), '2');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['bd1.txt', 'bd2.txt', 'missing.txt'] });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.deleted).toEqual(expect.arrayContaining(['bd1.txt', 'bd2.txt']));
|
||||
expect(res.body.failed).toHaveLength(1);
|
||||
expect(res.body.failed[0].path).toBe('missing.txt');
|
||||
await expect(fs.access(path.join(stacksDir, STACK, 'bd1.txt'))).rejects.toThrow();
|
||||
});
|
||||
|
||||
it('deletes a non-empty folder recursively', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'bdir/sub'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bdir/sub/x.txt'), 'x');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['bdir'] });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.deleted).toEqual(['bdir']);
|
||||
await expect(fs.access(path.join(stacksDir, STACK, 'bdir'))).rejects.toThrow();
|
||||
});
|
||||
|
||||
it('reports a protected file as a per-item failure, not a thrown request', async () => {
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['compose.yaml'] });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.deleted).toEqual([]);
|
||||
expect(res.body.failed[0].path).toBe('compose.yaml');
|
||||
// compose.yaml must survive.
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'compose.yaml'), 'utf-8')).toContain('services');
|
||||
});
|
||||
|
||||
it('normalizes a directly-submitted ancestor+descendant selection (no double-process)', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'ndir'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'ndir/child.txt'), 'c');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['ndir', 'ndir/child.txt'] });
|
||||
expect(res.status).toBe(200);
|
||||
// Only the ancestor is acted on; the descendant is dropped, so no spurious failure.
|
||||
expect(res.body.deleted).toEqual(['ndir']);
|
||||
expect(res.body.failed).toEqual([]);
|
||||
});
|
||||
|
||||
it('returns 403 for a read-only root before deleting', async () => {
|
||||
const readonlyRoot: StackFileRoot = {
|
||||
id: 'bind:robd', kind: 'bind', label: '/ro', hostPathOrName: path.join(stacksDir, 'ro'),
|
||||
mounts: [{ service: 'app', containerPath: '/ro', readOnly: true }],
|
||||
readonly: true, accessible: true, browsable: true, writable: false,
|
||||
chmodable: false, dangerous: false, managedSourceOverlap: false,
|
||||
warning: 'This location is read-only.', backend: 'fs',
|
||||
};
|
||||
const { StackFileRootsService } = await import('../services/StackFileRootsService');
|
||||
const spy = vi.spyOn(StackFileRootsService.prototype, 'resolveRoot').mockResolvedValue(readonlyRoot);
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-delete`)
|
||||
.query({ rootId: 'bind:robd' })
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ paths: ['x.txt'] });
|
||||
expect(res.status).toBe(403);
|
||||
expect(res.body.code).toBe('READONLY_ROOT');
|
||||
spy.mockRestore();
|
||||
});
|
||||
});
|
||||
|
||||
describe('POST /api/stacks/:stackName/files/bulk-move', () => {
|
||||
it('moves multiple entries into a destination folder', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'bmdest'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bm1.txt'), '1');
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bm2.txt'), '2');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-move`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: ['bm1.txt', 'bm2.txt'], toDir: 'bmdest' });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.moved).toEqual(expect.arrayContaining(['bm1.txt', 'bm2.txt']));
|
||||
expect(await fs.readFile(path.join(stacksDir, STACK, 'bmdest/bm1.txt'), 'utf-8')).toBe('1');
|
||||
await fs.rm(path.join(stacksDir, STACK, 'bmdest'), { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('reports a same-name collision at the destination as a per-item failure', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'bmd2'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bmcol.txt'), 'src');
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'bmd2/bmcol.txt'), 'existing');
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-move`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: ['bmcol.txt'], toDir: 'bmd2' });
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.body.moved).toEqual([]);
|
||||
expect(res.body.failed[0].path).toBe('bmcol.txt');
|
||||
await fs.rm(path.join(stacksDir, STACK, 'bmd2'), { recursive: true, force: true });
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'bmcol.txt'));
|
||||
});
|
||||
|
||||
it('rejects the whole request when the destination is inside a selected folder (400)', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'bmself/sub'), { recursive: true });
|
||||
const res = await request(app)
|
||||
.post(`/api/stacks/${STACK}/files/bulk-move`)
|
||||
.set('Cookie', adminCookie)
|
||||
.send({ from: ['bmself'], toDir: 'bmself/sub' });
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.code).toBe('INVALID_PATH');
|
||||
await fs.rm(path.join(stacksDir, STACK, 'bmself'), { recursive: true, force: true });
|
||||
});
|
||||
});
|
||||
|
||||
describe('GET /api/stacks/:stackName/files/bulk-download', () => {
|
||||
it('streams a .tar.gz of the selection (viewer/read access), de-duplicating nested paths', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'dl/sub'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'dl/sub/deep.txt'), 'deep');
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'dl-top.txt'), 'top');
|
||||
// Select the dir, a file inside it (redundant), and a top-level file.
|
||||
const res = await request(app)
|
||||
.get(`/api/stacks/${STACK}/files/bulk-download`)
|
||||
.query({ path: ['dl', 'dl/sub/deep.txt', 'dl-top.txt'] })
|
||||
.set('Cookie', viewerCookie) // a read-only viewer can download
|
||||
.buffer(true)
|
||||
.parse(binaryParser);
|
||||
expect(res.status).toBe(200);
|
||||
expect(res.headers['content-disposition']).toContain('.tar.gz');
|
||||
const entries = await extractTarGz(res.body as Buffer);
|
||||
// Nested selection de-duplicated: deep.txt appears once, under its dir path.
|
||||
expect(entries['dl/sub/deep.txt']).toBe('deep');
|
||||
expect(entries['dl-top.txt']).toBe('top');
|
||||
expect(Object.keys(entries)).toHaveLength(2);
|
||||
// Tar entry names are relative POSIX paths (no leading slash, no '..').
|
||||
for (const name of Object.keys(entries)) {
|
||||
expect(name.startsWith('/')).toBe(false);
|
||||
expect(name.includes('..')).toBe(false);
|
||||
}
|
||||
await fs.rm(path.join(stacksDir, STACK, 'dl'), { recursive: true, force: true });
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'dl-top.txt'));
|
||||
});
|
||||
|
||||
it('lets a viewer download but not bulk-delete or bulk-move', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'v.txt'), 'v');
|
||||
const dl = await request(app)
|
||||
.get(`/api/stacks/${STACK}/files/bulk-download`)
|
||||
.query({ path: ['v.txt'] })
|
||||
.set('Cookie', viewerCookie)
|
||||
.buffer(true)
|
||||
.parse(binaryParser);
|
||||
expect(dl.status).toBe(200);
|
||||
const del = await request(app).post(`/api/stacks/${STACK}/files/bulk-delete`).set('Cookie', viewerCookie).send({ paths: ['v.txt'] });
|
||||
expect(del.status).toBe(403);
|
||||
const mv = await request(app).post(`/api/stacks/${STACK}/files/bulk-move`).set('Cookie', viewerCookie).send({ from: ['v.txt'], toDir: '' });
|
||||
expect(mv.status).toBe(403);
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'v.txt'));
|
||||
});
|
||||
|
||||
// The helper (named-volume) bulk path (rootCaseSensitive helper branch,
|
||||
// FileRootGateway.stat/download helper branch) runs an Alpine container per
|
||||
// file and is validated on Linux / CI, not on this workstation.
|
||||
|
||||
it('returns 413 when the total byte cap is exceeded', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'big.bin'), 'x');
|
||||
const { FileRootGateway } = await import('../services/FileRootGateway');
|
||||
const spy = vi.spyOn(FileRootGateway.prototype, 'stat').mockResolvedValue({
|
||||
name: 'big.bin', type: 'file', size: 2 * 1024 * 1024 * 1024, mtime: 0, isProtected: false,
|
||||
});
|
||||
const res = await request(app)
|
||||
.get(`/api/stacks/${STACK}/files/bulk-download`)
|
||||
.query({ path: ['big.bin'] })
|
||||
.set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(413);
|
||||
expect(res.body.code).toBe('TOO_LARGE');
|
||||
spy.mockRestore();
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'big.bin'));
|
||||
});
|
||||
|
||||
it('returns 400 UNSUPPORTED when an entry is not archivable (e.g. a helper symlink/non-regular file)', async () => {
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'odd.bin'), 'x');
|
||||
// assertArchivable only rejects on helper roots; force the rejection so the
|
||||
// route's mapping (which fires for any backend) is covered without a volume.
|
||||
const { FileRootGateway } = await import('../services/FileRootGateway');
|
||||
const spy = vi.spyOn(FileRootGateway.prototype, 'assertArchivable').mockImplementation(() => {
|
||||
throw Object.assign(new Error('"odd.bin" cannot be downloaded from this volume'), { code: 'ARCHIVE_UNSUPPORTED' });
|
||||
});
|
||||
const res = await request(app)
|
||||
.get(`/api/stacks/${STACK}/files/bulk-download`)
|
||||
.query({ path: ['odd.bin'] })
|
||||
.set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(400);
|
||||
expect(res.body.code).toBe('UNSUPPORTED');
|
||||
spy.mockRestore();
|
||||
await fs.unlink(path.join(stacksDir, STACK, 'odd.bin'));
|
||||
});
|
||||
|
||||
it('returns 413 (before any archive byte) when the entry cap is exceeded', async () => {
|
||||
await fs.mkdir(path.join(stacksDir, STACK, 'huge'), { recursive: true });
|
||||
await fs.writeFile(path.join(stacksDir, STACK, 'huge/a.txt'), 'a');
|
||||
// A truncated directory listing means more entries than the archive cap.
|
||||
const { FileRootGateway } = await import('../services/FileRootGateway');
|
||||
const spy = vi.spyOn(FileRootGateway.prototype, 'listDir').mockResolvedValue({
|
||||
entries: [{ name: 'a.txt', type: 'file', size: 1, mtime: 0, isProtected: false }],
|
||||
total: 999999,
|
||||
truncated: true,
|
||||
});
|
||||
const res = await request(app)
|
||||
.get(`/api/stacks/${STACK}/files/bulk-download`)
|
||||
.query({ path: ['huge'] })
|
||||
.set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(413);
|
||||
expect(res.body.code).toBe('TOO_LARGE');
|
||||
spy.mockRestore();
|
||||
await fs.rm(path.join(stacksDir, STACK, 'huge'), { recursive: true, force: true });
|
||||
});
|
||||
|
||||
it('rejects an empty selection with 400', async () => {
|
||||
const res = await request(app).get(`/api/stacks/${STACK}/files/bulk-download`).set('Cookie', adminCookie);
|
||||
expect(res.status).toBe(400);
|
||||
});
|
||||
});
|
||||
|
||||
// ── PUT /:stackName/files/permissions ────────────────────────────────────────
|
||||
|
||||
describe('PUT /api/stacks/:stackName/files/permissions', () => {
|
||||
|
||||
@@ -1,8 +1,11 @@
|
||||
/**
|
||||
* Coverage for VolumeBrowserService pure helpers: path traversal sanitization,
|
||||
* volume-name validation, and binary detection. The Docker-facing exec path
|
||||
* is exercised in manual E2E only — mocking dockerode.run reliably is not
|
||||
* worth the brittleness for this PR.
|
||||
* is exercised in manual E2E only (mocking dockerode.run reliably is not worth
|
||||
* the brittleness here). That includes every helper that runs a script in the
|
||||
* Alpine container: list/read/write/writeFileStream/delete/rename/copy and
|
||||
* their exit-code-to-HTTP mappings (e.g. copy's 11 to 409, 12 to 400). Those
|
||||
* run on Linux nodes / CI against a real named volume, not on this workstation.
|
||||
*/
|
||||
import { describe, it, expect } from 'vitest';
|
||||
import {
|
||||
|
||||
+391
-10
@@ -1,6 +1,11 @@
|
||||
import { Router, type Request, type Response, type NextFunction } from 'express';
|
||||
import { z } from 'zod';
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import crypto from 'crypto';
|
||||
import zlib from 'zlib';
|
||||
import { promises as fsp } from 'fs';
|
||||
import * as tar from 'tar-stream';
|
||||
import { inspect } from 'node:util';
|
||||
import YAML from 'yaml';
|
||||
import multer from 'multer';
|
||||
@@ -32,6 +37,7 @@ import { StackOpLockService, type StackOpAction } from '../services/StackOpLockS
|
||||
import { StackOpMetricsService, type StackOpAction as StackMetricAction } from '../services/StackOpMetricsService';
|
||||
import { FileExplorerMetricsService, type FileExplorerOp } from '../services/FileExplorerMetricsService';
|
||||
import { isValidGitSourcePath, isValidStackName, isValidServiceName, isValidRelativeStackPath } from '../utils/validation';
|
||||
import { normalizeBulkPaths, destWithinAnySource } from '../utils/bulkPaths';
|
||||
import { getErrorMessage } from '../utils/errors';
|
||||
import { isDebugEnabled } from '../utils/debug';
|
||||
import { sanitizeForLog } from '../utils/safeLog';
|
||||
@@ -157,12 +163,48 @@ export async function resolveAllEnvFilePaths(nodeId: number, stackName: string):
|
||||
return dotenv ? [dotenv.resolvedPath as string] : [];
|
||||
}
|
||||
|
||||
// Uploads spool to disk (not memory) so a 25 MB upload is never held in RAM.
|
||||
// The temp dir lives under the OS temp root, deliberately outside COMPOSE_DIR and
|
||||
// any browsable volume, so a running container never observes a half-written
|
||||
// spool. SENCHO_UPLOAD_DIR relocates it (e.g. onto a larger volume).
|
||||
const UPLOAD_TMP_DIR = process.env.SENCHO_UPLOAD_DIR
|
||||
? path.resolve(process.env.SENCHO_UPLOAD_DIR)
|
||||
: path.join(os.tmpdir(), 'sencho-uploads');
|
||||
const upload = multer({
|
||||
storage: multer.memoryStorage(),
|
||||
storage: multer.diskStorage({
|
||||
destination: (_req, _file, cb) => {
|
||||
fsp.mkdir(UPLOAD_TMP_DIR, { recursive: true })
|
||||
.then(() => cb(null, UPLOAD_TMP_DIR))
|
||||
.catch((err: Error) => cb(err, UPLOAD_TMP_DIR));
|
||||
},
|
||||
filename: (_req, _file, cb) => {
|
||||
cb(null, `up-${process.pid}-${Date.now()}-${crypto.randomBytes(8).toString('hex')}`);
|
||||
},
|
||||
}),
|
||||
limits: { fileSize: 25 * 1024 * 1024, files: 1 },
|
||||
preservePath: true,
|
||||
});
|
||||
|
||||
/**
|
||||
* Best-effort cleanup of a spooled upload temp file; never throws (a failed
|
||||
* cleanup must not turn a successful upload into an error). A persistent failure
|
||||
* would silently grow the spool dir, so log it at diagnostic level rather than
|
||||
* swallowing it blind.
|
||||
*/
|
||||
async function cleanupUploadTemp(req: Request): Promise<void> {
|
||||
const tmp = req.file?.path;
|
||||
if (!tmp) return;
|
||||
// Canonical js/path-injection barrier inline with the unlink sink: the spool
|
||||
// path is multer-generated within UPLOAD_TMP_DIR (a random filename), but
|
||||
// static analysis taints req.file.*, so confirm containment before unlinking.
|
||||
const baseResolved = path.resolve(UPLOAD_TMP_DIR);
|
||||
const resolved = path.resolve(tmp);
|
||||
if (!resolved.startsWith(baseResolved + path.sep)) return;
|
||||
await fsp.unlink(resolved).catch((err: unknown) => {
|
||||
logFileDiag('upload temp cleanup failed', { path: resolved, errorCode: fsErrorCode(err) });
|
||||
});
|
||||
}
|
||||
|
||||
function getRelPath(req: Request): string {
|
||||
return typeof req.query.path === 'string' ? req.query.path : '';
|
||||
}
|
||||
@@ -2086,10 +2128,21 @@ stacksRouter.get('/:stackName/files/download', async (req: Request, res: Respons
|
||||
}
|
||||
});
|
||||
|
||||
type UploadStartedReq = Request & { _fileUploadStartedAt?: number };
|
||||
type UploadStartedReq = Request & { _fileUploadStartedAt?: number; _fileUploadRoot?: StackFileRoot };
|
||||
|
||||
stacksRouter.post(
|
||||
'/:stackName/files/upload',
|
||||
// Authorize BEFORE multer touches the body, so an unauthorized caller or a
|
||||
// read-only/non-existent root is rejected without ever spooling a temp file.
|
||||
// The resolved root is stashed for the handler so it is not resolved twice.
|
||||
async (req: Request, res: Response, next: NextFunction) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:edit', 'stack', stackName)) return;
|
||||
const root = await resolveRootForOp(req, res, stackName, 'write');
|
||||
if (!root) return;
|
||||
(req as UploadStartedReq)._fileUploadRoot = root;
|
||||
next();
|
||||
},
|
||||
(req: Request, res: Response, next: NextFunction) => {
|
||||
// Capture the time the upload entered the route so every downstream
|
||||
// metric reports the same latency window: the body-transfer +
|
||||
@@ -2107,7 +2160,11 @@ stacksRouter.post(
|
||||
errorCode: 'TOO_LARGE',
|
||||
});
|
||||
recordFileOp(req.nodeId, 'upload', startedAt, false);
|
||||
return res.status(413).json({ error: 'File exceeds 25 MB limit', code: 'TOO_LARGE' });
|
||||
// diskStorage may have spooled a partial file before the limit fired.
|
||||
void cleanupUploadTemp(req).finally(() =>
|
||||
res.status(413).json({ error: 'File exceeds 25 MB limit', code: 'TOO_LARGE' }),
|
||||
);
|
||||
return;
|
||||
}
|
||||
if (err) {
|
||||
logFileOperation('warn', 'upload failed', {
|
||||
@@ -2117,27 +2174,33 @@ stacksRouter.post(
|
||||
errorCode: 'MULTER_ERROR',
|
||||
});
|
||||
recordFileOp(req.nodeId, 'upload', startedAt, false);
|
||||
return res.status(500).json({ error: 'Upload failed' });
|
||||
void cleanupUploadTemp(req).finally(() => res.status(500).json({ error: 'Upload failed' }));
|
||||
return;
|
||||
}
|
||||
next();
|
||||
});
|
||||
},
|
||||
async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:edit', 'stack', stackName)) return;
|
||||
// The pre-multer middleware already authorized and resolved the root.
|
||||
const root = (req as UploadStartedReq)._fileUploadRoot;
|
||||
if (!root) {
|
||||
await cleanupUploadTemp(req);
|
||||
return res.status(500).json({ error: 'Upload failed' });
|
||||
}
|
||||
if (!req.file) {
|
||||
return res.status(400).json({ error: 'No file provided' });
|
||||
}
|
||||
const relPath = getRelPath(req);
|
||||
if (relPath !== '' && !isValidRelativeStackPath(relPath)) {
|
||||
await cleanupUploadTemp(req);
|
||||
return res.status(400).json({ error: 'Invalid path', code: 'INVALID_PATH' });
|
||||
}
|
||||
const originalName = req.file.originalname;
|
||||
if (!isSafeUploadFilename(originalName)) {
|
||||
await cleanupUploadTemp(req);
|
||||
return res.status(400).json({ error: 'Invalid filename' });
|
||||
}
|
||||
const root = await resolveRootForOp(req, res, stackName, 'write');
|
||||
if (!root) return;
|
||||
const targetRelPath = relPath ? `${relPath}/${originalName}` : originalName;
|
||||
const overwrite = String(req.query.overwrite) === '1';
|
||||
// The multer wrapper stashed the route-entry timestamp on the request so
|
||||
@@ -2179,11 +2242,21 @@ stacksRouter.post(
|
||||
},
|
||||
});
|
||||
}
|
||||
// Use the atomic exclusive create for the non-overwrite case so a file
|
||||
// created by another writer after the pathKind check above is not
|
||||
// Canonical js/path-injection barrier: the spool path is multer-generated
|
||||
// within UPLOAD_TMP_DIR (a random filename), but static analysis taints
|
||||
// req.file.*; confirm containment so the value handed to the gateway and
|
||||
// FileSystemService streaming sinks is credited as safe.
|
||||
const spoolBase = path.resolve(UPLOAD_TMP_DIR);
|
||||
const tempPath = path.resolve(req.file.path);
|
||||
if (!tempPath.startsWith(spoolBase + path.sep)) {
|
||||
return res.status(400).json({ error: 'Upload failed' });
|
||||
}
|
||||
// Copy the spooled temp file into place (the spool survives; the finally
|
||||
// removes it). The atomic exclusive create for the non-overwrite case means
|
||||
// a file created by another writer after the pathKind check above is not
|
||||
// silently clobbered (a race surfaces as FILE_EXISTS -> 409, same as the
|
||||
// pre-emptive check). overwrite=true intentionally allows the clobber.
|
||||
await gateway.writeBuffer(root, stackName, targetRelPath, req.file.buffer, !overwrite);
|
||||
await gateway.writeFromTemp(root, stackName, targetRelPath, tempPath, !overwrite);
|
||||
afterStackMutation(req, stackName);
|
||||
logFileOperation('info', 'mutate', {
|
||||
nodeId: req.nodeId,
|
||||
@@ -2209,6 +2282,10 @@ stacksRouter.post(
|
||||
});
|
||||
recordFileOp(req.nodeId, 'upload', startedAt, false);
|
||||
return sendFsError(res, err, 'Failed to upload file', { notFoundMessage: 'Target directory not found' });
|
||||
} finally {
|
||||
// writeFromTemp streams (copies) the spool into place, so the temp file
|
||||
// always remains and must be removed on every exit (success, conflict, error).
|
||||
await cleanupUploadTemp(req);
|
||||
}
|
||||
},
|
||||
);
|
||||
@@ -2410,6 +2487,310 @@ stacksRouter.patch('/:stackName/files/rename', async (req: Request, res: Respons
|
||||
}
|
||||
});
|
||||
|
||||
stacksRouter.post('/:stackName/files/copy', async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:edit', 'stack', stackName)) return;
|
||||
const { from, to } = req.body as { from?: unknown; to?: unknown };
|
||||
if (typeof from !== 'string' || !from) {
|
||||
return res.status(400).json({ error: '"from" must be a non-empty string' });
|
||||
}
|
||||
if (typeof to !== 'string' || !to) {
|
||||
return res.status(400).json({ error: '"to" must be a non-empty string' });
|
||||
}
|
||||
if (!isValidRelativeStackPath(from)) {
|
||||
return res.status(400).json({ error: 'Invalid source path', code: 'INVALID_PATH' });
|
||||
}
|
||||
if (!isValidRelativeStackPath(to)) {
|
||||
return res.status(400).json({ error: 'Invalid destination path', code: 'INVALID_PATH' });
|
||||
}
|
||||
const root = await resolveRootForOp(req, res, stackName, 'write');
|
||||
if (!root) return;
|
||||
const startedAt = Date.now();
|
||||
logFileDiag('copy start', { stackName, from, to, nodeId: req.nodeId, rootKind: root.kind });
|
||||
try {
|
||||
await FileRootGateway.getInstance(req.nodeId).copy(root, stackName, from, to);
|
||||
afterStackMutation(req, stackName);
|
||||
logFileOperation('info', 'mutate', {
|
||||
nodeId: req.nodeId,
|
||||
op: 'copy',
|
||||
stack: stackName,
|
||||
path: from,
|
||||
toPath: to,
|
||||
rootKind: root.kind,
|
||||
backend: root.backend,
|
||||
});
|
||||
logFileDiag('copy timing', { stackName, from, to, nodeId: req.nodeId, elapsedMs: Date.now() - startedAt });
|
||||
recordFileOp(req.nodeId, 'copy', startedAt, true);
|
||||
return res.status(204).send();
|
||||
} catch (err: unknown) {
|
||||
logFileOperation('warn', 'copy failed', {
|
||||
nodeId: req.nodeId,
|
||||
op: 'copy',
|
||||
stack: stackName,
|
||||
path: from,
|
||||
toPath: to,
|
||||
errorCode: fsErrorCode(err),
|
||||
});
|
||||
recordFileOp(req.nodeId, 'copy', startedAt, false);
|
||||
return sendFsError(res, err, 'Failed to copy');
|
||||
}
|
||||
});
|
||||
|
||||
// ── Bulk file operations (delete / move / download) ─────────────────────────
|
||||
|
||||
const MAX_BULK = 100; // selected paths accepted per bulk request
|
||||
const MAX_ARCHIVE_ENTRIES = 5000; // files packed into one bulk-download archive
|
||||
const MAX_ARCHIVE_BYTES = 1024 * 1024 * 1024; // 1 GiB uncompressed cap
|
||||
|
||||
/**
|
||||
* Helper-backed named volumes are Linux containers (case-sensitive). Filesystem
|
||||
* roots follow the host: Windows/macOS fold case, Linux does not.
|
||||
*/
|
||||
function rootCaseSensitive(root: StackFileRoot): boolean {
|
||||
if (root.backend === 'helper') return true;
|
||||
return process.platform !== 'win32' && process.platform !== 'darwin';
|
||||
}
|
||||
|
||||
/** Validate a bulk path array; sends the 400 and returns null on any problem. */
|
||||
function parseBulkPaths(value: unknown, res: Response): string[] | null {
|
||||
if (!Array.isArray(value) || value.length === 0) {
|
||||
res.status(400).json({ error: 'A non-empty list of paths is required' });
|
||||
return null;
|
||||
}
|
||||
if (value.length > MAX_BULK) {
|
||||
res.status(400).json({ error: `Select at most ${MAX_BULK} items at once`, code: 'TOO_MANY' });
|
||||
return null;
|
||||
}
|
||||
const out: string[] = [];
|
||||
for (const p of value) {
|
||||
if (typeof p !== 'string' || p === '' || !isValidRelativeStackPath(p)) {
|
||||
res.status(400).json({ error: 'Invalid path in selection', code: 'INVALID_PATH' satisfies FsErrorCode });
|
||||
return null;
|
||||
}
|
||||
out.push(p);
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
/** A clean per-item failure message for a bulk result, mapping the opaque
|
||||
* filesystem codes that carry no friendly message of their own. */
|
||||
function bulkItemError(err: unknown): string {
|
||||
const e = err as Error & { code?: string };
|
||||
switch (e.code) {
|
||||
case 'EXDEV': return 'Cannot move across a storage boundary';
|
||||
case 'EEXIST': return 'A file or folder with that name already exists';
|
||||
case 'ENOENT': return 'No longer exists';
|
||||
case 'EISDIR': case 'ENOTDIR': return 'Path type changed';
|
||||
default: return e.message || e.code || 'Operation failed';
|
||||
}
|
||||
}
|
||||
|
||||
function archiveTooLargeError(message: string): Error & { code: string } {
|
||||
return Object.assign(new Error(message), { code: 'ARCHIVE_TOO_LARGE' });
|
||||
}
|
||||
|
||||
/**
|
||||
* Walk the normalized selection and return every file to pack, enforcing the
|
||||
* entry and byte caps. Throws ARCHIVE_TOO_LARGE if either cap is exceeded (or an
|
||||
* fs directory listing is truncated), so the caller can 413 before any archive
|
||||
* byte is streamed. File sizes come from listDir; every directory is stat-ed once
|
||||
* (the walk recurses into each), but individual files in a listing are not re-stat-ed.
|
||||
*/
|
||||
async function enumerateArchiveFiles(
|
||||
gateway: FileRootGateway,
|
||||
root: StackFileRoot,
|
||||
stackName: string,
|
||||
selection: string[],
|
||||
): Promise<string[]> {
|
||||
const files: string[] = [];
|
||||
let totalBytes = 0;
|
||||
const addFile = (relPath: string, size: number): void => {
|
||||
files.push(relPath);
|
||||
totalBytes += size;
|
||||
if (files.length > MAX_ARCHIVE_ENTRIES) throw archiveTooLargeError('The selection has too many files to download');
|
||||
if (totalBytes > MAX_ARCHIVE_BYTES) throw archiveTooLargeError('The selection is too large to download');
|
||||
};
|
||||
const visit = async (relPath: string): Promise<void> => {
|
||||
const st = await gateway.stat(root, stackName, relPath);
|
||||
if (st.type !== 'directory') {
|
||||
gateway.assertArchivable(root, relPath, st);
|
||||
addFile(relPath, st.size);
|
||||
return;
|
||||
}
|
||||
// Request one over the remaining budget so a directory that would push us one
|
||||
// entry past the cap is detected: the overflow entry reaches addFile (which
|
||||
// throws on >), or for larger directories the fs listing reports truncated.
|
||||
const remaining = MAX_ARCHIVE_ENTRIES - files.length + 1;
|
||||
const { entries, truncated } = await gateway.listDir(root, stackName, relPath, remaining);
|
||||
if (truncated) throw archiveTooLargeError('A selected folder has too many files to download');
|
||||
for (const entry of entries) {
|
||||
const childRel = `${relPath}/${entry.name}`;
|
||||
if (entry.type === 'directory') await visit(childRel);
|
||||
else {
|
||||
gateway.assertArchivable(root, childRel, entry);
|
||||
addFile(childRel, entry.size);
|
||||
}
|
||||
}
|
||||
};
|
||||
for (const p of selection) await visit(p);
|
||||
return files;
|
||||
}
|
||||
|
||||
stacksRouter.post('/:stackName/files/bulk-delete', async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:edit', 'stack', stackName)) return;
|
||||
const parsed = parseBulkPaths((req.body as { paths?: unknown }).paths, res);
|
||||
if (!parsed) return;
|
||||
const root = await resolveRootForOp(req, res, stackName, 'write');
|
||||
if (!root) return;
|
||||
const normalized = normalizeBulkPaths(parsed, rootCaseSensitive(root));
|
||||
const gateway = FileRootGateway.getInstance(req.nodeId);
|
||||
const deleted: string[] = [];
|
||||
const failed: { path: string; error: string }[] = [];
|
||||
for (const relPath of normalized) {
|
||||
const startedAt = Date.now();
|
||||
try {
|
||||
await gateway.deletePath(root, stackName, relPath, true);
|
||||
deleted.push(relPath);
|
||||
recordFileOp(req.nodeId, 'delete', startedAt, true);
|
||||
} catch (err: unknown) {
|
||||
failed.push({ path: relPath, error: bulkItemError(err) });
|
||||
recordFileOp(req.nodeId, 'delete', startedAt, false);
|
||||
}
|
||||
}
|
||||
// Partial-success: invalidate the roots cache if anything actually changed.
|
||||
if (deleted.length > 0) afterStackMutation(req, stackName);
|
||||
logFileOperation('info', 'mutate', { nodeId: req.nodeId, op: 'bulkDelete', stack: stackName, deleted: deleted.length, failed: failed.length, rootKind: root.kind, backend: root.backend });
|
||||
return res.json({ deleted, failed });
|
||||
});
|
||||
|
||||
stacksRouter.post('/:stackName/files/bulk-move', async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:edit', 'stack', stackName)) return;
|
||||
const body = req.body as { from?: unknown; toDir?: unknown };
|
||||
const parsed = parseBulkPaths(body.from, res);
|
||||
if (!parsed) return;
|
||||
if (typeof body.toDir !== 'string') {
|
||||
return res.status(400).json({ error: '"toDir" must be a string (use "" for the root)' });
|
||||
}
|
||||
const toDir = body.toDir;
|
||||
if (toDir !== '' && !isValidRelativeStackPath(toDir)) {
|
||||
return res.status(400).json({ error: 'Invalid destination', code: 'INVALID_PATH' satisfies FsErrorCode });
|
||||
}
|
||||
const root = await resolveRootForOp(req, res, stackName, 'write');
|
||||
if (!root) return;
|
||||
const caseSensitive = rootCaseSensitive(root);
|
||||
const normalized = normalizeBulkPaths(parsed, caseSensitive);
|
||||
// Reject the whole request if the destination is one of the moved folders or
|
||||
// sits inside one (which would move a folder into its own subtree).
|
||||
if (destWithinAnySource(toDir, normalized, caseSensitive)) {
|
||||
return res.status(400).json({ error: 'Cannot move the selection into itself', code: 'INVALID_PATH' satisfies FsErrorCode });
|
||||
}
|
||||
const gateway = FileRootGateway.getInstance(req.nodeId);
|
||||
const moved: string[] = [];
|
||||
const failed: { path: string; error: string }[] = [];
|
||||
for (const fromRel of normalized) {
|
||||
const startedAt = Date.now();
|
||||
const name = fromRel.split('/').pop() as string;
|
||||
const toRel = toDir ? `${toDir}/${name}` : name;
|
||||
try {
|
||||
await gateway.rename(root, stackName, fromRel, toRel);
|
||||
moved.push(fromRel);
|
||||
recordFileOp(req.nodeId, 'rename', startedAt, true);
|
||||
} catch (err: unknown) {
|
||||
failed.push({ path: fromRel, error: bulkItemError(err) });
|
||||
recordFileOp(req.nodeId, 'rename', startedAt, false);
|
||||
}
|
||||
}
|
||||
if (moved.length > 0) afterStackMutation(req, stackName);
|
||||
logFileOperation('info', 'mutate', { nodeId: req.nodeId, op: 'bulkMove', stack: stackName, moved: moved.length, failed: failed.length, rootKind: root.kind, backend: root.backend });
|
||||
return res.json({ moved, failed });
|
||||
});
|
||||
|
||||
stacksRouter.get('/:stackName/files/bulk-download', async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:read', 'stack', stackName)) return;
|
||||
const raw = req.query.path;
|
||||
const list = Array.isArray(raw) ? raw : raw !== undefined ? [raw] : [];
|
||||
const parsed = parseBulkPaths(list, res);
|
||||
if (!parsed) return;
|
||||
const root = await resolveRootForOp(req, res, stackName, 'read');
|
||||
if (!root) return;
|
||||
const gateway = FileRootGateway.getInstance(req.nodeId);
|
||||
const normalized = normalizeBulkPaths(parsed, rootCaseSensitive(root));
|
||||
const startedAt = Date.now();
|
||||
|
||||
// Prewalk + cap enforcement BEFORE any response header is sent, so a too-large
|
||||
// selection fails as a clean 413 rather than a truncated archive.
|
||||
let files: string[];
|
||||
try {
|
||||
files = await enumerateArchiveFiles(gateway, root, stackName, normalized);
|
||||
} catch (err: unknown) {
|
||||
recordFileOp(req.nodeId, 'download', startedAt, false);
|
||||
const code = (err as { code?: string }).code;
|
||||
if (code === 'ARCHIVE_TOO_LARGE') {
|
||||
return res.status(413).json({ error: (err as Error).message, code: 'TOO_LARGE' });
|
||||
}
|
||||
if (code === 'ARCHIVE_UNSUPPORTED') {
|
||||
return res.status(400).json({ error: (err as Error).message, code: 'UNSUPPORTED' });
|
||||
}
|
||||
return sendFsError(res, err, 'Failed to prepare download');
|
||||
}
|
||||
if (files.length === 0) {
|
||||
recordFileOp(req.nodeId, 'download', startedAt, false);
|
||||
return res.status(404).json({ error: 'Nothing to download', code: 'NOT_FOUND' satisfies FsErrorCode });
|
||||
}
|
||||
|
||||
res.setHeader('Content-Type', 'application/gzip');
|
||||
res.setHeader('Content-Disposition', `attachment; filename="${stackName}-files.tar.gz"`);
|
||||
const pack = tar.pack();
|
||||
const gzip = zlib.createGzip();
|
||||
const onStreamError = (err: Error): void => {
|
||||
logFileOperation('warn', 'bulk download stream error', { nodeId: req.nodeId, stack: stackName, errorCode: fsErrorCode(err) });
|
||||
if (!res.writableEnded) res.destroy();
|
||||
};
|
||||
pack.on('error', onStreamError);
|
||||
gzip.on('error', onStreamError);
|
||||
// If the client aborts mid-download, stop fetching the remaining files (each
|
||||
// helper-volume read is a container exec) instead of streaming into a dead pipe.
|
||||
let aborted = false;
|
||||
res.on('close', () => {
|
||||
if (!res.writableEnded) {
|
||||
aborted = true;
|
||||
pack.destroy();
|
||||
}
|
||||
});
|
||||
pack.pipe(gzip).pipe(res);
|
||||
|
||||
try {
|
||||
for (const relPath of files) {
|
||||
if (aborted) break;
|
||||
const dl = await gateway.download(root, stackName, relPath);
|
||||
if (dl.kind === 'buffer') {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
pack.entry({ name: relPath }, dl.buffer, (err) => (err ? reject(err) : resolve()));
|
||||
});
|
||||
} else {
|
||||
await new Promise<void>((resolve, reject) => {
|
||||
const entry = pack.entry({ name: relPath, size: dl.size }, (err) => (err ? reject(err) : resolve()));
|
||||
dl.stream.on('error', reject);
|
||||
entry.on('error', reject);
|
||||
dl.stream.pipe(entry);
|
||||
});
|
||||
}
|
||||
}
|
||||
pack.finalize();
|
||||
recordFileOp(req.nodeId, 'download', startedAt, true);
|
||||
} catch (err: unknown) {
|
||||
// Headers are already sent, so surface the failure by tearing the stream
|
||||
// down rather than trying to change the status.
|
||||
logFileOperation('warn', 'bulk download failed mid-stream', { nodeId: req.nodeId, stack: stackName, errorCode: fsErrorCode(err) });
|
||||
recordFileOp(req.nodeId, 'download', startedAt, false);
|
||||
pack.destroy();
|
||||
if (!res.writableEnded) res.destroy();
|
||||
}
|
||||
});
|
||||
|
||||
stacksRouter.get('/:stackName/files/permissions', async (req: Request, res: Response) => {
|
||||
const stackName = req.params.stackName as string;
|
||||
if (!requirePermission(req, res, 'stack:read', 'stack', stackName)) return;
|
||||
|
||||
@@ -20,6 +20,7 @@ export type FileExplorerOp =
|
||||
| 'delete'
|
||||
| 'mkdir'
|
||||
| 'rename'
|
||||
| 'copy'
|
||||
| 'chmod';
|
||||
|
||||
interface FileExplorerOpStats {
|
||||
|
||||
@@ -13,9 +13,10 @@
|
||||
* within the same (seconds-resolution) second.
|
||||
*/
|
||||
import type { Readable } from 'stream';
|
||||
import { createReadStream, promises as fsp } from 'fs';
|
||||
|
||||
import { FileSystemService, type FileEntry, type FileRootScope } from './FileSystemService';
|
||||
import { VolumeBrowserService, makeHelperVersion, type VolumeEntry } from './VolumeBrowserService';
|
||||
import { VolumeBrowserService, makeHelperVersion, DOWNLOAD_MAX_BYTES, type VolumeEntry } from './VolumeBrowserService';
|
||||
import type { StackFileRoot } from './StackFileRootsService';
|
||||
|
||||
const HELPER_VIEW_MAX_BYTES = 2 * 1024 * 1024; // match the stack-source viewer cap
|
||||
@@ -51,7 +52,9 @@ export function parseFsVersion(raw: string | undefined): number | null {
|
||||
function volumeEntryToFileEntry(e: VolumeEntry): FileEntry {
|
||||
return {
|
||||
name: e.name,
|
||||
type: e.type === 'other' ? 'file' : e.type,
|
||||
// Preserve 'other' (non-regular entries) so the archive guard can reject
|
||||
// what the helper download path would refuse; the UI renders it like a file.
|
||||
type: e.type,
|
||||
size: e.size,
|
||||
mtime: e.mtime * 1000,
|
||||
isProtected: false,
|
||||
@@ -96,8 +99,12 @@ export class FileRootGateway {
|
||||
limit: number,
|
||||
): Promise<{ entries: FileEntry[]; total: number; truncated: boolean }> {
|
||||
if (root.backend === 'helper') {
|
||||
const entries = (await this.helper().listDir(root.hostPathOrName, relPath)).map(volumeEntryToFileEntry);
|
||||
return { entries, total: entries.length, truncated: false };
|
||||
// Ask for one over the limit so a fully-listed directory is distinguishable
|
||||
// from a truncated one without the helper buffering every entry.
|
||||
const raw = await this.helper().listDir(root.hostPathOrName, relPath, limit);
|
||||
const truncated = raw.length > limit;
|
||||
const entries = raw.slice(0, limit).map(volumeEntryToFileEntry);
|
||||
return { entries, total: entries.length, truncated };
|
||||
}
|
||||
return this.fs().listStackDirectoryPage(stackName, relPath, { limit, scope: this.scopeFor(root) });
|
||||
}
|
||||
@@ -173,22 +180,56 @@ export class FileRootGateway {
|
||||
return this.fs().pathKind(stackName, relPath, this.scopeFor(root));
|
||||
}
|
||||
|
||||
/** Upload write. `exclusive` rejects an existing target (no overwrite). */
|
||||
async writeBuffer(
|
||||
/** Stat a single entry (type + size). Used by bulk download to size the archive. */
|
||||
async stat(root: StackFileRoot, stackName: string, relPath: string): Promise<FileEntry> {
|
||||
if (root.backend === 'helper') {
|
||||
return volumeEntryToFileEntry(await this.helper().stat(root.hostPathOrName, relPath));
|
||||
}
|
||||
return this.fs().statStackEntry(stackName, relPath, this.scopeFor(root));
|
||||
}
|
||||
|
||||
/**
|
||||
* Reject a non-directory entry the backend's download path could not stream,
|
||||
* BEFORE the archive prewalk commits to sending response headers. The fs
|
||||
* backend streams any in-root file (and follows in-root symlinks), so it has
|
||||
* no constraint; the helper backend's download refuses symlinks/non-regular
|
||||
* files and caps each file at DOWNLOAD_MAX_BYTES, which must be enforced here
|
||||
* or a bulk download would tear mid-archive when gateway.download() later
|
||||
* throws. Throws ARCHIVE_UNSUPPORTED (-> 400) or ARCHIVE_TOO_LARGE (-> 413).
|
||||
*/
|
||||
assertArchivable(root: StackFileRoot, relPath: string, entry: FileEntry): void {
|
||||
if (root.backend !== 'helper') return;
|
||||
if (entry.type !== 'file') {
|
||||
throw Object.assign(new Error(`"${relPath}" cannot be downloaded from this volume`), { code: 'ARCHIVE_UNSUPPORTED' });
|
||||
}
|
||||
if (entry.size > DOWNLOAD_MAX_BYTES) {
|
||||
throw Object.assign(new Error(`"${relPath}" is too large to download from this volume`), { code: 'ARCHIVE_TOO_LARGE' });
|
||||
}
|
||||
}
|
||||
|
||||
/**
|
||||
* Upload write sourced from a temp file spooled to disk (multer diskStorage),
|
||||
* so the upload is never buffered in memory. `exclusive` rejects an existing
|
||||
* target (no overwrite). The caller owns deleting `tempPath`.
|
||||
*/
|
||||
async writeFromTemp(
|
||||
root: StackFileRoot,
|
||||
stackName: string,
|
||||
relPath: string,
|
||||
buffer: Buffer,
|
||||
tempPath: string,
|
||||
exclusive: boolean,
|
||||
): Promise<void> {
|
||||
if (root.backend === 'helper') {
|
||||
if (exclusive && (await this.helper().pathKind(root.hostPathOrName, relPath)) !== null) {
|
||||
throw Object.assign(new Error('File already exists'), { code: 'FILE_EXISTS' });
|
||||
}
|
||||
await this.helper().writeFile(root.hostPathOrName, relPath, buffer);
|
||||
// The helper writes via `cat`, which cannot report a short write; pass the
|
||||
// spooled byte count so writeFileStream can verify the volume got it all.
|
||||
const { size } = await fsp.stat(tempPath);
|
||||
await this.helper().writeFileStream(root.hostPathOrName, relPath, createReadStream(tempPath), size);
|
||||
return;
|
||||
}
|
||||
await this.fs().writeStackFileBuffer(stackName, relPath, buffer, { exclusive, scope: this.scopeFor(root) });
|
||||
await this.fs().writeScopedFileFromTemp(stackName, relPath, tempPath, { exclusive, scope: this.scopeFor(root) });
|
||||
}
|
||||
|
||||
async download(
|
||||
@@ -220,6 +261,12 @@ export class FileRootGateway {
|
||||
return this.fs().renameStackPath(stackName, fromRel, toRel, this.scopeFor(root));
|
||||
}
|
||||
|
||||
/** Copy a file or directory within a single root (cross-root copy is rejected at the route). */
|
||||
async copy(root: StackFileRoot, stackName: string, fromRel: string, toRel: string): Promise<void> {
|
||||
if (root.backend === 'helper') return this.helper().copy(root.hostPathOrName, fromRel, toRel);
|
||||
return this.fs().copyScopedPath(stackName, fromRel, toRel, this.scopeFor(root));
|
||||
}
|
||||
|
||||
async getMode(root: StackFileRoot, stackName: string, relPath: string): Promise<{ mode: number; octal: string }> {
|
||||
if (root.backend === 'helper') throw unsupportedOnHelperRoot();
|
||||
return this.fs().getStackEntryMode(stackName, relPath, this.scopeFor(root));
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
import path from 'path';
|
||||
import os from 'os';
|
||||
import crypto from 'crypto';
|
||||
import { promises as fsPromises, createReadStream } from 'fs';
|
||||
import { promises as fsPromises, createReadStream, createWriteStream } from 'fs';
|
||||
import type { Dirent } from 'fs';
|
||||
import type { Readable } from 'stream';
|
||||
import { Readable } from 'stream';
|
||||
import { pipeline } from 'stream/promises';
|
||||
import { NodeRegistry } from './NodeRegistry';
|
||||
import { isPathWithinBase, isValidStackName } from '../utils/validation';
|
||||
import { isBinaryBuffer } from '../utils/binaryDetect';
|
||||
@@ -11,7 +12,10 @@ import { sanitizeForLog } from '../utils/safeLog';
|
||||
|
||||
export interface FileEntry {
|
||||
name: string;
|
||||
type: 'file' | 'directory' | 'symlink';
|
||||
// 'other' covers non-regular helper-volume entries (fifo/socket/device): they
|
||||
// are unrepresentable on the fs backend but the helper can surface them, and
|
||||
// they must stay distinct from 'file' so the archive guard can reject them.
|
||||
type: 'file' | 'directory' | 'symlink' | 'other';
|
||||
size: number;
|
||||
mtime: number;
|
||||
isProtected: boolean;
|
||||
@@ -100,6 +104,18 @@ function fsCaseKey(s: string): string {
|
||||
return process.platform === 'win32' || process.platform === 'darwin' ? s.toLowerCase() : s;
|
||||
}
|
||||
|
||||
/**
|
||||
* True when resolved absolute path `candidate` is `parent` itself or sits inside
|
||||
* it, compared case-folded so the guard stays authoritative on a case-insensitive
|
||||
* filesystem. Used to block moving/copying a directory into its own subtree.
|
||||
*/
|
||||
function isSameOrDescendantFsPath(parent: string, candidate: string): boolean {
|
||||
const parentKey = fsCaseKey(parent);
|
||||
const parentKeyWithSep = parentKey.endsWith(path.sep) ? parentKey : parentKey + path.sep;
|
||||
const candidateKey = fsCaseKey(candidate);
|
||||
return candidateKey === parentKey || candidateKey.startsWith(parentKeyWithSep);
|
||||
}
|
||||
|
||||
function isProtectedRelPath(relPath: string): boolean {
|
||||
if (!relPath) return false;
|
||||
const normalized = stripTrailingSlash(relPath);
|
||||
@@ -1043,9 +1059,14 @@ export class FileSystemService {
|
||||
* compose. Tracked as a follow-up hardening, not a per-root regression.
|
||||
*/
|
||||
private async resolveSafePathWithin(rootAbsDir: string, relPath: string): Promise<string> {
|
||||
const target = relPath === '' ? rootAbsDir : path.resolve(rootAbsDir, relPath);
|
||||
|
||||
if (!isPathWithinBase(target, rootAbsDir)) {
|
||||
// Canonical js/path-injection barrier inline with the realpath sinks below:
|
||||
// isPathWithinBase performs the same containment check, but static analysis
|
||||
// only credits the path.resolve + startsWith form when it sits at the sink.
|
||||
// relPath === '' resolves to the (server-controlled) root itself and carries
|
||||
// no user input, so it needs no containment check.
|
||||
const baseResolved = path.resolve(rootAbsDir);
|
||||
const target = path.resolve(baseResolved, relPath);
|
||||
if (relPath !== '' && !target.startsWith(baseResolved + path.sep)) {
|
||||
throw Object.assign(new Error('Path escapes root directory'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
|
||||
@@ -1067,6 +1088,21 @@ export class FileSystemService {
|
||||
}
|
||||
suffix.unshift(path.basename(existing));
|
||||
existing = parent;
|
||||
if (existing === baseResolved) {
|
||||
// Reached the root: realpath the untainted base (never a tainted input)
|
||||
// and reattach the not-yet-existing suffix.
|
||||
const realBase = await fsPromises.realpath(baseResolved);
|
||||
if (!isPathWithinBase(realBase, rootAbsDir)) {
|
||||
throw Object.assign(new Error('Symlink escapes root directory'), { code: 'SYMLINK_ESCAPE' });
|
||||
}
|
||||
realTarget = path.join(realBase, ...suffix);
|
||||
break;
|
||||
}
|
||||
// Inline js/path-injection barrier: existing is now strictly below the
|
||||
// root, so the canonical path.resolve + startsWith form credits the sink.
|
||||
if (!existing.startsWith(baseResolved + path.sep)) {
|
||||
throw Object.assign(new Error('Path escapes root directory'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
try {
|
||||
const realExisting = await fsPromises.realpath(existing);
|
||||
if (!isPathWithinBase(realExisting, rootAbsDir)) {
|
||||
@@ -1255,7 +1291,7 @@ export class FileSystemService {
|
||||
*/
|
||||
private async writeStackFileAtomic(
|
||||
safePath: string,
|
||||
data: string | Buffer,
|
||||
data: string | Buffer | Readable,
|
||||
opts: { exclusive?: boolean } = {},
|
||||
): Promise<void> {
|
||||
await fsPromises.mkdir(path.dirname(safePath), { recursive: true });
|
||||
@@ -1265,13 +1301,28 @@ export class FileSystemService {
|
||||
const tmpPath = `${safePath}.sencho-tmp-${suffix}`;
|
||||
let stagedTmp = false;
|
||||
try {
|
||||
const fh = await fsPromises.open(tmpPath, 'wx');
|
||||
stagedTmp = true;
|
||||
try {
|
||||
await fh.writeFile(data);
|
||||
await fh.sync();
|
||||
} finally {
|
||||
await fh.close();
|
||||
if (data instanceof Readable) {
|
||||
// Stream a temp-file source (an upload spooled to disk) into the staging
|
||||
// file without buffering it in memory. 'wx' exclusively creates the
|
||||
// staging file; the random suffix already guarantees a fresh name.
|
||||
const ws = createWriteStream(tmpPath, { flags: 'wx' });
|
||||
stagedTmp = true;
|
||||
await pipeline(data, ws);
|
||||
const synced = await fsPromises.open(tmpPath, 'r+');
|
||||
try {
|
||||
await synced.sync();
|
||||
} finally {
|
||||
await synced.close();
|
||||
}
|
||||
} else {
|
||||
const fh = await fsPromises.open(tmpPath, 'wx');
|
||||
stagedTmp = true;
|
||||
try {
|
||||
await fh.writeFile(data);
|
||||
await fh.sync();
|
||||
} finally {
|
||||
await fh.close();
|
||||
}
|
||||
}
|
||||
if (opts.exclusive) {
|
||||
// link() is atomic against EEXIST. Tmp and target are guaranteed to live
|
||||
@@ -1306,14 +1357,22 @@ export class FileSystemService {
|
||||
await this.writeStackFileAtomic(safePath, content, opts);
|
||||
}
|
||||
|
||||
async writeStackFileBuffer(
|
||||
/**
|
||||
* Atomic, scoped write whose source is a temp file on disk (an upload spooled
|
||||
* by multer's diskStorage). Streams the temp file into a staging sibling in the
|
||||
* target's own directory, fsyncs, then links/renames into place, so a large
|
||||
* upload is never buffered in memory and the temp file's filesystem can differ
|
||||
* from the stack/volume filesystem (no cross-device rename). The caller owns
|
||||
* deleting tempPath.
|
||||
*/
|
||||
async writeScopedFileFromTemp(
|
||||
stackName: string,
|
||||
relPath: string,
|
||||
buffer: Buffer,
|
||||
tempPath: string,
|
||||
opts?: { exclusive?: boolean; scope?: FileRootScope },
|
||||
): Promise<void> {
|
||||
const safePath = await this.resolveScopedPath(stackName, relPath, opts?.scope);
|
||||
await this.writeStackFileAtomic(safePath, buffer, opts);
|
||||
await this.writeStackFileAtomic(safePath, createReadStream(tempPath), { exclusive: opts?.exclusive });
|
||||
}
|
||||
|
||||
/**
|
||||
@@ -1481,17 +1540,10 @@ export class FileSystemService {
|
||||
throw Object.assign(new Error('Invalid destination name'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
// Block moving a directory into itself or one of its own descendants; fs.rename
|
||||
// would otherwise fail with an opaque EINVAL/EPERM. Compare case-folded so the
|
||||
// guard stays authoritative when the source is supplied with non-disk casing on
|
||||
// a case-insensitive filesystem.
|
||||
// would otherwise fail with an opaque EINVAL/EPERM.
|
||||
const fromStat = await fsPromises.lstat(fromPath);
|
||||
if (fromStat.isDirectory()) {
|
||||
const fromKey = fsCaseKey(fromPath);
|
||||
const fromKeyWithSep = fromKey.endsWith(path.sep) ? fromKey : fromKey + path.sep;
|
||||
const toKey = fsCaseKey(toPath);
|
||||
if (toKey === fromKey || toKey.startsWith(fromKeyWithSep)) {
|
||||
throw Object.assign(new Error('Cannot move a folder into itself'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
if (fromStat.isDirectory() && isSameOrDescendantFsPath(fromPath, toPath)) {
|
||||
throw Object.assign(new Error('Cannot move a folder into itself'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
// Prevent overwriting an existing path. lstat (not access) so a dangling
|
||||
// symlink already at the destination still counts as occupied.
|
||||
@@ -1505,6 +1557,46 @@ export class FileSystemService {
|
||||
await fsPromises.rename(fromPath, toPath);
|
||||
}
|
||||
|
||||
/**
|
||||
* Copies a file or directory within a single root. The source resolves through
|
||||
* the leaf helper and the copy does not dereference symlinks, so a symlink
|
||||
* entry is copied as a link (matching the delete/rename leaf policy) rather
|
||||
* than followed to its target. Only the destination is protection-checked:
|
||||
* duplicating a protected file (e.g. compose.yaml) elsewhere is allowed, but a
|
||||
* copy cannot create a reserved name at a protected root. An existing
|
||||
* destination is rejected (surfaced as EEXIST, which the route maps to 409).
|
||||
*/
|
||||
async copyScopedPath(stackName: string, fromRel: string, toRel: string, scope?: FileRootScope): Promise<void> {
|
||||
if ((scope?.protectedEnabled ?? true) && isProtectedRelPath(toRel)) throw protectedFileError(toRel);
|
||||
const fromPath = await this.resolveScopedLeafPath(stackName, fromRel, scope);
|
||||
const toPath = await this.resolveScopedLeafPath(stackName, toRel, scope);
|
||||
const toName = path.basename(toPath);
|
||||
if (!toName || toName === '.' || toName === '..') {
|
||||
throw Object.assign(new Error('Invalid destination name'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
// Block copying a directory into itself or one of its own descendants;
|
||||
// fs.cp would otherwise recurse into the copy it is creating.
|
||||
const fromStat = await fsPromises.lstat(fromPath);
|
||||
if (fromStat.isDirectory() && isSameOrDescendantFsPath(fromPath, toPath)) {
|
||||
throw Object.assign(new Error('Cannot copy a folder into itself'), { code: 'INVALID_PATH' });
|
||||
}
|
||||
try {
|
||||
await fsPromises.cp(fromPath, toPath, {
|
||||
recursive: fromStat.isDirectory(),
|
||||
dereference: false,
|
||||
errorOnExist: true,
|
||||
force: false,
|
||||
});
|
||||
} catch (err: unknown) {
|
||||
// fs.cp raises ERR_FS_CP_EEXIST when the destination already exists; remap
|
||||
// to EEXIST so the route returns 409, matching rename's conflict handling.
|
||||
if ((err as NodeJS.ErrnoException).code === 'ERR_FS_CP_EEXIST') {
|
||||
throw Object.assign(new Error('A file or folder with that name already exists'), { code: 'EEXIST' });
|
||||
}
|
||||
throw err;
|
||||
}
|
||||
}
|
||||
|
||||
async getStackEntryMode(stackName: string, relPath: string, scope?: FileRootScope): Promise<{ mode: number; octal: string }> {
|
||||
const safePath = await this.resolveScopedPath(stackName, relPath, scope);
|
||||
const stat = await fsPromises.stat(safePath);
|
||||
|
||||
@@ -1,4 +1,5 @@
|
||||
import { Writable } from 'stream';
|
||||
import { Writable, Readable } from 'stream';
|
||||
import { pipeline } from 'stream/promises';
|
||||
import path from 'path';
|
||||
import { createHash } from 'crypto';
|
||||
import DockerController from './DockerController';
|
||||
@@ -10,7 +11,7 @@ const DEFAULT_MAX_BYTES = 5 * 1024 * 1024;
|
||||
// buffered up to this size and a larger file is rejected rather than silently
|
||||
// truncated. The bound matches the upload limit so the in-memory footprint is no
|
||||
// worse than the existing multipart upload path.
|
||||
const DOWNLOAD_MAX_BYTES = 25 * 1024 * 1024;
|
||||
export const DOWNLOAD_MAX_BYTES = 25 * 1024 * 1024;
|
||||
const EXEC_TIMEOUT_MS = 30_000;
|
||||
|
||||
// Containment guard run inside the helper after every `cd`. Even though
|
||||
@@ -27,6 +28,8 @@ const ROOT_GUARD =
|
||||
const LIST_SCRIPT = `set -e
|
||||
cd -- "$1" || exit 1
|
||||
${ROOT_GUARD}
|
||||
lim="$2"
|
||||
n=0
|
||||
for entry in * .[!.]* ..?*; do
|
||||
[ -e "$entry" ] || [ -L "$entry" ] || continue
|
||||
if [ -L "$entry" ]; then t=l; link=$(readlink -- "$entry" 2>/dev/null || echo "")
|
||||
@@ -37,6 +40,8 @@ for entry in * .[!.]* ..?*; do
|
||||
size=$(stat -c '%s' -- "$entry" 2>/dev/null || echo 0)
|
||||
mtime=$(stat -c '%Y' -- "$entry" 2>/dev/null || echo 0)
|
||||
printf '%s\\t%s\\t%s\\t%s\\t%s\\n' "$t" "$size" "$mtime" "$entry" "$link"
|
||||
n=$((n+1))
|
||||
if [ -n "$lim" ] && [ "$n" -ge "$lim" ]; then break; fi
|
||||
done`;
|
||||
|
||||
// Contain the parent before statting the leaf: cd into the leaf's directory and
|
||||
@@ -104,6 +109,26 @@ fd=$(dirname -- "$from"); td=$(dirname -- "$to")
|
||||
{ [ -e "$to" ] || [ -L "$to" ]; } && { echo "destination exists" >&2; exit 11; }
|
||||
mv -- "$from" "$to"`;
|
||||
|
||||
// $1 = from, $2 = to. Both parents are contained and the destination must not
|
||||
// exist. cp -RP recurses without dereferencing symlinks (copying links as links,
|
||||
// matching the fs backend) and does NOT preserve ownership, since the helper runs
|
||||
// unprivileged and cannot chown; new entries are owned by the helper user, like
|
||||
// the helper write. A directory may not be copied into itself or a descendant
|
||||
// (cp would otherwise recurse into the copy it is creating).
|
||||
const COPY_SCRIPT = `set -e
|
||||
from="$1"; to="$2"
|
||||
fd=$(dirname -- "$from"); td=$(dirname -- "$to")
|
||||
sp=$( cd -- "$fd" 2>/dev/null && pwd -P ) || { echo "source escapes volume root" >&2; exit 7; }
|
||||
case "$sp" in "${VOLUME_MOUNT}"|"${VOLUME_MOUNT}/"*) ;; *) echo "source escapes volume root" >&2; exit 7 ;; esac
|
||||
dp=$( cd -- "$td" 2>/dev/null && pwd -P ) || { echo "destination escapes volume root" >&2; exit 7; }
|
||||
case "$dp" in "${VOLUME_MOUNT}"|"${VOLUME_MOUNT}/"*) ;; *) echo "destination escapes volume root" >&2; exit 7 ;; esac
|
||||
{ [ -e "$to" ] || [ -L "$to" ]; } && { echo "destination exists" >&2; exit 11; }
|
||||
src="$sp/$(basename -- "$from")"
|
||||
if [ -d "$src" ] && [ ! -L "$src" ]; then
|
||||
case "$dp/" in "$src/"*) echo "cannot copy a folder into itself" >&2; exit 12 ;; esac
|
||||
fi
|
||||
cp -RP -- "$from" "$to"`;
|
||||
|
||||
// $1 = relative path. Prints directory|file|none. Used for upload-overwrite checks.
|
||||
const PATHKIND_SCRIPT = `set -e
|
||||
p="$1"
|
||||
@@ -186,7 +211,7 @@ export class VolumeBrowserService {
|
||||
return new VolumeBrowserService(nodeId ?? 1);
|
||||
}
|
||||
|
||||
async listDir(volumeName: string, relPath: string): Promise<VolumeEntry[]> {
|
||||
async listDir(volumeName: string, relPath: string, limit?: number): Promise<VolumeEntry[]> {
|
||||
const safe = sanitizeRelPath(relPath);
|
||||
await this.assertVolumeExists(volumeName);
|
||||
await this.ensureHelperImage();
|
||||
@@ -194,10 +219,12 @@ export class VolumeBrowserService {
|
||||
// Portable across BusyBox (Alpine) and GNU coreutils. Lists each
|
||||
// direct child with a tab-separated row: type<TAB>size<TAB>mtime<TAB>
|
||||
// name<TAB>symlinkTarget. We chdir to /v/<safe> first so user input is
|
||||
// never an argv element passed to find/stat.
|
||||
const script = LIST_SCRIPT;
|
||||
// never an argv element passed to find/stat. When a limit is given the
|
||||
// script breaks after limit+1 rows, so a huge directory is never fully
|
||||
// buffered; the caller detects truncation from the overflow row.
|
||||
const limArg = limit !== undefined ? String(limit + 1) : '';
|
||||
const { stdout, stderr, exitCode } = await this.runHelper(volumeName, [
|
||||
'sh', '-c', script, 'sh', `./${safe || ''}`,
|
||||
'sh', '-c', LIST_SCRIPT, 'sh', `./${safe || ''}`, limArg,
|
||||
]);
|
||||
|
||||
if (exitCode !== 0) {
|
||||
@@ -392,6 +419,39 @@ export class VolumeBrowserService {
|
||||
return { mtimeMs: meta.mtime * 1000, size: meta.size };
|
||||
}
|
||||
|
||||
/**
|
||||
* Like writeFile but streams the content from a Readable (an upload spooled to
|
||||
* a temp file) straight into the helper's stdin, so a large upload is never
|
||||
* held in memory. Non-atomic in-place write (`cat > file`), matching writeFile
|
||||
* (see writeFile for the ownership-preservation rationale); the caller owns the
|
||||
* source stream's underlying temp file. `expectedSize` is the source byte count:
|
||||
* `cat` cannot report a short write, so the post-write size is checked against
|
||||
* it to catch a truncated transfer that exited cleanly.
|
||||
*/
|
||||
async writeFileStream(volumeName: string, relPath: string, source: Readable, expectedSize: number): Promise<{ mtimeMs: number; size: number }> {
|
||||
const safe = sanitizeRelPath(relPath);
|
||||
if (!safe) throw new ExecError('Cannot write the volume root', 400);
|
||||
await this.assertVolumeExists(volumeName);
|
||||
await this.ensureHelperImage();
|
||||
|
||||
const { stderr, exitCode } = await this.runHelper(
|
||||
volumeName,
|
||||
['sh', '-c', WRITE_SCRIPT, 'sh', `./${safe}`],
|
||||
{ writable: true, stdin: source },
|
||||
);
|
||||
if (exitCode !== 0) {
|
||||
const msg = stderr.toString('utf-8').trim();
|
||||
if (/Permission denied/i.test(msg) || exitCode === 8) throw new ExecError('Permission denied', 403);
|
||||
if (exitCode === 9) throw new ExecError('Target is a directory', 400);
|
||||
throw new ExecError(`Write failed: ${msg.substring(0, 200) || 'unknown error'}`);
|
||||
}
|
||||
const meta = await this.stat(volumeName, safe);
|
||||
if (meta.size !== expectedSize) {
|
||||
throw new ExecError('Upload did not fully write to the volume', 500);
|
||||
}
|
||||
return { mtimeMs: meta.mtime * 1000, size: meta.size };
|
||||
}
|
||||
|
||||
/** Create a single directory; its parent must already exist. */
|
||||
async mkdir(volumeName: string, relPath: string): Promise<void> {
|
||||
const safe = sanitizeRelPath(relPath);
|
||||
@@ -450,6 +510,27 @@ export class VolumeBrowserService {
|
||||
}
|
||||
}
|
||||
|
||||
/** Copy a file or directory within the same volume (symlink-as-link, no chown). */
|
||||
async copy(volumeName: string, fromRel: string, toRel: string): Promise<void> {
|
||||
const from = sanitizeRelPath(fromRel);
|
||||
const to = sanitizeRelPath(toRel);
|
||||
if (!from || !to) throw new ExecError('Invalid copy path', 400);
|
||||
await this.assertVolumeExists(volumeName);
|
||||
await this.ensureHelperImage();
|
||||
const { stderr, exitCode } = await this.runHelper(
|
||||
volumeName,
|
||||
['sh', '-c', COPY_SCRIPT, 'sh', `./${from}`, `./${to}`],
|
||||
{ writable: true },
|
||||
);
|
||||
if (exitCode !== 0) {
|
||||
const msg = stderr.toString('utf-8').trim();
|
||||
if (/Permission denied/i.test(msg)) throw new ExecError('Permission denied', 403);
|
||||
if (exitCode === 11 || /exists/i.test(msg)) throw new ExecError('A file or folder with that name already exists', 409);
|
||||
if (exitCode === 12) throw new ExecError('Cannot copy a folder into itself', 400);
|
||||
throw new ExecError(`Copy failed: ${msg.substring(0, 200) || 'unknown error'}`);
|
||||
}
|
||||
}
|
||||
|
||||
// --- internals -----------------------------------------------------------
|
||||
|
||||
private async assertVolumeExists(volumeName: string): Promise<void> {
|
||||
@@ -493,7 +574,7 @@ export class VolumeBrowserService {
|
||||
private async runHelper(
|
||||
volumeName: string,
|
||||
cmd: string[],
|
||||
opts: { writable?: boolean; stdin?: Buffer } = {},
|
||||
opts: { writable?: boolean; stdin?: Buffer | Readable } = {},
|
||||
): Promise<{ stdout: Buffer; stderr: Buffer; exitCode: number }> {
|
||||
const docker = DockerController.getInstance(this.nodeId).getDocker();
|
||||
const stdoutChunks: Buffer[] = [];
|
||||
@@ -540,6 +621,7 @@ export class VolumeBrowserService {
|
||||
timer = setTimeout(() => reject(new ExecError('Helper exec timed out', 504)), EXEC_TIMEOUT_MS);
|
||||
});
|
||||
|
||||
let stdinError: unknown = null;
|
||||
const runPromise = (async () => {
|
||||
const stream = await container.attach({ stream: true, stdin: wantStdin, stdout: true, stderr: true, hijack: wantStdin });
|
||||
const streamEnded = new Promise<void>((resolve) => {
|
||||
@@ -551,16 +633,45 @@ export class VolumeBrowserService {
|
||||
if (wantStdin && opts.stdin) {
|
||||
// Feed the file content to the container's stdin, then close it so the
|
||||
// helper's `cat > file` sees EOF and exits.
|
||||
stream.write(opts.stdin);
|
||||
stream.end();
|
||||
if (Buffer.isBuffer(opts.stdin)) {
|
||||
stream.write(opts.stdin);
|
||||
stream.end();
|
||||
} else {
|
||||
// A Readable (an upload spooled to a temp file) is piped so a large
|
||||
// upload is never held in memory. pipeline tears the stdin side down
|
||||
// (destroys it) on a source error rather than ending it cleanly, so
|
||||
// cat sees an abnormal close instead of a normal EOF and does not
|
||||
// commit a truncated file as a success. Remember the error so the run
|
||||
// is reported as failed after the container exits.
|
||||
try {
|
||||
await pipeline(opts.stdin, stream);
|
||||
} catch (err) {
|
||||
stdinError = err;
|
||||
}
|
||||
}
|
||||
}
|
||||
const exitInfo = await container.wait();
|
||||
// Wait for the attach stream to finish flushing demuxed output.
|
||||
await streamEnded;
|
||||
const exitCode = typeof exitInfo?.StatusCode === 'number' ? exitInfo.StatusCode : 0;
|
||||
// A nonzero helper exit is the authoritative failure: its exit code carries
|
||||
// the intended 4xx mapping (target-is-a-directory, permission denied, etc.),
|
||||
// which a stdin EPIPE from the helper closing stdin early would otherwise
|
||||
// mask as a generic 500. Only surface the stream error when the helper
|
||||
// itself exited cleanly (or its status was unreadable).
|
||||
if (stdinError) {
|
||||
if (exitCode === 0) {
|
||||
throw new ExecError('Upload stream failed before the volume write completed', 500);
|
||||
}
|
||||
// The nonzero exit's mapping is the user-facing error, but the stream
|
||||
// failure is the root cause; log it so an intermittent upload failure is
|
||||
// debuggable rather than being silently attributed to the exit code.
|
||||
console.error('[VolumeBrowser] helper stdin stream error masked by nonzero exit', exitCode, stdinError);
|
||||
}
|
||||
return {
|
||||
stdout: Buffer.concat(stdoutChunks),
|
||||
stderr: Buffer.concat(stderrChunks),
|
||||
exitCode: typeof exitInfo?.StatusCode === 'number' ? exitInfo.StatusCode : 0,
|
||||
exitCode,
|
||||
};
|
||||
})();
|
||||
|
||||
|
||||
@@ -0,0 +1,54 @@
|
||||
/**
|
||||
* Normalization for bulk file-operation selections (delete / move / download).
|
||||
*
|
||||
* A client (or a direct API caller) can submit overlapping paths, e.g. both a
|
||||
* directory and a file inside it, or the same path twice. Acting on both would
|
||||
* double-process a file, duplicate an archive entry, or report spurious
|
||||
* per-item failures, so the route normalizes the selection before acting on it:
|
||||
* it dedupes and drops any path whose ancestor is also selected.
|
||||
*
|
||||
* Case-awareness is per root, not global. Filesystem roots on a case-insensitive
|
||||
* host (Windows/macOS) fold case so `Foo` and `foo` collapse; Linux filesystem
|
||||
* roots and helper-backed (named-volume) roots are case-sensitive, so `Foo` and
|
||||
* `foo` are distinct and both survive. The caller passes `caseSensitive` derived
|
||||
* from the root.
|
||||
*/
|
||||
export function normalizeBulkPaths(paths: string[], caseSensitive: boolean): string[] {
|
||||
const key = (p: string): string => (caseSensitive ? p : p.toLowerCase());
|
||||
|
||||
// Dedupe by key, keeping the first spelling seen.
|
||||
const seen = new Set<string>();
|
||||
const unique: string[] = [];
|
||||
for (const p of paths) {
|
||||
const k = key(p);
|
||||
if (!seen.has(k)) {
|
||||
seen.add(k);
|
||||
unique.push(p);
|
||||
}
|
||||
}
|
||||
|
||||
// Drop any path that has a selected ancestor. The check is key-based so a
|
||||
// case-insensitive root treats `Foo` as the ancestor of `foo/bar`.
|
||||
const keys = new Set(unique.map(key));
|
||||
return unique.filter((p) => {
|
||||
const segments = key(p).split('/');
|
||||
for (let i = 1; i < segments.length; i++) {
|
||||
if (keys.has(segments.slice(0, i).join('/'))) return false;
|
||||
}
|
||||
return true;
|
||||
});
|
||||
}
|
||||
|
||||
/**
|
||||
* True when `dir` (a normalized rel path, '' = root) is equal to or inside any
|
||||
* of the `sources` directories. Used to reject a bulk move whose destination is
|
||||
* one of the moved folders or a descendant of one.
|
||||
*/
|
||||
export function destWithinAnySource(dir: string, sources: string[], caseSensitive: boolean): boolean {
|
||||
const key = (p: string): string => (caseSensitive ? p : p.toLowerCase());
|
||||
const dirKey = key(dir);
|
||||
return sources.some((s) => {
|
||||
const sourceKey = key(s);
|
||||
return dirKey === sourceKey || dirKey.startsWith(`${sourceKey}/`);
|
||||
});
|
||||
}
|
||||
Reference in New Issue
Block a user