mirror of
https://github.com/projectsend/projectsend.git
synced 2026-09-17 00:55:07 +00:00
v2.3.0
53 Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 ignacionelson
|
97596da7d0 |
Refuse a stored path carrying a control character
Found reviewing the delivery work. The path is written into
X-Accel-Redirect or X-Sendfile, and a CR or LF in a header value is
header injection. PHP's header() refuses to emit one, so the real effect
is a 500 on every download, preview and thumbnail of that file rather
than a split response -- a file permanently broken by its own name.
Paths are generated here as Y/m/{uuid}.{ext}, so this should be
unreachable. The extension is not generated: it comes from the
uploader's filename, and on a migrated installation from a v1 database.
The upload routes all check the extension against an allowlist, which no
control character can match -- but upload_type_restriction can be set to
none, and the importer does not consult that policy at all.
assertRelative() was documented as the backstop for what a path may be
and only covered traversal, which is the half that cannot happen here.
Low severity, and the guard should have covered it either way.
|
||
|
ignacionelson
|
d6fd5a917d |
Send downloads the way the web server in front of us understands
Uploads live outside the web root, so PHP authorizes every download and then hands the file to the web server with a header naming it. Four routes decided that for themselves and all four hard-coded nginx's spelling. On Apache or LiteSpeed nothing acts on the header, so the empty body PHP sent goes to the visitor: files upload fine, thumbnails are broken images, and downloads arrive as 0 bytes, with every other page working. Reported as #1765 from an Apache 2.4 install, and before that as #1266, #1215, #870 and #1271. It is also a regression from v1, which had a download_method setting -- php, apache_xsendfile, litespeed, nginx_xaccel -- defaulting to php. v1 therefore worked on any server out of the box and v2 did not, and a v1 Apache user migrating lost every download with nothing to tell them why. So the four sites now go through one FileDelivery, and it picks: auto (default) nginx when SERVER_SOFTWARE says nginx, else php nginx X-Accel-Redirect, a URL path via the internal location xsendfile X-Sendfile, an absolute path (Apache mod_xsendfile, LiteSpeed) php BinaryFileResponse Defaulting to auto rather than nginx is the point of the change: a default that assumes nginx leaves an Apache install exactly as broken as it is today until somebody reads INSTALL.md. Slow beats empty. Auto never picks xsendfile, even where the module is loaded. mod_xsendfile also needs XSendFilePath to allow the storage directory, which cannot be seen from here, and choosing it on the strength of the module being present would trade a silent failure an administrator can diagnose from the dashboard for one nobody can. BinaryFileResponse rather than a readfile loop because it answers Range requests. nginx does that itself on the fast path, so hand-rolling it would have broken seeking through a video on exactly the installations this fallback exists for. Verified end to end: 206 with the right Content-Range through the live stack. Two guards. Every method checks the path cannot climb out of the storage area -- nginx resolves `..` in the URL it is handed as happily as PHP would -- and the two methods that hand over a filesystem path resolve it and prove it lands inside the root. Callers pass paths from rows they just authorized, so this is a backstop; it is here because the cost of being wrong once is handing over any file the web server can read. The dashboard's System panel names the method, with a warning icon and a dialog when PHP is doing the sending: what is happening, what it costs (one worker held for the whole of each download, so a few large simultaneous ones can occupy every worker while the processor sits idle), why it is set that way, and the three ways out. Written to be accurate rather than reassuring -- nothing is broken, it does not scale -- and the notice stays even when php was chosen deliberately, because the trade-off is the same either way. /system/settings/downloads repeats it, which is where somebody coming from v1 goes looking for the dropdown. An environment variable rather than a stored setting: it describes the server this installation runs on, not a preference, and a value in the database travels to a different server in a restore and is wrong there. Read only in config/projectsend.php, so config:cache cannot blank it. The suite pins itself to nginx. Left at auto it would detect no server at all, fall back to php, and quietly retire the coverage of the mechanism most installations actually use. |
||
|
ignacionelson
|
144f5fc578 |
Merge pull request #1747 from denkfabrik-li/fix/bulk-edit-skip-reason
Two different things stop a selected file being changed in a bulk edit, and bulkUpdate() reported both as the first one. Files dropped by the Gate::allows('update') filter are ones this staff member may not edit at all. A file that survives the filter and still changes nothing is a different case: it was editable, and every field they asked to change is one their role does not let them set -- expiry, download limit and categories each sit behind their own permission here, exactly as they do in the single-file editor. So a staff member with edit_files but without set_file_expiration_date, editing three files they own, was told "0 of 3 selected files were updated. The rest were skipped because you don't have permission to edit them." They own all three, and editing is precisely what they may do: the sentence was both wrong and unactionable, since nothing in it points at the permission that actually stopped the edit.
The two cases get their own sentences now. Every skip being a file they may not edit keeps the existing string, unchanged, so its sixteen translations stay in use. Anything else gets a new one, "because you don't have permission to make those changes", which is also true when both reasons are in play, so a mixed selection is described correctly rather than approximately. Which files get changed is untouched, as is the silent-skip convention and the 422 when nothing at all is authorised.
Verified before merging: 14 passed on the trial-merge, 2 failed / 12 passed with app/ reset -- the field-permission case and the mixture. The pure edit-permission case is green either way, which is what says the existing message was not disturbed. FilesController overlaps #1728, already merged, and its expiryDateFor work is intact in the merged tree.
The new string arrived English-only; the sixteen catalogs are filled in the commit that follows.
Reported and fixed by @denkfabrik-li.
|
||
|
ignacionelson
|
383c3b2ff5 |
Merge pull request #1746 from denkfabrik-li/fix/expired-file-staff-access-comment
File::isExpired() documented the rule the whole application is supposed to follow: once past, the file is hidden from clients and the public site but staff keep full access. The second half is not true of a client-scoped staff member. StaffLibraryScope::buildFiles() builds their library as own uploads plus what each assigned client may see, and that second half runs through File::scopeVisibleToClient, which ends in notExpired() -- a client-side rule. So an expired file they held only through a client leaves their library and answers 403 on download, while their own expired upload stays and an unscoped administrator is unaffected. Api\FilesController stated it the same way, "Only the client branch of the visibility rules drops them", which reads as though a staff caller is unaffected when a client-scoped one is reached through that very branch.
This does not change that behaviour.
|
||
|
ignacionelson
|
89b3d34c8f |
Merge pull request #1744 from denkfabrik-li/fix/version-link-duplicate-share-notice
FileVersions::link() resolves its audience before the merge, and its own comment says the ordering is the whole dedupe: these are the people who could already see both files, so anyone the merge is about to reach for the first time is excluded and gets file_shared from FileSharing::assign() instead. The merge then undid it. moveAssignmentsToRoot() handed every one of the revision's targets to assign() under the comment "firstOrCreate inside, so a target the root already has is a no-op rather than a duplicate notification" -- but firstOrCreate makes the assignment row idempotent, not the three side effects below it. The activity entry, the in-app notification and the digest all ran unconditionally, so a client who already held both files was told a file had been shared with them about a file they had had all along, on top of the file_new_version they were owed. Two notifications for one action, for exactly the people the early resolve exists to protect. A target the root already holds is now skipped rather than handed to assign(). Nobody is gaining access in that case, so the activity entry would have been as untrue as the notification -- which is the rule copyAssignmentsFrom() states outright for its own case, and why it inserts directly instead of going through FileSharing. The two stale comments are corrected with it. Deliberately not changed: assign() itself, and so the behaviour ShareNotificationsTest pins, where re-posting an existing assignment through the share endpoint still notifies again. That test says the condition for changing it -- it should stop for files and folders at once, which is the point of them sharing one implementation -- and a version merge is not somebody choosing to share again. Verified before merging: 10 passed on the trial-merge, 2 failed / 8 passed with app/ reset, and the whole tests/Feature/Files directory at 548 passed. The case where somebody genuinely gains the root still gets file_shared is green either way, which guards against skipping too much. The method was read whole rather than just the hunk: $file->assignments()->delete() still runs for a skipped target, so no row is left dangling and nobody loses reach. Reported and fixed by @denkfabrik-li. |
||
|
ignacionelson
|
b7a94d4479 |
Merge pull request #1742 from denkfabrik-li/fix/file-permissions-test-reads-config
FILES_WEB_SERVER_READABLE exists so a web server running as a different user can traverse the directories a download lives in. It asked for 0755 from a key that is never consulted: FilesystemManager::createLocalDriver() passes directory_visibility ?? visibility ?? private as the default visibility for directories, and this disk sets visibility to public two lines above with no directory_visibility, so Flysystem reads dir.public and never looks at dir.private. The mode came out 0755 anyway, because 0755 is Flysystem's default for a public directory -- the right answer from the wrong place, which is the kind that stops being right quietly. Adding a directory_visibility to this disk, an ordinary hardening move, or a change to that Flysystem default would have been enough to break the flag silently on exactly the hosts that need it. Both directory keys are now named, so the intent survives whichever branch Flysystem takes. Nothing widens: the flag-off path is still literally the old configuration, spread rather than ternary, and under the flag 0755 was already the effective mode. And the test could not have caught it, because it was not testing this configuration: filesDiskWith() restated the shipped branch inline, verbatim down to the 0755, so it kept passing against its own copy however the real one changed. It now requires config/filesystems.php and replaces only the root. Two housekeeping fixes ride along: the scratch root is per parallel worker, the way Tests\TestCase already does it for upload parts, because eight workers sharing one real directory means one worker's afterEach deletes another's tree mid-test; and the tree is cleared before each test as well as after, so a killed run does not poison the next one. Verified before merging: 3 passed on the trial-merge, and the mutation counter-check was run here rather than taken from the PR. With the shipped dir.public changed to 0750, this branch's test goes 1 failed / 2 passed and main's version of the same file goes 3 passed -- the old one genuinely could not see a change to the shipped configuration. Reported and fixed by @denkfabrik-li. |
||
|
ignacionelson
|
b6f4770795 |
Merge pull request #1731 from denkfabrik-li/fix/zip-build-failure-hygiene
BuildZipDownloadJob already draws this line in its write-failure branch: "What the requester sees stays generic: a libzip string means nothing to them and can name a server path. An operator needs the opposite, so the reason goes to the log instead." Thirty-seven lines below it, the catch-all around the whole build stored $e->getMessage() in the row the requester polls -- and ZipDownloadsController hands that column straight back to whoever asked, clients included. A client asking for an archive of a file whose disk is no longer configured read "Disk [a-disk-that-is-not-configured] does not have a configured driver." verbatim. The reason now goes to the log with the exception class, and the row carries the same kind of sentence fail() already uses. Two more in the same method. tempnam() creates the file, and $tempFiles[] was appended only after the copy finished, so every throw in between left a zip-src- file in the system temp directory that nothing ever removed; it is now registered the moment it exists. And the copy itself was unchecked -- a copy that stops early is a truncated member added to the archive as though it were the file, so the build reports ready and the recipient gets something that opens and is wrong. stream_copy_to_stream and the flushing fclose are both checked now, and both handles close on every path. Deliberately not changed: comparing the copied byte count against files.size, which would fail perfectly good archives whenever that column is stale; the write-failure branch and its wording; and the skipped-files reporting, which still says which files and why, so only the catch-all went generic. Verified before merging: 37 passed on the trial-merge, 2 failed / 35 passed with app/ reset. The leak was confirmed at the consuming end rather than inferred -- ZipDownloadsController:169 returns the error column to the requester. Reported and fixed by @denkfabrik-li. |
||
|
ignacionelson
|
d89807b237 |
Merge pull request #1726 from denkfabrik-li/fix/rendition-cleanup-independent
FileDiskCleanup::delete() wrapped two deletions in one try: the original upload, on whatever disk the row names, and every cached rendition, which is always on the local files disk. Storage::disk() throws outright for a name with no configured driver -- precisely the state the original's disk is in whenever this fails at all -- so the catch swallowed it and the renditions were never reached. Nothing looks for them afterwards: OrphanFileScanner skips the rendition directories on purpose, as derived artifacts rather than orphaned uploads. A file whose external disk had been removed or renamed therefore kept every cached copy of itself indefinitely on the disk that still worked, including the client-facing ones, which for a shared image may be the only copies anyone ever generated. The two attempts are now separate, each with the tolerance the class was written for: a storage failure still never turns a delete click into a 500, and the warning is still the whole report. Also corrected: File::booted() justified deferring the byte removal with "the worst case is bytes left on disk with no row, which OrphanFileScanner already finds and reports". That is not this path -- the row is soft-deleted, and knownPaths() counts a trashed row's path as claimed, deliberately, so a scan never offers to double-adopt a file still inside its erasure grace period. The comment now says what actually happens, which is that FileDiskCleanup's warning is the only record. Verified before merging: 8 passed on the trial-merge, 1 failed / 7 passed with app/ reset. Reported and fixed by @denkfabrik-li. |
||
|
denkfabrik-li
|
5e60d2ef88 |
Say what expiry does to a client-scoped staff member's library
File::isExpired() documents the rule the application is supposed to
follow: once past, the file is hidden from clients and the public site
"but staff keep full access to view, download, and manage it".
The second half is not true of a client-scoped staff member.
StaffLibraryScope::buildFiles() builds their library as their own uploads
union what each assigned client may see, and that second half runs
through File::scopeVisibleToClient, which ends in notExpired() -- a
client-side rule. Measured on main, with a rep holding one client and a
file the administrator uploaded and shared with that client:
before expiry in_library true GET .../download -> 200
after expiry in_library false GET .../download -> 403
the rep's own expired upload in_library true
an unscoped administrator, same expired file in_library true
Api\FilesController says it the same way -- "Only the client branch of
the visibility rules drops them" -- which reads as though a staff caller
is unaffected, when a client-scoped one is reached through that very
branch.
This does not change that behaviour.
|
||
|
denkfabrik-li
|
21cae2acb1 |
Stop a version link telling people about a file they already had
FileVersions::link() resolves its notification audience before the merge,
and says why:
RESOLVED BEFORE THE MERGE, and the ordering is the whole dedupe:
these are the people who could already see both files, so anyone the
merge below is about to reach for the first time is excluded here and
gets file_shared from FileSharing::assign() instead. Resolve it
afterwards and every newly-added client receives two emails about one
action.
The merge then undoes it. moveAssignmentsToRoot() hands every one of the
revision's targets to FileSharing::assign(), under a comment claiming
that firstOrCreate makes a target the root already has a no-op. It makes
the assignment row idempotent; the three side effects under it --
activity entry, in-app notification, digest -- run unconditionally.
Measured on main:
client already holds the root and the revision, then both are linked
file_shared (Report) <- wrong, they have had it all along
file_new_version (Report v2) <- right
assignment rows on the root: 1
client holds only the revision, then both are linked
file_shared (Report) <- right, the merge does hand it over
Two notifications for one action, for exactly the people the early
resolve was meant to protect.
So a target the root already holds is skipped rather than handed to
assign(). Nobody is gaining access in that case, and the activity entry
would be as untrue as the notification. copyAssignmentsFrom() directly
below already states that rule for its own case, which is why it inserts
directly instead of going through FileSharing. Both stale comments are
corrected with it.
Not changed: FileSharing::assign() itself, and so the behaviour
ShareNotificationsTest pins -- re-posting an existing assignment through
the share endpoint still notifies again. That test names the condition
for ever changing it, "it should stop being sent for both at once", and
that is a decision about files and folders together. This is narrower: a
version merge is not somebody choosing to share again, and it already
had a stated intent to send exactly one notification.
Three cases in ShareNotificationsTest -- the target already on the root,
the target gaining it, and a group already on the root. Reverting
FileVersions alone leaves 2 failed / 8 passed in that file; the middle
case passes without the fix, because it guards against skipping too much
rather than against the duplicate notice.
Suite 2108 passed / 2 skipped, 11415 assertions, PHPStan level 8 clean.
Measured on base
|
||
|
denkfabrik-li
|
b838036a9a |
Set the directory permission Flysystem actually reads
FILES_WEB_SERVER_READABLE asks for 0755 on the directories a download has to be traversed through, and asks for it from a key that is never consulted. FilesystemManager::createLocalDriver passes `directory_visibility ?? visibility ?? private` to PortableVisibilityConverter::fromArray() as the default visibility for directories. This disk sets `visibility` to public two lines above, and no `directory_visibility`, so directories are public and the converter reads `dir.public`. The configuration names only `dir.private`. The mode is 0755 regardless, because 0755 is Flysystem's default for a public directory -- the right answer from the wrong place. Adding `directory_visibility` to this disk, or a change to that default, is all it would take for the flag to stop doing what it says. Measured on main, with the flag on: dir.private 0755 → 0750 directory stays 0755 (nothing reads it) dir.public 0755 → 0750 directory becomes 0750 (this is the key) Both are named now, so the intent survives either way round. FilePermissionsTest could not have caught this, because it was not testing this configuration. filesDiskWith() restated the shipped branch inline, verbatim down to the 0755, so it went on passing against its own copy however the real one changed. It now requires config/filesystems.php and replaces only the root, which is what makes the mutation above visible to it. Two more things in the same helper, both about the suite rather than the subject: the scratch root is per worker now (Tests\TestCase does the same for upload parts, and eight workers sharing one directory means one worker's afterEach deletes another's tree mid-test), and it is cleared before each test as well as after, so a killed run does not poison the next one. |
||
|
denkfabrik-li
|
4b998cda92 |
Fail a zip build without handing the requester the server's reason
The write-failure branch already draws the line and says why: "What the requester sees stays generic: a libzip string means nothing to them and can name a server path. An operator needs the opposite ... so the reason goes to the log instead." Thirty-seven lines below it, the catch-all around the whole build stored $e->getMessage() in the row the requester polls. Measured, a client asking for an archive of a file on a disk that is no longer configured was told: "Disk [a-disk-that-is-not-configured] does not have a configured driver." The reason now goes to the log with the exception class, and the row carries the same kind of sentence fail() already uses. Second, the temp files. tempnam() creates the file, and $tempFiles[] was appended only after the copy had finished -- so every throw in between (a disk that will not resolve, a stream that will not open) left a zip-src- file in the system temp directory that nothing ever removes. It is now registered the moment it exists. Third, in the same method: the copy itself was unchecked. A copy that stops early is a truncated member added to the archive as though it were the file, so the build reports ready and the recipient gets something that opens and is wrong. Both the copy and the fclose that flushes it are checked now, and both handles close on every path. Two tests: the failure message names nothing about the server, and a build that throws mid-copy leaves no temp file behind. Both go red without the fix. |
||
|
denkfabrik-li
|
4164678ebc |
Delete a file's renditions even when its own disk cannot be resolved
FileDiskCleanup wraps both deletions in one try. The first is the original upload, on whatever disk the row names; the second is every cached rendition, always on the local files disk. Storage::disk() throws outright for a name with no configured driver -- which is the state the original's disk is in whenever this fails at all -- so the catch swallowed it and the renditions were never reached. Nothing looks for them afterwards. OrphanFileScanner skips the rendition directories on purpose (they are derived artifacts, never orphaned uploads), so a file whose external disk had been removed or renamed kept every cached copy of itself, indefinitely, on the disk that was working. The two attempts are now separate, each with the same tolerance the class was written for: a storage failure still never turns a delete click into a 500, and the warning is still the report. While here, the comment in File::booted() that justifies deferring the byte removal claimed "the worst case is bytes left on disk with no row, which OrphanFileScanner already finds and reports". Not on this path: the row is soft-deleted, and knownPaths() counts a trashed row's path as claimed -- deliberately, so a scan never offers to double-adopt a file still inside its erasure grace period. The comment now says what actually happens. One test: a file whose disk cannot be resolved loses its renditions. It goes red without the fix, next to the existing test that the delete itself still succeeds. |
||
|
denkfabrik-li
|
fc758c701a |
Write a rendition through a temporary file, and never serve an empty one
Both thumbnail routes treat "the file exists" as "the rendition is cached", and nothing ever invalidates one: RenderedImageCache::flush() runs on ImageRenderingChanged, which no core code raises. Whatever is at the path is what every later viewer gets. ThumbnailGenerator encoded straight onto that path. A render that died partway -- a full volume, a killed worker -- left a half-written file that was then served as the rendition for good, and two requests rendering the same file at once encoded into one path together. It now writes beside the destination and renames into place. rename() within a directory is atomic and replaces what is there, so the path is either the previous rendition or a complete new one, and the loser of a race leaves a whole image rather than a mixture of two. The temporary file is removed on the way out either way. The read side gets the other half: an empty file is not a rendition, so both routes replace one rather than serve it. Writing through a temporary file means this state can no longer be created here, but an installation that ran an older version can already have it on disk, and nothing else will ever clear it. Three tests: an empty rendition is replaced on the signed-in route and on the public one, and a successful render leaves nothing half-written behind. Without the fix the first two go red; the third is about the fix's own temporary file and passes either way. |
||
|
denkfabrik-li
|
763777d282 |
Say which permission a bulk edit was actually missing
Two different things stop a selected file being changed, and bulkUpdate()
reported both as the first one.
Files dropped by the Gate::allows('update') filter are ones the staff
member may not edit at all. A file that survives the filter and still
changes nothing is a different case: it was editable, and every field they
asked to change was one their role does not let them set -- expiry,
download limit, categories, each behind its own permission, exactly as the
single-file editor treats them.
Measured with edit_files but without set_file_expiration_date, three files
they own, expiry the only change: "0 of 3 selected files were updated. The
rest were skipped because you don't have permission to edit them." They
own all three and editing is precisely what they may do, so the sentence is
both wrong and unactionable.
The two cases now have their own sentences. The existing string is kept
for the case it describes -- every skip a file they may not edit -- so its
sixteen translations stay in use. The new one covers a field permission,
and covers a mixture of both reasons, since "permission to make those
changes" is true either way.
The new key is English only; a locale without it falls back to English,
which is a translated-but-wrong sentence traded for an untranslated
correct one.
Three tests: each reason on its own, and the mixture. Without the fix the
first and third go red.
|
||
|
Ignacio Nelson
|
046be36861 |
Merge pull request #1710 from denkfabrik-li/fix/folder-delete-file-authority
FoldersController::destroy() authorized delete on the folder and nothing else, while FolderService::delete() soft-deletes every file in the subtree and File's deleted hook takes the bytes off disk. So a staff member refused a file one route over could destroy it by deleting the folder around it -- permission and library boundary both unasked. MyFoldersController::destroy() already draws this line for the client half of the same cascade, and says why: owning the folder is not authority over content someone else put in it. This is the staff half of that sentence. Verified before merging: the four bug tests fail on main and pass here, and the SQL predicate was read line by line against FilePolicy::delete -- it is a faithful negation, including the null-uploader case and the short-circuit for an unscoped viewer holding both delete permissions. Membership of the check is one COUNT, not a policy call per file. Suite at 2099, PHPStan clean. Behaviour change, deliberately accepted: a folder delete that used to succeed now refuses, naming how many files are in the way. The likely case is somebody who owns a folder another account uploaded into. The alternative is irreversible loss of files the same person is refused individually. Not taken: deleting what the actor may and keeping the rest. Half a tree is worse than either answer. Naming the blocking files would be friendlier than counting them and is worth doing later -- the list has to hide any file the viewer cannot see, which is its own small design question. Reported and fixed by @denkfabrik-li. |
||
|
denkfabrik-li
|
b44c6bf098 |
Add a file to a zip once, however many ways the selection reaches it
BuildZipDownloadJob walks the loose file ids and then every selected
folder's subtree, and adds whatever each pass finds. A selection can
reach the same file from more than one of them, and nothing noticed:
file_ids [f], folder_ids [Reports]
-> ['report.pdf', 'Reports/report.pdf']
file_ids [f], folder_ids [Reports, Reports/Q1]
-> three entries, file_count 3, total_size three times the file
Two copies of the same bytes in one archive, and total_size is what the
size cap is checked against, so a selection could also be refused for a
weight it does not have.
The one that costs more than bandwidth is delivery. It logs one
FileDownloaded per contained file, and DownloadAllowance counts those
records -- so a file limited to a single download left in three copies
while the log recorded one. Measured: three entries, one record.
Two causes, so two halves.
`$added` is now keyed by id instead of being appended to a list, and the
folder pass skips a file already in the archive. A lookup rather than a
scan because the selection cap is 10000 sources. The loose pass runs
first, so a file picked both ways sits under its loose name; either
answer is defensible, but it has to be the same one every run.
And a selected folder inside another selected folder is dropped before
either is walked. Zipping both would reach every file in the inner one
twice, and which path the surviving entry ended up under would be decided
by the order the rows came back in. Keeping the outer folder keeps the
fuller path -- Reports/Q1/report.pdf rather than Q1/report.pdf.
Containment is decided on the materialized path, so it is one comparison
per pair with no queries: a folder's path starts with an ancestor's
subtreePathPrefix(), and both end in '/', so /5/ cannot match /50/.
Not changed: the per-file re-checks inside the folder pass. Visibility
and the download allowance are still re-derived per file, and the skip
happens before them, so a duplicate never spends an allowance twice
either. Nor the selection endpoint -- a caller may send whatever
selection they like, and the job is where it is resolved.
Four tests. Three measured red against the unfixed job (3 failed / 32
passed): the loose-plus-folder case, the nested-folder case, and the
three-way case asserted through delivery rather than through the archive.
The fourth -- two selected folders that merely share a name are both
zipped -- is green either way and guards the pruning against being about
names rather than containment.
Full suite passes (2052 passed / 2 skipped), PHPStan level 8 clean.
|
||
|
denkfabrik-li
|
26205082c2 |
Stop a folder deleting the files inside it that its owner may not delete
FoldersController::destroy() authorizes `delete` on the folder and nothing
else. FolderService::delete() then soft-deletes every file in the subtree,
and File::booted()'s `deleted` hook takes the bytes off disk. There is no
restore.
FilePolicy::delete asks two questions the folder route never reaches:
`delete_others_files` for somebody else's upload, and
StaffLibraryScope::allowsFile on top of it. Measured with a role holding
create_own_folders, delete_files, upload and edit_files -- the shape the
Client Manager system role already has, minus delete_others_files:
DELETE /files/{someone-elses} 403, the file is still there
DELETE /folders/{their-folder} 302, the file and its bytes are gone
MyFoldersController::destroy already refuses the client half of this exact
cascade, and says why: "Owning the folder is not authority over content
someone else put in it... Refuse rather than silently destroy them." This
is the staff half of the same sentence.
Counted rather than asked per file. A folder can hold thousands, Gate
resolves a fresh policy for every check, and a per-row policy check on a
listing is the cost
|
||
|
ignacionelson
|
5d99ab94fd |
Say on screen when nothing is building zip downloads
Zip building moved onto its own queue, which a manual install's worker has to be told about. update.sh repairs the service file and Docker is unaffected, so the population left is somebody upgrading by hand who skipped the release note — and for them the failure is the worst shape available. Email keeps going out perfectly. Zip downloads never finish. Nothing in any log says why, because nothing went wrong: the jobs sit on a queue nobody is reading. The person who missed it has no reason to suspect anything, so the notice has to go looking for them. The application cannot see its own worker processes, only whether work gets done, so the question is asked from the other end: was a build requested that no worker ever picked up? That needs a record of when a build *started*, which is what the new zip_downloads.started_at column is — stamped before any of the work, so it says a worker had the row, not that the row succeeded. Two conditions, because either alone cries wolf. A build has waited past five minutes and was never started, *and* no other build is in hand. The second matters because one worker builds one archive at a time: a queue behind a large build is a healthy queue, and its waiting rows look exactly like abandoned ones until you notice something running. "In hand" is bounded by the job's own timeout, so a worker that died holding a build stops counting as alive an hour later. The banner sits beside the stale-code one, on every staff page rather than the dashboard alone, gated on view_system_info for the reason that one already argues: a background worker not picking work up is a fact about the machine, not a feature of an edition. It names the fix rather than the symptom — "your worker command needs --queue=default,zips" — because somebody reading that downloads are not being processed still has to work out what to do about it. Eight tests, covering both halves of the discrimination rather than just the happy one: a queue waiting behind a live build stays quiet, and a build held by a worker that died does not. Translated into all sixteen locales in the same commit, since a release is close and a banner nobody can read is worse than none. Checked on screen as well as in assertions, with a real stalled row on the dev stack: the banner renders, wraps, and reads correctly. |
||
|
Ignacio Nelson
|
f12692520a |
Merge pull request #1694 from denkfabrik-li/fix/upload-folder-library-scope
Hold the folder an upload names to the same library boundary as everything else |
||
|
Ignacio Nelson
|
e4cd56f5d6 |
Merge pull request #1692 from denkfabrik-li/fix/zip-download-limit-at-delivery
Enforce the download limit when a zip is delivered |
||
|
Ignacio Nelson
|
9b3f7023d0 |
Merge pull request #1691 from denkfabrik-li/fix/file-bytes-after-commit
Delete a file's bytes when its transaction commits, not before |
||
|
Ignacio Nelson
|
09efad2d8c |
Merge pull request #1690 from denkfabrik-li/fix/client-portal-subfolder-names
Don't name a subfolder to a client who cannot open it |
||
|
ignacionelson
|
835943e1b6 |
Merge pull request #1686 from denkfabrik-li/fix/chunked-upload-complete-lock
Finalise each chunked upload once, under a per-session lock
Resolved a trivial conflict in ChunkedUploadsTest: this branch and
|
||
|
Ignacio Nelson
|
d36abd73ba |
Merge pull request #1682 from denkfabrik-li/fix/chunked-upload-max-size
Enforce the max file size against the bytes a chunked upload assembles |
||
|
Ignacio Nelson
|
e815ac8be5 |
Merge pull request #1681 from denkfabrik-li/fix/file-update-folder-scope
Scope a file's destination folder on update(), as move() already does |
||
|
ignacionelson
|
92a132d74f |
Give zip builds their own queue, so one archive cannot hold up the mail
The last piece of the #1687 follow-up. BuildZipDownloadJob allows itself an hour, every shipped topology runs exactly one worker, and everything shares the default queue -- so one large archive delayed every notification email queued behind it. The size cap and the one-build-per-person rule bounded that in July; they did not remove it. onQueue('zips') in the constructor rather than at the dispatch site, so a second caller cannot forget it. Both images grow a worker for it: compose.yaml gains worker-zips, supervisord gains [program:queue-zips], and the existing worker in each narrows to --queue=default. --tries=1 there matches the job, which records its own failure rather than being retried. The part that needs care is the manual install. A worker whose command still says plain `queue:work` consumes `default` only, so it would send email happily and never finish a single zip, with nothing in any log saying why. INSTALL.md's unit now reads --queue=default,zips -- one worker watching both, which is right for most installations -- and says what happens if you leave it off, with the two-worker split offered for anyone who would rather keep the two kinds of work apart. CHANGELOG carries it as an upgrade note, since it is something to do rather than something that was done. Verified in the dev stack rather than only in a test: dispatched a build and watched worker-zips take it while the default worker stayed idle. |
||
|
ignacionelson
|
41b4e477b5 |
Give each parallel test worker its own directory for upload parts
A full parallel run failed once and passed on retry while I was doing the #1703 follow-up. A flake is worse than a steady failure: it trains you to re-run rather than look, and it quietly weakens every green run reported beside it. Upload parts are real files under storage_path('app/uploads-tmp/{session_id}'), not a faked disk. Every parallel worker gets its own database, so session ids restart at 1 in each of them, and two workers writing parts land in the same directory. On top of that ChunkedUploadsTest's afterEach deleted the whole tree rather than its own share, for everybody. Six test files write parts, so this was reachable without anything I added. The same collision exists inside one worker: RefreshDatabase rolls back, so ids restart at 1 for every test, and a run that died before its cleanup leaves parts sitting under the id the next test is about to claim. LocalPartStore now reads its root from config, defaulting to exactly where it always was -- an installation with UPLOAD_PARTS_PATH unset behaves identically. Tests\TestCase points it at a per-worker directory and empties that directory per test, which closes the cross-worker, the cross-run and the intra-worker versions together. ChunkedUploadsTest's cleanup and its two directory assertions read the configured root rather than the hardcoded path, so they can no longer reach into a neighbour. Verified with eight consecutive parallel runs, green, and by watching the per-worker directories appear separately (w1, w2, w4 … w14) rather than one shared tree. The isolation itself cannot be asserted from inside a single test; what a test can pin is the mechanism it rests on, so one does: parts go where the configured root says. |
||
|
ignacionelson
|
d7e639b7af |
Close the two-request version of the deleted-folder target, and say why it failed
Follow-up to #1703, which made `exists:folders,id` mean what its ten readers already assumed. Two things it named and deliberately left. **The chunked upload is two requests.** store()'s rule only ever sees the first: POST /uploads records the resolved folder on the UploadSession and complete() reads it back from the session rather than from the caller, so deleting the folder while the bytes are in flight still files the assembled file into it -- the same orphan state #1703 removes, reached by a door a validation rule cannot watch. complete() now re-resolves through Folder::query() and files at the root when the folder has gone. Root rather than a refusal, because the two moments cost different things. At store() nothing has been sent, so refusing is free and honest, which is the call #1703 made. Here the bytes are already uploaded, and discarding somebody's finished transfer over a folder that vanished underneath them is the harsher of the two surprises. The file lands somewhere they can see it and move it. **The refusal now explains itself.** "The selected folder id is invalid" says nothing when the answer is that the folder has been deleted -- and that is the usual way to meet this rule, since a live id picked from a list is how anybody gets here. It matters most on the chunked path, the one place #1703 makes a previously-working request fail. A small ValidationRule object carries the message, which keeps the single definition Rules::folderId() exists for: a messages() array would have to be repeated at all ten call sites, and rules meaning different things in ten places is what went wrong in the first place. One note for whoever writes the next test here. Upload parts live in storage_path('app/uploads-tmp/{session_id}'), which is a real shared directory rather than a faked disk, and each parallel worker's database restarts session ids at 1 -- so two files writing parts on two workers collide, and ChunkedUploadsTest's afterEach deletes the whole tree for everybody. Six test files write parts today. These two cases live in ChunkedUploadsTest rather than beside the rest of their subject so this change does not add a seventh racer; the underlying isolation problem predates it and is worth its own fix. |
||
|
Ignacio Nelson
|
8d896191e4 |
Merge pull request #1703 from denkfabrik-li/fix/deleted-folder-upload-target
Say what `exists:folders,id` was already being read as |
||
|
Ignacio Nelson
|
18e4e014e6 |
Merge pull request #1683 from denkfabrik-li/fix/orphan-scanner-preview-renditions
Keep preview renditions out of the orphan-file scan |
||
|
ignacionelson
|
0a8b609e8b |
Build a scoped staff member's library query once per request, not once per row
#1698 moved the library boundary into FileCommentPolicy, where it belongs, and said plainly what that cost: the moderation screen went from 65 queries to 465 for a client-scoped moderator with five assigned clients. Measured here, those numbers are exactly right. The cost is not in asking. It is that StaffLibraryScope::files() rebuilds its query every time, and building one runs four immediate lookups per assigned client — the client's group ids, the same ids again inside Folder::sharedFolderIds(), that method's own assignment lookup, and the shared-folder get() in Folder::scopeVisibleToClient(). None of them depend on the query being built. Gate resolves a fresh policy for every check, so a listing paid for all of it once per row. The built query is now memoised per user and handed back as a clone, since every caller adds to it, and the scope is registered as `scoped` rather than transient so the memo survives a request. Scoped rather than a singleton on purpose: a long-lived queue worker keeps singletons between jobs, and a library query built from one job's data has no business answering the next one's question. That is 465 queries down to 60 on the same page — below the 65 it cost before #1698, because the memo also helps the callers that were already asking repeatedly. FileVersions::sharedAudience(), which runs the same helper twice per candidate while resolving notification recipients, gets it for free. So the answer to the question #1698 left open is neither of the two it offered. can_delete stays a real question asked of the policy; nothing restates the boundary; and the page is faster than it was before the fix. Three tests: one user's query never answers another's, one caller's constraints never follow the next, and the moderation screen does not ask once per row. |
||
|
denkfabrik-li
|
7727ad7616 |
Say what exists:folders,id was already being read as
Folder uses SoftDeletes. The `exists` rule runs against the table, so a folder in the trash passes it -- while every resolution that follows goes through Folder::query(), which honours the soft delete and finds nothing. Ten rules across five controllers rely on that check, and each one reads it as "this folder exists". Two of them then wrote the id anyway. Api\FilesController::store() resolves the folder, hands the null to Folder::uploadableBy(), is told yes -- correctly, that is the rule for a root upload -- and passes $validated['folder_id'] to the write. FilesController::store() is the same shape once #1694 gives it the guard. FilesController::update() and its API twin write it straight through with nothing in between. The result is a live file inside a deleted folder, which is a state nothing else in the application produces: FolderService::delete() deletes every file in the subtree along with it. The row is reachable by id, in search and over the API, and missing from the listing its uploader would look in. Rules::folderId() makes the check mean what its readers assume, once, where the reasoning can be written down -- the same argument slug() makes for itself one method above. Every site takes it, so the file cannot end up with two spellings of the same rule and no way to tell which is the safe one. What changes, path by path: - POST /files, POST /api/v1/files, PATCH /files/{file} and PATCH /api/v1/files/{file} refuse a folder in the trash instead of writing its id. This is the fix. - POST /uploads used to accept it and quietly file the upload at the root -- its guard and its write already agreed, on null. It now says so instead, which is what the other upload paths do. - files/{file}/move, files/bulk-edit, folders, folders/{folder}/move and the portal's my-folders already refused, through StaffLibraryScope::folders() or Folder::scopeVisibleToClient(), both of which drop trashed rows. They still refuse; the answer is now 422 naming folder_id rather than a bare 404. Those two guards are asking a different question -- "is this folder yours" -- and they keep asking it. No live folder id behaves differently anywhere, and the root (a null folder_id) is untouched. The published API document is unchanged: `exists` renders the same either way. Regenerated with php artisan scramble:export and byte-identical. |
||
|
denkfabrik-li
|
2c2b86ffa1 |
Hold the folder an upload names to the same library boundary as everything else
Folder::uploadableBy() returned true for any staff member without looking at the folder, on the strength of a comment saying staff had already validated folder_id through FilesController's own flow. No upload path did. FilesController::store() did not check the folder at all; the two that called uploadableBy() — the API upload and the chunked upload the browser actually posts to — called a guard that could only ever say yes. A client-scoped staff member could therefore name any folder id and put the file inside a subtree shared with somebody else's client, where File::scopeVisibleToClient hands it over without an assignment row ever being written. That is the boundary StaffLibraryScope's own docblock claims to hold everywhere. The staff branch now asks StaffLibraryScope::allowsFolder, which returns true for unscoped staff, so nothing changes for them. The client branch is untouched: a client is never client-scoped, and ownership or a public folder opting into client uploads remains the whole of their rule. The two folder pickers that fed those ids are narrowed the same way the listings around them already are. |
||
|
denkfabrik-li
|
acda732ff4 |
Enforce the download limit when a zip is delivered
A download limit is checked when an archive is ordered and again while it is built, but it is only spent when the archive is collected. Nothing about ordering or building moves the count, so every check along the way sees an allowance that is still untouched. That turns a prepared archive into a voucher. Order the same limited file into ten archives and all ten pass, because at the point each one is checked nothing has been taken yet. Collect them all and the file has been downloaded ten times against a limit of one. The three endpoints are independent of the interface that normally drives them, so this needs nothing more than calling store() in a loop — and no timing luck at all, since the archives can be collected minutes apart. DownloadAllowance says of itself that six routes put a file's bytes on the wire and that every one of them asks, precisely because there is no choke point to put the rule in. The zip pair asked in the two places that do not count and not in the one that does. So the delivery re-checks what the archive holds, where the count actually moves. Refusing is 403, matching the single-file download route for the same situation. It is also the only one of the two candidates that reaches the person: an archive is fetched by navigating to it, and there is no error view for 422, so the message would be replaced by the framework's generic "something is broken" page. One refused file refuses the whole delivery, because nothing can be taken out of a finished archive without building it again. Ordering the same selection afresh is the way through — the build leaves the spent file out and names it in skipped_files, which the poll already reports. This is stricter than store(), which drops spent files from a selection and refuses only when nothing survives: there, a selection can still be narrowed, and here it cannot. Checking costs nothing where nothing is limited. An unlimited file is answered from its own column and never reaches a count. Claiming the delivery is a conditional update now rather than a read followed by a write. Two fetches of one archive arriving together both saw delivered_at unset and both wrote a full set of downloads, counting a single delivery twice — the same shape as the conditional increment that guards a share link's max_downloads. Only the fetch that moves the column logs anything; the other still receives the archive, which is the existing rule that re-fetching one prepared zip is one delivery. Two things this deliberately leaves alone. Simultaneous downloads of one file can still both pass before either is logged: that race is documented in DownloadAllowance, and closing it needs the counter column it explains why it does not have. And an archive already delivered stays fetchable for its 24 hours even once the limit is spent — one delivery, re-fetched, which is what that rule is for. An archive built before the job recorded its contents is handed over the way it always was, without this check. What it holds can only be guessed at by resolving the selection a second time, and guessing is exactly what must not decide a refusal: the same reconstruction refuses over files the archive does not hold and misses files it does. Those rows stop existing within a day or two of an upgrade, and until then they behave as they did before this change rather than worse. |
||
|
denkfabrik-li
|
16787cf697 |
Record which files a zip actually contains
A zip download's row stores what was asked for — some file ids, some folder ids — and the download action resolved that selection a second time, when the archive was collected, to decide what to log as downloaded. The two are not the same thing. Folder contents are resolved against the scope as it stands at that moment, and an archive is written some time before it is fetched. Add a file to the folder in between and it was logged as downloaded without ever having been in the zip. Move one out of the folder and it was handed over without being logged at all. The same goes for a file that expired or otherwise left the requester's scope after the build: its bytes are in the archive either way. Nothing about this is visible to anyone — the download count on the file is simply wrong. The job already walks exactly the set that goes in, and already counted it for file_count. It now keeps the ids rather than a tally, and the download action logs those. count() gives back the number it was keeping before. Rows written before this column existed fall back to resolving the selection, which is what they were built for; the purge command clears them within a day. |
||
|
ignacionelson
|
073101d184 |
Put a ceiling on a zip download, and clean up after the ones that fail
Follow-up to #1687, which made a zip build report failure honestly. Four things it passed near, none of them regressions it introduced. A zip has never had a size limit — only a cap of 10,000 files, which bounds nothing that costs anything. Ten thousand spreadsheets zip in seconds; two hundred videos is an hour of stream-copying and an archive that fills the disk. Bytes are what a build actually costs, so the new Settings → Downloads screen caps the total size instead, at 2 GB out of the box. It is a setting rather than a constant because the safe figure depends on free disk, on whether sources live on a remote disk, and on the plan a hosted tenant is on — the file count stays fixed, since it is a foot-gun rail and not a knob anybody needs. The controller measures the selection at request time and names both numbers when it refuses; the job measures again, because it re-derives the selection at run time and a folder can grow while the job waits in the queue. Every shipped topology runs exactly one queue worker, and everything shares the default queue, so raising the job timeout to an hour handed any signed-in person an hour of everyone else's notification mail. There is now one build in progress per requester and a named throttle bucket on the endpoint, which had neither. A pending row older than an hour is treated as abandoned rather than in progress, so a worker killed hard enough to skip failed() cannot lock somebody out for good. Giving zip builds their own queue is the structural fix and wants its own change: it touches compose, supervisord and the systemd unit in INSTALL.md, and an install that upgrades without changing its worker command would stop building zips silently. zip_downloads.requested_by cascades on delete, so removing a user takes their rows with it and strands every archive they built — invisible to a purge that walks rows, and to OrphanFileScanner, which skips zips/ on purpose. The purge now also sweeps files in zips/ that no row explains, after a day's grace so a build in progress is never taken out from under itself. Two smaller things while in here. A build that failed because every file had already hit its download limit said only that nothing was available, and dropped the skipped list — the same distinction the store guard goes out of its way to draw at request time. And a failed close() now logs libzip's reason, which the @ silencing had been discarding: "the disk is full" and "the source vanished" are different problems for whoever has to fix one, while the requester still sees a message with no server paths in it. |
||
|
denkfabrik-li
|
65e7f37d36 |
Delete a file's bytes when its transaction commits, not before
File::booted() removed the bytes the moment a row was deleted. For a single file that is right. Two paths delete files inside a transaction, though, and both delete many at once: FolderService::delete() takes a folder's whole subtree, and DeletedAccountContent::cascadeDelete() takes everything an account uploaded. Anything that rolls either transaction back puts every row back while the bytes are already gone. A transaction exists to make a set of writes undoable, and removing the bytes was the one write in that set that nothing can undo. The account path is the sharper one: since content disposal is nested inside the caller's transaction, the write that fails need not be in this code at all. The two failure directions are not equal. Bytes gone with the rows restored leaves rows pointing at nothing and no way back. Rows gone with the bytes left leaves orphans on disk, which OrphanFileScanner already exists to find. Defer to the recoverable one. Three properties this relies on, all of them checked rather than assumed: without a pending transaction the callback runs immediately, so a single delete is unchanged; a savepoint committing inside a larger transaction does not fire it, which is exactly the account case; and the connection is the row's own rather than whichever is default. detachOnDelete stays inside the transaction — it repairs the version chain's pointers, which is database work that must roll back with everything else. |
||
|
denkfabrik-li
|
862765643b |
Don't name a subfolder to a client who cannot open it
MyFilesController::index() builds the portal's folder list in two branches. At the root it narrows to $visibleIds; one level in it listed every direct child of the folder being browsed, visible or not. Opening one was still refused — $current is resolved through visibleToClient() and 404s otherwise — so what escaped was the name, not the contents. A name is worth protecting here for the same reason VisibleCommentScope gives about its own boundary: it is what stops one customer learning that another exists. $visibleIds is computed once and wanted in three places. The root branch narrows by it, the breadcrumb narrows by it, and the nested branch did not. That is a gap rather than a distinction, and the class docblock had already promised the opposite: "Group and internal folder names never leak." Reachable only where a folder is visible for the created_by reason rather than by sharing. Inside a shared subtree every child matches by path prefix anyway, so nothing leaks there. A client with create_own_folders makes such a folder through POST /my-folders, and staff can file anything inside it — FoldersController::store resolves its parent through StaffLibraryScope, which is unfiltered for unscoped staff. Files were never affected: that branch's query already starts from File::query()->visibleToClient(). The breadcrumb already trims to the first visible ancestor. Neither is touched. |
||
|
denkfabrik-li
|
ff7758a31a |
Never mark a zip download ready over an archive that was not written
BuildZipDownloadJob deferred every write to ZipArchive::close() but then
marked the row STATUS_READY regardless of the result:
- close() returns false when a source file was deleted between addFile()
and close() (a concurrent staff delete runs FileDiskCleanup at once) or
the disk filled up; the row went ready over an archive libzip never
wrote, and the download controller X-Accel-served a path that isn't there.
- An archive that ended up with no entries (every selected file removed or
its allowance spent before the queued job ran) is written as no file at
all by libzip, yet close() still returns true — again marked ready.
Check both the close() return and the added-entry count, and fail the row
(deleting any partial archive) when either says nothing was written.
The job also had no $tries/$timeout/failed(): a build of up to MAX_FILES
sources runs past the worker's default 60s timeout, and the kill skips the
catch, stranding the row as PENDING while the frontend polls forever. Give
it room, run it once, and add a failed() backstop that fails a row still
pending (leaving an already-resolved one alone).
Finally, purge leftover zips/{id}.zip* by row id: a killed build leaves a
partial archive and libzip temp file with no path recorded, so the path
field alone never cleaned them up.
|
||
|
denkfabrik-li
|
329aef98e0 |
Finalise each chunked upload once, under a per-session lock
complete() assembled the received parts into the one target file and created the File row with no guard against a second complete() for the same session running at the same time -- an Uppy retry, a double submit, a resend after a lost connection. Two of them would interleave writes into the session's single `assembled` file (the stored bytes then no longer match the checksum computed from the in-memory buffers) and could each create a File row. Take a per-session lock around the finalisation and fail a second caller fast; the lock's TTL releases the claim if a completion dies mid-flight, so a genuine retry still works. The body moves to a finalise() helper so complete() reads as auth + lock + finalise. |
||
|
denkfabrik-li
|
67f340d23d |
Keep preview renditions out of the orphan-file scan
The orphan scanner skips derived artifacts by path prefix, but the list was a hard-coded ['thumbnails/', 'zips/'] that never learned about 'previews/'. ImageRendition::Preview caches under previews/ (and previews/external/) on the local files disk, so every cached preview was reported as an orphan: offered for import on the orphans screen, and deleted by the purge command once past the grace period. An imported preview also became a File row pointing at a path the rendition cache owns -- destroyed the moment its source file was deleted or the cache was flushed. Derive the rendition prefixes from ImageRendition::cases() rather than repeating them, so a future rendition can't be forgotten here the way previews were; 'zips/' (the download-bundle job's) stays as it was. |
||
|
denkfabrik-li
|
21fdf98981 |
Scope a file's destination folder on update(), as move() already does
Setting folder_id through update() is the same privileged reparent as move() and bulkUpdate(), but only those two verified the target folder was inside the caller's library (StaffLibraryScope::folders). update() validated it only for existence, so a client-scoped staff member could reparent an in-scope file into a folder shared with a client they are not assigned to -- which File::scopeVisibleToClient then exposes to that client, sidestepping the boundary the sharing endpoints enforce (guardAssignable), and likewise into a public folder without upload_public. Apply the same scope->folders()->findOrFail() guard on both the web and API update(), but only when folder_id actually changes, so re-saving a file that already sits in an out-of-scope folder (reachable via a direct client share) still works. |
||
|
denkfabrik-li
|
71d6b8937e |
Enforce the max file size against the bytes a chunked upload assembles
store() checks Setting::MaxFileSizeMb against the size the client declares when it opens the session, and complete() re-checks the storage quota against the real assembled byte count -- but nothing re-checked the size limit itself. A client that declared a one-byte upload and then streamed gigabytes of parts passed store()'s check and was never stopped, so the configured limit (which store() applies to everyone, staff included) did not hold for the resumable path that real uploads use. Re-check the assembled byte count against MaxFileSizeMb in complete(), cleaning up the assembled bytes and the session exactly as the quota branch already does. |
||
|
ignacionelson
|
a459d45c87 |
Store the bytes, or say you did not
Two bugs a green suite could not find, both from pointing the application at a real Google Cloud Storage bucket. The adapter attaches a legacy per-object ACL to every write, and a bucket with uniform bucket-level access — which our own setup instructions require, and which Google recommends — refuses it: "Cannot insert legacy ACL for an object when uniform bucket-level access is enabled". So the default configuration could not write to the recommended bucket. The library ships UniformBucketLevelAccessVisibility for exactly this, and nothing is lost by never setting an ACL: every object here is private and every read is a signed URL. The second is worse and was never about Google. Both file disks are configured 'throw' => false, so a refused write returns false rather than raising, and LocalPartStore ignored the return. The upload reported success, the File row was written, and the bytes were nowhere — the listing showed a file whose download could never work. An expired S3 credential did the same thing. It now checks, and the controller already turns that into a validation error rather than a 500, so the person uploading is told. Verified against a live bucket with a key scoped to roles/storage.objectAdmin: the probe lists, writes land, reads round-trip byte for byte, and a signed URL comes back 200 carrying "Informe año.pdf" intact through both the ASCII and RFC 8187 forms of Content-Disposition. |
||
|
ignacionelson
|
57540164fa |
Read a file from the disk it is actually on, everywhere
Two routes still assumed every file sits on local disk, which stopped
being true the moment external storage was switched on. A share link
answered with X-Accel-Redirect whatever the file's disk said, pointing
nginx at a path it has nothing behind; a public listing built a
thumbnail from Storage::disk('files')->path(), which for an externally
stored file is a path nobody ever wrote. Both fail only for installs
using S3, and only on those two routes, so the same file downloading
correctly from the file manager made the share link look like the
broken thing rather than where the file lives.
Neither is a new rule. FileDownloadController and
FileThumbnailController already did it right, which is the actual
finding: the knowledge was sitting in a private method on one class and
inline in another, so the next caller could not inherit it and did not.
Both are now objects with one job.
StoredFileResponse replaces InlineFileResponse and grows an
attachment() alongside inline(), since the two differ only by
disposition. LocalSourceFile takes a closure rather than returning a
path: the version that returned one also left the caller to unlink it,
and both of those are exactly the mistakes made here.
The regression tests fail against the previous controllers — checked in
both directions rather than assumed.
|
||
|
ignacionelson
|
503676647f |
Let a split-user host serve downloads
A download is not served by PHP. PHP authorizes it and hands the web server the path with X-Accel-Redirect, so the web server has to open a file PHP wrote. Where those are different users — cPanel and Plesk commonly arrange it that way — it cannot: uploads land 0600 inside a 0700 directory, and traversing 0700 means being its owner. Nothing else on the site shows a symptom. Uploading works, the library lists everything, and only downloads fail, as ERR_INVALID_RESPONSE in the browser and `open() ... failed (13: Permission denied)` in the web server's log. FILES_WEB_SERVER_READABLE writes uploads 0644/0755 instead. Opt-in and spread into the disk configuration rather than switched by a ternary, so an install that does not set it keeps byte-for-byte the configuration it had: the relaxed modes are readable by every account on the machine, which is the wrong trade wherever the web server and PHP are one user, as in the image and on most self-administered servers. The two halves are not enforced alike, which is the part worth knowing. `visibility` has Flysystem chmod each file after writing it, so 0644 holds under any umask. A directory is created by mkdir(), which masks its mode argument, so 0755 is a ceiling: a pool at umask 0077 still produces 0700 and still cannot be traversed. That cannot be fixed from config, so INSTALL.md carries it — how to tell the two users apart, the one-time chmod for files already on disk, and the pool setting for the umask. FilePermissionsTest asserts all three modes, umask cases included, since the asymmetry is invisible from the configuration. Reported by @denkfabrik-li (#1668), who diagnosed it and verified the remedy on the affected host. |
||
|
ignacionelson
|
51eea30dda |
Answer "did they ever actually get it?" from the file itself
The two things staff most often want to know about a file — who downloaded it, who looked at it — were answerable only by reading the whole activity log past everything else that had happened to it, or by going back to the library list for the details panel. The file's own page now has a Downloads & previews tab: the twenty most recent times it was taken or looked at, each with who did it and the address it went to, over a running count of both. Below them, two buttons open the file's full history already filtered — one to every download, one to every preview — so the narrow question is one click and the whole log is still one click further. Which filter value stands for "every download" is decided server-side and travels with the payload, because it is a fact about the log's vocabulary: downloads are three actions and share a group, previews are one action and are filtered by name. The history page now also keeps whatever filter it was sent with visible in its dropdown even at a count of zero, so a button cannot land somebody on an empty table above a select that has gone blank. |
||
|
ignacionelson
|
88c182cf3b |
Preview video, audio and PDF, not only images
v1 could preview four kinds of file in a modal — images, video, audio and PDF. v2 previewed only images, and not by decision: preview shipped as part of the image *thumbnail* work (1c68aa1), so "previewable" quietly became a synonym for "GD can decode it". FileThumbnailController::preview() gated on ThumbnailGenerator::SUPPORTED_MIME_TYPES, the frontend mirrored the same four types, and the dialog was a hardcoded <img>. Rather than widen that list — it drives pathFor(), extensionFor(), generate() and FileDiskCleanup, and a video reaching getimagesize() is a 500 — this separates the two questions. PreviewKind now answers "may these bytes be served inline, and what element renders them?", while ThumbnailGenerator keeps answering the narrower "can this app decode it itself?", which is what renditions, the cache and the watermark hook actually depend on. Image delegates to it so the two cannot drift. The allowlist stays a security boundary: mime_type is sniffed from the bytes, so text/html and image/svg+xml remain excluded, and PreviewKind is deliberately narrower than "formats a browser might cope with" — no quicktime, avi or matroska, because an embedded player for those shows a black rectangle. Those still download exactly as before. docs/security-audit-2026-08-05.md finding 1 recorded that adding application/pdf "should be a conscious decision". This is that decision, and three things were measured rather than assumed: - An <iframe sandbox> cannot be used. Chrome refuses to run its PDF viewer in a sandboxed frame at all (ERR_BLOCKED_BY_CLIENT, with or without allow-same-origin) — the attribute removes the feature, it does not harden it. - nginx's `Content-Security-Policy: sandbox; default-src 'none'` on /protected-files/ does work (a <video> frame lands in an opaque origin), but Chrome exempts its PDF viewer from it, so it is not what protects the PDF case. - What does is the allowlist plus the browser's own PDF sandbox, where PDF JavaScript has no DOM and no cookies. Range requests were verified end to end: 206 with a correct Content-Range, a byte-perfect file reassembled from three ranges, and a real browser seeking to 10s of a 20s clip. nginx drops the upstream Content-Length on the X-Accel path, so there is no collision. Two settings, both defaulting on so no installation loses what it has: clients_can_preview_files and public_listing_preview_enabled. Staff are never gated. The anonymous side needed a route of its own — there was no public preview endpoint — with its own throttle bucket, since a bare throttle: shares one counter across that whole block. A preview now logs at most one FilePreviewed per viewer per file per five minutes: a <video> turns one deliberate act into a long tail of Range requests, and a row each would bury the log. Also fixes a layout bug the tests could never catch. A portal file row was flex justify-between with three children — name, comment trigger, download — so the middle one settled wherever the name happened to end and the comment icon sat at a different place on every row. The name block now takes the slack and every action lives in one trailing group, with the comment trigger in a fixed-width slot so the icons form a column. And because half the previewable files have no thumbnail to click — a PDF, an mp3 and an mp4 all render as a generic icon — every row gains an explicit PreviewAction beside DownloadAction, matching whatever style that theme gives its download control. |
||
|
ignacionelson
|
cca3d9c314 |
Group only the downloads, which are the actions that need it
The grouped filter had a second member, "All previews", built on a public-preview action that does not exist: previewing is recorded one way today, so its own option already answers "who previewed this?" in full. Static analysis caught the reference; the group would have been unreachable even if it had compiled, since a group with a single present member is deliberately not offered. Previews get a group here the day a second way to preview a file is recorded separately, and the test now pins the single-member case on a file whose log holds one flavour of download. |