A client who may create folders creates them at the top of the library,
beside the ones staff made, and their uploads land at the root too. An
administrator opening /files gets one flat pile with nothing saying which
parts belong to whom.
With the new "Give each client a folder of their own" setting, every new
client gets a folder named after them and it acts as their root: what they
upload and any folder they create goes inside it. /files becomes a list of
clients rather than a pile.
The sentence this feature has to keep true: **the home is a default
location, not a boundary.** Folder::scopeVisibleToClient is untouched, so a
folder staff shared with a client still reaches them and sits beside their
own. Making the home a jail would have silently revoked every share that
already exists -- a data-access change wearing the clothes of a tidying-up
feature. There is a test named after that rule.
What the client sees is the *inside* of their folder, not a folder wearing
their own name, which is not information to them. The breadcrumb is trimmed
of it for the same reason: "Invoices", not "Acme Ltd / Invoices".
Some decisions worth naming:
- **A column, not a convention.** `folders.home_for_user_id`, unique.
Matching on the name breaks the moment two clients share one, and
`created_by` plus a null parent catches every root folder a client ever
made themselves. The question is asked on each upload and each portal
listing and the answer has to be exact.
- **created_by is the client**, because that is how scopeVisibleToClient
already grants somebody their own folder -- no assignment row to keep in
step with it. That is also why this writes the row rather than calling
FolderService::create(), which takes created_by from auth()->id().
- **On model events**, not in the services that make and rename clients.
There are nine of those (ClientAccounts, ClientProvisioning, the profile
screen, two update endpoints, AccountConversion, invitations, LDAP,
social) and a rule repeated in nine places is missing from the tenth.
- **Turning the setting on creates nothing.** Existing clients get a folder
when an administrator presses a button that says how many are waiting,
and it reports created/total/already-had afterwards. Somebody should be
able to switch this on, look, and switch it off without having
reorganised a library. It moves no files either.
- **Nobody deletes a home from a folder screen**, staff included, and the
client cannot rename theirs -- they own it, so ownership alone would have
let them, and its name follows the account anyway.
- **The name always follows the client**, over a hand-typed one. A folder
still called "Acme Ltd" under an account now called something else
misleads the administrator the feature exists for.
Verified in a real browser as well as in tests: the screen mounts, the
panel reads "24 of your existing clients have no folder yet", and pressing
the button answers "24 of 24 clients got a folder. 0 already had one."
The Test button asked the scanner on file, which makes it useless at
the moment it is most needed: the first attempt, before anything has
been saved. It now tries what is typed, falling back to the stored
address when the field is empty so the button still answers on a screen
nobody has touched. Nothing is written either way — testing is not
saving.
The address travels as a request field and is applied to the request's
own ScanningConfig, which is scoped so the screen and the scanner it
resolves share one. A preview address beats even a managed one, and is
set in exactly one place.
Quarantine now carries a count in the sidebar, like Comments — in amber
rather than the usual colour, because the others count work waiting and
this one counts something that went wrong. Shown only to whoever holds
the permission to act on it.
Both checked in a browser: an address typed and not saved came back
"No answer from tcp://escrito-a-mano.invalid:3310", the stored one
untouched, and the badge renders amber with the real count.
A row whose bytes are gone was recorded as "the scanner could not be
reached". Wrong on screen, and wrong underneath: that is the one reason
the hourly sweep re-queues, so every orphaned row would have been
rescanned hourly forever.
It is its own state now, `missing`, and withheld rather than offered:
a client who sees a file listed and gets an error on the download is
worse off than one who never saw it. Staff still see it, marked, which
is the point — somebody has to decide what to do about it. The refusal
says what it is ("no longer on the server") instead of sending somebody
looking for a permission that would let them through.
A daily `projectsend:check-missing-files` finds them, whether or not
this installation scans for viruses: it is not a virus question, and an
installation with no scanner has exactly the same problem. It compares
one disk listing against the rows rather than asking "does this exist?"
per file, which on object storage would be a request per file per day.
Files that come back — a remount, a restored backup — are picked up on
the next run and re-checked rather than left for dead.
They are listed beside the orphans, which is the same fault seen from
the other end: bytes with no row, rows with no bytes. The tab carries
the count, each row says where the file should be, and removing one
takes the record with it through the deletion that already exists.
The dashboard says how many there are, and so does
`projectsend:status`, because a fleet-wide jump in this is a storage
fault nothing else in that document would show.
Sharing a file that is still being checked writes the assignment and
says nothing. The announcement goes out when the file becomes
available — a clean scan, a file let through while the scanner was
down, or an administrator releasing it from quarantine — so nobody is
ever sent to a page that refuses them, and a file about to be
quarantined is not announced to everyone before anybody knows.
Recipients are derived from the assignments as they stand at that
moment, not remembered from the moment of sharing: a share taken back
in the meantime produces no email, and one added does. New-version
notices ride the same path, which they had to anyway — the audience
rule re-checks visibility, and a file being scanned is not visible.
Two bugs found while writing the tests, both in the hourly command:
Re-queuing a file marked it pending first. Pending means withheld, so
running --existing over a library that predates scanning would have
hidden every file in it from every client for as long as the backfill
ran, and then announced each one to its recipients a second time when
it came back. The job now knows which state it expects instead, and a
rescan leaves the file downloadable until a verdict actually arrives.
An infected file now goes somewhere rather than nowhere. Staff holding
the new release_quarantined_files permission get a Quarantine screen
listing what was refused, who uploaded it, and what the scanner called
it. They can delete it as they always could, or release it — which
needs a written reason, a password confirmation on top of the
permission, and lands in the activity log under their name.
Only the administrator role holds that permission by default. Deciding
a threat report is wrong is a different judgement from deciding a file
is no longer needed, which is why it is not delete_files.
Two notifications, two audiences: staff who can act on it, and the
person who uploaded it — for whom this is how they learn their own
machine has something on it. The people the file was shared with are
deliberately not told about a file they never received.
`projectsend:scan-files` runs hourly: it re-queues files still waiting,
and re-scans the ones that went out unscanned while the scanner was
unreachable, since it may be back. With --existing it also works
through a library uploaded before scanning was switched on, paced by a
setting so it does not starve today's uploads.
A file that was downloadable before it was caught says so on the
screen, with its download count, because that is the case where
somebody may already have a copy.
Every upload now starts as "being checked" and is not served to anyone
until a scanner has looked at it. Infected files are quarantined: kept
on disk, unreachable, waiting for an administrator.
The scanner is ClamAV, reached over a socket, streaming the file
wherever it is stored — no temporary copy for an S3 or GCS disk. What
the scanner answers is a fact; what it means for the file is this
installation's setting, so ClamAvScanner knows nothing about settings
and ScanPolicy knows nothing about sockets. Three of clamd's own alert
options are what make a file it could not open come back as an answer
rather than as "OK"; the client maps those to "too large" and
"encrypted" instead of to a threat.
Both policies default to letting files through, marked "not scanned",
which is the product owner's decision: a scanner that cannot answer must
not stop people working. Every such file is logged, and the screens that
say so come with the rest of this work.
Withholding is two rules. A file that is not available drops out of the
scopes that answer "what may this person see" — recipients and the
public listings, never the uploader's own copy. And every route that
puts bytes on the wire asks FileAvailability first: download, thumbnail,
preview, share link, the four public routes and both ends of a zip
build. A share link minted before the scan finishes says the file is
still being checked rather than 404ing.
Not yet here, and coming next: the quarantine screen and its permission,
the notifications, the settings screen, the hourly retry, the backfill
for existing libraries, and the Docker service.
A staff member limited to their own assigned clients could read the names
and ids of clients on nobody's roster but their own, out of ordinary file
metadata.
The file boundary was never wrong. Sharing means a file can legitimately
reach a scoped viewer through client A while client B uploaded it, or
while B also receives it -- StaffLibraryScope::buildFiles is right to
permit that, and a B-only file is still a 403. What was wrong is that
every response then went on to name B. FileResource serialised the loaded
uploader and each assignment unfiltered; ShareTargets::assigned took no
viewer at all, so the details panel published the recipient list as it
stands and forSubject narrowed available_clients while handing
assigned_clients straight through. FoldersController::fileRow,
FilesController::edit, FileDetailsController and ClientFilesController
each named the uploader the same way. The API's uploaded_by filter asked
the question without any name attached: it answered "does this client of
yours put files in front of a client of mine" for any id a caller cared
to try.
12a8ebe3 said the rule out loud while fixing topClientsByStorage -- "the
file was theirs to read and the uploader's name was not theirs to see" --
and then the rule stayed in that widget. So it is a class now.
ClientIdentityScope is the one decision, asked by every surface that
names a client, and it deliberately answers about clients only: a
colleague's name is not a client identity, and hiding it would hide who
uploaded most of the library from the people who work in it. Groups go
through it too, on the same argument -- a group is a list of clients
wearing one name -- which the report did not cover but is the same leak.
Two judgement calls worth naming. assigned() keeps returning the whole
truth and gains a warning, because VisibleCommentScope resolves
notification recipients from it and a recipient filtered out of that list
is one who never hears about a message addressed to them; assignedFor()
is the display half. And FileResource asks at serialisation rather than
in its callers' eager loads, which is the opposite of how the version
counterparts next door are narrowed: that one is set-shaped and folds
into a query, this one is a per-row roster check across eight call sites
in four controllers, two of them re-loading assignments after a write.
The tests assert on whole response bodies rather than on named keys. The
leak was never in one field -- the same name arrived through the
uploader, through the recipient list and through four screens -- so a
body that does not contain the name anywhere is the only assertion that
would have caught all of it. Ten of the eighteen fail without this
change; the rest are the negative controls, including that an unscoped
administrator still sees every name and that the uploaded_by filter still
works for a client on the roster and for staff.
Reported by @Noorkhalel, GHSA-whmp-p9hv-r7j7. Their write-up named every
affected surface and the root cause in each, which is most of why this
took one pass.
#1698 moved the library boundary into FileCommentPolicy, where it
belongs, and said plainly what that cost: the moderation screen went
from 65 queries to 465 for a client-scoped moderator with five assigned
clients. Measured here, those numbers are exactly right.
The cost is not in asking. It is that StaffLibraryScope::files() rebuilds
its query every time, and building one runs four immediate lookups per
assigned client — the client's group ids, the same ids again inside
Folder::sharedFolderIds(), that method's own assignment lookup, and the
shared-folder get() in Folder::scopeVisibleToClient(). None of them
depend on the query being built. Gate resolves a fresh policy for every
check, so a listing paid for all of it once per row.
The built query is now memoised per user and handed back as a clone,
since every caller adds to it, and the scope is registered as `scoped`
rather than transient so the memo survives a request. Scoped rather than
a singleton on purpose: a long-lived queue worker keeps singletons
between jobs, and a library query built from one job's data has no
business answering the next one's question.
That is 465 queries down to 60 on the same page — below the 65 it cost
before #1698, because the memo also helps the callers that were already
asking repeatedly. FileVersions::sharedAudience(), which runs the same
helper twice per candidate while resolving notification recipients, gets
it for free.
So the answer to the question #1698 left open is neither of the two it
offered. can_delete stays a real question asked of the policy; nothing
restates the boundary; and the page is faster than it was before the
fix. Three tests: one user's query never answers another's, one caller's
constraints never follow the next, and the moderation screen does not
ask once per row.
Client file sharing, rebuilt from the ground up: a private area per
client, resumable uploads, folders, groups and categories, sharing with
expiry dates and download limits, comments, file versions, an activity
log, a REST API, and sixteen languages.
This repository begins here. ProjectSend 2 was developed privately, and
that development history is not published — the previous generation
remains available, with its own history, at projectsend/legacy.
Free software under the GNU General Public License v2, or (at your
option) any later version.
ignacionelson