Sharing a file that is still being checked writes the assignment and
says nothing. The announcement goes out when the file becomes
available — a clean scan, a file let through while the scanner was
down, or an administrator releasing it from quarantine — so nobody is
ever sent to a page that refuses them, and a file about to be
quarantined is not announced to everyone before anybody knows.
Recipients are derived from the assignments as they stand at that
moment, not remembered from the moment of sharing: a share taken back
in the meantime produces no email, and one added does. New-version
notices ride the same path, which they had to anyway — the audience
rule re-checks visibility, and a file being scanned is not visible.
Two bugs found while writing the tests, both in the hourly command:
Re-queuing a file marked it pending first. Pending means withheld, so
running --existing over a library that predates scanning would have
hidden every file in it from every client for as long as the backfill
ran, and then announced each one to its recipients a second time when
it came back. The job now knows which state it expects instead, and a
rescan leaves the file downloadable until a verdict actually arrives.
An infected file now goes somewhere rather than nowhere. Staff holding
the new release_quarantined_files permission get a Quarantine screen
listing what was refused, who uploaded it, and what the scanner called
it. They can delete it as they always could, or release it — which
needs a written reason, a password confirmation on top of the
permission, and lands in the activity log under their name.
Only the administrator role holds that permission by default. Deciding
a threat report is wrong is a different judgement from deciding a file
is no longer needed, which is why it is not delete_files.
Two notifications, two audiences: staff who can act on it, and the
person who uploaded it — for whom this is how they learn their own
machine has something on it. The people the file was shared with are
deliberately not told about a file they never received.
`projectsend:scan-files` runs hourly: it re-queues files still waiting,
and re-scans the ones that went out unscanned while the scanner was
unreachable, since it may be back. With --existing it also works
through a library uploaded before scanning was switched on, paced by a
setting so it does not starve today's uploads.
A file that was downloadable before it was caught says so on the
screen, with its download count, because that is the case where
somebody may already have a copy.
Follow-up to #1687, which made a zip build report failure honestly. Four
things it passed near, none of them regressions it introduced.
A zip has never had a size limit — only a cap of 10,000 files, which
bounds nothing that costs anything. Ten thousand spreadsheets zip in
seconds; two hundred videos is an hour of stream-copying and an archive
that fills the disk. Bytes are what a build actually costs, so the new
Settings → Downloads screen caps the total size instead, at 2 GB out of
the box. It is a setting rather than a constant because the safe figure
depends on free disk, on whether sources live on a remote disk, and on
the plan a hosted tenant is on — the file count stays fixed, since it is
a foot-gun rail and not a knob anybody needs. The controller measures
the selection at request time and names both numbers when it refuses;
the job measures again, because it re-derives the selection at run time
and a folder can grow while the job waits in the queue.
Every shipped topology runs exactly one queue worker, and everything
shares the default queue, so raising the job timeout to an hour handed
any signed-in person an hour of everyone else's notification mail. There
is now one build in progress per requester and a named throttle bucket
on the endpoint, which had neither. A pending row older than an hour is
treated as abandoned rather than in progress, so a worker killed hard
enough to skip failed() cannot lock somebody out for good. Giving zip
builds their own queue is the structural fix and wants its own change:
it touches compose, supervisord and the systemd unit in INSTALL.md, and
an install that upgrades without changing its worker command would stop
building zips silently.
zip_downloads.requested_by cascades on delete, so removing a user takes
their rows with it and strands every archive they built — invisible to a
purge that walks rows, and to OrphanFileScanner, which skips zips/ on
purpose. The purge now also sweeps files in zips/ that no row explains,
after a day's grace so a build in progress is never taken out from under
itself.
Two smaller things while in here. A build that failed because every file
had already hit its download limit said only that nothing was available,
and dropped the skipped list — the same distinction the store guard goes
out of its way to draw at request time. And a failed close() now logs
libzip's reason, which the @ silencing had been discarding: "the disk is
full" and "the source vanished" are different problems for whoever has
to fix one, while the requester still sees a message with no server
paths in it.
BuildZipDownloadJob deferred every write to ZipArchive::close() but then
marked the row STATUS_READY regardless of the result:
- close() returns false when a source file was deleted between addFile()
and close() (a concurrent staff delete runs FileDiskCleanup at once) or
the disk filled up; the row went ready over an archive libzip never
wrote, and the download controller X-Accel-served a path that isn't there.
- An archive that ended up with no entries (every selected file removed or
its allowance spent before the queued job ran) is written as no file at
all by libzip, yet close() still returns true — again marked ready.
Check both the close() return and the added-entry count, and fail the row
(deleting any partial archive) when either says nothing was written.
The job also had no $tries/$timeout/failed(): a build of up to MAX_FILES
sources runs past the worker's default 60s timeout, and the kill skips the
catch, stranding the row as PENDING while the frontend polls forever. Give
it room, run it once, and add a failed() backstop that fails a row still
pending (leaving an already-resolved one alone).
Finally, purge leftover zips/{id}.zip* by row id: a killed build leaves a
partial archive and libzip temp file with no path recorded, so the path
field alone never cleaned them up.
Client file sharing, rebuilt from the ground up: a private area per
client, resumable uploads, folders, groups and categories, sharing with
expiry dates and download limits, comments, file versions, an activity
log, a REST API, and sixteen languages.
This repository begins here. ProjectSend 2 was developed privately, and
that development history is not published — the previous generation
remains available, with its own history, at projectsend/legacy.
Free software under the GNU General Public License v2, or (at your
option) any later version.
ignacionelson