A download limit is checked when an archive is ordered and again while it
is built, but it is only spent when the archive is collected. Nothing
about ordering or building moves the count, so every check along the way
sees an allowance that is still untouched.
That turns a prepared archive into a voucher. Order the same limited file
into ten archives and all ten pass, because at the point each one is
checked nothing has been taken yet. Collect them all and the file has
been downloaded ten times against a limit of one. The three endpoints are
independent of the interface that normally drives them, so this needs
nothing more than calling store() in a loop — and no timing luck at all,
since the archives can be collected minutes apart.
DownloadAllowance says of itself that six routes put a file's bytes on
the wire and that every one of them asks, precisely because there is no
choke point to put the rule in. The zip pair asked in the two places that
do not count and not in the one that does.
So the delivery re-checks what the archive holds, where the count
actually moves. Refusing is 403, matching the single-file download route
for the same situation. It is also the only one of the two candidates
that reaches the person: an archive is fetched by navigating to it, and
there is no error view for 422, so the message would be replaced by the
framework's generic "something is broken" page.
One refused file refuses the whole delivery, because nothing can be taken
out of a finished archive without building it again. Ordering the same
selection afresh is the way through — the build leaves the spent file out
and names it in skipped_files, which the poll already reports. This is
stricter than store(), which drops spent files from a selection and
refuses only when nothing survives: there, a selection can still be
narrowed, and here it cannot.
Checking costs nothing where nothing is limited. An unlimited file is
answered from its own column and never reaches a count.
Claiming the delivery is a conditional update now rather than a read
followed by a write. Two fetches of one archive arriving together both
saw delivered_at unset and both wrote a full set of downloads, counting a
single delivery twice — the same shape as the conditional increment that
guards a share link's max_downloads. Only the fetch that moves the column
logs anything; the other still receives the archive, which is the
existing rule that re-fetching one prepared zip is one delivery.
Two things this deliberately leaves alone. Simultaneous downloads of one
file can still both pass before either is logged: that race is documented
in DownloadAllowance, and closing it needs the counter column it explains
why it does not have. And an archive already delivered stays fetchable
for its 24 hours even once the limit is spent — one delivery, re-fetched,
which is what that rule is for.
An archive built before the job recorded its contents is handed over the
way it always was, without this check. What it holds can only be guessed
at by resolving the selection a second time, and guessing is exactly what
must not decide a refusal: the same reconstruction refuses over files the
archive does not hold and misses files it does. Those rows stop existing
within a day or two of an upgrade, and until then they behave as they did
before this change rather than worse.
A zip download's row stores what was asked for — some file ids, some
folder ids — and the download action resolved that selection a second
time, when the archive was collected, to decide what to log as
downloaded.
The two are not the same thing. Folder contents are resolved against the
scope as it stands at that moment, and an archive is written some time
before it is fetched. Add a file to the folder in between and it was
logged as downloaded without ever having been in the zip. Move one out
of the folder and it was handed over without being logged at all. The
same goes for a file that expired or otherwise left the requester's
scope after the build: its bytes are in the archive either way. Nothing
about this is visible to anyone — the download count on the file is
simply wrong.
The job already walks exactly the set that goes in, and already counted
it for file_count. It now keeps the ids rather than a tally, and the
download action logs those. count() gives back the number it was
keeping before.
Rows written before this column existed fall back to resolving the
selection, which is what they were built for; the purge command clears
them within a day.
Follow-up to #1687, which made a zip build report failure honestly. Four
things it passed near, none of them regressions it introduced.
A zip has never had a size limit — only a cap of 10,000 files, which
bounds nothing that costs anything. Ten thousand spreadsheets zip in
seconds; two hundred videos is an hour of stream-copying and an archive
that fills the disk. Bytes are what a build actually costs, so the new
Settings → Downloads screen caps the total size instead, at 2 GB out of
the box. It is a setting rather than a constant because the safe figure
depends on free disk, on whether sources live on a remote disk, and on
the plan a hosted tenant is on — the file count stays fixed, since it is
a foot-gun rail and not a knob anybody needs. The controller measures
the selection at request time and names both numbers when it refuses;
the job measures again, because it re-derives the selection at run time
and a folder can grow while the job waits in the queue.
Every shipped topology runs exactly one queue worker, and everything
shares the default queue, so raising the job timeout to an hour handed
any signed-in person an hour of everyone else's notification mail. There
is now one build in progress per requester and a named throttle bucket
on the endpoint, which had neither. A pending row older than an hour is
treated as abandoned rather than in progress, so a worker killed hard
enough to skip failed() cannot lock somebody out for good. Giving zip
builds their own queue is the structural fix and wants its own change:
it touches compose, supervisord and the systemd unit in INSTALL.md, and
an install that upgrades without changing its worker command would stop
building zips silently.
zip_downloads.requested_by cascades on delete, so removing a user takes
their rows with it and strands every archive they built — invisible to a
purge that walks rows, and to OrphanFileScanner, which skips zips/ on
purpose. The purge now also sweeps files in zips/ that no row explains,
after a day's grace so a build in progress is never taken out from under
itself.
Two smaller things while in here. A build that failed because every file
had already hit its download limit said only that nothing was available,
and dropped the skipped list — the same distinction the store guard goes
out of its way to draw at request time. And a failed close() now logs
libzip's reason, which the @ silencing had been discarding: "the disk is
full" and "the source vanished" are different problems for whoever has
to fix one, while the requester still sees a message with no server
paths in it.
Client file sharing, rebuilt from the ground up: a private area per
client, resumable uploads, folders, groups and categories, sharing with
expiry dates and download limits, comments, file versions, an activity
log, a REST API, and sixteen languages.
This repository begins here. ProjectSend 2 was developed privately, and
that development history is not published — the previous generation
remains available, with its own history, at projectsend/legacy.
Free software under the GNU General Public License v2, or (at your
option) any later version.
denkfabrik-li