Renditions are generated on demand and cached by existence, and nothing
between the callers stopped two requests decoding the same image at once.
The atomic rename settled which file survived; it never stopped both from
doing the work. So N concurrent requests for one cold rendition were N
full-size decodes, each holding four bytes per source pixel — up to 160 MB
at the 40-megapixel ceiling.
That is not an attack. A public listing emits a thumbnail URL per file, a
browser opens six or more connections at once, and the first visit to a
gallery of ordinary camera images was six simultaneous decodes on a
container sized for one. PublicGroupsController reaches the generator with
no account at all, so nothing about it required a customer to be signed
in, and the 240/min throttle bounds rate rather than concurrency.
Worse than a crash, it did not resolve itself: a render killed mid-flight
renames nothing, so the cache warmed only by whatever finished before the
kill and the page died again on the next visit.
A lock keyed on the destination path — which already encodes the file, the
audience and the rendition, so two requests collide exactly when they
would have written the same path. The waiter re-reads after acquiring,
which is what turns a wait into a cache hit rather than a second decode of
the same image.
Waiting rather than refusing, because the arithmetic says so: a waiting
request holds an idle worker at about 35 MB, a rendering one holds that
plus the whole source bitmap. Six waiters cost what one renderer costs.
On timeout it refuses instead of rendering anyway. Falling through would
reinstate the pile-on at the moment the system is already struggling, and
one failed thumbnail is a better outcome than a container that dies and
takes the warm cache with it.
The wait is configurable because the right number is a property of the
machine — a small VPS reading a large source off a slow disk wants longer
— and clamped to at least a second, since a stray empty variable would
otherwise make every concurrent request fail instantly, which is the
opposite of the point.
Eight tests. Two go red without the lock, and the clamp is asserted on the
resolved value rather than the clock, because block() measures in whole
seconds and a timing assertion there would be flaky rather than wrong.
Found by the session sizing free-tier containers, from the outside.
Both thumbnail routes treat "the file exists" as "the rendition is
cached", and nothing ever invalidates one: RenderedImageCache::flush()
runs on ImageRenderingChanged, which no core code raises. Whatever is at
the path is what every later viewer gets.
ThumbnailGenerator encoded straight onto that path. A render that died
partway -- a full volume, a killed worker -- left a half-written file
that was then served as the rendition for good, and two requests
rendering the same file at once encoded into one path together.
It now writes beside the destination and renames into place. rename()
within a directory is atomic and replaces what is there, so the path is
either the previous rendition or a complete new one, and the loser of a
race leaves a whole image rather than a mixture of two. The temporary
file is removed on the way out either way.
The read side gets the other half: an empty file is not a rendition, so
both routes replace one rather than serve it. Writing through a temporary
file means this state can no longer be created here, but an installation
that ran an older version can already have it on disk, and nothing else
will ever clear it.
Three tests: an empty rendition is replaced on the signed-in route and on
the public one, and a successful render leaves nothing half-written
behind. Without the fix the first two go red; the third is about the fix's
own temporary file and passes either way.
Client file sharing, rebuilt from the ground up: a private area per
client, resumable uploads, folders, groups and categories, sharing with
expiry dates and download limits, comments, file versions, an activity
log, a REST API, and sixteen languages.
This repository begins here. ProjectSend 2 was developed privately, and
that development history is not published — the previous generation
remains available, with its own history, at projectsend/legacy.
Free software under the GNU General Public License v2, or (at your
option) any later version.
ignacionelson