A quarantined file was listed in the library with every button a working
file has, and Download answered with an error page. Three changes, all
the same idea: do not offer what cannot be done.
The library no longer lists a file that is quarantined or missing from
storage. Those two live on the screens that exist to act on them —
Quarantine, and Files missing from storage — and both now link each row
to the file itself, which is where somebody deciding needs to look.
That page says why, at the top, in the colour the state deserves: red
for a threat, amber for bytes that are gone. And it stops offering the
download and the preview, because a button that answers 423 is not an
affordance.
A file still being checked stays in the library. It is about to be
usable, and its uploader should be able to see where it went.
The button was disabled on a library that had already been scanned
once, which is most of the time and exactly when somebody would press
it — after updating definitions, say. It now re-checks everything
rather than only what was never looked at, which is what its name says.
A rescan keeps each file available until its new verdict arrives, so a
full pass takes nothing offline. The two states it skips are a file
already waiting for its first verdict and one whose bytes are gone.
It is disabled for two honest reasons now — scanning is off, or a scan
is already running — and says which.
Results are green, amber and red: checked and fine, checked and could
not be read, checked and something was found. The badge gained a
warning variant to say the middle one, matching the amber the warning
alert already uses; before this a missing file wore the same red as a
virus.
A file found missing is also stamped with the time it was checked, so
it appears in the Activity list. It is a verdict like any other, and
without the stamp it was decided somewhere nobody could see.
A row whose bytes are gone was recorded as "the scanner could not be
reached". Wrong on screen, and wrong underneath: that is the one reason
the hourly sweep re-queues, so every orphaned row would have been
rescanned hourly forever.
It is its own state now, `missing`, and withheld rather than offered:
a client who sees a file listed and gets an error on the download is
worse off than one who never saw it. Staff still see it, marked, which
is the point — somebody has to decide what to do about it. The refusal
says what it is ("no longer on the server") instead of sending somebody
looking for a permission that would let them through.
A daily `projectsend:check-missing-files` finds them, whether or not
this installation scans for viruses: it is not a virus question, and an
installation with no scanner has exactly the same problem. It compares
one disk listing against the rows rather than asking "does this exist?"
per file, which on object storage would be a request per file per day.
Files that come back — a remount, a restored backup — are picked up on
the next run and re-checked rather than left for dead.
They are listed beside the orphans, which is the same fault seen from
the other end: bytes with no row, rows with no bytes. The tab carries
the count, each row says where the file should be, and removing one
takes the record with it through the deletion that already exists.
The dashboard says how many there are, and so does
`projectsend:status`, because a fleet-wide jump in this is a storage
fault nothing else in that document would show.
A backfill runs for minutes or hours inside a queue worker, where none
of it is visible. The third tab polls every four seconds and says what
is happening: whether anything is running, how many uploads are held,
how deep the queue is, how many files were checked in the last hour,
and the last twenty verdicts with what each one was. When nothing is
running, that same list is the record of the last run, which is what
somebody opening the tab after the fact came for.
Two things the live screen found that the tests had not:
**A backfill read as "nothing is being scanned."** Re-scanning a file
that already went out unchecked deliberately leaves it available, so it
is never "pending" — and the screen counted only pending files. It
counts the scans queue too, and the two are shown separately, because
"an upload nobody can download yet" and "work the scanner has not
reached" are different facts.
**A file whose bytes are missing was recorded as "the scanner could not
be reached."** Wrong on screen, and worse than wrong in behaviour: that
is the one reason the hourly sweep re-queues, so every orphaned row
would have been rescanned every hour forever. It has its own reason
now, and goes through the same policy as a file the scanner could not
open.
Both tabs also gained the header shortcut to Quarantine, and Quarantine
one back to the settings, each shown only to somebody the destination
will actually let in.
"Scan existing files" sat disabled on an installation with a library of
144 of them, and the hourly backfill would have found none either. Both
looked for `scan_note = 'before_scanning'`, and no file on any upgraded
installation carries it: the migration gives `scan_status` its default
and writes no note, and the v1 import inserts rows the same way. The
feature was inert on exactly the libraries it exists for.
A file with no reason beside its "not scanned" is now what it plainly
is — one nothing has ever looked at — through File::neverScanned(),
which the backfill, the counts and the badge all ask. Reported as
"Uploaded before virus scanning was switched on" rather than as a bare
"Not scanned", which is the one badge somebody would have had to come
and ask about.
Found on the real screen, not by a test. The tests now cover the shape
an upgrade actually produces.
"Uploads checked by: ClamAV 1.5.4" now sits beside "Downloads sent by"
and "Files stored on", and is always there. Same reasoning those two
already carry: being able to confirm at a glance that uploads are
checked is worth as much as being told when they are not.
Four states in one row. A working scanner is named. One that is not
answering says so. One letting files through is amber. No scanner at
all reads "Nothing", amber, and links to the screen that sets it up —
which is where the "turn it on" link now lives, so the big alert above
is left to the cases where a configured scanner is misbehaving. Absent
entirely where the scanner is not this installation's to connect.
Also fixes a line the dashboard itself exposed: the activity log read
'The file "" was quarantined'. The scan job has no actor and attaches no
subject, so those two templates have to take the name from their
context, not from :subject. There is a test now, which there was not
before, because a real screen caught it and a green suite did not.
Sharing a file that is still being checked writes the assignment and
says nothing. The announcement goes out when the file becomes
available — a clean scan, a file let through while the scanner was
down, or an administrator releasing it from quarantine — so nobody is
ever sent to a page that refuses them, and a file about to be
quarantined is not announced to everyone before anybody knows.
Recipients are derived from the assignments as they stand at that
moment, not remembered from the moment of sharing: a share taken back
in the meantime produces no email, and one added does. New-version
notices ride the same path, which they had to anyway — the audience
rule re-checks visibility, and a file being scanned is not visible.
Two bugs found while writing the tests, both in the hourly command:
Re-queuing a file marked it pending first. Pending means withheld, so
running --existing over a library that predates scanning would have
hidden every file in it from every client for as long as the backfill
ran, and then announced each one to its recipients a second time when
it came back. The job now knows which state it expects instead, and a
rescan leaves the file downloadable until a verdict actually arrives.
Every upload now starts as "being checked" and is not served to anyone
until a scanner has looked at it. Infected files are quarantined: kept
on disk, unreachable, waiting for an administrator.
The scanner is ClamAV, reached over a socket, streaming the file
wherever it is stored — no temporary copy for an S3 or GCS disk. What
the scanner answers is a fact; what it means for the file is this
installation's setting, so ClamAvScanner knows nothing about settings
and ScanPolicy knows nothing about sockets. Three of clamd's own alert
options are what make a file it could not open come back as an answer
rather than as "OK"; the client maps those to "too large" and
"encrypted" instead of to a threat.
Both policies default to letting files through, marked "not scanned",
which is the product owner's decision: a scanner that cannot answer must
not stop people working. Every such file is logged, and the screens that
say so come with the rest of this work.
Withholding is two rules. A file that is not available drops out of the
scopes that answer "what may this person see" — recipients and the
public listings, never the uploader's own copy. And every route that
puts bytes on the wire asks FileAvailability first: download, thumbnail,
preview, share link, the four public routes and both ends of a zip
build. A share link minted before the scan finishes says the file is
still being checked rather than 404ing.
Not yet here, and coming next: the quarantine screen and its permission,
the notifications, the settings screen, the hourly retry, the backfill
for existing libraries, and the Docker service.
ignacionelson