Merge pull request #1783 from projectsend/virus-scanning

Scan uploaded files for viruses
This commit is contained in:
Ignacio Nelson
2026-09-17 00:01:47 -03:00
committed by GitHub
79 changed files with 5912 additions and 52 deletions
+14
View File
@@ -20,6 +20,20 @@ PROJECTSEND_EDITION=community
# stream deliberately. The dashboard's System panel shows which is in use.
# PROJECTSEND_FILE_DELIVERY=auto
# Optional: the virus scanner every upload is checked against, as
# tcp://host:3310 or unix:///path/to/clamd.sock. Naming it here makes
# scanning managed: it is used, it cannot be switched off from the settings
# screen, and the address does not appear there. Leave it unset to
# configure scanning in Settings instead, which is the ordinary way.
# PROJECTSEND_SCANNER_ADDRESS=tcp://clamav:3310
# Optional: the scanner a fresh installation starts out pointed at, written
# into the settings on first boot and ignored on every later one. Unlike the
# variable above it leaves both the address and the switch on the settings
# screen, which is what a self-hosted install wants: configured out of the
# box, and still yours.
# PROJECTSEND_SCANNER_DEFAULT_ADDRESS=tcp://clamav:3310
# Optional: uid/gid the app/web containers' internal user runs as, so the
# bind-mounted repo needs no permission fixes. Defaults to 1000; override
# if your host user's `id -u`/`id -g` differ.
+23
View File
@@ -362,6 +362,29 @@ restored is a hypothesis, not a backup.
---
## Virus scanning
Uploads can be checked before anybody can download them. The scanner is an extra container, off
unless you ask for it:
```sh
docker compose --profile scanner up -d
```
Then go to **System → Settings → Virus scanning**, switch it on, and use `tcp://clamav:3310` as the
address. On a brand-new installation you can skip that step: uncomment
`PROJECTSEND_SCANNER_DEFAULT_ADDRESS` in the compose file before the first start and the site comes
up already pointed at the scanner. It is a starting value, not a lock — the address and the switch
stay on that screen. **Test scanner** sends a harmless standard test file and tells you whether it was actually
detected.
Two things to know before you turn it on. It needs about **11.5 GB of memory**, because the virus
definitions are held in memory. And the first start downloads those definitions, which takes a few
minutes — until it finishes, the scanner does not answer, and the Test button says so.
The scanner is reachable only from the application's own network. That is deliberate: ClamAV has no
password of any kind, so anything that can reach it can use it.
## Upgrading
With the data outside the containers, an upgrade touches only the containers:
+30
View File
@@ -517,6 +517,36 @@ the place to configure it — the settings screen also has a "send test email" b
save you a lot of guessing. The `MAIL_*` values in `.env` are only used until you fill that screen
in.
### Virus scanning
ProjectSend can check every upload before anyone can download it. It needs ClamAV, which you install
from your distribution's packages:
```sh
sudo apt install clamav-daemon # Debian/Ubuntu
sudo dnf install clamav-server # Fedora/RHEL
```
Then set these in `/etc/clamav/clamd.conf` (the paths differ per distribution) and restart the
daemon:
```
StreamMaxLength 512M
AlertExceedsMax yes
AlertEncrypted yes
AlertEncryptedArchive yes
AlertEncryptedDoc yes
```
Those `Alert` lines matter more than they look. Without them ClamAV answers "clean" for a file it
could not actually open — an encrypted zip, or one past a size limit — and ProjectSend would record
a scan that never happened.
Log in, go to **System → Settings → Virus scanning**, switch it on, and give it the socket, usually
`unix:///var/run/clamav/clamd.ctl`. Press **Test scanner**: it sends a harmless standard test file
and tells you whether the scanner actually detected it. Scanning happens in the background, so the
queue worker below must be running.
### Redis
If you have Redis available, it is faster than the database for sessions, cache and queues. Install
@@ -15,7 +15,9 @@ use App\Modules\Platform\Attribution\Attribution;
use App\Modules\Platform\Capabilities\CapabilityRegistry;
use App\Modules\Platform\Captcha\Captcha;
use App\Modules\Platform\Installation\Installation;
use App\Modules\Files\Models\File;
use App\Modules\Files\Queue\StalledZipBuilds;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Platform\Localization\LocaleRegistry;
use App\Modules\Platform\Localization\TimezoneRegistry;
use App\Modules\Platform\OfficialLinks;
@@ -183,6 +185,18 @@ class HandleInertiaRequests extends Middleware
$counts['comments'] = app(VisibleCommentScope::class)->pendingTotal($user);
}
if ($checker->allows($user, Permission::ReleaseQuarantinedFiles)) {
// Deliberately not library-scoped, unlike the comments count
// above: a quarantined file is not a file anybody is working
// with, it is one somebody has to decide about, and the
// permission is already narrow enough that whoever holds it
// is meant to see all of them.
$counts['quarantine'] = File::query()->whereIn('scan_status', [
ScanStatus::Infected->value,
ScanStatus::UnscannableBlocked->value,
])->count();
}
// Unlike the counts above, every authenticated user (staff or
// client) has their own personal notifications — no permission
// gate here.
+27
View File
@@ -75,6 +75,21 @@ enum Action: string
case FolderMadePrivate = 'folder.made_private';
case UploadAborted = 'upload.aborted';
case FileImported = 'file.imported';
// Virus scanning. A clean result is not logged: it is the ordinary
// outcome of every upload, and a row per upload would bury the ones
// that matter. Only the three that need answering are.
case FileQuarantined = 'file.quarantined';
case FileReleased = 'file.released';
// Allowed through without being checked — because the scanner could
// not be reached, or the file was too large or encrypted and this
// installation allows those. The context says which.
case FileNotScanned = 'file.not_scanned';
// The row is here and the bytes are not. Written by the daily check,
// so it has no actor: nobody did this, or nobody who was using the
// application did.
case FileMissing = 'file.missing';
case OrphanFileDeleted = 'orphan_file.deleted';
case OrphanFileAutoDeleted = 'orphan_file.auto_deleted';
case ExpiredFileDeleted = 'file.expired_deleted';
@@ -212,6 +227,14 @@ enum Action: string
self::CommentDeleted => 'Deleted a comment on the file ":subject"',
self::CommentApproved => 'Approved a comment on the file ":subject"',
self::FileImported => 'Imported the orphan file ":subject"',
// :name rather than :subject, unlike the file actions above
// it: these two are written by the scan job, which has no
// actor and attaches no subject, so the name has to travel in
// the context or the line reads 'The file "" was quarantined'.
self::FileQuarantined => 'The file ":name" was quarantined: :threat',
self::FileReleased => 'Released the quarantined file ":subject" (:reason)',
self::FileNotScanned => 'The file ":name" was not scanned for viruses: :reason',
self::FileMissing => 'The file ":name" is no longer on the server',
self::OrphanFileDeleted => 'Deleted the orphan file ":name"',
self::OrphanFileAutoDeleted => 'Deleted the orphan file ":name"',
self::ExpiredFileDeleted => 'Deleted the expired file ":name"',
@@ -316,6 +339,10 @@ enum Action: string
self::CommentDeleted => 'A comment was deleted',
self::CommentApproved => 'A comment was approved',
self::FileImported => 'An orphan file was imported',
self::FileQuarantined => 'A file was quarantined by the virus scanner',
self::FileReleased => 'A quarantined file was released by an administrator',
self::FileNotScanned => 'A file was allowed through without being scanned',
self::FileMissing => 'A file in the library was found to be missing from storage',
self::OrphanFileDeleted => 'An orphan file was deleted',
self::OrphanFileAutoDeleted => 'An orphan file was automatically deleted after its retention grace period passed',
self::ExpiredFileDeleted => 'An expired file was automatically deleted after its retention grace period passed',
@@ -17,6 +17,9 @@ use App\Modules\Clients\ClientStorageUsage;
use App\Modules\Files\Access\StaffLibraryScope;
use App\Modules\Files\Delivery\FileDelivery;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Groups\Models\Group;
use App\Modules\Identity\UserType;
use App\Modules\Platform\Capabilities\Capability;
@@ -480,7 +483,7 @@ class DashboardController extends Controller
}
/**
* @return array<string, array<string, bool|string|null>|bool|int|string|null>
* @return array<string, array<string, bool|int|string|null>|bool|int|string|null>
*/
private function systemInfo(): array
{
@@ -511,6 +514,73 @@ class DashboardController extends Controller
// able to confirm at a glance, not only worth warning about
// when it is false — the same reasoning as storage_durability.
'file_delivery' => $this->fileDelivery->describe(),
// Always stated, like delivery and storage above it: "my
// uploads are checked by ClamAV" is worth confirming at a
// glance, not only worth mentioning when it is false. Null
// only where this installation does not connect its own
// scanner at all.
'scanning' => $this->scanningState(),
// Rows this installation lists and cannot produce. Zero is the
// ordinary answer and says nothing on screen; anything else is
// somebody's files gone, which is worth interrupting for.
'missing_files' => File::query()->where('scan_status', ScanStatus::Missing)->count(),
];
}
/**
* Where this installation stands with virus scanning.
*
* Reported whether or not anything is wrong: the System card states
* how downloads leave and where files are stored for the same reason,
* and "nothing is checking my uploads" is exactly the fact an
* administrator will not go looking for.
*
* Null only when this installation does not connect its own scanner
* on a hosted one that is the platform's infrastructure, and a tenant
* reading about it could neither confirm nor fix it. See
* Capability::VirusScanningConnect.
*
* @return array{configured: bool, reachable: bool, engine: string|null, definitions_age_hours: int|null, let_through_24h: int, pending: int}|null
*/
private function scanningState(): ?array
{
if (! $this->capabilities->has(Capability::VirusScanningConnect)) {
return null;
}
$config = app(ScanningConfig::class);
if (! $config->enabled()) {
return [
'configured' => false,
'reachable' => false,
'engine' => null,
'definitions_age_hours' => null,
'let_through_24h' => 0,
'pending' => 0,
];
}
$scanner = app(VirusScanner::class)->status();
return [
'configured' => true,
'reachable' => $scanner->reachable,
'engine' => $scanner->engine,
'definitions_age_hours' => $scanner->definitionsAgeHours(),
// Files that went out unchecked in the last day. Zero is the
// only number that means "protected"; anything else is a
// scanner that was down, or files nobody could open.
'let_through_24h' => ActivityLog::query()
->where('action', Action::FileNotScanned)
->where('created_at', '>=', now()->subDay())
->count(),
// Waiting more than an hour: on an installation set to hold,
// this is what an outage looks like.
'pending' => File::query()
->where('scan_status', ScanStatus::Pending)
->where('created_at', '<=', now()->subHour())
->count(),
];
}
@@ -0,0 +1,77 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Console;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\MissingFileScanner;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use Illuminate\Console\Command;
/**
* Checks daily that the files this installation lists are actually there.
*
* Its own command rather than part of scanning, because a missing file is
* not a virus question and an installation with no scanner has exactly
* the same problem. It was only ever noticed when something tried to read
* the bytes a download, or a scan which means the first person to
* find out was a client clicking a link.
*
* Recovery is part of the job: storage comes back, and a library that
* kept insisting every file was gone would be its own bug.
*/
class CheckMissingFilesCommand extends Command
{
protected $signature = 'projectsend:check-missing-files';
protected $description = 'Check that every file in the database is still on disk (runs daily)';
public function handle(MissingFileScanner $scanner, ActivityLogger $activity, ScanningConfig $scanning): int
{
$gone = $scanner->scan();
$newlyGone = 0;
foreach (array_chunk($gone, 200) as $chunk) {
foreach (File::query()->whereIn('id', $chunk)->where('scan_status', '!=', ScanStatus::Missing)->get() as $file) {
// Stamped like any other verdict: this is the moment the
// file was last looked at, and without it a missing file
// never appears in the Activity list — which is exactly
// where somebody watching would look for it.
$file->forceFill([
'scan_status' => ScanStatus::Missing,
'scan_note' => null,
'scanned_at' => now(),
])->save();
$activity->logSystem(Action::FileMissing, ['id' => $file->id, 'name' => $file->name]);
$newlyGone++;
}
}
$back = $scanner->recovered();
foreach (array_chunk($back, 200) as $chunk) {
// Back to the start rather than to whatever it was before:
// nothing here knows what the scanner had decided about bytes
// that have since been away, and re-checking them is cheap
// next to trusting a verdict about a file that left.
File::query()->whereIn('id', $chunk)->update($scanning->enabled()
? ['scan_status' => ScanStatus::Pending->value, 'scan_note' => null]
: ['scan_status' => ScanStatus::NotScanned->value, 'scan_note' => NotScannedReason::BeforeScanning->value]);
}
$this->info(sprintf(
'%d file(s) are missing from storage (%d newly), %d came back.',
count($gone),
$newlyGone,
count($back),
));
return self::SUCCESS;
}
}
@@ -0,0 +1,113 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Console;
use App\Modules\Files\Jobs\ScanFileJob;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use Illuminate\Console\Command;
use Illuminate\Database\Eloquent\Builder;
/**
* Sends files back to the scanner: the ones still waiting, the ones that
* went through unscanned because it was down, and when asked the
* library that was already here before any of this existed.
*
* Hourly rather than daily. A file stuck pending is a file nobody can
* download, and an installation set to hold has no other way forward
* once its worker restarted and the job with it.
*/
class ScanFilesCommand extends Command
{
protected $signature = 'projectsend:scan-files
{--existing : also work through files that were never scanned because scanning was off}
{--all : check every file again, whatever it said last}';
protected $description = 'Scan files that are waiting, were missed, or were never checked (runs hourly)';
public function handle(ScanningConfig $config): int
{
if (! $config->enabled()) {
$this->info('Virus scanning is switched off.');
return self::SUCCESS;
}
$waiting = $this->dispatchFor(File::query()->where('scan_status', ScanStatus::Pending));
// Allowed through while the scanner was unreachable. Now that it
// may be back, they are asked again — a file found infected at
// this point is quarantined like any other, and its quarantine
// notice says it was available in the meantime.
$missed = $this->dispatchFor(
File::query()
->where('scan_status', ScanStatus::NotScanned)
->where('scan_note', NotScannedReason::ScannerUnavailable->value),
rescan: true,
);
$this->info("Re-queued {$waiting} waiting file(s) and {$missed} that were missed while the scanner was down.");
if ($this->option('all')) {
// Everything except the two states there is no point asking
// about: a file already waiting for its first verdict, and one
// whose bytes are not there to read. Files keep their current
// state — and stay downloadable — until a new verdict arrives.
$limit = $config->existingScanRatePerMinute() * 60;
$checked = $this->dispatchFor(
File::query()->whereNotIn('scan_status', [ScanStatus::Pending->value, ScanStatus::Missing->value]),
$limit,
rescan: true,
);
$this->info("Queued {$checked} file(s) to be checked again.");
return self::SUCCESS;
}
if ($this->option('existing')) {
// Paced, because this can be a whole library at once and the
// scanner is also serving today's uploads. An hour's worth per
// run, since that is how often this command runs.
$limit = $config->existingScanRatePerMinute() * 60;
$old = $this->dispatchFor(File::query()->neverScanned(), $limit, rescan: true);
$this->info("Queued {$old} file(s) that had never been scanned.");
}
return self::SUCCESS;
}
/**
* Nothing here changes a file's state before the scanner has spoken.
*
* An earlier version marked each file pending first, which reads as
* tidy and is wrong twice over: pending means "withheld", so a
* backfill would have hidden an entire library from its clients for
* as long as it ran, and every file would then have been announced to
* its recipients a second time when it came back. The job knows which
* state it expects instead see its $rescan.
*
* @param Builder<File> $query
*/
private function dispatchFor(Builder $query, ?int $limit = null, bool $rescan = false): int
{
if ($limit !== null) {
$query->limit($limit);
}
$ids = $query->orderBy('id')->pluck('id');
foreach ($ids as $id) {
ScanFileJob::dispatch((int) $id, $rescan);
}
return $ids->count();
}
}
@@ -0,0 +1,25 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Events;
use App\Modules\Files\Models\File;
/**
* A file that could not be handed to anyone now can be.
*
* Dispatched by FileAvailability::markAvailable(), from all three ways a
* file gets there: a clean scan, a scan this installation gave up waiting
* for, and an administrator releasing a quarantined file.
*
* It exists so that "tell the recipients" is written once rather than at
* each of those three, and so the private package can hook the same
* moment the same reasoning FileWasStored is dispatched under.
*/
final class FileBecameAvailable
{
public function __construct(
public readonly File $file,
) {}
}
@@ -5,13 +5,21 @@ declare(strict_types=1);
namespace App\Modules\Files;
use App\Modules\Files\Access\ClientIdentityScope;
use App\Modules\Files\Events\FileBecameAvailable;
use App\Modules\Files\Events\FileWasStored;
use App\Modules\Files\Listeners\AnnounceAvailableFile;
use App\Modules\Files\Access\StaffLibraryScope;
use App\Modules\Files\Models\File;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Jobs\ScanFileJob;
use App\Modules\Files\Notifications\FileShareDigestNotification;
use App\Modules\Files\Notifications\FileSharedNotification;
use App\Modules\Files\Notifications\NewVersionAvailableNotification;
use App\Modules\Files\Notifications\NewVersionDigestNotification;
use App\Modules\Files\Scanning\ClamAvScanner;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Files\Thumbnails\Events\ImageRenderingChanged;
use App\Modules\Files\Thumbnails\RenderedImageCache;
use App\Modules\Notifications\NotificationTypeDefinition;
@@ -35,6 +43,18 @@ class FilesServiceProvider extends ServiceProvider
// Same lifetime, same reason: the identity rule memoises a roster
// per viewer and the file listings ask it once per row.
$this->app->scoped(ClientIdentityScope::class);
// Scoped, so the settings screen and the scanner it resolves share
// one instance: that is what lets the Test button try the address
// being typed rather than the one on file. Scoped rather than a
// singleton so a queue worker starts each job with a clean one.
$this->app->scoped(ScanningConfig::class);
// One implementation ships, and the interface exists so the test
// suite can state a verdict instead of producing a file that
// provokes one — and so a commercial engine can be added later
// without touching the job or the policy.
$this->app->bind(VirusScanner::class, ClamAvScanner::class);
}
public function boot(): void
@@ -97,8 +117,47 @@ class FilesServiceProvider extends ServiceProvider
url: fn (array $data): string => route('my-files.index'),
));
// Two audiences, two types, because they need different words and
// different links. Staff get a queue to act on; the person who
// uploaded gets told their file did not go through.
$registry = $this->app->make(NotificationTypeRegistry::class);
$registry->register(new NotificationTypeDefinition(
key: 'file_quarantined',
label: 'A file was quarantined by the virus scanner',
template: 'A virus was found in ":itemName", uploaded by :uploaderName',
url: fn (array $data): string => route('files.quarantine'),
));
$registry->register(new NotificationTypeDefinition(
key: 'upload_blocked',
label: 'One of your uploads was blocked',
template: 'Your file ":itemName" was blocked: :threat',
// Their own files list. Deliberately not the quarantine
// screen, which they cannot open.
url: fn (array $data): string => route('my-files.index'),
));
// The other half of holding an announcement back while a file is
// being checked — see FileSharing::assign and
// AnnounceAvailableFile.
Event::listen(FileBecameAvailable::class, AnnounceAvailableFile::class);
// Every upload path converges on FileWasStored, so this is the
// one place a scan is started from. Dispatched rather than run
// inline: a 5 GB file takes minutes to read, and an upload must
// not wait for it — the file is already withheld until the
// verdict arrives.
Event::listen(FileWasStored::class, function (FileWasStored $event): void {
if ($event->file->scan_status === ScanStatus::Pending) {
ScanFileJob::dispatch($event->file->id);
}
});
if ($this->app->runningInConsole()) {
$this->commands([
Console\ScanFilesCommand::class,
Console\CheckMissingFilesCommand::class,
Console\PurgeStaleUploadsCommand::class,
Console\PurgeZipDownloadsCommand::class,
Console\PurgeExpiredFilesCommand::class,
@@ -18,6 +18,7 @@ use App\Modules\Files\Editing\ApplyFileEdits;
use App\Modules\Files\Editing\FileExpiry;
use App\Modules\Files\Http\Resources\Api\FileResource;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Storage\ResolvingUploadDisk;
use App\Modules\Files\Uploads\StoreUploadedFile;
@@ -107,6 +108,10 @@ class FilesController extends Controller
'downloads' => ['nullable', 'in:none,any'],
'version' => ['nullable', 'in:current,outdated'],
'expired' => ['nullable', 'boolean'],
// One of pending, clean, infected, released, not_scanned or
// unscannable_blocked — so an integration can wait for a file
// it just uploaded, or collect what is in quarantine.
'scan_status' => ['nullable', 'string', Rule::enum(ScanStatus::class)],
]);
$query = $this->viewable->for($user)
@@ -206,6 +211,10 @@ class FilesController extends Controller
$request->boolean('expired') ? $query->expired() : $query->notExpired();
}
if (($filters['scan_status'] ?? null) !== null) {
$query->where('files.scan_status', $filters['scan_status']);
}
return FileResource::collection($this->polling->paginate($request, $query, 'files'));
}
@@ -10,6 +10,7 @@ use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Access\DownloadAllowance;
use App\Modules\Files\Delivery\StoredFileResponse;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use Illuminate\Http\RedirectResponse;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Gate;
@@ -29,12 +30,18 @@ class FileDownloadController extends Controller
private readonly ActivityLogger $activity,
private readonly DownloadAllowance $allowance,
private readonly StoredFileResponse $bytes,
private readonly FileAvailability $availability,
) {}
public function __invoke(Request $request, File $file): Response|RedirectResponse
{
Gate::authorize('view', $file);
// Before the download limit and before the log: a file the scanner
// has not cleared is not served to anybody, and a refusal here is
// not a download to count.
$this->availability->guardDelivery($file);
// Separate from the policy on purpose: a spent download limit is
// not "you may not see this file" — the file stays listed, and
// the same person may still open its details. It is only the
@@ -10,6 +10,7 @@ use App\Modules\Files\Access\DownloadAllowance;
use App\Modules\Files\Delivery\FileDelivery;
use App\Modules\Files\Delivery\StoredFileResponse;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Preview\PreviewKind;
use App\Modules\Files\Preview\PreviewLog;
use App\Modules\Files\Thumbnails\Events\ResolvingImageRendering;
@@ -78,12 +79,18 @@ class FileThumbnailController extends Controller
private readonly LocalSourceFile $source,
private readonly Settings $settings,
private readonly FileDelivery $delivery,
private readonly FileAvailability $availability,
) {}
public function thumbnail(Request $request, File $file): Response
{
Gate::authorize('view', $file);
// A rendition is made by an image library reading the file, which
// is itself a way in — so an unchecked file is not rendered, not
// even as 300 pixels.
$this->availability->guardDelivery($file);
// This one route serves both the staff file manager and the client
// portal — the same URL, told apart only by who is asking. A client
// and a staff member looking at the same file get different cached
@@ -119,6 +126,8 @@ class FileThumbnailController extends Controller
{
Gate::authorize('view', $file);
$this->availability->guardDelivery($file);
// The inline allowlist. See the class docblock and PreviewKind —
// the stored mime type is sniffed from the bytes, so an allowed
// extension is not evidence of a safe-to-render payload.
@@ -177,6 +177,18 @@ class FilesController extends Controller
// 12th reopens showing the 11th.
'expires_at' => $this->expiry->asShown($file, $request->user()),
'expired' => $file->isExpired(),
// Said on the one screen that still shows a quarantined or
// missing file, since the library no longer lists it: a
// staff member who followed a link from Quarantine should
// not have to work out why the download refuses.
'scan_status' => $file->scan_status->value,
'scan_note' => $file->scan_note,
// Decided here rather than by the page comparing six
// states: whether there are bytes to hand over at all.
// Every button that would produce them is hidden when
// there are not — a download that answers 423 is not an
// affordance, it is a trap.
'scan_available' => $file->scan_status->isAvailable(),
'download_limit' => $file->download_limit,
'download_limit_scope' => ($file->download_limit_scope ?? DownloadLimitScope::Total)->value,
// The file's total downloads, so the editor can see what
@@ -18,6 +18,9 @@ use App\Modules\Files\Folders\BreadcrumbBuilder;
use App\Modules\Files\Folders\FolderService;
use App\Modules\Files\Models\Category;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Versions\FileVersionLinks;
use App\Modules\Groups\Models\Group;
@@ -128,7 +131,18 @@ class FoldersController extends Controller
// something on this install is actually limited.
$fileQuery = $this->allowance->withOwnCount(
$this->scope->files($user)->with('uploader.role', 'categories', 'folder')
->withCount(['assignments', 'downloads']),
->withCount(['assignments', 'downloads'])
// A file the scanner refused, or one whose bytes are gone,
// is not a file anybody can work with: every button on its
// row leads somewhere that refuses, and the download leads
// to an error page. They are listed on the two screens
// that exist to act on them — Quarantine, and Files
// missing from storage — and left out here.
->whereNotIn('scan_status', [
ScanStatus::Infected->value,
ScanStatus::UnscannableBlocked->value,
ScanStatus::Missing->value,
]),
$user,
);
@@ -278,6 +292,39 @@ class FoldersController extends Controller
]);
}
/**
* What the scanner made of a file, for a staff member's list.
*
* Staff see every file they always saw, with its state on it
* withholding applies to recipients, not to the library. Null while
* scanning is off so nothing is decorated on an installation that does
* not use it.
*
* @return array{status: string, note: string|null}|null
*/
private function scanState(File $file): ?array
{
if (! app(ScanningConfig::class)->enabled() && $file->scan_status === ScanStatus::NotScanned) {
return null;
}
// A file from before the scanner existed carries no reason — see
// File::scopeNeverScanned — and "Not scanned" with no explanation
// is the one badge somebody would have to come and ask about.
$note = $file->scan_note ?? ($file->scan_status === ScanStatus::NotScanned
? NotScannedReason::BeforeScanning->value
: null);
return [
'status' => $file->scan_status->value,
// A reason is a key and is translated here; a threat name is
// the scanner's own words and is passed through.
'note' => $note === null ? null : (NotScannedReason::tryFrom($note)?->label() !== null
? (string) __(NotScannedReason::from($note)->label())
: $note),
];
}
/**
* @return array<string, mixed>
*/
@@ -330,6 +377,9 @@ class FoldersController extends Controller
] : null,
'public' => $file->isEffectivelyPublic(),
'expired' => $file->isExpired(),
// Null while scanning is off, so a library that does not use
// it carries no badge.
'scan' => $this->scanState($file),
// No link at all once expired — the public route 404s past
// expiry too (see File::scopeNotExpired's callers), so there's
// no point offering a button that leads to a dead page.
@@ -7,7 +7,9 @@ namespace App\Modules\Files\Http\Controllers;
use App\Http\Controllers\Controller;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\OrphanFileScanner;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Uploads\StoreUploadedFile;
use App\Support\Pagination;
use Illuminate\Contracts\Filesystem\Filesystem;
@@ -45,6 +47,14 @@ class OrphanFilesController extends Controller
$validated = $request->validate(['search' => ['nullable', 'string', 'max:255']]);
$search = trim($validated['search'] ?? '');
// The mirror image of this screen, on the same screen: bytes with
// no row, and rows with no bytes. They are the same fault seen
// from either end, and an administrator looking into one has
// every reason to look at the other.
if ($request->query('tab') === 'missing') {
return $this->missing($request);
}
// A full disk scan (potentially thousands of entries, across
// every scanned disk) happens once per request regardless of
// page — Storage::allFiles() has no server-side paging of its
@@ -75,10 +85,51 @@ class OrphanFilesController extends Controller
);
return Inertia::render('files/orphans', [
'tab' => 'orphans',
'orphans' => $paginator->items(),
'pagination' => Pagination::meta($paginator),
'search' => $search,
'scanned_disks' => $this->scanner->scannedDisks(),
'missing_count' => File::query()->where('scan_status', ScanStatus::Missing)->count(),
]);
}
/**
* Files this installation lists and cannot produce.
*
* Read from the rows rather than from the disk: the daily check
* (projectsend:check-missing-files) has already done the comparing,
* and repeating a full disk listing on every page load would make
* this screen slower the worse the problem is.
*/
private function missing(Request $request): Response
{
$missing = File::query()
->where('scan_status', ScanStatus::Missing)
->with('uploader')
->orderBy('name')
->paginate(self::PER_PAGE)
->withQueryString();
$missing->through(fn (File $file): array => [
'id' => $file->id,
'name' => $file->name,
'original_name' => $file->original_name,
'size' => $file->size,
'disk' => $file->disk,
'path' => $file->path,
'uploader' => $file->uploader?->name,
'created_at' => $file->created_at?->toIso8601String(),
]);
return Inertia::render('files/orphans', [
'tab' => 'missing',
'orphans' => [],
'pagination' => Pagination::meta($missing),
'search' => '',
'scanned_disks' => $this->scanner->scannedDisks(),
'missing' => $missing->items(),
'missing_count' => $missing->total(),
]);
}
@@ -11,6 +11,8 @@ use App\Modules\Files\Access\DownloadAllowance;
use App\Modules\Files\Delivery\StoredFileResponse;
use App\Modules\Files\Models\Category;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Models\ShareLink;
use Illuminate\Http\RedirectResponse;
use Inertia\Inertia;
@@ -29,6 +31,7 @@ class PublicShareController extends Controller
private readonly ActivityLogger $activity,
private readonly DownloadAllowance $allowance,
private readonly StoredFileResponse $bytes,
private readonly FileAvailability $availability,
) {}
public function show(string $token): InertiaResponse
@@ -47,6 +50,16 @@ class PublicShareController extends Controller
return Inertia::render('share/show', ['status' => 'expired']);
}
// A link can be minted the moment a file is stored — the hosted
// free plan does exactly that — so the link routinely exists
// before the scanner has finished. It says so rather than 404ing:
// the visitor was sent a real link and it will work shortly.
if (! $this->availability->isAvailable($file)) {
return Inertia::render('share/show', [
'status' => $file->scan_status === ScanStatus::Pending ? 'checking' : 'unavailable',
]);
}
// Two separate caps reach the same page: the link's own
// max_downloads, and the file's. A visitor here has no account,
// so the file's limit is measured against the whole file — see
@@ -83,6 +96,12 @@ class PublicShareController extends Controller
return redirect()->route('share.show', $token);
}
// Same for a file still being checked, and for the same reason
// the limit is asked before the counter moves.
if (! $this->availability->isAvailable($file)) {
return redirect()->route('share.show', $token);
}
// Before the link's counter moves, not after: a download refused
// by the file's own limit must not spend one of the link's.
if (! $this->allowance->allows($file, null)) {
@@ -0,0 +1,114 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Http\Controllers;
use App\Http\Controllers\Controller;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanStatus;
use App\Support\Pagination;
use Illuminate\Http\RedirectResponse;
use Illuminate\Http\Request;
use Inertia\Inertia;
use Inertia\Response;
/**
* The files the virus scanner refused, and the one decision a person can
* make about them.
*
* Nothing is deleted here automatically and nothing expires out of this
* list: a quarantined file waits for somebody. Deleting one is the
* ordinary file deletion, with its ordinary permission this screen only
* adds the other answer, which is that the scanner was wrong.
*
* Releasing is gated by a permission of its own that only the
* administrator role holds by default, and by password confirmation on
* top of it, because it is the one action in the application that
* deliberately hands out a file something reported as malicious.
*/
class QuarantineController extends Controller
{
public function __construct(
private readonly ActivityLogger $activity,
private readonly FileAvailability $availability,
) {}
public function index(Request $request): Response
{
$files = File::query()
->whereIn('scan_status', [ScanStatus::Infected->value, ScanStatus::UnscannableBlocked->value])
->with('uploader')
->orderByDesc('scanned_at')
->paginate(25)
->withQueryString();
$files->through(fn (File $file): array => [
'id' => $file->id,
'name' => $file->name,
'original_name' => $file->original_name,
'size' => $file->size,
'uploader' => $file->uploader?->name,
// The threat name, or — for a file nothing could open — what
// stopped it being read.
'threat' => $file->scan_status === ScanStatus::UnscannableBlocked
? __(NotScannedReason::tryFrom((string) $file->scan_note)?->label() ?? 'Could not be scanned')
: $file->scan_note,
'status' => $file->scan_status->value,
'scanned_at' => $file->scanned_at?->toIso8601String(),
// True only for a file that went out unscanned while the
// scanner was unreachable and was caught later — which is the
// one case where somebody may already have a copy.
'was_available' => $file->scan_was_available,
'downloads_count' => $file->downloads()->count(),
]);
return Inertia::render('files/quarantine', [
'files' => $files->items(),
'pagination' => Pagination::meta($files),
]);
}
/**
* Overrule the scanner for one file.
*
* The reason is required and is recorded against the person who gave
* it. A release is not undone by a later scan: the file stays
* released until somebody deletes it, which is the point an
* administrator who has decided a detection is wrong should not have
* to decide it again every hour.
*/
public function release(Request $request, File $file): RedirectResponse
{
abort_unless($file->scan_status->isQuarantined(), 404);
$validated = $request->validate([
'reason' => ['required', 'string', 'max:500'],
]);
$actor = $request->user();
assert($actor !== null);
$file->forceFill([
'scan_status' => ScanStatus::Released,
'released_by' => $actor->id,
'released_at' => now(),
])->save();
$this->activity->log(Action::FileReleased, subject: $file, context: [
'reason' => $validated['reason'],
'threat' => $file->scan_note,
]);
// Everything that was waiting on this file — a share email, a new
// version notice — goes out now, exactly as it would have if the
// scan had passed.
$this->availability->markAvailable($file);
return back()->with('success', __('The file has been released.'));
}
}
@@ -0,0 +1,356 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Http\Controllers;
use App\Http\Controllers\Controller;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScannerAddress;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanOutcome;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Platform\Capabilities\Capability;
use App\Modules\Platform\Capabilities\CapabilityRegistry;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
use Illuminate\Http\JsonResponse;
use Illuminate\Support\Facades\Artisan;
use Illuminate\Http\RedirectResponse;
use Illuminate\Http\Request;
use Illuminate\Support\Facades\Queue;
use Illuminate\Validation\Rule;
use Inertia\Inertia;
use Inertia\Response;
/**
* The virus scanning screen.
*
* Two of these settings decide what happens when the scanner cannot
* answer, and both default to letting files through. That is a
* deliberate choice (see docs/feature-virus-scanning.md) and it is the
* reason this screen states the count of files currently allowed through
* unscanned rather than leaving it to be discovered: a scanner that has
* quietly stopped protecting anything looks exactly like one that is
* working.
*
* Where a managed configuration names a scanner, the connection is not
* this screen's to change and scanning cannot be switched off the
* policies still are. Same shape as the CAPTCHA screen under managed
* keys.
*/
class VirusScanningSettingsController extends Controller
{
public function __construct(
private readonly Settings $settings,
private readonly ScanningConfig $config,
private readonly ActivityLogger $activity,
private readonly CapabilityRegistry $capabilities,
) {}
public function edit(Request $request): Response
{
return Inertia::render('system/settings/virus-scanning', [
// Which half of the screen is open. The connection and the
// policies are two different jobs — one is done once when the
// scanner is set up, the other is revisited — and a single
// column of fields with two Save buttons reads as one form
// that saves half of itself.
'tab' => in_array($request->query('tab'), ['options', 'activity'], true)
? (string) $request->query('tab')
: 'scanner',
// Read from the session here rather than shared as a flash
// prop: HandleInertiaRequests shares `success` and `error` and
// nothing else, which is why the Test button appeared to do
// nothing at all. Same shape the CAPTCHA screen uses.
'test_result' => $request->session()->get('scanner_test_result'),
'enabled' => $this->config->enabled(),
// Two different reasons the connection is not this screen's to
// change: a managed configuration names the scanner, or this
// edition does not connect scanners at all. The screen says
// the same thing for both, since to the person reading it
// they are the same fact.
'managed' => $this->config->isManaged() || ! $this->canConnect(),
// Distinct from `managed`, which covers two different reasons
// the address is not editable. A managed installation still
// has a scanner worth testing; one that does not connect
// scanners at all has nothing to test, and the endpoint says
// so with a 403.
'can_test' => $this->canConnect(),
'address' => $this->config->isManaged() ? '' : $this->settings->get(Setting::VirusScannerAddress),
'max_size_mb' => $this->settings->get(Setting::VirusScanMaxSizeMb),
'unscannable_policy' => $this->settings->get(Setting::VirusUnscannablePolicy),
'scanner_down_policy' => $this->settings->get(Setting::VirusScannerDownPolicy),
'wait_minutes' => $this->settings->get(Setting::VirusScannerWaitMinutes),
'existing_rate_per_minute' => $this->settings->get(Setting::VirusScanExistingRatePerMinute),
'counts' => $this->counts(),
]);
}
public function update(Request $request): RedirectResponse
{
$validated = $request->validate([
'enabled' => ['required', 'boolean'],
'address' => ['nullable', 'string', 'max:255'],
'max_size_mb' => ['required', 'integer', 'min:0', 'max:4096'],
'unscannable_policy' => ['required', Rule::in(['allow', 'block'])],
'scanner_down_policy' => ['required', Rule::in(['allow', 'hold'])],
'wait_minutes' => ['required', 'integer', 'min:1', 'max:1440'],
'existing_rate_per_minute' => ['required', 'integer', 'min:1', 'max:6000'],
]);
// A managed installation may still choose its policies. The
// connection and the switch are not on the screen there, and a
// request that sends them anyway changes nothing.
if (! $this->config->isManaged() && $this->canConnect()) {
$address = trim((string) ($validated['address'] ?? ''));
// Refused rather than saved and quietly inert: switching this
// on with nowhere to send files would leave every upload
// waiting for a scanner that does not exist.
if ($request->boolean('enabled') && $address === '') {
return back()->withErrors(['address' => __('Enter the address of your scanner first.')]);
}
// Checked here rather than left to the socket, which accepts
// more than it should — see ScannerAddress.
if ($address !== '' && ! ScannerAddress::isValid($address)) {
return back()->withErrors(['address' => __(ScannerAddress::message())]);
}
$this->settings->set(Setting::VirusScannerAddress, $address);
$this->settings->set(Setting::VirusScanningEnabled, $request->boolean('enabled'));
}
$this->settings->set(Setting::VirusScanMaxSizeMb, (int) $validated['max_size_mb']);
$this->settings->set(Setting::VirusUnscannablePolicy, $validated['unscannable_policy']);
$this->settings->set(Setting::VirusScannerDownPolicy, $validated['scanner_down_policy']);
$this->settings->set(Setting::VirusScannerWaitMinutes, (int) $validated['wait_minutes']);
$this->settings->set(Setting::VirusScanExistingRatePerMinute, (int) $validated['existing_rate_per_minute']);
$this->activity->log(Action::SettingsUpdated, context: ['section' => 'virus_scanning']);
return back();
}
/**
* Prove the scanner is there, and that it is actually detecting.
*
* Three steps, reported separately, because "cannot connect" and
* "connects and finds nothing" are different problems and the second
* is the one that looks fine from the outside. The third sends the
* EICAR test string a harmless sequence every engine recognises by
* agreement so the answer is "it detected something" rather than
* "it did not complain".
*/
public function test(Request $request, VirusScanner $scanner): RedirectResponse
{
// Nothing to test where the connection is not this installation's
// to make.
abort_unless($this->canConnect(), 403);
$typed = trim((string) $request->input('address', ''));
// What the button is for: the address on screen, which on a first
// attempt has never been saved. Falls back to the stored one when
// the field is empty, so the button still answers on a screen
// somebody has not touched.
if ($typed !== '') {
if (! ScannerAddress::isValid($typed)) {
// Answered as a test result rather than as a field error:
// the person pressed Test, and this is what the test
// found. Nothing is dialled.
return back()->with('scanner_test_result', [
'ok' => false,
'message' => __(ScannerAddress::message()),
]);
}
$this->config->preview($typed);
}
$status = $scanner->status();
if (! $status->reachable) {
return back()->with('scanner_test_result', [
'ok' => false,
'message' => $status->error ?? __('The scanner could not be reached.'),
]);
}
$stream = fopen('php://temp', 'r+');
assert($stream !== false);
fwrite($stream, $this->eicar());
rewind($stream);
$verdict = $scanner->scan($stream, strlen($this->eicar()));
fclose($stream);
if ($verdict->outcome === ScanOutcome::Infected) {
return back()->with('scanner_test_result', [
'ok' => true,
'message' => __('Working. :engine detected the test file as ":threat".', [
'engine' => $status->engine ?? __('The scanner'),
'threat' => $verdict->detail ?? '',
]),
]);
}
// Reachable, and did not recognise a file every engine is supposed
// to. Almost always empty or broken virus definitions, which is
// exactly the failure nothing else would show.
return back()->with('scanner_test_result', [
'ok' => false,
'message' => __(':engine answered but did not detect the standard test file. Check that its virus definitions are installed and up to date.', [
'engine' => $status->engine ?? __('The scanner'),
]),
]);
}
/**
* Queue every file that has never been scanned.
*
* The work itself is the hourly command's, so this button does not
* hold a request open for a library of any size, and the pace is the
* setting above rather than "as fast as the queue will go".
*/
public function scanExisting(): RedirectResponse
{
abort_unless($this->config->enabled(), 422);
Artisan::queue('projectsend:scan-files', ['--existing' => true]);
// Onto the tab that shows it happening rather than back where they
// were: somebody who just started a scan wants to watch it, and a
// screen that looks unchanged reads as a button that did nothing.
return redirect()
->route('system-settings.virus-scanning.edit', ['tab' => 'activity'])
->with('success', __('The scan has started.'));
}
/**
* What the scanner is doing right now, and what it last decided.
*
* Polled by the Activity tab rather than rendered with the page: a
* backfill takes minutes to hours, and a screen that only tells you
* where things stood when you opened it is the screen somebody
* reloads repeatedly instead of watching.
*
* JSON rather than an Inertia partial, the way the notification bell
* and the zip builder already poll see use-notification-poll.ts.
*/
public function activity(): JsonResponse
{
$recent = File::query()
->whereNotNull('scanned_at')
->orderByDesc('scanned_at')
->limit(20)
->get(['id', 'name', 'scan_status', 'scan_note', 'scanned_at', 'scan_engine']);
$waiting = File::query()->where('scan_status', ScanStatus::Pending)->count();
// Counted as well as the files above, and this is the half that
// makes a backfill visible: re-scanning a file that already went
// out unchecked deliberately leaves it available, so it is not
// "pending" and a screen watching only that count says nothing is
// happening while the queue works through a whole library.
$queued = Queue::size('scans');
return response()->json([
// "Something is happening" is the one thing a person watching
// this screen wants to know, and it is worth being explicit
// about rather than left to be inferred from a count.
'running' => $waiting > 0 || $queued > 0,
'waiting' => $waiting,
'queued' => $queued,
'checked_last_hour' => File::query()->where('scanned_at', '>=', now()->subHour())->count(),
'last_scanned_at' => $recent->first()?->scanned_at?->toIso8601String(),
'never_scanned' => File::query()->neverScanned()->count(),
'quarantined' => File::query()->whereIn('scan_status', [
ScanStatus::Infected->value,
ScanStatus::UnscannableBlocked->value,
])->count(),
'recent' => $recent->map(fn (File $file): array => [
'id' => $file->id,
'name' => $file->name,
'status' => $file->scan_status->value,
// A reason is a key and is translated; a threat name is
// the scanner's own words and is passed through.
'note' => $this->noteFor($file),
'scanned_at' => $file->scanned_at?->toIso8601String(),
'engine' => $file->scan_engine,
])->all(),
]);
}
/**
* Whether this installation connects its own scanner.
*
* Community only, through the registry rather than an edition check
* see Capability::VirusScanningConnect for the division.
*/
private function canConnect(): bool
{
return $this->capabilities->has(Capability::VirusScanningConnect);
}
private function noteFor(File $file): ?string
{
$note = $file->scan_note;
if ($note === null) {
return $file->scan_status === ScanStatus::NotScanned
? (string) __(NotScannedReason::BeforeScanning->label())
: null;
}
$reason = NotScannedReason::tryFrom($note);
return $reason === null ? $note : (string) __($reason->label());
}
/**
* @return array<string, int>
*/
private function counts(): array
{
return [
'pending' => File::query()->where('scan_status', ScanStatus::Pending)->count(),
'quarantined' => File::query()->whereIn('scan_status', [
ScanStatus::Infected->value,
ScanStatus::UnscannableBlocked->value,
])->count(),
'never_scanned' => File::query()->neverScanned()->count(),
'let_through' => File::query()
->where('scan_status', ScanStatus::NotScanned)
->whereIn('scan_note', [
NotScannedReason::ScannerUnavailable->value,
NotScannedReason::TooLarge->value,
NotScannedReason::Encrypted->value,
])
->count(),
// What a New scan would actually check: everything except a
// file already waiting for its first verdict, and one whose
// bytes are gone.
'scannable' => File::query()
->whereNotIn('scan_status', [ScanStatus::Pending->value, ScanStatus::Missing->value])
->count(),
// So the New scan button can refuse a second scan while one is
// still working through the queue.
'queued' => Queue::size('scans'),
];
}
private function eicar(): string
{
// Assembled rather than written out, so the repository itself
// never contains the literal string: antivirus software on a
// developer's machine quarantines files that do, and a checkout
// that deletes its own test fixtures is a bad afternoon.
return 'X5O!P%@AP[4\\PZX54(P^)7CC)7}$'.'EICAR-STANDARD-'.'ANTIVIRUS-TEST-FILE!'.'$H+H*';
}
}
@@ -13,6 +13,7 @@ use App\Modules\Files\Access\DownloadAllowance;
use App\Modules\Files\Access\ViewableFileScope;
use App\Modules\Files\Jobs\BuildZipDownloadJob;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Models\ZipDownload;
use App\Modules\Files\Uploads\StoreUploadedFile;
@@ -45,6 +46,7 @@ class ZipDownloadsController extends Controller
private readonly ActivityLogger $activity,
private readonly ViewableFileScope $viewable,
private readonly DownloadAllowance $allowance,
private readonly FileAvailability $availability,
private readonly Settings $settings,
private readonly FileDelivery $delivery,
) {}
@@ -102,7 +104,11 @@ class ZipDownloadsController extends Controller
// as many times as they were meant to is the whole point of not
// hiding exhausted files.
$selected = $files->count();
$files = $files->filter(fn (File $file): bool => $this->allowance->allows($file, $user));
// A file still being checked, or quarantined, is left out of the
// selection the same way a spent allowance leaves one out: the zip
// is bytes leaving the server, and nothing unchecked goes into one.
$files = $files->filter(fn (File $file): bool => $this->availability->isAvailable($file)
&& $this->allowance->allows($file, $user));
abort_if(
$files->isEmpty() && $folders->isEmpty() && $selected > 0,
@@ -78,6 +78,18 @@ class FileResource extends JsonResource
'expires_at' => $this->expires_at?->toIso8601String(),
'expired' => $this->isExpired(),
// What the virus scanner made of this file. `pending` and
// `infected` mean the bytes are not available: the download
// endpoint answers 423 for both, and a caller that has just
// uploaded should poll this rather than the download. `note`
// carries the threat name, or why a file was not scanned.
'scan' => [
'status' => $this->scan_status->value,
'available' => $this->scan_status->isAvailable(),
'note' => $this->scan_note,
'scanned_at' => $this->scanned_at?->toIso8601String(),
],
// Null when the file may be downloaded any number of times.
// `download_limit_scope` says what the number counts —
// "total" across everyone, or "per_user" for each person
@@ -8,6 +8,7 @@ use App\Models\User;
use App\Modules\Files\Access\DownloadAllowance;
use App\Modules\Files\Access\ViewableFileScope;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Models\ZipDownload;
use App\Modules\Platform\Settings\Setting;
@@ -114,6 +115,7 @@ class BuildZipDownloadJob implements ShouldQueue
$visible = app(ViewableFileScope::class)->for($requester);
$allowance = app(DownloadAllowance::class);
$availability = app(FileAvailability::class);
try {
$relativePath = 'zips/'.$zipDownload->id.'.zip';
@@ -148,7 +150,11 @@ class BuildZipDownloadJob implements ShouldQueue
// Re-checked here for the same reason visibility is: the
// archive is built some time after it was asked for, and
// the allowance may have been spent in between.
if (! $allowance->allows($file, $requester)) {
// Availability is re-checked here for a sharper reason
// than the allowance is: a file can be quarantined between
// the request and the build, and an archive is exactly how
// an infected file would leave anyway.
if (! $availability->isAvailable($file) || ! $allowance->allows($file, $requester)) {
$skipped[] = ['id' => $file->id, 'name' => $file->name];
continue;
@@ -382,6 +388,7 @@ class BuildZipDownloadJob implements ShouldQueue
private function addFolder(ZipArchive $zip, Folder $folder, User $requester, array &$usedNames, array &$tempFiles, Builder $visible, array &$skipped, array &$added): int
{
$allowance = app(DownloadAllowance::class);
$availability = app(FileAvailability::class);
$subtreeIds = $folder->subtreeFolderIds();
/** @var Collection<int, Folder> $foldersById */
@@ -403,7 +410,7 @@ class BuildZipDownloadJob implements ShouldQueue
// inside it whose own allowance is spent — same reason the
// per-file visibility filter is re-derived rather than
// inherited from the folder.
if (! $allowance->allows($file, $requester)) {
if (! $availability->isAvailable($file) || ! $allowance->allows($file, $requester)) {
$skipped[] = ['id' => $file->id, 'name' => $file->name];
continue;
+182
View File
@@ -0,0 +1,182 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Jobs;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanOutcome;
use App\Modules\Files\Scanning\ScanPolicy;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\ScanVerdict;
use App\Modules\Files\Scanning\VirusScanner;
use Illuminate\Bus\Queueable;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Bus\Dispatchable;
use Illuminate\Queue\InteractsWithQueue;
use Illuminate\Queue\SerializesModels;
use Illuminate\Support\Facades\Log;
use Illuminate\Support\Facades\Storage;
use Throwable;
/**
* Reads one file to the scanner and records what comes back.
*
* On its own queue (`scans`) with its own worker, for the reason
* BuildZipDownloadJob has one: a 5 GB file streaming to a scanner would
* otherwise sit in front of every notification email on the default
* queue.
*
* Retries are about the scanner being down, not about the file. While it
* is unreachable the job puts itself back with a growing delay, and only
* once this installation's patience runs out does the configured policy
* decide the file's fate. An installation set to "hold" never runs out:
* the file stays pending and ScanFilesCommand keeps this job coming back.
*/
class ScanFileJob implements ShouldQueue
{
use Dispatchable, InteractsWithQueue, Queueable, SerializesModels;
/**
* Unlimited attempts, bounded by time instead see retryUntil(). A
* fixed count would give up on a scanner that is merely being
* restarted, and the file would be decided by a timeout rather than
* by the policy.
*/
public int $tries = 0;
public function __construct(
public readonly int $fileId,
/**
* A file that has already been through here one let through
* while the scanner was down, one that predates scanning, or one
* being checked again on purpose. It keeps its current state, and
* therefore stays downloadable, until a verdict actually arrives.
* Marking it pending first would take a library offline for the
* length of a backfill, and would announce every file a second
* time when it came back.
*/
public readonly bool $rescan = false,
) {
$this->onQueue('scans');
}
/**
* A day. Long enough that an overnight outage is survived by a
* "hold" installation, short enough that a job for a file somebody
* deleted does not live forever.
*/
public function retryUntil(): \DateTimeInterface
{
return now()->addDay();
}
public function handle(
VirusScanner $scanner,
ScanPolicy $policy,
ScanningConfig $config,
): void {
$file = File::query()->find($this->fileId);
if ($file === null) {
return;
}
// A new upload is only scanned while it is still pending: this job
// is dispatched from the upload and from the hourly sweep, and
// both can land on the same file.
if (! $this->rescan && $file->scan_status !== ScanStatus::Pending) {
return;
}
// A rescan checks a file again whatever it said last — after new
// definitions, or because somebody asked. The one state it leaves
// alone is a file already waiting for its first verdict, which
// belongs to the job above.
if ($this->rescan && $file->scan_status === ScanStatus::Pending) {
return;
}
if (! $config->enabled()) {
$policy->markNeverScanned($file);
return;
}
// A file identical to one already quarantined needs no second
// opinion, and asking for one would send the same malware past
// the scanner again. Checksums are already computed at upload.
$known = File::query()
->where('checksum', $file->checksum)
->where('scan_status', ScanStatus::Infected)
->whereKeyNot($file->id)
->first();
if ($known !== null) {
$policy->record($file, ScanVerdict::infected((string) $known->scan_note));
return;
}
$verdict = $this->read($file, $scanner);
if ($verdict->outcome === ScanOutcome::Unavailable && $this->keepWaiting($file, $config)) {
$file->forceFill(['scan_attempts' => $file->scan_attempts + 1])->save();
// 30 seconds, then a minute, then two, up to five. Long
// enough not to hammer a scanner that is starting up; short
// enough that a brief blip does not hold an upload for the
// whole patience window.
$this->release(min(300, 30 * (2 ** min(4, $file->scan_attempts))));
return;
}
$policy->record($file, $verdict);
}
/**
* Whether the file should wait rather than be decided now.
*
* "Hold" waits forever, by design. Otherwise the wait is measured
* from when the file was stored, not from this attempt: what the
* setting promises is that nobody's upload sits unavailable for
* longer than that, however many times the job has run.
*/
private function keepWaiting(File $file, ScanningConfig $config): bool
{
if ($config->holdsWhileUnavailable()) {
return true;
}
$storedAt = $file->created_at ?? now();
return $storedAt->copy()->addMinutes($config->unavailableWaitMinutes())->isFuture();
}
private function read(File $file, VirusScanner $scanner): ScanVerdict
{
try {
$stream = Storage::disk($file->disk)->readStream($file->path);
} catch (Throwable $e) {
$stream = null;
Log::warning("Could not open file {$file->id} for scanning: ".$e->getMessage());
}
if ($stream === null) {
// Not the scanner's fault, and not something waiting will fix
// — an orphaned row, or storage that moved. It goes through
// the same policy as a file the scanner could not open, and
// deliberately not through the scanner-unavailable path,
// which is retried hourly and would retry this forever.
return ScanVerdict::unreadable(__('The file could not be read from storage.'));
}
try {
return $scanner->scan($stream, $file->size);
} finally {
fclose($stream);
}
}
}
@@ -0,0 +1,96 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Listeners;
use App\Models\User;
use App\Modules\Files\Events\FileBecameAvailable;
use App\Modules\Files\Models\FileAssignment;
use App\Modules\Files\Versions\FileVersions;
use App\Modules\Groups\Models\Group;
use App\Modules\Notifications\NotificationDigester;
use App\Modules\Notifications\Notifier;
use Illuminate\Support\Collection;
/**
* Tells the people a file was shared with, once it can actually be had.
*
* Sharing a file that is still being scanned writes the assignment and
* says nothing (FileSharing::assign). This is the other half: when the
* scan finishes, or the file is let through, or an administrator releases
* it from quarantine, whoever it was shared with hears about it then.
*
* Recipients are derived from the assignments as they stand *now* rather
* than remembered from the moment of sharing. A share taken back while
* the file was being checked should not produce an email afterwards, and
* one added in the meantime should and deriving costs one query,
* against a table that already has to be read to answer the same question
* anywhere else.
*/
class AnnounceAvailableFile
{
public function __construct(
private readonly Notifier $notifier,
private readonly NotificationDigester $digester,
private readonly FileVersions $versions,
) {}
public function handle(FileBecameAvailable $event): void
{
$file = $event->file;
$recipients = $this->recipients($file->id);
if ($recipients->isNotEmpty()) {
$this->notifier->send('file_shared', $recipients, subject: $file, data: ['itemName' => $file->name]);
$this->digester->queue('file_shared', $recipients, $file->name, ['is_folder' => false]);
}
$previous = $file->previousVersion;
if ($previous === null) {
return;
}
// The same intersection rule the linking itself follows: only
// somebody who can see both files is told, and it is asked again
// here because while the new file was being checked the visibility
// scope hid it and the audience came out empty.
$audience = $this->versions->sharedAudience($file, $previous);
if ($audience->isNotEmpty()) {
$this->notifier->send('file_new_version', $audience, subject: $file, data: [
'itemName' => $file->name,
'previousName' => $previous->name,
]);
$this->digester->queue('file_new_version', $audience, $file->name, [
'previousName' => $previous->name,
]);
}
}
/**
* Everybody the file is assigned to, directly or through a group.
*
* @return Collection<int, User>
*/
private function recipients(int $fileId): Collection
{
return FileAssignment::query()
->where('file_id', $fileId)
->get()
->flatMap(function (FileAssignment $assignment): array {
$target = $assignment->assignable;
if ($target instanceof Group) {
return $target->members->all();
}
return $target instanceof User ? [$target] : [];
})
->unique(fn (User $user): int => $user->id)
->values();
}
}
+94
View File
@@ -0,0 +1,94 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanStatus;
use Illuminate\Support\Facades\Storage;
/**
* Rows whose bytes are not there the other half of the orphan problem.
*
* OrphanFileScanner finds bytes with no row. This finds rows with no
* bytes, which is the worse of the two: an orphan is disk space nobody
* claimed, while this is a file somebody was told they had. It happens
* when a volume is remounted somewhere else, when a backup is restored
* without its storage, when an external bucket is swapped, and when
* something deleted the bytes behind the application's back.
*
* Asked by listing each disk once and comparing, rather than by asking
* "does this exist?" per row: on object storage that would be one request
* per file, and a library of ten thousand files would answer with ten
* thousand HEADs every day.
*
* Only disks this installation can enumerate are checked, which is the
* same set the orphan scan walks. A row on any other disk is left alone
* rather than declared missing never having looked is not evidence.
*/
class MissingFileScanner
{
public function __construct(
private readonly OrphanFileScanner $orphans,
) {}
/**
* The files whose bytes are gone, as ids.
*
* @return list<int>
*/
public function scan(): array
{
$missing = [];
foreach (array_keys($this->orphans->scannedDisks()) as $diskName) {
$onDisk = array_flip(Storage::disk($diskName)->allFiles());
File::query()
->where('disk', $diskName)
->select(['id', 'path'])
->chunkById(500, function ($files) use ($onDisk, &$missing): void {
foreach ($files as $file) {
if (! isset($onDisk[$file->path])) {
$missing[] = (int) $file->id;
}
}
});
}
return $missing;
}
/**
* Files this installation has marked missing whose bytes are back.
*
* A remount, a restored backup, a bucket reconnected. Recovery is not
* optional politeness: the alternative is an administrator who fixed
* their storage and still has a library that says every file is gone.
*
* @return list<int>
*/
public function recovered(): array
{
$back = [];
foreach (array_keys($this->orphans->scannedDisks()) as $diskName) {
$onDisk = array_flip(Storage::disk($diskName)->allFiles());
File::query()
->where('disk', $diskName)
->where('scan_status', ScanStatus::Missing)
->select(['id', 'path'])
->chunkById(500, function ($files) use ($onDisk, &$back): void {
foreach ($files as $file) {
if (isset($onDisk[$file->path])) {
$back[] = (int) $file->id;
}
}
});
}
return $back;
}
}
+67 -4
View File
@@ -10,6 +10,8 @@ use App\Modules\Audit\ActivityLog;
use App\Modules\Files\Access\SharingIdentity;
use App\Modules\Files\DownloadLimitScope;
use App\Modules\Files\FileDiskCleanup;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Versions\FileVersions;
use App\Modules\Groups\Models\Group;
use App\Support\Concerns\HasUniqueSlug;
@@ -40,6 +42,14 @@ use Illuminate\Support\Carbon;
* @property string $mime_type
* @property int $size
* @property string $checksum
* @property ScanStatus $scan_status
* @property string|null $scan_note the threat name, or a NotScannedReason
* @property Carbon|null $scanned_at
* @property string|null $scan_engine
* @property int $scan_attempts
* @property bool $scan_was_available
* @property int|null $released_by
* @property Carbon|null $released_at
* @property bool $public
* @property Carbon|null $expires_at
* @property int|null $download_limit
@@ -74,6 +84,14 @@ class File extends Model
{
return [
'public' => 'boolean',
// Where this file stands with the virus scanner. Cast to the
// enum so nothing compares raw strings — see ScanStatus and
// FileAvailability.
'scan_status' => ScanStatus::class,
'scanned_at' => 'datetime',
'released_at' => 'datetime',
'scan_attempts' => 'integer',
'scan_was_available' => 'boolean',
'commentable' => 'boolean',
'expires_at' => 'datetime',
'download_limit' => 'integer',
@@ -271,6 +289,50 @@ class File extends Model
return $this->expires_at !== null && $this->expires_at->isPast();
}
/**
* Files the virus scanner has finished with, one way or another.
*
* Sits beside notExpired() in every scope that answers "what may this
* person be shown", and for the same reason: a file nobody has
* checked yet is not a file anybody may be handed. The uploader is
* the exception their own upload stays on their screen while it is
* being checked, marked as such, because a file that vanishes for ten
* minutes after you send it reads as a failed upload.
*
* @param Builder<File> $query
*/
public function scopeAvailable(Builder $query, ?User $viewer = null): void
{
$query->where(function (Builder $inner) use ($viewer): void {
$inner->whereIn('scan_status', ScanStatus::availableValues());
if ($viewer !== null) {
$inner->orWhere('uploaded_by', $viewer->id);
}
});
}
/**
* Files nothing has ever looked at.
*
* Two ways to be one, and the second is the common one: a file stored
* while scanning was off carries the reason, and a file that predates
* the scanner entirely carries none at all the migration gives the
* column its default and writes no note, and the v1 import inserts
* rows the same way. Reading only the reason missed every file on
* every real installation, which is exactly the set "Scan existing
* files" exists for.
*
* @param Builder<File> $query
*/
public function scopeNeverScanned(Builder $query): void
{
$query->where('scan_status', ScanStatus::NotScanned)
->where(fn (Builder $inner) => $inner
->whereNull('scan_note')
->orWhere('scan_note', NotScannedReason::BeforeScanning->value));
}
/**
* @param Builder<File> $query
*/
@@ -387,7 +449,7 @@ class File extends Model
$outer->orWhere('uploaded_by', $client->id);
});
$query->notExpired();
$query->notExpired()->available($client);
}
/**
@@ -428,7 +490,7 @@ class File extends Model
$outer->orWhereIn('folder_id', $subtreeFolderIds);
});
$query->notExpired();
$query->notExpired()->available();
}
/**
@@ -442,7 +504,7 @@ class File extends Model
*/
public function scopePubliclyVisibleForFolder(Builder $query, Folder $folder): void
{
$query->whereIn('folder_id', $folder->subtreeFolderIds())->notExpired();
$query->whereIn('folder_id', $folder->subtreeFolderIds())->notExpired()->available();
}
/**
@@ -493,6 +555,7 @@ class File extends Model
->where(function (Builder $folder) use ($publicFolderSubtreeIds): void {
$folder->whereNull('folder_id')->orWhereNotIn('folder_id', $publicFolderSubtreeIds);
})
->notExpired();
->notExpired()
->available();
}
}
@@ -0,0 +1,283 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use Illuminate\Support\Carbon;
use Throwable;
/**
* Talks to ClamAV's daemon, `clamd`, over a Unix socket or TCP.
*
* The protocol is small enough to own: a command is `z<COMMAND>\0`, and
* INSTREAM is that followed by length-prefixed chunks and a zero-length
* chunk to finish. Taking a library for this would be more dependency
* than code.
*
* **Three of clamd's own settings decide whether this class can tell the
* truth**, and without them a file it could not open comes back as `OK`:
* `AlertExceedsMax`, `AlertEncrypted` and its two companions turn those
* cases into answers, which arrive here as `Heuristics.Limits.Exceeded.*`
* and `Heuristics.Encrypted.*` and are mapped below to tooLarge and
* encrypted rather than to a threat. An encrypted archive full of malware
* reported as clean is the failure this exists to prevent, so the
* shipped Docker configuration sets all of them and the documentation
* says so for manual installs.
*
* Nothing here throws for a scanner that is down, slow or misconfigured:
* the caller has a policy for that, and an exception would read as a bug
* in the job rather than as the state of somebody's server.
*/
class ClamAvScanner implements VirusScanner
{
/** 64 KiB — clamd's own read buffer size, and small enough to stream 5 GB without holding it. */
private const CHUNK = 65536;
/**
* What the daemon said it was, the first time this instance asked.
*
* Every verdict records the engine and definitions that reached it, so
* a stored "clean" can be read back against what knew it. Asking on
* every scan would double the connections; asking once per instance
* means once per queue job, and the worker is recycled hourly.
*/
private ?string $engine = null;
public function __construct(
private readonly ScanningConfig $config,
) {}
public function scan(mixed $stream, int $size): ScanVerdict
{
$max = $this->config->maxScanBytes();
// Asked before opening a socket: a file this installation has
// decided not to scan should not spend a connection, and clamd
// would refuse it anyway once it passed StreamMaxLength.
if ($max > 0 && $size > $max) {
return ScanVerdict::tooLarge($this->engine());
}
if (! $this->addressIsUsable()) {
return ScanVerdict::unavailable(__(ScannerAddress::message()));
}
$socket = $this->connect();
if ($socket === null) {
return ScanVerdict::unavailable(__('The scanner could not be reached at :address.', [
'address' => $this->config->address(),
]));
}
try {
fwrite($socket, "zINSTREAM\0");
while (! feof($stream)) {
$chunk = fread($stream, self::CHUNK);
if ($chunk === false) {
return ScanVerdict::unavailable(__('The file could not be read for scanning.'));
}
if ($chunk === '') {
continue;
}
// Big-endian length, then the bytes. A short write here
// means clamd hung up mid-stream — usually its own size
// limit — and the reply below says which.
if (fwrite($socket, pack('N', strlen($chunk)).$chunk) === false) {
break;
}
}
fwrite($socket, pack('N', 0));
$reply = $this->readReply($socket);
} catch (Throwable $e) {
return ScanVerdict::unavailable($e->getMessage());
} finally {
fclose($socket);
}
if ($reply === null) {
return ScanVerdict::unavailable(__('The scanner did not answer in time.'));
}
return $this->verdictFor($reply, $this->engine());
}
public function status(): ScannerStatus
{
// Named rather than reported as "no answer". A managed address
// comes from the environment and never passed the settings
// screen's validation, so this is the only place it is checked —
// and the socket would accept a malformed one by reading the
// digits at the front of the port and ignoring the rest, which is
// how an address with a typo on the end came to look like it
// worked.
if (! $this->addressIsUsable()) {
return ScannerStatus::unreachable(__(ScannerAddress::message()));
}
$socket = $this->connect();
if ($socket === null) {
return ScannerStatus::unreachable(__('No answer from :address.', ['address' => $this->config->address()]));
}
try {
fwrite($socket, "zVERSION\0");
$reply = $this->readReply($socket);
} catch (Throwable $e) {
return ScannerStatus::unreachable($e->getMessage());
} finally {
fclose($socket);
}
if ($reply === null || $reply === '') {
return ScannerStatus::unreachable(__('The scanner did not answer in time.'));
}
// "ClamAV 1.4.1/27412/Mon Sep 15 09:12:03 2026" — engine,
// signature database number, and when that database was built.
// Older builds answer with the engine alone, so every part after
// the first is optional rather than assumed.
$parts = explode('/', $reply);
$definitions = isset($parts[1]) && is_numeric(trim($parts[1])) ? (int) trim($parts[1]) : null;
$built = null;
if (isset($parts[2])) {
try {
$built = Carbon::parse(trim($parts[2]));
} catch (Throwable) {
$built = null;
}
}
return new ScannerStatus(true, trim($parts[0]), $definitions, $built);
}
private function verdictFor(string $reply, ?string $engine): ScanVerdict
{
if (str_ends_with($reply, 'OK')) {
return ScanVerdict::clean($engine);
}
// "stream: Win.Test.EICAR_HDB-1 FOUND"
if (str_ends_with($reply, 'FOUND')) {
$threat = trim(str_replace(['stream:', 'FOUND'], '', $reply));
// Not threats: clamd's way of saying "I could not look
// inside". Which one it is decides the file's fate, and both
// are the installation's policy rather than a detection.
if (str_contains($threat, 'Heuristics.Encrypted')) {
return ScanVerdict::encrypted($engine);
}
if (str_contains($threat, 'Heuristics.Limits.Exceeded')) {
return ScanVerdict::tooLarge($engine);
}
return ScanVerdict::infected($threat === '' ? 'unknown' : $threat, $engine);
}
// "INSTREAM size limit exceeded. ERROR" — the stream was longer
// than clamd's StreamMaxLength. Same meaning as the heuristic
// above, reached when this installation's own maximum is the
// larger of the two.
if (str_contains($reply, 'size limit exceeded')) {
return ScanVerdict::tooLarge($engine);
}
return ScanVerdict::unavailable($reply);
}
/**
* "ClamAV 1.5.4/28122" engine and signature database, as recorded
* against every verdict. Null when the daemon did not say.
*/
private function engine(): ?string
{
if ($this->engine !== null) {
return $this->engine;
}
$status = $this->status();
if (! $status->reachable || $status->engine === null) {
return null;
}
return $this->engine = $status->definitionsVersion === null
? $status->engine
: $status->engine.'/'.$status->definitionsVersion;
}
/** Whether the configured address is one at all — see ScannerAddress. */
private function addressIsUsable(): bool
{
return ScannerAddress::isValid($this->config->address());
}
/** @return resource|null */
private function connect(): mixed
{
$address = $this->config->address();
if ($address === '' || ! $this->addressIsUsable()) {
return null;
}
$socket = @stream_socket_client(
$address,
$code,
$message,
$this->config->connectTimeoutSeconds(),
STREAM_CLIENT_CONNECT,
);
if ($socket === false) {
return null;
}
// Without this a scanner that accepts the connection and then
// stops answering holds the worker open indefinitely.
stream_set_timeout($socket, $this->config->replyTimeoutSeconds());
return $socket;
}
/**
* clamd's replies end with a NUL in `z` mode. Returns null when the
* socket timed out rather than answered.
*
* @param resource $socket
*/
private function readReply(mixed $socket): ?string
{
$reply = '';
while (! feof($socket)) {
$byte = fread($socket, 1);
if ($byte === false || $byte === '') {
break;
}
if ($byte === "\0") {
break;
}
$reply .= $byte;
}
if (stream_get_meta_data($socket)['timed_out']) {
return null;
}
return trim($reply);
}
}
@@ -0,0 +1,88 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use App\Modules\Files\Events\FileBecameAvailable;
use App\Modules\Files\Models\File;
use Illuminate\Support\Facades\Event;
/**
* Whether a file may be seen and served, and what happens the moment it
* may be.
*
* The one predicate every other rule asks. Three states mean yes and
* three mean no (see ScanStatus), and the reason this is a class rather
* than a comparison at each call site is that the list of "yes" states
* has already changed once `released` was added when quarantine gained
* an override and the day it changes again, it has to change in one
* place or a file becomes downloadable through one route and not another.
*
* "Available" is about everyone *other than* staff and the uploader. Staff
* see their library at all times, with each file's state on it; what
* availability governs is whether recipients and visitors see a file at
* all, and whether its bytes may leave the server.
*/
class FileAvailability
{
public function isAvailable(File $file): bool
{
return $file->scan_status->isAvailable();
}
/**
* Refuse to serve a file's bytes unless it is available.
*
* Called by every route that puts bytes on the wire the download,
* the thumbnail, the preview, the share link, the public listing and
* the zip builder. Not by the listings: a staff member's library shows
* a pending file with its state on it, and the uploader sees their own.
* What this governs is the bytes.
*
* It refuses everybody, including staff and the file's own uploader.
* A file the scanner has not cleared is not one this application
* hands out, and an administrator who wants it anyway has a way to say
* so on the record: release it from quarantine.
*
* 423 rather than 403: the refusal is about the file's state and it is
* temporary in the pending case, which is exactly what "Locked" means
* and what "Forbidden" does not. ProblemDetails renders it as JSON for
* the API, which shares these controllers.
*/
public function guardDelivery(File $file): void
{
if ($this->isAvailable($file)) {
return;
}
abort(423, match ($file->scan_status) {
ScanStatus::Pending => __('This file is still being checked for viruses.'),
// Said plainly, because it is not a refusal: there is nothing
// to serve, and whoever hits this can stop looking for a
// permission that would let them through.
ScanStatus::Missing => __('This file is no longer on the server.'),
default => __('This file is not available.'),
});
}
/**
* A file has finished being checked, one way or another.
*
* Three roads lead here and they are not interchangeable: the scan
* passed, the scanner could not be reached and this installation lets
* files through, or an administrator released it from quarantine. What
* they share is the only thing this announces the file can now be
* had by the people it was shared with, which is when everything that
* was waiting on it (a share email, a new-version notice) is allowed
* to go out.
*/
public function markAvailable(File $file): void
{
if (! $this->isAvailable($file)) {
return;
}
Event::dispatch(new FileBecameAvailable($file));
}
}
@@ -0,0 +1,38 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
/**
* Why a file carries ScanStatus::NotScanned stored in `scan_note`.
*
* Four different things to say to a person, and two of them are the
* installation's own doing rather than the file's, so a single "not
* scanned" badge with no reason would be unactionable.
*/
enum NotScannedReason: string
{
/** Bigger than the largest file this installation scans. */
case TooLarge = 'too_large';
/** An encrypted archive or document the scanner cannot open. */
case Encrypted = 'encrypted';
/** The scanner could not be reached in time, and the policy lets files through. */
case ScannerUnavailable = 'scanner_unavailable';
/** Uploaded before scanning was switched on, or while it is off. */
case BeforeScanning = 'before_scanning';
public function label(): string
{
return match ($this) {
self::TooLarge => 'Too large to scan',
self::Encrypted => 'Encrypted, so it could not be scanned',
self::ScannerUnavailable => 'The scanner could not be reached',
self::BeforeScanning => 'Uploaded before virus scanning was switched on',
};
}
}
@@ -0,0 +1,71 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use App\Models\User;
use App\Modules\Files\Models\File;
use App\Modules\Identity\Permissions\Permission;
use App\Modules\Identity\Permissions\PermissionChecker;
use App\Modules\Identity\UserType;
use App\Modules\Notifications\Notifier;
/**
* Who hears about a quarantined file.
*
* Two audiences, deliberately not three. Staff who can do something about
* it are told, because a file sitting in quarantine that nobody looks at
* is the same as a file silently lost. The person who uploaded it is
* told, because on an honest account this is how they find out their own
* machine has something on it and because otherwise their file simply
* never arrives and they have no idea why.
*
* The people the file was shared with are **not** told. They never
* received it, and a message about a virus in a file they never saw
* would alarm without informing.
*
* Recipients are resolved here rather than inside Notifier, which
* authorizes nothing by design see its security contract.
*/
class QuarantineNotifier
{
public function __construct(
private readonly Notifier $notifier,
private readonly PermissionChecker $permissions,
) {}
public function quarantined(File $file, string $threat): void
{
$uploader = $file->uploader;
$this->notifier->send('file_quarantined', $this->staff(), subject: $file, data: [
'itemName' => $file->name,
'uploaderName' => $uploader->name ?? __('a deleted account'),
'threat' => $threat,
]);
// The uploader hears it once. Without this check a staff member
// who uploaded an infected file would get both messages, which
// read as two different files.
if ($uploader !== null && ! $this->staff()->contains(fn (User $staff): bool => $staff->is($uploader))) {
$this->notifier->send('upload_blocked', [$uploader], subject: $file, data: [
'itemName' => $file->name,
'threat' => $threat,
]);
}
}
/**
* @return \Illuminate\Support\Collection<int, User>
*/
private function staff(): \Illuminate\Support\Collection
{
return User::query()
->where('type', UserType::Staff)
->where('active', true)
->get()
->filter(fn (User $staff): bool => $this->permissions->allows($staff, Permission::ReleaseQuarantinedFiles))
->values();
}
}
@@ -0,0 +1,17 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
enum ScanOutcome
{
case Clean;
case Infected;
case TooLarge;
case Encrypted;
/** The file's own bytes could not be read. Nothing to do with the scanner. */
case Unreadable;
case Unavailable;
}
+201
View File
@@ -0,0 +1,201 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\Thumbnails\ThumbnailGenerator;
use Illuminate\Support\Facades\Storage;
/**
* What a verdict means for a file, on this installation.
*
* The scanner answers a question of fact clean, infected, could not
* open it, did not answer. Three of those four are only half an answer:
* whether a file nobody could check may be handed to a client is a
* decision about somebody's business, not about the file, so it is a
* setting and it is applied here. Keeping that split is why ClamAvScanner
* knows nothing about settings and this class knows nothing about
* sockets.
*
* Every write to a file's scan columns goes through this class. They are
* not fillable and nothing else sets them.
*/
class ScanPolicy
{
public function __construct(
private readonly ScanningConfig $config,
private readonly FileAvailability $availability,
private readonly ActivityLogger $activity,
private readonly QuarantineNotifier $notifier,
) {}
/**
* Record a verdict, and return the state the file ended up in.
*
* Returns null when the verdict was "the scanner did not answer" and
* this installation waits: nothing is written, the file stays
* pending, and the caller retries.
*/
public function record(File $file, ScanVerdict $verdict): ?ScanStatus
{
return match ($verdict->outcome) {
ScanOutcome::Clean => $this->settle($file, ScanStatus::Clean, null, $verdict->engine),
ScanOutcome::Infected => $this->quarantine($file, $verdict->detail ?? 'unknown', $verdict->engine),
ScanOutcome::TooLarge => $this->unscannable($file, NotScannedReason::TooLarge, $verdict->engine),
ScanOutcome::Encrypted => $this->unscannable($file, NotScannedReason::Encrypted, $verdict->engine),
ScanOutcome::Unreadable => $this->missing($file),
ScanOutcome::Unavailable => $this->unavailable($file, $verdict->detail),
};
}
/**
* The file existed before there was a scanner, or scanning is off.
* Not a verdict, so it is never logged: nothing happened to this
* file, it simply was never looked at.
*/
public function markNeverScanned(File $file): void
{
$file->forceFill([
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::BeforeScanning->value,
])->save();
}
/**
* A threat was found. The bytes stay a scanner can be wrong, and an
* administrator may release it but nothing may reach them, and the
* thumbnails already rendered from this file have to go: they are
* derived from the same bytes and are served by their own routes.
*/
private function quarantine(File $file, string $threat, ?string $engine): ScanStatus
{
$wasAvailable = $this->availability->isAvailable($file);
$file->forceFill(['scan_was_available' => $wasAvailable])->save();
$this->settle($file, ScanStatus::Infected, $threat, $engine);
$this->purgeRenditions($file);
$this->activity->logSystem(Action::FileQuarantined, [
'id' => $file->id,
'name' => $file->name,
'threat' => $threat,
// Said out loud because it changes what an administrator has
// to do: a file that was downloadable while it waited for a
// scanner may already be on somebody's machine, and its
// download history is the only way to know.
'was_available' => $wasAvailable,
]);
$this->notifier->quarantined($file, $threat);
return ScanStatus::Infected;
}
/**
* The row is here and the bytes are not.
*
* Not a scanning verdict at all, and deliberately not run through the
* unscannable policy: "allow files nobody could scan" is a decision
* about risk, and there is no risk in a file that cannot be served.
* What there is, is a problem somebody has to look at see
* MissingFileScanner and the Files Missing screen.
*/
private function missing(File $file): ScanStatus
{
$this->settle($file, ScanStatus::Missing, null, null);
return ScanStatus::Missing;
}
/** The scanner could not open the file: too large, or encrypted. */
private function unscannable(File $file, NotScannedReason $reason, ?string $engine): ScanStatus
{
if ($this->config->blocksUnscannable()) {
$this->settle($file, ScanStatus::UnscannableBlocked, $reason->value, $engine);
$this->purgeRenditions($file);
$this->activity->logSystem(Action::FileQuarantined, [
'id' => $file->id,
'name' => $file->name,
'threat' => $reason->label(),
'was_available' => false,
]);
$this->notifier->quarantined($file, $reason->label());
return ScanStatus::UnscannableBlocked;
}
return $this->letThrough($file, $reason, $engine);
}
/** The scanner never answered. Either wait for it, or let the file go. */
private function unavailable(File $file, ?string $reason): ?ScanStatus
{
if ($this->config->holdsWhileUnavailable()) {
return null;
}
return $this->letThrough($file, NotScannedReason::ScannerUnavailable, null);
}
/**
* Allowed through without being checked.
*
* Always logged, even though it is the configured behaviour: this is
* the state where the installation looks protected and is not, and
* the log is what makes "we were unprotected between these two dates"
* answerable afterwards.
*/
private function letThrough(File $file, NotScannedReason $reason, ?string $engine): ScanStatus
{
$this->settle($file, ScanStatus::NotScanned, $reason->value, $engine);
$this->activity->logSystem(Action::FileNotScanned, [
'id' => $file->id,
'name' => $file->name,
'reason' => $reason->value,
]);
return ScanStatus::NotScanned;
}
private function settle(File $file, ScanStatus $status, ?string $note, ?string $engine): ScanStatus
{
// Asked before the write, because what the announcement means is
// "this can now be had" and a file that could already be had has
// nothing to announce. Without this, re-scanning a file that went
// out unscanned would tell its recipients a second time.
$wasAvailable = $this->availability->isAvailable($file);
$file->forceFill([
'scan_status' => $status,
'scan_note' => $note,
'scanned_at' => now(),
'scan_engine' => $engine,
])->save();
if (! $wasAvailable) {
$this->availability->markAvailable($file);
}
return $status;
}
/**
* Thumbnails and previews are cached copies of the same bytes, served
* by routes of their own, so a quarantined file with a rendition
* already on disk would still be showing part of itself.
*/
private function purgeRenditions(File $file): void
{
foreach (ThumbnailGenerator::pathsFor($file->id, $file->mime_type) as $path) {
Storage::disk('files')->delete($path);
}
}
}
+92
View File
@@ -0,0 +1,92 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
/**
* Where a file stands with the virus scanner.
*
* Availability is not a case here on purpose: three of these mean the
* file may be served and three mean it may not, and asking
* FileAvailability rather than comparing cases is what keeps that rule in
* one place. See docs/feature-virus-scanning.md.
*/
enum ScanStatus: string
{
/** Waiting to be scanned, or being scanned right now. */
case Pending = 'pending';
/** Scanned, nothing found. */
case Clean = 'clean';
/** A threat was found. Quarantined; `scan_note` is the threat name. */
case Infected = 'infected';
/** Was infected, and an administrator decided to allow it anyway. */
case Released = 'released';
/** Not checked, and allowed through. `scan_note` is a NotScannedReason. */
case NotScanned = 'not_scanned';
/** Could not be checked, and this installation blocks those. Quarantined. */
case UnscannableBlocked = 'unscannable_blocked';
/**
* The row is here and the bytes are not.
*
* Its own state rather than a kind of "not scanned", because what it
* means for the file is different: nothing can be served, so nothing
* is offered. A client listing it and getting an error on the
* download is worse than not seeing it, and staff need to see it
* precisely because somebody has to decide what to do about it.
*/
case Missing = 'missing';
/**
* Whether a file in this state may be seen and downloaded by people
* other than staff and its uploader.
*/
public function isAvailable(): bool
{
return match ($this) {
self::Clean, self::Released, self::NotScanned => true,
self::Pending, self::Infected, self::UnscannableBlocked, self::Missing => false,
};
}
/**
* The states a query may hand to somebody other than staff.
*
* @return list<string>
*/
public static function availableValues(): array
{
return array_values(array_map(
fn (self $status): string => $status->value,
array_filter(self::cases(), fn (self $status): bool => $status->isAvailable()),
));
}
/** Whether this state is waiting on an administrator's decision. */
public function isQuarantined(): bool
{
return $this === self::Infected || $this === self::UnscannableBlocked;
}
/**
* English, and the translation key what staff see on the file.
*/
public function label(): string
{
return match ($this) {
self::Pending => 'Checking for viruses',
self::Clean => 'Checked',
self::Infected => 'Quarantined',
self::Released => 'Released by an administrator',
self::NotScanned => 'Not scanned',
self::UnscannableBlocked => 'Blocked: could not be scanned',
self::Missing => 'Missing from storage',
};
}
}
@@ -0,0 +1,57 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
/**
* What a scanner answered about one file.
*
* Five outcomes rather than a boolean, because four of them are not
* "clean or not": a file the scanner refused to open, one too big for it,
* and a scanner that never answered are three different facts, and this
* installation's settings decide what each one means for the file. That
* decision lives in ScanPolicy, not here.
*/
final class ScanVerdict
{
private function __construct(
public readonly ScanOutcome $outcome,
/** The threat name, the reason a scan was refused, or null. */
public readonly ?string $detail = null,
/** Engine and definitions, as the scanner reported them. */
public readonly ?string $engine = null,
) {}
public static function clean(?string $engine = null): self
{
return new self(ScanOutcome::Clean, null, $engine);
}
public static function infected(string $threat, ?string $engine = null): self
{
return new self(ScanOutcome::Infected, $threat, $engine);
}
public static function tooLarge(?string $engine = null): self
{
return new self(ScanOutcome::TooLarge, null, $engine);
}
public static function encrypted(?string $engine = null): self
{
return new self(ScanOutcome::Encrypted, null, $engine);
}
/** The file could not be read, so nothing was scanned. */
public static function unreadable(string $reason): self
{
return new self(ScanOutcome::Unreadable, $reason);
}
/** The scanner could not be reached, or did not answer in time. */
public static function unavailable(string $reason): self
{
return new self(ScanOutcome::Unavailable, $reason);
}
}
@@ -0,0 +1,56 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
/**
* Whether a scanner address is one, and not merely one PHP will accept.
*
* `stream_socket_client()` reads a port the way `atoi` does: it takes the
* digits at the front and ignores whatever follows. So
* `tcp://clamav:3310djlkasjdlk` connects happily to port 3310, and an
* address with a typo on the end is saved, tested, and reported as
* working until the day something parses it differently. Meanwhile
* `tcp://clamav:33101` goes somewhere else entirely and fails, so the
* feedback an operator gets is inconsistent with the mistake they made.
*
* This refuses both, and says so while the field is still on screen.
*/
final class ScannerAddress
{
/**
* A TCP address: host, then a port of one to five digits and nothing
* after it. The host is a hostname, an IPv4 address, or an IPv6
* address in brackets the three forms PHP itself accepts.
*/
private const TCP = '#^tcp://(?:\[[0-9a-fA-F:]+\]|[a-zA-Z0-9._-]+):([0-9]{1,5})$#';
/** A Unix socket: an absolute path, and nothing clever. */
private const UNIX = '#^unix://(/[^\x00]+)$#';
public static function isValid(string $address): bool
{
$address = trim($address);
if (preg_match(self::UNIX, $address) === 1) {
return true;
}
if (preg_match(self::TCP, $address, $matches) !== 1) {
return false;
}
$port = (int) $matches[1];
return $port >= 1 && $port <= 65535;
}
/**
* English, and the translation key: what to type instead.
*/
public static function message(): string
{
return 'Enter the scanner as tcp://host:3310 or unix:///path/to/clamd.sock.';
}
}
@@ -0,0 +1,47 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use Illuminate\Support\Carbon;
/**
* What the scanner said about itself for the Test button, the dashboard
* warning and `projectsend:status`.
*/
final class ScannerStatus
{
public function __construct(
public readonly bool $reachable,
/** e.g. "ClamAV 1.4.1", or null when unreachable. */
public readonly ?string $engine = null,
/** The signature database number, when the scanner reports one. */
public readonly ?int $definitionsVersion = null,
public readonly ?Carbon $definitionsDate = null,
/** Why it could not be reached, for a person to act on. */
public readonly ?string $error = null,
) {}
public static function unreachable(string $error): self
{
return new self(false, error: $error);
}
/**
* How old the definitions are, in hours. Null when the scanner does
* not say absent and zero are different answers, and a caller
* warning on "older than three days" must not treat "did not say" as
* "brand new".
*/
public function definitionsAgeHours(): ?int
{
if ($this->definitionsDate === null) {
return null;
}
// diffInHours() answers with a float; whole hours is what the
// warning threshold and the status document both speak in.
return (int) $this->definitionsDate->diffInHours(now());
}
}
@@ -0,0 +1,132 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
/**
* What this installation's scanning setup actually is, once the managed
* configuration and the settings screen have both had their say.
*
* The rule is the one Captcha::resolve() already follows: an address
* named in the environment wins, and where it wins the screen stops
* offering the choice. That is how a hosted fleet points every site at one
* scanning service without a per-site setting to get wrong, and it is
* deliberately not an edition check a self-hosted operator who prefers
* to configure this in the environment gets the same behaviour. Edition
* differences flow through the capability registry; this is not one.
*
* The two policies stay editable either way. What to do with a file that
* cannot be scanned, and what to do while the scanner is down, are
* decisions about somebody's own files.
*/
class ScanningConfig
{
public function __construct(
private readonly Settings $settings,
) {}
/**
* Whether new uploads are scanned at all.
*
* Forced on under a managed configuration: a platform that supplies
* the scanner is not offering the tenant a switch for it.
*/
public function enabled(): bool
{
return $this->isManaged() || $this->settings->get(Setting::VirusScanningEnabled) === true;
}
/**
* An address to use instead of the stored one, for this request only.
*
* The Test button exists to answer "is *this* address right?", and
* the address in question is the one being typed testing what is
* saved would make the button useless exactly when it is needed, on
* the first attempt, before anything is saved. Set by
* VirusScanningSettingsController::test() and never persisted.
*/
private ?string $preview = null;
public function preview(string $address): void
{
$this->preview = trim($address);
}
public function isManaged(): bool
{
return $this->managedAddress() !== '';
}
public function address(): string
{
// Ahead of the managed address too: an operator on a managed
// installation has no field to type in, so nothing sets this
// there — and where something does, it was asked for.
if ($this->preview !== null && $this->preview !== '') {
return $this->preview;
}
if ($this->isManaged()) {
return $this->managedAddress();
}
$stored = $this->settings->get(Setting::VirusScannerAddress);
return is_string($stored) ? trim($stored) : '';
}
/**
* The largest file this installation sends to the scanner, in bytes.
* Zero means no limit of our own clamd's StreamMaxLength still
* applies, and answers with tooLarge when it is reached.
*/
public function maxScanBytes(): int
{
return max(0, (int) $this->settings->get(Setting::VirusScanMaxSizeMb)) * 1024 * 1024;
}
/** What happens to a file the scanner could not open. */
public function blocksUnscannable(): bool
{
return $this->settings->get(Setting::VirusUnscannablePolicy) === 'block';
}
/** Whether uploads wait for a scanner that is not answering. */
public function holdsWhileUnavailable(): bool
{
return $this->settings->get(Setting::VirusScannerDownPolicy) === 'hold';
}
/** How long a file waits for an unreachable scanner before the policy applies. */
public function unavailableWaitMinutes(): int
{
return max(1, (int) $this->settings->get(Setting::VirusScannerWaitMinutes));
}
/** How many already-stored files an hour-long backfill may scan per minute. */
public function existingScanRatePerMinute(): int
{
return max(1, (int) $this->settings->get(Setting::VirusScanExistingRatePerMinute));
}
public function connectTimeoutSeconds(): int
{
return max(1, (int) config('projectsend.scanning.connect_timeout', 5));
}
public function replyTimeoutSeconds(): int
{
return max(1, (int) config('projectsend.scanning.reply_timeout', 600));
}
private function managedAddress(): string
{
$address = config('projectsend.scanning.address', '');
return is_string($address) ? trim($address) : '';
}
}
@@ -0,0 +1,35 @@
<?php
declare(strict_types=1);
namespace App\Modules\Files\Scanning;
/**
* The seam between this application and whatever actually reads the
* bytes.
*
* One implementation ships (ClamAvScanner) and one more lives in the test
* suite. It exists as an interface because a commercial engine is a
* plausible later addition and because every test that is *about* policy
* what happens to a file the scanner could not open should be able to
* state the verdict rather than produce a file that provokes it.
*/
interface VirusScanner
{
/**
* Read a file and say what it is.
*
* Implementations never throw for a scanner that is down or slow:
* that is ScanVerdict::unavailable(), because the caller has a policy
* for it and an exception would look like a bug in the job.
*
* @param resource $stream the file's bytes, at position 0
* @param int $size the file's size in bytes
*/
public function scan(mixed $stream, int $size): ScanVerdict;
/**
* Whether the scanner answers, and what it is running.
*/
public function status(): ScannerStatus;
}
+14
View File
@@ -8,6 +8,7 @@ use App\Models\User;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Models\FileAssignment;
use App\Modules\Groups\Models\Group;
use App\Modules\Notifications\NotificationDigester;
@@ -35,6 +36,7 @@ class FileSharing
private readonly ActivityLogger $activity,
private readonly NotificationDigester $digester,
private readonly Notifier $notifier,
private readonly FileAvailability $availability,
) {}
/**
@@ -51,6 +53,18 @@ class FileSharing
$this->activity->log(Action::FileAssigned, subject: $file, context: ['target' => $targetName]);
// Sharing itself is never held up — the assignment above is
// written, and the file is theirs the moment it can be had. What
// waits is the telling: a file still being checked for viruses
// cannot be downloaded, so an email now would send somebody to a
// page that refuses them, and a file about to be quarantined would
// have been announced to everyone before anybody knew. The
// announcement goes out from AnnounceAvailableFile instead, on the
// event that says the file can be handed over.
if (! $this->availability->isAvailable($file)) {
return;
}
$recipients = $this->recipients($assignable);
$this->notifier->send('file_shared', $recipients, subject: $file, data: ['itemName' => $file->name]);
@@ -10,6 +10,9 @@ use Illuminate\Support\Facades\Event;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLogger;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\ScanningConfig;
/**
* The single place a stored payload becomes a File record shared by
@@ -20,6 +23,7 @@ class StoreUploadedFile
{
public function __construct(
private readonly ActivityLogger $activity,
private readonly ScanningConfig $scanning,
) {}
public function create(
@@ -37,6 +41,8 @@ class StoreUploadedFile
): File {
$originalName = self::sanitizeFilename($originalName);
$scanning = $this->scanning->enabled();
$file = File::query()->create([
'uploaded_by' => $uploader->id,
'folder_id' => $folderId,
@@ -50,6 +56,13 @@ class StoreUploadedFile
'mime_type' => $mimeType,
'size' => $size,
'checksum' => $checksum,
// Decided in the same insert as the row rather than a moment
// later: a file is unavailable from the instant it exists, or
// there is a window in which it is neither scanned nor
// withheld. Every upload path arrives here, so this is the
// only place that has to be right.
'scan_status' => $scanning ? ScanStatus::Pending : ScanStatus::NotScanned,
'scan_note' => $scanning ? null : NotScannedReason::BeforeScanning->value,
]);
$this->activity->log($action, $uploader, $file);
+5 -2
View File
@@ -151,11 +151,14 @@ class FileVersions
*
* Candidates come from the previous file's own audience rather than a
* broad user query, then each is re-checked against both files with the
* authoritative visibility scope.
* authoritative visibility scope which is also why this is public:
* while the new file is being scanned that scope hides it, so the
* audience is empty and nothing is sent. AnnounceAvailableFile asks
* again once the file can actually be had.
*
* @return Collection<int, User>
*/
private function sharedAudience(File $file, File $previous): Collection
public function sharedAudience(File $file, File $previous): Collection
{
$candidateIds = FileAssignment::query()
->where('file_id', $previous->sharingOwnerId())
@@ -13,6 +13,7 @@ use App\Modules\Files\Delivery\FileDelivery;
use App\Modules\Files\Delivery\StoredFileResponse;
use App\Modules\Files\Models\Category;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\FileAvailability;
use App\Modules\Files\Models\Folder;
use App\Modules\Files\Preview\PreviewKind;
use App\Modules\Files\Preview\PreviewLog;
@@ -81,6 +82,7 @@ class PublicGroupsController extends Controller
private readonly ActivityLogger $activity,
private readonly PreviewLog $previews,
private readonly DownloadAllowance $allowance,
private readonly FileAvailability $availability,
private readonly ThumbnailGenerator $thumbnails,
private readonly PublicThemeRegistry $themes,
private readonly CapabilityRegistry $capabilities,
@@ -192,6 +194,7 @@ class PublicGroupsController extends Controller
$this->guardSlug($publicSlug);
abort_unless($file->isEffectivelyPublic() && ! $file->isExpired(), 404);
abort_unless($this->availability->isAvailable($file), 404);
$file->loadMissing('categories');
@@ -243,6 +246,7 @@ class PublicGroupsController extends Controller
$this->guardSlug($publicSlug);
abort_unless($file->isEffectivelyPublic() && ! $file->isExpired(), 404);
abort_unless($this->availability->isAvailable($file), 404);
abort_unless(ThumbnailGenerator::supports($file->mime_type), 404);
// Always the external variant — nobody reaching a public listing is
@@ -300,6 +304,7 @@ class PublicGroupsController extends Controller
$this->guardSlug($publicSlug);
abort_unless($file->isEffectivelyPublic() && ! $file->isExpired(), 404);
abort_unless($this->availability->isAvailable($file), 404);
abort_unless($this->settings->get(Setting::PublicListingPreviewEnabled) === true, 404);
abort_if(PreviewKind::forMime($file->mime_type) === null, 404);
@@ -346,6 +351,7 @@ class PublicGroupsController extends Controller
$this->guardSlug($publicSlug);
abort_unless($file->isEffectivelyPublic() && ! $file->isExpired(), 404);
abort_unless($this->availability->isAvailable($file), 404);
// 403 rather than 404, unlike the checks above it: the file is
// genuinely here and genuinely public, it has simply been taken
@@ -35,6 +35,13 @@ enum Permission: string
// rather than a key nobody can reach.
case ModerateComments = 'moderate_comments';
// Overrule the virus scanner: let a quarantined file out. Its own key
// rather than riding on delete_files, because deciding that a threat
// report is wrong is a different judgement from deciding a file is no
// longer needed — and only the administrator role holds it by
// default. See docs/feature-virus-scanning.md.
case ReleaseQuarantinedFiles = 'release_quarantined_files';
// Categories
case CreateCategories = 'create_categories';
case EditCategories = 'edit_categories';
@@ -95,6 +102,7 @@ enum Permission: string
self::ImportOrphans => 'Import orphan files',
self::LimitDownloads => 'Limit download counts',
self::ModerateComments => 'Moderate comments',
self::ReleaseQuarantinedFiles => 'Release quarantined files',
self::CreateCategories => 'Create categories',
self::EditCategories => 'Edit categories',
self::DeleteCategories => 'Delete categories',
@@ -186,6 +194,7 @@ enum Permission: string
self::ImportOrphans,
self::LimitDownloads,
self::ModerateComments => PermissionCategory::Files,
self::ReleaseQuarantinedFiles => PermissionCategory::Files,
self::CreateCategories,
self::EditCategories,
@@ -29,6 +29,16 @@ enum Capability: string
// the bucket is provisioned, what goes in it is not.
case UsersManage = 'users.manage';
case StorageConfigure = 'storage.configure';
// Connecting this installation to a virus scanner, and being told on
// the dashboard when it has none. Community only, and the division is
// the one managed storage already draws: on a hosted installation the
// scanner is infrastructure the platform runs, so its address is not a
// tenant's to set and its absence is not a tenant's to fix. What stays
// on both editions is what to *do* with a file nobody could scan —
// that is a decision about somebody's own files, not about
// infrastructure. See docs/feature-virus-scanning.md.
case VirusScanningConnect = 'scanning.connect';
case EmailTransportConfigure = 'email.transport.configure';
case SystemUpdates = 'system.updates';
@@ -166,6 +176,7 @@ enum Capability: string
{
return match ($this) {
self::StorageConfigure,
self::VirusScanningConnect,
self::EmailTransportConfigure,
self::SystemUpdates,
self::NewsConfigure,
@@ -57,8 +57,10 @@ class SchedulerMonitoringController extends Controller
'projectsend:purge-zip-downloads' => (string) __('Purge zip downloads'),
'projectsend:check-for-updates' => (string) __('Check for updates'),
'projectsend:fetch-news' => (string) __('Fetch dashboard news'),
'projectsend:scan-files' => (string) __('Scan files for viruses'),
'projectsend:purge-expired-files' => (string) __('Purge expired files'),
'projectsend:purge-orphan-files' => (string) __('Purge orphan files'),
'projectsend:check-missing-files' => (string) __('Check for files missing from storage'),
'projectsend:purge-api-request-logs' => (string) __('Purge API request logs'),
'projectsend:purge-failed-jobs' => (string) __('Purge failed jobs'),
'projectsend:purge-notifications' => (string) __('Purge read notifications'),
@@ -8,6 +8,9 @@ use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLog;
use App\Modules\Clients\ClientStorageUsage;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanningConfig;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Identity\TwoFactor\TwoFactorEnforcement;
use App\Modules\Identity\UserType;
use App\Modules\Platform\Capabilities\CapabilityRegistry;
@@ -269,6 +272,7 @@ class StatusCommand extends Command
],
'storage' => $this->storage(),
'usage' => $this->usage(),
'scanning' => $this->scanning(),
'health' => $this->health(),
'settings' => [
// Echoed back rather than assumed: an operator writes the
@@ -333,6 +337,7 @@ class StatusCommand extends Command
$this->line('Health: '.$status['health']['pending_migrations'].' migrations pending, '
.$status['health']['failed_jobs'].' failed jobs, '
.array_sum(array_filter($status['health']['queues'], 'is_int')).' queued');
$this->line('Scanning: '.$this->scanningSummary($status['scanning']));
$this->line('Scheduler: '.($status['health']['scheduler']['last_run_at'] ?? 'never run')
.' ('.$status['health']['scheduler']['failing'].' failing)');
$this->line('Last '.self::USAGE_WINDOW_DAYS.'d: '
@@ -392,6 +397,82 @@ class StatusCommand extends Command
/**
* @return array{pending_migrations: int, failed_jobs: int, failed_jobs_latest_at: string|null, queues: array<string, int|null>, scheduler: array{last_run_at: string|null, failing: int}}
*/
/**
* Whether this installation is actually checking what it accepts.
*
* The block exists because the honest answer to "are you protected?"
* is not a boolean. Scanning can be on, the scanner unreachable, and
* every upload sailing through marked "not scanned" which is the
* configured behaviour and looks, from every screen a customer sees,
* exactly like a working installation. `let_through_24h` is the
* number that gives that away, and `reachable` is measured now rather
* than remembered.
*
* Absent, null and zero stay distinct here as everywhere else in this
* document: `enabled: false` is a decision, `reachable: null` is
* "nothing to reach because it is off", and `reachable: false` is a
* scanner that should be answering and is not.
*
* @return array<string, mixed>
*/
private function scanning(): array
{
$config = app(ScanningConfig::class);
if (! $config->enabled()) {
return [
'enabled' => false,
'managed' => $config->isManaged(),
'reachable' => null,
'engine' => null,
'definitions_age_hours' => null,
'pending' => 0,
'quarantined' => 0,
'let_through_24h' => 0,
];
}
$scanner = app(VirusScanner::class)->status();
return [
'enabled' => true,
'managed' => $config->isManaged(),
'reachable' => $scanner->reachable,
'engine' => $scanner->engine,
'definitions_age_hours' => $scanner->definitionsAgeHours(),
'pending' => File::query()->where('scan_status', ScanStatus::Pending)->count(),
'quarantined' => File::query()->whereIn('scan_status', [
ScanStatus::Infected->value,
ScanStatus::UnscannableBlocked->value,
])->count(),
// Files that went out unchecked in the last day. Zero is the
// only number that means "protected"; anything else is a
// scanner that was down, or files nobody could open.
'let_through_24h' => ActivityLog::query()
->where('action', Action::FileNotScanned)
->where('created_at', '>=', now()->subDay())
->count(),
];
}
/**
* @param array<string, mixed> $scanning
*/
private function scanningSummary(array $scanning): string
{
if ($scanning['enabled'] !== true) {
return 'off';
}
$reachable = $scanning['reachable'] === true ? 'reachable' : 'UNREACHABLE';
return "{$reachable}, {$scanning['pending']} waiting, {$scanning['quarantined']} quarantined, "
."{$scanning['let_through_24h']} let through in 24h";
}
/**
* @return array<string, mixed>
*/
private function health(): array
{
return [
@@ -405,7 +486,12 @@ class StatusCommand extends Command
'queues' => [
'default' => $this->queueDepth('default'),
'zips' => $this->queueDepth('zips'),
'scans' => $this->queueDepth('scans'),
],
// Rows whose bytes are gone. A fleet-wide jump in this is a
// storage fault, not a user one, and nothing else in this
// document would show it.
'missing_files' => File::query()->where('scan_status', ScanStatus::Missing)->count(),
'scheduler' => $this->scheduler(),
];
}
@@ -60,9 +60,43 @@ class SeedSettingsCommand extends Command
$this->seedTwoFactorEnforcement($settings, $enforcement);
}
$scanner = config('projectsend.scanning.default_address');
if (is_string($scanner) && trim($scanner) !== '') {
$this->seedScanner($settings, trim($scanner));
}
return self::SUCCESS;
}
/**
* Point a fresh installation at its scanner, and switch scanning on.
*
* For the operator who brings up the optional scanner container beside
* the application: without this they would have to find the settings
* screen and type an address the compose file already knows. Unlike
* PROJECTSEND_SCANNER_ADDRESS this leaves both the address and the
* switch editable afterwards it is a starting value, not a policy.
*
* Both are seeded together or neither: an address with scanning off
* would look configured and check nothing, and scanning on with no
* address would hold every upload.
*/
private function seedScanner(Settings $settings, string $address): void
{
// The address, asked of the table for the reason given below: its
// default is the empty string, so get() cannot tell "never set"
// from "deliberately cleared".
if (StoredSetting::query()->where('key', Setting::VirusScannerAddress->value)->exists()) {
return;
}
$settings->set(Setting::VirusScannerAddress, $address);
$settings->set(Setting::VirusScanningEnabled, true);
$this->info("Virus scanning seeded to '{$address}' and switched on (first boot).");
}
private function seedTwoFactorEnforcement(Settings $settings, string $value): void
{
if (TwoFactorEnforcement::tryFrom($value) === null) {
+51
View File
@@ -175,6 +175,27 @@ enum Setting: string
// of this stored value — whenever external storage is active; this
// feature only ever operates on the local disk (see
// PurgeOrphanFilesCommand and FileRetentionSettingsController).
// Virus scanning — see docs/feature-virus-scanning.md and
// App\Modules\Files\Scanning\ScanningConfig, which is what reads
// these (the address and the on/off switch can both be overruled by a
// managed configuration in the environment).
case VirusScanningEnabled = 'virus_scanning_enabled';
case VirusScannerAddress = 'virus_scanner_address';
case VirusScanMaxSizeMb = 'virus_scan_max_size_mb';
// 'allow' or 'block' — what happens to a file the scanner cannot open
// (too large, or encrypted). Allowing marks it "not scanned" rather
// than calling it clean.
case VirusUnscannablePolicy = 'virus_unscannable_policy';
// 'allow' or 'hold' — what happens to new uploads while the scanner
// is unreachable.
case VirusScannerDownPolicy = 'virus_scanner_down_policy';
// How long a file waits for an unreachable scanner before the policy
// above is applied.
case VirusScannerWaitMinutes = 'virus_scanner_wait_minutes';
// How fast "Scan existing files" works through a library that was
// uploaded before scanning was switched on.
case VirusScanExistingRatePerMinute = 'virus_scan_existing_rate_per_minute';
case OrphanFilesAutoDeleteEnabled = 'orphan_files_auto_delete_enabled';
case OrphanFilesDeleteAfterDays = 'orphan_files_delete_after_days';
@@ -353,6 +374,9 @@ enum Setting: string
self::ClientsCanSelectGroup,
self::TwoFactorEnforcement,
self::UploadTypeRestriction,
self::VirusScannerAddress,
self::VirusUnscannablePolicy,
self::VirusScannerDownPolicy,
self::DownloadIpLogging,
self::CommentsScope,
self::CommentsAuthors,
@@ -394,6 +418,7 @@ enum Setting: string
self::CaptchaOnPasswordReset,
self::CaptchaOnPublicComments,
self::PasswordRejectBreached,
self::VirusScanningEnabled,
self::GettingStartedPending => SettingType::Boolean,
self::ClientsAutoGroup,
@@ -410,6 +435,9 @@ enum Setting: string
self::ExpiredFilesDeleteAfterDays,
self::CommentsEditWindowMinutes,
self::OrphanFilesDeleteAfterDays,
self::VirusScanMaxSizeMb,
self::VirusScannerWaitMinutes,
self::VirusScanExistingRatePerMinute,
self::PasswordMinLength => SettingType::Integer,
self::AdminNotificationEmails,
@@ -449,6 +477,9 @@ enum Setting: string
self::PublicListingEnabled,
self::ExpiredFilesAutoDeleteEnabled,
self::PublicCommentsEnabled,
// Off until somebody points it at a scanner, or a managed
// configuration names one — see ScanningConfig.
self::VirusScanningEnabled,
self::OrphanFilesAutoDeleteEnabled => false,
self::CheckForUpdates,
@@ -482,6 +513,22 @@ enum Setting: string
self::OrphanFilesDeleteAfterDays => 30,
self::CommentsEditWindowMinutes => 15,
// Large enough for ordinary documents and archives, small
// enough that one upload does not hold the scanner for
// minutes. Must stay under clamd's own StreamMaxLength.
self::VirusScanMaxSizeMb => 512,
self::VirusScannerWaitMinutes => 10,
self::VirusScanExistingRatePerMinute => 60,
// Both of these let files through, which is the product
// owner's decision (2026-09-14): a scanner that cannot answer
// must not stop people working. What pays for it is
// visibility — every file allowed through this way is marked
// "not scanned", and the dashboard and projectsend:status both
// say so while it is happening.
self::VirusUnscannablePolicy,
self::VirusScannerDownPolicy => 'allow',
self::AdminNotificationEmails => [],
// English only out of the box: a fresh install offering
@@ -501,6 +548,10 @@ enum Setting: string
self::CommentsScope => 'all',
self::CommentsAuthors => 'staff_and_clients',
self::PublicListingSlug => 'public',
// Empty means no scanner configured here. A managed
// configuration in the environment beats this when present —
// ScanningConfig, not this default, is what a caller asks.
self::VirusScannerAddress => '',
self::DefaultLocale => '',
self::Timezone => '',
self::Theme => 'default',
+42
View File
@@ -91,6 +91,27 @@ services:
redis:
condition: service_started
# Scans get their own worker for the reason zips do: reading a 5 GB file
# to the scanner takes minutes, and on the default queue it would sit in
# front of every notification email.
worker-scans:
build:
context: .
dockerfile: docker/app/Dockerfile
args:
WWWUSER: ${WWWUSER:-1000}
WWWGROUP: ${WWWGROUP:-1000}
command: php artisan queue:work --queue=scans --tries=1
volumes:
- .:/var/www/html
- ../packages:/var/www/packages
restart: unless-stopped
depends_on:
db:
condition: service_healthy
redis:
condition: service_started
scheduler:
build:
context: .
@@ -169,6 +190,27 @@ services:
profiles:
- dev
# The virus scanner, off unless you ask for it:
# docker compose --profile scanner up -d
# then point Settings → Virus scanning at tcp://clamav:3310.
#
# It costs about 1-1.5 GB of memory, because the virus definitions are
# held in memory, and the first start downloads them before it answers.
# That is why this is a profile rather than a service everybody runs.
clamav:
image: clamav/clamav:stable
# No ports. clamd has no authentication and no encryption of any kind,
# so anything that can reach it can use it, and file contents cross
# that connection in the clear. It is reachable from the application
# on this network and from nowhere else.
volumes:
- clamav-data:/var/lib/clamav
- ./docker/clamav/clamd.conf:/etc/clamav/clamd.conf:ro
restart: unless-stopped
profiles:
- scanner
volumes:
db-data:
redis-data:
clamav-data:
+38
View File
@@ -180,6 +180,44 @@ return [
],
],
/*
|--------------------------------------------------------------------------
| Virus scanning
|--------------------------------------------------------------------------
|
| Naming an address here makes scanning *managed*: that scanner is used,
| scanning cannot be switched off from the settings screen, and the
| connection fields are hidden. It is how a hosted fleet points every
| site at one scanning service, and a self-hosted operator who would
| rather configure this in the environment than in the database can use
| it too. Everything else what to do with files that cannot be scanned,
| and what to do while the scanner is down stays a setting either way,
| because those are the site's decisions rather than the platform's.
|
| Format: unix:///var/run/clamav/clamd.sock, or tcp://host:3310.
|
*/
'scanning' => [
'address' => env('PROJECTSEND_SCANNER_ADDRESS'),
// The other way to point an installation at a scanner, and the
// opposite of the one above: written into the settings table on
// first boot and then owned by whoever administers the
// installation, who can change it or switch scanning off like any
// other setting. It is what a self-hosted operator who brings up
// the optional scanner container wants — the site arrives
// configured, without the platform taking the switch away. Ignored
// on any boot where the setting already has a value.
'default_address' => env('PROJECTSEND_SCANNER_DEFAULT_ADDRESS'),
// How long to wait for the socket, and then for each reply. A scan
// streams the whole file before the reply comes, so the second one
// has to allow for the largest file this installation accepts.
'connect_timeout' => (int) env('PROJECTSEND_SCANNER_CONNECT_TIMEOUT', 5),
'reply_timeout' => (int) env('PROJECTSEND_SCANNER_REPLY_TIMEOUT', 600),
],
/*
|--------------------------------------------------------------------------
| Release identity
@@ -0,0 +1,57 @@
<?php
declare(strict_types=1);
use Illuminate\Database\Migrations\Migration;
use Illuminate\Database\Schema\Blueprint;
use Illuminate\Support\Facades\Schema;
return new class extends Migration
{
public function up(): void
{
Schema::table('files', function (Blueprint $table) {
// Where this file stands with the virus scanner. Every rule
// about who may see or download it reads this one column
// through FileAvailability, and nothing else compares the
// strings. Existing rows default to "not scanned": they were
// uploaded before there was a scanner, which is a fact about
// them rather than a verdict.
$table->string('scan_status', 24)->default('not_scanned')->index()->after('checksum');
// The threat name when infected, or why it was not scanned.
// See ScanStatus and NotScannedReason for the two vocabularies
// this column carries.
$table->string('scan_note')->nullable()->after('scan_status');
$table->timestamp('scanned_at')->nullable()->after('scan_note');
// Engine and definitions, as the scanner reported them at the
// time. Kept so a verdict can be read back against what knew
// it — definitions change daily.
$table->string('scan_engine')->nullable()->after('scanned_at');
$table->unsignedInteger('scan_attempts')->default(0)->after('scan_engine');
// Whether this file could be downloaded before it was
// quarantined — true only for one that went out unscanned
// while the scanner was unreachable and was caught later.
// Recorded on the file because it changes what an
// administrator has to do, and because reconstructing it from
// the activity log afterwards means reading every entry.
$table->boolean('scan_was_available')->default(false)->after('scan_attempts');
// Who overruled a quarantine, and when. The reason they gave
// is in the activity log; this is what the file itself shows.
$table->foreignId('released_by')->nullable()->after('scan_attempts')->constrained('users')->nullOnDelete();
$table->timestamp('released_at')->nullable()->after('released_by');
});
}
public function down(): void
{
Schema::table('files', function (Blueprint $table) {
$table->dropConstrainedForeignId('released_by');
$table->dropIndex(['scan_status']);
$table->dropColumn(['scan_status', 'scan_note', 'scanned_at', 'scan_engine', 'scan_attempts', 'scan_was_available', 'released_at']);
});
}
};
+34
View File
@@ -0,0 +1,34 @@
# clamd settings ProjectSend depends on.
#
# The three Alert* options are not cosmetic. Without them clamd answers
# "OK" for a file it could not actually look inside — an encrypted archive,
# or one that hit a size limit — and ProjectSend would record a clean scan
# for a file nobody scanned. With them, those come back as
# Heuristics.Encrypted.* and Heuristics.Limits.Exceeded.*, which the client
# maps to "encrypted" and "too large" rather than to a threat, and this
# installation's own policy decides what happens to the file.
LogTime yes
Foreground yes
# Listen for the application on this network only. There is no
# authentication in this protocol; see the note in compose.yaml.
TCPSocket 3310
TCPAddr 0.0.0.0
# The largest stream clamd will accept. ProjectSend refuses anything over
# its own "largest file to scan" setting before it gets here, and that
# setting must not exceed this number.
StreamMaxLength 512M
MaxFileSize 512M
MaxScanSize 1024M
# Archive limits, which is where zip bombs live. Reaching one is reported
# rather than passed, thanks to AlertExceedsMax below.
MaxRecursion 16
MaxFiles 10000
AlertExceedsMax yes
AlertEncrypted yes
AlertEncryptedArchive yes
AlertEncryptedDoc yes
+26
View File
@@ -56,6 +56,13 @@ services:
SESSION_DRIVER: redis
QUEUE_CONNECTION: redis
# Uncomment together with the clamav service at the bottom of this
# file. It is written into the settings once, on first boot, so the
# site arrives configured — and it stays yours afterwards: the
# address and the switch are both on System → Settings → Virus
# scanning, and this line is ignored on every later boot.
# PROJECTSEND_SCANNER_DEFAULT_ADDRESS: tcp://clamav:3310
# Mail is easier to configure from System → Settings → Email once you
# are logged in — it has a "send test" button. These are the fallback
# until then.
@@ -116,7 +123,26 @@ services:
volumes:
- redis-data:/data
# Virus scanning, off unless you ask for it:
# docker compose --profile scanner up -d
# then point Settings → Virus scanning at tcp://clamav:3310.
#
# Budget about 1-1.5 GB of memory: the virus definitions are held in
# memory. The first start downloads them and does not answer until it
# has, which the Test button on that screen reports plainly.
clamav:
image: clamav/clamav:stable
restart: unless-stopped
# Deliberately no ports. clamd has no authentication and no
# encryption, so anything that can reach it can use it, and files
# cross that connection in the clear.
volumes:
- clamav-data:/var/lib/clamav
profiles:
- scanner
volumes:
storage:
db-data:
redis-data:
clamav-data:
+15
View File
@@ -61,6 +61,21 @@ stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; Virus scans get their own worker, for the reason zips do: reading a 5 GB
; file to the scanner takes minutes, and on the shared queue it would sit
; in front of every notification email. Harmless on an installation with no
; scanner configured — the queue is simply empty.
[program:queue-scans]
command=su-exec www-data php /var/www/html/artisan queue:work --queue=scans --max-time=3600 --tries=1
autostart=true
autorestart=true
priority=32
stopwaitsecs=3630
stdout_logfile=/dev/stdout
stdout_logfile_maxbytes=0
stderr_logfile=/dev/stderr
stderr_logfile_maxbytes=0
; schedule:work is the long-running equivalent of a per-minute cron entry,
; which is what a container should use — there is no crond here.
[program:scheduler]
+63
View File
@@ -1860,6 +1860,20 @@
"null"
]
}
},
{
"name": "scan_status",
"in": "query",
"schema": {
"anyOf": [
{
"$ref": "#/components/schemas/ScanStatus"
},
{
"type": "null"
}
]
}
}
],
"responses": {
@@ -3632,6 +3646,10 @@
"folder.made_private",
"upload.aborted",
"file.imported",
"file.quarantined",
"file.released",
"file.not_scanned",
"file.missing",
"orphan_file.deleted",
"orphan_file.auto_deleted",
"file.expired_deleted",
@@ -4060,6 +4078,36 @@
"expired": {
"type": "boolean"
},
"scan": {
"type": "object",
"description": "What the virus scanner made of this file. `pending` and\n`infected` mean the bytes are not available: the download\nendpoint answers 423 for both, and a caller that has just\nuploaded should poll this rather than the download. `note`\ncarries the threat name, or why a file was not scanned.",
"properties": {
"status": {
"type": "string"
},
"available": {
"type": "boolean"
},
"note": {
"type": [
"string",
"null"
]
},
"scanned_at": {
"type": [
"string",
"null"
]
}
},
"required": [
"status",
"available",
"note",
"scanned_at"
]
},
"download_limit": {
"type": [
"integer",
@@ -4243,6 +4291,7 @@
"commentable",
"expires_at",
"expired",
"scan",
"download_limit",
"download_limit_scope",
"downloads_used",
@@ -4324,6 +4373,20 @@
],
"title": "GroupResource"
},
"ScanStatus": {
"type": "string",
"description": "Where a file stands with the virus scanner. Availability is not a case here on purpose: three of these mean the file may be served and three mean it may not, and asking FileAvailability rather than comparing cases is what keeps that rule in one place. See docs/feature-virus-scanning.md.\n| |\n|---|\n| `pending` <br/> Waiting to be scanned, or being scanned right now. |\n| `clean` <br/> Scanned, nothing found. |\n| `infected` <br/> A threat was found. Quarantined; `scan_note` is the threat name. |\n| `released` <br/> Was infected, and an administrator decided to allow it anyway. |\n| `not_scanned` <br/> Not checked, and allowed through. `scan_note` is a NotScannedReason. |\n| `unscannable_blocked` <br/> Could not be checked, and this installation blocks those. Quarantined. |\n| `missing` <br/> The row is here and the bytes are not. Its own state rather than a kind of \"not scanned\", because what it means for the file is different: nothing can be served, so nothing is offered. A client listing it and getting an error on the download is worse than not seeing it, and staff need to see it precisely because somebody has to decide what to do about it. |",
"enum": [
"pending",
"clean",
"infected",
"released",
"not_scanned",
"unscannable_blocked",
"missing"
],
"title": "ScanStatus"
},
"StaffUserResource": {
"type": "object",
"properties": {
+13 -24
View File
@@ -5,30 +5,7 @@ import { Sidebar, SidebarContent, SidebarFooter, SidebarHeader, SidebarMenu, Sid
import { useTranslation } from '@/hooks/use-translation';
import { type NavGroup, type SharedData } from '@/types';
import { Link, usePage } from '@inertiajs/react';
import {
Activity,
ArrowLeftRight,
BookOpen,
Boxes,
Contact,
Download,
FileCode,
FileText,
FileWarning,
History,
KeyRound,
LayoutGrid,
ListChecks,
MailPlus,
MessageSquare,
Settings,
ShieldCheck,
Tags,
Upload,
UserCheck,
UserPlus,
Users,
} from 'lucide-react';
import { Activity, ArrowLeftRight, BookOpen, Boxes, Contact, Download, FileCode, FileText, FileWarning, History, KeyRound, LayoutGrid, ListChecks, MailPlus, MessageSquare, Settings, ShieldAlert, ShieldCheck, Tags, Upload, UserCheck, UserPlus, Users } from 'lucide-react';
import AppLogo from './app-logo';
export function AppSidebar() {
@@ -87,6 +64,17 @@ export function AppSidebar() {
if (can('import_orphans')) {
fileItems.push({ title: t('Import orphan files'), url: '/files/orphans', icon: FileWarning });
}
if (can('release_quarantined_files')) {
// Amber rather than the usual badge colour: the others count work
// waiting, this one counts something that went wrong.
fileItems.push({
title: t('Quarantine'),
url: '/files/quarantine',
icon: ShieldAlert,
badge: pending.quarantine,
badgeTone: 'warning',
});
}
if (can('moderate_comments')) {
// Just "Comments" — the old "Comments awaiting approval" wrapped
// and pushed its own count badge out of the sidebar, and the screen
@@ -247,6 +235,7 @@ export function AppSidebar() {
{ title: t('Storage'), url: '/system/settings/storage', when: settings && capabilities.includes('storage.configure') },
{ title: t('Downloads'), url: '/system/settings/downloads', when: settings },
{ title: t('File retention'), url: '/system/settings/file-retention', when: settings },
{ title: t('Virus scanning'), url: '/system/settings/virus-scanning', when: settings },
{ title: t('Comments'), url: '/system/settings/comments', when: settings },
{ title: t('Public listing'), url: '/system/settings/public-listing', when: settings },
{ title: t('Privacy'), url: '/system/settings/privacy', when: settings },
@@ -1,6 +1,6 @@
import { type SharedData } from '@/types';
import { usePage } from '@inertiajs/react';
import { AlertTriangle, ArrowUpCircle, HardDrive } from 'lucide-react';
import { Link, usePage } from '@inertiajs/react';
import { AlertTriangle, ArrowUpCircle, HardDrive, ShieldAlert } from 'lucide-react';
import { useState } from 'react';
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
@@ -33,6 +33,24 @@ export interface SystemInfo {
install_kind: InstallKind;
/** How downloads leave the server — see FileDeliveryDialog. */
file_delivery: FileDelivery;
/**
* Only ever present when something is wrong with virus scanning, and
* null when scanning is off. A scanner that has stopped answering
* looks, from every other screen, exactly like one that is working:
* uploads keep arriving and downloads keep working, because that is
* the configured behaviour. This is where that gets said out loud.
*/
scanning: {
/** False means no scanner is configured at all. */
configured: boolean;
reachable: boolean;
engine: string | null;
definitions_age_hours: number | null;
let_through_24h: number;
pending: number;
} | null;
/** Files in the library whose bytes are gone. Zero is the ordinary answer. */
missing_files: number;
}
/**
@@ -101,6 +119,43 @@ function StorageDurabilityNotice({ durability }: { durability: StorageDurability
return null;
}
/**
* What the scanning row says, and whether it is a warning.
*
* Four states in one line, because the row is always there: no scanner at
* all, one that is not answering, one letting files through, and one
* quietly working — which is the common case and the only one that is not
* a warning.
*/
function scanningRow(
scanning: NonNullable<SystemInfo['scanning']>,
t: (key: string, replacements?: Record<string, string | number>) => string,
): { value: string; warning: boolean; title: string } {
if (!scanning.configured) {
return {
value: t('Nothing'),
warning: true,
title: t('Uploads are passed on without being checked for viruses.'),
};
}
const engine = scanning.engine ?? t('A virus scanner');
if (!scanning.reachable) {
return { value: t(':engine (not answering)', { engine }), warning: true, title: t('The scanner could not be reached.') };
}
if (scanning.let_through_24h > 0 || scanning.pending > 0) {
return {
value: engine,
warning: true,
title: t('Some files were not checked. Open the virus scanning settings for the detail.'),
};
}
return { value: engine, warning: false, title: '' };
}
export function SystemWidget({ system, onViewReleaseNotes }: { system: SystemInfo; onViewReleaseNotes: () => void }) {
const { t } = useTranslation();
const { update_notice } = usePage<SharedData>().props;
@@ -109,12 +164,60 @@ export function SystemWidget({ system, onViewReleaseNotes }: { system: SystemInf
// Only PHP is worth flagging. The other two are the file being handed
// to the web server, which is the outcome this is watching for.
const deliveryNeedsAttention = system.file_delivery.method === 'php';
const scanning = system.scanning ? scanningRow(system.scanning, t) : null;
return (
<div>
{/* Before the update notice on purpose: losing the files outranks
being a version behind. */}
{durability && <StorageDurabilityNotice durability={durability} />}
{/* Only for a scanner that is configured and misbehaving. An
installation with no scanner at all says so on its own row
below, with the same link — two warnings for one fact would
make the card noisier without saying more. */}
{system.missing_files > 0 && (
<Alert variant="destructive" className="mb-3">
<AlertTriangle className="size-4" />
<AlertTitle>{t(':count files are missing from storage', { count: system.missing_files })}</AlertTitle>
<AlertDescription>
{t('They are listed in the library and cannot be downloaded. Their bytes are not where this installation expects them.')}
<Link href="/files/orphans?tab=missing" className="mt-1 inline-block underline hover:no-underline">
{t('See which files')}
</Link>
</AlertDescription>
</Alert>
)}
{system.scanning?.configured && scanning?.warning && (
<Alert variant="warning" className="mb-3">
<ShieldAlert className="size-4" />
<AlertTitle>
{system.scanning.reachable ? t('Files are going out unscanned') : t('The virus scanner is not answering')}
</AlertTitle>
<AlertDescription>
<ul className="list-inside list-disc">
{!system.scanning.reachable && <li>{t('Uploads cannot be checked until it is back.')}</li>}
{system.scanning.let_through_24h > 0 && (
<li>
{t(':count files were allowed through without being scanned in the last 24 hours.', {
count: system.scanning.let_through_24h,
})}
</li>
)}
{system.scanning.pending > 0 && (
<li>{t(':count files have been waiting to be checked for over an hour.', { count: system.scanning.pending })}</li>
)}
{system.scanning.definitions_age_hours !== null && system.scanning.definitions_age_hours >= 72 && (
<li>
{t('The virus definitions are :hours hours old.', { hours: system.scanning.definitions_age_hours })}
</li>
)}
</ul>
<Link href="/system/settings/virus-scanning" className="mt-1 inline-block underline hover:no-underline">
{t('Virus scanning settings')}
</Link>
</AlertDescription>
</Alert>
)}
{system.update_available && (
<Alert variant="warning" className="mb-3">
<ArrowUpCircle className="size-4" />
@@ -214,6 +317,42 @@ export function SystemWidget({ system, onViewReleaseNotes }: { system: SystemInf
)}
</dd>
</div>
{/* Same rule as the row above, and the reason this one is
never hidden: an installation checking nothing looks
exactly like one that is. Absent only where the scanner
is not this installation's to connect. */}
{scanning && (
<div className="flex justify-between gap-2">
<dt>
{scanning.warning ? (
<Link
href="/system/settings/virus-scanning"
className="font-medium text-amber-600 underline underline-offset-2 hover:no-underline dark:text-amber-500"
title={scanning.title}
>
{t('Uploads checked by')}
</Link>
) : (
<span className="text-muted-foreground">{t('Uploads checked by')}</span>
)}
</dt>
<dd>
{scanning.warning ? (
<Link
href="/system/settings/virus-scanning"
className="flex items-center gap-1.5 font-medium text-amber-600 hover:underline dark:text-amber-500"
aria-label={scanning.title}
title={scanning.title}
>
{scanning.value}
<AlertTriangle className="size-4" />
</Link>
) : (
<span>{scanning.value}</span>
)}
</dd>
</div>
)}
{/* Stated even when everything is correct: "my files are on a
host directory" is worth being able to confirm at a glance,
not only worth warning about when it is false. */}
@@ -0,0 +1,62 @@
import { Badge } from '@/components/ui/badge';
import { useTranslation } from '@/hooks/use-translation';
export interface ScanState {
status: 'pending' | 'clean' | 'infected' | 'released' | 'not_scanned' | 'unscannable_blocked' | 'missing';
/** The threat name, or why it was not scanned. Already translated. */
note: string | null;
}
/**
* What the virus scanner made of this file, for a staff member's list.
*
* Nothing at all for a clean file, which is the common case: a badge on
* every row would say nothing and cost the eye something. The four that
* do show are the ones somebody may have to act on.
*
* Recipients never see this — a file they should not have simply is not
* there. This is a staff-side affordance only.
*/
export function ScanBadge({ scan }: { scan: ScanState | null | undefined }) {
const { t } = useTranslation();
if (!scan || scan.status === 'clean') return null;
if (scan.status === 'pending') {
return (
<Badge variant="outline" className="text-[11px] font-normal" title={t('Nobody can download it until this finishes.')}>
{t('Checking')}
</Badge>
);
}
if (scan.status === 'infected' || scan.status === 'unscannable_blocked') {
return (
<Badge variant="destructive" className="text-[11px] font-normal" title={scan.note ?? undefined}>
{t('Quarantined')}
</Badge>
);
}
if (scan.status === 'missing') {
return (
<Badge variant="warning" className="text-[11px] font-normal" title={t('The file is no longer in storage.')}>
{t('Missing')}
</Badge>
);
}
if (scan.status === 'released') {
return (
<Badge variant="secondary" className="text-[11px] font-normal" title={t('An administrator released this file from quarantine.')}>
{t('Released')}
</Badge>
);
}
return (
<Badge variant="warning" className="text-[11px] font-normal" title={scan.note ?? undefined}>
{t('Not scanned')}
</Badge>
);
}
+7 -1
View File
@@ -85,7 +85,13 @@ export function NavMain({ groups = [] }: { groups: NavGroup[] }) {
)}
</SidebarMenuButton>
{item.badge !== undefined && item.badge > 0 && (
<SidebarMenuBadge className="bg-primary text-primary-foreground peer-hover/menu-button:text-primary-foreground peer-data-[active=true]/menu-button:text-primary-foreground rounded-full">
<SidebarMenuBadge
className={
item.badgeTone === 'warning'
? 'rounded-full bg-amber-500 text-amber-950 peer-hover/menu-button:text-amber-950 peer-data-[active=true]/menu-button:text-amber-950'
: 'bg-primary text-primary-foreground peer-hover/menu-button:text-primary-foreground peer-data-[active=true]/menu-button:text-primary-foreground rounded-full'
}
>
{item.badge}
</SidebarMenuBadge>
)}
+5
View File
@@ -12,6 +12,11 @@ const badgeVariants = cva(
secondary: 'border-transparent bg-secondary text-secondary-foreground hover:bg-secondary/80',
destructive: 'border-transparent bg-destructive text-destructive-foreground hover:bg-destructive/80',
success: 'border-transparent bg-success text-success-foreground hover:bg-success/80',
// Amber, matching the warning alert: something to look at,
// not something that failed. The two are different
// answers and a reader should not have to read the words
// to tell them apart.
warning: 'border-transparent bg-amber-500 text-amber-950 hover:bg-amber-500/80 dark:bg-amber-500 dark:text-amber-950',
outline: 'text-foreground',
},
},
@@ -0,0 +1,200 @@
import { router } from '@inertiajs/react';
import { CheckCircle2, Loader2, ShieldAlert, ShieldQuestion } from 'lucide-react';
import { useEffect, useState } from 'react';
import HeadingSmall from '@/components/heading-small';
import { TableShell } from '@/components/table-shell';
import { Badge } from '@/components/ui/badge';
import { useFormatDate } from '@/hooks/use-format-date';
import { useTranslation } from '@/hooks/use-translation';
/** Often enough to feel live, rarely enough to be a read every few seconds. */
const POLL_INTERVAL_MS = 4000;
interface ScannedFile {
id: number;
name: string;
status: string;
note: string | null;
scanned_at: string | null;
engine: string | null;
}
interface Activity {
running: boolean;
/** New uploads withheld until they are checked. */
waiting: number;
/** Jobs on the scans queue — a backfill lives here, not in `waiting`. */
queued: number;
checked_last_hour: number;
last_scanned_at: string | null;
never_scanned: number;
quarantined: number;
recent: ScannedFile[];
}
/**
* What the scanner is doing, refreshed while somebody is watching.
*
* A backfill runs for minutes or hours in a queue worker, where nothing
* about it is visible: this is the only place it can be watched. When
* nothing is running the same list is the record of what was decided
* last, which is what a person opening this tab after the fact is
* looking for.
*/
export function VirusScanActivity() {
const { t } = useTranslation();
const { dateTime } = useFormatDate();
const [activity, setActivity] = useState<Activity | null>(null);
const [failed, setFailed] = useState(false);
useEffect(() => {
let stopped = false;
const poll = () => {
fetch(route('system-settings.virus-scanning.activity'), {
credentials: 'same-origin',
headers: { Accept: 'application/json' },
})
.then((r) => r.json())
.then((body: Activity) => {
if (!stopped) {
setActivity(body);
setFailed(false);
}
})
.catch(() => {
// A missed poll tries again on the next tick. Only say
// so once nothing has ever arrived, or a blip would
// replace a screen full of real numbers with an error.
if (!stopped) setFailed(true);
});
};
poll();
const intervalId = window.setInterval(poll, POLL_INTERVAL_MS);
// A scan started from the Options tab lands here as a redirect
// back; refresh then rather than waiting out the interval.
const stopOnSuccess = router.on('success', poll);
return () => {
stopped = true;
window.clearInterval(intervalId);
stopOnSuccess();
};
}, []);
if (activity === null) {
return (
<p className="text-muted-foreground text-sm">
{failed ? t('Could not read what the scanner is doing.') : t('Loading…')}
</p>
);
}
const badge = (file: ScannedFile) => {
// Green, amber, red: checked and fine, checked and could not be
// read, checked and found something. The colour says which before
// the words do.
if (file.status === 'clean') {
return (
<Badge variant="success" className="gap-1 font-normal">
<CheckCircle2 className="size-3" /> {t('Clean')}
</Badge>
);
}
if (file.status === 'infected' || file.status === 'unscannable_blocked') {
return (
<Badge variant="destructive" className="gap-1 font-normal">
<ShieldAlert className="size-3" /> {file.note ?? t('Quarantined')}
</Badge>
);
}
if (file.status === 'missing') {
return (
<Badge variant="warning" className="gap-1 font-normal">
<ShieldQuestion className="size-3" /> {t('Missing from storage')}
</Badge>
);
}
if (file.status === 'released') {
return (
<Badge variant="secondary" className="font-normal">
{t('Released')}
</Badge>
);
}
return (
<Badge variant="warning" className="gap-1 font-normal">
<ShieldQuestion className="size-3" /> {file.note ?? t('Not scanned')}
</Badge>
);
};
return (
<div className="space-y-6">
<div className="rounded-lg border p-4">
<div className="flex items-center gap-2">
{activity.running && <Loader2 className="text-muted-foreground size-4 animate-spin" />}
<HeadingSmall
title={activity.running ? t('Scanning now') : t('Nothing is being scanned')}
description={
activity.running
? t(':count files still to check.', { count: Math.max(activity.waiting, activity.queued) })
: activity.last_scanned_at
? t('Last checked :when.', { when: dateTime(activity.last_scanned_at) })
: t('Nothing has been checked yet.')
}
/>
</div>
<dl className="mt-4 grid grid-cols-2 gap-x-6 gap-y-2 text-sm sm:grid-cols-5">
<div>
{/* Two different facts, and the difference matters:
an upload nobody can download yet, and work the
scanner has not reached. A backfill shows up in
the second and never in the first. */}
<dt className="text-muted-foreground">{t('Uploads held')}</dt>
<dd className="font-medium">{activity.waiting}</dd>
</div>
<div>
<dt className="text-muted-foreground">{t('In the queue')}</dt>
<dd className="font-medium">{activity.queued}</dd>
</div>
<div>
<dt className="text-muted-foreground">{t('Checked in the last hour')}</dt>
<dd className="font-medium">{activity.checked_last_hour}</dd>
</div>
<div>
<dt className="text-muted-foreground">{t('In quarantine')}</dt>
<dd className="font-medium">{activity.quarantined}</dd>
</div>
<div>
<dt className="text-muted-foreground">{t('Never scanned')}</dt>
<dd className="font-medium">{activity.never_scanned}</dd>
</div>
</dl>
</div>
<TableShell
columns={[t('File'), t('Result'), t('Checked')]}
isEmpty={activity.recent.length === 0}
emptyMessage={<>{t('No file has been checked yet.')}</>}
>
{activity.recent.map((file) => (
<tr key={file.id} className="border-b last:border-0">
<td className="px-4 py-2.5 font-medium">{file.name}</td>
<td className="px-4 py-2.5">{badge(file)}</td>
<td className="text-muted-foreground px-4 py-2.5">{dateTime(file.scanned_at)}</td>
</tr>
))}
</TableShell>
{failed && <p className="text-muted-foreground text-xs">{t('The last refresh did not go through. Still trying.')}</p>}
</div>
);
}
+52 -9
View File
@@ -1,10 +1,11 @@
import { type BreadcrumbItem } from '@/types';
import { Head, router, useForm, usePage } from '@inertiajs/react';
import { Check, Copy, Download, Eye, File as FileIcon, Loader2, X } from 'lucide-react';
import { Head, Link, router, useForm, usePage } from '@inertiajs/react';
import { Check, Copy, Download, Eye, File as FileIcon, Loader2, ShieldAlert, X } from 'lucide-react';
import { FormEventHandler, useEffect, useState } from 'react';
import { CommentThread } from '@/components/comments/comment-thread';
import { ConfirmDialog } from '@/components/confirm-dialog';
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
import { FilePreviewDialog } from '@/components/file-preview-dialog';
import { FileVersionField, type ChainEntry, type VersionLink } from '@/components/files/file-version-field';
import { InheritedSharingNotice, type SharingRoot } from '@/components/files/inherited-sharing-notice';
@@ -93,6 +94,11 @@ interface FilesEditProps {
slug: string;
expires_at: string | null;
expired: boolean;
/** What the virus scanner made of it — see the notice at the top of this page. */
scan_status?: string;
scan_note?: string | null;
/** Whether there are bytes to hand over: false hides everything that would ask for them. */
scan_available?: boolean;
download_limit: number | null;
download_limit_scope: string | null;
downloads_used: number;
@@ -304,9 +310,41 @@ export default function FilesEdit({
<Head title={file.name} />
<div className="px-4 py-6">
{/* The library no longer lists a quarantined or missing
file, so this is where somebody arrives from Quarantine
or from Files missing from storage — and the page is
full of buttons that will refuse. Say why, once, at the
top. */}
{(file.scan_status === 'infected' || file.scan_status === 'unscannable_blocked') && (
<Alert variant="destructive" className="mb-4">
<ShieldAlert className="size-4" />
<AlertTitle>{t('This file is in quarantine')}</AlertTitle>
<AlertDescription>
{t('The virus scanner reported: :threat. Nobody can download it, and it is not listed in the library.', {
threat: file.scan_note ?? t('a threat'),
})}
<Link href="/files/quarantine" className="mt-1 inline-block underline hover:no-underline">
{t('Quarantine')}
</Link>
</AlertDescription>
</Alert>
)}
{file.scan_status === 'missing' && (
<Alert variant="warning" className="mb-4">
<ShieldAlert className="size-4" />
<AlertTitle>{t('This file is missing from storage')}</AlertTitle>
<AlertDescription>
{t('The record is here and the file itself is not, so nothing can be downloaded. It is not listed in the library.')}
<Link href="/files/orphans?tab=missing" className="mt-1 inline-block underline hover:no-underline">
{t('Files missing from storage')}
</Link>
</AlertDescription>
</Alert>
)}
<div className="flex items-start justify-between">
<div className="flex items-start gap-4">
{isPreviewable(file.mime_type) && (
{file.scan_available !== false && isPreviewable(file.mime_type) && (
<FilePreviewDialog
previewUrl={route('files.preview', file.id)}
mimeType={file.mime_type}
@@ -335,12 +373,17 @@ export default function FilesEdit({
/>
</div>
<div className="flex gap-2">
<Button variant="outline" asChild>
<a href={route('files.download', file.id)}>
<Download className="size-4" />
{t('Download')}
</a>
</Button>
{/* Offered only when there are bytes to hand over.
The notice above says why, so a missing button
is not a mystery. */}
{file.scan_available !== false && (
<Button variant="outline" asChild>
<a href={route('files.download', file.id)}>
<Download className="size-4" />
{t('Download')}
</a>
</Button>
)}
{can_delete && (
<ConfirmDialog
trigger={
+5
View File
@@ -13,6 +13,7 @@ import { DetailsPanel, DetailsTarget } from '@/components/details-panel';
import { DragChip, DragData, DropZone, useFolderDrop, useRowDrag } from '@/components/file-dnd';
import { FilePreviewDialog } from '@/components/file-preview-dialog';
import { PreviewAction } from '@/components/preview-action';
import { ScanBadge, type ScanState } from '@/components/files/scan-badge';
import { VersionBadge, type VersionLinks } from '@/components/files/version-badge';
import Heading from '@/components/heading';
import { FilterField, ListToolbar } from '@/components/list-toolbar';
@@ -60,6 +61,8 @@ interface FileRow {
public: boolean;
public_url: string | null;
expired: boolean;
/** What the virus scanner made of it. Null when scanning is off. */
scan: ScanState | null;
assignments_count: number;
downloads_count: number;
/**
@@ -811,6 +814,7 @@ function FileRow({
{t('Expired')}
</Badge>
)}
<ScanBadge scan={row.scan} />
<VersionBadge version={row.version} />
</div>
<p className="text-muted-foreground text-xs">
@@ -1053,6 +1057,7 @@ function FileCard({
{t('Expired')}
</Badge>
)}
<ScanBadge scan={row.scan} />
<VersionBadge version={row.version} />
</div>
<p className="text-muted-foreground truncate text-xs">
Binary file not shown.
+183
View File
@@ -0,0 +1,183 @@
import { type BreadcrumbItem, type SharedData } from '@/types';
import { Head, Link, useForm, usePage } from '@inertiajs/react';
import { ShieldAlert } from 'lucide-react';
import { useState } from 'react';
import Heading from '@/components/heading';
import InputError from '@/components/input-error';
import { Pagination, PaginationMeta } from '@/components/pagination';
import { TableShell } from '@/components/table-shell';
import { Alert, AlertDescription } from '@/components/ui/alert';
import { Badge } from '@/components/ui/badge';
import { Button } from '@/components/ui/button';
import {
Dialog,
DialogContent,
DialogDescription,
DialogFooter,
DialogHeader,
DialogTitle,
DialogTrigger,
} from '@/components/ui/dialog';
import { Label } from '@/components/ui/label';
import { Textarea } from '@/components/ui/textarea';
import { useFormatDate } from '@/hooks/use-format-date';
import { useTranslation } from '@/hooks/use-translation';
import AppLayout from '@/layouts/app-layout';
import { formatBytes } from '@/lib/format-bytes';
interface QuarantinedFile {
id: number;
name: string;
original_name: string;
size: number;
uploader: string | null;
threat: string | null;
status: string;
scanned_at: string | null;
/** It could be downloaded before it was flagged — so somebody may already have it. */
was_available: boolean;
downloads_count: number;
}
interface QuarantineProps {
files: QuarantinedFile[];
pagination: PaginationMeta;
}
function ReleaseDialog({ file }: { file: QuarantinedFile }) {
const { t } = useTranslation();
const [open, setOpen] = useState(false);
const { data, setData, post, processing, errors, reset } = useForm({ reason: '' });
return (
<Dialog
open={open}
onOpenChange={(next) => {
setOpen(next);
if (!next) reset();
}}
>
<DialogTrigger asChild>
<Button variant="outline" size="sm">
{t('Release')}
</Button>
</DialogTrigger>
<DialogContent>
<DialogHeader>
<DialogTitle>{t('Release ":name"?', { name: file.name })}</DialogTitle>
<DialogDescription>
{t(
'The scanner reported a threat in this file. Releasing it makes it downloadable again for everyone it was shared with. Only do this if you are sure the report is wrong.',
)}
</DialogDescription>
</DialogHeader>
<div className="grid gap-2">
<Label htmlFor="reason">{t('Why are you releasing it?')}</Label>
<Textarea id="reason" value={data.reason} onChange={(e) => setData('reason', e.target.value)} required />
<p className="text-muted-foreground text-xs">{t('This is recorded in the activity log, with your name.')}</p>
<InputError message={errors.reason} />
</div>
<DialogFooter>
<Button variant="ghost" type="button" onClick={() => setOpen(false)}>
{t('Cancel')}
</Button>
<Button
type="button"
disabled={processing || data.reason.trim() === ''}
onClick={() =>
post(route('files.release', file.id), {
preserveScroll: true,
onSuccess: () => setOpen(false),
})
}
>
{t('Release file')}
</Button>
</DialogFooter>
</DialogContent>
</Dialog>
);
}
export default function Quarantine({ files, pagination }: QuarantineProps) {
const { t } = useTranslation();
const { dateTime } = useFormatDate();
const { auth } = usePage<SharedData>().props;
const breadcrumbs: BreadcrumbItem[] = [
{ title: t('All files'), href: '/files' },
{ title: t('Quarantine'), href: '/files/quarantine' },
];
return (
<AppLayout breadcrumbs={breadcrumbs}>
<Head title={t('Quarantine')} />
<div className="px-4 py-6">
<div className="flex items-start justify-between">
<Heading title={t('Quarantine')} description={t('Files the virus scanner refused. Nobody can download these.')} />
{/* The way back to the screen that decides what gets
refused, for whoever may change it. */}
{auth.permissions.includes('edit_settings') && (
<Button variant="outline" asChild>
<Link href={route('system-settings.virus-scanning.edit')}>{t('Virus scanning settings')}</Link>
</Button>
)}
</div>
{files.some((file) => file.was_available) && (
<Alert variant="destructive" className="mb-4">
<ShieldAlert className="size-4" />
<AlertDescription>
{t(
'Some of these were downloadable before they were checked, because the scanner was unreachable at the time. Their download history shows whether anybody took a copy.',
)}
</AlertDescription>
</Alert>
)}
<TableShell
columns={[t('File'), t('Uploaded by'), t('Found'), t('Detected'), null]}
isEmpty={files.length === 0}
emptyMessage={<>{t('Nothing is in quarantine.')}</>}
>
{files.map((file) => (
<tr key={file.id} className="border-b last:border-0">
<td className="px-4 py-2.5">
{/* The file itself, for somebody deciding
whether the scanner is right: who it was
shared with, where it came from. It is no
longer listed in the library. */}
<Link href={route('files.edit', file.id)} className="font-medium hover:underline">
{file.name}
</Link>
<div className="text-muted-foreground text-xs">
{file.original_name} · {formatBytes(file.size)}
</div>
</td>
<td className="text-muted-foreground px-4 py-2.5">{file.uploader ?? t('(deleted account)')}</td>
<td className="px-4 py-2.5">
<Badge variant="destructive">{file.threat ?? t('Unknown')}</Badge>
{file.was_available && (
<div className="text-muted-foreground mt-1 text-xs">
{t('Was downloadable before this. Downloads so far: :count', { count: file.downloads_count })}
</div>
)}
</td>
<td className="text-muted-foreground px-4 py-2.5">{dateTime(file.scanned_at)}</td>
<td className="px-4 py-2.5 text-right">
<ReleaseDialog file={file} />
</td>
</tr>
))}
</TableShell>
<Pagination meta={pagination} />
</div>
</AppLayout>
);
}
+14 -4
View File
@@ -8,7 +8,7 @@ import AuthLayout from '@/layouts/auth-layout';
import { formatBytes } from '@/lib/format-bytes';
interface ShareShowProps {
status: 'active' | 'expired' | 'limit_reached' | 'not_found';
status: 'active' | 'expired' | 'limit_reached' | 'not_found' | 'checking' | 'unavailable';
file?: {
original_name: string;
size: number;
@@ -26,11 +26,21 @@ export default function ShareShow({ status, file, download_url }: ShareShowProps
? t('This link has expired.')
: status === 'limit_reached'
? t('This link has reached its download limit.')
: t("This link doesn't exist or has been revoked.");
: // A link can exist before its file has been checked for
// viruses — on some installations one is created the
// moment a file is uploaded — so this is "come back in a
// minute", not "something is wrong".
status === 'checking'
? t('This file is still being checked for viruses. Please try again in a few minutes.')
: status === 'unavailable'
? t('This file is not available.')
: t("This link doesn't exist or has been revoked.");
const title = status === 'checking' ? t('Almost ready') : t('Link unavailable');
return (
<AuthLayout title={t('Link unavailable')} description={description}>
<Head title={t('Link unavailable')} />
<AuthLayout title={title} description={description}>
<Head title={title} />
</AuthLayout>
);
}
@@ -0,0 +1,360 @@
import { type BreadcrumbItem, type SharedData } from '@/types';
import { Head, Link, router, useForm, usePage } from '@inertiajs/react';
import { CheckCircle2, ShieldAlert, TriangleAlert } from 'lucide-react';
import { FormEventHandler } from 'react';
import Heading from '@/components/heading';
import HeadingSmall from '@/components/heading-small';
import InputError from '@/components/input-error';
import { SaveButton } from '@/components/save-button';
import { Alert, AlertDescription, AlertTitle } from '@/components/ui/alert';
import { Button } from '@/components/ui/button';
import { Checkbox } from '@/components/ui/checkbox';
import { Input } from '@/components/ui/input';
import { Label } from '@/components/ui/label';
import { Select, SelectContent, SelectItem, SelectTrigger, SelectValue } from '@/components/ui/select';
import { VirusScanActivity } from '@/components/virus-scan-activity';
import { useTranslation } from '@/hooks/use-translation';
import AppLayout from '@/layouts/app-layout';
type Tab = 'scanner' | 'options' | 'activity';
interface VirusScanningProps {
tab: Tab;
/** The Test button's answer, carried through the session. */
test_result: { ok: boolean; message: string } | null;
enabled: boolean;
/** The scanner is supplied by the platform: no address to set, and no switch. */
managed: boolean;
/** Whether this installation connects its own scanner, and so has one to test. */
can_test: boolean;
address: string;
max_size_mb: number;
unscannable_policy: 'allow' | 'block';
scanner_down_policy: 'allow' | 'hold';
wait_minutes: number;
existing_rate_per_minute: number;
counts: {
pending: number;
quarantined: number;
never_scanned: number;
let_through: number;
/** What a New scan would check: everything with bytes and a verdict to revisit. */
scannable: number;
/** Jobs still on the scans queue: a scan already under way. */
queued: number;
};
}
export default function VirusScanningSettings({
tab,
test_result,
enabled,
managed,
can_test,
address,
max_size_mb,
unscannable_policy,
scanner_down_policy,
wait_minutes,
existing_rate_per_minute,
counts,
}: VirusScanningProps) {
const { t } = useTranslation();
const { auth } = usePage<SharedData>().props;
const breadcrumbs: BreadcrumbItem[] = [
{ title: t('Settings'), href: '/system/settings' },
{ title: t('Virus scanning'), href: '/system/settings/virus-scanning' },
];
// One form behind both tabs, and one Save. The server takes every
// field on every save, so switching tabs never loses what was typed on
// the other one.
const { data, setData, patch, errors, processing, recentlySuccessful } = useForm({
enabled,
address,
max_size_mb,
unscannable_policy,
scanner_down_policy,
wait_minutes,
existing_rate_per_minute,
});
const submit: FormEventHandler = (e) => {
e.preventDefault();
patch(route('system-settings.virus-scanning.update', tab === 'options' ? { tab: 'options' } : {}), { preserveScroll: true });
};
const tabs: { key: Tab; label: string }[] = [
{ key: 'scanner', label: t('Scanner') },
{ key: 'options', label: t('Options') },
{ key: 'activity', label: t('Activity') },
];
return (
<AppLayout breadcrumbs={breadcrumbs}>
<Head title={t('Virus scanning')} />
<div className="space-y-6 px-4 py-6">
<div className="flex items-start justify-between">
<Heading title={t('Virus scanning')} description={t('Uploaded files are checked before anyone can download them')} />
{/* The screen this one leads to: whatever the scanner
actually refused. Only for somebody who may act on
it — the quarantine screen answers 403 otherwise,
and a button that leads to a refusal is worse than
no button. */}
<div className="flex items-start gap-2">
{auth.permissions.includes('release_quarantined_files') && (
<Button variant="outline" asChild>
<Link href={route('files.quarantine')}>
{counts.quarantined > 0
? t('Quarantine (:count)', { count: counts.quarantined })
: t('Quarantine')}
</Link>
</Button>
)}
{/* The screen's one action, where an action belongs.
Disabled rather than hidden when there is nothing
to do, so it can say why. */}
<Button
type="button"
disabled={!enabled || counts.scannable === 0 || counts.queued > 0}
title={
!enabled
? t('Switch scanning on first.')
: counts.queued > 0
? t('A scan is already running.')
: counts.scannable === 0
? t('There is nothing to scan.')
: t('Checks all :count files again, including the ones already checked.', {
count: counts.scannable,
})
}
onClick={() => router.post(route('system-settings.virus-scanning.scan-existing'), {}, { preserveScroll: false })}
>
{t('New scan')}
</Button>
</div>
</div>
{counts.let_through > 0 && (
<Alert variant="destructive" className="max-w-xl">
<TriangleAlert className="size-4" />
<AlertTitle>{t(':count files were allowed through without being scanned', { count: counts.let_through })}</AlertTitle>
<AlertDescription>
{t(
'They are marked "not scanned" and can be downloaded. This happens when a file is too large or encrypted, or when the scanner could not be reached.',
)}
</AlertDescription>
</Alert>
)}
<div className="border-border flex gap-1 border-b">
{tabs.map(({ key, label }) => (
<Link
key={key}
href={route('system-settings.virus-scanning.edit', key === 'scanner' ? {} : { tab: key })}
preserveScroll
className={`-mb-px border-b-2 px-3 py-2 text-sm ${
tab === key ? 'border-primary text-foreground font-medium' : 'text-muted-foreground border-transparent'
}`}
>
{label}
</Link>
))}
</div>
{tab === 'scanner' && (
<div className="max-w-xl space-y-6">
<form onSubmit={submit} className="space-y-6">
{managed ? (
<Alert>
<ShieldAlert className="size-4" />
<AlertTitle>{t('Scanning is managed for you')}</AlertTitle>
<AlertDescription>
{t(
'Every upload on this site is scanned. The scanner itself is run for you, so there is nothing to connect here.',
)}
</AlertDescription>
</Alert>
) : (
<>
<div className="grid gap-2">
<div className="flex items-center gap-2">
<Checkbox
id="enabled"
checked={data.enabled}
onCheckedChange={(checked) => setData('enabled', checked === true)}
/>
<Label htmlFor="enabled" className="font-normal">
{t('Scan uploaded files for viruses')}
</Label>
</div>
<InputError message={errors.enabled} />
</div>
<div className="grid gap-2">
<Label htmlFor="address">{t('Scanner address')}</Label>
<Input
id="address"
value={data.address}
onChange={(e) => setData('address', e.target.value)}
placeholder="tcp://clamav:3310"
/>
<p className="text-muted-foreground text-sm">
{t('A ClamAV daemon, as tcp://host:3310 or unix:///path/to/clamd.sock.')}
</p>
<InputError message={errors.address} />
</div>
</>
)}
{/* Under the address it tests, and above the
Save button, which nothing goes below. */}
{can_test && (
<div className="space-y-3 rounded-lg border p-4">
<HeadingSmall
title={t('Check the connection')}
description={t('Sends the standard test file, which is harmless and every scanner recognises.')}
/>
<Button
type="button"
variant="outline"
className="w-fit"
// The address on screen, not the one on
// file: the question is whether what is
// being typed works.
onClick={() =>
router.post(
route('system-settings.virus-scanning.test'),
{ address: data.address },
{ preserveScroll: true },
)
}
>
{t('Test scanner')}
</Button>
{test_result && (
<Alert variant={test_result.ok ? 'default' : 'destructive'}>
{test_result.ok ? <CheckCircle2 className="size-4" /> : <TriangleAlert className="size-4" />}
<AlertDescription>{test_result.message}</AlertDescription>
</Alert>
)}
</div>
)}
<SaveButton processing={processing} recentlySuccessful={recentlySuccessful} />
</form>
</div>
)}
{tab === 'options' && (
<div className="max-w-xl space-y-6">
<form onSubmit={submit} className="space-y-6">
<div className="grid gap-2">
<Label htmlFor="max_size_mb">{t('Largest file to scan (MB)')}</Label>
<Input
id="max_size_mb"
type="number"
min={0}
max={4096}
value={data.max_size_mb}
onChange={(e) => setData('max_size_mb', Number(e.target.value))}
/>
<p className="text-muted-foreground text-sm">
{t(
'Bigger files are handled by the rule below. Your scanner has its own limit too, and this should not exceed it.',
)}
</p>
<InputError message={errors.max_size_mb} />
</div>
<div className="grid gap-3">
<HeadingSmall
title={t('Files that cannot be scanned')}
description={t('Too large, or an encrypted archive or document the scanner cannot open.')}
/>
<Select
value={data.unscannable_policy}
onValueChange={(value: string) => setData('unscannable_policy', value as 'allow' | 'block')}
>
<SelectTrigger id="unscannable_policy" className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="allow">{t('Allow them, marked "not scanned"')}</SelectItem>
<SelectItem value="block">{t('Block them, like an infected file')}</SelectItem>
</SelectContent>
</Select>
<InputError message={errors.unscannable_policy} />
</div>
<div className="grid gap-3">
<HeadingSmall
title={t('If the scanner cannot be reached')}
description={t('What happens to new uploads while the scanner is down.')}
/>
<Select
value={data.scanner_down_policy}
onValueChange={(value: string) => setData('scanner_down_policy', value as 'allow' | 'hold')}
>
<SelectTrigger id="scanner_down_policy" className="w-full">
<SelectValue />
</SelectTrigger>
<SelectContent>
<SelectItem value="allow">{t('Allow them after the wait below, marked "not scanned"')}</SelectItem>
<SelectItem value="hold">{t('Hold them until the scanner is back')}</SelectItem>
</SelectContent>
</Select>
<p className="text-muted-foreground text-sm">
{t('Files allowed through this way are scanned again automatically once the scanner answers.')}
</p>
<InputError message={errors.scanner_down_policy} />
</div>
<div className="grid gap-2">
<Label htmlFor="wait_minutes">{t('How long to wait for the scanner (minutes)')}</Label>
<Input
id="wait_minutes"
type="number"
min={1}
max={1440}
value={data.wait_minutes}
onChange={(e) => setData('wait_minutes', Number(e.target.value))}
/>
<InputError message={errors.wait_minutes} />
</div>
<div className="grid gap-2">
<Label htmlFor="existing_rate_per_minute">{t('Files to scan per minute')}</Label>
<Input
id="existing_rate_per_minute"
type="number"
min={1}
max={6000}
className="w-32"
value={data.existing_rate_per_minute}
onChange={(e) => setData('existing_rate_per_minute', Number(e.target.value))}
/>
<p className="text-muted-foreground text-sm">
{t('The pace of a New scan, so working through a whole library does not starve the scanner of new uploads.')}
</p>
<InputError message={errors.existing_rate_per_minute} />
</div>
<SaveButton processing={processing} recentlySuccessful={recentlySuccessful} />
</form>
</div>
)}
{/* Mounted only while the tab is open, which is also what
starts and stops its polling. */}
{tab === 'activity' && <VirusScanActivity />}
</div>
</AppLayout>
);
}
+8
View File
@@ -24,6 +24,12 @@ export interface NavItem {
isActive?: boolean;
items?: NavItem[];
badge?: number;
/**
* What the badge is saying. The default reads as "there is work
* here"; "warning" reads as "something is wrong here", which is the
* difference between a queue and a quarantine.
*/
badgeTone?: 'default' | 'warning';
/**
* Leaves this installation. Rendered as a plain anchor opening in a
* new tab rather than an Inertia <Link>, which would try to fetch a
@@ -130,6 +136,8 @@ export interface SharedData {
membership_requests?: number;
/** Comments held for approval anywhere in this viewer's library. */
comments?: number;
/** Files the scanner refused, waiting for somebody to decide. */
quarantine?: number;
notifications_unread?: number;
};
update_notice: {
+7
View File
@@ -21,8 +21,15 @@ Schedule::command('projectsend:expire-client-accounts')->hourly();
Schedule::command('projectsend:purge-zip-downloads')->daily();
Schedule::command('projectsend:check-for-updates')->daily();
Schedule::command('projectsend:fetch-news')->daily();
// Hourly, not daily: a file stuck waiting for a scanner is a file nobody
// can download, and an installation set to hold has no other way forward.
Schedule::command('projectsend:scan-files')->hourly();
Schedule::command('projectsend:purge-expired-files')->daily();
Schedule::command('projectsend:purge-orphan-files')->daily();
// The other half of the orphan question: rows whose bytes are gone. Daily
// and independent of virus scanning, because an installation with no
// scanner has exactly the same problem.
Schedule::command('projectsend:check-missing-files')->daily();
Schedule::command('projectsend:purge-api-request-logs')->daily();
Schedule::command('projectsend:purge-failed-jobs')->daily();
Schedule::command('projectsend:purge-notifications')->daily();
+15
View File
@@ -7,6 +7,7 @@ use App\Modules\Comments\Http\Controllers\CommentSettingsController;
use App\Modules\Files\Http\Controllers\DownloadSettingsController;
use App\Modules\Files\Http\Controllers\FileRetentionSettingsController;
use App\Modules\Files\Http\Controllers\UploadSettingsController;
use App\Modules\Files\Http\Controllers\VirusScanningSettingsController;
use App\Modules\Identity\Http\Controllers\ApiTokensController;
use App\Modules\Identity\Http\Controllers\ConnectedAccountsController;
use App\Modules\Identity\Http\Controllers\LdapSettingsController;
@@ -185,6 +186,20 @@ Route::middleware('auth')->group(function () {
Route::patch('system/settings/downloads', [DownloadSettingsController::class, 'update'])->name('system-settings.downloads.update');
Route::get('system/settings/file-retention', [FileRetentionSettingsController::class, 'edit'])->name('system-settings.file-retention.edit');
Route::patch('system/settings/file-retention', [FileRetentionSettingsController::class, 'update'])->name('system-settings.file-retention.update');
Route::get('system/settings/virus-scanning', [VirusScanningSettingsController::class, 'edit'])->name('system-settings.virus-scanning.edit');
Route::patch('system/settings/virus-scanning', [VirusScanningSettingsController::class, 'update'])->name('system-settings.virus-scanning.update');
// Named buckets, like the email and CAPTCHA test buttons beside
// them: both of these reach out or start work, and neither should
// be a button somebody can sit on.
Route::post('system/settings/virus-scanning/test', [VirusScanningSettingsController::class, 'test'])
->middleware('throttle:12,1,scanner-test')->name('system-settings.virus-scanning.test');
// Polled every few seconds while the Activity tab is open, so it
// is a plain JSON read with its own generous bucket rather than
// an Inertia render.
Route::get('system/settings/virus-scanning/activity', [VirusScanningSettingsController::class, 'activity'])
->middleware('throttle:120,1,scanner-activity')->name('system-settings.virus-scanning.activity');
Route::post('system/settings/virus-scanning/scan-existing', [VirusScanningSettingsController::class, 'scanExisting'])
->middleware('throttle:6,1,scanner-backfill')->name('system-settings.virus-scanning.scan-existing');
Route::get('system/settings/comments', [CommentSettingsController::class, 'edit'])->name('system-settings.comments.edit');
Route::patch('system/settings/comments', [CommentSettingsController::class, 'update'])->name('system-settings.comments.update');
Route::get('system/settings/email', [EmailSettingsController::class, 'edit'])->name('system-settings.email.edit');
+11
View File
@@ -29,6 +29,7 @@ use App\Modules\Files\Http\Controllers\FoldersController;
use App\Modules\Files\Http\Controllers\MyFilesController;
use App\Modules\Files\Http\Controllers\MyFoldersController;
use App\Modules\Files\Http\Controllers\OrphanFilesController;
use App\Modules\Files\Http\Controllers\QuarantineController;
use App\Modules\Files\Http\Controllers\PublicShareController;
use App\Modules\Files\Http\Controllers\ShareLinksController;
use App\Modules\Files\Http\Controllers\ZipDownloadsController;
@@ -128,6 +129,16 @@ Route::middleware(['auth'])->group(function () {
Route::post('files/orphans/delete', [OrphanFilesController::class, 'destroy'])->name('orphan-files.delete');
});
// Before files/{file}, or "quarantine" is swallowed as a file key —
// the same ordering the orphans block above depends on.
Route::get('files/quarantine', [QuarantineController::class, 'index'])
->middleware(['staff', 'can:release_quarantined_files'])->name('files.quarantine');
// Password confirmation on top of the permission: this is the one
// action that deliberately hands out a file the scanner called
// malicious, and it is the same bar minting an API token has to clear.
Route::post('files/{file}/release', [QuarantineController::class, 'release'])
->middleware(['staff', 'can:release_quarantined_files', 'password.confirm'])->name('files.release');
Route::get('files/{file}', [FilesController::class, 'edit'])->middleware('staff')->name('files.edit');
Route::get('files/{file}/details', [FileDetailsController::class, 'show'])->middleware('staff')->name('files.details');
Route::get('files/{file}/activity', [FileDetailsController::class, 'activity'])->middleware('staff')->name('files.activity');
+22
View File
@@ -274,3 +274,25 @@ test('a token cannot download a file outside its scope', function () {
$this->withToken($token)->getJson("/api/v1/files/{$unrelated->id}/download")->assertForbidden();
});
test('a file says where it stands with the virus scanner, and can be filtered by it', function () {
$pending = File::factory()->create([
'uploaded_by' => $this->admin->id,
'scan_status' => App\Modules\Files\Scanning\ScanStatus::Pending,
]);
File::factory()->create(['uploaded_by' => $this->admin->id]);
$this->withToken($this->token)->getJson("/api/v1/files/{$pending->id}")
->assertOk()
->assertJsonPath('data.scan.status', 'pending')
->assertJsonPath('data.scan.available', false);
// The download says "not yet" rather than "no": 423, and the caller
// can poll the field above.
$this->withToken($this->token)->get("/api/v1/files/{$pending->id}/download")->assertStatus(423);
$ids = $this->withToken($this->token)->getJson('/api/v1/files?scan_status=pending')
->assertOk()->json('data.*.id');
expect($ids)->toBe([$pending->id]);
});
+179
View File
@@ -0,0 +1,179 @@
<?php
declare(strict_types=1);
use App\Models\User;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLog;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
beforeEach(function () {
Storage::fake('files');
$this->admin = User::factory()->create();
app(Settings::class)->set(Setting::VirusScanningEnabled, false);
app(Settings::class)->set(Setting::VirusScannerAddress, '');
});
/** A row with bytes behind it, as an upload would leave it. */
function fileWithBytes(array $overrides = []): File
{
$path = 'uploads/'.Str::uuid()->toString().'.pdf';
Storage::disk('files')->put($path, 'some bytes');
return File::factory()->create(array_merge([
'path' => $path,
'disk' => 'files',
'size' => 10,
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::BeforeScanning->value,
], $overrides));
}
test('the daily check finds a row whose bytes are gone, and says so once', function () {
$here = fileWithBytes();
$gone = fileWithBytes(['name' => 'Contrato']);
Storage::disk('files')->delete($gone->path);
$this->artisan('projectsend:check-missing-files')->assertSuccessful();
expect($gone->refresh()->scan_status)->toBe(ScanStatus::Missing)
->and($here->refresh()->scan_status)->toBe(ScanStatus::NotScanned);
$entry = ActivityLog::query()->where('action', Action::FileMissing)->sole();
expect($entry->context['name'])->toBe('Contrato');
// Stamped like any other verdict, so it shows up on the Activity tab
// rather than being decided somewhere nobody can see.
expect($gone->refresh()->scanned_at)->not->toBeNull();
// Run again: the file is still gone, and that is not news.
$this->artisan('projectsend:check-missing-files')->assertSuccessful();
expect(ActivityLog::query()->where('action', Action::FileMissing)->count())->toBe(1);
});
test('a file that comes back is checked again rather than left for dead', function () {
$file = fileWithBytes();
$path = $file->path;
Storage::disk('files')->delete($path);
$this->artisan('projectsend:check-missing-files')->assertSuccessful();
expect($file->refresh()->scan_status)->toBe(ScanStatus::Missing);
// A remount, a restored backup, a bucket reconnected.
Storage::disk('files')->put($path, 'some bytes');
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
$this->artisan('projectsend:check-missing-files')->assertSuccessful();
// Back to the start: nothing here knows what the scanner had decided
// about bytes that have since been away.
expect($file->refresh()->scan_status)->toBe(ScanStatus::Pending);
});
test('a file that comes back on an installation with no scanner is simply available again', function () {
$file = fileWithBytes();
$path = $file->path;
Storage::disk('files')->delete($path);
$this->artisan('projectsend:check-missing-files');
Storage::disk('files')->put($path, 'some bytes');
$this->artisan('projectsend:check-missing-files')->assertSuccessful();
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::NotScanned)
->and($file->scan_status->isAvailable())->toBeTrue();
});
test('a missing file is not offered to anybody', function () {
$client = User::factory()->client()->create();
$file = fileWithBytes(['uploaded_by' => $this->admin->id]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
expect(File::query()->visibleToClient($client)->count())->toBe(1);
Storage::disk('files')->delete($file->path);
$this->artisan('projectsend:check-missing-files');
expect(File::query()->visibleToClient($client)->count())->toBe(0);
// And says what it is rather than "not available", which would send
// somebody looking for a permission that would let them through.
$this->actingAs($this->admin)->get("/files/{$file->id}/download")
->assertStatus(423)
->assertSee('no longer on the server', false);
});
/*
|--------------------------------------------------------------------------
| The screen
|--------------------------------------------------------------------------
*/
test('missing files are listed beside the orphans, which are the same fault from the other end', function () {
$gone = fileWithBytes(['name' => 'Contrato']);
Storage::disk('files')->delete($gone->path);
$this->artisan('projectsend:check-missing-files');
$this->actingAs($this->admin)->get('/files/orphans')->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page
->where('tab', 'orphans')
// Carried on both tabs, so the count in the tab label is there
// before anybody clicks it.
->where('missing_count', 1),
);
$this->actingAs($this->admin)->get('/files/orphans?tab=missing')->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page
->where('tab', 'missing')
->where('missing.0.name', 'Contrato')
->where('missing.0.path', $gone->path),
);
});
test('the screen needs the permission it always needed', function () {
$staff = User::factory()->role(App\Modules\Identity\Permissions\SystemRole::Uploader)->create();
App\Modules\Identity\Models\RolePermission::query()
->where('role_id', $staff->role_id)
->where('permission', App\Modules\Identity\Permissions\Permission::ImportOrphans->value)
->delete();
forgetRequestState();
$this->actingAs($staff)->get('/files/orphans?tab=missing')->assertForbidden();
});
test('removing a missing file takes the record with it', function () {
$gone = fileWithBytes(['uploaded_by' => $this->admin->id]);
Storage::disk('files')->delete($gone->path);
$this->artisan('projectsend:check-missing-files');
$this->actingAs($this->admin)->delete("/files/{$gone->id}")->assertRedirect();
expect(File::query()->whereKey($gone->id)->exists())->toBeFalse();
});
test('the dashboard and the status document both count them', function () {
app(Settings::class)->set(Setting::GettingStartedPending, false);
app(Settings::class)->set(Setting::UpdateWelcomeTo, '');
$gone = fileWithBytes();
Storage::disk('files')->delete($gone->path);
$this->artisan('projectsend:check-missing-files');
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page->where('system.missing_files', 1),
);
Illuminate\Support\Facades\Artisan::call('projectsend:status', ['--json' => true]);
$status = json_decode(Illuminate\Support\Facades\Artisan::output(), true);
expect($status['health']['missing_files'])->toBe(1);
});
+173
View File
@@ -0,0 +1,173 @@
<?php
declare(strict_types=1);
use App\Models\User;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLog;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\ScanVerdict;
use App\Modules\Identity\Models\Role;
use App\Modules\Identity\Models\RolePermission;
use App\Modules\Identity\Permissions\Permission;
use App\Modules\Notifications\InAppNotification;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
use Inertia\Testing\AssertableInertia;
beforeEach(function () {
Storage::fake('files');
$this->admin = User::factory()->create();
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
app(Settings::class)->set(Setting::VirusUnscannablePolicy, 'allow');
app(Settings::class)->set(Setting::VirusScannerDownPolicy, 'allow');
app(Settings::class)->set(Setting::EmailNotificationsEnabled, false);
});
function quarantined(array $overrides = []): File
{
$path = 'uploads/'.Str::uuid()->toString().'.pdf';
Storage::disk('files')->put($path, 'some bytes');
return File::factory()->create(array_merge([
'path' => $path,
'disk' => 'files',
'size' => 10,
'scan_status' => ScanStatus::Infected,
'scan_note' => 'Eicar-Test-Signature',
'scanned_at' => now(),
], $overrides));
}
/*
|--------------------------------------------------------------------------
| Who may open it
|--------------------------------------------------------------------------
*/
test('the quarantine screen needs its own permission', function () {
$role = Role::query()->create(['name' => 'No release '.Str::random(4)]);
RolePermission::query()->create(['role_id' => $role->id, 'permission' => Permission::DeleteOthersFiles->value]);
$staff = User::factory()->create(['role_id' => $role->id]);
$this->actingAs($staff)->get('/files/quarantine')->assertForbidden();
// Deleting a file is not the same judgement as deciding the scanner
// was wrong, which is why this permission exists separately.
$file = quarantined();
$this->actingAs($staff)->post("/files/{$file->id}/release", ['reason' => 'looks fine'])->assertForbidden();
expect($file->refresh()->scan_status)->toBe(ScanStatus::Infected);
});
test('a client cannot reach it at all', function () {
$client = User::factory()->client()->create();
// EnsureStaff sends a client to their own dashboard rather than
// answering 403 — what matters here is that the screen is not served.
$this->actingAs($client)->get('/files/quarantine')->assertRedirect(route('dashboard'));
});
test('an administrator sees what is quarantined, and what got out first', function () {
$uploader = User::factory()->client()->create(['name' => 'Cliente Uno']);
$file = quarantined(['name' => 'Factura', 'uploaded_by' => $uploader->id, 'scan_was_available' => true]);
$this->actingAs($this->admin)->get('/files/quarantine')->assertInertia(
fn (AssertableInertia $page) => $page
->component('files/quarantine')
->where('files.0.name', 'Factura')
->where('files.0.uploader', 'Cliente Uno')
->where('files.0.threat', 'Eicar-Test-Signature')
->where('files.0.was_available', true),
);
});
/*
|--------------------------------------------------------------------------
| Releasing
|--------------------------------------------------------------------------
*/
test('releasing needs a reason, and records who gave it', function () {
$file = quarantined();
confirmPassword($this->admin);
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => ''])
->assertSessionHasErrors('reason');
expect($file->refresh()->scan_status)->toBe(ScanStatus::Infected);
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => 'False positive, reported upstream'])
->assertSessionHasNoErrors();
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::Released)
->and($file->released_by)->toBe($this->admin->id)
->and($file->released_at)->not->toBeNull();
$entry = ActivityLog::query()->where('action', Action::FileReleased)->sole();
expect($entry->actor_id)->toBe($this->admin->id)
->and($entry->context['reason'])->toBe('False positive, reported upstream');
});
test('a released file downloads again', function () {
$file = quarantined(['uploaded_by' => $this->admin->id]);
$this->actingAs($this->admin)->get("/files/{$file->id}/download")->assertStatus(423);
confirmPassword($this->admin);
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => 'Known false positive']);
$this->actingAs($this->admin)->get("/files/{$file->id}/download")->assertOk();
});
test('releasing asks for the password first', function () {
$file = quarantined();
// No confirmPassword() here: the middleware should send the request
// to the confirmation screen rather than release the file.
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => 'sure'])
->assertRedirect(route('password.confirm'));
expect($file->refresh()->scan_status)->toBe(ScanStatus::Infected);
});
test('a file that is not quarantined cannot be released', function () {
$file = quarantined(['scan_status' => ScanStatus::Clean, 'scan_note' => null]);
confirmPassword($this->admin);
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => 'why not'])->assertNotFound();
});
/*
|--------------------------------------------------------------------------
| Who is told
|--------------------------------------------------------------------------
*/
test('a quarantined file tells the administrators and the uploader, and nobody else', function () {
$uploader = User::factory()->client()->create();
$bystander = User::factory()->client()->create();
$file = quarantined(['scan_status' => ScanStatus::Pending, 'scan_note' => null, 'uploaded_by' => $uploader->id]);
app(App\Modules\Files\Scanning\ScanPolicy::class)->record($file, ScanVerdict::infected('Eicar-Test-Signature'));
$told = InAppNotification::query()->pluck('type', 'user_id');
expect($told[$this->admin->id] ?? null)->toBe('file_quarantined')
->and($told[$uploader->id] ?? null)->toBe('upload_blocked')
->and($told->has($bystander->id))->toBeFalse();
});
test('a staff member who uploaded it is told once, as staff', function () {
$file = quarantined(['scan_status' => ScanStatus::Pending, 'scan_note' => null, 'uploaded_by' => $this->admin->id]);
app(App\Modules\Files\Scanning\ScanPolicy::class)->record($file, ScanVerdict::infected('Eicar-Test-Signature'));
expect(InAppNotification::query()->where('user_id', $this->admin->id)->count())->toBe(1);
});
@@ -0,0 +1,534 @@
<?php
declare(strict_types=1);
use App\Models\User;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScannerStatus;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\ScanVerdict;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
use Inertia\Testing\AssertableInertia;
use Tests\Support\FakeVirusScanner;
beforeEach(function () {
$this->admin = User::factory()->create();
app(Settings::class)->set(Setting::VirusScanningEnabled, false);
app(Settings::class)->set(Setting::VirusScannerAddress, '');
app(Settings::class)->set(Setting::VirusUnscannablePolicy, 'allow');
app(Settings::class)->set(Setting::VirusScannerDownPolicy, 'allow');
config()->set('projectsend.scanning.address', null);
// The dashboard test below lands on the greeting instead of the
// dashboard otherwise — and settings survive RefreshDatabase's
// rollback in the cache, so both markers are stated rather than
// assumed.
app(Settings::class)->set(Setting::GettingStartedPending, false);
app(Settings::class)->set(Setting::UpdateWelcomeTo, '');
});
test('the screen shows what is configured and what is outstanding', function () {
File::factory()->create(['scan_status' => ScanStatus::NotScanned, 'scan_note' => NotScannedReason::ScannerUnavailable->value]);
File::factory()->create(['scan_status' => ScanStatus::NotScanned, 'scan_note' => NotScannedReason::BeforeScanning->value]);
// The shape every upgraded installation is actually in: the status
// from the column default, and no reason beside it. Counted, or the
// "Scan existing files" button sits disabled on a library of
// thousands.
File::factory()->create(['scan_status' => ScanStatus::NotScanned, 'scan_note' => null]);
File::factory()->create(['scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page
->component('system/settings/virus-scanning')
->where('managed', false)
->where('counts.let_through', 1)
->where('counts.never_scanned', 2)
->where('counts.quarantined', 1),
);
});
test('scanning cannot be switched on without an address', function () {
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => true,
'address' => '',
'max_size_mb' => 512,
'unscannable_policy' => 'allow',
'scanner_down_policy' => 'allow',
'wait_minutes' => 10,
'existing_rate_per_minute' => 60,
])->assertSessionHasErrors('address');
expect(app(Settings::class)->get(Setting::VirusScanningEnabled))->toBeFalse();
});
test('an address and the policies are saved', function () {
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => true,
'address' => 'tcp://clamav:3310',
'max_size_mb' => 256,
'unscannable_policy' => 'block',
'scanner_down_policy' => 'hold',
'wait_minutes' => 20,
'existing_rate_per_minute' => 30,
])->assertSessionHasNoErrors();
$settings = app(Settings::class);
expect($settings->get(Setting::VirusScanningEnabled))->toBeTrue()
->and($settings->get(Setting::VirusScannerAddress))->toBe('tcp://clamav:3310')
->and($settings->get(Setting::VirusUnscannablePolicy))->toBe('block')
->and($settings->get(Setting::VirusScannerDownPolicy))->toBe('hold');
});
test('a managed installation keeps its policies but not the connection', function () {
config()->set('projectsend.scanning.address', 'tcp://fleet-scanner:3310');
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('managed', true)->where('enabled', true)->where('address', ''),
);
// Sending the fields anyway changes nothing about the connection, and
// cannot switch scanning off.
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => false,
'address' => 'tcp://somewhere-else:3310',
'max_size_mb' => 100,
'unscannable_policy' => 'block',
'scanner_down_policy' => 'hold',
'wait_minutes' => 10,
'existing_rate_per_minute' => 60,
])->assertSessionHasNoErrors();
$settings = app(Settings::class);
expect($settings->get(Setting::VirusScannerAddress))->toBe('')
->and(app(App\Modules\Files\Scanning\ScanningConfig::class)->enabled())->toBeTrue()
->and($settings->get(Setting::VirusUnscannablePolicy))->toBe('block');
});
/*
|--------------------------------------------------------------------------
| The test button
|--------------------------------------------------------------------------
*/
test('the test button says when the scanner cannot be reached', function () {
app()->instance(VirusScanner::class, (new FakeVirusScanner)->reports(ScannerStatus::unreachable('No answer from tcp://clamav:3310.')));
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test')
->assertSessionHas('scanner_test_result', fn (array $result): bool => $result['ok'] === false);
});
test('the answer actually reaches the screen', function () {
// It did not, at first: the page read a flash prop that nothing
// shares, so the button appeared to do nothing at all. The result is
// handed over as a page prop, like the CAPTCHA screen's.
app()->instance(VirusScanner::class, new FakeVirusScanner(ScanVerdict::infected('Eicar-Test-Signature')));
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test');
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('test_result.ok', true),
);
});
test('the screen opens on the scanner tab, and the other one is a link away', function () {
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('tab', 'scanner'),
);
$this->actingAs($this->admin)->get('/system/settings/virus-scanning?tab=options')->assertInertia(
fn (AssertableInertia $page) => $page->where('tab', 'options'),
);
// Anything else is the default rather than an error: a stale
// bookmark should open the page, not break it.
$this->actingAs($this->admin)->get('/system/settings/virus-scanning?tab=nonsense')->assertInertia(
fn (AssertableInertia $page) => $page->where('tab', 'scanner'),
);
});
test('the test button says when the scanner answers but detects nothing', function () {
// The failure that looks like success: reachable, and blind. Empty or
// broken virus definitions do exactly this.
app()->instance(VirusScanner::class, new FakeVirusScanner(ScanVerdict::clean('FakeAV 1.0')));
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test')
->assertSessionHas('scanner_test_result', fn (array $result): bool => $result['ok'] === false
&& str_contains($result['message'], 'did not detect'));
});
test('the test button confirms a working scanner', function () {
app()->instance(VirusScanner::class, new FakeVirusScanner(ScanVerdict::infected('Eicar-Test-Signature')));
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test')
->assertSessionHas('scanner_test_result', fn (array $result): bool => $result['ok'] === true);
});
test('the test button sends the standard test file, not an empty stream', function () {
$scanner = new FakeVirusScanner(ScanVerdict::infected('Eicar-Test-Signature'));
app()->instance(VirusScanner::class, $scanner);
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test');
expect($scanner->scans)->toBe(1)
->and($scanner->sizes[0])->toBe(68);
});
test('only somebody who can edit settings may test or save', function () {
$staff = User::factory()->role(App\Modules\Identity\Permissions\SystemRole::Uploader)->create();
$this->actingAs($staff)->get('/system/settings/virus-scanning')->assertForbidden();
$this->actingAs($staff)->post('/system/settings/virus-scanning/test')->assertForbidden();
});
/*
|--------------------------------------------------------------------------
| Saying so where nobody is looking
|--------------------------------------------------------------------------
*/
/** The scanning block of the status document, as the fleet probe reads it. */
function scanningStatus(): array
{
Illuminate\Support\Facades\Artisan::call('projectsend:status', ['--json' => true]);
/** @var array<string, mixed> $document */
$document = json_decode(Illuminate\Support\Facades\Artisan::output(), true);
return $document['scanning'];
}
test('the status command reports scanning as off when it is off', function () {
$this->artisan('projectsend:status')->assertSuccessful();
// statusJson() lives in StatusCommandTest — Pest loads every test
// file into one process, so a second copy here would be a redeclare.
$status = scanningStatus();
expect($status['enabled'])->toBeFalse()
// Null, not false: there is nothing to reach. A watcher must be
// able to tell that from a scanner that should answer and does not.
->and($status['reachable'])->toBeNull();
});
test('the status command reports an unreachable scanner and what got through', function () {
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://nowhere.test:3310');
app()->instance(VirusScanner::class, (new FakeVirusScanner)->reports(ScannerStatus::unreachable('no answer')));
app(App\Modules\Audit\ActivityLogger::class)->logSystem(App\Modules\Audit\Action::FileNotScanned, [
'id' => 1, 'name' => 'x', 'reason' => 'scanner_unavailable',
]);
// statusJson() lives in StatusCommandTest — Pest loads every test
// file into one process, so a second copy here would be a redeclare.
$status = scanningStatus();
expect($status['enabled'])->toBeTrue()
->and($status['reachable'])->toBeFalse()
->and($status['let_through_24h'])->toBe(1);
});
test('the dashboard reports a healthy scanner, and says so when it stops answering', function () {
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
app()->instance(VirusScanner::class, new FakeVirusScanner);
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page->where('system.scanning.reachable', true),
);
app()->instance(VirusScanner::class, (new FakeVirusScanner)->reports(ScannerStatus::unreachable('no answer')));
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page->where('system.scanning.reachable', false),
);
});
/*
|--------------------------------------------------------------------------
| Nothing is checking what this installation accepts
|--------------------------------------------------------------------------
*/
test('an installation with no scanner is told so on the dashboard', function () {
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page
->where('system.scanning.configured', false)
->where('system.scanning.reachable', false),
);
});
test('a hosted installation is not told: the scanner is not its job', function () {
// The capability, not an edition check — see
// Capability::VirusScanningConnect. The System card is community-only
// in its own right, so on a hosted installation the whole card is
// absent and the notice with it; the assertion below is about the
// card, and the one on the settings screen (further down) is what
// pins the capability itself.
config(['projectsend.edition' => App\Modules\Platform\Capabilities\Edition::Cloud]);
forgetRequestState();
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page->where('system', null),
);
});
test('a working scanner is still reported, by name', function () {
// The row is always there, like the delivery and storage rows beside
// it: "my uploads are checked by ClamAV" is worth confirming at a
// glance, not only worth saying when it is false.
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
app()->instance(VirusScanner::class, new FakeVirusScanner);
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page
->where('system.scanning.configured', true)
->where('system.scanning.reachable', true)
->where('system.scanning.engine', 'FakeAV 1.0')
->where('system.scanning.let_through_24h', 0),
);
});
test('a hosted installation cannot connect a scanner of its own, but keeps its policies', function () {
config(['projectsend.edition' => App\Modules\Platform\Capabilities\Edition::Cloud]);
forgetRequestState();
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('managed', true),
);
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/test')->assertForbidden();
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => true,
'address' => 'tcp://mine:3310',
'max_size_mb' => 256,
'unscannable_policy' => 'block',
'scanner_down_policy' => 'hold',
'wait_minutes' => 10,
'existing_rate_per_minute' => 60,
])->assertSessionHasNoErrors();
expect(app(Settings::class)->get(Setting::VirusScannerAddress))->toBe('')
->and(app(Settings::class)->get(Setting::VirusUnscannablePolicy))->toBe('block');
});
/*
|--------------------------------------------------------------------------
| Watching a scan happen
|--------------------------------------------------------------------------
*/
test('the activity endpoint says what is running and what was decided', function () {
$waiting = File::factory()->create(['scan_status' => ScanStatus::Pending]);
$done = File::factory()->create([
'name' => 'Contrato',
'scan_status' => ScanStatus::Infected,
'scan_note' => 'Eicar-Test-Signature',
'scanned_at' => now()->subMinute(),
]);
$body = $this->actingAs($this->admin)->getJson('/system/settings/virus-scanning/activity')->assertOk()->json();
expect($body['running'])->toBeTrue()
->and($body['waiting'])->toBe(1)
->and($body['checked_last_hour'])->toBe(1)
->and($body['quarantined'])->toBe(1)
->and($body['recent'][0]['name'])->toBe('Contrato')
->and($body['recent'][0]['note'])->toBe('Eicar-Test-Signature');
expect($waiting->fresh()->scan_status)->toBe(ScanStatus::Pending)
->and($done->fresh()->scan_status)->toBe(ScanStatus::Infected);
});
test('with nothing waiting it reports the last run rather than nothing at all', function () {
File::factory()->create([
'scan_status' => ScanStatus::Clean,
'scanned_at' => now()->subDays(2),
]);
$body = $this->actingAs($this->admin)->getJson('/system/settings/virus-scanning/activity')->assertOk()->json();
expect($body['running'])->toBeFalse()
->and($body['last_scanned_at'])->not->toBeNull()
->and($body['recent'])->toHaveCount(1);
});
test('a file that was never scanned reads as such rather than as a bare "not scanned"', function () {
File::factory()->create(['scan_status' => ScanStatus::NotScanned, 'scan_note' => null, 'scanned_at' => now()]);
$body = $this->actingAs($this->admin)->getJson('/system/settings/virus-scanning/activity')->assertOk()->json();
expect($body['recent'][0]['note'])->toContain('before virus scanning');
});
test('watching a scan needs the same permission as changing its settings', function () {
$staff = User::factory()->role(App\Modules\Identity\Permissions\SystemRole::Uploader)->create();
$this->actingAs($staff)->getJson('/system/settings/virus-scanning/activity')->assertForbidden();
});
test('a backfill counts as running even though it holds nothing back', function () {
// The case the first version of this screen got wrong: re-scanning
// files that already went out deliberately leaves them available, so
// nothing is "pending" and a screen watching only that said nothing
// was happening while the queue worked through a whole library.
Illuminate\Support\Facades\Queue::fake();
App\Modules\Files\Jobs\ScanFileJob::dispatch(1, true);
$body = $this->actingAs($this->admin)->getJson('/system/settings/virus-scanning/activity')->assertOk()->json();
expect($body['waiting'])->toBe(0)
->and($body['queued'])->toBe(1)
->and($body['running'])->toBeTrue();
});
test('starting a scan lands on the tab that shows it happening', function () {
// A button whose screen looks unchanged afterwards reads as a button
// that did nothing.
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
$this->actingAs($this->admin)->post('/system/settings/virus-scanning/scan-existing')
->assertRedirect(route('system-settings.virus-scanning.edit', ['tab' => 'activity']));
});
test('the screen says whether a scan is already under way', function () {
Illuminate\Support\Facades\Queue::fake();
App\Modules\Files\Jobs\ScanFileJob::dispatch(1, true);
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('counts.queued', 1),
);
});
test('a hosted installation is not offered a test it cannot run', function () {
config(['projectsend.edition' => App\Modules\Platform\Capabilities\Edition::Cloud]);
forgetRequestState();
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('can_test', false),
);
});
test('an installation that connects its own scanner is', function () {
$this->actingAs($this->admin)->get('/system/settings/virus-scanning')->assertInertia(
fn (AssertableInertia $page) => $page->where('can_test', true),
);
});
test('the test button tries the address on screen, not the one on file', function () {
// The real client, deliberately: a fake would answer whatever it was
// told and prove nothing about which address was used. Neither
// address has a scanner behind it, so the answer names the one it
// actually tried.
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://saved.invalid:3310');
$this->actingAs($this->admin)
->post('/system/settings/virus-scanning/test', ['address' => 'tcp://typed.invalid:3310'])
->assertSessionHas('scanner_test_result', fn (array $result): bool => $result['ok'] === false
&& str_contains($result['message'], 'typed.invalid'));
// And the stored address is untouched: testing is not saving.
expect(app(Settings::class)->get(Setting::VirusScannerAddress))->toBe('tcp://saved.invalid:3310');
});
test('an empty field falls back to the address on file', function () {
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://saved.invalid:3310');
$this->actingAs($this->admin)
->post('/system/settings/virus-scanning/test', ['address' => ''])
->assertSessionHas('scanner_test_result', fn (array $result): bool => str_contains($result['message'], 'saved.invalid'));
});
test('the sidebar carries a count of what is in quarantine', function () {
App\Modules\Files\Models\File::factory()->create(['scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
App\Modules\Files\Models\File::factory()->create(['scan_status' => ScanStatus::Clean]);
$this->actingAs($this->admin)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page->where('pending.quarantine', 1),
);
});
test('somebody who cannot release is not shown the count', function () {
$staff = User::factory()->role(App\Modules\Identity\Permissions\SystemRole::Uploader)->create();
App\Modules\Files\Models\File::factory()->create(['scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
$this->actingAs($staff)->get('/dashboard')->assertInertia(
fn (AssertableInertia $page) => $page->missing('pending.quarantine'),
);
});
/*
|--------------------------------------------------------------------------
| An address that only looks like one
|--------------------------------------------------------------------------
|
| stream_socket_client() reads a port the way atoi does digits at the
| front, the rest ignored so tcp://clamav:3310djlkasjdlk connects
| happily to 3310. An address with a typo on the end would be saved,
| tested, and reported as working.
|
*/
test('a malformed address is refused when saving', function () {
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => true,
'address' => 'tcp://clamav:3310djlkasjdlk',
'max_size_mb' => 512,
'unscannable_policy' => 'allow',
'scanner_down_policy' => 'allow',
'wait_minutes' => 10,
'existing_rate_per_minute' => 60,
])->assertSessionHasErrors('address');
expect(app(Settings::class)->get(Setting::VirusScannerAddress))->toBe('');
});
test('a malformed address is refused when testing, without dialling anything', function () {
$scanner = new FakeVirusScanner(ScanVerdict::infected('Eicar-Test-Signature'));
app()->instance(VirusScanner::class, $scanner);
$this->actingAs($this->admin)
->post('/system/settings/virus-scanning/test', ['address' => 'tcp://clamav:3310djlkasjdlk'])
->assertSessionHas('scanner_test_result', fn (array $result): bool => $result['ok'] === false);
expect($scanner->scans)->toBe(0);
});
test('the real client refuses a malformed address rather than connecting anyway', function () {
// The socket would take this one: PHP reads 3310 and drops the rest.
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://clamav:3310djlkasjdlk');
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
$status = app(App\Modules\Files\Scanning\ClamAvScanner::class)->status();
// Says which problem it is. "No answer" would be true of any
// unreachable scanner and would prove nothing about this guard — and
// would send an operator looking at their network for a typo.
expect($status->reachable)->toBeFalse()
->and($status->error)->toContain('tcp://host:3310');
});
test('the addresses people actually type are accepted', function () {
foreach (['tcp://clamav:3310', 'unix:///var/run/clamav/clamd.ctl', 'tcp://[::1]:3310', 'tcp://10.0.0.5:3310'] as $address) {
$this->actingAs($this->admin)->patch('/system/settings/virus-scanning', [
'enabled' => true,
'address' => $address,
'max_size_mb' => 512,
'unscannable_policy' => 'allow',
'scanner_down_policy' => 'allow',
'wait_minutes' => 10,
'existing_rate_per_minute' => 60,
])->assertSessionHasNoErrors("{$address} was refused");
expect(app(Settings::class)->get(Setting::VirusScannerAddress))->toBe($address);
}
});
+623
View File
@@ -0,0 +1,623 @@
<?php
declare(strict_types=1);
use App\Models\User;
use App\Modules\Audit\Action;
use App\Modules\Audit\ActivityLog;
use App\Modules\Files\Jobs\ScanFileJob;
use App\Modules\Files\Models\File;
use App\Modules\Files\Scanning\NotScannedReason;
use App\Modules\Files\Scanning\ScanStatus;
use App\Modules\Files\Scanning\ScanVerdict;
use App\Modules\Files\Scanning\VirusScanner;
use App\Modules\Platform\Settings\Setting;
use App\Modules\Platform\Settings\Settings;
use Illuminate\Support\Facades\Storage;
use Illuminate\Support\Str;
use Tests\Support\FakeVirusScanner;
beforeEach(function () {
Storage::fake('files');
$this->admin = User::factory()->create();
// Settings survive RefreshDatabase's rollback in the cache, so every
// value this file depends on is stated rather than assumed.
app(Settings::class)->set(Setting::VirusScanningEnabled, true);
app(Settings::class)->set(Setting::VirusScannerAddress, 'tcp://scanner.test:3310');
app(Settings::class)->set(Setting::VirusScanMaxSizeMb, 512);
app(Settings::class)->set(Setting::VirusUnscannablePolicy, 'allow');
app(Settings::class)->set(Setting::VirusScannerDownPolicy, 'allow');
app(Settings::class)->set(Setting::VirusScannerWaitMinutes, 10);
});
function fakeScanner(?ScanVerdict $verdict = null): FakeVirusScanner
{
$scanner = new FakeVirusScanner($verdict);
app()->instance(VirusScanner::class, $scanner);
return $scanner;
}
/** A stored file with real bytes, as an upload would leave it. */
function scannableFile(array $overrides = []): File
{
$path = 'uploads/'.Str::uuid()->toString().'.pdf';
Storage::disk('files')->put($path, 'some bytes');
return File::factory()->create(array_merge([
'path' => $path,
'disk' => 'files',
'size' => 10,
'scan_status' => ScanStatus::Pending,
'checksum' => hash('sha256', $path),
], $overrides));
}
/** Run the job the way the queue would, with this test's fake scanner. */
function runScan(File $file): void
{
(new ScanFileJob($file->id))->handle(
app(VirusScanner::class),
app(App\Modules\Files\Scanning\ScanPolicy::class),
app(App\Modules\Files\Scanning\ScanningConfig::class),
);
}
/*
|--------------------------------------------------------------------------
| A verdict, and what this installation does with it
|--------------------------------------------------------------------------
*/
test('a clean file becomes available', function () {
fakeScanner(ScanVerdict::clean('FakeAV 1.0'));
$file = scannableFile();
runScan($file);
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::Clean)
->and($file->scan_status->isAvailable())->toBeTrue()
->and($file->scanned_at)->not->toBeNull()
->and($file->scan_engine)->toBe('FakeAV 1.0');
});
test('an infected file is quarantined and logged', function () {
fakeScanner(ScanVerdict::infected('Eicar-Test-Signature'));
$file = scannableFile();
runScan($file);
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::Infected)
->and($file->scan_status->isAvailable())->toBeFalse()
->and($file->scan_note)->toBe('Eicar-Test-Signature');
$entry = ActivityLog::query()->where('action', Action::FileQuarantined)->sole();
expect($entry->context['threat'])->toBe('Eicar-Test-Signature')
->and($entry->context['was_available'])->toBeFalse();
});
test('a quarantined file loses the thumbnails already rendered from it', function () {
fakeScanner(ScanVerdict::infected('Some.Threat'));
$file = scannableFile(['mime_type' => 'image/png']);
$paths = App\Modules\Files\Thumbnails\ThumbnailGenerator::pathsFor($file->id, 'image/png');
expect($paths)->not->toBeEmpty();
foreach ($paths as $path) {
Storage::disk('files')->put($path, 'rendered');
}
runScan($file);
foreach ($paths as $path) {
expect(Storage::disk('files')->exists($path))->toBeFalse();
}
});
test('a file the scanner cannot open is allowed through, marked, and logged', function () {
fakeScanner(ScanVerdict::encrypted());
$file = scannableFile();
runScan($file);
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::NotScanned)
->and($file->scan_note)->toBe(NotScannedReason::Encrypted->value)
->and($file->scan_status->isAvailable())->toBeTrue();
expect(ActivityLog::query()->where('action', Action::FileNotScanned)->count())->toBe(1);
});
test('the same file is blocked when this installation says to block', function () {
app(Settings::class)->set(Setting::VirusUnscannablePolicy, 'block');
fakeScanner(ScanVerdict::encrypted());
$file = scannableFile();
runScan($file);
expect($file->refresh()->scan_status)->toBe(ScanStatus::UnscannableBlocked)
->and($file->scan_status->isAvailable())->toBeFalse();
});
test('a file larger than the maximum never reaches the scanner at all', function () {
app(Settings::class)->set(Setting::VirusScanMaxSizeMb, 1);
// The real client, deliberately: refusing an oversized file before a
// socket is opened is its job, and a fake that answered anyway would
// hide the day that check moves or disappears. There is no scanner at
// the configured address, so anything but an early refusal here would
// come back as "unavailable" instead.
$stream = fopen('php://memory', 'r+');
$verdict = app(App\Modules\Files\Scanning\ClamAvScanner::class)->scan($stream, 2 * 1024 * 1024);
fclose($stream);
expect($verdict->outcome)->toBe(App\Modules\Files\Scanning\ScanOutcome::TooLarge);
});
/*
|--------------------------------------------------------------------------
| A scanner that is not answering
|--------------------------------------------------------------------------
*/
test('a file waits while the scanner is down, then goes through', function () {
fakeScanner(ScanVerdict::unavailable('connection refused'));
$file = scannableFile();
runScan($file);
expect($file->refresh()->scan_status)->toBe(ScanStatus::Pending);
// Past the ten minutes this installation is willing to wait.
$this->travel(11)->minutes();
runScan($file);
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::NotScanned)
->and($file->scan_note)->toBe(NotScannedReason::ScannerUnavailable->value);
});
test('an installation set to hold keeps waiting however long it takes', function () {
app(Settings::class)->set(Setting::VirusScannerDownPolicy, 'hold');
fakeScanner(ScanVerdict::unavailable('connection refused'));
$file = scannableFile();
runScan($file);
$this->travel(3)->days();
runScan($file);
expect($file->refresh()->scan_status)->toBe(ScanStatus::Pending);
});
test('an upload identical to a quarantined file is quarantined without a scan', function () {
$scanner = fakeScanner(ScanVerdict::clean());
$known = scannableFile(['scan_status' => ScanStatus::Infected, 'scan_note' => 'Known.Threat']);
$copy = scannableFile(['checksum' => $known->checksum]);
runScan($copy);
expect($copy->refresh()->scan_status)->toBe(ScanStatus::Infected)
->and($copy->scan_note)->toBe('Known.Threat')
->and($scanner->scans)->toBe(0);
});
/*
|--------------------------------------------------------------------------
| Uploads
|--------------------------------------------------------------------------
*/
test('a new upload is pending while scanning is on', function () {
fakeScanner();
$file = app(App\Modules\Files\Uploads\StoreUploadedFile::class)->create(
uploader: $this->admin,
originalName: 'report.pdf',
path: 'uploads/report.pdf',
mimeType: 'application/pdf',
size: 10,
checksum: str_repeat('b', 64),
);
expect($file->scan_status)->toBe(ScanStatus::Pending);
});
test('a new upload is marked never scanned while scanning is off', function () {
app(Settings::class)->set(Setting::VirusScanningEnabled, false);
app(Settings::class)->set(Setting::VirusScannerAddress, '');
$file = app(App\Modules\Files\Uploads\StoreUploadedFile::class)->create(
uploader: $this->admin,
originalName: 'report.pdf',
path: 'uploads/report.pdf',
mimeType: 'application/pdf',
size: 10,
checksum: str_repeat('c', 64),
);
expect($file->scan_status)->toBe(ScanStatus::NotScanned)
->and($file->scan_note)->toBe(NotScannedReason::BeforeScanning->value);
});
/*
|--------------------------------------------------------------------------
| Nothing unchecked leaves the server
|--------------------------------------------------------------------------
*/
test('no route serves the bytes of a file that is still being checked', function () {
$file = scannableFile(['uploaded_by' => $this->admin->id, 'mime_type' => 'image/png']);
foreach ([
"/files/{$file->id}/download",
"/files/{$file->id}/thumbnail",
"/files/{$file->id}/preview",
] as $path) {
$this->actingAs($this->admin)->get($path)->assertStatus(423, "{$path} served a pending file");
}
});
test('a quarantined file is refused for the same routes', function () {
$file = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
$this->actingAs($this->admin)->get("/files/{$file->id}/download")->assertStatus(423);
});
test('a clean file downloads normally', function () {
$file = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Clean]);
$this->actingAs($this->admin)->get("/files/{$file->id}/download")->assertOk();
});
test('a share link says the file is still being checked', function () {
$file = scannableFile();
$link = App\Modules\Files\Models\ShareLink::query()->create([
'shareable_type' => $file->getMorphClass(),
'shareable_id' => $file->id,
'token' => Str::random(32),
'created_by' => $this->admin->id,
]);
$this->get("/s/{$link->token}")->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page->component('share/show')->where('status', 'checking'),
);
$this->get("/s/{$link->token}/download")->assertRedirect(route('share.show', $link->token));
});
test('a pending file is not visible to the client it was shared with', function () {
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $this->admin->id]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
expect(File::query()->visibleToClient($client)->count())->toBe(0);
$file->forceFill(['scan_status' => ScanStatus::Clean])->save();
expect(File::query()->visibleToClient($client)->count())->toBe(1);
});
test('a client still sees their own upload while it is being checked', function () {
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $client->id]);
expect(File::query()->visibleToClient($client)->pluck('id')->all())->toBe([$file->id]);
});
test('a pending file is left out of a zip', function () {
$pending = scannableFile(['uploaded_by' => $this->admin->id]);
$clean = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Clean]);
$this->actingAs($this->admin)
->postJson('/zip-downloads', ['file_ids' => [$pending->id, $clean->id], 'folder_ids' => []])
->assertOk();
$zip = App\Modules\Files\Models\ZipDownload::query()->latest('id')->sole();
expect($zip->file_ids)->toBe([$clean->id]);
});
/*
|--------------------------------------------------------------------------
| Nobody is told about a file they cannot have yet
|--------------------------------------------------------------------------
*/
test('sharing a file still being checked tells nobody, and tells them when it clears', function () {
fakeScanner(ScanVerdict::clean());
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $this->admin->id]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(0);
runScan($file);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->where('type', 'file_shared')->count())->toBe(1);
});
test('a file found infected is never announced', function () {
fakeScanner(ScanVerdict::infected('Some.Threat'));
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $this->admin->id]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
runScan($file);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(0);
});
test('a share taken back while the file was being checked produces no email afterwards', function () {
fakeScanner(ScanVerdict::clean());
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $this->admin->id]);
$sharing = app(App\Modules\Files\Sharing\FileSharing::class);
$sharing->assign($file, $client, $client->name);
$sharing->unassign($file, $client, $client->name);
runScan($file);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(0);
});
test('re-scanning a file that was let through keeps it available, and does not announce it twice', function () {
// The hourly command asks again about files that went out unchecked
// while the scanner was down. Their recipients already have them, so
// they must not lose access while the answer comes back, and must not
// be told a second time when it does.
$client = User::factory()->client()->create();
$file = scannableFile([
'uploaded_by' => $this->admin->id,
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::ScannerUnavailable->value,
]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(1);
fakeScanner(ScanVerdict::clean());
$this->artisan('projectsend:scan-files')->assertSuccessful();
// Still theirs throughout, and now actually checked.
expect(File::query()->visibleToClient($client)->count())->toBe(1)
->and($file->refresh()->scan_status)->toBe(ScanStatus::Clean)
->and(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(1);
});
test('a backfill never hides the library it is working through', function () {
$client = User::factory()->client()->create();
$file = scannableFile([
'uploaded_by' => $this->admin->id,
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::BeforeScanning->value,
]);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
// Queued rather than run, which is the state a real backfill spends
// almost all of its time in: dispatched, not yet scanned.
Illuminate\Support\Facades\Queue::fake();
fakeScanner(ScanVerdict::clean());
$this->artisan('projectsend:scan-files', ['--existing' => true])->assertSuccessful();
expect($file->refresh()->scan_status)->toBe(ScanStatus::NotScanned)
->and(File::query()->visibleToClient($client)->count())->toBe(1);
});
test('a released file is announced then, not before', function () {
$client = User::factory()->client()->create();
$file = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
app(App\Modules\Files\Sharing\FileSharing::class)->assign($file, $client, $client->name);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->count())->toBe(0);
confirmPassword($this->admin);
$this->actingAs($this->admin)->post("/files/{$file->id}/release", ['reason' => 'False positive']);
expect(App\Modules\Notifications\InAppNotification::query()->where('user_id', $client->id)->where('type', 'file_shared')->count())->toBe(1);
});
test('the activity log names the file it quarantined', function () {
// The scan job has no actor and attaches no subject, so a template
// written with :subject renders 'The file "" was quarantined'. Caught
// on a real dashboard, not by a test, which is why there is one now.
fakeScanner(ScanVerdict::infected('Eicar-Test-Signature'));
$file = scannableFile(['name' => 'Contrato firmado']);
runScan($file);
$entry = ActivityLog::query()->where('action', Action::FileQuarantined)->sole();
$presented = app(App\Modules\Audit\ActivityPresenter::class)->present($entry);
$line = strtr($presented['template'], collect($presented['replacements'])
->mapWithKeys(fn (string $value, string $key): array => [":{$key}" => $value])
->all());
expect($line)->toContain('Contrato firmado')
->toContain('Eicar-Test-Signature');
});
test('a library from before the scanner is what "scan existing files" actually finds', function () {
// The migration gives the column its default and writes no reason, so
// every file on every upgraded installation has scan_note = null. A
// backfill that looked for the reason found none of them — the whole
// feature was inert on exactly the libraries it exists for.
$old = File::factory()->create(['scan_status' => ScanStatus::NotScanned, 'scan_note' => null]);
$stated = File::factory()->create([
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::BeforeScanning->value,
]);
$letThrough = File::factory()->create([
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::ScannerUnavailable->value,
]);
$found = File::query()->neverScanned()->pluck('id')->all();
expect($found)->toContain($old->id)
->toContain($stated->id)
// Not this one: it was offered to a scanner that could not answer,
// and the hourly sweep already re-scans those.
->not->toContain($letThrough->id);
});
test('the backfill queues those files', function () {
fakeScanner(ScanVerdict::clean());
$old = scannableFile(['scan_status' => ScanStatus::NotScanned, 'scan_note' => null]);
Illuminate\Support\Facades\Queue::fake();
$this->artisan('projectsend:scan-files', ['--existing' => true])->assertSuccessful();
Illuminate\Support\Facades\Queue::assertPushed(ScanFileJob::class, fn (ScanFileJob $job): bool => $job->fileId === $old->id);
});
test('a file whose bytes are gone says so, and is not retried forever', function () {
// An orphaned row, or storage that moved. It used to be recorded as
// "the scanner could not be reached" — wrong on screen, and wrong in
// behaviour: that is the one reason the hourly sweep re-queues, so
// every missing file would have been rescanned every hour for good.
$scanner = fakeScanner(ScanVerdict::clean());
$file = scannableFile();
Storage::disk('files')->delete($file->path);
runScan($file);
$file->refresh();
expect($file->scan_status)->toBe(ScanStatus::Missing)
// Withheld: there is nothing to serve, and a client shown a file
// whose download fails is worse off than one who never saw it.
->and($file->scan_status->isAvailable())->toBeFalse()
// Never offered to the scanner: there was nothing to offer.
->and($scanner->scans)->toBe(0);
Illuminate\Support\Facades\Queue::fake();
$this->artisan('projectsend:scan-files')->assertSuccessful();
Illuminate\Support\Facades\Queue::assertNothingPushed();
});
test('a missing file is missing whatever the unscannable policy says', function () {
// "Allow files nobody could scan" is a decision about risk, and there
// is no risk in a file that cannot be served — only a problem
// somebody has to look at.
app(App\Modules\Platform\Settings\Settings::class)->set(Setting::VirusUnscannablePolicy, 'allow');
fakeScanner(ScanVerdict::clean());
$file = scannableFile();
Storage::disk('files')->delete($file->path);
runScan($file);
expect($file->refresh()->scan_status)->toBe(ScanStatus::Missing);
});
test('a new scan checks files that already have a verdict', function () {
// "New scan" after an engine update means "check my library again".
// The first version only queued files nothing had ever looked at, so
// on a library already scanned once the button did nothing — and was
// disabled for saying so.
fakeScanner(ScanVerdict::clean());
$clean = scannableFile(['scan_status' => ScanStatus::Clean]);
$letThrough = scannableFile([
'scan_status' => ScanStatus::NotScanned,
'scan_note' => NotScannedReason::TooLarge->value,
]);
$waiting = scannableFile(['scan_status' => ScanStatus::Pending]);
$gone = scannableFile(['scan_status' => ScanStatus::Missing]);
Illuminate\Support\Facades\Queue::fake();
$this->artisan('projectsend:scan-files', ['--all' => true])->assertSuccessful();
foreach ([$clean, $letThrough] as $file) {
Illuminate\Support\Facades\Queue::assertPushed(ScanFileJob::class, fn (ScanFileJob $job): bool => $job->fileId === $file->id);
}
// Nothing to re-ask about a file with no bytes.
Illuminate\Support\Facades\Queue::assertNotPushed(ScanFileJob::class, fn (ScanFileJob $job): bool => $job->fileId === $gone->id);
// The one already waiting is picked up by the ordinary sweep at the
// top of the command, as a first scan rather than as a rescan — the
// distinction the job itself turns on.
Illuminate\Support\Facades\Queue::assertPushed(
ScanFileJob::class,
fn (ScanFileJob $job): bool => $job->fileId === $waiting->id && $job->rescan === false,
);
});
test('a rescan of a clean file re-checks it, and finds what is there now', function () {
// What an engine update is for: the same bytes, a newer opinion.
fakeScanner(ScanVerdict::infected('Newly.Known.Threat'));
$file = scannableFile(['scan_status' => ScanStatus::Clean]);
(new ScanFileJob($file->id, true))->handle(
app(VirusScanner::class),
app(App\Modules\Files\Scanning\ScanPolicy::class),
app(App\Modules\Files\Scanning\ScanningConfig::class),
);
expect($file->refresh()->scan_status)->toBe(ScanStatus::Infected)
->and($file->scan_note)->toBe('Newly.Known.Threat');
});
test('a rescan leaves a file that is waiting for its first verdict alone', function () {
$scanner = fakeScanner(ScanVerdict::clean());
$file = scannableFile(['scan_status' => ScanStatus::Pending]);
(new ScanFileJob($file->id, true))->handle(
app(VirusScanner::class),
app(App\Modules\Files\Scanning\ScanPolicy::class),
app(App\Modules\Files\Scanning\ScanningConfig::class),
);
expect($scanner->scans)->toBe(0)
->and($file->refresh()->scan_status)->toBe(ScanStatus::Pending);
});
test('the library does not list a file nobody can use', function () {
// Every button on such a row leads somewhere that refuses, and the
// download leads to an error page. They live on the two screens that
// exist to act on them.
$clean = scannableFile(['uploaded_by' => $this->admin->id, 'name' => 'Usable', 'scan_status' => ScanStatus::Clean]);
$quarantined = scannableFile(['uploaded_by' => $this->admin->id, 'name' => 'Infectado', 'scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
$gone = scannableFile(['uploaded_by' => $this->admin->id, 'name' => 'Sin bytes', 'scan_status' => ScanStatus::Missing]);
$waiting = scannableFile(['uploaded_by' => $this->admin->id, 'name' => 'Esperando', 'scan_status' => ScanStatus::Pending]);
$names = collect($this->actingAs($this->admin)->get('/files')->viewData('page')['props']['files'])->pluck('name');
expect($names)->toContain('Usable')
// Still listed: it is about to be usable, and its uploader should
// see where it went.
->toContain('Esperando')
->not->toContain('Infectado')
->not->toContain('Sin bytes');
expect([$clean->id, $quarantined->id, $gone->id, $waiting->id])->toHaveCount(4);
});
test('the file editor says why a quarantined file refuses everything', function () {
$file = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Infected, 'scan_note' => 'Eicar-Test-Signature']);
$this->actingAs($this->admin)->get("/files/{$file->id}")->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page
->where('file.scan_status', 'infected')
->where('file.scan_note', 'Eicar-Test-Signature'),
);
});
test('the editor offers no download for a file it cannot produce', function () {
$quarantined = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Infected, 'scan_note' => 'X']);
$clean = scannableFile(['uploaded_by' => $this->admin->id, 'scan_status' => ScanStatus::Clean]);
$this->actingAs($this->admin)->get("/files/{$quarantined->id}")->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page->where('file.scan_available', false),
);
$this->actingAs($this->admin)->get("/files/{$clean->id}")->assertInertia(
fn (Inertia\Testing\AssertableInertia $page) => $page->where('file.scan_available', true),
);
});
@@ -45,7 +45,7 @@ test('the scheduler page lists every known command, flagging ones that have neve
]);
$response = $this->actingAs($this->admin)->get('/system/settings/scheduler');
$response->assertInertia(fn (AssertableInertia $page) => $page->component('system/settings/scheduler')->has('tasks', 12));
$response->assertInertia(fn (AssertableInertia $page) => $page->component('system/settings/scheduler')->has('tasks', 14));
$tasks = collect(schedulerPageProps($response)['tasks'])->keyBy('command');
expect($tasks->get('projectsend:purge-expired-files')['status'])->toBe('success')
@@ -95,3 +95,50 @@ test('the seeded policy is in force for the first account the same boot creates'
expect(app(Settings::class)->get(Setting::TwoFactorEnforcement))->toBe('staff')
->and($admin->hasTwoFactorEnabled())->toBeFalse();
});
/*
|--------------------------------------------------------------------------
| The virus scanner
|--------------------------------------------------------------------------
|
| The opposite of PROJECTSEND_SCANNER_ADDRESS, which is a policy the
| platform keeps. This is a starting value for an operator who brought up
| the optional scanner container beside the application: it arrives
| configured, and stays theirs to change.
|
*/
test('a first boot points the installation at the scanner named in its environment', function () {
config(['projectsend.scanning.default_address' => 'tcp://clamav:3310']);
$this->artisan('projectsend:seed-settings')->assertSuccessful();
$settings = app(App\Modules\Platform\Settings\Settings::class);
expect($settings->get(App\Modules\Platform\Settings\Setting::VirusScannerAddress))->toBe('tcp://clamav:3310')
// Both together: an address with scanning off would look
// configured and check nothing.
->and($settings->get(App\Modules\Platform\Settings\Setting::VirusScanningEnabled))->toBeTrue();
});
test('it never argues with an administrator who has already chosen', function () {
$settings = app(App\Modules\Platform\Settings\Settings::class);
$settings->set(App\Modules\Platform\Settings\Setting::VirusScannerAddress, '');
$settings->set(App\Modules\Platform\Settings\Setting::VirusScanningEnabled, false);
config(['projectsend.scanning.default_address' => 'tcp://clamav:3310']);
$this->artisan('projectsend:seed-settings')->assertSuccessful();
// Cleared on purpose is a decision, and a restart must not undo it.
expect($settings->get(App\Modules\Platform\Settings\Setting::VirusScannerAddress))->toBe('')
->and($settings->get(App\Modules\Platform\Settings\Setting::VirusScanningEnabled))->toBeFalse();
});
test('an installation with no scanner named in its environment is left alone', function () {
config(['projectsend.scanning.default_address' => null]);
$this->artisan('projectsend:seed-settings')->assertSuccessful();
expect(app(App\Modules\Platform\Settings\Settings::class)->get(App\Modules\Platform\Settings\Setting::VirusScanningEnabled))->toBeFalse();
});
+79
View File
@@ -0,0 +1,79 @@
<?php
declare(strict_types=1);
namespace Tests\Support;
use App\Modules\Files\Scanning\ScannerStatus;
use App\Modules\Files\Scanning\ScanVerdict;
use App\Modules\Files\Scanning\VirusScanner;
/**
* A scanner that answers whatever the test says.
*
* Every question worth asking about scanning is about what happens
* *after* a verdict whether the file can be downloaded, who is told,
* what the policy does with a file nobody could open. Producing a real
* file that provokes each of those from ClamAV would mean shipping
* malware samples and an encrypted archive in the repository, and would
* still not let a test say "the scanner is down".
*
* The real client has its own test against a live clamd, which skips
* unless one is reachable.
*/
class FakeVirusScanner implements VirusScanner
{
/** @var list<ScanVerdict> */
private array $verdicts = [];
public int $scans = 0;
/** @var list<int> the sizes it was asked to read, in order */
public array $sizes = [];
private ScannerStatus $status;
public function __construct(?ScanVerdict $verdict = null)
{
if ($verdict !== null) {
$this->verdicts[] = $verdict;
}
$this->status = new ScannerStatus(true, 'FakeAV 1.0', 1, now());
}
/** Answer this next. Queued, so a test can say "down, then up". */
public function willAnswer(ScanVerdict ...$verdicts): self
{
foreach ($verdicts as $verdict) {
$this->verdicts[] = $verdict;
}
return $this;
}
public function reports(ScannerStatus $status): self
{
$this->status = $status;
return $this;
}
public function scan(mixed $stream, int $size): ScanVerdict
{
$this->scans++;
$this->sizes[] = $size;
// The last answer stands once the queue runs dry: a test that
// says "infected" once means it, however many times the job is
// retried.
return count($this->verdicts) > 1
? array_shift($this->verdicts)
: ($this->verdicts[0] ?? ScanVerdict::clean('FakeAV 1.0'));
}
public function status(): ScannerStatus
{
return $this->status;
}
}