mirror of
https://github.com/projectsend/projectsend.git
synced 2026-09-18 01:25:09 +00:00
ignacionelson
f937b4398d
A row whose bytes are gone was recorded as "the scanner could not be
reached". Wrong on screen, and wrong underneath: that is the one reason
the hourly sweep re-queues, so every orphaned row would have been
rescanned hourly forever.
It is its own state now, `missing`, and withheld rather than offered:
a client who sees a file listed and gets an error on the download is
worse off than one who never saw it. Staff still see it, marked, which
is the point — somebody has to decide what to do about it. The refusal
says what it is ("no longer on the server") instead of sending somebody
looking for a permission that would let them through.
A daily `projectsend:check-missing-files` finds them, whether or not
this installation scans for viruses: it is not a virus question, and an
installation with no scanner has exactly the same problem. It compares
one disk listing against the rows rather than asking "does this exist?"
per file, which on object storage would be a request per file per day.
Files that come back — a remount, a restored backup — are picked up on
the next run and re-checked rather than left for dead.
They are listed beside the orphans, which is the same fault seen from
the other end: bytes with no row, rows with no bytes. The tab carries
the count, each row says where the file should be, and removing one
takes the record with it through the deletion that already exists.
The dashboard says how many there are, and so does
`projectsend:status`, because a fleet-wide jump in this is a storage
fault nothing else in that document would show.
70 lines
2.6 KiB
PHP
70 lines
2.6 KiB
PHP
<?php
|
|
|
|
declare(strict_types=1);
|
|
|
|
namespace App\Modules\Files\Console;
|
|
|
|
use App\Modules\Audit\Action;
|
|
use App\Modules\Audit\ActivityLogger;
|
|
use App\Modules\Files\MissingFileScanner;
|
|
use App\Modules\Files\Models\File;
|
|
use App\Modules\Files\Scanning\NotScannedReason;
|
|
use App\Modules\Files\Scanning\ScanningConfig;
|
|
use App\Modules\Files\Scanning\ScanStatus;
|
|
use Illuminate\Console\Command;
|
|
|
|
/**
|
|
* Checks daily that the files this installation lists are actually there.
|
|
*
|
|
* Its own command rather than part of scanning, because a missing file is
|
|
* not a virus question and an installation with no scanner has exactly
|
|
* the same problem. It was only ever noticed when something tried to read
|
|
* the bytes — a download, or a scan — which means the first person to
|
|
* find out was a client clicking a link.
|
|
*
|
|
* Recovery is part of the job: storage comes back, and a library that
|
|
* kept insisting every file was gone would be its own bug.
|
|
*/
|
|
class CheckMissingFilesCommand extends Command
|
|
{
|
|
protected $signature = 'projectsend:check-missing-files';
|
|
|
|
protected $description = 'Check that every file in the database is still on disk (runs daily)';
|
|
|
|
public function handle(MissingFileScanner $scanner, ActivityLogger $activity, ScanningConfig $scanning): int
|
|
{
|
|
$gone = $scanner->scan();
|
|
$newlyGone = 0;
|
|
|
|
foreach (array_chunk($gone, 200) as $chunk) {
|
|
foreach (File::query()->whereIn('id', $chunk)->where('scan_status', '!=', ScanStatus::Missing)->get() as $file) {
|
|
$file->forceFill(['scan_status' => ScanStatus::Missing, 'scan_note' => null])->save();
|
|
|
|
$activity->logSystem(Action::FileMissing, ['id' => $file->id, 'name' => $file->name]);
|
|
$newlyGone++;
|
|
}
|
|
}
|
|
|
|
$back = $scanner->recovered();
|
|
|
|
foreach (array_chunk($back, 200) as $chunk) {
|
|
// Back to the start rather than to whatever it was before:
|
|
// nothing here knows what the scanner had decided about bytes
|
|
// that have since been away, and re-checking them is cheap
|
|
// next to trusting a verdict about a file that left.
|
|
File::query()->whereIn('id', $chunk)->update($scanning->enabled()
|
|
? ['scan_status' => ScanStatus::Pending->value, 'scan_note' => null]
|
|
: ['scan_status' => ScanStatus::NotScanned->value, 'scan_note' => NotScannedReason::BeforeScanning->value]);
|
|
}
|
|
|
|
$this->info(sprintf(
|
|
'%d file(s) are missing from storage (%d newly), %d came back.',
|
|
count($gone),
|
|
$newlyGone,
|
|
count($back),
|
|
));
|
|
|
|
return self::SUCCESS;
|
|
}
|
|
}
|