feat(torrents): support remote torrent metadata

This commit is contained in:
NimBold
2026-08-02 00:29:49 +03:30
parent 0f1f4e8003
commit d0541308c7
6 changed files with 281 additions and 4 deletions
+73
View File
@@ -0,0 +1,73 @@
# Firelink Torrent feature matrix
This is the current product-facing comparison for BitTorrent features exposed
by Firelink's bundled Aria2 engine. It intentionally excludes Aria2's generic
HTTP/FTP/Metalink options, shell hooks, and daemon-admin RPC methods that do not
belong in the download UI. The Aria2 reference is the [1.37.0 manual](https://aria2.github.io/manual/en/html/aria2c.html).
## Implemented
- Local `.torrent` files, magnet links, and remote HTTP(S) `.torrent` metadata.
Remote metadata is bounded, SSRF-checked, redirect-checked, parsed, and
cached before it enters the normal `addTorrent` path.
- Bencode validation, canonical info-hash checks, safe output paths, managed
metadata retention, selected-file preview, `select-file`, and `index-out`.
- Firelink queue admission, per-queue/global permits, pause/resume, cancel,
retry/GID replacement, restart recovery, terminal reconciliation, and
output ownership for Torrent lifecycles.
- Optional seeding by time and/or ratio, upload progress, upload limits,
seeders telemetry, per-Torrent maximum peers, and the Aria2
`bt-request-peer-speed-limit` threshold.
- Global DHT, IPv6 DHT, PEX, and Local Peer Discovery toggles.
- Optional piece-integrity verification, including the explicit policy that
disables unverified seeding when verification is requested.
- Additional per-Torrent tracker URLs through `bt-tracker`, with bounded and
credential-free HTTP/HTTPS/UDP validation.
- Deterministic local Aria2 smoke coverage for metadata resolution, selected
output, pause/resume, ownership, cancellation/removal, unavailable trackers,
and daemon failure; RPC-boundary coverage is separate.
## Priority tiers for remaining work
### Tier 0 — reliability and user-visible control
1. **Stall timeout** — expose `bt-stop-timeout` with clear semantics for a
Torrent that has no download progress. The queue must reconcile the Aria2
stop/error outcome and release its permit without turning an intentional
stall policy into a stale retry loop.
2. **Peer diagnostics** — expose `aria2.getPeers` as bounded, redacted,
read-only detail for the selected Torrent. Keep the current counts as the
fast summary and treat peer IPs/IDs as sensitive display data.
3. **Tracker exclusion** — add `bt-exclude-tracker` alongside the existing
additional-tracker list. It must be persisted, re-normalized, and re-applied
on every retry/GID replacement. Document that it filters announce URLs only;
it does not disable DHT or PEX.
### Tier 1 — transfer policy and storage behavior
1. **Piece/file priority** — expose `bt-prioritize-piece` for head/tail
previewing and a deliberate file-priority model beyond the current binary
selected/unselected state.
2. **Safe removal of unselected files** — expose
`bt-remove-unselected-file` only as an explicit destructive choice, with
ownership-aware confirmation and tests for cancellation, retry, and
reconfiguration.
3. **Encryption policy** — expose `bt-force-encryption`,
`bt-require-crypto`, and `bt-min-crypto-level` as one validated policy so
users cannot accidentally select contradictory combinations.
4. **Tracker timing controls** — expose tracker connect timeout, request
timeout, and interval only when their effect on battery/network behavior is
explained and persisted.
### Tier 2 — advanced networking and daemon tuning
1. Configurable TCP/UDP listen ports, external IP, DHT entry points, IPv6 DHT
listen address, and LPD interface, with platform/firewall warnings.
2. Global BitTorrent open-file limits and peer identity/agent controls.
3. Aria2 `follow-torrent`/in-memory follow behavior for generic downloads only
if the resulting child-GID ownership model can be represented safely; the
current explicit metadata path intentionally avoids unmapped child jobs.
The first implementation in this task is the former missing Tier 0 intake
capability: remote `.torrent` metadata now uses Firelink's existing safe,
cached, lifecycle-aware Torrent path.
+157
View File
@@ -1,6 +1,7 @@
#![allow(unexpected_cfgs)]
// Learn more about Tauri commands at https://tauri.app/develop/calling-rust/
use futures_util::StreamExt;
use regex::Regex;
use serde::Serialize;
use std::collections::{BTreeSet, HashMap, HashSet, VecDeque};
@@ -1743,6 +1744,125 @@ async fn validate_url_ssrf(url: &str) -> Result<Option<(String, std::net::Socket
resolve_and_validate_url_host(&parsed).await.map(Some)
}
const MAX_REMOTE_TORRENT_REDIRECTS: usize = 5;
/// Fetch remote `.torrent` metadata through the same SSRF, redirect, proxy,
/// timeout, and bounded-body rules used by Firelink's metadata path. The
/// bytes are parsed and cached before they can enter the Aria2 lifecycle.
async fn fetch_remote_torrent_bytes(
source: &str,
proxy: Option<&str>,
) -> Result<Vec<u8>, String> {
ensure_reqwest_crypto_provider();
let proxy = proxy
.map(crate::queue::aria2_all_proxy_value)
.transpose()?
.flatten();
let mut current = reqwest::Url::parse(source)
.map_err(|_| "SSRF blocked: Invalid URL".to_string())?;
for redirect_count in 0..=MAX_REMOTE_TORRENT_REDIRECTS {
if !matches!(current.scheme(), "http" | "https") {
return Err("Remote torrent metadata must use HTTP or HTTPS".to_string());
}
if !current.username().is_empty() || current.password().is_some() {
return Err("Torrent metadata URLs must not contain credentials".to_string());
}
let (host, address) = validate_url_ssrf(current.as_str())
.await?
.ok_or_else(|| "SSRF blocked: No host".to_string())?;
let mut builder = reqwest::Client::builder()
.redirect(reqwest::redirect::Policy::none())
.timeout(FILE_METADATA_TIMEOUT)
.user_agent("Firelink torrent metadata");
match proxy.as_deref().map(str::trim) {
Some("") => builder = builder.no_proxy(),
Some(proxy) => {
builder = builder.proxy(reqwest::Proxy::all(proxy).map_err(|error| error.to_string())?);
}
None => {}
}
builder = builder.resolve(&host, address);
let client = builder
.build()
.map_err(|error| format!("could not create torrent metadata client: {error}"))?;
let response = client
.get(current.clone())
.header(
reqwest::header::ACCEPT,
"application/x-bittorrent, application/octet-stream",
)
.send()
.await
.map_err(|error| {
format!(
"remote torrent metadata request failed: {}",
crate::redact_sensitive_text(&error.to_string())
)
})?;
if response.status().is_redirection() {
if redirect_count == MAX_REMOTE_TORRENT_REDIRECTS {
return Err("Too many redirects while fetching torrent metadata".to_string());
}
let location = response
.headers()
.get(reqwest::header::LOCATION)
.and_then(|value| value.to_str().ok())
.ok_or_else(|| "Torrent metadata redirect has no valid location".to_string())?;
current = current
.join(location)
.map_err(|_| "Torrent metadata redirect URL is invalid".to_string())?;
continue;
}
if !response.status().is_success() {
return Err(format!(
"Remote torrent metadata request returned HTTP {}",
response.status().as_u16()
));
}
if response
.content_length()
.is_some_and(|length| length > crate::torrent::MAX_TORRENT_BYTES as u64)
{
return Err(format!(
"torrent metadata must be at most {} bytes",
crate::torrent::MAX_TORRENT_BYTES
));
}
let mut bytes = Vec::new();
let mut stream = response.bytes_stream();
while let Some(chunk) = stream.next().await {
let chunk = chunk.map_err(|error| {
format!(
"remote torrent metadata response failed: {}",
crate::redact_sensitive_text(&error.to_string())
)
})?;
if bytes
.len()
.checked_add(chunk.len())
.is_none_or(|length| length > crate::torrent::MAX_TORRENT_BYTES)
{
return Err(format!(
"torrent metadata must be at most {} bytes",
crate::torrent::MAX_TORRENT_BYTES
));
}
bytes.extend_from_slice(&chunk);
}
if bytes.is_empty() {
return Err("remote torrent metadata response was empty".to_string());
}
return Ok(bytes);
}
Err("Too many redirects while fetching torrent metadata".to_string())
}
fn same_origin(left: &reqwest::Url, right: &reqwest::Url) -> bool {
left.scheme() == right.scheme()
&& left.host() == right.host()
@@ -5878,6 +5998,23 @@ async fn inspect_torrent(
.await
.map_err(AppError::Internal);
}
if crate::torrent::is_remote_torrent_url(&source) {
let remote_source = source.trim();
let bytes = fetch_remote_torrent_bytes(remote_source, proxy.as_deref())
.await
.map_err(AppError::Internal)?;
let parsed = crate::torrent::parse_torrent_bytes(&bytes).map_err(AppError::Internal)?;
let torrent_path = if cache != Some(false) {
Some(
crate::torrent::cache_torrent_bytes(&app_handle, &id, &bytes)
.await
.map_err(AppError::Internal)?,
)
} else {
None
};
return Ok(crate::torrent::to_metadata(parsed, torrent_path));
}
if cache == Some(false) {
return crate::torrent::inspect_source(&source)
.map(|parsed| crate::torrent::to_metadata(parsed, None))
@@ -7573,6 +7710,26 @@ mod tests {
);
}
#[tokio::test]
async fn remote_torrent_fetch_rejects_non_http_and_embedded_credentials() {
assert_eq!(
super::fetch_remote_torrent_bytes("ftp://example.com/sample.torrent", None).await,
Err("Remote torrent metadata must use HTTP or HTTPS".to_string())
);
assert_eq!(
super::fetch_remote_torrent_bytes("https://user:pass@example.com/sample.torrent", None)
.await,
Err("Torrent metadata URLs must not contain credentials".to_string())
);
let proxy_error = super::fetch_remote_torrent_bytes(
"https://example.com/sample.torrent",
Some("socks5://127.0.0.1:1080"),
)
.await
.expect_err("remote Torrent metadata must use the shared proxy policy");
assert!(proxy_error.contains("SOCKS"));
}
#[tokio::test]
async fn enqueue_uri_validation_covers_mirrors_not_only_the_primary_url() {
assert_eq!(
+25
View File
@@ -407,6 +407,20 @@ pub fn inspect_source(source: &str) -> Result<ParsedTorrent, String> {
parse_torrent_bytes(&bytes)
}
/// Remote torrent metadata is fetched and cached before enqueue so it follows
/// the same validated `addTorrent`, ownership, retry, and restart path as
/// local metadata and magnets.
pub fn is_remote_torrent_url(source: &str) -> bool {
let Ok(parsed) = url::Url::parse(source.trim()) else {
return false;
};
matches!(parsed.scheme(), "http" | "https")
&& parsed
.path_segments()
.and_then(|segments| segments.last())
.is_some_and(|name| name.to_ascii_lowercase().ends_with(".torrent"))
}
pub fn to_metadata(parsed: ParsedTorrent, torrent_path: Option<String>) -> TorrentMetadata {
TorrentMetadata {
name: parsed.name,
@@ -743,6 +757,17 @@ mod tests {
);
}
#[test]
fn recognizes_only_http_torrent_metadata_urls() {
assert!(is_remote_torrent_url("https://example.com/files/sample.torrent"));
assert!(is_remote_torrent_url("http://example.com/sample.TORRENT?download=1"));
assert!(!is_remote_torrent_url("https://example.com/files/sample.zip"));
assert!(!is_remote_torrent_url("ftp://example.com/files/sample.torrent"));
assert!(!is_remote_torrent_url(
"magnet:?xt=urn:btih:0123456789012345678901234567890123456789"
));
}
#[test]
fn rejects_noncanonical_lengths_and_invalid_files_field() {
assert!(parse_torrent_bytes(b"d4:infod6:lengthi5e4:name04:testee").is_err());
+1 -3
View File
@@ -574,9 +574,7 @@ export const AddDownloadsModal = () => {
const requestContext = requestContextForUrl(contextUrl);
if (row.isTorrent) {
const torrentCacheId = row.torrentCacheId || `${row.id}-${row.generation}`;
const proxy = row.sourceUrl.trim().toLowerCase().startsWith('magnet:')
? await getProxyArgs(settingsStore)
: undefined;
const proxy = await getProxyArgs(settingsStore);
const torrentData = await invoke('inspect_torrent', {
source: row.sourceUrl,
id: torrentCacheId,
+14
View File
@@ -12,6 +12,7 @@ import {
mediaTypeForFormat,
metadataSummaryMessage,
isYouTubePlaylistUrl,
isRemoteTorrentUrl,
playlistFilePrefix,
reconcileDownloadRows,
refreshFailedMetadataRows,
@@ -106,6 +107,19 @@ describe('add download metadata workflow', () => {
expect(rows[1].torrentCacheId).toBe(`${rows[1].id}-1`);
});
it('admits remote .torrent URLs through the Torrent metadata path', () => {
expect(isRemoteTorrentUrl('https://example.com/files/sample.torrent?download=1')).toBe(true);
expect(isRemoteTorrentUrl('https://example.com/files/sample.zip')).toBe(false);
const rows = reconcileDownloadRows('https://example.com/files/sample.torrent?download=1', []);
expect(rows[0]).toMatchObject({
isTorrent: true,
isMedia: false,
status: 'loading'
});
});
it('gives refreshed torrent metadata a new cache identity', () => {
const existing = row({
id: 'torrent-row',
+11 -1
View File
@@ -91,6 +91,16 @@ const isLocalTorrentPath = (value: string): boolean => {
&& (value.startsWith('/') || /^[a-z]:[\\/]/i.test(value));
};
export const isRemoteTorrentUrl = (value: string): boolean => {
try {
const parsed = new URL(value);
return (parsed.protocol === 'http:' || parsed.protocol === 'https:')
&& parsed.pathname.toLowerCase().endsWith('.torrent');
} catch {
return false;
}
};
type ParsedInput = {
identity: string;
sourceUrl: string;
@@ -153,7 +163,7 @@ const parseInputLines = (
if (!isTorrent) {
const url = new URL(line);
valid = ALLOWED_SCHEMES.has(url.protocol);
isTorrent = valid && url.protocol === 'magnet:';
isTorrent = valid && (url.protocol === 'magnet:' || isRemoteTorrentUrl(sourceUrl));
if (valid) sourceUrl = url.href;
}
} catch {