Compare commits

..

10 Commits

Author SHA1 Message Date
leo 11de0bd408 reorder changelog 2026-09-02 14:15:23 +02:00
leo 5632e44a87 reorder changelog 2026-09-02 14:14:05 +02:00
leo 71f9032681 update upgrade.md to reflect options 2026-09-02 14:00:32 +02:00
leo 88dc205ac3 revert change regarding change in worker 2026-09-02 13:49:48 +02:00
leo 4b7e212db8 wip 2026-09-02 12:20:17 +02:00
leo b852f40778 lint 2026-09-02 12:11:14 +02:00
leo 56f9605bd3 address issues 2026-09-02 12:04:03 +02:00
leo 65bd76698f fix tets 2026-09-02 11:39:23 +02:00
leo 653e67de56 wip 2026-09-02 11:31:58 +02:00
leo 9cb19e1906 (backend) add per-recording encoding config to start-recording API
Add new options to query start-start recording API. A resolution
("540p", "720p", "1080p") and a profile ("talking_heads", "text", "mixed")
are resolved to provide a width, height, fps and bitrate which
are passed on to the encoder. Using profiles allows for some flexibility
on quality if necessary without changing front facing user config.

Co-authored-by: sarthakbahal <sarthakbahal.45@gmail.com>
2026-09-02 10:39:12 +02:00
29 changed files with 1122 additions and 714 deletions
+5 -2
View File
@@ -10,8 +10,11 @@ and this project adheres to
### Added
- ✨(frontend) add 1080p sending resolution option #1660
- ✨(backend) add Traefik support via configurable media-auth url header #1649
- ✨(backend) add per-recording encoding quality presets to start-recording API
### Changed
git
- 💥(backend) replace recording encoding options with a profile model
### Fixed
+122
View File
@@ -16,6 +16,128 @@ the following command inside your docker container:
## [Unreleased]
### Recording encoding settings replaced by a resolution/profile model
The `RECORDING_ENCODING_*` settings introduced in v1.16.0 exposed raw encoder
values (width, height, framerate, bitrate). They are replaced by two named and configurable sets of
dimensions, a **resolution** (default: `540p`, `720p`, `1080p`) and a **profile**
(default: `talking_heads`, `text`, `mixed`, `full`), which are resolved to the width, height,
fps and video bitrate.
**The following environment variables are no longer read. If they are still set in
your deployment they are silently ignored, and your recordings will be encoded with
the new defaults instead of your tuned values.**
| Removed variable | Replaced by |
| --------------------------------------- | ------------------------------------------------------------------------------------------------------------- |
| `RECORDING_ENCODING_ENABLED` | Nothing. A default encoding is now always built (see below). **Not** `RECORDING_CUSTOM_ENCODING_ENABLED`, which gates a different feature. |
| `RECORDING_ENCODING_WIDTH` | The `width` of the entry selected by `RECORDING_ENCODING_DEFAULT_RESOLUTION` in `RECORDING_ENCODING_AVAILABLE_RESOLUTIONS`. |
| `RECORDING_ENCODING_HEIGHT` | The `height` of that same entry. |
| `RECORDING_ENCODING_FRAMERATE` | The `fps` of the profile selected by `RECORDING_ENCODING_DEFAULT_PROFILE` in `RECORDING_ENCODING_AVAILABLE_PROFILES`. |
| `RECORDING_ENCODING_VIDEO_BITRATE_KBPS` | That profile's `kbps`. |
`RECORDING_ENCODING_AUDIO_BITRATE_KBPS` and `RECORDING_ENCODING_KEY_FRAME_INTERVAL_S`
keep their names and meaning. The keyframe interval now defaults to `0` (unset,
encoder's choice) instead of `4.0`.
#### If you never set `RECORDING_ENCODING_ENABLED=True`
The shipped defaults (`RECORDING_ENCODING_DEFAULT_PROFILE=full`,
`RECORDING_ENCODING_DEFAULT_RESOLUTION=720p`) match LiveKit's built-in
`H264_720P_30` preset: 1280×720, 30 fps, 3000 kbps H.264 MAIN, 128 kbps AAC.
Video output is therefore unchanged.
Audio and keyframing may not be. These values are now sent explicitly as advanced
`EncodingOptions` rather than relying on LiveKit's preset, so
`RECORDING_ENCODING_AUDIO_BITRATE_KBPS` and `RECORDING_ENCODING_KEY_FRAME_INTERVAL_S`
now apply to every recording. They previously applied only when
`RECORDING_ENCODING_ENABLED` was `True`. **If you set either of them while the
feature was disabled, they had no effect and now do**; check them before upgrading.
If you never set them, no action is required: 128 kbps AAC is what the preset used,
and the keyframe interval now defaults to `0`, which leaves the field unset so the
encoder keeps picking it as before. Set `RECORDING_ENCODING_KEY_FRAME_INTERVAL_S=4.0`
if you want fixed 4-second keyframes (the value the setting defaulted to while it
was gated behind `RECORDING_ENCODING_ENABLED`).
To keep letting LiveKit pick the encoding instead, set either default to an empty
value:
```
RECORDING_ENCODING_DEFAULT_RESOLUTION=
RECORDING_ENCODING_DEFAULT_PROFILE=
```
#### If you had tuned `RECORDING_ENCODING_*` values
Translate your old values into a default resolution and a default profile. Declare your own resolution and/or profile. Both maps are read from the
environment as a single-line Python/JSON dict literal (parsed with
`ast.literal_eval`, so use double-quoted keys and no trailing commas, and do not
add outer quotes in `.env`-style files):
```bash
RECORDING_ENCODING_AVAILABLE_RESOLUTIONS={"540p": {"width": 960, "height": 540}, "720p": {"width": 1280, "height": 720}, "1080p": {"width": 1920, "height": 1080}}
RECORDING_ENCODING_AVAILABLE_PROFILES={"my_old_profile": {"fps": 15, "kbps": {"540p": 350, "720p": 600, "1080p": 1100}}}
RECORDING_ENCODING_DEFAULT_RESOLUTION=720p
RECORDING_ENCODING_DEFAULT_PROFILE=my_old_profile
```
Both maps are validated at startup and a malformed one raises a `ValueError`:
- every entry of `RECORDING_ENCODING_AVAILABLE_RESOLUTIONS` must declare `width` and
`height`, and every entry of `RECORDING_ENCODING_AVAILABLE_PROFILES` an `fps` and a
`kbps` map;
- every profile must define a `kbps` entry for **exactly** the keys of
`RECORDING_ENCODING_AVAILABLE_RESOLUTIONS`; overriding one of the two maps usually
means overriding both;
- `RECORDING_ENCODING_DEFAULT_RESOLUTION` and `RECORDING_ENCODING_DEFAULT_PROFILE`,
when non-empty, must be keys of their respective map.
#### Breaking: custom worker services must accept `encoding_options`
Only concerns deployments pointing `RECORDING_WORKER_CLASSES` at their own worker
class. The shipped `VideoCompositeEgressService` and `AudioCompositeEgressService`
are already updated.
The `WorkerService` protocol's `start()` takes a third argument, and the mediator
now always passes it as a keyword when the recording carries no per-recording encoding:
```python
# before
def start(self, room_id: str, recording_id: str) -> str: ...
# now
def start(
self,
room_id: str,
recording_id: str,
encoding_options: Optional[Dict[str, Any]] = None,
) -> str: ...
```
#### Optional: per-recording encoding
`RECORDING_CUSTOM_ENCODING_ENABLED` (default `False`) toggles whether the
start-recording API accepts an `encoding` object
(`{"resolution": "720p", "profile": "talking_heads"}`, `profile` optional. It
falls back to `RECORDING_ENCODING_DEFAULT_PROFILE`) that overrides the default for
a single recording. It does not enable or disable the
default encoding, which is built from the two `RECORDING_ENCODING_DEFAULT_*`
settings either way. Leaving it at `False` preserves the previous behaviour, where
every recording uses the server-side encoding: requests carrying
`options.encoding` are rejected with a `400` before the recording is created, so
nothing is persisted and no egress is started.
Before enabling it:
- clients can only pick keys you declared; there is no way to send a raw width or bitrate
- as of this implementation, the frontend never sends `encoding`
- `encoding` is accepted but ignored for `transcript` recordings, whose audio-only
egress has no video encoding to configure.
See [docs/features/recording.md](docs/features/recording.md#tuning-recording-encoding)
for the full setting reference, the shipped profile table and the tuning caveats.
## v1.30.0
### Removing S3 storage-event webhooks for recordings
-1
View File
@@ -65,7 +65,6 @@ RUN apk update && apk upgrade \
musl \
musl-utils \
zlib>=1.3.2-r0 \
libexpat>=2.8.4-r0 \
&& apk del curl
USER nginx
+41 -34
View File
@@ -93,13 +93,13 @@ sequenceDiagram
| **RECORDING_WORKER_CLASSES** | Dict | `{ "screen_recording": "core.recording.worker.services.VideoCompositeEgressService", "transcript": "core.recording.worker.services.AudioCompositeEgressService" }` | Maps recording types to their worker service classes. |
| **RECORDING_EXPIRATION_DAYS** | Integer | `None` | Number of days before recordings expire. Should match bucket lifecycle policy. Set to `None` for no expiration. |
| **RECORDING_MAX_DURATION** | Integer | `None` | Maximum duration of a recording in milliseconds. Must be synced with the LiveKit Egress configuration. Set to None for unlimited duration. When the maximum duration is reached, the recording is automatically stopped and saved, and the user is prompted in the frontend with an alert message. |
| **RECORDING_ENCODING_ENABLED** | Boolean | `False` | When `False`, LiveKit Egress uses its built-in `H264_720P_30` preset. When `True`, the `RECORDING_ENCODING_*` values below are sent to LiveKit as advanced `EncodingOptions`. See [Tuning recording encoding](#tuning-recording-encoding). |
| **RECORDING_ENCODING_WIDTH** | Integer | `1280` | Recording video width in pixels. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_ENCODING_HEIGHT** | Integer | `720` | Recording video height in pixels. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_ENCODING_FRAMERATE** | Integer | `30` | Recording video framerate (fps). Directly impacts egress worker CPU (roughly linear). Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_ENCODING_VIDEO_BITRATE_KBPS** | Integer | `3000` | H.264 MAIN video bitrate in kbps. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_ENCODING_AUDIO_BITRATE_KBPS** | Integer | `128` | AAC audio bitrate in kbps. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_ENCODING_KEY_FRAME_INTERVAL_S** | Float | `4.0` | Keyframe interval in seconds. Drives seek granularity in the recorded MP4 (a player can only seek to keyframe boundaries). Larger values give the encoder slightly more bits for non-keyframe content at a fixed bitrate. `4.0` is a standard VOD value. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
| **RECORDING_CUSTOM_ENCODING_ENABLED** | Boolean | `False` | Whether the start-recording API accepts a per-recording `encoding` object (resolution/profile) that overrides the default. When `False`, the API rejects per-recording `encoding`; when `True`, clients may pick from the available resolutions/profiles. The default encoding below is applied regardless of this flag. See [Tuning recording encoding](#tuning-recording-encoding). |
| **RECORDING_ENCODING_AVAILABLE_RESOLUTIONS** | Dict | `{"540p": {"width": 960, "height": 540}, "720p": {"width": 1280, "height": 720}, "1080p": {"width": 1920, "height": 1080}}` | Maps a resolution name to its `{"width", "height"}` in pixels. Both the default encoding and the per-recording start-recording API pick from these keys. |
| **RECORDING_ENCODING_AVAILABLE_PROFILES** | Dict | `{"full": {"fps": 30, "kbps": {…}}, …}` | Maps a profile name to `{"fps", "kbps": {resolution: video_bitrate_kbps}}`. Every profile must define a bitrate for each available resolution (validated at startup). |
| **RECORDING_ENCODING_DEFAULT_RESOLUTION** | String | `"720p"` | Resolution used by the default encoding. When set, must be a key of `RECORDING_ENCODING_AVAILABLE_RESOLUTIONS`. Leave unset (together with, or instead of, the default profile) to disable the custom default encoding and fall back to LiveKit's built-in preset (a startup warning is emitted). |
| **RECORDING_ENCODING_DEFAULT_PROFILE** | String | `"full"` | Profile used by the default encoding. When set, must be a key of `RECORDING_ENCODING_AVAILABLE_PROFILES`. Leave unset (together with, or instead of, the default resolution) to disable the custom default encoding and fall back to LiveKit's built-in preset (a startup warning is emitted). |
| **RECORDING_ENCODING_AUDIO_BITRATE_KBPS** | Integer | `128` | AAC audio bitrate in kbps used in the default encoding. |
| **RECORDING_ENCODING_KEY_FRAME_INTERVAL_S** | Float | `0.0` | Keyframe interval in seconds. Drives seek granularity in the recorded MP4 (a player can only seek to keyframe boundaries). Larger values give the encoder slightly more bits for non-keyframe content at a fixed bitrate. `0` leaves the field unset, letting the encoder pick; `4.0` is a standard VOD value. |
> [!NOTE]
@@ -130,52 +130,59 @@ This allows you to verify which recordings are in progress, troubleshoot egress
## Tuning recording encoding
By default, LiveKit Egress records with the built-in `H264_720P_30` preset: 1280×720 at 30 fps, 3000 kbps H.264 MAIN video and 128 kbps AAC audio. For a one-hour meeting this produces a file of roughly **1.4 GB**, which is often heavier than necessary for talking-head content and screen sharing.
Every video recording is encoded from a default resolved from `RECORDING_ENCODING_DEFAULT_PROFILE` + `RECORDING_ENCODING_DEFAULT_RESOLUTION` and passed to LiveKit as advanced `EncodingOptions`. The shipped defaults (`full` profile) match LiveKit's built-in `H264_720P_30` preset. For a one-hour meeting that produces a file of roughly **1.4 GB**, which is often heavier than necessary for talking-head content and screen sharing; lowering the default profile/resolution shrinks it. If either default is left unset, no custom default encoding is built: a warning is logged at startup and LiveKit's built-in preset is used instead.
The `RECORDING_ENCODING_*` settings let operators override this preset without modifying the source. Values are passed straight through LiveKit's `EncodingOptions.advanced` to the GStreamer pipeline (`x264enc` for video, `faac` for audio), so there are no hidden conversions — what you set is what the encoder receives.
Encoding is chosen from two maps: `RECORDING_ENCODING_AVAILABLE_RESOLUTIONS` (`resolution → {"width", "height"}`) and `RECORDING_ENCODING_AVAILABLE_PROFILES` (`profile → {"fps", "kbps": {resolution: video_bitrate_kbps}}`):
- **Default**: `RECORDING_ENCODING_DEFAULT_PROFILE` + `RECORDING_ENCODING_DEFAULT_RESOLUTION` set the encoding used by every recording that doesn't override it. Leave either unset to fall back to LiveKit's built-in preset (a startup warning is emitted).
- **Per recording (opt-in)**: set `RECORDING_CUSTOM_ENCODING_ENABLED=True` to let clients override the default per recording. The start-recording API then accepts an `encoding` object selecting a `resolution` (required) and `profile` (optional): a resolution-only request keeps `RECORDING_ENCODING_DEFAULT_PROFILE` for fps and bitrate, so clients can only pick from pre-defined values. When `RECORDING_CUSTOM_ENCODING_ENABLED=False`, the API rejects any per-recording `encoding` and the default is used.
The resolved values are passed straight through LiveKit's `EncodingOptions.advanced` to the GStreamer pipeline (`x264enc` for video, `faac` for audio), so there are no hidden conversions — what the profile/resolution resolve to is what the encoder receives.
### How values map to GStreamer
| Setting | GStreamer element | Property |
| ------------------------------------- | ----------------- | ---------------------------------- |
| `RECORDING_ENCODING_WIDTH/HEIGHT` | capsfilter | `video/x-raw,width=W,height=H` |
| `RECORDING_ENCODING_FRAMERATE` | capsfilter | `framerate=F/1` |
| `RECORDING_ENCODING_VIDEO_BITRATE_KBPS` | `x264enc` | `bitrate=kbps` (kilobits) |
| `RECORDING_ENCODING_KEY_FRAME_INTERVAL_S` | `x264enc` | `key-int-max = interval × fps` |
| `RECORDING_ENCODING_AUDIO_BITRATE_KBPS` | `faac` | `bitrate = kbps × 1000` (bits) |
| Resolved value | GStreamer element | Property |
| ----------------------------------------- | ----------------- | ---------------------------------- |
| resolution `width` / `height` | capsfilter | `video/x-raw,width=W,height=H` |
| profile `fps` | capsfilter | `framerate=F/1` |
| profile `kbps[resolution]` | `x264enc` | `bitrate=kbps` (kilobits) |
| `RECORDING_ENCODING_KEY_FRAME_INTERVAL_S` | `x264enc` | `key-int-max = interval × fps` |
| `RECORDING_ENCODING_AUDIO_BITRATE_KBPS` | `faac` | `bitrate = kbps × 1000` (bits) |
The H.264 profile is fixed to MAIN and the x264 `speed-preset` to `veryfast` by LiveKit (real-time constraint) — lowering the framerate is therefore the main lever to save CPU, while lowering the bitrate is the main lever to shrink the output file.
### Reference profiles
### Built-in profiles
Rough 30-minute file-size estimates assume video + audio bitrate multiplied by duration. Actual sizes vary with content (static talking heads compress better than heavy screen motion). Egress CPU figures are indicative, measured on a single Ryzen laptop core saturated by the default preset (= 100 %); scaling is roughly linear with `framerate × bitrate` but the absolute numbers depend on the host hardware.
The default `RECORDING_ENCODING_AVAILABLE_PROFILES` ship four profiles. Framerate is fixed per profile; video bitrate (kbps) scales with resolution so quality stays consistent across sizes. File size scales roughly with `framerate × bitrate`, and so does egress CPU cost.
| Profile | Resolution | FPS | Video (kbps) | Audio (kbps) | Keyframe (s) | ~ size / 30 min | Egress CPU (vs. default) | Suitable for |
| ---------------------- | ---------- | --- | ------------ | ------------ | ------------ | --------------- | ------------------------ | --------------------------------------------------- |
| Default (preset) | 1280×720 | 30 | 3000 | 128 | 4 | **~690 MB** | 100 % | Unchanged LiveKit behaviour |
| Balanced | 1280×720 | 20 | 1000 | 96 | 4 | ~240 MB | ~67 % | Mixed content, moderate motion |
| **Low CPU / small file** | 1280×720 | 15 | 600 | 64 | 4 | **~150 MB** | ~50 % | Talking-head dominant meetings + occasional slides ★ |
| Slide-heavy | 1280×720 | 15 | 900 | 64 | 4 | ~210 MB | ~55 % | Frequent dense screen sharing (decks, IDE, docs) |
| Minimum CPU | 960×540 | 15 | 500 | 64 | 4 | ~125 MB | ~30 % | Voice-first meetings, readable text not required |
| Audio-heavy fallback | 1280×720 | 10 | 400 | 96 | 4 | ~110 MB | ~35 % | Long webinars, low motion |
| Profile | FPS | 540p (kbps) | 720p (kbps) | 1080p (kbps) | Suitable for |
| --------------- | --- | ----------- | ----------- | ------------ | -------------------------------------------------- |
| `talking_heads` | 15 | 400 | 700 | 1200 | Talking-head dominant meetings + occasional slides |
| `text` | 15 | 600 | 1000 | 1800 | Frequent dense screen sharing (decks, IDE, docs) |
| `mixed` | 20 | 900 | 1500 | 2500 | Mixed content, moderate motion |
| `full` | 30 | 2000 | 3000 | 4500 | Highest fidelity; closest to the LiveKit default preset |
★ Recommended starting point for typical LaSuite Meet usage.
To pick a profile per recording (requires `RECORDING_CUSTOM_ENCODING_ENABLED=True`), the client sends it in the start-recording request:
Environment variables for the **Low CPU / small file** profile:
```json
{
"mode": "screen_recording",
"options": {"encoding": {"resolution": "720p", "profile": "talking_heads"}}
}
```
To change the default encoding applied to every recording:
```bash
RECORDING_ENCODING_ENABLED=True
RECORDING_ENCODING_WIDTH=1280
RECORDING_ENCODING_HEIGHT=720
RECORDING_ENCODING_FRAMERATE=15
RECORDING_ENCODING_VIDEO_BITRATE_KBPS=600
RECORDING_ENCODING_DEFAULT_RESOLUTION=720p
RECORDING_ENCODING_DEFAULT_PROFILE=talking_heads
RECORDING_ENCODING_AUDIO_BITRATE_KBPS=64
RECORDING_ENCODING_KEY_FRAME_INTERVAL_S=4.0
```
### Caveats
- **Screen-share readability — think bits/frame, not bitrate**: at 720p, text legibility starts to break down below ~40 kbits/frame (= `bitrate ÷ framerate`). The recommended preset (600 kbps × 15 fps) sits at exactly that threshold, comfortable for talking heads with occasional slide sharing. The same 600 kbps at 30 fps would only deliver 20 kbits/frame and visibly blur dense slides — which is why **lowering framerate is a more screen-share-friendly lever than lowering bitrate**. For deck-heavy or IDE-share meetings, prefer the **Slide-heavy** profile (900 kbps × 15 fps ≈ 60 kbits/frame).
- **Screen-share readability — think bits/frame, not bitrate**: at 720p, text legibility starts to break down below ~40 kbits/frame (= `bitrate ÷ framerate`). The `talking_heads` profile (700 kbps × 15 fps) sits just above that threshold, comfortable for talking heads with occasional slide sharing. The same bitrate at 30 fps would only deliver ~23 kbits/frame and visibly blur dense slides — which is why **lowering framerate is a more screen-share-friendly lever than lowering bitrate**. For deck-heavy or IDE-share meetings, prefer the **`text`** profile (1000 kbps × 15 fps ≈ 67 kbits/frame).
- **Motion handling**: the `veryfast` x264 preset is set by LiveKit and cannot be overridden here. Low-bitrate settings will therefore show more artefacts on fast motion than an offline re-encode with a slower preset would. This is the other reason FPS reduction is the safer tuning lever for meeting recordings.
- **Audio**: AAC at 64 kbps stereo is transparent for voice but starts to compress music noticeably. Keep 128 kbps if you expect music playback in meetings.
- **Codec choice**: H.264 MAIN is hardcoded on purpose. Switching to HEVC or VP9 would increase egress CPU cost 2×–5×, defeating the goal of this tuning.
+50 -2
View File
@@ -13,7 +13,12 @@ from django.core.exceptions import SuspiciousOperation
from django.utils.translation import gettext_lazy as _
from django_pydantic_field.rest_framework import SchemaField
from pydantic import BaseModel, Field, field_serializer
from pydantic import (
BaseModel,
Field,
field_serializer,
field_validator,
)
from pydantic import ValidationError as PydanticValidationError
from rest_framework import serializers
from rest_framework.exceptions import PermissionDenied
@@ -244,6 +249,49 @@ class BaseValidationOnlySerializer(serializers.Serializer):
raise NotImplementedError(f"{self.__class__.__name__} is validation-only")
class EncodingConfig(BaseModel):
"""Configuration options for recording encoding.
The allowed `resolution` and `profile` values are derived at validation time
from ``settings.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS`` and
``settings.RECORDING_ENCODING_AVAILABLE_PROFILES``, so adding a resolution or profile
to those maps is enough to make it accepted here.
Attributes:
resolution: Target video resolution.
profile: Encoding profile to fps and kbps. When `None`,
`settings.RECORDING_ENCODING_DEFAULT_PROFILE` applies.
"""
resolution: str
profile: str | None = None
model_config = {"extra": "forbid"}
@field_validator("resolution")
@classmethod
def _validate_resolution(cls, value):
"""Reject resolutions absent from RECORDING_ENCODING_AVAILABLE_RESOLUTIONS."""
allowed = set(settings.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS)
if value not in allowed:
raise ValueError(
f"Invalid resolution '{value}'. Choose from {sorted(allowed)}."
)
return value
@field_validator("profile")
@classmethod
def _validate_profile(cls, value):
"""Reject profiles absent from RECORDING_ENCODING_AVAILABLE_PROFILES."""
if value is None:
return None
allowed = set(settings.RECORDING_ENCODING_AVAILABLE_PROFILES)
if value not in allowed:
raise ValueError(
f"Invalid profile '{value}'. Choose from {sorted(allowed)}."
)
return value
class RecordingOptions(BaseModel):
"""Configuration options for recording.
@@ -264,7 +312,7 @@ class RecordingOptions(BaseModel):
transcribe: bool | None = None
collect_metadata: bool | None = None
original_mode: Literal["screen_recording", "transcript"] | None = None
encoding: EncodingConfig | None = None
model_config = {"extra": "forbid"}
+31 -18
View File
@@ -55,6 +55,7 @@ from core.recording.worker.exceptions import (
RecordingStopError,
)
from core.recording.worker.factories import (
build_encoding_options,
get_worker_service,
)
from core.recording.worker.mediator import (
@@ -400,12 +401,34 @@ class RoomViewSet(
options = serializer.validated_data.get("options")
room = self.get_object()
if (
options is not None
and options.encoding is not None
and not settings.RECORDING_CUSTOM_ENCODING_ENABLED
):
# Per-recording encoding selection is gated by
# RECORDING_CUSTOM_ENCODING_ENABLED. When disabled, recordings use
# encoding defined by RECORDING_ENCODING_DEFAULT_RESOLUTION
# and RECORDING_ENCODING_DEFAULT_PROFILE.
return drf_response.Response(
{"detail": "Per-recording encoding selection is disabled."},
status=drf_status.HTTP_400_BAD_REQUEST,
)
options_data = options.model_dump(exclude_none=True) if options else {}
if options is not None and options.encoding is not None:
# Persist the resolved encoding (concrete width/height/framerate/
# bitrate) alongside the requested resolution/profile for traceability.
options_data["encoding"]["resolved"] = build_encoding_options(
options.encoding.resolution, options.encoding.profile
)
try:
with transaction.atomic():
recording = models.Recording.objects.create(
room=room,
mode=mode,
options=options.model_dump(exclude_none=True) if options else {},
options=options_data,
)
models.RecordingAccess.objects.create(
user=self.request.user,
@@ -1076,10 +1099,9 @@ class RecordingViewSet(
def _auth_get_original_url(self, request):
"""
Extracts and parses the original URL from the configured header.
Extracts and parses the original URL from the "HTTP_X_ORIGINAL_URL" header.
Raises PermissionDenied if the header is missing.
The original url is passed by the reverse proxy in the header named by the
MEDIA_AUTH_ORIGINAL_URL_HEADER setting, which defaults to "HTTP_X_ORIGINAL_URL".
The original url is passed by nginx in the "HTTP_X_ORIGINAL_URL" header.
See corresponding ingress configuration in Helm chart and read about the
nginx.ingress.kubernetes.io/auth-url annotation to understand how the Nginx ingress
is configured to do this.
@@ -1089,13 +1111,9 @@ class RecordingViewSet(
reasons.
"""
# Extract the original URL from the request header
original_url = request.META.get(settings.MEDIA_AUTH_ORIGINAL_URL_HEADER)
original_url = request.META.get("HTTP_X_ORIGINAL_URL")
if not original_url:
logger.warning(
"Missing %s header in subrequest. Set MEDIA_AUTH_ORIGINAL_URL_HEADER "
"to the header your reverse proxy sends.",
settings.MEDIA_AUTH_ORIGINAL_URL_HEADER,
)
logger.warning("Missing HTTP_X_ORIGINAL_URL header in subrequest")
raise drf_exceptions.PermissionDenied()
logger.debug("Original url: '%s'", original_url)
@@ -1420,8 +1438,7 @@ class FileViewSet(
Authorize access based on the original URL of an Nginx subrequest
and user permissions. Returns a dictionary of URL parameters if authorized.
The original url is passed by the reverse proxy in the header named by the
MEDIA_AUTH_ORIGINAL_URL_HEADER setting, which defaults to "HTTP_X_ORIGINAL_URL".
The original url is passed by nginx in the "HTTP_X_ORIGINAL_URL" header.
See corresponding ingress configuration in Helm chart and read about the
nginx.ingress.kubernetes.io/auth-url annotation to understand how the Nginx ingress
is configured to do this.
@@ -1440,13 +1457,9 @@ class FileViewSet(
- PermissionDenied if authorization fails.
"""
# Extract the original URL from the request header
original_url = request.META.get(settings.MEDIA_AUTH_ORIGINAL_URL_HEADER)
original_url = request.META.get("HTTP_X_ORIGINAL_URL")
if not original_url:
logger.warning(
"Missing %s header in subrequest. Set MEDIA_AUTH_ORIGINAL_URL_HEADER "
"to the header your reverse proxy sends.",
settings.MEDIA_AUTH_ORIGINAL_URL_HEADER,
)
logger.warning("Missing HTTP_X_ORIGINAL_URL header in subrequest")
raise drf_exceptions.PermissionDenied()
parsed_url = urlparse(original_url)
+55 -16
View File
@@ -22,6 +22,46 @@ _RECORDING_AUDIO_CODEC = livekit_api.AudioCodec.AAC
_RECORDING_AUDIO_FREQUENCY_HZ = 48000
def build_encoding_options(resolution, profile=None):
"""Assemble the LiveKit ``EncodingOptions`` kwargs for a resolution/profile.
Single source of truth shared by the default encoding
(``WorkerServiceConfig.from_settings``) and the per-recording encoding
persisted by the start-recording API, so both paths always produce the
same shape.
The profile-independent fields (audio bitrate, keyframe interval and the
pinned codec / frequency constants) are always included.
An omitted profile falls back to RECORDING_ENCODING_DEFAULT_PROFILE.
Framerate and bitrate are left to LiveKit only when the operator
declared no default profile at all.
"""
profile = profile or settings.RECORDING_ENCODING_DEFAULT_PROFILE
options: Dict[str, Any] = {
"audio_bitrate": settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS,
"key_frame_interval": settings.RECORDING_ENCODING_KEY_FRAME_INTERVAL_S,
"video_codec": _RECORDING_VIDEO_CODEC,
"audio_codec": _RECORDING_AUDIO_CODEC,
"audio_frequency": _RECORDING_AUDIO_FREQUENCY_HZ,
}
if resolution:
resolution_config = settings.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS[
resolution
]
options["width"] = resolution_config["width"]
options["height"] = resolution_config["height"]
if resolution and profile:
profile_config = settings.RECORDING_ENCODING_AVAILABLE_PROFILES[profile]
options["framerate"] = profile_config["fps"]
options["video_bitrate"] = profile_config["kbps"][resolution]
return options
@dataclass(frozen=True)
class WorkerServiceConfig:
"""Declare Worker Service common configurations"""
@@ -38,22 +78,16 @@ class WorkerServiceConfig:
logger.debug("Loading WorkerServiceConfig from settings.")
# The default encoding is resolved from the default profile/resolution and
# applied to every recording that carries no per-recording encoding.
# When either default is missing, we leave this as None so LiveKit falls
# back to its built-in preset.
resolution = settings.RECORDING_ENCODING_DEFAULT_RESOLUTION
profile = settings.RECORDING_ENCODING_DEFAULT_PROFILE
encoding_options: Optional[Dict[str, Any]] = None
if settings.RECORDING_ENCODING_ENABLED:
# Single source of truth for the EncodingOptions kwargs:
# operator-tunable values live in Django settings, codec / frequency
# are pinned constants. The services layer only unpacks this dict.
encoding_options = {
"width": settings.RECORDING_ENCODING_WIDTH,
"height": settings.RECORDING_ENCODING_HEIGHT,
"framerate": settings.RECORDING_ENCODING_FRAMERATE,
"video_bitrate": settings.RECORDING_ENCODING_VIDEO_BITRATE_KBPS,
"audio_bitrate": settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS,
"key_frame_interval": settings.RECORDING_ENCODING_KEY_FRAME_INTERVAL_S,
"video_codec": _RECORDING_VIDEO_CODEC,
"audio_codec": _RECORDING_AUDIO_CODEC,
"audio_frequency": _RECORDING_AUDIO_FREQUENCY_HZ,
}
if resolution and profile:
encoding_options = build_encoding_options(resolution, profile)
return cls(
output_folder=settings.RECORDING_OUTPUT_FOLDER,
@@ -78,7 +112,12 @@ class WorkerService(Protocol):
def __init__(self, config: WorkerServiceConfig):
"""Initialize the service with the given configuration."""
def start(self, room_id: str, recording_id: str) -> str:
def start(
self,
room_id: str,
recording_id: str,
encoding_options: Optional[Dict[str, Any]] = None,
) -> str:
"""Start a recording for a specified room."""
def stop(self, worker_id: str) -> str:
@@ -51,8 +51,11 @@ class WorkerServiceMediator:
raise RecordingStartError()
room_name = str(recording.room.id)
encoding_options = (recording.options.get("encoding") or {}).get("resolved")
try:
worker_id = self._worker_service.start(room_name, recording.id)
worker_id = self._worker_service.start(
room_name, recording.id, encoding_options=encoding_options
)
except (WorkerRequestError, WorkerConnectionError, WorkerResponseError) as e:
logger.exception(
"Failed to start recording for room %s: %s", recording.room.slug, e
+19 -14
View File
@@ -76,28 +76,28 @@ class BaseEgressService:
return "FAILED_TO_STOP"
def start(self, room_name, recording_id):
def start(self, room_name, recording_id, encoding_options=None):
"""Start the egress process for a recording (not implemented in the base class).
Each derived class must implement this method, providing the necessary parameters for
its specific egress type (e.g. audio_only, streaming output).
"""
raise NotImplementedError("Subclass must implement this method.")
def _build_encoding_options(self):
"""Build a LiveKit EncodingOptions from the service config, or None.
def _resolve_encoding_options(self, encoding_options):
"""Build a LiveKit EncodingOptions from a resolved kwargs dict, or None.
``encoding_options`` is the per-recording dict persisted by the API in
``recording.options["encoding"]["resolved"]``; it falls back to the
default encoding carried by the service config.
When None is returned, the caller should omit the `advanced` field so
LiveKit Egress falls back to its built-in preset (H264_720P_30).
The full EncodingOptions kwargs (operator-tunable values + pinned
codec / frequency constants) are assembled in `WorkerServiceConfig`,
so this method is a thin protobuf adapter.
"""
opts = self._config.encoding_options
if not opts:
encoding_options = encoding_options or self._config.encoding_options
if not encoding_options:
return None
return livekit_api.EncodingOptions(**opts)
return livekit_api.EncodingOptions(**encoding_options)
class VideoCompositeEgressService(BaseEgressService):
@@ -105,7 +105,7 @@ class VideoCompositeEgressService(BaseEgressService):
hrid = "video-recording-composite-livekit-egress"
def start(self, room_name, recording_id):
def start(self, room_name, recording_id, encoding_options=None):
"""Start the video composite egress process for a recording."""
# Save room's recording as a mp4 video file.
@@ -126,7 +126,7 @@ class VideoCompositeEgressService(BaseEgressService):
"layout": "speaker-light",
}
advanced = self._build_encoding_options()
advanced = self._resolve_encoding_options(encoding_options)
if advanced is not None:
request_kwargs["advanced"] = advanced
@@ -145,8 +145,13 @@ class AudioCompositeEgressService(BaseEgressService):
hrid = "audio-recording-composite-livekit-egress"
def start(self, room_name, recording_id):
"""Start the audio composite egress process for a recording."""
def start(self, room_name, recording_id, encoding_options=None):
"""Start the audio composite egress process for a recording.
``encoding_options`` is accepted for signature compatibility with the
WorkerService protocol but ignored: audio-only egress has no
encoding to configure.
"""
# Save room's recording as an ogg audio file.
file_type = livekit_api.EncodedFileType.OGG
@@ -7,7 +7,6 @@ from urllib.parse import quote, urlparse
from django.conf import settings
from django.core.files.storage import default_storage
from django.test import override_settings
from django.utils import timezone
import pytest
@@ -144,59 +143,3 @@ def test_api_files_media_auth_own_file_deleted():
)
assert response.status_code == 403
@override_settings(MEDIA_AUTH_ORIGINAL_URL_HEADER="HTTP_X_FORWARDED_URI")
def test_api_files_media_auth_custom_original_url_header():
"""
Authorization should honour the configured original-url header.
Covers the attachment subrequest path, which resolves the header separately
from the recording one. Reverse proxies other than nginx-ingress use
different headers: Traefik's ForwardAuth sends X-Forwarded-Uri and cannot
emit X-Original-URL at all.
"""
user = factories.UserFactory()
file = factories.FileFactory(
type=models.FileTypeChoices.BACKGROUND_IMAGE,
update_upload_state=models.FileUploadStateChoices.READY,
creator=user,
)
client = APIClient()
client.force_login(user)
default_storage.save(file.file_key, BytesIO(b"my prose"))
original_url = f"http://localhost/media/{file.file_key:s}"
response = client.get(
"/api/v1.0/files/media-auth/", HTTP_X_FORWARDED_URI=original_url
)
assert response.status_code == 200
assert "AWS4-HMAC-SHA256 Credential=" in response["Authorization"]
@override_settings(MEDIA_AUTH_ORIGINAL_URL_HEADER="HTTP_X_FORWARDED_URI")
def test_api_files_media_auth_default_header_ignored_when_reconfigured():
"""
Only the configured header should be honoured, never a hardcoded fallback.
"""
user = factories.UserFactory()
file = factories.FileFactory(
type=models.FileTypeChoices.BACKGROUND_IMAGE,
update_upload_state=models.FileUploadStateChoices.READY,
creator=user,
)
client = APIClient()
client.force_login(user)
original_url = f"http://localhost/media/{file.file_key:s}"
response = client.get(
"/api/v1.0/files/media-auth/", HTTP_X_ORIGINAL_URL=original_url
)
assert response.status_code == 403
@@ -8,7 +8,6 @@ from uuid import uuid4
from django.conf import settings
from django.core.files.storage import default_storage
from django.test import override_settings
from django.utils import timezone
import pytest
@@ -283,63 +282,3 @@ def test_api_recordings_media_auth_success_administrator(mode):
timeout=1,
)
assert response.content.decode("utf-8") == "my prose"
def test_api_recordings_media_auth_missing_header():
"""
Test that a subrequest without the configured original-url header is rejected.
"""
user = UserFactory()
client = APIClient()
client.force_login(user)
response = client.get("/api/v1.0/recordings/media-auth/")
assert response.status_code == 403
@override_settings(MEDIA_AUTH_ORIGINAL_URL_HEADER="HTTP_X_FORWARDED_URI")
def test_api_recordings_media_auth_custom_original_url_header():
"""
Test that the header carrying the original URL can be configured.
Reverse proxies other than nginx-ingress use different headers: Traefik's
ForwardAuth sends X-Forwarded-Uri and cannot emit X-Original-URL at all.
"""
user = UserFactory()
client = APIClient()
client.force_login(user)
original_url = f"http://localhost/media/recordings/{uuid4()!s}.mp4"
response = client.get(
"/api/v1.0/recordings/media-auth/", HTTP_X_FORWARDED_URI=original_url
)
# The header was read and parsed: we get as far as looking the recording up,
# rather than being rejected for a missing header.
assert response.status_code == 404
@override_settings(MEDIA_AUTH_ORIGINAL_URL_HEADER="HTTP_X_FORWARDED_URI")
def test_api_recordings_media_auth_default_header_ignored_when_reconfigured():
"""
Test that only the configured header is honoured.
Guards against the header being read from a hardcoded name in parallel with
the setting.
"""
user = UserFactory()
client = APIClient()
client.force_login(user)
original_url = f"http://localhost/media/recordings/{uuid4()!s}.mp4"
response = client.get(
"/api/v1.0/recordings/media-auth/", HTTP_X_ORIGINAL_URL=original_url
)
assert response.status_code == 403
@@ -0,0 +1,169 @@
"""Tests for the per-recording encoding resolution in BaseEgressService."""
# pylint: disable=protected-access,redefined-outer-name,unused-argument,no-member
from unittest.mock import Mock
from django.conf import settings
from django.test import override_settings
import pytest
from livekit import api as livekit_api
from pydantic import ValidationError as PydanticValidationError
from core.api.serializers import EncodingConfig
from core.recording.worker.factories import build_encoding_options
from core.recording.worker.services import VideoCompositeEgressService
def make_config():
"""Build a minimal WorkerServiceConfig-like mock for service instantiation."""
config = Mock()
config.bucket_args = {
"endpoint": "https://s3.test.com",
"access_key": "test_key",
"secret": "test_secret",
"region": "test-region",
"bucket": "test-bucket",
"force_path_style": True,
}
config.encoding_options = None
return config
@pytest.fixture
def service():
"""Return a VideoCompositeEgressService with mocked handle_request."""
svc = VideoCompositeEgressService(make_config())
svc._handle_request = Mock()
return svc
# --- build_encoding_options ---
def test_build_options_without_profile_uses_default_profile():
"""A resolution-only config should fall back to the default profile.
Left unset, framerate and video_bitrate take LiveKit's own EncodingOptions
defaults. The profile-independent fields (audio bitrate, keyframe interval,
codec/frequency pins) are always present, matching the default encoding.
"""
default_profile = settings.RECORDING_ENCODING_AVAILABLE_PROFILES[
settings.RECORDING_ENCODING_DEFAULT_PROFILE
]
resolved = build_encoding_options("540p")
assert resolved == {
"audio_bitrate": settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS,
"key_frame_interval": settings.RECORDING_ENCODING_KEY_FRAME_INTERVAL_S,
"video_codec": livekit_api.VideoCodec.H264_MAIN,
"audio_codec": livekit_api.AudioCodec.AAC,
"audio_frequency": 48000,
"width": 960,
"height": 540,
"framerate": default_profile["fps"],
"video_bitrate": default_profile["kbps"]["540p"],
}
@override_settings(RECORDING_ENCODING_DEFAULT_PROFILE="")
def test_build_options_omits_profile_fields_without_default_profile():
"""With no default profile declared, framerate/bitrate are left to LiveKit."""
resolved = build_encoding_options("720p", None)
assert resolved == {
"audio_bitrate": settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS,
"key_frame_interval": settings.RECORDING_ENCODING_KEY_FRAME_INTERVAL_S,
"video_codec": livekit_api.VideoCodec.H264_MAIN,
"audio_codec": livekit_api.AudioCodec.AAC,
"audio_frequency": 48000,
"width": 1280,
"height": 720,
}
assert "framerate" not in resolved
assert "video_bitrate" not in resolved
def test_encoding_config_requires_resolution():
"""A profile-only or empty encoding config should be rejected at validation."""
with pytest.raises(PydanticValidationError):
EncodingConfig(profile="mixed")
with pytest.raises(PydanticValidationError):
EncodingConfig()
# --- _resolve_encoding_options ---
@pytest.mark.parametrize("encoding_options", [None, {}])
def test_resolve_options_returns_none_when_empty(service, encoding_options):
"""Resolver should return None when the resolved dict is empty or missing."""
assert service._resolve_encoding_options(encoding_options) is None
@pytest.mark.parametrize(
"resolution",
list(settings.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS),
)
@pytest.mark.parametrize(
"profile",
list(settings.RECORDING_ENCODING_AVAILABLE_PROFILES),
)
def test_resolve_profile_resolution_combinations(service, profile, resolution):
"""Every (profile, resolution) pair should resolve to the values from settings."""
resolution_config = settings.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS[resolution]
expected_width = resolution_config["width"]
expected_height = resolution_config["height"]
profile_config = settings.RECORDING_ENCODING_AVAILABLE_PROFILES[profile]
expected_fps = profile_config["fps"]
expected_bitrate = profile_config["kbps"][resolution]
resolved = build_encoding_options(resolution, profile)
result = service._resolve_encoding_options(resolved)
assert result.width == expected_width
assert result.height == expected_height
assert result.framerate == expected_fps
assert result.video_bitrate == expected_bitrate
# Profile-independent fields match the default encoding, never dropped.
assert result.audio_bitrate == settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS
assert result.video_codec == livekit_api.VideoCodec.H264_MAIN
assert result.audio_codec == livekit_api.AudioCodec.AAC
assert result.audio_frequency == 48000
def test_resolve_options_none_profile_uses_default_profile(service):
"""A missing profile should resolve to the default profile's fps/bitrate."""
default_profile = settings.RECORDING_ENCODING_AVAILABLE_PROFILES[
settings.RECORDING_ENCODING_DEFAULT_PROFILE
]
resolved = build_encoding_options("720p", None)
result = service._resolve_encoding_options(resolved)
assert result.width == 1280
assert result.height == 720
assert result.framerate == default_profile["fps"]
assert result.video_bitrate == default_profile["kbps"]["720p"]
assert result.audio_bitrate == settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS
assert result.video_codec == livekit_api.VideoCodec.H264_MAIN
@override_settings(RECORDING_ENCODING_DEFAULT_PROFILE="")
def test_resolve_options_passes_zero_when_no_default_profile(service):
"""With no default profile, fps/bitrate reach LiveKit unset (protobuf 0).
The pinned codec / audio fields are still applied.
"""
resolved = build_encoding_options("720p", None)
result = service._resolve_encoding_options(resolved)
assert result.width == 1280
assert result.height == 720
assert result.framerate == 0
assert result.video_bitrate == 0
assert result.audio_bitrate == settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS
assert result.video_codec == livekit_api.VideoCodec.H264_MAIN
assert result.audio_codec == livekit_api.AudioCodec.AAC
@@ -40,6 +40,16 @@ def test_settings():
"AWS_S3_SECRET_ACCESS_KEY": "test_secret",
"AWS_S3_REGION_NAME": "test-region",
"AWS_STORAGE_BUCKET_NAME": "test-bucket",
"RECORDING_ENCODING_AVAILABLE_RESOLUTIONS": {
"720p": {"width": 1280, "height": 720}
},
"RECORDING_ENCODING_AVAILABLE_PROFILES": {
"full": {"fps": 30, "kbps": {"720p": 3000}}
},
"RECORDING_ENCODING_DEFAULT_RESOLUTION": "720p",
"RECORDING_ENCODING_DEFAULT_PROFILE": "full",
"RECORDING_ENCODING_AUDIO_BITRATE_KBPS": 128,
"RECORDING_ENCODING_KEY_FRAME_INTERVAL_S": 4.0,
}
# Use override_settings to properly patch Django settings
@@ -66,8 +76,18 @@ def test_config_initialization(default_config):
"bucket": "test-bucket",
"force_path_style": True,
}
# Encoding override is opt-in; disabled by default.
assert default_config.encoding_options is None
# The default encoding is always resolved from the default profile/resolution.
assert default_config.encoding_options == {
"width": 1280,
"height": 720,
"framerate": 30,
"video_bitrate": 3000,
"audio_bitrate": 128,
"key_frame_interval": 4.0,
"video_codec": livekit_api_codec.VideoCodec.H264_MAIN,
"audio_codec": livekit_api_codec.AudioCodec.AAC,
"audio_frequency": 48000,
}
def test_config_immutability(default_config):
@@ -76,6 +96,7 @@ def test_config_immutability(default_config):
default_config.output_folder = "new/path"
@pytest.mark.parametrize("custom_encoding_enabled", [True, False])
@override_settings(
RECORDING_OUTPUT_FOLDER="/test/output",
LIVEKIT_CONFIGURATION={"server": "test.example.com"},
@@ -84,23 +105,25 @@ def test_config_immutability(default_config):
AWS_S3_SECRET_ACCESS_KEY="test_secret",
AWS_S3_REGION_NAME="test-region",
AWS_STORAGE_BUCKET_NAME="test-bucket",
RECORDING_ENCODING_ENABLED=True,
RECORDING_ENCODING_WIDTH=1280,
RECORDING_ENCODING_HEIGHT=720,
RECORDING_ENCODING_FRAMERATE=15,
RECORDING_ENCODING_VIDEO_BITRATE_KBPS=600,
RECORDING_ENCODING_AVAILABLE_RESOLUTIONS={"720p": {"width": 1280, "height": 720}},
RECORDING_ENCODING_AVAILABLE_PROFILES={"low": {"fps": 15, "kbps": {"720p": 600}}},
RECORDING_ENCODING_DEFAULT_RESOLUTION="720p",
RECORDING_ENCODING_DEFAULT_PROFILE="low",
RECORDING_ENCODING_AUDIO_BITRATE_KBPS=64,
RECORDING_ENCODING_KEY_FRAME_INTERVAL_S=10.0,
)
def test_config_encoding_options_enabled():
"""When RECORDING_ENCODING_ENABLED is True, encoding options are populated.
def test_config_encoding_options_default(custom_encoding_enabled):
"""The default encoding is always resolved from the default profile/resolution.
The dict mixes operator-tunable values from settings with pinned codec /
frequency constants, so the services layer can simply unpack it.
The default fallback resolves the default profile/resolution and mixes those
operator-tunable values with pinned codec / frequency constants. This works
regardless of RECORDING_CUSTOM_ENCODING_ENABLED, which only gates the
per-recording API, so both toggle states produce the same default.
"""
WorkerServiceConfig.from_settings.cache_clear()
config = WorkerServiceConfig.from_settings()
with override_settings(RECORDING_CUSTOM_ENCODING_ENABLED=custom_encoding_enabled):
WorkerServiceConfig.from_settings.cache_clear()
config = WorkerServiceConfig.from_settings()
assert config.encoding_options == {
"width": 1280,
@@ -115,6 +138,27 @@ def test_config_encoding_options_enabled():
}
@pytest.mark.parametrize(
("default_resolution", "default_profile"),
[("", "full"), ("720p", ""), ("", "")],
)
def test_config_encoding_options_none_when_default_missing(
test_settings, default_resolution, default_profile
):
"""A missing default resolution/profile leaves encoding_options None.
The service then omits the `advanced` field so LiveKit uses its built-in preset.
"""
with override_settings(
RECORDING_ENCODING_DEFAULT_RESOLUTION=default_resolution,
RECORDING_ENCODING_DEFAULT_PROFILE=default_profile,
):
WorkerServiceConfig.from_settings.cache_clear()
config = WorkerServiceConfig.from_settings()
assert config.encoding_options is None
@override_settings(
RECORDING_OUTPUT_FOLDER="/test/output",
LIVEKIT_CONFIGURATION={"server": "test.example.com"},
@@ -50,7 +50,7 @@ def test_start_recording_success(mock_update_metadata, mediator, mock_worker_ser
# Verify worker service call
expected_room_name = str(mock_recording.room.id)
mock_worker_service.start.assert_called_once_with(
expected_room_name, mock_recording.id
expected_room_name, mock_recording.id, encoding_options=None
)
# Verify recording updates
@@ -64,6 +64,38 @@ def test_start_recording_success(mock_update_metadata, mediator, mock_worker_ser
)
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
def test_start_recording_passes_resolved_encoding(
mock_update_metadata, mediator, mock_worker_service
):
"""The resolved encoding persisted in recording.options reaches the worker."""
mock_worker_service.start.return_value = "test-worker-123"
resolved = {
"key_frame_interval": 4.0,
"width": 1280,
"height": 720,
"framerate": 15,
"video_bitrate": 700,
}
mock_recording = RecordingFactory(
status=RecordingStatusChoices.INITIATED,
worker_id=None,
options={
"encoding": {
"resolution": "720p",
"profile": "talking_heads",
"resolved": resolved,
}
},
)
mediator.start(mock_recording)
mock_worker_service.start.assert_called_once_with(
str(mock_recording.room.id), mock_recording.id, encoding_options=resolved
)
@pytest.mark.parametrize(
"error_class", [WorkerRequestError, WorkerConnectionError, WorkerResponseError]
)
@@ -2,11 +2,12 @@
Test rooms API endpoints in the Meet core app: start recording.
"""
# pylint: disable=redefined-outer-name,unused-argument
# pylint: disable=redefined-outer-name,unused-argument,no-member
from unittest import mock
import pytest
from livekit import api as livekit_api
from rest_framework.test import APIClient
from ...factories import RoomFactory, UserFactory
@@ -470,6 +471,224 @@ def test_start_recording_options_unknown_field_rejected(settings):
assert response.status_code == 400
def test_start_recording_options_encoding_valid(
settings, mock_worker_service_factory, mock_worker_manager
):
"""Should accept a valid encoding configuration."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{
"mode": "screen_recording",
"options": {"encoding": {"resolution": "720p", "profile": "talking_heads"}},
},
format="json",
)
assert response.status_code == 201
def test_start_recording_options_encoding_rejected_when_custom_encoding_disabled(
settings, mock_worker_service_factory, mock_worker_manager
):
"""Per-recording encoding is rejected when RECORDING_CUSTOM_ENCODING_ENABLED is off."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = False
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{
"mode": "screen_recording",
"options": {"encoding": {"resolution": "720p", "profile": "talking_heads"}},
},
format="json",
)
assert response.status_code == 400
assert not Recording.objects.filter(room=room).exists()
def test_start_recording_persists_resolved_encoding(
settings, mock_worker_service_factory, mock_worker_manager
):
"""The resolved encoding should be persisted in recording.options alongside
the requested resolution/profile for traceability."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{
"mode": "screen_recording",
"options": {"encoding": {"resolution": "720p", "profile": "talking_heads"}},
},
format="json",
)
assert response.status_code == 201
recording = Recording.objects.get(room=room)
assert recording.options["encoding"] == {
"resolution": "720p",
"profile": "talking_heads",
"resolved": {
"audio_bitrate": settings.RECORDING_ENCODING_AUDIO_BITRATE_KBPS,
"key_frame_interval": settings.RECORDING_ENCODING_KEY_FRAME_INTERVAL_S,
"video_codec": livekit_api.VideoCodec.H264_MAIN,
"audio_codec": livekit_api.AudioCodec.AAC,
"audio_frequency": 48000,
"width": 1280,
"height": 720,
"framerate": 15,
"video_bitrate": 700,
},
}
def test_start_recording_resolution_only_uses_default_profile(
settings, mock_worker_service_factory, mock_worker_manager
):
"""An encoding without a profile should resolve the default profile."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
settings.RECORDING_ENCODING_DEFAULT_PROFILE = "talking_heads"
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording", "options": {"encoding": {"resolution": "540p"}}},
format="json",
)
assert response.status_code == 201
recording = Recording.objects.get(room=room)
resolved = recording.options["encoding"]["resolved"]
assert resolved["width"] == 960
assert resolved["height"] == 540
assert resolved["framerate"] == 15
assert resolved["video_bitrate"] == 400
# The requested payload is persisted as sent: no profile was asked for.
assert "profile" not in recording.options["encoding"]
def test_start_recording_forwards_resolved_encoding_to_worker(
settings, mock_worker_service, mock_worker_service_factory
):
"""The resolved encoding should passed on to the worker."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
mock_worker_service.start.return_value = "egress-123"
with mock.patch("core.services.room_management.RoomManagement.update_metadata"):
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{
"mode": "screen_recording",
"options": {
"encoding": {"resolution": "720p", "profile": "talking_heads"}
},
},
format="json",
)
assert response.status_code == 201
recording = Recording.objects.get(room=room)
mock_worker_service.start.assert_called_once_with(
str(room.id),
recording.id,
encoding_options=recording.options["encoding"]["resolved"],
)
def test_start_recording_options_encoding_invalid_resolution(settings):
"""Should reject invalid encoding resolution values."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording", "options": {"encoding": {"resolution": "4K"}}},
format="json",
)
assert response.status_code == 400
def test_start_recording_options_encoding_unknown_key_rejected(settings):
"""Should reject unknown keys in encoding configuration."""
settings.RECORDING_ENABLE = True
settings.RECORDING_CUSTOM_ENCODING_ENABLED = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{
"mode": "screen_recording",
"options": {"encoding": {"bitrate": 9000}},
},
format="json",
)
assert response.status_code == 400
def test_start_recording_options_without_encoding_unchanged(
settings, mock_worker_service_factory, mock_worker_manager
):
"""Requests without encoding should keep existing options behavior."""
settings.RECORDING_ENABLE = True
room = RoomFactory()
user = UserFactory()
room.accesses.create(user=user, role="owner")
client = APIClient()
client.force_login(user)
response = client.post(
f"/api/v1.0/rooms/{room.id}/start-recording/",
{"mode": "screen_recording", "options": {"language": "fr"}},
format="json",
)
assert response.status_code == 201
recording = Recording.objects.get(room=room)
assert recording.options == {"language": "fr"}
@pytest.mark.parametrize("value", ["foo", 12])
def test_start_recording_options_invalid_transcribe_type(settings, value):
"""Should reject non-boolean transcribe values."""
+173 -29
View File
@@ -23,6 +23,8 @@ import dj_database_url
import sentry_sdk
from configurations import Configuration, values
from lasuite.configuration.values import SecretFileValue
from pydantic import BaseModel, PositiveInt, TypeAdapter
from pydantic import ValidationError as PydanticValidationError
from sentry_sdk.integrations.django import DjangoIntegration
from sentry_sdk.integrations.logging import ignore_logger
@@ -53,6 +55,28 @@ def get_release():
return "NA" # Default: not available
class Resolution(BaseModel):
"""Shape of a RECORDING_ENCODING_AVAILABLE_RESOLUTIONS entry."""
model_config = {"extra": "forbid"}
width: PositiveInt
height: PositiveInt
class Profile(BaseModel):
"""Shape of a RECORDING_ENCODING_AVAILABLE_PROFILES entry."""
model_config = {"extra": "forbid"}
fps: PositiveInt
kbps: dict[str, PositiveInt]
RESOLUTION_MAP_ADAPTER = TypeAdapter(dict[str, Resolution])
PROFILE_MAP_ADAPTER = TypeAdapter(dict[str, Profile])
class Base(Configuration):
"""
This is the base configuration every configuration (aka environment) should inherit from. It
@@ -129,15 +153,6 @@ class Base(Configuration):
MEDIA_BASE_URL = values.Value(
"", environ_name="MEDIA_BASE_URL", environ_prefix=None
)
# Header the reverse proxy uses to pass the original request URL to the
# media-auth subrequest views. nginx-ingress sends X-Original-URL, which is
# the default. Other proxies use different headers -- Traefik's ForwardAuth,
# for instance, sends X-Forwarded-Uri and cannot emit X-Original-URL at all.
MEDIA_AUTH_ORIGINAL_URL_HEADER = values.Value(
default="HTTP_X_ORIGINAL_URL",
environ_name="MEDIA_AUTH_ORIGINAL_URL_HEADER",
environ_prefix=None,
)
SITE_ID = 1
@@ -754,35 +769,81 @@ class Base(Configuration):
# These settings affect screen recordings handled by VideoCompositeEgressService;
# they are silently ignored by AudioCompositeEgressService (audio-only transcript
# recordings), whose request never carries advanced EncodingOptions.
# When disabled, LiveKit falls back to its built-in H264_720P_30 preset
# (1280x720, 30 fps, 3000 kbps H.264 MAIN video, 128 kbps AAC audio).
# When enabled, the values below are passed to LiveKit as EncodingOptions
# (advanced) and replace the preset. Lowering framerate and bitrate reduces
# output file size and CPU load on the egress worker.
RECORDING_ENCODING_ENABLED = values.BooleanValue(
False, environ_name="RECORDING_ENCODING_ENABLED", environ_prefix=None
#
# A default encoding is applied to every recording: it is resolved from the default
# profile and resolution below and passed to LiveKit as EncodingOptions (advanced),
# replacing LiveKit's built-in H264_720P_30 preset. Lowering framerate and bitrate
# reduces output file size and CPU load on the egress worker. If either
# RECORDING_ENCODING_DEFAULT_RESOLUTION or RECORDING_ENCODING_DEFAULT_PROFILE is
# unset, no default encoding is built (a startup warning is emitted) and LiveKit's
# built-in preset is used instead.
#
# RECORDING_CUSTOM_ENCODING_ENABLED gates whether the start-recording API lets a
# client override that default per recording (via an `encoding` object selecting a
# resolution/profile). When False, the API rejects per-recording `encoding` and
# every recording uses the default; when True, clients may pick from the
# available resolutions/profiles below.
RECORDING_CUSTOM_ENCODING_ENABLED = values.BooleanValue(
False, environ_name="RECORDING_CUSTOM_ENCODING_ENABLED", environ_prefix=None
)
RECORDING_ENCODING_WIDTH = values.PositiveIntegerValue(
1280, environ_name="RECORDING_ENCODING_WIDTH", environ_prefix=None
)
RECORDING_ENCODING_HEIGHT = values.PositiveIntegerValue(
720, environ_name="RECORDING_ENCODING_HEIGHT", environ_prefix=None
)
RECORDING_ENCODING_FRAMERATE = values.PositiveIntegerValue(
30, environ_name="RECORDING_ENCODING_FRAMERATE", environ_prefix=None
)
RECORDING_ENCODING_VIDEO_BITRATE_KBPS = values.PositiveIntegerValue(
3000,
environ_name="RECORDING_ENCODING_VIDEO_BITRATE_KBPS",
# Map resolution string -> {"width", "height"} in pixels.
RECORDING_ENCODING_AVAILABLE_RESOLUTIONS = values.DictValue(
{
"540p": {"width": 960, "height": 540},
"720p": {"width": 1280, "height": 720},
"1080p": {"width": 1920, "height": 1080},
},
environ_name="RECORDING_ENCODING_AVAILABLE_RESOLUTIONS",
environ_prefix=None,
)
# Map profile string -> {"fps", "kbps": {resolution: video_bitrate_kbps}}.
# Bitrate scales with resolution so quality stays consistent across sizes.
RECORDING_ENCODING_AVAILABLE_PROFILES = values.DictValue(
{
"talking_heads": {
"fps": 15,
"kbps": {"540p": 400, "720p": 700, "1080p": 1200},
},
"text": {
"fps": 15,
"kbps": {"540p": 600, "720p": 1000, "1080p": 1800},
},
"mixed": {
"fps": 20,
"kbps": {"540p": 900, "720p": 1500, "1080p": 2500},
},
"full": {
"fps": 30,
"kbps": {"540p": 2000, "720p": 3000, "1080p": 4500},
},
},
environ_name="RECORDING_ENCODING_AVAILABLE_PROFILES",
environ_prefix=None,
)
# Defaults used when no profile/resolution is specified per recording.
# Must be keys of the two dicts above (validated at startup).
RECORDING_ENCODING_DEFAULT_PROFILE = values.Value(
"full",
environ_name="RECORDING_ENCODING_DEFAULT_PROFILE",
environ_prefix=None,
)
RECORDING_ENCODING_DEFAULT_RESOLUTION = values.Value(
"720p",
environ_name="RECORDING_ENCODING_DEFAULT_RESOLUTION",
environ_prefix=None,
)
# Settings independent of profile/resolution.
RECORDING_ENCODING_AUDIO_BITRATE_KBPS = values.PositiveIntegerValue(
128,
environ_name="RECORDING_ENCODING_AUDIO_BITRATE_KBPS",
environ_prefix=None,
)
RECORDING_ENCODING_KEY_FRAME_INTERVAL_S = values.FloatValue(
4.0,
0.0,
environ_name="RECORDING_ENCODING_KEY_FRAME_INTERVAL_S",
environ_prefix=None,
)
@@ -790,6 +851,7 @@ class Base(Configuration):
SUMMARY_SERVICE_VERSION = values.PositiveIntegerValue(
1, environ_name="SUMMARY_SERVICE_VERSION", environ_prefix=None
)
SUMMARY_SERVICE_ENDPOINT = values.Value(
None, environ_name="SUMMARY_SERVICE_ENDPOINT", environ_prefix=None
)
@@ -1178,6 +1240,86 @@ class Base(Configuration):
},
}
@classmethod
def _check_recording_encoding_maps(cls):
"""Ensure the per-recording encoding maps are well-formed and consistent.
Each entry of RECORDING_ENCODING_AVAILABLE_RESOLUTIONS must declare a width and
a height, each entry of RECORDING_ENCODING_AVAILABLE_PROFILES an fps and a kbps
map, and every profile must define a bitrate for each declared resolution.
The default profile / resolution feed the default encoding. When either is
missing, no custom default encoding can be built: a warning is emitted and
recordings fall back to LiveKit's built-in preset. When both are set, they
must reference keys that actually exist in the maps above.
"""
resolutions = set(cls.RECORDING_ENCODING_AVAILABLE_RESOLUTIONS)
profiles = set(cls.RECORDING_ENCODING_AVAILABLE_PROFILES)
for name, adapter in (
("RECORDING_ENCODING_AVAILABLE_RESOLUTIONS", RESOLUTION_MAP_ADAPTER),
("RECORDING_ENCODING_AVAILABLE_PROFILES", PROFILE_MAP_ADAPTER),
):
try:
adapter.validate_python(getattr(cls, name))
except PydanticValidationError as exc:
raise ValueError(f"{name} is malformed: {exc}") from exc
for (
profile,
profile_config,
) in cls.RECORDING_ENCODING_AVAILABLE_PROFILES.items(): # pylint: disable=no-member
profile_resolutions = set(profile_config["kbps"])
if profile_resolutions != resolutions:
raise ValueError(
f"Profile '{profile}' in RECORDING_ENCODING_AVAILABLE_PROFILES must "
"define a bitrate for exactly the resolutions in "
"RECORDING_ENCODING_AVAILABLE_RESOLUTIONS, mismatch on: "
f"{resolutions ^ profile_resolutions}"
)
# Check that default resolutions and profiles are actually defined
if (
cls.RECORDING_ENCODING_DEFAULT_RESOLUTION
and cls.RECORDING_ENCODING_DEFAULT_RESOLUTION not in resolutions
):
raise ValueError(
"RECORDING_ENCODING_DEFAULT_RESOLUTION "
f"'{cls.RECORDING_ENCODING_DEFAULT_RESOLUTION}' is not a key of "
f"RECORDING_ENCODING_AVAILABLE_RESOLUTIONS ({sorted(resolutions)})."
)
if (
cls.RECORDING_ENCODING_DEFAULT_PROFILE
and cls.RECORDING_ENCODING_DEFAULT_PROFILE not in profiles
):
raise ValueError(
"RECORDING_ENCODING_DEFAULT_PROFILE "
f"'{cls.RECORDING_ENCODING_DEFAULT_PROFILE}' is not a key of "
f"RECORDING_ENCODING_AVAILABLE_PROFILES ({sorted(profiles)})."
)
missing = [
name
for name, value in (
(
"RECORDING_ENCODING_DEFAULT_RESOLUTION",
cls.RECORDING_ENCODING_DEFAULT_RESOLUTION,
),
(
"RECORDING_ENCODING_DEFAULT_PROFILE",
cls.RECORDING_ENCODING_DEFAULT_PROFILE,
),
)
if not value
]
if missing:
warnings.warn(
f"{' and '.join(missing)} not set; recordings will use LiveKit's "
"built-in encoding preset instead of a custom default encoding.",
UserWarning,
stacklevel=2,
)
@classmethod
def post_setup(cls):
"""Post setup configuration.
@@ -1191,6 +1333,8 @@ class Base(Configuration):
"FILE_UPLOAD_TMP_PATH cannot be the same as FILE_UPLOAD_PATH"
)
cls._check_recording_encoding_maps()
if (
cls.SUMMARY_SERVICE_VERSION == 1
and cls.SUMMARY_SERVICE_ENDPOINT is not None
-1
View File
@@ -53,7 +53,6 @@ RUN apk update && apk upgrade \
musl \
musl-utils \
zlib>=1.3.2-r0 \
libexpat>=2.8.4-r0 \
&& apk del curl
USER nginx
@@ -8,28 +8,6 @@ const IGNORED_EXCEPTION_PATTERNS = [
// the close reason is already logged by the SDK.
// See: https://github.com/livekit/client-sdk-js/issues/2062
/^Event captured as exception with keys: isTrusted$/,
// MediaPipe's WASM writes its native logs to stderr, which Emscripten
// routes to console.error, which PostHog's console capture then promotes
// to an $exception — even though nothing was thrown. Two flavors:
//
// 1. "INFO: ..." lines are purely informational. In particular
// "INFO: Created TensorFlow Lite XNNPACK delegate for CPU." is a
// SUCCESS message: TFLite prints it when it lazily initializes CPU
// inference on the first segmented frame. It fires on every effects
// init, on every browser and delegate (the GPU delegate still
// instantiates the CPU/XNNPACK delegate for non-delegated ops), so it
// was our single noisiest "error" while carrying zero signal.
/^INFO: /,
//
// 2. absl-formatted log lines, e.g.
// "E0901 19:21:45.443000 1880752 gl_graph_runner_internal.cc:260]
// StartGraph failed: ..."
// (severity letter E/W/I/F + MMDD + timestamp). These are the stderr
// *copies* of failures that MediaPipe also raises as real JS
// exceptions, which we already capture via reportError / thrown
// errors. Dropping them de-duplicates each incident (previously
// counted 2-3x) without losing the actual error report.
/^[EWIF]\d{4} \d{2}:\d{2}:\d{2}\./,
]
const shouldIgnoreException = (value: unknown): boolean =>
@@ -17,7 +17,7 @@ import {
type ProcessorType,
MEDIAPIPE_PATH_WASM,
} from '.'
import { captureEvent, reportError } from '@/features/analytics/telemetry'
import { captureEvent } from '@/features/analytics/telemetry.ts'
const PROCESSING_WIDTH = 256
const PROCESSING_HEIGHT = 144
@@ -26,40 +26,6 @@ const SEGMENTATION_MASK_CANVAS_ID = 'background-blur-local-segmentation'
const BLUR_CANVAS_ID = 'background-blur-local'
const DEFAULT_BLUR = '10'
const CONCEALING_BLUR = '25'
const FRAME_INTERVAL_MS = 1000 / 30
// After this many consecutive failed frames, stop segmenting and fall back to
// publishing a fully blurred frame: the user keeps a live camera instead of a
// frozen one, without ever exposing the surroundings they chose to conceal.
const MAX_CONSECUTIVE_ERRORS = 5
let webgl2Supported: boolean | undefined
/**
* MediaPipe's ImageSegmenter requires a WebGL2 context on the web even with
* `delegate: 'CPU'` (only inference runs on CPU; the mask post-processing in
* TensorsToSegmentationCalculator is GL-based). Without this check, machines
* with WebGL disabled or blocklisted fail at StartGraph with
* `emscripten_webgl_create_context() returned error 0`.
*
* The result is cached and the probe context is explicitly released so that
* repeated support checks do not count against the browser's limit on live
* WebGL contexts.
*/
const isWebGL2Supported = () => {
if (webgl2Supported === undefined) {
try {
const canvas = document.createElement('canvas')
const gl = canvas.getContext('webgl2')
webgl2Supported = !!gl
gl?.getExtension('WEBGL_lose_context')?.loseContext()
} catch {
webgl2Supported = false
}
}
return webgl2Supported
}
/**
* This implementation of video blurring is made to be run on CPU for browser that are
@@ -76,12 +42,14 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
source?: MediaStreamTrack
sourceSettings?: MediaTrackSettings
videoElement?: HTMLVideoElement
videoElementLoaded?: boolean
// Canvas containing the video processing result, of which we extract as stream.
outputCanvas?: HTMLCanvasElement
outputCanvasCtx?: CanvasRenderingContext2D
imageSegmenter?: ImageSegmenter
imageSegmenterResult?: ImageSegmenterResult
// Canvas used for resizing video source and projecting mask.
segmentationMaskCanvas?: HTMLCanvasElement
@@ -98,13 +66,6 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
type: ProcessorType
virtualBackgroundImage?: HTMLImageElement
private virtualBackgroundImagePath?: string
private destroyed = false
private degraded = false
private consecutiveErrors = 0
private processing?: Promise<void>
private onVideoLoaded?: () => void
constructor(opts: ProcessorConfig) {
this.name = 'blur'
this.options = opts
@@ -112,10 +73,7 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
}
static get isSupported() {
return (
navigator.userAgent.toLowerCase().includes('firefox') &&
isWebGL2Supported()
)
return navigator.userAgent.toLowerCase().includes('firefox')
}
async init(opts: ProcessorOptions<Track.Kind>) {
@@ -123,10 +81,6 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
throw new Error('Element is required for processing')
}
this.destroyed = false
this.degraded = false
this.consecutiveErrors = 0
this.source = opts.track as MediaStreamTrack
this.sourceSettings = this.source!.getSettings()
this.videoElement = opts.element as HTMLVideoElement
@@ -143,56 +97,26 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
this.processedTrack = tracks[0]
this.segmentationMask = new ImageData(PROCESSING_WIDTH, PROCESSING_HEIGHT)
const t0 = performance.now()
await this.initSegmenter()
const segmenterInitMs = Math.round(performance.now() - t0)
this._initWorker()
captureEvent('legacy-background-processor', {
effect_type: this.options.type,
hw_concurrency: navigator.hardwareConcurrency,
video_width: this.videoElement?.videoWidth,
video_height: this.videoElement?.videoHeight,
segmenter_init_ms: segmenterInitMs,
})
captureEvent('firefox-blurring-init', {})
}
_initVirtualBackgroundImage() {
if (this.options.type !== 'virtual' || !this.options.imagePath) {
if (this.options.type !== 'virtual') {
return
}
if (
const needsUpdate =
this.options.imagePath &&
this.virtualBackgroundImage &&
this.virtualBackgroundImagePath === this.options.imagePath
) {
return
this.virtualBackgroundImage.src !== this.options.imagePath
if (this.options.imagePath || needsUpdate) {
this.virtualBackgroundImage = document.createElement('img')
this.virtualBackgroundImage.crossOrigin = 'anonymous'
this.virtualBackgroundImage.src = this.options.imagePath!
}
const image = document.createElement('img')
image.crossOrigin = 'anonymous'
image.src = this.options.imagePath
// Surface load failures once instead of letting drawImage throw on a
// broken image inside the processing loop.
image.decode().catch((error) => {
reportError('effects_processor_failure', error, {
context: 'Failed to load virtual background image',
image_path:
this.options.type === 'virtual' ? this.options.imagePath : undefined,
})
})
this.virtualBackgroundImage = image
this.virtualBackgroundImagePath = this.options.imagePath
}
_isVirtualBackgroundImageReady() {
return (
!!this.virtualBackgroundImage &&
this.virtualBackgroundImage.complete &&
this.virtualBackgroundImage.naturalWidth > 0
)
}
async update(opts: ProcessorConfig): Promise<void> {
@@ -205,26 +129,26 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
name: 'Blurring',
})
this.timerWorker.onmessage = (data) => this.onTimerMessage(data)
const startLoop = () => {
this.onVideoLoaded = undefined
this._syncOutputCanvasSize()
this._scheduleNextFrame()
}
if (this.videoElement!.readyState >= HTMLMediaElement.HAVE_CURRENT_DATA) {
startLoop()
} else {
this.onVideoLoaded = startLoop
this.videoElement!.addEventListener('loadeddata', this.onVideoLoaded, {
once: true,
// When hiding camera then showing it again, the onloadeddata callback is not fired again.
if (this.videoElementLoaded) {
this.timerWorker!.postMessage({
id: SET_TIMEOUT,
timeMs: 1000 / 30,
})
} else {
this.videoElement!.onloadeddata = () => {
this.videoElementLoaded = true
this.timerWorker!.postMessage({
id: SET_TIMEOUT,
timeMs: 1000 / 30,
})
}
}
}
onTimerMessage(response: { data: { id: number } }) {
if (response.data.id === TIMEOUT_TICK) {
this.processing = this.process()
this.process()
}
}
@@ -270,47 +194,30 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
*/
async segment() {
const startTimeMs = performance.now()
return new Promise<void>((resolve, reject) => {
try {
this.imageSegmenter!.segmentForVideo(
this.sourceImageData!,
startTimeMs,
(result: ImageSegmenterResult) => {
try {
// The mask is only valid for the duration of this callback:
// MediaPipe frees the underlying WASM memory as soon as it
// returns, so the data must be copied out synchronously here.
this._applyMaskToAlphaChannel(result)
resolve()
} catch (error) {
reject(error)
}
}
)
} catch (error) {
reject(error)
}
return new Promise<void>((resolve) => {
this.imageSegmenter!.segmentForVideo(
this.sourceImageData!,
startTimeMs,
(result: ImageSegmenterResult) => {
this.imageSegmenterResult = result
resolve()
}
)
})
}
_applyMaskToAlphaChannel(result: ImageSegmenterResult) {
const categoryMask = result.categoryMask
if (!categoryMask) {
return
}
const mask = categoryMask.getAsUint8Array()
const alpha = this.segmentationMask!.data
const length = Math.min(mask.length, alpha.length / 4)
for (let i = 0; i < length; ++i) {
alpha[i * 4 + 3] = 255 - mask[i]
}
}
/**
* Composite the segmentation mask over the output canvas: mask first, then
* the clear body, leaving the background to be filled by the caller.
* TODO: future improvement with WebGL.
*/
_compositeMaskAndBody() {
async blur() {
if (this.options.type !== 'blur') {
throw new Error('Blurring is only supported for blur background')
}
const mask = this.imageSegmenterResult!.categoryMask!.getAsUint8Array()
for (let i = 0; i < mask.length; ++i) {
this.segmentationMask!.data[i * 4 + 3] = 255 - mask[i]
}
this.segmentationMaskCanvasCtx!.putImageData(this.segmentationMask!, 0, 0)
this.outputCanvasCtx!.globalCompositeOperation = 'copy'
@@ -333,16 +240,6 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
this.outputCanvasCtx!.globalCompositeOperation = 'source-in'
this.outputCanvasCtx!.filter = 'none'
this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
}
/**
* TODO: future improvement with WebGL.
*/
async blur() {
if (this.options.type !== 'blur') {
throw new Error('Blurring is only supported for blur background')
}
this._compositeMaskAndBody()
// Draw blurry background.
this.outputCanvasCtx!.globalCompositeOperation = 'destination-over'
@@ -354,150 +251,87 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
* TODO: future improvement with WebGL.
*/
async drawVirtualBackground() {
this._compositeMaskAndBody()
this.outputCanvasCtx!.globalCompositeOperation = 'destination-over'
this.outputCanvasCtx!.filter = 'none'
if (this._isVirtualBackgroundImageReady()) {
// Draw virtual background.
this.outputCanvasCtx!.drawImage(
this.virtualBackgroundImage!,
0,
0,
this.outputCanvas!.width,
this.outputCanvas!.height
)
} else {
// Image not decoded (yet, or failed to load): fill the background with
// a heavy blur instead. Never fall back to the raw video here — the
// user selected this effect to conceal their surroundings, so the
// fallback must keep concealing them.
this.outputCanvasCtx!.filter = `blur(${CONCEALING_BLUR}px)`
this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
const mask = this.imageSegmenterResult!.categoryMask!.getAsUint8Array()
for (let i = 0; i < mask.length; ++i) {
this.segmentationMask!.data[i * 4 + 3] = 255 - mask[i]
}
}
/**
* Draw the whole frame heavily blurred (person included). Used when
* segmentation is broken: the outgoing video keeps flowing instead of
* freezing on a stale frame, while the surroundings the user chose to
* conceal stay concealed. Requires no segmenter, only one filtered draw.
*/
_drawDegradedFrame() {
this.segmentationMaskCanvasCtx!.putImageData(this.segmentationMask!, 0, 0)
this.outputCanvasCtx!.globalCompositeOperation = 'copy'
this.outputCanvasCtx!.filter = `blur(${CONCEALING_BLUR}px)`
this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
this.outputCanvasCtx!.filter = 'blur(8px)'
// Put opacity mask.
this.outputCanvasCtx!.drawImage(
this.segmentationMaskCanvas!,
0,
0,
PROCESSING_WIDTH,
PROCESSING_HEIGHT,
0,
0,
this.videoElement!.videoWidth,
this.videoElement!.videoHeight
)
// Draw clear body.
this.outputCanvasCtx!.globalCompositeOperation = 'source-in'
this.outputCanvasCtx!.filter = 'none'
this.outputCanvasCtx!.drawImage(this.videoElement!, 0, 0)
// Draw virtual background.
this.outputCanvasCtx!.globalCompositeOperation = 'destination-over'
this.outputCanvasCtx!.drawImage(
this.virtualBackgroundImage!,
0,
0,
this.outputCanvas!.width,
this.outputCanvas!.height
)
}
async process() {
if (this.destroyed) {
return
await this.sizeSource()
await this.segment()
if (this.options.type === 'blur') {
await this.blur()
} else {
await this.drawVirtualBackground()
}
try {
this._syncOutputCanvasSize()
// No decoded frame available (e.g. right after a device switch): skip
// this tick rather than processing a 0x0 source.
if (
!this.videoElement ||
this.videoElement.videoWidth === 0 ||
this.videoElement.videoHeight === 0
) {
this._scheduleNextFrame()
return
}
if (this.degraded) {
this._drawDegradedFrame()
this._scheduleNextFrame()
return
}
await this.sizeSource()
await this.segment()
if (this.destroyed) {
return
}
if (this.options.type === 'blur') {
await this.blur()
} else {
await this.drawVirtualBackground()
}
this.consecutiveErrors = 0
} catch (error) {
if (this.destroyed) {
return
}
this.consecutiveErrors += 1
if (this.consecutiveErrors >= MAX_CONSECUTIVE_ERRORS) {
// Degrade to a fully blurred frame: a live camera beats a frozen
// one, and concealment must survive the failure.
this.degraded = true
reportError('effects_processor_failure', error, {
context:
'Background processing failed repeatedly, falling back to fully blurred video',
consecutive_errors: this.consecutiveErrors,
})
this.imageSegmenter?.close()
this.imageSegmenter = undefined
}
}
this._scheduleNextFrame()
}
_scheduleNextFrame() {
if (this.destroyed) {
return
}
this.timerWorker?.postMessage({
this.timerWorker!.postMessage({
id: SET_TIMEOUT,
timeMs: FRAME_INTERVAL_MS,
timeMs: 1000 / 30,
})
}
/**
* Keep the output canvas in sync with the actual decoded video dimensions.
* `MediaStreamTrack.getSettings()` can be incomplete or stale on Firefox,
* so the video element is the source of truth.
*/
_syncOutputCanvasSize() {
const width = this.videoElement?.videoWidth
const height = this.videoElement?.videoHeight
if (!width || !height || !this.outputCanvas) {
return
}
if (
this.outputCanvas.width !== width ||
this.outputCanvas.height !== height
) {
this.outputCanvas.width = width
this.outputCanvas.height = height
}
}
_createMainCanvas() {
const width =
this.sourceSettings?.width || this.videoElement?.videoWidth || 1280
const height =
this.sourceSettings?.height || this.videoElement?.videoHeight || 720
this.outputCanvas = this._createCanvas(BLUR_CANVAS_ID, width, height)
this.outputCanvas = document.querySelector(
'canvas#background-blur-local'
) as HTMLCanvasElement
if (!this.outputCanvas) {
this.outputCanvas = this._createCanvas(
BLUR_CANVAS_ID,
this.sourceSettings!.width!,
this.sourceSettings!.height!
)
}
this.outputCanvasCtx = this.outputCanvas.getContext('2d')!
}
_createMaskCanvas() {
this.segmentationMaskCanvas = this._createCanvas(
SEGMENTATION_MASK_CANVAS_ID,
PROCESSING_WIDTH,
PROCESSING_HEIGHT
)
// getImageData is called on this canvas 30 times per second: opt out of
// GPU backing to avoid a costly readback on every frame.
this.segmentationMaskCanvasCtx = this.segmentationMaskCanvas.getContext(
'2d',
{ willReadFrequently: true }
)!
this.segmentationMaskCanvas = document.querySelector(
`#${SEGMENTATION_MASK_CANVAS_ID}`
) as HTMLCanvasElement
if (!this.segmentationMaskCanvas) {
this.segmentationMaskCanvas = this._createCanvas(
SEGMENTATION_MASK_CANVAS_ID,
PROCESSING_WIDTH,
PROCESSING_HEIGHT
)
}
this.segmentationMaskCanvasCtx =
this.segmentationMaskCanvas.getContext('2d')!
}
_createCanvas(id: string, width: number, height: number) {
@@ -514,39 +348,11 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
}
async destroy() {
this.destroyed = true
this.timerWorker?.postMessage({
id: CLEAR_TIMEOUT,
})
// Let any in-flight frame finish before releasing the resources it uses,
// so segmentForVideo is never called on a closed segmenter.
try {
await this.processing
} catch {
// Failures are already handled inside process().
}
this.processing = undefined
if (this.onVideoLoaded && this.videoElement) {
this.videoElement.removeEventListener('loadeddata', this.onVideoLoaded)
}
this.onVideoLoaded = undefined
this.timerWorker?.terminate()
this.timerWorker = undefined
this.imageSegmenter?.close()
this.imageSegmenter = undefined
this.processedTrack?.stop()
this.processedTrack = undefined
this.outputCanvas = undefined
this.outputCanvasCtx = undefined
this.segmentationMaskCanvas = undefined
this.segmentationMaskCanvasCtx = undefined
this.sourceImageData = undefined
}
}
@@ -6,7 +6,6 @@ import type { Track, TrackProcessor } from 'livekit-client'
import { BackgroundCustomProcessor } from './BackgroundCustomProcessor'
import { UnifiedBackgroundTrackProcessor } from './UnifiedBackgroundTrackProcessor'
import { FaceLandmarksOptions } from './FaceLandmarksProcessor'
import { captureEvent } from '@/features/analytics/telemetry'
export const SELFIE_SEGMENTER_MODEL_PATH =
'/assets/mediapipe/models/selfie_segmenter_landscape.tflite'
@@ -32,29 +31,15 @@ export interface BackgroundProcessorInterface extends TrackProcessor<Track.Kind>
options: ProcessorConfig
}
let unsupportedReported = false
export class BackgroundProcessorFactory {
private static _isSupported?: boolean
static hasModernApiSupport() {
return ProcessorWrapper.hasModernApiSupport
}
static isSupported() {
if (this._isSupported === undefined) {
this._isSupported =
supportsBackgroundProcessors() || BackgroundCustomProcessor.isSupported
}
if (!this._isSupported && !unsupportedReported) {
unsupportedReported = true
captureEvent('background-processor-unsupported', {
path: 'isSupported',
})
}
return this._isSupported
return (
supportsBackgroundProcessors() || BackgroundCustomProcessor.isSupported
)
}
static getProcessor(
@@ -63,8 +48,6 @@ export class BackgroundProcessorFactory {
const isBlur = config.type === ProcessorType.BLUR
const isVirtual = config.type === ProcessorType.VIRTUAL
return new BackgroundCustomProcessor(config)
if (!isBlur && !isVirtual) return undefined
if (supportsBackgroundProcessors()) {
@@ -75,12 +58,6 @@ export class BackgroundProcessorFactory {
return new BackgroundCustomProcessor(config)
}
if (!unsupportedReported) {
captureEvent('background-processor-unsupported', {
path: 'getProcessor',
})
}
return undefined
}
@@ -25,11 +25,7 @@ import {
} from '@/features/files/api/listFiles.ts'
import { useCreateFile } from '@/features/files/api/createFile.ts'
import { FileTrigger } from 'react-aria-components'
import {
RiDeleteBinLine,
RiImageAddFill,
RiProhibitedLine,
} from '@remixicon/react'
import { RiDeleteBinLine, RiImageAddFill } from '@remixicon/react'
import { useDeleteFile } from '@/features/files/api/deleteFile.ts'
import { useUser } from '@/features/auth/api/useUser'
import { ApiFileItem } from '@/features/files/api/types.ts'
@@ -201,23 +197,10 @@ export const EffectsConfiguration = ({
*
* We arrive in this condition when we enter the room with the camera already off.
*/
try {
const newProcessorTmp =
BackgroundProcessorFactory.getProcessor(config)!
await toggle(true, {
processor: newProcessorTmp,
})
} catch (error) {
reportError('effects_processor_failure', error, {
context: 'Error applying effect while enabling camera:',
})
saveProcessorConfig(undefined)
try {
await toggle(true)
} catch {
// Camera errors are handled by the toggle's own error path.
}
}
const newProcessorTmp = BackgroundProcessorFactory.getProcessor(config)!
await toggle(true, {
processor: newProcessorTmp,
})
setTimeout(() => setProcessorPending(false))
return
}
@@ -259,14 +242,6 @@ export const EffectsConfiguration = ({
reportError('effects_processor_failure', error, {
context: 'Error applying effect:',
})
try {
if (videoTrack.getProcessor()) {
await videoTrack.stopProcessor()
}
} catch {
// Best effort: the processor may already be broken.
}
saveProcessorConfig(undefined)
} finally {
// Without setTimeout the DOM is not refreshing when updating the options.
setTimeout(() => setProcessorPending(false))
@@ -275,24 +250,6 @@ export const EffectsConfiguration = ({
[enabled, selectedId, toggle, updateEffectStatusMessage, videoTrack]
)
const clearEffect = useCallback(async () => {
if (selectedId === 'none') return
setProcessorPending(true)
try {
if (videoTrack?.getProcessor()) {
await videoTrack.stopProcessor()
}
saveProcessorConfig(undefined)
announceEffectStatusMessage(t('blur.status.none'))
} catch (error) {
reportError('effects_processor_failure', error, {
context: 'Error clearing effect:',
})
} finally {
setTimeout(() => setProcessorPending(false))
}
}, [announceEffectStatusMessage, selectedId, t, videoTrack])
const { data: appConfig } = useConfig()
const { isLoggedIn } = useUser()
const canUploadBackground =
@@ -690,17 +647,6 @@ export const EffectsConfiguration = ({
gap: '1.25rem',
})}
>
<ToggleButton
variant="bigSquare"
aria-label={t('clear')}
tooltip={t('clear')}
isDisabled={processorOptions.isDisabled}
onChange={clearEffect}
isSelected={selectedId === 'none'}
data-attr="toggle-effect-none"
>
<RiProhibitedLine />
</ToggleButton>
{processorOptions.blurBased.map(({ Icon, ...option }) => (
<ToggleButton
key={option.id}
@@ -22,12 +22,10 @@ import { VOICE_AUDIO_CONSTRAINTS } from '../utils/constants'
import {
saveAudioInputDeviceId,
saveAudioInputEnabled,
saveProcessorConfig,
saveVideoInputDeviceId,
saveVideoInputEnabled,
userChoicesStore,
} from '@/stores/userChoices'
import { reportError } from '@/features/analytics/telemetry'
import { useSyncTrackDeviceId } from './useSyncTrackDeviceId'
// Module-level: effect dependencies, must be referentially stable.
@@ -223,32 +221,15 @@ export function useJoinTracks(): {
[audioDeviceId]
)
const createVideo = useCallback(async () => {
const processor =
BackgroundProcessorFactory.fromProcessorConfig(processorConfig)
if (!processor) {
return createLocalVideoTrack({ deviceId: videoDeviceId })
}
try {
return await createLocalVideoTrack({
const createVideo = useCallback(
() =>
createLocalVideoTrack({
deviceId: videoDeviceId,
processor,
})
} catch (error) {
// A camera problem (permission, device missing/busy) is not the
// effect's fault: let the normal media error handling deal with it
// without touching the user's saved effect.
const e = getMediaDeviceFailure(error as Error)
if (e !== MediaDeviceFailure.Other && !!e) {
throw error
}
reportError('effects_processor_failure', error, {
context: 'Restoring saved effect failed, retrying without it',
})
saveProcessorConfig(undefined)
return createLocalVideoTrack({ deviceId: videoDeviceId })
}
}, [videoDeviceId, processorConfig])
processor:
BackgroundProcessorFactory.fromProcessorConfig(processorConfig),
}),
[videoDeviceId, processorConfig]
)
const audioTrack = useLocalTrack({
ready: audioReady,
@@ -18,7 +18,6 @@ import {
saveVideoPublishResolution,
saveVideoSubscribeQuality,
userChoicesStore,
VIDEO_RESOLUTIONS,
VideoResolution,
} from '@/stores/userChoices'
import { RowWrapper } from './layout/RowWrapper'
@@ -71,7 +70,7 @@ export const VideoTab = ({ id }: VideoTabProps) => {
isDisabled: true,
}
const handleVideoResolutionChange = async (key: VideoResolution) => {
const handleVideoResolutionChange = async (key: 'h720' | 'h360' | 'h180') => {
saveVideoPublishResolution(key)
const videoTrack = localParticipant.getTrackPublication(
Track.Source.Camera
@@ -125,13 +124,20 @@ export const VideoTab = ({ id }: VideoTabProps) => {
}, [videoDeviceId, videoElement])
const resolutionItems = useMemo(() => {
const labels: Record<VideoResolution, string> = {
h1080: `${t('resolution.publish.items.veryHigh')} (1080p)`,
h720: `${t('resolution.publish.items.high')} (720p)`,
h360: `${t('resolution.publish.items.medium')} (360p)`,
h180: `${t('resolution.publish.items.low')} (180p)`,
}
return VIDEO_RESOLUTIONS.map((value) => ({ value, label: labels[value] }))
return [
{
value: 'h720',
label: `${t('resolution.publish.items.high')} (720p)`,
},
{
value: 'h360',
label: `${t('resolution.publish.items.medium')} (360p)`,
},
{
value: 'h180',
label: `${t('resolution.publish.items.low')} (180p)`,
},
]
}, [t])
const videoQualityItems = useMemo(() => {
@@ -56,7 +56,6 @@
"publish": {
"label": "Wähle die maximale Auflösung beim Senden",
"items": {
"veryHigh": "Sehr hohe Auflösung",
"high": "Hohe Auflösung",
"medium": "Mittlere Auflösung",
"low": "Niedrige Auflösung"
@@ -56,7 +56,6 @@
"publish": {
"label": "Select your sending resolution (max.)",
"items": {
"veryHigh": "Very high definition",
"high": "High definition",
"medium": "Standard definition",
"low": "Low definition"
@@ -56,7 +56,6 @@
"publish": {
"label": "Selecciona tu resolución de envío (máx.)",
"items": {
"veryHigh": "Muy alta definición",
"high": "Alta definición",
"medium": "Definición estándar",
"low": "Baja definición"
@@ -56,7 +56,6 @@
"publish": {
"label": "Sélectionner votre résolution d'envoi (max.)",
"items": {
"veryHigh": "Très haute définition",
"high": "Haute définition",
"medium": "Définition standard",
"low": "Basse définition"
@@ -56,7 +56,6 @@
"publish": {
"label": "Selecteer uw verzendresolutie (max.)",
"items": {
"veryHigh": "Zeer hoge definitie",
"high": "Hoge definitie",
"medium": "Standaarddefinitie",
"low": "Lage definitie"
+2 -11
View File
@@ -10,12 +10,7 @@ import {
} from '@livekit/components-core'
import { VideoQuality } from 'livekit-client'
export const VIDEO_RESOLUTIONS = ['h1080', 'h720', 'h360', 'h180'] as const
export type VideoResolution = (typeof VIDEO_RESOLUTIONS)[number]
const isVideoResolution = (value: unknown): value is VideoResolution =>
VIDEO_RESOLUTIONS.includes(value as VideoResolution)
export type VideoResolution = 'h720' | 'h360' | 'h180'
export type LocalUserChoices = Omit<LocalUserChoicesLK, 'username'> & {
processorConfig?: ProcessorConfig
@@ -26,17 +21,13 @@ export type LocalUserChoices = Omit<LocalUserChoicesLK, 'username'> & {
}
function getUserChoicesState(): LocalUserChoices {
const stored: LocalUserChoices = {
return {
noiseReductionEnabled: false,
audioOutputDeviceId: 'default', // Use 'default' to match LiveKit's standard device selection behavior
videoPublishResolution: 'h720',
videoSubscribeQuality: VideoQuality.HIGH,
...loadUserChoices(),
}
if (!isVideoResolution(stored.videoPublishResolution)) {
stored.videoPublishResolution = 'h720'
}
return stored
}
export const userChoicesStore = proxy<LocalUserChoices>(getUserChoicesState())