🔥(backend) remove the S3 storage-event webhook for recordings

Recordings used to be finalized by an inbound notifications sent by
S3. This tied the recording lifecycle to bucket notifications, adding
complexity and dependency to limited S3 services.

The LiveKit egress_ended webhook, added as a fallback in aee1847, does
the same job without any object-storage dependency. Make it the only
mechanism: RecordingEventsService.handle_complete is now called on
EGRESS_COMPLETE / EGRESS_LIMIT_REACHED unconditionally, instead of only
when RECORDING_STORAGE_EVENT_ENABLE is False. Remove the storage-event
path entirely.
This commit is contained in:
leo
2026-08-26 15:19:22 +02:00
committed by aleb_the_flash
parent f1d3799434
commit c7e3168ba3
21 changed files with 116 additions and 1272 deletions
+5
View File
@@ -8,6 +8,11 @@ and this project adheres to
## [Unreleased]
### Changed
- 🔥(backend) remove the S3 storage-event webhook for recordings
- ♻️(backend) always finalize recordings using the LiveKit egress_ended webhook
## [1.29.0] - 2026-08-25
### Added
+17
View File
@@ -16,6 +16,23 @@ the following command inside your docker container:
## [Unreleased]
### Removing S3 storage-event webhooks for recordings
Recordings were previously confirmed as saved by an S3 storage-event webhook posting to `/api/v1.0/recordings/storage-hook/`. That endpoint has been removed: recordings are now always finalized from LiveKit's own `egress_ended` webhook, which has been the default path since v1.22.0.
**Required for every deployment:** LiveKit must be able to deliver webhooks to the backend at `/api/v1.0/rooms/webhooks-livekit/`. This is now the only way a recording reaches a saved state; if `egress_ended` is never delivered, recordings stay in the `active` state.
For hosters who had configured storage-event webhooks:
- Recordings reach the same final state, but they are now finalized when LiveKit reports the egress as ended rather than when the storage backend reports the upload.
- Remove the event notification from your bucket configuration: it now targets a non-existent endpoint and will fail on every delivery.
For hosters who had **not** configured storage-event webhooks:
- Nothing changes. Recordings have been finalized from the `egress_ended` webhook since v1.22.0.
In both cases, the following settings are no longer used and can be removed from your env: `RECORDING_EVENT_PARSER_CLASS`, `RECORDING_ENABLE_STORAGE_EVENT_AUTH`, `RECORDING_STORAGE_EVENT_ENABLE`, `RECORDING_STORAGE_EVENT_TOKEN`.
On completion of the egress, a recording moves to `notification_succeeded`, or to `saved` if notifying external services failed.
## v1.23.0
As part of the 1.23.0 release, the legacy `api/v1` implementation has been removed from the _experimental_ Summary service and Meet has been migrated to the new `api/v2`.
-16
View File
@@ -46,21 +46,6 @@ services:
/usr/bin/mc mb meet/meet-media-storage && \
exit 0;"
createwebhook:
image: minio/mc
depends_on:
minio:
condition: service_healthy
restart: true
entrypoint: >
sh -c "
/usr/bin/mc alias set meet http://minio:9000 meet password &&
/usr/bin/mc admin config set meet notify_webhook:meet-webhook endpoint='http://app-dev:8000/api/v1.0/recordings/storage-hook/' auth_token='Bearer password' &&
/usr/bin/mc admin service restart meet --wait --json &&
sleep 15 &&
/usr/bin/mc event add meet/meet-media-storage arn:minio:sqs::meet-webhook:webhook --event put --prefix "recordings" &&
exit 0;"
app-dev:
build:
context: .
@@ -86,7 +71,6 @@ services:
- mailcatcher
- redis
- createbuckets
- createwebhook
extra_hosts:
- "127.0.0.1.nip.io:host-gateway"
networks:
+2 -22
View File
@@ -23,14 +23,11 @@ It uses LiveKit Egress to record room sessions. For reference, see the [LiveKit
To use the room recording feature, the following components are required:
- A running [LiveKit Egress](https://github.com/livekit/egress) server capable of handling room composite recordings.
- A S3-compatible object storage that supports webhook events to notify the backend when recordings are uploaded.
- A S3-compatible object storage where the egress uploads the recorded files.
- An email service to notify room owners when a recording is available for download.
- Webhook events configured between LiveKit Server and the backend.
> [!CAUTION]
> Minio supports lifecycle events; other providers may not work out of the box. There is currently a dependency on Minio, which is planned to be refactored in the future.
> [!NOTE]
> Celery isnt in use for these async tasks yet. Its something wed like to add, but its not planned at this stage.
@@ -75,7 +72,7 @@ sequenceDiagram
LiveKit->>Egress: Stop recording
Egress->>Storage: Upload recorded file
Storage->>Backend: Storage event notification
LiveKit->>Backend: POST /api/v1.0/rooms/webhooks-livekit/ (egress_ended)
Backend->>Backend: Update Recording status to SAVED
Backend->>Email: Send notification to room owner
@@ -94,10 +91,6 @@ sequenceDiagram
| **RECORDING_ENABLE** | Boolean | `False` | Enable or disable the room recording feature. |
| **RECORDING_OUTPUT_FOLDER** | String | `"recordings"` | Folder/prefix where recordings are stored in the object storage. |
| **RECORDING_WORKER_CLASSES** | Dict | `{ "screen_recording": "core.recording.worker.services.VideoCompositeEgressService", "transcript": "core.recording.worker.services.AudioCompositeEgressService" }` | Maps recording types to their worker service classes. |
| **RECORDING_EVENT_PARSER_CLASS** | String | `"core.recording.event.parsers.MinioParser"` | Class responsible for parsing storage events and updating the backend. |
| **RECORDING_ENABLE_STORAGE_EVENT_AUTH** | Boolean | `True` | Enable authentication for storage event webhook requests. |
| **RECORDING_STORAGE_EVENT_ENABLE** | Boolean | `False` | Enable handling of storage events (must configure webhook in storage). If `False`, fallback to LiveKit egress complete webhook. |
| **RECORDING_STORAGE_EVENT_TOKEN** | Secret/File | `None` | Token used to authenticate storage webhook requests, if `RECORDING_ENABLE_STORAGE_EVENT_AUTH` is enabled. |
| **RECORDING_EXPIRATION_DAYS** | Integer | `None` | Number of days before recordings expire. Should match bucket lifecycle policy. Set to `None` for no expiration. |
| **RECORDING_MAX_DURATION** | Integer | `None` | Maximum duration of a recording in milliseconds. Must be synced with the LiveKit Egress configuration. Set to None for unlimited duration. When the maximum duration is reached, the recording is automatically stopped and saved, and the user is prompted in the frontend with an alert message. |
| **RECORDING_ENCODING_ENABLED** | Boolean | `False` | When `False`, LiveKit Egress uses its built-in `H264_720P_30` preset. When `True`, the `RECORDING_ENCODING_*` values below are sent to LiveKit as advanced `EncodingOptions`. See [Tuning recording encoding](#tuning-recording-encoding). |
@@ -109,19 +102,6 @@ sequenceDiagram
| **RECORDING_ENCODING_KEY_FRAME_INTERVAL_S** | Float | `4.0` | Keyframe interval in seconds. Drives seek granularity in the recorded MP4 (a player can only seek to keyframe boundaries). Larger values give the encoder slightly more bits for non-keyframe content at a fixed bitrate. `4.0` is a standard VOD value. Only applied when `RECORDING_ENCODING_ENABLED` is `True`. |
### Manual Storage Webhook
Storage events must be configured manually; the Kubernetes chart does not do this automatically.
1. Configure your S3 bucket to send file creation events to the backend webhook.
2. Enable events and token in settings:
```python
RECORDING_STORAGE_EVENT_ENABLE = True
RECORDING_ENABLE_STORAGE_EVENT_AUTH = True
RECORDING_STORAGE_EVENT_TOKEN = <token>
```
> [!NOTE]
> Questions? Open an issue on [GitHub](https://github.com/suitenumerique/meet/issues/new?assignees=&labels=bug&template=Bug_report.md) or join our [Matrix community](https://matrix.to/#/#meet-official:matrix.org).
-4
View File
@@ -406,10 +406,6 @@ These are the environmental options available on meet backend.
| RECORDING_ENABLE | Record meeting option | false |
| RECORDING_OUTPUT_FOLDER | Folder to store meetings | recordings |
| RECORDING_WORKER_CLASSES | Worker classes for recording | {"screen_recording": "core.recording.worker.services.VideoCompositeEgressService","transcript": "core.recording.worker.services.AudioCompositeEgressService"} |
| RECORDING_EVENT_PARSER_CLASS | Storage event engine for recording | core.recording.event.parsers.MinioParser |
| RECORDING_ENABLE_STORAGE_EVENT_AUTH | Enable storage event authorization | true |
| RECORDING_STORAGE_EVENT_ENABLE | Enable recording storage events. If false, fallback to egress webhook. | false |
| RECORDING_STORAGE_EVENT_TOKEN | Recording storage event token | |
| RECORDING_EXPIRATION_DAYS | Recording expiration in days | |
| RECORDING_MAX_DURATION | Maximum recording duration in milliseconds. Must match LiveKit Egress configuration exactly. | |
| SCREEN_RECORDING_BASE_URL | Screen recording base URL | |
-2
View File
@@ -63,8 +63,6 @@ ALLOW_UNREGISTERED_ROOMS=False
# Recording
RECORDING_ENABLE=True
RECORDING_STORAGE_EVENT_ENABLE=False
RECORDING_STORAGE_EVENT_TOKEN=password
SUMMARY_SERVICE_ENDPOINT=http://app-summary-dev:8000/api/v2/async-jobs/transcribe/
SUMMARY_SERVICE_API_TOKEN=password
SUMMARY_SERVICE_WEBHOOK_API_TOKEN=webhook-password
-1
View File
@@ -11,7 +11,6 @@ class FeatureFlag:
FLAGS = {
"recording": "RECORDING_ENABLE",
"storage_event": "RECORDING_STORAGE_EVENT_ENABLE",
"subtitle": "ROOM_SUBTITLE_ENABLED",
"file_upload": "FILE_UPLOAD_ENABLED",
"addons": "ADDONS_ENABLED",
+1 -65
View File
@@ -45,25 +45,11 @@ from core.api import throttling
from core.api.filters import ListFileFilter
from core.enums import MEDIA_STORAGE_URL_PATTERN
from core.recording.enums import FileExtension
from core.recording.event.authentication import (
RecordingProcessWebhookAuthentication,
StorageEventAuthentication,
)
from core.recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
from core.recording.event.parsers import get_parser
from core.recording.event.authentication import RecordingProcessWebhookAuthentication
from core.recording.services.metadata_collector import (
MetadataCollectorException,
MetadataCollectorService,
)
from core.recording.services.recording_events import (
RecordingEventsService,
RecordingNotSavableError,
)
from core.recording.worker.exceptions import (
RecordingStartError,
RecordingStopError,
@@ -1034,56 +1020,6 @@ class RecordingViewSet(
.filter(Q(accesses__user=user) | Q(accesses__team__in=user.get_teams()))
)
@decorators.action(
detail=False,
methods=["post"],
url_path="storage-hook",
authentication_classes=[StorageEventAuthentication],
)
@FeatureFlag.require("storage_event")
def on_storage_event_received(self, request, pk=None): # pylint: disable=unused-argument
"""Handle incoming storage hook events for recordings."""
parser = get_parser()
try:
recording_id = parser.get_recording_id(request.data)
except ParsingEventDataError as e:
raise drf_exceptions.PermissionDenied("Invalid request data.") from e
except InvalidBucketError as e:
raise drf_exceptions.PermissionDenied("Invalid bucket specified.") from e
except InvalidFilepathError:
return drf_response.Response(
{"message": "Notification ignored."},
)
except InvalidFileTypeError:
return drf_response.Response(
{"message": "Notification ignored."},
)
try:
recording = models.Recording.objects.get(id=recording_id)
except models.Recording.DoesNotExist as e:
raise drf_exceptions.NotFound("No recording found for this event.") from e
# Save recording
recording_events_service = RecordingEventsService()
try:
recording_events_service.handle_complete(recording)
except RecordingNotSavableError:
raise drf_exceptions.PermissionDenied(
f"Recording with ID {recording_id} cannot be saved because it is either,"
" in an error state or has already been saved."
) from None
return drf_response.Response(
{"message": "Event processed."},
)
@decorators.action(
detail=False,
methods=["post"],
@@ -1,4 +1,4 @@
"""Authentication class for storage event token validation."""
"""Authentication classes for server-to-server webhook token validation."""
import logging
import secrets
@@ -12,9 +12,9 @@ logger = logging.getLogger(__name__)
class MachineUser:
"""Represent a non-interactive system user for automated storage operations."""
"""Represent a non-interactive system user for automated operations."""
def __init__(self, username: str = "storage_event_user") -> None:
def __init__(self, username: str = "machine_user") -> None:
self.pk = None
self.username = username
self.is_active = True
@@ -41,24 +41,17 @@ class HeaderBasedAuthentication(BaseAuthentication):
TOKEN_TYPE = "Bearer" # noqa S105
REALM = ""
IS_ENFORCED_SETTINGS_KEY = None
EXPECTED_TOKEN_SETTINGS_KEY = None
def authenticate(self, request):
"""Validate the Bearer token from the Authorization header."""
if self.IS_ENFORCED_SETTINGS_KEY is not None:
if not getattr(settings, self.IS_ENFORCED_SETTINGS_KEY):
return MachineUser(), None
if (
self.EXPECTED_TOKEN_SETTINGS_KEY is None
or (required_token := getattr(settings, self.EXPECTED_TOKEN_SETTINGS_KEY))
is None
):
raise AuthenticationFailed(
"Authentication is enabled but token is not configured."
)
raise AuthenticationFailed("Authentication token is not configured.")
auth_header = request.headers.get(self.AUTH_HEADER)
if not auth_header:
@@ -88,18 +81,6 @@ class HeaderBasedAuthentication(BaseAuthentication):
return f"{self.TOKEN_TYPE} realm='{self.REALM}'"
class StorageEventAuthentication(HeaderBasedAuthentication):
"""Authenticate requests using a Bearer token for storage event integration.
This class validates Bearer tokens for storage events that don't map to database users.
It's designed for S3-compatible storage integrations and similar use cases.
Events are submitted when a webhook is configured on some bucket's events.
"""
REALM = "Storage event API"
IS_ENFORCED_SETTINGS_KEY = "RECORDING_ENABLE_STORAGE_EVENT_AUTH"
EXPECTED_TOKEN_SETTINGS_KEY = "RECORDING_STORAGE_EVENT_TOKEN" # noqa S105
class RecordingProcessWebhookAuthentication(HeaderBasedAuthentication):
"""
Custom authentication class for recording process webhook requests.
@@ -1,17 +0,0 @@
"""Storage parsers specific exceptions."""
class ParsingEventDataError(Exception):
"""Raised when the request data is malformed, incomplete, or missing."""
class InvalidBucketError(Exception):
"""Raised when the bucket name in the request does not match the expected one."""
class InvalidFileTypeError(Exception):
"""Raised when the file type in the request is not supported."""
class InvalidFilepathError(Exception):
"""Raised when the filepath in the request is invalid."""
-178
View File
@@ -1,178 +0,0 @@
"""Meet storage event parser classes."""
import logging
import mimetypes
import re
from dataclasses import dataclass
from functools import lru_cache
from typing import Any, Dict, Optional, Protocol
from urllib.parse import quote
from django.conf import settings
from django.utils.module_loading import import_string
from core.enums import FILE_EXT_REGEX, UUID_REGEX
from .exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
# Additional MIME type mapping
mimetypes.add_type("audio/ogg", ".ogg")
logger = logging.getLogger(__name__)
@dataclass
class StorageEvent:
"""Represents a storage event with relevant metadata.
Attributes:
filepath: Identifier for the affected recording
filetype: Type of storage event
bucket_name: When the event occurred
metadata: Additional event data
"""
filepath: str
filetype: str
bucket_name: str
metadata: Optional[Dict[str, Any]]
def __post_init__(self):
if self.filepath is None:
raise TypeError("filepath cannot be None")
if self.filetype is None:
raise TypeError("filetype cannot be None")
if self.bucket_name is None:
raise TypeError("bucket_name cannot be None")
class EventParser(Protocol):
"""Interface for parsing storage events."""
def __init__(self, bucket_name, allowed_filetypes=None):
"""Initialize parser with bucket name and optional allowed filetypes."""
def parse(self, data: Dict) -> StorageEvent:
"""Extract storage event data from raw dictionary input."""
def validate(self, data: StorageEvent) -> str:
"""Verify storage event data meets all requirements."""
def get_recording_id(self, data: Dict) -> str:
"""Extract recording ID from event dictionary."""
@lru_cache(maxsize=1)
def get_parser() -> EventParser:
"""Return cached instance of configured event parser.
Uses function memoization instead of a factory class since the only
varying parameter is the parser class from settings. A factory class
would add unnecessary complexity when a cached function provides the
same singleton behavior with simpler code.
"""
event_parser_cls = import_string(settings.RECORDING_EVENT_PARSER_CLASS)
return event_parser_cls(bucket_name=settings.AWS_STORAGE_BUCKET_NAME)
class BaseS3Parser:
"""Base class for handling parsing and validation of S3-compatible storage events."""
def __init__(self, bucket_name: str, allowed_filetypes=None):
"""Initialize parser with target bucket name and accepted filetypes."""
if not bucket_name:
raise ValueError("Bucket name cannot be None or empty")
self._bucket_name = bucket_name
self._allowed_filetypes = allowed_filetypes or {"audio/ogg", "video/mp4"}
# pylint: disable=line-too-long
self._filepath_regex = re.compile(
rf"(?P<url_encoded_folder_path>(?:[^%]+%2F)+)?{settings.RECORDING_OUTPUT_FOLDER}%2F(?P<recording_id>{UUID_REGEX})\.(?P<extension>{FILE_EXT_REGEX})"
)
def validate(self, event_data: StorageEvent) -> str:
"""Verify StorageEvent matches bucket, filetype and filepath requirements."""
if event_data.bucket_name != self._bucket_name:
raise InvalidBucketError(
f"Invalid bucket: expected {self._bucket_name}, got {event_data.bucket_name}"
)
if event_data.filetype not in self._allowed_filetypes:
raise InvalidFileTypeError(
f"Invalid file type, expected {self._allowed_filetypes},"
f"got '{event_data.filetype}'"
)
match = self._filepath_regex.match(event_data.filepath)
if not match:
raise InvalidFilepathError(
f"Invalid filepath structure: {event_data.filepath}"
)
recording_id = match.group("recording_id")
return recording_id
def get_recording_id(self, data):
"""Extract recording ID from S3 event through parsing and validation."""
event_data = self.parse(data)
return self.validate(event_data)
def parse(self, data: Dict) -> StorageEvent:
"""To be implemented by subclasses."""
raise NotImplementedError("Subclasses must implement parse()")
class MinioParser(BaseS3Parser):
"""Minio specific event parsing."""
def parse(self, data: Dict) -> StorageEvent:
if not data:
raise ParsingEventDataError("Received empty data.")
try:
record = data["Records"][0]
s3 = record["s3"]
return StorageEvent(
filepath=s3["object"]["key"],
filetype=s3["object"]["contentType"], # Minio-specific field
bucket_name=s3["bucket"]["name"],
metadata=None,
)
except (KeyError, IndexError) as e:
raise ParsingEventDataError(f"Malformed Minio event: {e}") from e
except TypeError as e:
raise ParsingEventDataError(f"Missing essential data fields: {e}") from e
class S3Parser(BaseS3Parser):
"""AWS S3 specific event parsing."""
def parse(self, data: Dict) -> StorageEvent:
if not data:
raise ParsingEventDataError("Received empty data.")
try:
# AWS S3 structure can slightly differ from Minio implementation
record = data["Records"][0]
s3 = record["s3"]
filepath = s3["object"]["key"]
if not filepath:
raise ParsingEventDataError("Missing object key name")
filetype, _ = mimetypes.guess_type(filepath)
# Normalize raw S3-compatible object keys without re-encoding
# already encoded AWS S3 notification keys.
filepath = quote(filepath, safe="%+")
return StorageEvent(
filepath=filepath,
filetype=filetype,
bucket_name=s3["bucket"]["name"],
metadata=None,
)
except (KeyError, IndexError) as e:
raise ParsingEventDataError(f"Malformed S3 event: {e}") from e
+2 -4
View File
@@ -220,10 +220,8 @@ class LiveKitEventsService:
f"Failed to process limit reached event for recording {recording}"
) from e
# Fallback for completion when no MinIO/S3 webhooks are configured
if (
not settings.RECORDING_STORAGE_EVENT_ENABLE
) and data.egress_info.status in [
# Finalize the recording, the egress has uploaded the file to the storage
if data.egress_info.status in [
api.EgressStatus.EGRESS_COMPLETE,
api.EgressStatus.EGRESS_LIMIT_REACHED,
]:
@@ -11,135 +11,98 @@ from rest_framework.exceptions import AuthenticationFailed
from core.recording.event.authentication import (
MachineUser,
StorageEventAuthentication,
RecordingProcessWebhookAuthentication,
)
def test_successful_authentication(settings):
"""Test successful authentication with valid token."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer valid-test-token"}
user, token = StorageEventAuthentication().authenticate(request)
user, token = RecordingProcessWebhookAuthentication().authenticate(request)
assert token == "valid-test-token"
assert isinstance(user, MachineUser)
def test_disabled_authentication_with_header(settings):
"""Authentication should pass when no auth is configured, and header is present."""
settings.RECORDING_STORAGE_EVENT_TOKEN = None
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer some-token"}
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_disabled_authentication_without_header(settings):
"""Authentication should pass when no auth is configured, and no header is present."""
settings.RECORDING_STORAGE_EVENT_TOKEN = None
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_authentication_when_disabled(settings):
"""Authentication should pass when disabled, regardless of token configuration."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "some-token"
settings.RECORDING_ENABLE_STORAGE_EVENT_AUTH = False
request = RequestFactory().get("/")
user, token = StorageEventAuthentication().authenticate(request)
assert token is None
assert isinstance(user, MachineUser)
def test_authentication_fails_when_token_not_configured(settings):
"""Authentication should fail when authentication is enabled but no token is configured."""
"""Authentication should fail when no token is configured."""
# By default RECORDING_ENABLE_STORAGE_EVENT_AUTH should be True
settings.RECORDING_STORAGE_EVENT_TOKEN = None
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = None
request = RequestFactory().get("/")
with pytest.raises(
AuthenticationFailed,
match="Authentication is enabled but token is not configured",
match="Authentication token is not configured",
):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_missing_auth_header(settings):
"""Test failure when Authorization header is missing."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {}
with pytest.raises(AuthenticationFailed, match="Authorization header is required"):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_invalid_auth_header_format(settings):
"""Test failure when Authorization header has invalid format."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "InvalidFormat"}
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_invalid_token_type(settings):
"""Test failure when token type is not Bearer."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Basic some-token"}
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_invalid_token(settings):
"""Test failure when token is invalid."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer wrong-token"}
with pytest.raises(AuthenticationFailed, match="Invalid token"):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_malformed_auth_header(settings):
"""Test failure when Authorization header is malformed."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "valid-test-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer"} # Missing token part
with pytest.raises(AuthenticationFailed, match="Invalid authorization header"):
StorageEventAuthentication().authenticate(request)
RecordingProcessWebhookAuthentication().authenticate(request)
def test_authenticate_header():
"""Test the WWW-Authenticate header value."""
request = RequestFactory().get("/")
header = StorageEventAuthentication().authenticate_header(request)
assert header == "Bearer realm='Storage event API'"
header = RecordingProcessWebhookAuthentication().authenticate_header(request)
assert header == "Bearer realm='External process webhook API'"
def test_multiple_spaces_in_auth_header(settings):
"""Test success when Authorization header contains multiple spaces."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "valid-test-token"
"""Extra spaces between the scheme and the token should be tolerated."""
settings.SUMMARY_SERVICE_WEBHOOK_API_TOKEN = "extra-spaces-token"
request = RequestFactory().get("/")
request.headers = {"Authorization": "Bearer extra-spaces-token"}
header = StorageEventAuthentication().authenticate_header(request)
assert header == "Bearer realm='Storage event API'"
user, token = RecordingProcessWebhookAuthentication().authenticate(request)
assert token == "extra-spaces-token"
assert isinstance(user, MachineUser)
@@ -1,512 +0,0 @@
"""
Test event parsers.
"""
# pylint: disable=protected-access,redefined-outer-name,unused-argument
from unittest import mock
from django.conf import settings
import pytest
from core.recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
from core.recording.event.parsers import (
MinioParser,
S3Parser,
StorageEvent,
get_parser,
)
# MinioParser
@pytest.fixture
def valid_minio_event():
"""Mock a valid Minio event."""
return {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"contentType": "audio/ogg",
},
}
}
]
}
@pytest.fixture
def minio_parser():
"""Mock a Minio parser."""
return MinioParser(bucket_name="test-bucket")
def test_minio_parse_valid_event(minio_parser, valid_minio_event):
"""Test parsing a valid Minio event."""
event = minio_parser.parse(valid_minio_event)
assert isinstance(event, StorageEvent)
assert event.filepath == "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg"
assert event.filetype == "audio/ogg"
assert event.bucket_name == "test-bucket"
assert event.metadata is None
def test_minio_parse_with_video_type(minio_parser):
"""Test parsing event with video file type."""
video_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "46d1a121-2426-484d-8fb3-09b5d886f7a8.mp4",
"contentType": "video/mp4",
},
}
}
]
}
event = minio_parser.parse(video_event)
assert event.filetype == "video/mp4"
assert event.filepath.endswith(".mp4")
def test_minio_parse_empty_data(minio_parser):
"""Test parsing empty event data raises error."""
with pytest.raises(ParsingEventDataError, match="Received empty data."):
minio_parser.parse({})
def test_minio_parse_missing_keys(minio_parser):
"""Test parsing event with missing key."""
invalid_minio_event = {
"Records": [
{
"s3": {
"bucket": {"name": None},
# Missing 'object' key
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Malformed Minio event:"):
minio_parser.parse(invalid_minio_event)
def test_minio_parse_none_key(minio_parser):
"""Test parsing event with None field."""
invalid_minio_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"contentType": None, # 'contentType' should not be None
},
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Missing essential data fields"):
minio_parser.parse(invalid_minio_event)
def test_minio_validate_invalid_bucket(minio_parser):
"""Test validation with wrong bucket name."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="wrong-bucket",
metadata=None,
)
with pytest.raises(InvalidBucketError):
minio_parser.validate(event)
def test_minio_validate_invalid_filetype(minio_parser):
"""Test validation with unsupported file type."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.txt",
filetype="text/plain", # Not included in the default allowed filetypes
bucket_name="test-bucket",
metadata=None,
)
with pytest.raises(InvalidFileTypeError):
minio_parser.validate(event)
@pytest.mark.parametrize(
"invalid_filepath",
[
"invalid_filepath", # totally invalid string
"recordings/46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"recordings/46d1a121-2426-484d-8fb3-09b5d886f7a8", # missing extension
"46d1a121-2426-484d-8fb3-09b5d886f7a8", # missing url_encoded_folder_path and extension
"", # empty string
"46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # no folder at all
"uploads%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # wrong folder name
"folder%2Fuploads%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # nested but no recordings/
],
)
def test_minio_validate_invalid_filepath(invalid_filepath, minio_parser):
"""Test validation with malformed filepath."""
event = StorageEvent(
filepath=invalid_filepath,
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
with pytest.raises(InvalidFilepathError):
minio_parser.validate(event)
def test_minio_validate_valid_event(minio_parser):
"""Test validation with valid event data."""
event = StorageEvent(
filepath="recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
recording_id = minio_parser.validate(event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_minio_get_recording_id_success(minio_parser, valid_minio_event):
"""Test successful extraction of recording ID."""
recording_id = minio_parser.get_recording_id(valid_minio_event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_minio_validate_filepath_with_folder(minio_parser):
"""Test validation of filepath with folder structure."""
event = StorageEvent(
filepath="parent_folder%2Frecordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
)
recording_id = minio_parser.validate(event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
def test_minio_empty_allowed_filetypes():
"""Test MinioParser with empty allowed_filetypes."""
empty_types = set()
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes=empty_types)
assert parser._allowed_filetypes == {"audio/ogg", "video/mp4"}
def test_minio_custom_allowed_filetypes():
"""Test MinioParser with empty allowed_filetypes."""
custom_types = {"audio/mp3", "video/mov"}
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes=custom_types)
assert parser._allowed_filetypes == {"audio/mp3", "video/mov"}
def test_minio_validate_custom_filetypes():
"""Test validation of filepath with folder structure."""
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes={"audio/mp3"})
event = StorageEvent(
filepath="parent_folder%2Frecordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/mp3",
bucket_name="test-bucket",
metadata=None,
)
parser.validate(event)
def test_minio_constructor_none_bucket():
"""Test MinioParser constructor with None bucket name."""
with pytest.raises(ValueError, match="Bucket name cannot be None or empty"):
MinioParser(bucket_name=None)
def test_minio_constructor_empty_bucket():
"""Test MinioParser constructor with empty bucket name."""
with pytest.raises(ValueError, match="Bucket name cannot be None or empty"):
MinioParser(bucket_name="")
# S3Parser
@pytest.fixture
def valid_s3_event():
"""Mock a valid S3 event."""
return {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
},
}
}
]
}
@pytest.fixture
def s3_parser():
"""Mock an S3 parser."""
return S3Parser(bucket_name="test-bucket")
def test_s3_parse_valid_event(s3_parser, valid_s3_event):
"""Test parsing a valid S3 event."""
event = s3_parser.parse(valid_s3_event)
assert isinstance(event, StorageEvent)
assert event.filepath == "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg"
assert event.filetype == "audio/ogg"
assert event.bucket_name == "test-bucket"
assert event.metadata is None
def test_s3_parse_empty_data(s3_parser):
"""Test parsing empty S3 event data raises error."""
with pytest.raises(ParsingEventDataError, match="Received empty data."):
s3_parser.parse({})
def test_s3_parse_missing_keys(s3_parser):
"""Test parsing S3 event with missing key."""
invalid_s3_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
# Missing 'object' key
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Malformed S3 event:"):
s3_parser.parse(invalid_s3_event)
def test_s3_parse_none_key(s3_parser):
"""Test parsing S3 event with None field."""
invalid_s3_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": None,
},
}
}
]
}
with pytest.raises(ParsingEventDataError, match="Missing object key name"):
s3_parser.parse(invalid_s3_event)
def test_s3_parse_with_video_type(s3_parser):
"""Test parsing S3 event with mp4 file extension."""
video_event = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.mp4",
},
}
}
]
}
event = s3_parser.parse(video_event)
assert event.filetype == "video/mp4"
assert event.filepath.endswith(".mp4")
def test_s3_parse_unrecognized_extension(s3_parser):
"""Test parsing S3 event with unrecognized file extension."""
event_with_unknown_ext = {
"Records": [
{
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.zzunknown999",
},
}
}
]
}
with pytest.raises(TypeError, match="filetype cannot be None"):
s3_parser.parse(event_with_unknown_ext)
def test_s3_parser_keeps_encoded_filepath_compatible(settings):
"""Test S3 parser keeps already encoded object keys compatible."""
settings.RECORDING_OUTPUT_FOLDER = "recordings"
recording_id = "80ae9fe5-639a-438b-b86e-9e3dd2d55f4d"
parser = S3Parser(bucket_name="recordings-bucket")
data = {
"Records": [
{
"s3": {
"bucket": {"name": "recordings-bucket"},
"object": {
"key": f"recordings%2F{recording_id}.mp4",
},
}
}
]
}
assert parser.get_recording_id(data) == recording_id
def test_s3_parser_accepts_unencoded_filepath(settings):
"""Test S3 parser accepts raw object keys with slash separators."""
settings.RECORDING_OUTPUT_FOLDER = "recordings"
recording_id = "80ae9fe5-639a-438b-b86e-9e3dd2d55f4d"
parser = S3Parser(bucket_name="recordings-bucket")
data = {
"Records": [
{
"s3": {
"bucket": {"name": "recordings-bucket"},
"object": {
"key": f"recordings/{recording_id}.mp4",
},
}
}
]
}
assert parser.get_recording_id(data) == recording_id
def test_s3_parser_preserves_plus_signs_in_encoded_filepath(settings):
"""Test S3 parser preserves plus signs in already encoded object keys."""
settings.RECORDING_OUTPUT_FOLDER = "recordings"
recording_id = "80ae9fe5-639a-438b-b86e-9e3dd2d55f4d"
parser = S3Parser(bucket_name="recordings-bucket")
data = {
"Records": [
{
"s3": {
"bucket": {"name": "recordings-bucket"},
"object": {
"key": f"folder+name%2Frecordings%2F{recording_id}.mp4",
},
}
}
]
}
assert parser.get_recording_id(data) == recording_id
def test_s3_get_recording_id_success(s3_parser, valid_s3_event):
"""Test successful extraction of recording ID from S3 event."""
recording_id = s3_parser.get_recording_id(valid_s3_event)
assert recording_id == "46d1a121-2426-484d-8fb3-09b5d886f7a8"
# get_parser
@pytest.fixture
def clear_lru_cache():
"""Fixture to clear the LRU cache between tests."""
get_parser.cache_clear()
yield
get_parser.cache_clear()
def test_returns_correct_instance(clear_lru_cache):
"""Test if get_parser returns the correct parser instance."""
settings.AWS_STORAGE_BUCKET_NAME = "test-bucket"
parser = get_parser()
assert isinstance(parser, MinioParser)
assert parser._bucket_name == "test-bucket"
def test_caching_behavior(clear_lru_cache):
"""Test if the function properly caches the parser instance."""
settings.AWS_STORAGE_BUCKET_NAME = "test-bucket"
parser1 = get_parser()
parser2 = get_parser()
assert parser1 is parser2 # Check object identity
def test_different_settings_new_instance():
"""Test if changing settings creates a new instance."""
settings.AWS_STORAGE_BUCKET_NAME = "different-bucket"
parser = get_parser()
assert parser._bucket_name == "different-bucket"
def test_import_error_handling(clear_lru_cache):
"""Test handling of import errors for invalid parser class."""
settings.RECORDING_EVENT_PARSER_CLASS = "invalid.parser.path"
with pytest.raises(ImportError):
get_parser()
@mock.patch("core.recording.event.parsers.import_string")
def test_parser_instantiation_called_once(mock_import_string, clear_lru_cache):
"""Test that parser class is instantiated only once due to caching."""
mock_parser_cls = type(
"MockParser",
(),
{
"__init__": lambda self, bucket_name: setattr(
self, "_bucket_name", bucket_name
)
},
)
mock_import_string.return_value = mock_parser_cls
# First call
parser1 = get_parser()
# Second call
parser2 = get_parser()
# Verify import_string was called only once
mock_import_string.assert_called_once_with(settings.RECORDING_EVENT_PARSER_CLASS)
assert parser1 is parser2
def test_cache_clear_behavior(clear_lru_cache, settings):
"""Test that cache clearing creates new instance."""
settings.RECORDING_EVENT_PARSER_CLASS = "core.recording.event.parsers.MinioParser"
parser1 = get_parser()
get_parser.cache_clear()
parser2 = get_parser()
assert parser1 is not parser2 # Should be different instances after cache clear
@@ -12,6 +12,7 @@ from core.factories import RecordingFactory
from core.recording.services.recording_events import (
RecordingEventsError,
RecordingEventsService,
RecordingNotSavableError,
)
from core.utils import NotificationError
@@ -70,3 +71,56 @@ def test_handle_limit_reached_error(mock_notify, mode, notification_type, servic
mock_notify.assert_called_once_with(
room_name=str(recording.room.id), notification_data={"type": notification_type}
)
@pytest.mark.parametrize("status", ["active", "stopped"])
@pytest.mark.parametrize(
("notify_return_value", "expected_status"),
((True, "notification_succeeded"), (False, "saved")),
)
@mock.patch(
"core.recording.services.recording_events.notification_service."
"notify_external_services"
)
def test_handle_complete_saves_recording( # pylint: disable=too-many-arguments, too-many-positional-arguments
mock_notify_external_services,
notify_return_value,
expected_status,
status,
service,
):
"""Test handle_complete notifies external services and saves a savable recording."""
mock_notify_external_services.return_value = notify_return_value
recording = RecordingFactory(status=status)
service.handle_complete(recording)
mock_notify_external_services.assert_called_once_with(recording)
recording.refresh_from_db()
assert recording.status == expected_status
@pytest.mark.parametrize(
"status",
["initiated", "saved", "notification_succeeded", "aborted", "failed_to_start"],
)
@mock.patch(
"core.recording.services.recording_events.notification_service."
"notify_external_services"
)
def test_handle_complete_non_savable_recording(
mock_notify_external_services, status, service
):
"""Test handle_complete refuses recordings that are already saved or in error."""
recording = RecordingFactory(status=status)
with pytest.raises(RecordingNotSavableError):
service.handle_complete(recording)
mock_notify_external_services.assert_not_called()
recording.refresh_from_db()
assert recording.status == status
@@ -1,267 +0,0 @@
"""
Test recordings API endpoints in the Meet core app: save recording.
"""
# pylint: disable=redefined-outer-name,unused-argument
import uuid
from unittest import mock
import pytest
from rest_framework.test import APIClient
from ...factories import RecordingFactory
from ...models import Recording, RecordingStatusChoices
from ...recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
pytestmark = pytest.mark.django_db
@pytest.fixture
def recording_settings(settings):
"""Configure recording-related and storage event Django settings."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
settings.RECORDING_STORAGE_EVENT_ENABLE = True
return settings
@pytest.fixture
def mock_get_parser():
"""Mock 'get_parser' factory function."""
with mock.patch("core.api.viewsets.get_parser") as mock_parser:
yield mock_parser
def test_save_recording_anonymous(settings, client):
"""Anonymous users should not be allowed to save room recordings."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
RecordingFactory(status="active")
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
)
assert response.status_code == 401
assert Recording.objects.count() == 1
def test_save_recording_wrong_bearer(settings, client):
"""Requests with incorrect bearer token should be rejected when auth is required."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer wrongAuthToken",
)
assert response.status_code == 401
def test_save_recording_permission_needed(settings, client):
"""Recordings should not be saved when feature is disabled."""
settings.RECORDING_STORAGE_EVENT_TOKEN = "testAuthToken"
settings.RECORDING_STORAGE_EVENT_ENABLE = False
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 404
assert response.json() == {"detail": "Not found."}
def test_save_recording_parsing_error(recording_settings, mock_get_parser, client):
"""Test handling of parsing errors in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = ParsingEventDataError("Error message")
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid request data."}
def test_save_recording_bucket_error(recording_settings, mock_get_parser, client):
"""Test handling of invalid storage bucket errors in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidBucketError("Error message")
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid bucket specified."}
def test_save_recording_filetype_error(recording_settings, mock_get_parser):
"""Test handling of unsupported file types in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidFileTypeError(
"unsupported '.json'"
)
mock_get_parser.return_value = mock_parser
client = APIClient()
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Notification ignored."}
def test_save_recording_filepath_error(recording_settings, mock_get_parser):
"""Test handling of unsupported filepath in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidFilepathError(
"Invalid filepath structure: parent/folder/recording.jpeg"
)
mock_get_parser.return_value = mock_parser
client = APIClient()
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Notification ignored."}
def test_save_recording_unknown_recording(recording_settings, mock_get_parser, client):
"""Test handling of events for non-existent recordings."""
RecordingFactory(status="active")
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = uuid.uuid4()
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 404
assert response.json() == {"detail": "No recording found for this event."}
@pytest.mark.parametrize(
"status", ["failed_to_start", "aborted", "failed_to_stop", "saved", "initiated"]
)
def test_save_recording_non_savable_recording(
recording_settings, mock_get_parser, client, status
):
"""Test that recordings in non-savable states cannot be saved."""
recording = RecordingFactory(status=status)
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = recording.id
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 403
assert response.json() == {
"detail": f"Recording with ID {recording.id} cannot be saved because it is either,"
" in an error state or has already been saved."
}
@pytest.mark.parametrize("status", ["active", "stopped"])
def test_save_recording_success(recording_settings, mock_get_parser, client, status):
"""Test successful saving of recordings in valid states."""
recording = RecordingFactory(status=status)
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = recording.id
mock_get_parser.return_value = mock_parser
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Event processed."}
recording.refresh_from_db()
assert recording.status == RecordingStatusChoices.SAVED
@mock.patch(
"core.recording.services.recording_events.notification_service."
"notify_external_services"
)
@pytest.mark.parametrize("notification_succeeded", [True, False])
def test_save_recording_notifies_external_services(
mock_notify_external_services,
recording_settings,
mock_get_parser,
client,
notification_succeeded,
):
"""External services should be notified when a recording is saved."""
recording = RecordingFactory(status="active")
mock_parser = mock.Mock()
mock_parser.get_recording_id.return_value = recording.id
mock_get_parser.return_value = mock_parser
mock_notify_external_services.return_value = notification_succeeded
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Event processed."}
mock_notify_external_services.assert_called_once_with(recording)
recording.refresh_from_db()
assert recording.status == (
RecordingStatusChoices.NOTIFICATION_SUCCEEDED
if notification_succeeded
else RecordingStatusChoices.SAVED
)
@@ -75,12 +75,11 @@ def test_initialization(
)
@mock.patch("core.utils.notify_participants")
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
def test_handle_egress_ended_success( # noqa: PLR0913, PLR0917 # pylint: disable=too-many-arguments, too-many-positional-arguments
mock_update_metadata, mock_notify, mode, notification_type, service, settings
def test_handle_egress_ended_success( # pylint: disable=too-many-arguments, too-many-positional-arguments
mock_update_metadata, mock_notify, mode, notification_type, service
):
"""Should successfully stop recording and notifies all participant."""
settings.RECORDING_STORAGE_EVENT_ENABLE = False
recording = RecordingFactory(worker_id="worker-1", mode=mode, status="active")
mock_data = mock.MagicMock()
mock_data.egress_info.egress_id = recording.worker_id
@@ -159,12 +158,11 @@ def test_handle_egress_updated_non_handled(
)
@mock.patch("core.utils.notify_participants")
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
def test_handle_egress_ended_metadata_update_fails( # noqa: PLR0913, PLR0917 # pylint: disable=too-many-arguments, too-many-positional-arguments
mock_update_metadata, mock_notify, mode, notification_type, service, settings
def test_handle_egress_ended_metadata_update_fails( # pylint: disable=too-many-arguments, too-many-positional-arguments
mock_update_metadata, mock_notify, mode, notification_type, service
):
"""Should successfully stop and save recording when metadata's update fails."""
settings.RECORDING_STORAGE_EVENT_ENABLE = False
recording = RecordingFactory(worker_id="worker-1", mode=mode, status="active")
mock_data = mock.MagicMock()
mock_data.egress_info.egress_id = recording.worker_id
@@ -358,12 +356,10 @@ def test_handle_egress_ended_finalizes_recording( # noqa: PLR0913, PLR0917
recording_status,
egress_status,
service,
settings,
): # pylint: disable=too-many-arguments,too-many-positional-arguments
"""Should notify external services and save the recording on egress completion
(EGRESS_COMPLETE or EGRESS_LIMIT_REACHED) when RECORDING_STORAGE_EVENT_ENABLE is False.
(EGRESS_COMPLETE or EGRESS_LIMIT_REACHED).
"""
settings.RECORDING_STORAGE_EVENT_ENABLE = False
mock_notify_external_services.return_value = notify_return_value
recording = RecordingFactory(worker_id="worker-1", status="active")
@@ -379,47 +375,6 @@ def test_handle_egress_ended_finalizes_recording( # noqa: PLR0913, PLR0917
assert recording.status == recording_status
@mock.patch(
"core.recording.services.recording_events.notification_service."
"notify_external_services"
)
@mock.patch("core.utils.notify_participants")
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
@pytest.mark.parametrize(
"egress_status, expected_status",
[
(EgressStatus.EGRESS_COMPLETE, "active"),
(EgressStatus.EGRESS_LIMIT_REACHED, "stopped"),
],
)
def test_handle_egress_ended_does_not_finalize_when_webhooks_enabled( # noqa: PLR0913, PLR0917
mock_update_metadata,
mock_notify,
mock_notify_external_services,
egress_status,
expected_status,
service,
settings,
): # pylint: disable=too-many-arguments,too-many-positional-arguments
"""When storage event webhooks are enabled, egress_ended must not finalize the
recording: external services are never notified. EGRESS_LIMIT_REACHED still stops
the recording, EGRESS_COMPLETE leaves it active.
"""
settings.RECORDING_STORAGE_EVENT_ENABLE = True
recording = RecordingFactory(worker_id="worker-1", status="active")
mock_data = mock.MagicMock()
mock_data.egress_info.egress_id = recording.worker_id
mock_data.egress_info.status = egress_status
service._handle_egress_ended(mock_data)
mock_notify_external_services.assert_not_called()
recording.refresh_from_db()
assert recording.status == expected_status
@pytest.mark.parametrize(
"egress_status",
[
@@ -432,10 +387,9 @@ def test_handle_egress_ended_does_not_finalize_when_webhooks_enabled( # noqa: P
)
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
def test_handle_egress_ended_does_not_save_on_wrong_status(
mock_update_metadata, egress_status, service, settings
mock_update_metadata, egress_status, service
):
"""Shouldn't save on invalid status."""
settings.RECORDING_STORAGE_EVENT_ENABLE = False
recording = RecordingFactory(worker_id="worker-1", status="active")
mock_data = mock.MagicMock()
@@ -453,14 +407,13 @@ def test_handle_egress_ended_does_not_save_on_wrong_status(
)
@mock.patch("core.services.room_management.RoomManagement.update_metadata")
def test_handle_egress_ended_ignores_non_savable_recording(
mock_update_metadata, status, service, settings
mock_update_metadata, status, service
):
"""Should handle non-savable recordings idempotently without raising.
'egress_ended' may be redelivered (e.g. for an already-saved recording);
this must not raise, otherwise the webhook would 500 and LiveKit would retry.
"""
settings.RECORDING_STORAGE_EVENT_ENABLE = False
recording = RecordingFactory(worker_id="worker-1", status=status)
mock_data = mock.MagicMock()
-14
View File
@@ -729,20 +729,6 @@ class Base(Configuration):
environ_name="RECORDING_WORKER_CLASSES",
environ_prefix=None,
)
RECORDING_EVENT_PARSER_CLASS = values.Value(
"core.recording.event.parsers.MinioParser",
environ_name="RECORDING_EVENT_PARSER_CLASS",
environ_prefix=None,
)
RECORDING_ENABLE_STORAGE_EVENT_AUTH = values.BooleanValue(
True, environ_name="RECORDING_ENABLE_STORAGE_EVENT_AUTH", environ_prefix=None
)
RECORDING_STORAGE_EVENT_ENABLE = values.BooleanValue(
False, environ_name="RECORDING_STORAGE_EVENT_ENABLE", environ_prefix=None
)
RECORDING_STORAGE_EVENT_TOKEN = SecretFileValue(
None, environ_name="RECORDING_STORAGE_EVENT_TOKEN", environ_prefix=None
)
# Number of days before recordings expire - must be synced with bucket lifecycle policy
# Set to None for no expiration
RECORDING_EXPIRATION_DAYS = values.IntegerValue(
-2
View File
@@ -186,8 +186,6 @@ backend:
CELERY_BROKER_URL: redis://default:pass@redis-master:6379/1
# Recording & Transcription
RECORDING_ENABLE: True
RECORDING_STORAGE_EVENT_ENABLE: True
RECORDING_STORAGE_EVENT_TOKEN: password
SUMMARY_SERVICE_ENDPOINT: http://meet-summary:80/api/v2/async-jobs/transcribe/
SUMMARY_SERVICE_API_TOKEN: password
SUMMARY_SERVICE_WEBHOOK_API_TOKEN: webhook-password
-23
View File
@@ -113,26 +113,3 @@ spec:
exit 0
restartPolicy: Never
backoffLimit: 3
---
apiVersion: batch/v1
kind: Job
metadata:
name: minio-webhook
spec:
template:
spec:
containers:
- name: mc
image: minio/mc
command:
- /bin/sh
- -c
- |
/usr/bin/mc alias set meet http://minio:9000 meet password && \
/usr/bin/mc admin config set meet notify_webhook:meet-webhook endpoint="https://meet.127.0.0.1.nip.io/api/v1.0/recordings/storage-hook/" auth_token="Bearer password" && \
/usr/bin/mc admin service restart meet --wait --json && \
sleep 15 && \
/usr/bin/mc event add meet/meet-media-storage arn:minio:sqs::meet-webhook:webhook --event put --prefix "recordings" && \
exit 0
restartPolicy: Never
backoffLimit: 3
-7
View File
@@ -22,13 +22,6 @@ Configure your env values in `env.d/summary` to properly set up WhisperX and the
make run
```
When the stack is up, configure the MinIO webhook
*(TODO: add this step to `make bootstrap`)*
```sh
make minio-webhook-setup
```
If you want to develop on the Celery workers with hot reloading, run:
```sh