Compare commits

...

21 Commits

Author SHA1 Message Date
leo 2a00e3a72e fix linting 2026-03-17 18:00:34 +01:00
leo efa0d48598 revert readme change 2026-03-17 17:42:39 +01:00
leo 873cc083ab refactor env proposal 2026-03-17 17:42:39 +01:00
leo 8fc7ccf33e update make test 2026-03-17 17:42:39 +01:00
leo d650dbc988 👷(CI) add summary service testing to CI
Add summary service testing to CI.
2026-03-17 17:42:39 +01:00
leo c08411d133 (summary) add unit and API tests for summary service
Summary service currently has no tests. Add unit and API tests to
summary service.
2026-03-17 17:42:39 +01:00
leo cb4502354e 🔧(build) update openssl and libssl3t64 versions to fix build
Update OpenSSL and libssl3t64 package versions to resolve a build failure
caused by version regression.
2026-03-17 17:28:40 +01:00
lebaudantoine 7347fc7c86 🚨(doc) fix changelog linting
Merge an outside contribution, didn't notice it broke the changelog fix it.
2026-03-17 16:45:04 +01:00
Hadrien Blanc ada7d9a666 🐛(frontend) fix dimension mismatch in BackgroundCustomProcessor
The getImageData call was using PROCESSING_WIDTH for both dimensions
instead of PROCESSING_WIDTH and PROCESSING_HEIGHT. This caused the
source image data to be 256x256 instead of the expected 256x144, leading
to buffer overflow when writing to the segmentation mask and potential
visual artifacts in Firefox background effects.
2026-03-17 16:41:40 +01:00
lebaudantoine 0cb3fb8e3c 🔧(ci) explicitly set Docker Hub CI permissions to read-only
Define read-only permissions in the workflow to clarify the
expected access level and follow the principle of least
privilege.
2026-03-15 16:53:07 +01:00
lebaudantoine dcb788b57b 🔒️(backend) avoid information exposure through exception messages
Sanitize error handling to prevent leaking internal details when
invalid or malicious requests are sent to the API.

Return generic error responses to reduce the risk of information
disclosure during probing attempts.
2026-03-13 17:33:55 +01:00
lebaudantoine 73bcb9d598 🐛(backend) fix unescaped dot in regex pattern
The dot before (?P<extension>...) was not escaped and matched any
character instead of a literal period.

Escape it to align with MEDIA_STORAGE_URL_PATTERN, which correctly
uses \. for the file extension separator.
2026-03-13 16:19:36 +01:00
lebaudantoine f3e90c3999 ♻️(backend) clarify storage hook message for intentionally ignored req
Update the response message to explicitly state that certain
notifications are ignored on purpose, avoiding confusion
during debugging or log inspection.
2026-03-13 15:42:30 +01:00
lebaudantoine cb4ed3c9d7 🩹(backend) ignore non-recording uploads in storage webhook handler
With the introduction of background file uploads, a misconfigured
MinIO webhook could trigger the storage hook endpoint for unrelated
files.

While the dev setup now filters events via the MinIO lifecycle
configuration, add a safeguard at the application level.

Enforce a stricter filepath regex when parsing storage hook events
and ignore files outside the recording output directory.

Return a clean 200 response to acknowledge the webhook while
avoiding unnecessary processing.
2026-03-13 15:34:02 +01:00
lebaudantoine f8b0746e73 ♻️(backend) factorize regex used in the S3 event parser
Avoid duplicating patterns that match the same logical unit by
sharing common parts between the two regexes.

This logical factorization improves maintainability and ensures
consistent parsing behavior across S3 event handlers.
2026-03-13 15:34:02 +01:00
lebaudantoine d8ad7a743e ♻️(backend) align recording file regex with MEDIA_URL-based pattern
Initially I avoided coupling the recording path regex with the
setting that provides the Django static URL. However the new
file viewset already relies on it, and Drive does as well.

For consistency, update the recording regex to use the same
STATIC_URL-based approach.

While the coupling may not feel ideal, having two different
regex strategies for similar file paths would be worse.
2026-03-13 15:34:02 +01:00
lebaudantoine 0ba445895c ♻️(backend) merge room name regex and centralize patterns in enums
Combine the regex used in Drive with the one from the recording
feature and centralize them in the enums module.

Although the module name suggests only enums, it also hosts
shared constants used across the codebase.
2026-03-13 15:34:02 +01:00
Florent Chehab 3b719ab9ba (settings) disable file upload by default & max count
Add FILE_UPLOAD_ENABLED setting (default to False, to avoid
a breaking change). Also adds a max_count_by_user sub setting
to restrict the number of uploaded files per user.
2026-03-12 14:55:33 +01:00
Florent Chehab 8887e811d3 🔨(tilt) improve tilt stack stability
* Increase the backoff limit on jobs
* Add redis dependency for livekit
2026-03-12 14:41:58 +01:00
Florent Chehab cb9e994749 ️(helm) reduce initialDelaySeconds and add periods seconds
InitialDelaySeconds was set to 30s which caused the tilt stack
startup to be very slow. In this commit I split the initial delay
and the period arg.
5s seems to be relevant given the nature of the app.
2026-03-12 14:41:57 +01:00
Florent Chehab 1841533d2c 🔨(makefile) add default to meet namespace
Avoids resources being deployed in the wrong namespace
depending on user config.
2026-03-12 14:41:40 +01:00
44 changed files with 833 additions and 138 deletions
+15
View File
@@ -12,6 +12,9 @@ on:
branches:
- 'main'
permissions:
contents: read
env:
DOCKER_USER: 1001:127
DOCKER_CONTAINER_REGISTRY_HOSTNAME: docker.io
@@ -20,6 +23,8 @@ env:
jobs:
build-and-push-backend:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
-
name: Checkout repository
@@ -63,6 +68,8 @@ jobs:
build-and-push-frontend-generic:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
-
name: Checkout repository
@@ -107,6 +114,8 @@ jobs:
build-and-push-frontend-dinum:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
-
name: Checkout repository
@@ -151,6 +160,8 @@ jobs:
build-and-push-summary:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
-
name: Checkout repository
@@ -197,6 +208,8 @@ jobs:
build-and-push-agents:
runs-on: ubuntu-latest
permissions:
contents: read
steps:
-
name: Checkout repository
@@ -242,6 +255,8 @@ jobs:
labels: ${{ steps.meta.outputs.labels }}
notify-argocd:
permissions:
contents: read
needs:
- build-and-push-frontend-generic
- build-and-push-frontend-dinum
+26 -2
View File
@@ -180,7 +180,7 @@ jobs:
- name: Lint code with ruff
run: ~/.local/bin/ruff check .
test-back:
test-backend:
runs-on: ubuntu-latest
needs: build-mails
permissions:
@@ -294,9 +294,33 @@ jobs:
- name: Generate a MO file from strings extracted from the project
run: uv run python manage.py compilemessages
- name: Run tests
- name: Run backend tests
run: uv run pytest -n 2
test-summary:
runs-on: ubuntu-latest
permissions:
contents: read
defaults:
run:
working-directory: src/summary
steps:
- name: Checkout repository
uses: actions/checkout@v6
- name: Install Python
uses: actions/setup-python@v6
with:
python-version: "3.13"
cache: "pip"
- name: Install development dependencies
run: pip install --user .[dev]
- name: Run summary tests
run: ~/.local/bin/pytest
lint-front:
runs-on: ubuntu-latest
permissions:
+6
View File
@@ -14,6 +14,7 @@ and this project adheres to
- ✨(helm) support ingress for custom background image #1124
- ✨(backend) add authenticated user rate throttling on request-entry #1129
- ✨(backend) expose `is_active` field for Application in Django admin #1133
- ✨(file-upload) disable by default & limit count by user #1141
### Changed
@@ -26,12 +27,17 @@ and this project adheres to
- 🔨(python-env) migrate meet main app to UV #1120
- ♻️(backend) align Application model field with `is_active` convention #1133
- 🔐(backend) avoids revealing the inactive status of an application #1135
- ⚡️(helm) reduce initialDelaySeconds and add periods seconds #1139
- 🔒️(backend) avoid information exposure through exception messages #1144
### Fixed
- 🐛(frontend) fix hand icon and queue position alignment and position #1119
- 🩹(backend) add page_size to pagination for room endpoints #1131
- 🐛(backend) refactor lobby throttling to use participant id #1129
- 🩹(backend) ignore non-recording uploads in storage webhook handler #1142
- 🐛(frontend) fix dimension mismatch in BackgroundCustomProcessor #1116
## [1.10.0] - 2026-03-05
+9 -3
View File
@@ -191,6 +191,7 @@ lint-pylint: ## lint back-end python sources with pylint only on changed files f
test: ## run project tests
@$(MAKE) test-back-parallel
@$(MAKE) test-summary
.PHONY: test
test-back: ## run back-end tests
@@ -203,6 +204,11 @@ test-back-parallel: ## run all back-end tests in parallel
bin/pytest -n auto $${args:-${1}}
.PHONY: test-back-parallel
test-summary: ## run summary service tests
@args="$(filter-out $@,$(MAKECMDGOALS))" && \
bin/pytest-summary $${args:-${1}}
.PHONY: test-summary
makemigrations: ## run django makemigrations for the Meet project.
@echo "$(BOLD)Running makemigrations$(RESET)"
@$(COMPOSE) up -d postgresql
@@ -354,13 +360,13 @@ install-external-secrets: ## install the kubernetes secrets from Vaultwarden
.PHONY: build-k8s-cluster
start-tilt: ## start the kubernetes cluster using kind
tilt up -f ./bin/Tiltfile
tilt up --namespace=meet -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
start-tilt-keycloak: ## start the kubernetes cluster using kind, without Pro Connect for authentication, use keycloak
DEV_ENV=dev-keycloak tilt up -f ./bin/Tiltfile
DEV_ENV=dev-keycloak tilt up --namespace=meet -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
start-tilt-dinum: ## start the kubernetes cluster using kind, without Pro Connect for authentication, but with DINUM styles
DEV_ENV=dev-dinum tilt up -f ./bin/Tiltfile
DEV_ENV=dev-dinum tilt up --namespace=meet -f ./bin/Tiltfile
.PHONY: build-k8s-cluster
+1
View File
@@ -103,6 +103,7 @@ k8s_resource('meet-celery-backend', resource_deps=['redis'])
k8s_resource('meet-celery-summarize', resource_deps=['redis'])
k8s_resource('meet-celery-transcribe', resource_deps=['redis'])
k8s_resource('meet-backend-migrate', resource_deps=['meet-backend'])
k8s_resource('livekit-livekit-server', resource_deps=['redis'])
k8s_resource('livekit-livekit-server-test-connection', resource_deps=['livekit-livekit-server'])
k8s_resource('keycloak', resource_deps=['kc-postgresql'])
k8s_resource('meet-backend-createsuperuser', resource_deps=['meet-backend-migrate'])
+2 -1
View File
@@ -1,8 +1,9 @@
#!/usr/bin/env bash
# NB: this file is used locally only. In CI, it is overwritten by pytest install
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
-e DJANGO_CONFIGURATION=Test \
app-dev \
pytest "$@"
pytest "$@"
+7
View File
@@ -0,0 +1,7 @@
#!/usr/bin/env bash
source "$(dirname "${BASH_SOURCE[0]}")/_config.sh"
_dc_run \
app-summary-dev \
python -m pytest "$@"
+1
View File
@@ -28,6 +28,7 @@ AWS_S3_ENDPOINT_URL=http://minio:9000
AWS_S3_ACCESS_KEY_ID=meet
AWS_S3_SECRET_ACCESS_KEY=password
MEDIA_BASE_URL=http://localhost:8083
FILE_UPLOAD_ENABLED=True
# OIDC
OIDC_OP_JWKS_ENDPOINT=http://nginx:8083/realms/meet/protocol/openid-connect/certs
+8 -7
View File
@@ -1,13 +1,14 @@
FROM python:3.13-slim AS base
# Install system dependencies required by LiveKit
RUN apt-get update && apt-get install -y \
libglib2.0-0 \
libgobject-2.0-0 \
"openssl=3.5.4-1~deb13u2" \
"libssl3t64=3.5.4-1~deb13u2" \
&& rm -rf /var/lib/apt/lists/*
RUN apt-get update \
&& apt-get install -y --no-install-recommends \
libglib2.0-0 \
libgobject-2.0-0 \
&& apt-get upgrade -y openssl libssl3t64 \
&& apt-get clean \
&& rm -rf /var/lib/apt/lists/*
FROM base AS builder
WORKDIR /builder
+15
View File
@@ -43,6 +43,21 @@ def get_frontend_configuration(request):
"expiration_days": settings.RECORDING_EXPIRATION_DAYS,
"max_duration": settings.RECORDING_MAX_DURATION,
},
"background_image": {
"upload_is_enabled": settings.FILE_UPLOAD_ENABLED,
"max_count_by_user": settings.FILE_UPLOAD_RESTRICTIONS["background_image"][
"max_count_by_user"
],
"max_size": settings.FILE_UPLOAD_RESTRICTIONS["background_image"][
"max_size"
],
"allowed_extensions": settings.FILE_UPLOAD_RESTRICTIONS["background_image"][
"allowed_extensions"
],
"allowed_mimetypes": settings.FILE_UPLOAD_RESTRICTIONS["background_image"][
"allowed_mimetypes"
],
},
"telephony": {
"enabled": settings.ROOM_TELEPHONY_ENABLED,
"phone_number": settings.ROOM_TELEPHONY_PHONE_NUMBER
+1
View File
@@ -13,6 +13,7 @@ class FeatureFlag:
"recording": "RECORDING_ENABLE",
"storage_event": "RECORDING_STORAGE_EVENT_ENABLE",
"subtitle": "ROOM_SUBTITLE_ENABLED",
"file_upload": "FILE_UPLOAD_ENABLED",
}
@classmethod
+8
View File
@@ -1,5 +1,6 @@
"""Permission handlers for the Meet core app."""
from django.conf import settings
from django.http import Http404
from rest_framework import permissions
@@ -116,6 +117,13 @@ class FilePermission(IsAuthenticated):
Handling soft deletions specificities
"""
def has_permission(self, request, view):
"""Allow access only to authenticated users."""
if not settings.FILE_UPLOAD_ENABLED:
raise Http404
return super().has_permission(request, view)
def has_object_permission(self, request, view, obj):
"""
Return a 404 on deleted files or if the user is not the owner
+35 -19
View File
@@ -1,7 +1,6 @@
"""API endpoints"""
# pylint: disable=too-many-lines
import re
import uuid
from logging import getLogger
from urllib.parse import unquote, urlparse
@@ -12,6 +11,7 @@ from django.db.models import Q
from django.http import Http404
from django.shortcuts import get_object_or_404
from django.utils.text import slugify
from django.utils.translation import gettext_lazy as _
from django_filters import rest_framework as django_filters
from rest_framework import (
@@ -33,10 +33,12 @@ from rest_framework import (
from core import enums, models, utils
from core.api.filters import ListFileFilter
from core.enums import MEDIA_STORAGE_URL_PATTERN
from core.recording.enums import FileExtension
from core.recording.event.authentication import StorageEventAuthentication
from core.recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
@@ -78,17 +80,6 @@ from .feature_flag import FeatureFlag
logger = getLogger(__name__)
FILE_FOLDER = settings.FILE_UPLOAD_PATH
UUID_REGEX = (
r"[a-fA-F0-9]{8}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{4}-[a-fA-F0-9]{12}"
)
FILE_EXT_REGEX = r"[\d\w]+"
MEDIA_STORAGE_URL_PATTERN = re.compile(
f"{settings.MEDIA_URL:s}"
rf"(?P<key>{FILE_FOLDER:s}/(?P<pk>{UUID_REGEX:s})\.{FILE_EXT_REGEX:s})$"
)
class NestedGenericViewSet(viewsets.GenericViewSet):
"""
A generic Viewset aims to be used in a nested route context.
@@ -497,9 +488,7 @@ class RoomViewSet(
if status_code == drf_status.HTTP_500_INTERNAL_SERVER_ERROR:
raise e
return drf_response.Response(
{"status": "error", "message": str(e)}, status=status_code
)
return drf_response.Response({"status": "error"}, status=status_code)
@decorators.action(
detail=False,
@@ -766,14 +755,19 @@ class RecordingViewSet(
recording_id = parser.get_recording_id(request.data)
except ParsingEventDataError as e:
raise drf_exceptions.PermissionDenied(f"Invalid request data: {e}") from e
raise drf_exceptions.PermissionDenied("Invalid request data.") from e
except InvalidBucketError as e:
raise drf_exceptions.PermissionDenied("Invalid bucket specified") from e
raise drf_exceptions.PermissionDenied("Invalid bucket specified.") from e
except InvalidFileTypeError as e:
except InvalidFilepathError:
return drf_response.Response(
{"message": f"Ignore this file type, {e}"},
{"message": "Notification ignored."},
)
except InvalidFileTypeError:
return drf_response.Response(
{"message": "Notification ignored."},
)
try:
@@ -973,6 +967,26 @@ class FileViewSet(
def perform_create(self, serializer):
"""Set the current user as creator of the newly created file."""
if settings.FILE_UPLOAD_APPLY_RESTRICTIONS:
file_type = serializer.validated_data["type"]
config_for_file_type = settings.FILE_UPLOAD_RESTRICTIONS[file_type]
count = models.File.objects.filter(
creator=self.request.user,
deleted_at__isnull=True,
type=file_type,
).count()
if count >= config_for_file_type["max_count_by_user"]:
logger.info(
"create_item: user reached max files per user for type %s",
file_type,
)
raise serializers.PermissionDenied(
_("You have reached the maximum number of files for this type.")
)
serializer.save(creator=self.request.user)
def perform_destroy(self, instance):
@@ -980,6 +994,7 @@ class FileViewSet(
instance.soft_delete()
@decorators.action(detail=True, methods=["post"], url_path="upload-ended")
@FeatureFlag.require("file_upload")
def upload_ended(self, request, *args, **kwargs):
"""
Check the actual uploaded file and mark it as ready.
@@ -1162,6 +1177,7 @@ class FileViewSet(
return url_params, request.user.id, file
@decorators.action(detail=False, methods=["get"], url_path="media-auth")
@FeatureFlag.require("file_upload")
def media_auth(self, request, *args, **kwargs):
"""
This view is used by an Nginx subrequest to control access to an file's
+7 -1
View File
@@ -14,9 +14,15 @@ FILE_EXT_REGEX = r"[a-zA-Z0-9]{1,10}"
# pylint: disable=line-too-long
RECORDING_STORAGE_URL_PATTERN = re.compile(
f"/media/{settings.RECORDING_OUTPUT_FOLDER}/(?P<recording_id>{UUID_REGEX:s}).(?P<extension>{FILE_EXT_REGEX:s})"
rf"{settings.MEDIA_URL:s}{settings.RECORDING_OUTPUT_FOLDER}/(?P<recording_id>{UUID_REGEX:s})\.(?P<extension>{FILE_EXT_REGEX:s})"
)
MEDIA_STORAGE_URL_PATTERN = re.compile(
f"{settings.MEDIA_URL:s}"
rf"(?P<key>{settings.FILE_UPLOAD_PATH:s}/(?P<pk>{UUID_REGEX:s})\.{FILE_EXT_REGEX:s})$"
)
# Django sets `LANGUAGES` by default with all supported languages. We can use it for
# the choice of languages which should not be limited to the few languages active in
# the app.
+3 -1
View File
@@ -9,6 +9,8 @@ from typing import Any, Dict, Optional, Protocol
from django.conf import settings
from django.utils.module_loading import import_string
from core.enums import FILE_EXT_REGEX, UUID_REGEX
from .exceptions import (
InvalidBucketError,
InvalidFilepathError,
@@ -86,7 +88,7 @@ class MinioParser:
# pylint: disable=line-too-long
self._filepath_regex = re.compile(
r"(?P<url_encoded_folder_path>(?:[^%]+%2F)+)?(?P<recording_id>[0-9a-fA-F\-]{36})\.(?P<extension>[a-zA-Z0-9]+)"
rf"(?P<url_encoded_folder_path>(?:[^%]+%2F)+)?{settings.RECORDING_OUTPUT_FOLDER}%2F(?P<recording_id>{UUID_REGEX})\.(?P<extension>{FILE_EXT_REGEX})"
)
@staticmethod
@@ -174,6 +174,26 @@ def test_api_files_create_file_authenticated_extension_case_insensitive():
assert file.title == "file"
def test_api_files_create_file_disabled(settings):
"""
Creating a file is denied if file upload is disabled
"""
settings.FILE_UPLOAD_ENABLED = False
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.post(
"/api/v1.0/files/",
{
"type": FileTypeChoices.BACKGROUND_IMAGE,
"filename": "file.JPG",
},
format="json",
)
assert response.status_code == 404
assert not File.objects.exists()
def test_api_files_create_file_authenticated_not_checking_extension(settings):
"""
Creating a file with an extension not allowed should not fail when restrictions are disabled.
@@ -239,6 +259,48 @@ def test_api_files_create_file_authenticated_hidden_file_but_checking_extension_
assert response.json() == {"filename": ["This file extension is not allowed."]}
def test_api_files_create_file_too_many(
settings,
):
"""
Creating a file is forbidden if above user limit.
"""
settings.FILE_UPLOAD_APPLY_RESTRICTIONS = True
settings.FILE_UPLOAD_RESTRICTIONS = {
"background_image": {
**settings.FILE_UPLOAD_RESTRICTIONS["background_image"],
"max_count_by_user": 1,
},
}
user = factories.UserFactory()
client = APIClient()
client.force_login(user)
response = client.post(
"/api/v1.0/files/",
{
"type": FileTypeChoices.BACKGROUND_IMAGE,
"filename": "1.png",
},
)
assert response.status_code == 201
response = client.post(
"/api/v1.0/files/",
{
"type": FileTypeChoices.BACKGROUND_IMAGE,
"filename": "2.png",
},
)
assert response.status_code == 403
assert response.json() == {
"detail": "You have reached the maximum number of files for this type."
}
assert File.objects.count() == 1
def test_api_files_create_force_id_success():
"""It should be possible to force the item ID when creating a item."""
user = factories.UserFactory()
@@ -32,7 +32,7 @@ def valid_minio_event():
"s3": {
"bucket": {"name": "test-bucket"},
"object": {
"key": "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"key": "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"contentType": "audio/ogg",
},
}
@@ -51,7 +51,7 @@ def test_parse_valid_event(minio_parser, valid_minio_event):
"""Test parsing a valid Minio event."""
event = minio_parser.parse(valid_minio_event)
assert isinstance(event, StorageEvent)
assert event.filepath == "recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg"
assert event.filepath == "recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg"
assert event.filetype == "audio/ogg"
assert event.bucket_name == "test-bucket"
assert event.metadata is None
@@ -130,11 +130,13 @@ def test_validate_invalid_filetype(minio_parser):
"invalid_filepath",
[
"invalid_filepath", # totally invalid string
"recording/46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"recording/46d1a121-2426-484d-8fb3-09b5d886f7a8", # missing extension
"recordings/46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
"recordings/46d1a121-2426-484d-8fb3-09b5d886f7a8", # missing extension
"46d1a121-2426-484d-8fb3-09b5d886f7a8", # missing url_encoded_folder_path and extension
"", # empty string
"recording%2F46d1a1212426484d8fb309b5d886f7a8.ogg",
"46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # no folder at all
"uploads%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # wrong folder name
"folder%2Fuploads%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg", # nested but no recordings/
],
)
def test_validate_invalid_filepath(invalid_filepath, minio_parser):
@@ -152,7 +154,7 @@ def test_validate_invalid_filepath(invalid_filepath, minio_parser):
def test_validate_valid_event(minio_parser):
"""Test validation with valid event data."""
event = StorageEvent(
filepath="recording%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filepath="recordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
@@ -170,7 +172,7 @@ def test_get_recording_id_success(minio_parser, valid_minio_event):
def test_validate_filepath_with_folder(minio_parser):
"""Test validation of filepath with folder structure."""
event = StorageEvent(
filepath="parent_folder%2Ffolder%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filepath="parent_folder%2Frecordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/ogg",
bucket_name="test-bucket",
metadata=None,
@@ -219,7 +221,7 @@ def test_validate_custom_filetypes():
parser = MinioParser(bucket_name="test-bucket", allowed_filetypes={"audio/mp3"})
event = StorageEvent(
filepath="parent_folder%2Ffolder%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filepath="parent_folder%2Frecordings%2F46d1a121-2426-484d-8fb3-09b5d886f7a8.ogg",
filetype="audio/mp3",
bucket_name="test-bucket",
metadata=None,
@@ -14,6 +14,7 @@ from ...factories import RecordingFactory
from ...models import Recording, RecordingStatusChoices
from ...recording.event.exceptions import (
InvalidBucketError,
InvalidFilepathError,
InvalidFileTypeError,
ParsingEventDataError,
)
@@ -94,7 +95,7 @@ def test_save_recording_parsing_error(recording_settings, mock_get_parser, clien
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid request data: Error message"}
assert response.json() == {"detail": "Invalid request data."}
def test_save_recording_bucket_error(recording_settings, mock_get_parser, client):
@@ -111,7 +112,7 @@ def test_save_recording_bucket_error(recording_settings, mock_get_parser, client
)
assert response.status_code == 403
assert response.json() == {"detail": "Invalid bucket specified"}
assert response.json() == {"detail": "Invalid bucket specified."}
def test_save_recording_filetype_error(recording_settings, mock_get_parser):
@@ -132,7 +133,28 @@ def test_save_recording_filetype_error(recording_settings, mock_get_parser):
)
assert response.status_code == 200
assert response.json() == {"message": "Ignore this file type, unsupported '.json'"}
assert response.json() == {"message": "Notification ignored."}
def test_save_recording_filepath_error(recording_settings, mock_get_parser):
"""Test handling of unsupported filepath in recording event data."""
mock_parser = mock.Mock()
mock_parser.get_recording_id.side_effect = InvalidFilepathError(
"Invalid filepath structure: parent/folder/recording.jpeg"
)
mock_get_parser.return_value = mock_parser
client = APIClient()
response = client.post(
"/api/v1.0/recordings/storage-hook/",
{"recording_data": "valid-data"},
HTTP_AUTHORIZATION="Bearer testAuthToken",
)
assert response.status_code == 200
assert response.json() == {"message": "Notification ignored."}
def test_save_recording_unknown_recording(recording_settings, mock_get_parser, client):
@@ -77,7 +77,6 @@ def test_missing_auth_header(client, serialized_event_data, mock_livekit_config)
assert response.status_code == 401
assert response.json() == {
"status": "error",
"message": "Authorization header missing",
}
@@ -91,7 +90,7 @@ def test_invalid_payload(client, auth_token, mock_livekit_config):
)
assert response.status_code == 400
assert response.json() == {"status": "error", "message": "Invalid webhook payload"}
assert response.json() == {"status": "error"}
def test_unknown_event_type(client, mock_livekit_config):
@@ -116,7 +115,6 @@ def test_unknown_event_type(client, mock_livekit_config):
assert response.status_code == 422
assert response.json() == {
"status": "error",
"message": "Unknown webhook type: unknown_event_type",
}
+11 -7
View File
@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-03-11 17:31+0000\n"
"POT-Creation-Date: 2026-03-12 13:46+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -76,11 +76,11 @@ msgstr "%(count)s Aufnahme(n) erfolgreich als Fehler beim Stoppen markiert
msgid "Skipped %(count)s recording(s) with an ineligible status."
msgstr "%(count)s abgelaufene Aufnahme(n) übersprungen."
#: core/admin.py:341
#: core/admin.py:342
msgid "No scopes"
msgstr "Keine Scopes"
#: core/admin.py:343
#: core/admin.py:344
msgid "Scopes"
msgstr "Scopes"
@@ -98,6 +98,10 @@ msgstr ""
msgid "This file extension is not allowed."
msgstr "Diese Dateiendung ist nicht erlaubt."
#: core/api/serializers.py:533
msgid "You have reached the maximum number of files for this type."
msgstr "Sie haben die maximale Anzahl an Dateien dieses Typs erreicht."
#: core/models.py:37
msgid "Member"
msgstr "Mitglied"
@@ -600,18 +604,18 @@ msgstr ""
" Wenn Sie Fragen haben oder Unterstützung benötigen, wenden Sie sich bitte "
"an unser Support-Team unter %(support_email)s. "
#: meet/settings.py:215
#: meet/settings.py:223
msgid "English"
msgstr "Englisch"
#: meet/settings.py:216
#: meet/settings.py:224
msgid "French"
msgstr "Französisch"
#: meet/settings.py:217
#: meet/settings.py:225
msgid "Dutch"
msgstr "Niederländisch"
#: meet/settings.py:218
#: meet/settings.py:226
msgid "German"
msgstr "Deutsch"
+11 -7
View File
@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-03-11 17:31+0000\n"
"POT-Creation-Date: 2026-03-12 13:46+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -76,11 +76,11 @@ msgstr "%(count)s recording(s) successfully marked as 'Failed to Stop'."
msgid "Skipped %(count)s recording(s) with an ineligible status."
msgstr "Skipped %(count)s expired recording(s)."
#: core/admin.py:341
#: core/admin.py:342
msgid "No scopes"
msgstr "No scopes"
#: core/admin.py:343
#: core/admin.py:344
msgid "Scopes"
msgstr "Scopes"
@@ -96,6 +96,10 @@ msgstr "You must be administrator or owner of a room to add accesses to it."
msgid "This file extension is not allowed."
msgstr "This file extension is not allowed."
#: core/api/serializers.py:533
msgid "You have reached the maximum number of files for this type."
msgstr "You have reached the maximum number of files for this type."
#: core/models.py:37
msgid "Member"
msgstr "Member"
@@ -596,18 +600,18 @@ msgstr ""
" If you have any questions or need assistance, please contact our support "
"team at %(support_email)s. "
#: meet/settings.py:215
#: meet/settings.py:223
msgid "English"
msgstr "English"
#: meet/settings.py:216
#: meet/settings.py:224
msgid "French"
msgstr "French"
#: meet/settings.py:217
#: meet/settings.py:225
msgid "Dutch"
msgstr "Dutch"
#: meet/settings.py:218
#: meet/settings.py:226
msgid "German"
msgstr "German"
+13 -8
View File
@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-03-11 17:31+0000\n"
"POT-Creation-Date: 2026-03-12 13:46+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: antoine.lebaud@mail.numerique.gouv.fr\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -68,7 +68,8 @@ msgstr "Marquer les enregistrements sélectionnés comme « Échec darrêt »
#: core/admin.py:218
#, python-format
msgid "%(count)s recording(s) successfully marked as 'Failed to Stop'."
msgstr "%(count)s enregistrement(s) marqué(s) avec succès comme « Échec darrêt »."
msgstr ""
"%(count)s enregistrement(s) marqué(s) avec succès comme « Échec darrêt »."
#: core/admin.py:226
#, fuzzy, python-format
@@ -76,11 +77,11 @@ msgstr "%(count)s enregistrement(s) marqué(s) avec succès comme « Échec d
msgid "Skipped %(count)s recording(s) with an ineligible status."
msgstr "%(count)s enregistrement(s) avec un statut inéligible ignoré(s)."
#: core/admin.py:341
#: core/admin.py:342
msgid "No scopes"
msgstr "Aucun scopes"
#: core/admin.py:343
#: core/admin.py:344
msgid "Scopes"
msgstr "Scopes"
@@ -98,6 +99,10 @@ msgstr ""
msgid "This file extension is not allowed."
msgstr "Cette extension n'est pas autorisée"
#: core/api/serializers.py:533
msgid "You have reached the maximum number of files for this type."
msgstr "Vous avez atteint le nombre maximum de fichiers de ce type"
#: core/models.py:37
msgid "Member"
msgstr "Membre"
@@ -601,18 +606,18 @@ msgstr ""
" Si vous avez des questions ou besoin d'assistance, veuillez contacter notre "
"équipe d'assistance à %(support_email)s. "
#: meet/settings.py:215
#: meet/settings.py:223
msgid "English"
msgstr "Anglais"
#: meet/settings.py:216
#: meet/settings.py:224
msgid "French"
msgstr "Français"
#: meet/settings.py:217
#: meet/settings.py:225
msgid "Dutch"
msgstr "Néerlandais"
#: meet/settings.py:218
#: meet/settings.py:226
msgid "German"
msgstr "Allemand"
+11 -7
View File
@@ -8,7 +8,7 @@ msgid ""
msgstr ""
"Project-Id-Version: PACKAGE VERSION\n"
"Report-Msgid-Bugs-To: \n"
"POT-Creation-Date: 2026-03-11 17:31+0000\n"
"POT-Creation-Date: 2026-03-12 13:46+0000\n"
"PO-Revision-Date: YEAR-MO-DA HO:MI+ZONE\n"
"Last-Translator: FULL NAME <EMAIL@ADDRESS>\n"
"Language-Team: LANGUAGE <LL@li.org>\n"
@@ -76,11 +76,11 @@ msgstr "%(count)s opname(s) succesvol gemarkeerd als 'Mislukt bij stoppen'."
msgid "Skipped %(count)s recording(s) with an ineligible status."
msgstr "%(count)s opname(s) met een niet-toegestane status overgeslagen."
#: core/admin.py:341
#: core/admin.py:342
msgid "No scopes"
msgstr "Geen scopes"
#: core/admin.py:343
#: core/admin.py:344
msgid "Scopes"
msgstr "Scopes"
@@ -97,6 +97,10 @@ msgstr ""
msgid "This file extension is not allowed."
msgstr "Deze bestandsextensie is niet toegestaan."
#: core/api/serializers.py:533
msgid "You have reached the maximum number of files for this type."
msgstr "Het maximale aantal bestanden voor dit type is bereikt."
#: core/models.py:37
msgid "Member"
msgstr "Lid"
@@ -595,18 +599,18 @@ msgstr ""
" Als je vragen hebt of hulp nodig hebt, neem dan contact op met ons support "
"team via %(support_email)s. "
#: meet/settings.py:215
#: meet/settings.py:223
msgid "English"
msgstr "Engels"
#: meet/settings.py:216
#: meet/settings.py:224
msgid "French"
msgstr "Frans"
#: meet/settings.py:217
#: meet/settings.py:225
msgid "Dutch"
msgstr "Nederlands"
#: meet/settings.py:218
#: meet/settings.py:226
msgid "German"
msgstr "Duits"
+9
View File
@@ -176,6 +176,13 @@ class Base(Configuration):
environ_prefix=None,
)
FILE_UPLOAD_ENABLED = values.BooleanValue(
# False to avoid a breaking change for now
default=False,
environ_name="FILE_UPLOAD_ENABLED",
environ_prefix=None,
)
FILE_UPLOAD_PATH = values.Value(
"files", environ_name="FILE_UPLOAD_PATH", environ_prefix=None
)
@@ -188,6 +195,7 @@ class Base(Configuration):
{
"background_image": {
"max_size": 2 * MB,
"max_count_by_user": 10,
"allowed_extensions": [".jpeg", ".jpg", ".png"],
"allowed_mimetypes": ["image/jpeg", "image/png"],
},
@@ -973,6 +981,7 @@ class Test(Base):
APPLICATION_JWT_AUDIENCE = "Test inc."
CELERY_TASK_ALWAYS_EAGER = True
FILE_UPLOAD_ENABLED = True
def __init__(self):
# pylint: disable=invalid-name
@@ -187,7 +187,7 @@ export class BackgroundCustomProcessor implements BackgroundProcessorInterface {
0,
0,
PROCESSING_WIDTH,
PROCESSING_WIDTH
PROCESSING_HEIGHT
)
}
+2 -2
View File
@@ -112,7 +112,7 @@ spec:
/usr/bin/mc mb meet/meet-media-storage && \
exit 0
restartPolicy: Never
backoffLimit: 1
backoffLimit: 3
---
apiVersion: batch/v1
kind: Job
@@ -135,4 +135,4 @@ spec:
/usr/bin/mc event add meet/meet-media-storage arn:minio:sqs::meet-webhook:webhook --event put --prefix "recordings" && \
exit 0
restartPolicy: Never
backoffLimit: 1
backoffLimit: 3
+23 -11
View File
@@ -279,22 +279,28 @@ backend:
## @param backend.probes.liveness.path [nullable] Configure path for backend HTTP liveness probe
## @param backend.probes.liveness.targetPort [nullable] Configure port for backend HTTP liveness probe
## @param backend.probes.liveness.initialDelaySeconds [nullable] Configure initial delay for backend liveness probe
## @param backend.probes.liveness.initialDelaySeconds [nullable] Configure timeout for backend liveness probe
## @param backend.probes.liveness.periodSeconds [nullable] Configure period for backend liveness probe
## @param backend.probes.liveness.timeoutSeconds [nullable] Configure timeout for backend liveness probe
## @param backend.probes.startup.path [nullable] Configure path for backend HTTP startup probe
## @param backend.probes.startup.targetPort [nullable] Configure port for backend HTTP startup probe
## @param backend.probes.startup.initialDelaySeconds [nullable] Configure initial delay for backend startup probe
## @param backend.probes.startup.initialDelaySeconds [nullable] Configure timeout for backend startup probe
## @param backend.probes.startup.periodSeconds [nullable] Configure period for backend startup probe
## @param backend.probes.startup.timeoutSeconds [nullable] Configure timeout for backend startup probe
## @param backend.probes.readiness.path [nullable] Configure path for backend HTTP readiness probe
## @param backend.probes.readiness.targetPort [nullable] Configure port for backend HTTP readiness probe
## @param backend.probes.readiness.initialDelaySeconds [nullable] Configure initial delay for backend readiness probe
## @param backend.probes.readiness.initialDelaySeconds [nullable] Configure timeout for backend readiness probe
## @param backend.probes.readiness.periodSeconds [nullable] Configure period for backend readiness probe
## @param backend.probes.readiness.timeoutSeconds [nullable] Configure timeout for backend readiness probe
probes:
liveness:
path: /__heartbeat__
initialDelaySeconds: 30
initialDelaySeconds: 5
periodSeconds: 30
readiness:
path: /__lbheartbeat__
initialDelaySeconds: 30
initialDelaySeconds: 5
periodSeconds: 30
## @param backend.resources Resource requirements for the backend container
resources: {}
@@ -555,23 +561,29 @@ summary:
## @param summary.probes.liveness.path [nullable] Configure path for summary HTTP liveness probe
## @param summary.probes.liveness.targetPort [nullable] Configure port for summary HTTP liveness probe
## @param summary.probes.liveness.initialDelaySeconds [nullable] Configure initial delay for summary liveness probe
## @param summary.probes.liveness.initialDelaySeconds [nullable] Configure timeout for summary liveness probe
## @param summary.probes.liveness.initialDelaySeconds [nullable] Configure initial delay summary liveness probe
## @param summary.probes.liveness.periodSeconds [nullable] Configure the period for the summary liveness probe
## @param summary.probes.liveness.timeoutSeconds [nullable] Configure timeout for summary liveness probe
## @param summary.probes.startup.path [nullable] Configure path for summary HTTP startup probe
## @param summary.probes.startup.targetPort [nullable] Configure port for summary HTTP startup probe
## @param summary.probes.startup.initialDelaySeconds [nullable] Configure initial delay for summary startup probe
## @param summary.probes.startup.initialDelaySeconds [nullable] Configure timeout for summary startup probe
## @param summary.probes.startup.periodSeconds [nullable] Configure period for the summary startup probe
## @param summary.probes.startup.timeoutSeconds [nullable] Configure timeout for summary startup probe
## @param summary.probes.readiness.path [nullable] Configure path for summary HTTP readiness probe
## @param summary.probes.readiness.targetPort [nullable] Configure port for summary HTTP readiness probe
## @param summary.probes.readiness.initialDelaySeconds [nullable] Configure initial delay for summary readiness probe
## @param summary.probes.readiness.initialDelaySeconds [nullable] Configure timeout for summary readiness probe
## @param summary.probes.readiness.periodSeconds [nullable] Configure period for the summary readiness probe
## @param summary.probes.readiness.timeoutSeconds [nullable] Configure timeout for summary readiness probe
probes:
liveness:
path: /__heartbeat__
initialDelaySeconds: 30
initialDelaySeconds: 5
periodSeconds: 30
readiness:
path: /__lbheartbeat__
initialDelaySeconds: 30
initialDelaySeconds: 5
periodSeconds: 30
## @param summary.resources Resource requirements for the summary container
resources: {}
+11 -3
View File
@@ -21,8 +21,17 @@ dependencies = [
[project.optional-dependencies]
dev = [
"ruff==0.14.4",
"pytest==9.0.2",
"responses>=0.25.8",
]
[tool.pytest.ini_options]
markers = [
"unit: Test individual components",
"integration: Test the API",
]
testpaths = ["tests"]
[build-system]
requires = ["setuptools>=61.0"]
build-backend = "setuptools.build_meta"
@@ -49,11 +58,10 @@ select = [
"T20", # flake8-print
"W", # pycodestyle warning
]
ignore= ["DJ001", "PLR2004"]
[tool.ruff.lint.per-file-ignores]
"tests/*" = [
"S101", # use of assert
]
"tests/*" = ["S", "SLF"]
[tool.ruff.lint.pydocstyle]
# Use Google-style docstrings.
+4 -5
View File
@@ -12,8 +12,6 @@ from summary.core.celery_worker import (
)
from summary.core.config import get_settings
settings = get_settings()
class TranscribeSummarizeTaskCreation(BaseModel):
"""Transcription and summarization parameters."""
@@ -34,10 +32,11 @@ class TranscribeSummarizeTaskCreation(BaseModel):
@classmethod
def validate_language(cls, v):
"""Validate 'language' parameter."""
if v is not None and v not in settings.whisperx_allowed_languages:
allowed = get_settings().whisperx_allowed_languages
if v is not None and v not in allowed:
raise ValueError(
f"Language '{v}' is not allowed. "
f"Allowed languages: {', '.join(settings.whisperx_allowed_languages)}"
f"Allowed languages: {', '.join(allowed)}"
)
return v
@@ -62,7 +61,7 @@ async def create_transcribe_summarize_task(request: TranscribeSummarizeTaskCreat
request.download_link,
request.context_language,
],
queue=settings.transcribe_queue,
queue=get_settings().transcribe_queue,
)
return {"id": task.id, "message": "Task created"}
+3 -2
View File
@@ -12,7 +12,6 @@ from posthog import Posthog
from summary.core.config import get_settings
logger = get_task_logger(__name__)
settings = get_settings()
class AnalyticsException(Exception):
@@ -26,6 +25,7 @@ class Analytics:
def __init__(self):
"""Initialize a client if settings are configure."""
settings = get_settings()
self._client = None
if settings.posthog_api_key and settings.posthog_enabled:
logger.info("Initialize analytics client")
@@ -71,6 +71,7 @@ class MetadataManager:
def __init__(self):
"""Initialize the task tracker with analytics client."""
settings = get_settings()
self._redis = redis.from_url(settings.task_tracker_redis_url)
self._key_prefix = settings.task_tracker_prefix
self._analytics = get_analytics()
@@ -117,7 +118,7 @@ class MetadataManager:
start_time = time.time()
initial_metadata = {
"start_time": start_time,
"asr_model": settings.whisperx_asr_model,
"asr_model": get_settings().whisperx_asr_model,
"retries": 0,
"filename": filename,
"email": email,
+2 -2
View File
@@ -140,7 +140,7 @@ def format_transcript(
)
def format_actions(llm_output: dict) -> str:
def _format_actions(llm_output: dict) -> str:
"""Format the actions from the LLM output into a markdown list.
fomat:
@@ -328,7 +328,7 @@ def summarize_transcription(
response_format=FORMAT_NEXT_STEPS,
)
next_steps = format_actions(json.loads(next_steps))
next_steps = _format_actions(json.loads(next_steps))
logger.info("Next steps generated")
+29
View File
@@ -1,5 +1,6 @@
"""Application configuration and settings."""
import os
from functools import lru_cache
from typing import Annotated, List, Literal, Optional, Set
@@ -91,9 +92,37 @@ class Settings(BaseSettings):
task_tracker_prefix: str = "task_metadata:"
class TestSettings(Settings):
"""Settings with safe defaults for testing."""
model_config = SettingsConfigDict(env_file=None)
app_api_token: SecretStr = SecretStr("test-api-token")
aws_storage_bucket_name: str = "test-bucket"
aws_s3_endpoint_url: str = "http://localhost:9000"
aws_s3_access_key_id: str = "test-access-key"
aws_s3_secret_access_key: SecretStr = SecretStr("test-secret-key")
aws_s3_secure_access: bool = False
whisperx_api_key: SecretStr = SecretStr("test-whisperx-key")
whisperx_base_url: str = "http://localhost:8000/v1"
llm_base_url: str = "http://localhost:8001/v1"
llm_api_key: SecretStr = SecretStr("test-llm-key")
llm_model: str = "test-model"
webhook_api_token: SecretStr = SecretStr("test-webhook-token")
webhook_url: str = "http://localhost:8002/webhook"
celery_broker_url: str = "memory://"
celery_result_backend: str = "cache+memory://"
posthog_enabled: bool = False
sentry_is_enabled: bool = False
langfuse_enabled: bool = False
task_tracker_redis_url: str = "redis://localhost:6379/0"
@lru_cache
def get_settings():
"""Load and cache application settings."""
if os.environ.get("SUMMARY_ENV") == "test":
return TestSettings()
return Settings()
+9 -11
View File
@@ -13,9 +13,6 @@ from minio.error import MinioException, S3Error
from summary.core.config import get_settings
settings = get_settings()
logger = logging.getLogger(__name__)
@@ -31,23 +28,24 @@ class FileService:
def __init__(self):
"""Initialize FileService with MinIO client and configuration."""
endpoint = (
settings.aws_s3_endpoint_url.removeprefix("https://")
get_settings()
.aws_s3_endpoint_url.removeprefix("https://")
.removeprefix("http://")
.rstrip("/")
)
self._minio_client = Minio(
endpoint,
access_key=settings.aws_s3_access_key_id,
secret_key=settings.aws_s3_secret_access_key.get_secret_value(),
secure=settings.aws_s3_secure_access,
access_key=get_settings().aws_s3_access_key_id,
secret_key=get_settings().aws_s3_secret_access_key.get_secret_value(),
secure=get_settings().aws_s3_secure_access,
)
self._bucket_name = settings.aws_storage_bucket_name
self._bucket_name = get_settings().aws_storage_bucket_name
self._stream_chunk_size = 32 * 1024
self._allowed_extensions = settings.recording_allowed_extensions
self._max_duration = settings.recording_max_duration
self._allowed_extensions = get_settings().recording_allowed_extensions
self._max_duration = get_settings().recording_max_duration
def _download_from_minio(self, remote_object_key) -> Path:
"""Download file from MinIO to local temporary file.
@@ -174,7 +172,7 @@ class FileService:
extension = downloaded_path.suffix.lower()
if extension in settings.recording_video_extensions:
if extension in get_settings().recording_video_extensions:
logger.info("Video file detected, extracting audio...")
extracted_audio_path = self._extract_audio_from_video(downloaded_path)
processed_path = extracted_audio_path
+10 -13
View File
@@ -8,9 +8,6 @@ from langfuse import Langfuse
from summary.core.config import get_settings
settings = get_settings()
logger = logging.getLogger(__name__)
@@ -34,28 +31,28 @@ class LLMObservability:
self.session_id = session_id
self.user_id = user_id
if settings.langfuse_enabled:
if get_settings().langfuse_enabled:
def masking_function(data, **kwargs):
if (
user_has_tracing_consent
or settings.langfuse_environment != "production"
or get_settings().langfuse_environment != "production"
):
return data
return "[REDACTED]"
if not settings.langfuse_secret_key:
if not get_settings().langfuse_secret_key:
raise ValueError(
"langfuse_secret_key is not configured. "
"Please set the secret key or disable Langfuse."
)
self._observability_client = Langfuse(
secret_key=settings.langfuse_secret_key.get_secret_value(),
public_key=settings.langfuse_public_key,
host=settings.langfuse_host,
environment=settings.langfuse_environment,
secret_key=get_settings().langfuse_secret_key.get_secret_value(),
public_key=get_settings().langfuse_public_key,
host=get_settings().langfuse_host,
environment=get_settings().langfuse_environment,
mask=masking_function,
)
@@ -72,8 +69,8 @@ class LLMObservability:
to Langfuse for observability when enabled.
"""
base_args = {
"base_url": settings.llm_base_url,
"api_key": settings.llm_api_key.get_secret_value(),
"base_url": get_settings().llm_base_url,
"api_key": get_settings().llm_api_key.get_secret_value(),
}
if not self.is_enabled:
@@ -120,7 +117,7 @@ class LLMService:
"""
try:
params: dict[str, Any] = {
"model": settings.llm_model,
"model": get_settings().llm_model,
"messages": [
{"role": "system", "content": system_prompt},
{"role": "user", "content": user_prompt},
@@ -6,8 +6,6 @@ from typing import Optional, Tuple
from summary.core.config import get_settings
from summary.core.locales import LocaleStrings
settings = get_settings()
logger = logging.getLogger(__name__)
@@ -23,7 +21,7 @@ class TranscriptFormatter:
def __init__(self, locale: LocaleStrings):
"""Initialize formatter with settings and locale."""
self.hallucination_patterns = settings.hallucination_patterns
self.hallucination_patterns = get_settings().hallucination_patterns
self._locale = locale
def _get_segments(self, transcription):
+8 -8
View File
@@ -9,8 +9,6 @@ from urllib3.util import Retry
from summary.core.config import get_settings
settings = get_settings()
logger = logging.getLogger(__name__)
@@ -18,9 +16,9 @@ def _create_retry_session():
"""Create an HTTP session configured with retry logic."""
session = Session()
retries = Retry(
total=settings.webhook_max_retries,
backoff_factor=settings.webhook_backoff_factor,
status_forcelist=settings.webhook_status_forcelist,
total=get_settings().webhook_max_retries,
backoff_factor=get_settings().webhook_backoff_factor,
status_forcelist=get_settings().webhook_status_forcelist,
allowed_methods={"POST"},
)
session.mount("https://", HTTPAdapter(max_retries=retries))
@@ -31,7 +29,9 @@ def _post_with_retries(url, data):
"""Send POST request with automatic retries."""
session = _create_retry_session()
session.headers.update(
{"Authorization": f"Bearer {settings.webhook_api_token.get_secret_value()}"}
{
"Authorization": f"Bearer {get_settings().webhook_api_token.get_secret_value()}" # noqa: E501
}
)
try:
response = session.post(url, json=data)
@@ -53,10 +53,10 @@ def submit_content(content, title, email, sub):
"sub": sub,
}
logger.debug("Submitting to %s", settings.webhook_url)
logger.debug("Submitting to %s", get_settings().webhook_url)
logger.debug("Request payload: %s", json.dumps(data, indent=2))
response = _post_with_retries(settings.webhook_url, data)
response = _post_with_retries(get_settings().webhook_url, data)
try:
response_data = response.json()
+1
View File
@@ -0,0 +1 @@
"""Tests for the summary service."""
+7
View File
@@ -0,0 +1,7 @@
"""Shared test fixtures and environment setup for the summary service tests."""
import os
# Activate TestSettings (safe defaults for all required env vars)
# before any summary module is imported.
os.environ["SUMMARY_ENV"] = "test"
@@ -0,0 +1 @@
"""Integration tests for the summary service."""
+23
View File
@@ -0,0 +1,23 @@
"""Integration test configuration. Provides shared fixtures."""
import pytest
from fastapi.testclient import TestClient
from summary.core.celery_worker import celery
from summary.main import app
@pytest.fixture()
def client():
"""Provide a FastAPI TestClient for integration tests."""
return TestClient(app)
@pytest.fixture()
def eager_celery():
"""Run Celery tasks synchronously in the same process."""
celery.conf.task_always_eager = True
celery.conf.task_eager_propagates = True
yield
celery.conf.task_always_eager = False
celery.conf.task_eager_propagates = False
@@ -0,0 +1,21 @@
"""Integration tests for the health check endpoints."""
class TestHeartbeat:
"""Tests for the /__heartbeat__ endpoint."""
def test_returns_200(self, client):
"""The heartbeat endpoint responds with 200 OK without a token."""
response = client.get("/__heartbeat__")
assert response.status_code == 200
class TestLBHeartbeat:
"""Tests for the /__lbheartbeat__ endpoint."""
def test_returns_200(self, client):
"""The load-balancer heartbeat endpoint responds with 200 OK without a token."""
response = client.get("/__lbheartbeat__")
assert response.status_code == 200
@@ -0,0 +1,130 @@
"""Integration test for the transcribe-and-summarize task flow via the API."""
import json
from contextlib import contextmanager
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import responses
from summary.core.config import get_settings
API_PREFIX = "/api/v1"
AUTH_HEADER = {
"Authorization": f"Bearer {get_settings().app_api_token.get_secret_value()}"
}
WEBHOOK_URL = get_settings().webhook_url
class TestTranscribeSummarizeFlow:
"""End-to-end test: POST /tasks/ triggers transcription and summary via webhook."""
@responses.activate
@patch("summary.core.celery_worker.analytics")
@patch("summary.core.celery_worker.LLMObservability")
@patch("summary.core.celery_worker.LLMService")
@patch("summary.core.celery_worker.metadata_manager")
@patch("summary.core.celery_worker.openai")
@patch("summary.core.celery_worker.file_service")
def test_transcription_and_summary_are_submitted( # noqa: PLR0913
self,
mock_file_service,
mock_openai,
mock_metadata,
mock_llm_cls,
mock_observability_cls,
mock_analytics,
client,
eager_celery,
):
"""Creating a task produces a transcription and summary sent to the webhook."""
# Stub file service
fake_audio = MagicMock()
@contextmanager
def fake_prepare(filename):
yield fake_audio, {"duration": 60.0}
mock_file_service.prepare_audio_file = fake_prepare
# Stub WhisperX transcription
fake_transcription = SimpleNamespace(
segments=[
{"speaker": "SPEAKER_00", "text": "Hello everyone."},
{"speaker": "SPEAKER_01", "text": "Let's discuss the roadmap."},
],
)
mock_client = MagicMock()
mock_client.audio.transcriptions.create.return_value = fake_transcription
mock_openai.OpenAI.return_value = mock_client
# Stub analytics to enable summary
mock_analytics.is_feature_enabled.return_value = True
# Stub LLM for summarization
mock_llm = MagicMock()
mock_llm_cls.return_value = mock_llm
plan_json = json.dumps({"titles": ["Roadmap"]})
next_steps_json = json.dumps(
{
"actions": [
{
"title": "Draft roadmap",
"assignees": ["Aleb"],
"due_date": "2026-03-04",
}
]
}
)
mock_llm.call.side_effect = [
"### TL;DR\nQuick overview.", # tldr
plan_json, # parts plan
"### Roadmap\nDetails.", # part content
next_steps_json, # next steps
"Cleaned summary.", # cleaning
]
mock_observability_cls.return_value = MagicMock()
# Stub webhook (called twice: transcription + summary)
responses.post(WEBHOOK_URL, json={"id": "doc-1"}, status=200)
responses.post(WEBHOOK_URL, json={"id": "doc-2"}, status=200)
payload = {
"owner_id": "owner-1",
"filename": "recording.webm",
"email": "user@example.com",
"sub": "user-sub-id",
"room": "Visio room",
"recording_date": "2026-03-04",
"recording_time": "09:00",
"language": "en",
"download_link": "https://example.com/rec.webm",
"context_language": "en",
}
response = client.post(
f"{API_PREFIX}/tasks/", json=payload, headers=AUTH_HEADER
)
assert response.status_code == 200
body = response.json()
assert "id" in body
assert body["message"] == "Task created"
# Verify the webhook received the transcription
assert len(responses.calls) >= 1
transcript_payload = json.loads(responses.calls[0].request.body)
assert "SPEAKER_00" in transcript_payload["content"]
assert "Hello everyone." in transcript_payload["content"]
assert "Visio room" in transcript_payload["title"]
assert transcript_payload["email"] == "user@example.com"
assert transcript_payload["sub"] == "user-sub-id"
# Verify the webhook received the summary
assert len(responses.calls) == 2
summary_payload = json.loads(responses.calls[1].request.body)
assert "TL;DR" in summary_payload["content"]
assert "Cleaned summary." in summary_payload["content"]
assert "Draft roadmap" in summary_payload["content"]
+1
View File
@@ -0,0 +1 @@
"""Unit tests for the summary service."""
@@ -0,0 +1,249 @@
"""Tests for the celery_worker module."""
import json
from contextlib import contextmanager
from types import SimpleNamespace
from unittest.mock import MagicMock, patch
import pytest
import responses
from summary.core.celery_worker import (
format_transcript,
summarize_transcription,
transcribe_audio,
)
from summary.core.config import get_settings
from summary.core.file_service import FileServiceException
WEBHOOK_URL = get_settings().webhook_url
# ---------------------------------------------------------------------------
# transcribe_audio
# ---------------------------------------------------------------------------
class TestTranscribeAudio:
"""Tests for the transcribe_audio function."""
@patch("summary.core.celery_worker.metadata_manager")
@patch("summary.core.celery_worker.openai")
@patch("summary.core.celery_worker.file_service")
def test_success(self, mock_file_service, mock_openai, mock_metadata):
"""Transcription succeeds and returns the transcription object."""
fake_audio = MagicMock()
fake_metadata = {"duration": 120.5}
@contextmanager
def fake_prepare(filename):
yield fake_audio, fake_metadata
mock_file_service.prepare_audio_file = fake_prepare
fake_transcription = SimpleNamespace(
segments=[{"speaker": "SPEAKER_00", "text": "Hello"}],
)
mock_client = MagicMock()
mock_client.audio.transcriptions.create.return_value = fake_transcription
mock_openai.OpenAI.return_value = mock_client
result = transcribe_audio("task-1", "recording.ogg", "en")
assert result is fake_transcription
mock_client.audio.transcriptions.create.assert_called_once()
call_kwargs = mock_client.audio.transcriptions.create.call_args
assert call_kwargs.kwargs["language"] == "en"
assert call_kwargs.kwargs["file"] is fake_audio
@patch("summary.core.celery_worker.metadata_manager")
@patch("summary.core.celery_worker.openai")
@patch("summary.core.celery_worker.file_service")
def test_file_service_error_returns_none(
self, mock_file_service, mock_openai, mock_metadata
):
"""Returns None when the file cannot be retrieved."""
@contextmanager
def failing_prepare(filename):
raise FileServiceException("download failed")
yield # NOSONAR - yield required for contextmanager
mock_file_service.prepare_audio_file = failing_prepare
result = transcribe_audio("task-1", "recording.ogg", "en")
assert result is None
mock_openai.OpenAI.return_value.audio.transcriptions.create.assert_not_called()
# ---------------------------------------------------------------------------
# format_transcript
# ---------------------------------------------------------------------------
class TestFormatTranscript:
"""Tests for the format_transcript function."""
def test_with_segments(self):
"""Formats a transcription with segments into content and title."""
transcription = {
"segments": [
{"speaker": "SPEAKER_00", "text": "Hello everyone."},
{"speaker": "SPEAKER_01", "text": "Good morning."},
],
}
content, title = format_transcript(
transcription,
context_language="en",
language="en",
room="Daily standup",
recording_date="2026-03-04",
recording_time="09:00",
download_link="https://example.com/rec.ogg",
)
assert "SPEAKER_00" in content
assert "Hello everyone." in content
assert "SPEAKER_01" in content
assert "Good morning." in content
assert "Daily standup" in title
assert "2026-03-04" in title
assert "09:00" in title
@pytest.mark.parametrize(
"context_language, expected_string",
[
("en", "Download your recording"),
("fr", "Télécharger votre enregistrement"),
("de", "diesem Link folgen"),
("nl", "Download uw opname door"),
],
)
def test_context_language(self, context_language, expected_string):
"""Context language parameter modifies output."""
transcription = {
"segments": [
{"speaker": "SPEAKER_00", "text": "Hello everyone."},
],
}
content, _ = format_transcript(
transcription,
context_language=context_language,
language="en",
room="Daily standup",
recording_date="2026-03-04",
recording_time="09:00",
download_link="https://example.com/rec.ogg",
)
assert expected_string in content
def test_empty_segments(self):
"""Returns empty-transcription message when there are no segments."""
transcription = {"segments": []}
content, title = format_transcript(
transcription,
context_language="en",
language="en",
room=None,
recording_date=None,
recording_time=None,
download_link=None,
)
assert "No audio content" in content or "Transcription" in title
# ---------------------------------------------------------------------------
# summarize_transcription
# ---------------------------------------------------------------------------
class TestSummarizeTranscription:
"""Tests for the summarize_transcription Celery task."""
@responses.activate
@patch("summary.core.celery_worker.LLMService")
@patch("summary.core.celery_worker.LLMObservability")
@patch("summary.core.celery_worker.analytics")
def test_generates_and_submits_summary(
self, mock_analytics, mock_observability_cls, mock_llm_cls
):
"""Assembles TLDR + parts + next steps + cleaning, then submits."""
mock_analytics.is_feature_enabled.return_value = False
# Mock the webhook HTTP endpoint
responses.post(
WEBHOOK_URL,
json={"id": "doc-42"},
status=200,
)
mock_llm = MagicMock()
mock_llm_cls.return_value = mock_llm
plan_json = json.dumps({"titles": ["Topic A", "Topic B"]})
next_steps_json = json.dumps(
{
"actions": [
{
"title": "What's nice about Visio",
"assignees": ["Aleb"],
"due_date": "2026-03-04",
}
]
}
)
# LLM calls in order: tldr, parts (plan), part A, part B, next-steps, cleaning
mock_llm.call.side_effect = [
"### TL;DR\nShort summary.", # tldr
plan_json, # parts plan
"### Topic A\nDetails about A.", # part A
"### Topic B\nDetails about B.", # part B
next_steps_json, # next steps
"Cleaned summary content.", # cleaning
]
mock_observability = MagicMock()
mock_observability_cls.return_value = mock_observability
# Push a fake request context so self.request.id is available
summarize_transcription.push_request(id="summary-task-1")
try:
summarize_transcription.run(
"owner-1",
"Full transcript text",
"user@example.com",
"oidc-sub-123",
"99.999% uptime. Is it reasonable ?",
)
finally:
summarize_transcription.pop_request()
# Verify the webhook was called with the assembled summary
assert len(responses.calls) == 1
webhook_request = responses.calls[0]
submitted_payload = json.loads(webhook_request.request.body)
assert "TL;DR" in submitted_payload["content"]
assert "Cleaned summary content." in submitted_payload["content"]
assert "What's nice about Visio" in submitted_payload["content"]
assert "99.999% uptime. Is it reasonable ?" in submitted_payload["title"]
assert submitted_payload["email"] == "user@example.com"
assert submitted_payload["sub"] == "oidc-sub-123"
# Verify auth header was sent
assert (
webhook_request.request.headers["Authorization"]
== f"Bearer {get_settings().webhook_api_token.get_secret_value()}"
)
# LLM was called for: tldr, plan, part A, part B, next-steps, cleaning
expected_llm_calls = 6
assert mock_llm.call.call_count == expected_llm_calls
mock_observability.flush.assert_called_once()