Compare commits

...

14 Commits

Author SHA1 Message Date
lebaudantoine 73a7841b96 🔖(minor) bump release to 1.18.0 2026-06-03 14:46:26 +02:00
Florent Chehab 6c25d0a525 🐛(backend) update alter file state migration predecessor
Last PR was merged a bit too quicly and another migration
was added before the PR one.
2026-06-03 14:18:58 +02:00
Florent Chehab d13e3a8a5d ️(backend) change db state instead of using long running row lock
When analysing a file, the previous commit introduced a row level lock
to make sure we would analyse and promote a single file.
This commit changes the locking mechanism so that it happens with
the upload state which avoids long running db locks and potential
perf issues.
2026-06-03 13:45:32 +02:00
Florent Chehab a82d7f885a 🔒️(backend) prevnt file change post checks
Before this commit, post a file check, the policy could be
reused to change the verified file.
Now, files are uplaoded to a temporary location, then inside
a transaction that prevents concurrent calls, the file is
copied to its final destination and the checks are run on that one.
A new file can still be updated with the policy but it will never be read, etc.

As part of this change, all files in the new tmp directory
on s3 should have an expiration policy.
2026-06-03 13:45:31 +02:00
lebaudantoine 5ef6e8b5ea 🔖(helm) release chart 0.0.22 2026-06-03 13:25:25 +02:00
lebaudantoine 913d4f91ae 👷(helm) add Kubernetes job for duplicate user merge command
Introduce a Kubernetes job to run the `merge_duplicate_users`
management command on the backend, as suggested by @rouja.

This provides a standard and reproducible way to execute the user merge
process in deployed environments without requiring manual access to
application containers.

The job can be used alongside the command's dry-run mode to validate
the impact of a merge operation before applying changes.
2026-06-03 13:19:26 +02:00
lebaudantoine 1f437089ad 📝(helm) correct chart value docstring for createsuperuser
Update the chart value docstring to reference the `createsuperuser`
command instead of `migrate`.
2026-06-02 21:55:46 +02:00
lebaudantoine 85eff8afaf 🧑‍💻(backend) add email filter to target subset of users for merge cmd
Add an email substring filter to the user merge management command,
allowing selection of a subset of users concerned by the merge process.

This enables safer incremental execution of the command by testing it
on a controlled group of users before applying it globally.

The goal is to validate behavior and ensure the merge process does not
introduce unexpected side effects or inconsistencies at scale.
2026-06-02 21:55:46 +02:00
lebaudantoine 29b0a6fcb4 🧑‍💻(backend) add management command to merge duplicate users
Add a management command to merge duplicate users and reassign all
granted resources to the most recent user account in the database.

Support a dry-run mode to estimate the impact of the operation before
applying any changes. This helps validate the command and identify
potential issues in environments where realistic testing is difficult.

Add unit tests to verify the command behavior and ensure database
integrity is preserved during the merge process.
2026-06-02 21:55:46 +02:00
lebaudantoine 3554b2eb53 ♻️(backend) defend user provisioning against race condition
Move user provisioning logic out of the external token viewset into a
dedicated service to keep the viewset lightweight and easier to
maintain.

While extracting the logic, refactor user object handling to improve
robustness and make the provisioning workflow easier to reason about.

Defend against race conditions when concurrent requests attempt to
provision the same user. Rely on the existing database constraints to
guarantee uniqueness and gracefully handle integrity errors raised by
concurrent creations.
2026-06-02 20:13:16 +02:00
lebaudantoine e25aa6ce05 (backend) add test coverage for blank sub behavior
Update the test to document the actual contract without modifying the
underlying model behavior.

These tests act as non-regression coverage and explicitly assert that
users may have a null sub or email. They are intended to document the
current behavior of the initial user model rather than evolve or
constrain it.

Ghost rows have not been reported as an operational issue. For the sake
of simplicity, avoid changing the model unless required by a concrete
issue or unless the benefits clearly outweigh the added complexity.
2026-06-02 20:13:16 +02:00
lebaudantoine a8b79740e9 🐛(backend) prevent duplicate pending users on concurrent requests
Fix a race condition in the external viewset where concurrent requests
could create multiple pending users with the same email address.

No database constraint enforced email uniqueness for pending users,
allowing duplicates to be created under load.

This caused issues during user reconciliation, which expects a single
matching pending user and raises a warning when multiple records are
found.

Add a narrowly scoped migration to enforce the uniqueness constraint
and address the identified issue.
2026-06-02 20:13:16 +02:00
lebaudantoine 28f652e035 🔧(backend) backport logging configuration from docs
Backport the logging configuration from docs to LaSuite Meet after
discussion with @lunika.

Add a proper base logging setup to restore usable logs in production.
The initial repository bootstrap lacked a complete logging
configuration, resulting in limited operational visibility.
2026-06-02 14:27:58 +02:00
lebaudantoine a4997e7431 🔖(minor) bump release to 1.17.0 2026-05-31 18:18:00 +02:00
33 changed files with 1351 additions and 138 deletions
+17 -1
View File
@@ -8,6 +8,21 @@ and this project adheres to
## [Unreleased]
## [1.18.0] - 2026-06-03
### Added
- 🔧(backend) backport logging configuration from docs
- 🧑‍💻(backend) add management command to merge duplicate users
- 👷(helm) add Kubernetes job for duplicate user merge command
### Fixed
- 🐛(backend) prevent duplicate pending users on concurrent requests
- 🔒️(backend) prevent file change post checks #1377
## [1.17.0] - 2026-05-31
### Added
- ✨(fullstack) allow participants to mute others based on room configuration
@@ -18,7 +33,7 @@ and this project adheres to
- ✨(backend) add core.recording.event.parsers.S3Parser
- ✨(summary) extended support for all video / audio files #1358
### Changed
### Changed
- ♻️(fullstack) simplify source serialization
- ✨(backend) expose room configuration to all API consumers
@@ -28,6 +43,7 @@ and this project adheres to
- ♻️(backend) prefix Swagger routes with /api
### Fixed
- 🩹(backend) fix swagger and redoc documentation URLs
## [1.16.0] - 2026-05-13
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "agents"
version = "1.16.0"
version = "1.18.0"
requires-python = ">=3.12"
dependencies = [
"livekit-agents==1.4.5",
+1 -1
View File
@@ -9,7 +9,7 @@ resolution-markers = [
[[package]]
name = "agents"
version = "1.16.0"
version = "1.18.0"
source = { virtual = "." }
dependencies = [
{ name = "livekit-agents" },
+101 -73
View File
@@ -1196,96 +1196,124 @@ class FileViewSet(
"""
Check the actual uploaded file and mark it as ready.
"""
# Ensures we go through authorization checks
file = self.get_object()
if not file.is_pending_upload:
# Try to update the file with the new state. If the file is already in this state
# we are in a concurrent request, and we should reject that request
updated_rows = models.File.objects.filter(
upload_state=models.FileUploadStateChoices.PENDING,
pk=kwargs["pk"],
).update(upload_state=models.FileUploadStateChoices.ANALYZING)
if updated_rows != 1:
raise drf_exceptions.ValidationError(
{"file": "This action is only available for files in PENDING state."},
code="file_upload_state_not_pending",
)
file.refresh_from_db()
s3_client = default_storage.connection.meta.client
validation_error = None
head_response = s3_client.head_object(
Bucket=default_storage.bucket_name, Key=file.file_key
)
file_size = head_response["ContentLength"]
if settings.FILE_UPLOAD_APPLY_RESTRICTIONS:
config_for_file_type = settings.FILE_UPLOAD_RESTRICTIONS[file.type]
if file_size > config_for_file_type["max_size"]:
self._complete_file_deletion(file)
logger.info(
"upload_ended: file size (%s) for file %s higher than the allowed max size",
file_size,
file.file_key,
)
raise drf_exceptions.ValidationError(
detail="The file size is higher than the allowed max size.",
code="file_size_exceeded",
)
# python-magic recommends using at least the first 2048 bytes
# to reduce incorrect identification.
# This is a tradeoff between pulling in the whole file and the most likely relevant bytes
# of the file for mime type identification.
if file_size > 2048:
range_response = s3_client.get_object(
Bucket=default_storage.bucket_name,
Key=file.file_key,
Range="bytes=0-2047",
)
file_head = range_response["Body"].read()
else:
file_head = s3_client.get_object(
Bucket=default_storage.bucket_name, Key=file.file_key
)["Body"].read()
# Use improved MIME type detection combining magic bytes and file extension
logger.info("upload_ended: detecting mimetype for file: %s", file.file_key)
mimetype = utils.detect_mimetype(file_head, filename=file.filename)
if settings.FILE_UPLOAD_APPLY_RESTRICTIONS:
config_for_file_type = settings.FILE_UPLOAD_RESTRICTIONS[file.type]
allowed_file_mimetypes = config_for_file_type["allowed_mimetypes"]
if mimetype not in allowed_file_mimetypes:
self._complete_file_deletion(file)
logger.warning(
"upload_ended: mimetype not allowed %s for file %s",
mimetype,
file.file_key,
)
raise drf_exceptions.ValidationError(
detail="The file type is not allowed.",
code="file_type_not_allowed",
)
file.upload_state = models.FileUploadStateChoices.READY
file.mimetype = mimetype
file.size = file_size
file.save(update_fields=["upload_state", "mimetype", "size"])
if head_response["ContentType"] != mimetype:
logger.info(
"upload_ended: content type mismatch between object storage and file,"
" updating from %s to %s",
head_response["ContentType"],
mimetype,
)
try:
# We copy the file to its final destination, we will run the checks on that
# final file and ignore any updates to the temporary file. (We cannot revoke the policy,
# so the temporary file might still be updated after that.)
# The temporary folders will need to be cleaned periodically
s3_client.copy_object(
Bucket=default_storage.bucket_name,
Key=file.file_key,
CopySource={
"Bucket": default_storage.bucket_name,
"Key": file.file_key,
"Key": file.temporary_file_key,
},
ContentType=mimetype,
Metadata=head_response["Metadata"],
MetadataDirective="REPLACE",
)
head_response = s3_client.head_object(
Bucket=default_storage.bucket_name, Key=file.file_key
)
file_size = head_response["ContentLength"]
# python-magic recommends using at least the first 2048 bytes
# to reduce incorrect identification.
# This is a tradeoff between pulling in the whole file and
# the most likely relevant bytes
# of the file for mime type identification.
if file_size > 2048:
range_response = s3_client.get_object(
Bucket=default_storage.bucket_name,
Key=file.file_key,
Range="bytes=0-2047",
)
file_head = range_response["Body"].read()
else:
file_head = s3_client.get_object(
Bucket=default_storage.bucket_name, Key=file.file_key
)["Body"].read()
logger.info("upload_ended: detecting mimetype for file: %s", file.file_key)
mimetype = utils.detect_mimetype(file_head, filename=file.filename)
if settings.FILE_UPLOAD_APPLY_RESTRICTIONS:
config_for_file_type = settings.FILE_UPLOAD_RESTRICTIONS[file.type]
if file_size > config_for_file_type["max_size"]:
logger.info(
"upload_ended: file size (%s) for file %s higher than the allowed max size",
file_size,
file.file_key,
)
validation_error = drf_exceptions.ValidationError(
detail="The file size is higher than the allowed max size.",
code="file_size_exceeded",
)
else:
# Use improved MIME type detection combining magic bytes and file extension
allowed_file_mimetypes = config_for_file_type["allowed_mimetypes"]
if mimetype not in allowed_file_mimetypes:
logger.warning(
"upload_ended: mimetype not allowed %s for file %s",
mimetype,
file.file_key,
)
validation_error = drf_exceptions.ValidationError(
detail="The file type is not allowed.",
code="file_type_not_allowed",
)
if validation_error is not None:
self._complete_file_deletion(file)
else:
file.upload_state = models.FileUploadStateChoices.READY
file.mimetype = mimetype
file.size = file_size
file.save(update_fields=["upload_state", "mimetype", "size"])
if head_response["ContentType"] != mimetype:
logger.info(
"upload_ended: content type mismatch between object storage and file,"
" updating from %s to %s",
head_response["ContentType"],
mimetype,
)
s3_client.copy_object(
Bucket=default_storage.bucket_name,
Key=file.file_key,
CopySource={
"Bucket": default_storage.bucket_name,
"Key": file.file_key,
},
ContentType=mimetype,
Metadata=head_response["Metadata"],
MetadataDirective="REPLACE",
)
except Exception as e:
logger.exception("Failed to analyze file, reverting to pending state")
file.upload_state = models.FileUploadStateChoices.PENDING
file.save()
raise e
if validation_error:
raise validation_error
# Not yet implemented
# Change the file.upload_state when this will be done
# malware_detection.analyse_file(file.file_key, file_id=file.id)
+14 -35
View File
@@ -4,7 +4,7 @@ from logging import getLogger
from django.conf import settings
from django.contrib.auth.hashers import check_password
from django.core.exceptions import SuspiciousOperation, ValidationError
from django.core.exceptions import ValidationError
from django.core.validators import validate_email
from lasuite.oidc_resource_server.authentication import ResourceServerAuthentication
@@ -23,6 +23,11 @@ from core import api, models
from core.api.feature_flag import FeatureFlag
from core.services.jwt_token import JwtTokenService
from ..services.provisional_user_service import (
ProvisionalUserCreationDisabledError,
ProvisionalUserIntegrityError,
ProvisionalUserService,
)
from . import authentication, permissions, serializers
logger = getLogger(__name__)
@@ -94,40 +99,14 @@ class ApplicationViewSet(viewsets.ViewSet):
)
try:
user = models.User.objects.get(email__iexact=email)
except models.User.DoesNotExist as e:
if (
settings.APPLICATION_ALLOW_USER_CREATION
and settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION
and not settings.OIDC_USER_SUB_FIELD_IMMUTABLE
):
# Create a provisional user without `sub`, identified by email only.
#
# This relies on Django LaSuite implicitly updating the `sub` field on the
# user's first successful OIDC authentication. If this stops working,
# check for behavior changes in Django LaSuite.
#
# `OIDC_USER_SUB_FIELD_IMMUTABLE` comes from Django LaSuite and prevents `sub`
# updates. We override its default value to allow setting `sub` for
# provisional users.
user = models.User(
sub=None,
email=email,
)
user.set_unusable_password()
user.save()
logger.info(
"Provisional user created via application: user_id=%s, email=%s, client_id=%s",
user.id,
email,
application.client_id,
)
else:
raise drf_exceptions.NotFound("User not found.") from e
except models.User.MultipleObjectsReturned as e:
raise SuspiciousOperation(
"Multiple user accounts share a common email."
) from e
user, _ = ProvisionalUserService().get_or_create(email, client_id)
except ProvisionalUserCreationDisabledError as not_found_error:
raise drf_exceptions.NotFound("User not found.") from not_found_error
except ProvisionalUserIntegrityError:
return drf_response.Response(
{"error": "Failed to create or retrieve provisional user."},
status=drf_status.HTTP_409_CONFLICT,
)
scope = " ".join(application.scopes or [])
@@ -0,0 +1,168 @@
"""Management command to merge duplicate users based on their email address."""
# pylint: disable=too-many-locals
from django.contrib.auth import get_user_model
from django.core.management.base import BaseCommand, CommandError
from django.db import transaction
from django.db.models import Count
from core.models import File, RecordingAccess, ResourceAccess, RoleChoices
User = get_user_model()
ROLE_PRIORITY = {
RoleChoices.OWNER: 3,
RoleChoices.ADMIN: 2,
RoleChoices.MEMBER: 1,
}
class Command(BaseCommand):
"""
Merge duplicate users sharing the same email into the most recently created one.
The KEPT user is the most recently created. All room memberships, recording
accesses and files are transferred to it. When a conflict exists, the
higher-privilege role wins. Stale users are then deleted.
Each email group is processed inside a single database transaction.
"""
help = __doc__
def add_arguments(self, parser):
parser.add_argument(
"--dry-run",
action="store_true",
help="Simulate the merge without writing any changes to the database.",
)
parser.add_argument(
"--email-filter",
type=str,
default=None,
help="Only merge users whose email contains this string (e.g. '@example.com').",
)
def handle(self, *args, **options):
"""Execute the management command."""
dry_run = options["dry_run"]
email_filter = options["email_filter"]
if dry_run:
self.stdout.write("[DRY-RUN] No changes will be written.\n")
users_qs = User.objects.all()
if email_filter:
users_qs = users_qs.filter(email__icontains=email_filter)
self.stdout.write(f"[INFO] Filtering emails containing '{email_filter}'.\n")
duplicate_emails = (
users_qs.exclude(email__isnull=True)
.exclude(email="")
.values("email")
.annotate(cnt=Count("id"))
.filter(cnt__gt=1)
.values_list("email", flat=True)
)
if not duplicate_emails:
self.stdout.write("[INFO] No duplicate users found. Nothing to do.")
return
self.stdout.write(
f"[INFO] Found {len(duplicate_emails)} email(s) with duplicate users."
)
total_merged = 0
total_deleted = 0
failed_emails = []
for email in duplicate_emails:
# Secondary sort by id ensures a stable, deterministic order when
# created_at timestamps are equal (common in tests and bulk imports).
users = list(User.objects.filter(email=email).order_by("created_at", "id"))
kept_user = users[-1]
stale_users = users[:-1]
self.stdout.write(
f"\n[INFO] Email '{email}': {len(users)} users — "
f"keeping {kept_user.id} (created {kept_user.created_at.date()})."
)
for u in stale_users:
self.stdout.write(
f" stale: {u.id} (created {u.created_at.date()})"
)
if dry_run:
ra_count = ResourceAccess.objects.filter(user__in=stale_users).count()
rca_count = RecordingAccess.objects.filter(user__in=stale_users).count()
f_count = File.objects.filter(creator__in=stale_users).count()
self.stdout.write(
f" [DRY-RUN] Would migrate: {ra_count} ResourceAccess, "
f"{rca_count} RecordingAccess, {f_count} File(s)."
)
continue
try:
group_deleted = 0
with transaction.atomic():
for stale_user in stale_users:
self._merge_resource_accesses(stale_user, kept_user)
self._merge_recording_accesses(stale_user, kept_user)
self._merge_files(stale_user, kept_user)
stale_user.delete()
group_deleted += 1
total_deleted += group_deleted
total_merged += 1
except Exception as exc: # noqa: BLE001 #pylint: disable=broad-exception-caught
failed_emails.append(email)
self.stderr.write(f"[ERROR] Failed to merge '{email}': {exc}")
if failed_emails:
raise CommandError(
f"Failed to merge {len(failed_emails)} email group(s): {', '.join(failed_emails)}"
)
self.stdout.write(
self.style.SUCCESS(
f"\n[DONE] Merged {total_merged} group(s), deleted {total_deleted} user(s)."
)
)
def _merge_resource_accesses(self, stale_user, kept_user):
"""Transfer room memberships from stale_user to kept_user."""
for ra in ResourceAccess.objects.filter(user=stale_user):
existing = ResourceAccess.objects.filter(
user=kept_user, resource=ra.resource
).first()
if existing is None:
ra.user = kept_user
ra.save(update_fields=["user"])
else:
if ROLE_PRIORITY.get(ra.role, 0) > ROLE_PRIORITY.get(existing.role, 0):
existing.role = ra.role
existing.save(update_fields=["role"])
ra.delete()
def _merge_recording_accesses(self, stale_user, kept_user):
"""Transfer recording accesses from stale_user to kept_user."""
for rca in RecordingAccess.objects.filter(user=stale_user):
existing = RecordingAccess.objects.filter(
user=kept_user, recording=rca.recording
).first()
if existing is None:
rca.user = kept_user
rca.save(update_fields=["user"])
else:
if ROLE_PRIORITY.get(rca.role, 0) > ROLE_PRIORITY.get(existing.role, 0):
existing.role = rca.role
existing.save(update_fields=["role"])
rca.delete()
def _merge_files(self, stale_user, kept_user):
"""Re-assign files created by stale_user to kept_user."""
File.objects.filter(creator=stale_user).update(creator=kept_user)
@@ -0,0 +1,19 @@
# Generated by Django 5.2.14 on 2026-06-02 17:31
import django.db.models.functions.text
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('auth', '0012_alter_user_first_name_max_length'),
('core', '0018_rename_active_application_is_active'),
]
operations = [
migrations.AddConstraint(
model_name='user',
constraint=models.UniqueConstraint(django.db.models.functions.text.Lower('email'), condition=models.Q(('sub__isnull', True)), name='unique_email_when_sub_is_null'),
),
]
@@ -0,0 +1,18 @@
# Generated by Django 5.2.14 on 2026-06-03 12:14
from django.db import migrations, models
class Migration(migrations.Migration):
dependencies = [
('core', '0019_user_unique_email_when_sub_is_null'),
]
operations = [
migrations.AlterField(
model_name='file',
name='upload_state',
field=models.CharField(choices=[('pending', 'Pending'), ('analyzing', 'Analyzing'), ('ready', 'Ready')], max_length=25),
),
]
+24 -1
View File
@@ -211,6 +211,13 @@ class User(AbstractBaseUser, BaseModel, auth_models.PermissionsMixin):
ordering = ("-created_at",)
verbose_name = _("user")
verbose_name_plural = _("users")
constraints = [
models.UniqueConstraint(
models.functions.Lower("email"),
condition=models.Q(sub__isnull=True),
name="unique_email_when_sub_is_null",
)
]
def __str__(self):
return self.email or self.admin_email or str(self.id)
@@ -839,8 +846,8 @@ class FileUploadStateChoices(models.TextChoices):
"""Possible states of a file."""
PENDING = "pending", _("Pending")
ANALYZING = "analyzing", _("Analyzing")
# Commented out for now, as we may need this when we implement the malware detection logic.
# ANALYZING = "analyzing", _("Analyzing")
# SUSPICIOUS = "suspicious", _("Suspicious")
# FILE_TOO_LARGE_TO_ANALYZE = (
# "file_too_large_to_analyze",
@@ -947,6 +954,16 @@ class File(BaseModel):
return f"{settings.FILE_UPLOAD_PATH}/{self.pk!s}"
@property
def temporary_key_base(self):
"""Temporary key base used while upload is still pending."""
if not self.pk:
raise RuntimeError(
"The file instance must be saved before requesting a storage key."
)
return f"{settings.FILE_UPLOAD_TMP_PATH}/{self.pk!s}"
@property
def file_key(self):
"""Key used to store the file in object storage."""
@@ -955,6 +972,12 @@ class File(BaseModel):
# leaking Personal Information in logs, etc.
return f"{self.key_base}{extension!s}"
@property
def temporary_file_key(self):
"""Temporary key used to upload the file before it is finalized."""
_, extension = splitext(self.filename)
return f"{self.temporary_key_base}{extension!s}"
def get_abilities(self, user):
"""
Compute and return abilities for a given user on the file.
@@ -0,0 +1,110 @@
"""Service for provisional user creation."""
import logging
from django.conf import settings
from django.core.exceptions import SuspiciousOperation, ValidationError
from django.db import IntegrityError
from core import models
logger = logging.getLogger(__name__)
class ProvisionalUserError(Exception):
"""Base exception for provisional user service errors."""
class ProvisionalUserCreationDisabledError(ProvisionalUserError):
"""Raised when provisional user creation is disabled by configuration."""
class ProvisionalUserIntegrityError(ProvisionalUserError):
"""Raised when a provisional user cannot be created or retrieved after a race condition."""
class ProvisionalUserService:
"""Handles creation and retrieval of provisional users.
A provisional user is created without a `sub`, identified by email only.
The `sub` is set on first successful OIDC authentication via Django LaSuite.
"""
def __init__(self):
"""Initialize the service."""
# `OIDC_USER_SUB_FIELD_IMMUTABLE` comes from Django LaSuite and prevents `sub`
# updates. We override its default value to allow setting `sub` for
# provisional users.
self._is_creation_enabled = (
settings.APPLICATION_ALLOW_USER_CREATION
and settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION
and not settings.OIDC_USER_SUB_FIELD_IMMUTABLE
)
def _get_by_email(self, email: str) -> models.User | None:
"""Return the user with this email, or None if not found."""
try:
return models.User.objects.get(email__iexact=email)
except models.User.DoesNotExist:
return None
except models.User.MultipleObjectsReturned as e:
raise SuspiciousOperation(
"Multiple user accounts share a common email."
) from e
def get_or_create(
self, email: str, client_id: str
) -> tuple[models.User | None, bool]:
"""Get or create a provisional user identified by email.
Args:
email: The email address to identify the user.
client_id: The application client_id, used for audit logging only.
Returns:
A (user, created) tuple — mirrors get_or_create conventions.
Raises:
ProvisionalUserError: If creation and retrieval both fail.
"""
user = self._get_by_email(email)
if user:
return user, False
if not self._is_creation_enabled:
raise ProvisionalUserCreationDisabledError(
"Provisional user creation is disabled by configuration."
)
# Create a provisional user without `sub`, identified by email only.
# This relies on Django LaSuite implicitly updating the `sub` field on the
# user's first successful OIDC authentication. If this stops working,
# check for behavior changes in Django LaSuite.
try:
user = models.User(sub=None, email=email)
user.set_unusable_password()
user.save()
logger.info(
"Provisional user created via application: user_id=%s, email=%s, client_id=%s",
user.id,
email,
client_id,
)
return user, True
except (IntegrityError, ValidationError) as e:
logger.warning(
"Race condition on provisional user creation, fetching existing: "
"email=%s, client_id=%s",
email,
client_id,
)
user = self._get_by_email(email)
if user:
return user, False
raise ProvisionalUserIntegrityError(
"Failed to create or retrieve provisional user."
) from e
@@ -346,8 +346,11 @@ def test_authentication_getter_existing_user_change_fields(
# One and only one additional update query when a field has changed
# Note: .save() triggers uniqueness validation queries for unique fields,
# adding extra SELECT queries before the UPDATE (e.g., checking unique=True on 'sub')
with django_assert_num_queries(3):
# adding extra SELECT queries before the UPDATE:
# - unique=True on 'sub'
# - unique=True on 'admin_email'
# - partial unique index 'unique_email_when_sub_is_null'
with django_assert_num_queries(5):
authenticated_user = klass.get_or_create_user(
access_token="test-token", id_token=None, payload=None
)
@@ -118,7 +118,7 @@ def test_api_files_create_file_authenticated_success():
assert policy_parsed.scheme == "http"
assert policy_parsed.netloc in ["minio:9000", "localhost:9000"]
assert policy_parsed.path == f"/meet-media-storage/files/{file.id!s}.png"
assert policy_parsed.path == f"/meet-media-storage/tmp/files/{file.id!s}.png"
query_params = parse_qs(policy_parsed.query)
@@ -1,6 +1,7 @@
"""Test related to item upload ended API."""
import logging
from concurrent.futures import ThreadPoolExecutor
from io import BytesIO
from django.core.files.storage import default_storage
@@ -81,7 +82,7 @@ def test_api_file_upload_ended_success(settings):
)
default_storage.save(
file.file_key,
file.temporary_file_key,
BytesIO(b"my prose"),
)
@@ -97,6 +98,7 @@ def test_api_file_upload_ended_success(settings):
assert response.json()["mimetype"] == "text/plain"
@pytest.mark.django_db(transaction=True)
def test_api_file_upload_ended_mimetype_not_allowed(settings, caplog):
"""
Test that the API returns a 400 when the mimetype is not allowed.
@@ -119,7 +121,7 @@ def test_api_file_upload_ended_mimetype_not_allowed(settings, caplog):
)
default_storage.save(
file.file_key,
file.temporary_file_key,
BytesIO(b"my prose"),
)
@@ -156,7 +158,7 @@ def test_api_file_upload_ended_mimetype_not_allowed_not_checking_mimetype(settin
)
default_storage.save(
file.file_key,
file.temporary_file_key,
BytesIO(b"my prose"),
)
@@ -200,7 +202,7 @@ def test_api_upload_ended_mismatch_mimetype_with_object_storage(settings, caplog
s3_client.put_object(
Bucket=default_storage.bucket_name,
Key=file.file_key,
Key=file.temporary_file_key,
ContentType="text/html",
Body=BytesIO(
b'<meta http-equiv="refresh" content="0; url=https://fichiers.numerique.gouv.fr">'
@@ -211,7 +213,7 @@ def test_api_upload_ended_mismatch_mimetype_with_object_storage(settings, caplog
)
head_object = s3_client.head_object(
Bucket=default_storage.bucket_name, Key=file.file_key
Bucket=default_storage.bucket_name, Key=file.temporary_file_key
)
assert head_object["ContentType"] == "text/html"
@@ -234,9 +236,10 @@ def test_api_upload_ended_mismatch_mimetype_with_object_storage(settings, caplog
assert head_object["Metadata"] == {"foo": "bar"}
@pytest.mark.django_db(transaction=True)
def test_api_upload_ended_file_size_exceeded(settings, caplog):
"""
Test when the file size exceed the allowed max upload file size
Test when the file size exceeds the allowed max upload file size
should return a 400 and delete the file.
"""
@@ -256,7 +259,7 @@ def test_api_upload_ended_file_size_exceeded(settings, caplog):
)
default_storage.save(
file.file_key,
file.temporary_file_key,
BytesIO(b"my prose"),
)
@@ -270,3 +273,48 @@ def test_api_upload_ended_file_size_exceeded(settings, caplog):
assert not models.File.objects.filter(id=file.id).exists()
assert not default_storage.exists(file.file_key)
@pytest.mark.django_db(transaction=True)
def test_api_file_upload_ended_concurrent_calls_are_serialized(settings):
"""Only one concurrent upload-ended call can finalize a pending upload."""
settings.FILE_UPLOAD_APPLY_RESTRICTIONS = True
settings.FILE_UPLOAD_RESTRICTIONS = {
"background_image": {
**settings.FILE_UPLOAD_RESTRICTIONS["background_image"],
"allowed_mimetypes": ["text/plain"],
},
}
user = factories.UserFactory()
file = factories.FileFactory(
type=FileTypeChoices.BACKGROUND_IMAGE,
filename="my_file.txt",
creator=user,
)
default_storage.save(file.temporary_file_key, BytesIO(b"my prose"))
def call_upload_ended():
client = APIClient()
client.force_login(user)
return client.post(f"/api/v1.0/files/{file.id!s}/upload-ended/")
with ThreadPoolExecutor(max_workers=2) as executor:
futures = [
executor.submit(call_upload_ended),
executor.submit(call_upload_ended),
]
responses = [future.result() for future in futures]
status_codes = sorted(response.status_code for response in responses)
assert status_codes == [200, 400]
failed_response = next(
response for response in responses if response.status_code == 400
)
assert failed_response.json() == {
"file": "This action is only available for files in PENDING state."
}
file.refresh_from_db()
assert file.upload_state == FileUploadStateChoices.READY
@@ -0,0 +1,460 @@
"""Tests for the merge_duplicate_users management command."""
from unittest import mock
from django.core.management import base, call_command
import pytest
from core.factories import (
FileFactory,
UserFactory,
UserRecordingAccessFactory,
UserResourceAccessFactory,
)
from core.models import RecordingAccess, ResourceAccess, RoleChoices, User
pytestmark = pytest.mark.django_db
# pylint: disable=W0613
def test_merge_no_duplicates_does_nothing():
"""Command should do nothing when no duplicate users exist."""
user = UserFactory(email="unique@example.com")
call_command("merge_duplicate_users")
assert User.objects.count() == 1
assert User.objects.filter(id=user.id).exists()
def test_merge_keeps_most_recently_created_user():
"""Command should keep the most recently created user when duplicates exist."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
call_command("merge_duplicate_users")
assert not User.objects.filter(id=user1.id).exists()
assert User.objects.filter(id=user2.id).exists()
def test_merge_deletes_all_stale_users():
"""Command should delete all stale users and keep only the most recently created one."""
email = "many@example.com"
UserFactory(email=email)
UserFactory(email=email)
user_kept = UserFactory(email=email)
call_command("merge_duplicate_users")
assert User.objects.filter(email=email).count() == 1
assert User.objects.filter(id=user_kept.id).exists()
# ── ResourceAccess ─────────────────────────────────────────────────────────────
def test_merge_transfers_resource_access_to_kept_user():
"""ResourceAccess should be transferred to the kept user when stale user is merged."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
ra = UserResourceAccessFactory(user=user1)
call_command("merge_duplicate_users")
ra.refresh_from_db()
assert ra.user == user2
def test_merge_transfers_multiple_room_accesses():
"""All ResourceAccesses should be transferred to the kept user when stale user is merged."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
accesses = UserResourceAccessFactory.create_batch(3, user=user1)
call_command("merge_duplicate_users")
assert not ResourceAccess.objects.filter(user=user1).exists()
for ra in accesses:
assert ResourceAccess.objects.filter(user=user2, resource=ra.resource).exists()
def test_merge_all_resource_accesses_owned_by_kept_user_nothing_changes():
"""ResourceAccesses should remain unchanged when all are already owned by the kept user."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
accesses = UserResourceAccessFactory.create_batch(3, user=user2)
call_command("merge_duplicate_users")
assert not ResourceAccess.objects.filter(user=user1).exists()
for ra in accesses:
ra.refresh_from_db()
assert ra.user == user2
def test_merge_resource_access_conflict_upgrades_to_owner():
"""ResourceAccess role should be upgraded to owner when stale user has a higher role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
ra1 = UserResourceAccessFactory(user=user1, role=RoleChoices.OWNER)
ra2 = UserResourceAccessFactory(
user=user2, resource=ra1.resource, role=RoleChoices.MEMBER
)
other_accesses = UserResourceAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
ra2.refresh_from_db()
assert ra2.role == RoleChoices.OWNER
assert not ResourceAccess.objects.filter(user=user1).exists()
for ra in other_accesses:
assert ResourceAccess.objects.filter(
user=user2, resource=ra.resource, role=RoleChoices.MEMBER
).exists()
def test_merge_resource_access_conflict_upgrades_to_admin():
"""ResourceAccess role should be upgraded to admin when stale user has a higher role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
ra1 = UserResourceAccessFactory(user=user1, role=RoleChoices.ADMIN)
ra2 = UserResourceAccessFactory(
user=user2, resource=ra1.resource, role=RoleChoices.MEMBER
)
other_accesses = UserResourceAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
ra2.refresh_from_db()
assert ra2.role == RoleChoices.ADMIN
assert not ResourceAccess.objects.filter(user=user1).exists()
for ra in other_accesses:
assert ResourceAccess.objects.filter(
user=user2, resource=ra.resource, role=RoleChoices.MEMBER
).exists()
def test_merge_resource_access_conflict_does_not_downgrade_role():
"""ResourceAccess role should not be downgraded when stale user has a lower role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
ra1 = UserResourceAccessFactory(user=user1, role=RoleChoices.MEMBER)
ra2 = UserResourceAccessFactory(
user=user2, resource=ra1.resource, role=RoleChoices.OWNER
)
other_accesses = UserResourceAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
ra2.refresh_from_db()
assert ra2.role == RoleChoices.OWNER
assert not ResourceAccess.objects.filter(user=user1).exists()
for ra in other_accesses:
assert ResourceAccess.objects.filter(
user=user2, resource=ra.resource, role=RoleChoices.MEMBER
).exists()
def test_merge_resource_access_conflict_equal_role_keeps_single_access():
"""ResourceAccess should keep one entry for the kept user when both ones have the same role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
ra1 = UserResourceAccessFactory(user=user1, role=RoleChoices.MEMBER)
UserResourceAccessFactory(
user=user2, resource=ra1.resource, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
accesses = ResourceAccess.objects.filter(resource=ra1.resource)
assert accesses.count() == 1
assert accesses.first().user == user2
assert accesses.first().role == RoleChoices.MEMBER
# ── RecordingAccess ────────────────────────────────────────────────────────────
def test_merge_transfers_recording_access_to_kept_user():
"""RecordingAccess should be transferred to the kept user when stale user is merged."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
rca = UserRecordingAccessFactory(user=user1)
call_command("merge_duplicate_users")
rca.refresh_from_db()
assert rca.user == user2
def test_merge_transfers_multiple_recording_accesses():
"""All RecordingAccesses should be transferred to the kept user when stale user is merged."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
accesses = UserRecordingAccessFactory.create_batch(3, user=user1)
call_command("merge_duplicate_users")
assert not RecordingAccess.objects.filter(user=user1).exists()
for rca in accesses:
assert RecordingAccess.objects.filter(
user=user2, recording=rca.recording
).exists()
def test_merge_all_recording_accesses_owned_by_kept_user_nothing_changes():
"""RecordingAccesses should remain unchanged when all are already owned by the kept user."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
accesses = UserRecordingAccessFactory.create_batch(3, user=user2)
call_command("merge_duplicate_users")
assert not RecordingAccess.objects.filter(user=user1).exists()
for rca in accesses:
rca.refresh_from_db()
assert rca.user == user2
def test_merge_recording_access_conflict_upgrades_to_owner():
"""RecordingAccess role should be upgraded to owner when stale user has a higher role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
rca1 = UserRecordingAccessFactory(user=user1, role=RoleChoices.OWNER)
rca2 = UserRecordingAccessFactory(
user=user2, recording=rca1.recording, role=RoleChoices.MEMBER
)
other_accesses = UserRecordingAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
rca2.refresh_from_db()
assert rca2.role == RoleChoices.OWNER
assert not RecordingAccess.objects.filter(user=user1).exists()
for rca in other_accesses:
assert RecordingAccess.objects.filter(
user=user2, recording=rca.recording, role=RoleChoices.MEMBER
).exists()
def test_merge_recording_access_conflict_upgrades_to_admin():
"""RecordingAccess role should be upgraded to admin when stale user has a higher role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
rca1 = UserRecordingAccessFactory(user=user1, role=RoleChoices.ADMIN)
rca2 = UserRecordingAccessFactory(
user=user2, recording=rca1.recording, role=RoleChoices.MEMBER
)
other_accesses = UserRecordingAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
rca2.refresh_from_db()
assert rca2.role == RoleChoices.ADMIN
assert not RecordingAccess.objects.filter(user=user1).exists()
for rca in other_accesses:
assert RecordingAccess.objects.filter(
user=user2, recording=rca.recording, role=RoleChoices.MEMBER
).exists()
def test_merge_recording_access_conflict_does_not_downgrade_role():
"""RecordingAccess role should not be downgraded when stale user has a lower role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
rca1 = UserRecordingAccessFactory(user=user1, role=RoleChoices.MEMBER)
rca2 = UserRecordingAccessFactory(
user=user2, recording=rca1.recording, role=RoleChoices.OWNER
)
other_accesses = UserRecordingAccessFactory.create_batch(
3, user=user1, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
rca2.refresh_from_db()
assert rca2.role == RoleChoices.OWNER
assert not RecordingAccess.objects.filter(user=user1).exists()
for rca in other_accesses:
assert RecordingAccess.objects.filter(
user=user2, recording=rca.recording, role=RoleChoices.MEMBER
).exists()
def test_merge_recording_access_conflict_equal_role_keeps_single_access():
"""RecordingAccess should keep one entry for the user when both users have the same role."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
rca1 = UserRecordingAccessFactory(user=user1, role=RoleChoices.MEMBER)
UserRecordingAccessFactory(
user=user2, recording=rca1.recording, role=RoleChoices.MEMBER
)
call_command("merge_duplicate_users")
accesses = RecordingAccess.objects.filter(recording=rca1.recording)
assert accesses.count() == 1
assert accesses.first().user == user2
assert accesses.first().role == RoleChoices.MEMBER
# ── Files ──────────────────────────────────────────────────────────────────────
def test_merge_reassigns_files_to_kept_user():
"""Files should be reassigned to the kept user when stale user is merged."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
files = FileFactory.create_batch(3, creator=user1)
call_command("merge_duplicate_users")
for f in files:
f.refresh_from_db()
assert f.creator == user2
def test_merge_kept_user_own_files_untouched():
"""Files already owned by the kept user should remain unchanged after merge."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
FileFactory(creator=user1)
kept_file = FileFactory(creator=user2)
call_command("merge_duplicate_users")
kept_file.refresh_from_db()
assert kept_file.creator == user2
# ── Dry-run ────────────────────────────────────────────────────────────────────
def test_merge_dry_run_does_not_delete_users():
"""Command should not delete any users when dry-run is enabled."""
UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
call_command("merge_duplicate_users", dry_run=True)
assert User.objects.filter(email="dup@example.com").count() == 2
def test_merge_dry_run_does_not_move_resource_access():
"""Command should not move resource accesses when dry-run is enabled."""
user1 = UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
ra = UserResourceAccessFactory(user=user1)
call_command("merge_duplicate_users", dry_run=True)
ra.refresh_from_db()
assert ra.user == user1
def test_merge_dry_run_does_not_move_recording_access():
"""Command should not move recording accesses when dry-run is enabled."""
user1 = UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
ra = UserRecordingAccessFactory(user=user1)
call_command("merge_duplicate_users", dry_run=True)
ra.refresh_from_db()
assert ra.user == user1
def test_merge_dry_run_does_not_move_files():
"""Command should not reassign files when dry-run is enabled."""
user1 = UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
f = FileFactory(creator=user1)
call_command("merge_duplicate_users", dry_run=True)
f.refresh_from_db()
assert f.creator == user1
# ── Isolation ──────────────────────────────────────────────────────────────────
def test_merge_non_duplicate_users_untouched():
"""Non-duplicate users should remain untouched when other duplicates are merged."""
unique = UserFactory(email="unique@example.com")
UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
call_command("merge_duplicate_users")
assert User.objects.filter(id=unique.id).exists()
def test_merge_non_duplicate_resource_access_untouched():
"""ResourceAccess of non-duplicate users should remain untouched."""
unique = UserFactory(email="unique@example.com")
ra = UserResourceAccessFactory(user=unique)
UserFactory(email="dup@example.com")
UserFactory(email="dup@example.com")
call_command("merge_duplicate_users")
ra.refresh_from_db()
assert ra.user == unique
def test_merge_multiple_email_groups_all_merged():
"""Command should merge all duplicate email groups in a single run."""
for i in range(3):
UserFactory(email=f"group{i}@example.com")
UserFactory(email=f"group{i}@example.com")
call_command("merge_duplicate_users")
for i in range(3):
assert User.objects.filter(email=f"group{i}@example.com").count() == 1
assert User.objects.count() == 3
# ── NULL / blank email guard ───────────────────────────────────────────────────
def test_merge_does_not_merge_users_with_null_email():
"""Users with NULL email must never be merged together, even if multiple exist."""
user1 = UserFactory(email=None)
user2 = UserFactory(email=None)
call_command("merge_duplicate_users")
assert User.objects.filter(id=user1.id).exists()
assert User.objects.filter(id=user2.id).exists()
def test_merge_does_not_merge_users_with_blank_email():
"""Users with empty-string email must never be merged together, even if multiple exist."""
user1 = UserFactory(email="")
user2 = UserFactory(email="")
call_command("merge_duplicate_users")
assert User.objects.filter(id=user1.id).exists()
assert User.objects.filter(id=user2.id).exists()
# ── Atomicity ──────────────────────────────────────────────────────────────────
@mock.patch(
"core.management.commands.merge_duplicate_users.Command._merge_recording_accesses",
side_effect=Exception("forced failure"),
)
def test_merge_is_atomic_rolls_back_all_on_any_failure(mock_reassign_files):
"""Merge should be fully rolled back when any step fails."""
user1 = UserFactory(email="dup@example.com")
user2 = UserFactory(email="dup@example.com")
resource_accesses = UserResourceAccessFactory.create_batch(3, user=user1)
recording_accesses = UserRecordingAccessFactory.create_batch(3, user=user1)
files = FileFactory.create_batch(3, creator=user1)
with pytest.raises(base.CommandError):
call_command("merge_duplicate_users")
assert User.objects.filter(id=user1.id).exists()
assert User.objects.filter(id=user2.id).exists()
for ra in resource_accesses:
ra.refresh_from_db()
assert ra.user == user1
for rca in recording_accesses:
rca.refresh_from_db()
assert rca.user == user1
for f in files:
f.refresh_from_db()
assert f.creator == user1
# ── Email filter ───────────────────────────────────────────────────────────────
def test_merge_email_filter_only_merges_matching_emails():
"""Command should only merge users whose email matches the filter."""
UserFactory(email="user1@example.com")
UserFactory(email="user1@example.com")
other1 = UserFactory(email="user1@other.com")
other2 = UserFactory(email="user1@other.com")
call_command("merge_duplicate_users", email_filter="@example.com")
assert User.objects.filter(email="user1@example.com").count() == 1
assert User.objects.filter(id=other1.id).exists()
assert User.objects.filter(id=other2.id).exists()
def test_merge_email_filter_no_match_does_nothing():
"""Command should do nothing when the email filter matches no users."""
UserFactory(email="user1@example.com")
UserFactory(email="user1@example.com")
call_command("merge_duplicate_users", email_filter="@nomatch.com")
assert User.objects.filter(email="user1@example.com").count() == 2
def test_merge_email_filter_is_case_insensitive():
"""Command should match emails case-insensitively when filtering."""
UserFactory(email="user1@Example.com")
UserFactory(email="user1@Example.com")
call_command("merge_duplicate_users", email_filter="@example.com")
assert User.objects.filter(email="user1@Example.com").count() == 1
@@ -4,6 +4,8 @@ Tests for external API /token endpoint
# pylint: disable=W0621
from unittest import mock
import jwt
import pytest
from freezegun import freeze_time
@@ -15,6 +17,7 @@ from core.factories import (
UserFactory,
)
from core.models import ApplicationScope, User
from core.services import provisional_user_service
pytestmark = pytest.mark.django_db
@@ -436,3 +439,88 @@ def test_api_applications_token_existing_user(settings):
"delegated": True,
"scope": "rooms:list rooms:create",
}
@mock.patch.object(provisional_user_service.ProvisionalUserService, "_get_by_email")
def test_api_applications_token_new_user_race_condition(mock_get_by_email, settings):
"""Should handle race condition where two concurrent requests create the same user."""
settings.APPLICATION_ALLOW_USER_CREATION = True
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
settings.OIDC_USER_SUB_FIELD_IMMUTABLE = False
application = ApplicationFactory(
is_active=True, scopes=[ApplicationScope.ROOMS_LIST]
)
plain_secret = "test-secret-123"
application.client_secret = plain_secret
application.save()
email = "john.doe@example.com"
# First call: lie and say user doesn't exist, simulating the race window
# Second call (recovery path): return the real user
existing_user = UserFactory(sub=None, email=email)
mock_get_by_email.side_effect = [None, existing_user]
client = APIClient()
response = client.post(
"/external-api/v1.0/application/token/",
{
"client_id": application.client_id,
"client_secret": plain_secret,
"grant_type": "client_credentials",
"scope": email,
},
format="json",
)
assert response.status_code == 200
assert mock_get_by_email.call_count == 2
token = response.data["access_token"]
payload = jwt.decode(
token,
settings.APPLICATION_JWT_SECRET_KEY,
algorithms=[settings.APPLICATION_JWT_ALG],
issuer=settings.APPLICATION_JWT_ISSUER,
audience=settings.APPLICATION_JWT_AUDIENCE,
)
assert payload["user_id"] == str(existing_user.id)
assert User.objects.filter(email=email).count() == 1
@mock.patch.object(
provisional_user_service.ProvisionalUserService,
"get_or_create",
side_effect=provisional_user_service.ProvisionalUserIntegrityError,
)
def test_api_applications_token_new_user_race_condition_unrecoverable(
mock_get_or_create, settings
):
"""Should return 500 when ProvisionalUserIntegrityError is raised."""
settings.APPLICATION_ALLOW_USER_CREATION = True
settings.OIDC_FALLBACK_TO_EMAIL_FOR_IDENTIFICATION = True
settings.OIDC_USER_SUB_FIELD_IMMUTABLE = False
application = ApplicationFactory(
is_active=True, scopes=[ApplicationScope.ROOMS_LIST]
)
plain_secret = "test-secret-123"
application.client_secret = plain_secret
application.save()
client = APIClient()
response = client.post(
"/external-api/v1.0/application/token/",
{
"client_id": application.client_id,
"client_secret": plain_secret,
"grant_type": "client_credentials",
"scope": "john.doe@example.com",
},
format="json",
)
assert response.status_code == 409
assert mock_get_or_create.call_count == 1
@@ -44,3 +44,73 @@ def test_models_users_send_mail_main_missing():
user.email_user("my subject", "my message")
assert str(excinfo.value) == "User has no email address."
def test_models_users_email_unique_when_sub_is_null():
"""Email should be unique among users with no sub (pending users)."""
user = factories.UserFactory(sub=None, email="test@example.com")
with pytest.raises(
ValidationError, match="Constraint “unique_email_when_sub_is_null” is violated."
):
factories.UserFactory(sub=None, email=user.email)
def test_models_users_email_unique_case_insensitive_when_sub_is_null():
"""Email uniqueness should be case-insensitive among users with no sub (pending users)."""
factories.UserFactory(sub=None, email="Test@example.com")
with pytest.raises(
ValidationError, match="Constraint “unique_email_when_sub_is_null” is violated."
):
factories.UserFactory(sub=None, email="test@example.com")
def test_models_users_email_not_unique_when_sub_is_set():
"""Email uniqueness should not be enforced when users have a sub."""
user = factories.UserFactory(sub="sub-1", email="test@example.com")
user2 = factories.UserFactory(sub="sub-2", email=user.email)
assert user2.email == user.email
def test_models_users_email_not_unique_between_sub_null_and_sub_set():
"""A user with a sub and a pending user (sub=None) can share the same email."""
user = factories.UserFactory(sub="sub-1", email="test@example.com")
user2 = factories.UserFactory(sub=None, email=user.email)
assert user2.email == user.email
def test_models_users_email_unique_constraint_allows_multiple_null_emails():
"""Multiple users with sub=None and email=None should be allowed."""
factories.UserFactory(sub=None, email=None)
factories.UserFactory(sub=None, email=None)
def test_models_users_sub_null_email_null_does_not_prevent_creation():
"""Multiple pending users (sub=None, email=None) can be created without conflict.
sub=None is not unique-constrained. email uniqueness is only enforced among
sub=None users with a non-null email, so email=None bypasses it (NULL != NULL in SQL).
"""
# Ghost row can still appear from bad code path
u1 = factories.UserFactory(sub=None, email=None)
u2 = factories.UserFactory(sub=None, email=None)
assert u1.pk != u2.pk
def test_models_users_sub_can_be_null():
"""sub is nullable: pending users exist before OIDC activation."""
user = factories.UserFactory(sub=None)
user.refresh_from_db()
assert user.sub is None
def test_models_users_sub_null_does_not_prevent_creation():
"""Multiple users can be created with sub=None (pending state)."""
u1 = factories.UserFactory(sub=None)
u2 = factories.UserFactory(sub=None)
assert u1.pk != u2.pk
def test_models_users_sub_blank_is_accepted():
"""sub='' passes validation because blank=True; null is preferred but not enforced."""
user = factories.UserFactory.build(sub="")
user.full_clean()
+1 -1
View File
@@ -426,7 +426,7 @@ def generate_upload_policy(file):
Originally taken from https://github.com/suitenumerique/drive/blob/564822d31f071c6dfacd112ef4b7146c73077cd9/src/backend/core/api/utils.py#L102 # pylint: disable=line-too-long
"""
key = file.file_key
key = file.temporary_file_key
# This settings should be used if the backend application and the frontend application
# can't connect to the object storage with the same domain. This is the case in the
+46
View File
@@ -186,6 +186,9 @@ class Base(Configuration):
FILE_UPLOAD_PATH = values.Value(
"files", environ_name="FILE_UPLOAD_PATH", environ_prefix=None
)
FILE_UPLOAD_TMP_PATH = values.Value(
"tmp/files", environ_name="FILE_UPLOAD_TMP_PATH", environ_prefix=None
)
FILE_UPLOAD_APPLY_RESTRICTIONS = values.BooleanValue(
default=True, environ_name="FILE_UPLOAD_APPLY_RESTRICTIONS", environ_prefix=None
@@ -1011,6 +1014,44 @@ class Base(Configuration):
environ_prefix=None,
)
# Logging
# We want to make it easy to log to console but by default we log production
# to Sentry and don't want to log to console.
LOGGING = {
"version": 1,
"disable_existing_loggers": False,
"formatters": {
"simple": {
"format": "{asctime} {name} {levelname} {message}",
"style": "{",
},
},
"handlers": {
"console": {
"class": "logging.StreamHandler",
"formatter": "simple",
},
},
# Override root logger to send it to console
"root": {
"handlers": ["console"],
"level": values.Value(
"INFO", environ_name="LOGGING_LEVEL_LOGGERS_ROOT", environ_prefix=None
),
},
"loggers": {
"core": {
"handlers": ["console"],
"level": values.Value(
"INFO",
environ_name="LOGGING_LEVEL_LOGGERS_APP",
environ_prefix=None,
),
"propagate": False,
},
},
}
# pylint: disable=invalid-name
@property
def ENVIRONMENT(self):
@@ -1049,6 +1090,11 @@ class Base(Configuration):
"""
super().post_setup()
if cls.FILE_UPLOAD_TMP_PATH == cls.FILE_UPLOAD_PATH:
raise ValueError(
"FILE_UPLOAD_TMP_PATH cannot be the same as FILE_UPLOAD_PATH"
)
# The SENTRY_DSN setting should be available to activate sentry for an environment
if cls.SENTRY_DSN is not None:
sentry_sdk.init(
+1 -1
View File
@@ -7,7 +7,7 @@ build-backend = "uv_build"
[project]
name = "meet"
version = "1.16.0"
version = "1.18.0"
authors = [{ "name" = "DINUM", "email" = "dev@mail.numerique.gouv.fr" }]
classifiers = [
"Development Status :: 5 - Production/Stable",
+1 -1
View File
@@ -1173,7 +1173,7 @@ wheels = [
[[package]]
name = "meet"
version = "1.16.0"
version = "1.18.0"
source = { editable = "." }
dependencies = [
{ name = "aiohttp" },
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "meet",
"version": "1.16.0",
"version": "1.18.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "meet",
"version": "1.16.0",
"version": "1.18.0",
"dependencies": {
"@fontsource-variable/atkinson-hyperlegible-next": "5.2.6",
"@fontsource-variable/lexend": "5.2.11",
+1 -1
View File
@@ -1,7 +1,7 @@
{
"name": "meet",
"private": true,
"version": "1.16.0",
"version": "1.18.0",
"type": "module",
"scripts": {
"dev": "panda codegen && vite",
+1 -1
View File
@@ -1,4 +1,4 @@
apiVersion: v2
type: application
name: meet
version: 0.0.21
version: 0.0.22
@@ -0,0 +1,126 @@
{{- if .Values.backend.mergeDuplicateUsers.enabled }}
{{- $envVars := include "meet.common.env" (list . .Values.backend) -}}
{{- $fullName := include "meet.backend.fullname" . -}}
{{- $component := "backend-merge-duplicate-users" -}}
apiVersion: batch/v1
kind: Job
metadata:
name: {{ $fullName }}-merge-duplicate-users
namespace: {{ .Release.Namespace | quote }}
annotations:
argocd.argoproj.io/sync-options: Replace=true,Force=true
{{- with .Values.backend.mergeDuplicateUsers.annotations }}
{{- toYaml . | nindent 4 }}
{{- end }}
labels:
{{- include "meet.common.labels" (list . $component) | nindent 4 }}
spec:
ttlSecondsAfterFinished: {{ .Values.backend.jobs.ttlSecondsAfterFinished }}
backoffLimit: {{ .Values.backend.jobs.backoffLimit }}
template:
metadata:
annotations:
{{- with .Values.backend.podAnnotations }}
{{- toYaml . | nindent 8 }}
{{- end }}
labels:
{{- include "meet.common.selectorLabels" (list . $component) | nindent 8 }}
spec:
{{- if $.Values.image.credentials }}
imagePullSecrets:
- name: {{ include "meet.secret.dockerconfigjson.name" (dict "fullname" (include "meet.fullname" .) "imageCredentials" $.Values.image.credentials) }}
{{- end }}
shareProcessNamespace: {{ .Values.backend.shareProcessNamespace }}
{{- with .Values.backend.podSecurityContext }}
securityContext:
{{- toYaml . | nindent 8 }}
{{- end }}
containers:
{{- with .Values.backend.sidecars }}
{{- toYaml . | nindent 8 }}
{{- end }}
- name: {{ .Chart.Name }}
image: "{{ (.Values.backend.image | default dict).repository | default .Values.image.repository }}:{{ (.Values.backend.image | default dict).tag | default .Values.image.tag }}"
imagePullPolicy: {{ (.Values.backend.image | default dict).pullPolicy | default .Values.image.pullPolicy }}
{{- with .Values.backend.mergeDuplicateUsers.command }}
command:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.backend.args }}
args:
{{- toYaml . | nindent 12 }}
{{- end }}
env:
{{- if $envVars }}
{{- $envVars | indent 12 }}
{{- end }}
{{- with .Values.backend.securityContext }}
securityContext:
{{- toYaml . | nindent 12 }}
{{- end }}
{{- with .Values.backend.resources }}
resources:
{{- toYaml . | nindent 12 }}
{{- end }}
volumeMounts:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
mountPath: {{ $value.path }}
subPath: content
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
mountPath: "{{ $volume.mountPath }}"
{{- end }}
{{- range .Values.backend.extraVolumeMounts }}
- name: {{ .name }}
mountPath: {{ .mountPath }}
subPath: {{ .subPath | default "" }}
readOnly: {{ .readOnly }}
{{- end }}
{{- with .Values.backend.nodeSelector }}
nodeSelector:
{{- toYaml . | nindent 8 }}
{{- end }}
{{- with .Values.backend.tolerations }}
tolerations:
{{- toYaml . | nindent 8 }}
{{- end }}
restartPolicy: {{ .Values.backend.mergeDuplicateUsers.restartPolicy }}
volumes:
{{- range $index, $value := .Values.mountFiles }}
- name: "files-{{ $index }}"
configMap:
name: "{{ include "meet.fullname" $ }}-files-{{ $index }}"
{{- end }}
{{- range $name, $volume := .Values.backend.persistence }}
- name: "{{ $name }}"
{{- if eq $volume.type "emptyDir" }}
emptyDir: {}
{{- else }}
persistentVolumeClaim:
claimName: "{{ $fullName }}-{{ $name }}"
{{- end }}
{{- end }}
{{- range .Values.backend.extraVolumes }}
- name: {{ .name }}
{{- if .existingClaim }}
persistentVolumeClaim:
claimName: {{ .existingClaim }}
{{- else if .hostPath }}
hostPath:
{{ toYaml .hostPath | nindent 12 }}
{{- else if .csi }}
csi:
{{- toYaml .csi | nindent 12 }}
{{- else if .configMap }}
configMap:
{{- toYaml .configMap | nindent 12 }}
{{- else if .emptyDir }}
emptyDir:
{{- toYaml .emptyDir | nindent 12 }}
{{- else }}
emptyDir: {}
{{- end }}
{{- end }}
{{- end }}
+13 -2
View File
@@ -262,8 +262,8 @@ backend:
python manage.py migrate --no-input
restartPolicy: Never
## @param backend.createsuperuser.command backend migrate command
## @param backend.createsuperuser.restartPolicy backend migrate job restart policy
## @param backend.createsuperuser.command backend createsuperuser command
## @param backend.createsuperuser.restartPolicy backend createsuperuser job restart policy
createsuperuser:
command:
- "/bin/sh"
@@ -279,6 +279,17 @@ backend:
python manage.py createsuperuser --email $DJANGO_SUPERUSER_EMAIL --password $DJANGO_SUPERUSER_PASSWORD
restartPolicy: Never
## @param backend.mergeDuplicateUsers.command backend merge_duplicate_users command
## @param backend.mergeDuplicateUsers.restartPolicy backend merge_duplicate_users job restart policy
mergeDuplicateUsers:
annotations: {}
enabled: false
command:
- python
- manage.py
- merge_duplicate_users
restartPolicy: Never
## @param backend.probes.liveness.path [nullable] Configure path for backend HTTP liveness probe
## @param backend.probes.liveness.targetPort [nullable] Configure port for backend HTTP liveness probe
## @param backend.probes.liveness.initialDelaySeconds [nullable] Configure initial delay for backend liveness probe
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "mail_mjml",
"version": "1.16.0",
"version": "1.18.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "mail_mjml",
"version": "1.16.0",
"version": "1.18.0",
"license": "MIT",
"dependencies": {
"@html-to/text-cli": "0.5.4",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "mail_mjml",
"version": "1.16.0",
"version": "1.18.0",
"description": "An util to generate html and text django's templates from mjml templates",
"type": "module",
"dependencies": {
+2 -2
View File
@@ -1,12 +1,12 @@
{
"name": "sdk",
"version": "1.16.0",
"version": "1.18.0",
"lockfileVersion": 3,
"requires": true,
"packages": {
"": {
"name": "sdk",
"version": "1.16.0",
"version": "1.18.0",
"license": "ISC",
"workspaces": [
"./library",
+1 -1
View File
@@ -1,6 +1,6 @@
{
"name": "sdk",
"version": "1.16.0",
"version": "1.18.0",
"author": "",
"license": "ISC",
"description": "",
+1 -1
View File
@@ -1,7 +1,7 @@
[project]
name = "summary"
version = "1.16.0"
version = "1.18.0"
dependencies = [
"fastapi[standard]>=0.105.0",
"uvicorn>=0.24.0",