* Removed constraint on file extension
* Infer audio/video streams from the media with ffmpeg
* Infer the correct processed audio file extension based on actual
codec to avoid ffmpeg errors
We need to support more extensions and make audio extraction dynamic,
as we shipped transcript in production and it led to user complaints
requesting more formats.
Speaker-to-participant assignment relie on WhisperX word timings, but
incorrect word durations in the output can lead to inaccurate overlap
scoring and wrong user attribution. Add a custom heuristic to trim
overly long word durations before computing assignments.
When duration is not reported in the files metadata,
we directly infer the duration from the audio packets.
This prevents errors on webm files.
Very simple audio & video test files have been added
that cover relevant usecases to prevent regressions.
Introduce a new user assignment mechanism to for more friendly output
than the current (SPEAKER_0, SPEAKER_1, ...). Use the VAD metadata to
compare speech intervals with those returned by WhisperX. User with the
highest overlap score above a defined threshold is assigned to each segment.
This method allows for multi-speaker scenarios for a single account.
The tasks endpoint used non-timezone-aware date and time values and split
them into separate variables, which is unconventional. Refactor the
implementation to use timezone-aware datetime objects and align transcription
formatting with the user-declared timezone. Update the source of truth for
recording start time to FileInfo.started_at for improved precision. Adjust
the task signature in preparation for upcoming user assignment work, which
will require `started_at`, `ended_at`, and `metadata_filename`.
Updated taskV2 API contract to be closer to the target gateway contract.
GET operations return the same things as the webhook payload.
Also store the summary on S3 to be iso with transcript.
Add multitenancy support to Summary sub-app. The V1 routes / tasks
behave like before, with the default tenant being "meet".
V2 routes / tasks support being called frm any tenant, and don't have
meet related logic.
V2 tasks are created in separate queues to avoid mix / match,i
Update the link label to use explicit text "Download your recording"
instead of generic "following this link."
This ensures blind users understand the purpose of the link
and the behavior of opening a new window.
Refactor the summary service to better separate concerns, making components
easier to isolate and test. Unify logging logic to ensure consistent
behavior and reduce duplication across the service layer. These changes
set up the codebase for granular testing.
Transcription and summarization results were always generated
using a French text structure (e.g. "Réunion du..."), regardless
of user preference or meeting language. Introduced basic localization
support to adapt generated string languages.
Pip was removed before copying the builder stage output, which caused
it to be reinstalled unintentionally. Adjust the order to align with
the backend image behavior.
Reduce surface area and keep the runtime image minimal.
Alpine 3.22 provides ffmpeg v6 as the latest version.
Alpine 3.23 does not include ffmpeg v7, so upgrade directly to v8.
Install pip temporarily for build steps, then remove it from the
production image.
Based on a request from our European partners, introduce new languages for the
transcription feature. Dutch and German are now supported, which is a great
addition.
It closes#837.
WhisperX is expected to support both languages.
Fix MinIO client configuration: I was incorrectly passing a full URL instead of
an endpoint, which caused errors in staging. Local development values did not
reflect the staging setup and were also out of sync with the backend.
Link the transcription document to its related recording by adding a short
header explaining that users can download the audio file via a dedicated link.
This was a highly requested feature, as many users need to keep their audio
files.
As part of a small refactor, remove the argument length check in the metadata
analytics class. The hardcoded argument count made code evolution harder and was
easy to forget updating. Argument unwrapping remains fragile and should be
redesigned later to be more robust.
The backend is responsible for generating the download link to ensure
consistency and reliability.
I tried adding a divider, but the Markdown-to-Yjs conversion is very lossy and
almost never handles it correctly. Only about one out of ten conversions works
as expected.
Video files are heavy recording files, sometimes several hours long.
Previously, recordings were naively submitted to the Whisper API without
chunking, resulting in very large requests that could take a long time
to process. Video files are much larger than audio-only files, which
could cause performance issues during upload.
Introduce an extra step to extract the audio component from MP4 files,
producing a lighter audio-only file (to be confirmed). No re-encoding
is done, just a minimal FFmpeg extraction based on community guidance,
since I’m not an FFmpeg expert.
This feature is experimental and may introduce regressions, especially
if audio quality or sampling is impacted, which could reduce Whisper’s
accuracy. Early tests with the ASR model worked, but it has not been
tested on long recordings (e.g., 3-hour meetings),
which some users have.
Screen recording are MP4 files containing video)
The current approach is suboptimal: the microservice will later be updated to
extract audio paths from video, which can be heavy to send to the Whisper
service.
This implementation is straightforward, but the notification service is now
handling many responsibilities through conditional logic. A refactor with a
more configurable approach (mapping attributes to processing steps via
settings) would be cleaner and easier to maintain.
For now, this works; further improvements can come later.
I follow the KISS principle, and try to make this new feature implemented
with the lesser impact on the codebase. This isn’t perfect.
The previous code lacked proper encapsulation, resulting in an overly complex
worker. While the initial naive approach was great for bootstrapping the
feature, the refactor introduces more maturity with dedicated service classes
that have clear, single responsibilities.
During the extraction to services, several minor issues were fixed:
1) Properly closing the MinIO response.
2) Enhanced validation of object filenames and extensions to ensure
correct file handling.
3) Introduced a context manager to automatically clean up temporary
local files, removing reliance on developers.
4) Slightly improved logging and naming for clarity.
5) Dynamic temporary file extension handling when it was previously
always an hardcoded .ogg file, even when it was not the case.
lebaudantoine