Compare commits

...

6 Commits

Author SHA1 Message Date
lebaudantoine 28826bf459 (frontend) introduce performance mode with auto-detection and telemetry
Provide CPU relief during video meetings for low-spec and constrained
devices by dynamically reducing video quality.

Context and motivation:

Video processing can be intensive on resource-constrained hardware.
To ensure accessibility on any device, this feature introduces both
automatic, proactive CPU-constraint detection and manual controls in
settings, so users can reduce CPU load while keeping control over
their experience.

Key changes:

* Performance mode store
  - Central, non-persisted store as the single source of truth for
    performance mode.
  - Decouples inputs (e.g. `CpuConstrainedObserver`, settings
    toggle) from effects (`PerformanceModeController` for publishing
    encodings, `VideoResolutionSubscription` for incoming quality
    caps).
  - Intentionally session-scoped to prevent sticky low-quality
    modes across meetings.

* Auto-detection and user toast notification
  - Automatically activate performance mode when the encoder
    reports CPU constraints.
  - Show a 30-second toast letting users override ("keep my
    quality"), which restores encodings and suppresses further
    auto-activations for the session.

* Video settings tab
  - Update the Videos tab in the settings so it reflects the
    current performance mode state and lets users toggle it
    manually.

* Hardware telemetry snapshot
  - Add a non-throwing, best-effort environment snapshot (core
    count, memory, battery status) attached to performance events,
    to help analyze hardware patterns across sessions.
2026-08-13 20:28:02 +02:00
lebaudantoine 77c5329f8a 🐛(analytics) filter benign ResizeObserver loop error in Sentry/PostHog
Filter out harmless `ResizeObserver loop limit exceeded` and
`ResizeObserver loop completed with undelivered notifications`
errors via `beforeSend`.

Why this is safe:

* These are W3C spec-mandated browser guards that defer notification
  delivery to the next frame when callbacks alter layout during
  render. They do not cause JS runtime exceptions or break the UX.

Why we actually need to filter them:

* Telemetry platforms like PostHog do not stack/group these well,
  frequently generating distinct error events per browser engine
  and version.
* The unique variants flood reporting dashboards and trigger
  false-positive alerts that clutter real issue triage.
2026-08-13 14:37:55 +02:00
lebaudantoine c8ec1c8a9d 🐛(frontend) fix toolbar ResizeObserver loop and alignment drift
* Switch toolbar horizontal alignment from `marginRight` to
  `transform: translateX()`, so it no longer triggers layout reflows
  during ResizeObserver cycles and stops the "ResizeObserver loop"
  error.
* Replace the unstable `shift * 2` margin heuristic with a direct
  1:1 positional delta (`offsetX + shift`).
* Decouple CSS transitions: use the individual CSS `translate`
  property for the slide-up/down animations, leaving `transform`
  free for dynamic horizontal positioning.
2026-08-13 14:37:55 +02:00
lebaudantoine 01e004e272 🐛(frontend) vendor formatChatMessageLinks and trim surrounding newlines
Copy the `formatChatMessageLinks` function locally so we can iterate
on it without patching the upstream dependency.

Use the local copy to trim `\n` characters at the beginning and end
of chat messages, which were leaking into the rendered output.
2026-08-13 14:37:55 +02:00
lebaudantoine cbfb97eb54 🐛(frontend) implement hysteresis band for the control bar layout
Introduce dual thresholds (1100px wide, 1050px narrow) for switching
the control bar between the expanded inline controls and the
collapsed menu.

The 50px deadband absorbs the width changes caused by rendering
5 buttons vs. 1 button, preventing an infinite layout oscillation
and the resulting `ResizeObserver loop` errors.
2026-08-13 14:37:55 +02:00
lebaudantoine ac503b3ae5 🔥(frontend) drop unused vendored ConnectionObserver
The vendored ConnectionObserver collected connection data that never
turned out to be useful for debugging.

Remove it to reduce dead code, and re-add a targeted observer later
if a concrete debugging need shows up.
2026-08-13 10:34:18 +02:00
34 changed files with 681 additions and 194 deletions
+12
View File
@@ -8,11 +8,23 @@ and this project adheres to
## [Unreleased]
### Added
- ✨(frontend) introduce performance mode with auto-detection and telemetry
### Changed
- 🔥(frontend) drop unused vendored ConnectionObserver
- 🐛(frontend) vendor formatChatMessageLinks and trim surrounding newlines
### Fixed
- 📈(frontend) downgrade unreachable external home URL from error to event
- 🐛(frontend) handle 401 responses when syncing user preferences
- 🐛(frontend) harden speaker test against missing sinks and play errors
- 🐛(frontend) implement hysteresis band for the control bar layout
- 🐛(frontend) fix toolbar ResizeObserver loop and alignment drift
- 🐛(analytics) filter benign ResizeObserver loop error in Sentry/PostHog
## [1.26.0] - 2026-08-12
@@ -0,0 +1,24 @@
import type { CaptureResult } from 'posthog-js'
const IGNORED_EXCEPTION_PATTERNS = [
/ResizeObserver loop (completed with undelivered notifications|limit exceeded)/,
]
const shouldIgnoreException = (value: unknown): boolean =>
typeof value === 'string' &&
IGNORED_EXCEPTION_PATTERNS.some((pattern) => pattern.test(value))
export const filterExceptions = (
event: CaptureResult | null
): CaptureResult | null => {
if (event?.event !== '$exception') return event
const exceptionList = event.properties?.['$exception_list']
const values: unknown[] = Array.isArray(exceptionList)
? exceptionList.map((exception) => exception?.value)
: []
values.push(event.properties?.['$exception_message'])
return values.some(shouldIgnoreException) ? null : event
}
@@ -0,0 +1,68 @@
export interface HardwareSnapshot {
/** navigator.hardwareConcurrency — logical CPU cores. */
cpu_cores: number | null
/** navigator.deviceMemory — RAM in GiB, bucketed by the browser (Chromium only). */
device_memory_gb: number | null
/** performance.memory.jsHeapSizeLimit in MB (Chromium only, non-standard). */
js_heap_limit_mb: number | null
/** performance.memory.usedJSHeapSize in MB (Chromium only, non-standard). */
js_heap_used_mb: number | null
/** Battery level 0..1 via navigator.getBattery() (Chromium only). */
battery_level: number | null
/** Whether the device is plugged in, via navigator.getBattery(). */
battery_charging: boolean | null
}
const BATTERY_TIMEOUT_MS = 1_000
const toMb = (bytes: unknown): number | null =>
typeof bytes === 'number' ? Math.round(bytes / (1024 * 1024)) : null
export const collectHardwareSnapshot = async (): Promise<HardwareSnapshot> => {
const snapshot: HardwareSnapshot = {
cpu_cores: null,
device_memory_gb: null,
js_heap_limit_mb: null,
js_heap_used_mb: null,
battery_level: null,
battery_charging: null,
}
try {
snapshot.cpu_cores = navigator.hardwareConcurrency ?? null
const nav = navigator as Navigator & {
deviceMemory?: number
userAgentData?: { mobile?: boolean; platform?: string }
getBattery?: () => Promise<{ level: number; charging: boolean }>
}
snapshot.device_memory_gb = nav.deviceMemory ?? null
const memory = (
performance as Performance & {
memory?: { jsHeapSizeLimit?: number; usedJSHeapSize?: number }
}
).memory
snapshot.js_heap_limit_mb = toMb(memory?.jsHeapSizeLimit)
snapshot.js_heap_used_mb = toMb(memory?.usedJSHeapSize)
if (typeof nav.getBattery === 'function') {
// getBattery can hang on some platforms — don't let it delay the event.
const battery = await Promise.race([
nav.getBattery(),
new Promise<null>((resolve) =>
setTimeout(() => resolve(null), BATTERY_TIMEOUT_MS)
),
]).catch(() => null)
if (battery) {
snapshot.battery_level = battery.level
snapshot.battery_charging = battery.charging
}
}
} catch {
// telemetry must never break the app
}
return snapshot
}
@@ -2,6 +2,7 @@ import { useEffect } from 'react'
import { type ApiUser } from '@/features/auth/api/ApiUser'
import { useUser } from '@/features/auth/api/useUser'
import { getPosthog } from '../utils'
import { filterExceptions } from '../exceptionFilters'
export const startAnalyticsSession = (data: ApiUser) => {
getPosthog().then((ph) => {
@@ -47,6 +48,7 @@ export const useAnalytics = ({
capture_unhandled_rejections: true,
capture_console_errors: true,
},
before_send: filterExceptions,
})
})
}, [id, host, flags_api_host, isDisabled])
@@ -30,6 +30,7 @@ export type LogCode =
| 'clipboard_failure'
| 'fullscreen_failure'
| 'publish_sources_failure'
| 'performance_mode_failure'
| 'disconnect_failure'
| 'generic_failure'
@@ -1,6 +1,6 @@
import { ChatRow } from '@/stores/chat'
import React, { useMemo } from 'react'
import { formatChatMessageLinks } from '@livekit/components-react'
import { formatChatMessageLinks } from '../utils'
import { css } from '@/styled-system/css'
import { Text } from '@/primitives'
+32
View File
@@ -0,0 +1,32 @@
import { tokenize, createDefaultGrammar } from '@livekit/components-core'
import { ReactNode } from 'react'
const defaultGrammar = Object.freeze(createDefaultGrammar())
export function formatChatMessageLinks(message: string): ReactNode {
const trimmedMessage = message.replace(/^[\r\n]+|[\r\n]+$/g, '')
return tokenize(trimmedMessage, defaultGrammar).map((tok, i) => {
if (typeof tok === `string`) {
return tok
} else {
const content = tok.content.toString()
const href =
tok.type === `url`
? /^http(s?):\/\//.test(content)
? content
: `https://${content}`
: `mailto:${content}`
return (
<a
className="lk-chat-link"
key={i}
href={href}
target="_blank"
rel="noreferrer"
>
{content}
</a>
)
}
})
}
@@ -3,6 +3,7 @@ export enum ToastDuration {
MEDIUM = 4000,
LONG = 5000,
EXTRA_LONG = 7000,
UNDO_WINDOW = 30000,
}
export const NotificationDuration = {
@@ -15,4 +16,5 @@ export const NotificationDuration = {
REACTION_RECEIVED: ToastDuration.SHORT,
RECORDING_REQUESTED: ToastDuration.LONG,
ROLE_CHANGED: ToastDuration.LONG,
CPU_CONSTRAINED: ToastDuration.UNDO_WINDOW,
} as const
@@ -18,4 +18,5 @@ export enum NotificationType {
RecordingSaving = 'recordingSaving',
PermissionsRemoved = 'permissionsRemoved',
RoleChanged = 'roleChanged',
CpuConstrained = 'cpuConstrained',
}
@@ -0,0 +1,61 @@
import { useToast } from 'react-aria'
import { useRef } from 'react'
import { type ToastProps } from './Toast'
import { VStack } from '@/styled-system/jsx'
import { useTranslation } from 'react-i18next'
import { Button, Text } from '@/primitives'
import { css } from '@/styled-system/css'
import { StyledToastContainer } from './StyledToastContainer'
import { disablePerformanceMode } from '@/stores/performanceMode'
import { captureEvent } from '@/features/analytics/telemetry'
// todo - make it closable
export function ToastCpuConstrained({ state, ...props }: Readonly<ToastProps>) {
const { t } = useTranslation('notifications', {
keyPrefix: 'cpuConstrained',
})
const ref = useRef(null)
const { toastProps, contentProps } = useToast(props, state, ref)
const toast = props.toast
const handleKeepQuality = () => {
captureEvent('cpu-constrained-degradation-cancelled')
disablePerformanceMode({ declinedAuto: true })
state.close(toast.key)
}
return (
<StyledToastContainer {...toastProps} ref={ref}>
<VStack
justify="start"
alignItems="self-start"
{...contentProps}
maxWidth="370px"
gap="0.75rem"
padding={14}
>
<Text
margin={false}
className={css({
wordBreak: 'break-word',
overflowWrap: 'break-word',
whiteSpace: 'normal',
})}
>
{t('message')}
</Text>
<Button
size="sm"
variant="text"
className={css({
color: 'primary.300',
})}
onPress={() => handleKeepQuality()}
>
{t('keepQuality')}
</Button>
</VStack>
</StyledToastContainer>
)
}
@@ -15,6 +15,7 @@ import { ToastPermissionsRemoved } from './ToastPermissionsRemoved'
import { ToastRecordingRequest } from './ToastRecordingRequest'
import { ToastAutoMuteLargeRoom } from './ToastAutoMuteLargeRoom'
import { ToastRoleChanged } from '@/features/notifications/components/ToastRoleChanged'
import { ToastCpuConstrained } from './ToastCpuConstrained'
interface ToastRegionProps extends AriaToastRegionProps {
state: ToastState<ToastData>
@@ -74,6 +75,9 @@ const renderToast = (
case NotificationType.RoleChanged:
return <ToastRoleChanged key={toast.key} toast={toast} state={state} />
case NotificationType.CpuConstrained:
return <ToastCpuConstrained key={toast.key} toast={toast} state={state} />
default:
return <Toast key={toast.key} toast={toast} state={state} />
}
@@ -15,6 +15,15 @@ export const notifyAutoMutedOnJoin = () => {
)
}
export const notifyCpuConstrained = () => {
toastQueue.add(
{
type: NotificationType.CpuConstrained,
},
{ timeout: NotificationDuration.CPU_CONSTRAINED }
)
}
export const showLowerHandToast = (
participant: Participant,
onClose: () => void
@@ -0,0 +1,71 @@
import { useEffect, useRef } from 'react'
import { useRoomContext } from '@livekit/components-react'
import {
LocalTrackPublication,
LocalVideoTrack,
ParticipantEvent,
Track,
} from 'livekit-client'
import { captureEvent } from '@/features/analytics/telemetry'
import { collectHardwareSnapshot } from '@/features/analytics/hardware'
import { notifyCpuConstrained } from '@/features/notifications/utils'
import { isFireFox } from '@/utils/livekit'
import {
enablePerformanceMode,
performanceModeStore,
} from '@/stores/performanceMode'
export const CpuConstrainedObserver = () => {
const room = useRoomContext()
const degradedTracksRef = useRef(new WeakSet<LocalVideoTrack>())
useEffect(() => {
const localParticipant = room.localParticipant
const handleCpuConstrained = (
track: LocalVideoTrack,
publication: LocalTrackPublication
) => {
const { enabled, userDeclinedAuto } = performanceModeStore
const shouldDegrade =
publication.source === Track.Source.Camera &&
!enabled &&
!userDeclinedAuto &&
!degradedTracksRef.current.has(track)
void collectHardwareSnapshot().then((hardware) => {
captureEvent('cpu-constrained', {
firefox: isFireFox(),
source: publication.source,
degraded: shouldDegrade,
trackOptions: publication.options,
performance_mode_enabled: enabled,
user_declined_auto: userDeclinedAuto,
...hardware,
})
})
if (!shouldDegrade) return
degradedTracksRef.current.add(track)
enablePerformanceMode('cpu')
notifyCpuConstrained()
}
localParticipant.on(
ParticipantEvent.LocalTrackCpuConstrained,
handleCpuConstrained
)
return () => {
localParticipant.off(
ParticipantEvent.LocalTrackCpuConstrained,
handleCpuConstrained
)
}
}, [room])
return null
}
@@ -0,0 +1,108 @@
import { useEffect } from 'react'
import { useRoomContext } from '@livekit/components-react'
import {
LocalTrackPublication,
LocalVideoTrack,
ParticipantEvent,
Track,
TrackEvent,
} from 'livekit-client'
import { useSnapshot } from 'valtio'
import { reportError } from '@/features/analytics/telemetry'
import {
disablePerformanceMode,
performanceModeStore,
} from '@/stores/performanceMode'
import { degradeVideoTrack, restoreVideoTrack } from '../degradation'
/** Delay to re-apply degradation after restart to avoid racing LiveKit's encoding recompute. */
const REAPPLY_AFTER_RESTART_MS = 1_000
/** Syncs performance mode store state to outbound camera track encoding settings. */
export const PerformanceModeController = () => {
const room = useRoomContext()
const { enabled } = useSnapshot(performanceModeStore)
// Manage degradation application and track lifecycle events
useEffect(() => {
const localParticipant = room.localParticipant
const getCameraTrack = () => {
const pub = localParticipant.getTrackPublication(Track.Source.Camera)
return pub?.track instanceof LocalVideoTrack ? pub.track : null
}
const track = getCameraTrack()
// Restore track quality if performance mode is disabled
if (!enabled) {
if (track) {
restoreVideoTrack(track).catch((err) =>
reportError('performance_mode_failure', err, { action: 'restore' })
)
}
return
}
const applyDegradation = (t: LocalVideoTrack, action = 'degrade') => {
degradeVideoTrack(t).catch((err) =>
reportError('performance_mode_failure', err, { action })
)
}
// Re-apply degradation after track restarts (e.g. device/resolution changes)
const watchTrackRestart = (t: LocalVideoTrack) => {
let timeoutId: ReturnType<typeof setTimeout>
const onRestarted = () => {
clearTimeout(timeoutId)
timeoutId = setTimeout(() => {
if (performanceModeStore.enabled) applyDegradation(t, 'reapply')
}, REAPPLY_AFTER_RESTART_MS)
}
t.on(TrackEvent.Restarted, onRestarted)
return () => {
clearTimeout(timeoutId)
t.off(TrackEvent.Restarted, onRestarted)
}
}
let unwatchRestart: (() => void) | undefined
if (track) {
applyDegradation(track)
unwatchRestart = watchTrackRestart(track)
}
// Apply degradation to newly published camera tracks
const handlePublished = (pub: LocalTrackPublication) => {
if (
pub.source === Track.Source.Camera &&
pub.track instanceof LocalVideoTrack
) {
unwatchRestart?.()
applyDegradation(pub.track)
unwatchRestart = watchTrackRestart(pub.track)
}
}
localParticipant.on(ParticipantEvent.LocalTrackPublished, handlePublished)
return () => {
localParticipant.off(
ParticipantEvent.LocalTrackPublished,
handlePublished
)
unwatchRestart?.()
}
}, [room, enabled])
// Reset auto (CPU-triggered) performance mode on unmount/leave room
useEffect(() => {
return () => {
if (performanceModeStore.trigger === 'cpu') {
disablePerformanceMode()
}
}
}, [])
return null
}
@@ -0,0 +1,126 @@
import { LocalVideoTrack } from 'livekit-client'
import { isFireFox } from '@/utils/livekit'
/** Target degraded encoding for layer 0 (~360p @ 15fps, ≤300kbps). */
export const DEGRADED_MAX_HEIGHT = 360
export const DEGRADED_MAX_FRAMERATE = 15
export const DEGRADED_MAX_BITRATE = 300_000
/**
* Firefox ignores `active = false` on RTCRtpSender.
* We starve higher layers using LiveKit's sentinel workaround values instead.
*/
const FF_DISABLED_SCALE_DOWN = 4
const FF_DISABLED_MAX_BITRATE = 10
const FF_DISABLED_MAX_FRAMERATE = 2
type SavedEncoding = Pick<
RTCRtpEncodingParameters,
'active' | 'scaleResolutionDownBy' | 'maxBitrate' | 'maxFramerate'
>
/** Pre-degradation encoding snapshots keyed by track to prevent leaks. */
const savedEncodingsByTrack = new WeakMap<LocalVideoTrack, SavedEncoding[]>()
export const isTrackDegraded = (track: LocalVideoTrack) =>
savedEncodingsByTrack.has(track)
/**
* Revertibly degrades local video quality without unpublishing.
* Avoids `prioritizePerformance()` because its internal flag disables dynacast permanently.
*
* - Layer 0: Capped to 360p / 15fps / 300kbps.
* - Other layers: Set to `active: false` (or starved on Firefox).
*/
export const degradeVideoTrack = async (track: LocalVideoTrack) => {
const sender = track.sender
if (!sender) {
throw new Error('sender not found')
}
const params = sender.getParameters()
if (!params.encodings || params.encodings.length === 0) {
return
}
// Snapshot once so re-applications don't overwrite pristine encodings
if (!savedEncodingsByTrack.has(track)) {
savedEncodingsByTrack.set(
track,
params.encodings.map((e) => ({
active: e.active,
scaleResolutionDownBy: e.scaleResolutionDownBy,
maxBitrate: e.maxBitrate,
maxFramerate: e.maxFramerate,
}))
)
}
const captureHeight =
track.mediaStreamTrack.getSettings().height ?? DEGRADED_MAX_HEIGHT
params.encodings = params.encodings.map((encoding, idx) => {
if (idx === 0) {
return {
...encoding,
active: true,
scaleResolutionDownBy: Math.max(
1,
Math.ceil(captureHeight / DEGRADED_MAX_HEIGHT)
),
maxFramerate: DEGRADED_MAX_FRAMERATE,
maxBitrate: Math.min(
encoding.maxBitrate ?? DEGRADED_MAX_BITRATE,
DEGRADED_MAX_BITRATE
),
}
}
if (isFireFox()) {
const starved: RTCRtpEncodingParameters = {
...encoding,
// Firefox workaround: active=false prevents LiveKit re-encodes, while starved values limit bitrate
active: false,
scaleResolutionDownBy: FF_DISABLED_SCALE_DOWN,
maxBitrate: FF_DISABLED_MAX_BITRATE,
maxFramerate: FF_DISABLED_MAX_FRAMERATE,
}
// LiveKit legacy property fallback for Firefox
;(starved as Record<string, unknown>).maxFrameRate =
FF_DISABLED_MAX_FRAMERATE
return starved
}
return { ...encoding, active: false }
})
await sender.setParameters(params)
}
/** Restores encodings captured prior to degradation. */
export const restoreVideoTrack = async (track: LocalVideoTrack) => {
const saved = savedEncodingsByTrack.get(track)
savedEncodingsByTrack.delete(track)
const sender = track.sender
if (!saved || !sender) {
return
}
const params = sender.getParameters()
if (!params.encodings || params.encodings.length !== saved.length) {
return
}
params.encodings = params.encodings.map((encoding, idx) => {
const restored: RTCRtpEncodingParameters = {
...encoding,
...saved[idx],
}
// Clean up Firefox legacy property if set during degradation
;(restored as Record<string, unknown>).maxFrameRate = undefined
return restored
})
await sender.setParameters(params)
}
@@ -26,8 +26,8 @@ const StyledContainer = styled('div', {
backgroundColor: 'primaryDark.100',
maxWidth: '100%',
opacity: 0,
transform: 'translateY(3.25rem)',
transition: 'opacity, transform',
translate: '0 3.25rem',
transition: 'opacity, translate',
transitionDuration: '0.5s',
transitionTimingFunction: 'cubic-bezier(0.4, 0, 0.2, 1)',
pointerEvents: 'none',
@@ -36,7 +36,7 @@ const StyledContainer = styled('div', {
isVisible: {
true: {
opacity: 1,
transform: 'translateY(0)',
translate: '0 0',
pointerEvents: 'auto',
},
},
@@ -84,7 +84,7 @@ export const ReactionButtonsContainer = ({
shouldBeCenteredWithToggleButton,
setShouldBeCenteredWithToggleButton,
] = useState(false)
const [rightOffset, setRightOffset] = useState(0)
const [offsetX, setOffsetX] = useState(0)
const updateArrows = useCallback(() => {
const el = scrollRef.current
@@ -115,7 +115,7 @@ export const ReactionButtonsContainer = ({
useLayoutEffect(() => {
if (!shouldBeCenteredWithToggleButton || isMobile) {
setRightOffset(0)
setOffsetX(0)
return
}
@@ -133,7 +133,7 @@ export const ReactionButtonsContainer = ({
const containerCenterX = containerRect.left + containerRect.width / 2
const shift = toggleCenterX - containerCenterX
if (Math.abs(shift) < 0.5) return
setRightOffset((prev) => prev - shift * 2)
setOffsetX((prev) => prev + shift)
}
const schedule = () => {
@@ -182,7 +182,7 @@ export const ReactionButtonsContainer = ({
isVisible={isVisible}
style={
shouldBeCenteredWithToggleButton && !isMobile && adjustedCentering
? { marginRight: `${rightOffset}px` }
? { transform: `translateX(${offsetX}px)` }
: { margin: '0 15px' }
}
>
@@ -30,7 +30,6 @@ import { useConfig } from '@/api/useConfig'
import { isFireFox } from '@/utils/livekit'
import { useIsMobile } from '@/utils/useIsMobile'
import { navigateTo } from '@/navigation/navigateTo'
import { connectionObserverStore } from '@/stores/connectionObserver'
import { PictureInPictureConference } from '@/features/pip/components/PictureInPictureConference'
import { notifyAutoMutedOnJoin } from '@/features/notifications/utils'
import { useSnapshot } from 'valtio'
@@ -247,23 +246,8 @@ export const Conference = ({
onDisconnected={(e) => {
const metadata = {
room_id: roomId,
pc_publisher: connectionObserverStore.publisher && {
...connectionObserverStore.publisher,
},
pc_subscriber: connectionObserverStore.subscriber && {
...connectionObserverStore.subscriber,
},
pc_publisher_changes_count:
connectionObserverStore.publisherChangesCount,
pc_subscriber_changes_count:
connectionObserverStore.subscriberChangesCount,
}
connectionObserverStore.publisher = null
connectionObserverStore.publisherChangesCount = 0
connectionObserverStore.subscriber = null
connectionObserverStore.subscriberChangesCount = 0
switch (e) {
case DisconnectReason.CLIENT_INITIATED:
navigateTo(
@@ -5,7 +5,6 @@ import { useTranslation } from 'react-i18next'
import { styled, VStack } from '@/styled-system/jsx'
import { Button as RACButton } from 'react-aria-components'
import { useIsAnalyticsEnabled } from '@/features/analytics/hooks/useIsAnalyticsEnabled'
import type { CandidateInfo } from '@/stores/connectionObserver'
import { captureEvent } from '@/features/analytics/telemetry'
const Card = styled('div', {
@@ -240,10 +239,6 @@ const ConfirmationMessage = ({ onNext }: { onNext: () => void }) => {
type RatingMetadata = {
room_id?: string
pc_publisher?: CandidateInfo
pc_subscriber?: CandidateInfo
pc_publisher_changes_count?: number
pc_subscriber_changes_count?: number
}
export const Rating = ({
@@ -9,14 +9,9 @@ import { useSnapshot } from 'valtio'
import { DisconnectReason, RoomEvent } from 'livekit-client'
import { userPreferencesStore } from '@/stores/userPreferences'
import { connectionObserverStore } from '@/stores/connectionObserver'
import { useFeatureFlagEnabled } from 'posthog-js/react'
import { isMobileBrowser } from '@livekit/components-core'
import { FeatureFlags } from '@/features/analytics/enums'
import { captureEvent, captureMediaEvent } from '@/features/analytics/telemetry'
const CANDIDATE_POLL_INTERVAL_MS = 5000
import { connectionObserverStore } from '@/stores/connectionObserver'
export const ConnectionObserver = () => {
const room = useRoomContext()
@@ -24,13 +19,6 @@ export const ConnectionObserver = () => {
const { data } = useConfig()
const isAnalyticsEnabled = useIsAnalyticsEnabled()
const featureEnabled = useFeatureFlagEnabled(FeatureFlags.candidatePolling)
const isMobile = isMobileBrowser()
const isAdvancedConnectionObserverEnabled =
!isMobile && isAnalyticsEnabled && featureEnabled
const userPreferencesSnap = useSnapshot(userPreferencesStore)
const idleDisconnectModalTimeoutRef = useRef<ReturnType<
@@ -80,100 +68,6 @@ export const ConnectionObserver = () => {
userPreferencesSnap.is_idle_disconnect_modal_enabled,
])
useEffect(() => {
if (!isAdvancedConnectionObserverEnabled) return
if (!room) return
let interval: ReturnType<typeof setInterval> | null = null
const pollCandidate = async (
label: 'publisher' | 'subscriber',
// eslint-disable-next-line @typescript-eslint/no-explicit-any
pc?: any
) => {
if (!pc) return
let stats: RTCStatsReport
try {
stats = await pc.getStats()
} catch {
return
}
// eslint-disable-next-line @typescript-eslint/no-explicit-any
stats.forEach((report: any) => {
if (
report.type === 'candidate-pair' &&
report.state === 'succeeded' &&
report.nominated
) {
const remoteCandidate = stats.get(report.remoteCandidateId)
if (!remoteCandidate) return
const next = {
type: remoteCandidate.candidateType,
address: remoteCandidate.address,
protocol: remoteCandidate.protocol,
}
const current = connectionObserverStore[label]
const hasChanged =
current?.type !== next.type ||
current?.address !== next.address ||
current?.protocol !== next.protocol
if (hasChanged) {
connectionObserverStore[label] = next
const key = `${label}ChangesCount` as const
connectionObserverStore[key] =
(connectionObserverStore[key] || 0) + 1
}
}
})
}
const poll = async () => {
const publisher = room.engine?.pcManager?.publisher
const subscriber = room.engine?.pcManager?.subscriber
await Promise.all([
pollCandidate('publisher', publisher),
pollCandidate('subscriber', subscriber),
])
}
const startPolling = async () => {
if (interval) return // prevent duplicates
// Initial snapshot
await poll()
interval = setInterval(poll, CANDIDATE_POLL_INTERVAL_MS)
}
const stopPolling = () => {
if (!interval) return
clearInterval(interval)
interval = null
}
room.on(RoomEvent.Connected, startPolling)
room.on(RoomEvent.Reconnected, startPolling)
room.on(RoomEvent.Reconnecting, stopPolling)
room.on(RoomEvent.Disconnected, stopPolling)
return () => {
stopPolling()
room.off(RoomEvent.Connected, startPolling)
room.off(RoomEvent.Reconnected, startPolling)
room.off(RoomEvent.Reconnecting, stopPolling)
room.off(RoomEvent.Disconnected, stopPolling)
}
}, [room, isAdvancedConnectionObserverEnabled])
useEffect(() => {
if (!isAnalyticsEnabled) return
@@ -9,15 +9,18 @@ import {
} from 'livekit-client'
import { useSnapshot } from 'valtio'
import { userChoicesStore } from '@/stores/userChoices'
import { performanceModeStore } from '@/stores/performanceMode'
/**
* Sets initial video quality for new participants as they join.
* LiveKit doesn't allow handling video quality preferences at the room level.
*/
export const VideoResolutionSubscription = () => {
const { videoSubscribeQuality } = useSnapshot(userChoicesStore)
const { enabled: isPerformanceModeEnabled } =
useSnapshot(performanceModeStore)
const room = useRoomContext()
const effectiveQuality = isPerformanceModeEnabled
? VideoQuality.LOW
: (videoSubscribeQuality ?? VideoQuality.HIGH)
useEffect(() => {
if (!room) return
@@ -25,18 +28,12 @@ export const VideoResolutionSubscription = () => {
publication: RemoteTrackPublication,
_participant: RemoteParticipant
) => {
// By default, the maximum quality is set to high
if (
videoSubscribeQuality === undefined ||
videoSubscribeQuality === VideoQuality.HIGH
)
return
if (effectiveQuality === VideoQuality.HIGH) return
if (
publication.kind === Track.Kind.Video &&
publication.source !== Track.Source.ScreenShare
) {
publication.setVideoQuality(videoSubscribeQuality)
publication.setVideoQuality(effectiveQuality)
}
}
@@ -44,7 +41,20 @@ export const VideoResolutionSubscription = () => {
return () => {
room.off(RoomEvent.TrackPublished, handleTrackPublished)
}
}, [room, videoSubscribeQuality])
}, [room, effectiveQuality])
useEffect(() => {
if (!room) return
room.remoteParticipants.forEach((participant) => {
participant.videoTrackPublications.forEach((publication) => {
if (publication.source === Track.Source.ScreenShare) return
if (publication.videoQuality !== effectiveQuality) {
publication.setVideoQuality(effectiveQuality)
}
})
})
}, [room, effectiveQuality])
return null
}
@@ -12,7 +12,8 @@ import type { ToggleButtonProps } from '@/primitives/ToggleButton'
import { RiArrowDownSLine, RiArrowUpSLine } from '@remixicon/react'
import { useTranslation } from 'react-i18next'
const CONTROL_BAR_BREAKPOINT = 1100
const CONTROL_BAR_BREAKPOINT_WIDE = 1100
const CONTROL_BAR_BREAKPOINT_NARROW = 1050
const NavigationControls = ({
onPress,
@@ -65,10 +66,9 @@ export const LateralMenu = () => {
</DialogTrigger>
)
}
interface BreakpointObserverProps {
containerRef: RefObject<HTMLDivElement>
onWideChange: (isWide: boolean) => void
onWideChange: (isWide: boolean | null) => void
}
const BreakpointObserver = ({
@@ -76,7 +76,20 @@ const BreakpointObserver = ({
onWideChange,
}: BreakpointObserverProps) => {
const { width } = useSize(containerRef)
const isWide = width > CONTROL_BAR_BREAKPOINT
const [isWide, setIsWide] = useState<boolean | null>(null)
useEffect(() => {
if (!width) {
return
}
if (width > CONTROL_BAR_BREAKPOINT_WIDE) {
setIsWide(true)
} else if (width <= CONTROL_BAR_BREAKPOINT_NARROW) {
setIsWide(false)
} else {
setIsWide((prev) => (prev === null ? false : prev))
}
}, [width])
useEffect(() => {
onWideChange(isWide)
@@ -90,7 +103,7 @@ export const MoreOptions = ({
}: {
parentElement: RefObject<HTMLDivElement>
}) => {
const [isWide, setIsWide] = useState(false)
const [isWide, setIsWide] = useState<boolean | null>(null)
return (
<nav
@@ -107,7 +120,7 @@ export const MoreOptions = ({
containerRef={parentElement}
onWideChange={setIsWide}
/>
{isWide ? <NavigationControls /> : <LateralMenu />}
{isWide !== null && (isWide ? <NavigationControls /> : <LateralMenu />)}
</nav>
)
}
@@ -27,6 +27,8 @@ import { PinAnnouncer } from '@/features/layout/components/PinAnnouncer'
import { ChatProvider } from '@/features/chat/components/ChatProvider'
import { SyncDevicePreferences } from '@/features/rooms/livekit/components/SyncDevicePreferences'
import { RoomSilentMicDetector } from '@/features/rooms/components/SilentMicDetector'
import { CpuConstrainedObserver } from '@/features/performance/components/CpuConstrainedObserver'
import { PerformanceModeController } from '@/features/performance/components/PerformanceModeController'
/**
* @public
@@ -70,6 +72,8 @@ export function VideoConference({ ...props }: VideoConferenceProps) {
<MediaStateObserver />
<ChatProvider />
<VideoResolutionSubscription />
<PerformanceModeController />
<CpuConstrainedObserver />
<div
className="lk-video-conference"
{...props}
@@ -6,7 +6,6 @@ import { Rating } from '@/features/rooms/components/Rating.tsx'
import { useLocation } from 'wouter'
import { useMemo } from 'react'
import { DisconnectReason } from 'livekit-client'
import type { CandidateInfo } from '@/stores/connectionObserver'
// fixme - duplicated with home, refactor in a proper style
const Heading = styled('h1', {
@@ -48,10 +47,6 @@ const FeedbackRoute = () => {
const state = window.history.state
return {
room_id: state?.room_id as string,
pc_publisher: state?.pc_publisher as CandidateInfo,
pc_publisher_changes_count: state?.pc_publisher_changes_count as number,
pc_subscriber: state?.pc_subscriber as CandidateInfo,
pc_subscriber_changes_count: state?.pc_subscriber_changes_count as number,
}
}, [])
@@ -22,6 +22,11 @@ import {
} from '@/stores/userChoices'
import { RowWrapper } from './layout/RowWrapper'
import { useSnapshot } from 'valtio'
import {
disablePerformanceMode,
enablePerformanceMode,
performanceModeStore,
} from '@/stores/performanceMode'
export type VideoTabProps = Pick<DialogProps, 'onOpenChange'> &
Pick<TabPanelProps, 'id'>
@@ -32,7 +37,7 @@ const EMPTY_PROPS = {}
export const VideoTab = ({ id }: VideoTabProps) => {
const { t } = useTranslation('settings', { keyPrefix: 'video' })
const { localParticipant, remoteParticipants } = useRoomContext()
const { localParticipant } = useRoomContext()
const {
videoDeviceId,
@@ -41,6 +46,9 @@ export const VideoTab = ({ id }: VideoTabProps) => {
videoSubscribeQuality,
} = useSnapshot(userChoicesStore)
const { enabled: isPerformanceModeEnabled } =
useSnapshot(performanceModeStore)
const [videoElement, setVideoElement] = useState<HTMLVideoElement | null>(
null
)
@@ -85,22 +93,6 @@ export const VideoTab = ({ id }: VideoTabProps) => {
}
}
/**
* Updates video quality for all existing remote video tracks when user preference changes.
* LiveKit doesn't support setting video quality preferences at the room level for remote participants,
* so this function applies the selected quality to all existing remote video tracks.
* Hook useVideoResolutionSubscription updates quality preferences of new participants joining.
*/
const updateExistingRemoteVideoQuality = (selectedQuality: VideoQuality) => {
remoteParticipants.forEach((participant) => {
participant.videoTrackPublications.forEach((publication) => {
if (publication.videoQuality !== selectedQuality) {
publication.setVideoQuality(selectedQuality)
}
})
})
}
useEffect(() => {
let videoTrack: LocalVideoTrack | null = null
@@ -217,10 +209,13 @@ export const VideoTab = ({ id }: VideoTabProps) => {
type="select"
label={t('resolution.publish.label')}
items={resolutionItems}
selectedKey={videoPublishResolution}
selectedKey={
isPerformanceModeEnabled ? 'h360' : videoPublishResolution
}
onSelectionChange={async (key) => {
await handleVideoResolutionChange(key as VideoResolution)
}}
isDisabled={isPerformanceModeEnabled}
style={{
width: '100%',
}}
@@ -232,19 +227,43 @@ export const VideoTab = ({ id }: VideoTabProps) => {
type="select"
label={t('resolution.subscribe.label')}
items={videoQualityItems}
selectedKey={videoSubscribeQuality?.toString()}
selectedKey={
isPerformanceModeEnabled
? VideoQuality.LOW.toString()
: videoSubscribeQuality?.toString()
}
onSelectionChange={(key) => {
if (key == undefined) return
const selectedQuality = Number(String(key))
saveVideoSubscribeQuality(selectedQuality)
updateExistingRemoteVideoQuality(selectedQuality)
}}
isDisabled={isPerformanceModeEnabled}
style={{
width: '100%',
}}
/>
<></>
</RowWrapper>
<RowWrapper heading={t('performance.heading')}>
<Field
type="switch"
label={t('performance.label')}
description={t('performance.description')}
isSelected={isPerformanceModeEnabled}
onChange={(value) => {
if (value) {
enablePerformanceMode('manual')
} else {
disablePerformanceMode()
}
}}
wrapperProps={{
noMargin: true,
fullWidth: true,
}}
/>
<></>
</RowWrapper>
</TabPanel>
)
}
@@ -18,6 +18,10 @@
"auto": "Sie haben an einer großen Besprechung teilgenommen. Ihr Mikrofon wurde automatisch stummgeschaltet.",
"dismiss": "Stummschaltung aufheben"
},
"cpuConstrained": {
"message": "Ihr Gerät ist stark ausgelastet. Die Videoqualität wurde reduziert, um das Gespräch flüssig zu halten.",
"keepQuality": "Ursprüngliche Qualität beibehalten"
},
"reaction": {
"description": "{{name}} hat mit {{emoji}} reagiert"
},
@@ -70,6 +70,11 @@
}
}
},
"performance": {
"heading": "Leistung",
"label": "Leistung priorisieren",
"description": "Reduziert die Qualität Ihrer Videos, um Ihr Gerät zu entlasten. Wird bei Überlastung automatisch aktiviert. Deaktivieren Sie diese Funktion, um wieder die maximale Qualität zu erhalten."
},
"permissionsRequired": "Berechtigungen erforderlich"
},
"transcription": {
@@ -18,6 +18,10 @@
"auto": "You have joined a large meeting. Your microphone has been automatically muted.",
"dismiss": "Unmute"
},
"cpuConstrained": {
"message": "Your device is running low on processing power. Video quality has been reduced to keep the call smooth.",
"keepQuality": "Keep original quality"
},
"reaction": {
"description": "{{name}} reacted with {{emoji}}"
},
@@ -70,6 +70,11 @@
}
}
},
"performance": {
"heading": "Performance",
"label": "Prioritize performance",
"description": "Reduces the quality of your videos to take the strain off your device. Automatically activates in case of overload. Disable it to restore maximum quality."
},
"permissionsRequired": "Permissions required"
},
"transcription": {
@@ -18,6 +18,10 @@
"auto": "Vous venez de rejoindre une grande réunion. Votre micro a été automatiquement coupé.",
"dismiss": "Rétablir le micro"
},
"cpuConstrained": {
"message": "Votre appareil manque de puissance. La qualité vidéo a été réduite pour préserver la fluidité de l'appel.",
"keepQuality": "Conserver la qualité d'origine"
},
"reaction": {
"description": "{{name}} a reagi avec {{emoji}}"
},
@@ -70,6 +70,11 @@
}
}
},
"performance": {
"heading": "Performances",
"label": "Privilégier les performances",
"description": "Réduit la qualité de vos vidéos pour soulager votre appareil. Sactive automatiquement en cas de surcharge. Désactivez-la pour retrouver la qualité maximale."
},
"permissionsRequired": "Autorisations nécessaires"
},
"transcription": {
@@ -18,6 +18,10 @@
"auto": "U bent zojuist lid geworden van een grote vergadering. Uw microfoon is automatisch gedempt.",
"dismiss": "Microfoon inschakelen"
},
"cpuConstrained": {
"message": "Uw apparaat is zwaar belast. De videokwaliteit is verlaagd om het gesprek soepel te laten verlopen.",
"keepQuality": "Oorspronkelijke kwaliteit behouden"
},
"reaction": {
"description": "{{name}} reageerde met {{emoji}}"
},
@@ -70,6 +70,11 @@
}
}
},
"performance": {
"heading": "Prestaties",
"label": "Prestaties prioriteren",
"description": "Vermindert de kwaliteit van je video's om je apparaat te ontlasten. Wordt automatisch geactiveerd bij overbelasting. Schakel het uit om de maximale kwaliteit te herstellen."
},
"permissionsRequired": "Machtigingen vereist"
},
"transcription": {
@@ -1,23 +1,9 @@
import { proxy } from 'valtio'
export type CandidateInfo = {
type: string
address: string
protocol: string
}
type State = {
isIdleDisconnectModalOpen: boolean
publisher: CandidateInfo | null
publisherChangesCount: number
subscriber: CandidateInfo | null
subscriberChangesCount: number
}
export const connectionObserverStore = proxy<State>({
isIdleDisconnectModalOpen: false,
publisher: null,
publisherChangesCount: 0,
subscriber: null,
subscriberChangesCount: 0,
})
@@ -0,0 +1,30 @@
import { proxy } from 'valtio'
export type PerformanceModeTrigger = 'cpu' | 'manual'
export const performanceModeStore = proxy<{
enabled: boolean
trigger: PerformanceModeTrigger | null
userDeclinedAuto: boolean
}>({
enabled: false,
trigger: null,
userDeclinedAuto: false,
})
export const enablePerformanceMode = (trigger: PerformanceModeTrigger) => {
if (performanceModeStore.enabled) return
performanceModeStore.enabled = true
performanceModeStore.trigger = trigger
}
export const disablePerformanceMode = ({
declinedAuto = false,
}: { declinedAuto?: boolean } = {}) => {
if (declinedAuto) {
performanceModeStore.userDeclinedAuto = true
}
if (!performanceModeStore.enabled) return
performanceModeStore.enabled = false
performanceModeStore.trigger = null
}