Compare commits

...

4 Commits

Author SHA1 Message Date
lebaudantoine e7d47afb49 🚸(frontend) bring raised hands into view in the carousel
A raised hand matters as much as someone speaking, but it went
unnoticed when the tile was scrolled out of the carousel. It is now
brought to the top like a new speaker and stays visible while raised.
2026-11-13 19:23:23 +01:00
lebaudantoine c80e9ef3ef 🚸(frontend) bring new speakers to the top of the carousel
In the focus layout, when someone off screen started speaking, their
tile replaced whichever tile was evicted, often at the bottom of the
column. A screen share that started did the same. Users had to scan
the whole column to find who was talking, and the spot changed every
time.

New speakers now always appear at the top, right below the local
camera and the screen shares, and the other tiles slide down by one.
The most relevant tiles are where the eye already is, and the column
moves in a predictable way.

The ordering still relies on LiveKit's sort and stable update; only
the position of the incoming tile changes. This is the enhancement
#1805 was after.
2026-11-13 18:43:20 +01:00
lebaudantoine e80dfc3305 🐛(frontend) promote the active speaker in the carousel layout
Regression: in the focus layout, a participant who started speaking
while their tile was outside the visible part of the carousel was no
longer brought into view.

Cause: 81f23aeb moved useTracks in StageLayout to `updateOnlyOn: []`
so the whole stage stops re-rendering on every ActiveSpeakersChanged.
useVisualStableUpdate only promotes speakers when its host renders, so
it needs something to wake it up. The grid layout got that with
useSpeakerPromotionTrigger in the same commit; the carousel did not,
and nothing else re-renders it on speaker events.

Fix: the carousel now uses the same tripwire. Unlike the grid, which
passes its current page, the carousel renders every tile in a scroll
container, so only the first maxVisibleTiles are passed as visible.

The tripwire also takes the full list of tiles the layout shows and
ignores speakers who have none. In the focus layout the pinned camera
is rendered by the focus view, not the carousel: without this, each
speaker event from that participant would force a render that cannot
promote anyone. The grid passes all its tracks, so its behaviour is
unchanged.

Known limit, not addressed here: when a single tile fits, it stays on
the local camera, which useVisualStableUpdate sorts first, so a remote
speaker cannot be promoted and the tripwire keeps firing for them.

This supersedes #1805, which will be closed. On top of this fix, it
pinned the active speaker to a fixed slot next to the local camera,
first with a 2s hold, a reserved slot and an eligibility check
(867a14d0), then simplified to the last remote speaker with no timer
(378e3d01). That is an enhancement rather than part of the fix, and it
is left out here to keep the regression fix small. It can come back in
a dedicated PR; it would also cover the single-tile case above.
2026-11-13 18:10:28 +01:00
lebaudantoine 2eea8e6c18 🐛(frontend) apply the sans font token on every page
Since routes are lazy loaded, the LiveKit stylesheet only ships
with the room chunk. It was the only rule applying `--fonts-sans`
to the document, so pages outside a room fell back to Panda's
preflight system font stack, ignoring custom CSS themes (e.g.
Marianne in the DINUM image) and accessibility fonts.

Point Panda's `--global-font-body` at `--fonts-sans` so the
document font follows the token regardless of which chunks are
loaded.
2026-11-13 15:06:03 +01:00
7 changed files with 373 additions and 26 deletions
+4
View File
@@ -27,6 +27,8 @@ and this project adheres to
- 🔒️(frontend) send the display name in the join request body
- 🔥(backend) drop is_administrable from unregistered room payloads
- 🍱(frontend) add Portuguese glyphs to Marianne subsetted fonts
- 🚸(frontend) bring new speakers to the top of the carousel
- 🚸(frontend) bring raised hands into view in the carousel
### Fixed
@@ -42,6 +44,8 @@ and this project adheres to
- 🔒️(docker) move frontend to nginx/nginx-unprivileged image
- 🐛(frontend) accept meeting links with query params in join dialog
- 🐛(backend) allow muting when everyone_can_mute is stored as null #1942
- 🐛(frontend) apply the sans font token on every page
- 🐛(frontend) promote the active speaker in the carousel layout
## [1.37.0] - 2026-11-01
@@ -1,8 +1,10 @@
import type { TrackReferenceOrPlaceholder } from '@livekit/components-core'
import { getScrollBarWidth } from '@livekit/components-core'
import * as React from 'react'
import { TrackLoop, useVisualStableUpdate } from '@livekit/components-react'
import { TrackLoop } from '@livekit/components-react'
import { useSize } from '@/features/rooms/livekit/hooks/useResizeObserver'
import { useCarouselTiles } from '../hooks/useCarouselTiles'
import { useSpeakerPromotionTrigger } from '../hooks/useSpeakerPromotionTrigger'
const MIN_HEIGHT = 130
const MIN_WIDTH = 140
@@ -23,8 +25,9 @@ export interface CarouselLayoutProps extends React.HTMLAttributes<HTMLMediaEleme
* The `CarouselLayout` component displays a list of tracks in a scroll container.
* It will display as many tiles as possible and overflow the rest.
* @remarks
* To ensure visual stability when tiles are reordered due to track updates,
* the component uses the `useVisualStableUpdate` hook.
* Tiles are ordered by `useCarouselTiles`: screen shares, new speakers and
* raised hands enter at the top, right below the local camera, and the other
* tiles keep their order.
* @example
* ```tsx
* const tracks = useTracks([Track.Source.Camera]);
@@ -66,7 +69,11 @@ export function CarouselLayout({
setPrevTiles(tilesThatFit)
}
const sortedTiles = useVisualStableUpdate(tracks, maxVisibleTiles)
const sortedTiles = useCarouselTiles(tracks, maxVisibleTiles)
// The carousel scrolls, so only the first tiles are guaranteed on screen.
useSpeakerPromotionTrigger(sortedTiles.slice(0, maxVisibleTiles), tracks, {
raisedHands: true,
})
React.useLayoutEffect(() => {
if (asideEl.current) {
@@ -46,7 +46,7 @@ export function GridLayout({ tracks, ...props }: GridLayoutProps) {
const { layout } = useGridLayout(gridEl, tracks.length)
const pagination = usePagination(layout.maxTiles, tracks)
useSpeakerPromotionTrigger(pagination.tracks)
useSpeakerPromotionTrigger(pagination.tracks, tracks)
useSwipe(gridEl, {
onLeftSwipe: pagination.nextPage,
@@ -0,0 +1,23 @@
import type { TrackReferenceOrPlaceholder } from '@livekit/components-core'
import { useRef } from 'react'
import { orderCarouselTiles } from '../utils/orderCarouselTiles'
/**
* Carousel counterpart of LiveKit's `useVisualStableUpdate`: same sort, same
* stable update, but screen shares and new speakers end up at the top. See
* `orderCarouselTiles`.
*
* Like `useVisualStableUpdate`, it only reorders when its host renders.
*/
export function useCarouselTiles(
tracks: TrackReferenceOrPlaceholder[],
maxVisibleTiles: number
) {
const previous = useRef<TrackReferenceOrPlaceholder[]>([])
previous.current = orderCarouselTiles(
previous.current,
tracks,
maxVisibleTiles
)
return previous.current
}
@@ -8,41 +8,65 @@ import { useEffect, useReducer, useRef } from 'react'
* Tripwire for speaker promotion.
*
* Listens to `RoomEvent.ActiveSpeakersChanged` imperatively and forces ONE
* re-render of the host component only when an active speaker has none of
* their tiles are within the visible span (`maxVisibleTiles`). That render
* re-runs `useVisualStableUpdate`, which reads live `participant.isSpeaking`
* state and performs the actual swap.
* re-render of the host when a speaker has a tile in `allTiles` but none in
* `visibleTiles`. That render re-runs the host's tile ordering, which reads
* live `participant.isSpeaking` state and performs the actual swap.
*
* Speakers already visible are ignored. Everything else costs zero React work.
* Speakers already visible cost nothing, nor do speakers the layout doesn't
* show at all (e.g. the pinned participant, rendered by the focus view).
*
* Requires the parent to NOT re-render on speaker events itself, i.e.
* With `raisedHands`, it also trips when a hand goes up or down while a
* raised hand is hidden: the hand that just went up, or the next one waiting
* for the slot of a hand that went down. Only for hosts whose ordering
* promotes raised hands; others would render for nothing.
*
* Requires the parent to NOT re-render on these events itself, i.e.
* `useTracks(..., { updateOnlyOn: [] })` upstream.
*/
export function useSpeakerPromotionTrigger(
sortedTiles: TrackReferenceOrPlaceholder[]
/** Tiles inside the span the layout guarantees to speakers. */
visibleTiles: TrackReferenceOrPlaceholder[],
/** Every tile the layout shows, visible or not. */
allTiles: TrackReferenceOrPlaceholder[],
{ raisedHands = false }: { raisedHands?: boolean } = {}
) {
const room = useRoomContext()
const [, forceRender] = useReducer((n: number) => n + 1, 0)
// Refs so the listener reads current values without re-subscribing
// and without itself being a render dependency.
const tilesRef = useRef(sortedTiles)
tilesRef.current = sortedTiles
// Read by the listeners at event time, so they never re-subscribe.
const tilesRef = useRef({ visibleTiles, allTiles })
tilesRef.current = { visibleTiles, allTiles }
useEffect(() => {
const onActiveSpeakersChanged = (speakers: Participant[]) => {
const tiles = tilesRef.current
const hiddenSpeakerExists = speakers.some(
(speaker) =>
!tiles.some((t) => t.participant.identity === speaker.identity)
const renderIfHidden = (participants: Participant[]) => {
const { visibleTiles, allTiles } = tilesRef.current
const visible = new Set(visibleTiles.map((t) => t.participant.identity))
const shown = new Set(allTiles.map((t) => t.participant.identity))
const hidden = participants.some(
({ identity }) => shown.has(identity) && !visible.has(identity)
)
if (hiddenSpeakerExists) {
if (hidden) {
forceRender()
}
}
room.on(RoomEvent.ActiveSpeakersChanged, onActiveSpeakersChanged)
return () => {
room.off(RoomEvent.ActiveSpeakersChanged, onActiveSpeakersChanged)
const onAttributesChanged = (changed: Record<string, string>) => {
if ('handRaisedAt' in changed) {
renderIfHidden(
tilesRef.current.allTiles
.map((t) => t.participant)
.filter((p) => !!p.attributes.handRaisedAt)
)
}
}
}, [room])
room.on(RoomEvent.ActiveSpeakersChanged, renderIfHidden)
if (raisedHands) {
room.on(RoomEvent.ParticipantAttributesChanged, onAttributesChanged)
}
return () => {
room.off(RoomEvent.ActiveSpeakersChanged, renderIfHidden)
room.off(RoomEvent.ParticipantAttributesChanged, onAttributesChanged)
}
}, [room, raisedHands])
}
@@ -0,0 +1,201 @@
import type { TrackReferenceOrPlaceholder } from '@livekit/components-core'
import { Track } from 'livekit-client'
import { describe, expect, it } from 'vitest'
import { orderCarouselTiles } from './orderCarouselTiles'
type Tile = TrackReferenceOrPlaceholder
// Placeholder tiles are enough: the real LiveKit sort and update run on them.
const tile = (
identity: string,
joinedAt: number,
{
source = Track.Source.Camera,
isLocal = false,
isSpeaking = false,
hasSpoken = isSpeaking,
handRaisedAt,
}: {
source?: Track.Source
isLocal?: boolean
isSpeaking?: boolean
hasSpoken?: boolean
handRaisedAt?: string
} = {}
) =>
({
participant: {
identity,
isLocal,
isSpeaking,
audioLevel: isSpeaking ? 1 : 0,
lastSpokeAt: hasSpoken ? new Date(joinedAt) : undefined,
joinedAt: new Date(joinedAt),
isCameraEnabled: false,
attributes: handRaisedAt ? { handRaisedAt } : {},
},
source,
}) as unknown as Tile
const names = (tiles: Tile[]) =>
tiles.map((t) =>
t.source === Track.Source.ScreenShare
? `${t.participant.identity}:screen`
: t.participant.identity
)
const me = tile('me', 0, { isLocal: true })
const [a, b, c, d, e] = ['a', 'b', 'c', 'd', 'e'].map((id, i) =>
tile(id, i + 1)
)
const speaking = (t: Tile) =>
tile(t.participant.identity, t.participant.joinedAt!.getTime(), {
isSpeaking: true,
})
const quietAgain = (t: Tile) =>
tile(t.participant.identity, t.participant.joinedAt!.getTime(), {
hasSpoken: true,
})
const handUp = (t: Tile, minute = 0) =>
tile(t.participant.identity, t.participant.joinedAt!.getTime(), {
isLocal: t.participant.isLocal,
handRaisedAt: new Date(Date.UTC(2026, 0, 1, 10, minute)).toISOString(),
})
const screenOf = (t: Tile) =>
tile(t.participant.identity, t.participant.joinedAt!.getTime(), {
source: Track.Source.ScreenShare,
})
describe('orderCarouselTiles', () => {
it('follows the LiveKit sort on the first render', () => {
expect(names(orderCarouselTiles([], [d, c, b, a, me], 3))).toEqual([
'me',
'a',
'b',
'c',
'd',
])
})
it('puts a new speaker right below the local camera', () => {
const previous = orderCarouselTiles([], [me, a, b, c, d, e], 4)
const next = orderCarouselTiles(previous, [me, a, b, c, d, speaking(e)], 4)
// LiveKit alone would give me, a, b, e: `e` takes the slot of `c`.
expect(names(next).slice(0, 4)).toEqual(['me', 'e', 'a', 'b'])
})
it('keeps the order of the tiles that stay visible', () => {
let tiles = orderCarouselTiles([], [me, a, b, c, d, e], 4)
tiles = orderCarouselTiles(tiles, [me, a, b, c, speaking(d), e], 4)
expect(names(tiles).slice(0, 4)).toEqual(['me', 'd', 'a', 'b'])
// `d` went quiet, `e` speaks: `d` stays ahead of `a`, `b` is evicted.
tiles = orderCarouselTiles(
tiles,
[me, a, b, c, quietAgain(d), speaking(e)],
4
)
expect(names(tiles).slice(0, 4)).toEqual(['me', 'e', 'd', 'a'])
})
it('does not move a visible participant who speaks', () => {
const previous = orderCarouselTiles([], [me, a, b, c, d], 4)
const next = orderCarouselTiles(previous, [me, a, b, speaking(c), d], 4)
expect(names(next)).toEqual(['me', 'a', 'b', 'c', 'd'])
})
it('keeps screen shares above a new speaker', () => {
const previous = orderCarouselTiles([], [me, screenOf(a), a, b, c, d], 4)
expect(names(previous).slice(0, 4)).toEqual(['me', 'a:screen', 'a', 'b'])
const next = orderCarouselTiles(
previous,
[me, screenOf(a), a, b, c, speaking(d)],
4
)
expect(names(next).slice(0, 4)).toEqual(['me', 'a:screen', 'd', 'a'])
})
it('puts a screen share that starts below the local camera', () => {
const previous = orderCarouselTiles([], [me, a, b, c, d], 4)
const next = orderCarouselTiles(previous, [me, a, b, c, d, screenOf(d)], 4)
expect(names(next).slice(0, 4)).toEqual(['me', 'd:screen', 'a', 'b'])
})
it('fills the slot of a participant who left without reordering', () => {
const previous = orderCarouselTiles([], [me, a, b, c, d], 4)
const next = orderCarouselTiles(previous, [me, b, c, d], 4)
expect(names(next)).toEqual(['me', 'd', 'b', 'c'])
})
it('brings a raised hand to the top like a speaker', () => {
const previous = orderCarouselTiles([], [me, a, b, c, d, e], 4)
const next = orderCarouselTiles(previous, [me, a, b, c, d, handUp(e)], 4)
expect(names(next).slice(0, 4)).toEqual(['me', 'e', 'a', 'b'])
})
it('keeps a raised hand visible when others speak', () => {
let tiles = orderCarouselTiles([], [me, a, b, c, d, e], 3)
tiles = orderCarouselTiles(tiles, [me, a, b, c, d, handUp(e)], 3)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'e', 'a'])
// `d` speaks: it evicts `a`, not the raised hand.
tiles = orderCarouselTiles(tiles, [me, a, b, c, speaking(d), handUp(e)], 3)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'd', 'e'])
})
it('lets the current speaker in when hands fill the carousel', () => {
let tiles = orderCarouselTiles([], [me, handUp(a, 1), handUp(b, 2), c], 3)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'a', 'b'])
// The last hand raised gives way to the speaker.
tiles = orderCarouselTiles(
tiles,
[me, handUp(a, 1), handUp(b, 2), speaking(c)],
3
)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'c', 'a'])
})
it('gives the slot of a lowered hand to the next one waiting', () => {
let tiles = orderCarouselTiles(
[],
[me, handUp(a, 1), handUp(b, 2), handUp(c, 3)],
3
)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'a', 'b'])
// `a` lowers their hand: `c` was waiting and takes over.
tiles = orderCarouselTiles(tiles, [me, a, handUp(b, 2), handUp(c, 3)], 3)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'c', 'b'])
})
it('shows the first hands raised when there are too many', () => {
const tiles = orderCarouselTiles(
[],
[me, a, handUp(b, 3), handUp(c, 1), handUp(d, 2)],
3
)
expect(names(tiles).slice(0, 3)).toEqual(['me', 'c', 'd'])
})
it('keeps the local camera first when the local hand is raised', () => {
const tiles = orderCarouselTiles([], [a, screenOf(a), b, handUp(me)], 3)
expect(names(tiles)).toEqual(['me', 'a:screen', 'a', 'b'])
})
it('is stable when run twice on the same input', () => {
const tracks = [me, a, b, c, d, speaking(e)]
const previous = orderCarouselTiles([], [me, a, b, c, d, e], 4)
const once = orderCarouselTiles(previous, tracks, 4)
expect(names(orderCarouselTiles(once, tracks, 4))).toEqual(names(once))
})
})
@@ -0,0 +1,88 @@
import {
log,
sortTrackReferences,
updatePages,
type TrackReferenceOrPlaceholder,
} from '@livekit/components-core'
import { Track } from 'livekit-client'
// LiveKit's track reference id changes when a placeholder gets its track;
// this key doesn't, so turning a camera on is not seen as a new tile.
const keyOf = (tile: TrackReferenceOrPlaceholder) =>
`${tile.participant.identity}_${tile.source}`
/** When the participant raised their hand, if it is still raised. */
const handRaisedAt = (tile: TrackReferenceOrPlaceholder) => {
if (tile.source !== Track.Source.Camera) return undefined
const time = Date.parse(tile.participant.attributes.handRaisedAt ?? '')
return Number.isNaN(time) ? undefined : time
}
/**
* LiveKit's sort knows nothing about raised hands. This moves them right
* after the tiles that must stay ahead (local camera, screen shares, whoever
* is speaking now), first raised first, so they are among the first tiles
* picked as visible for as long as the hand is up.
*/
const sortWithRaisedHands = (tracks: TrackReferenceOrPlaceholder[]) => {
const rank = (tile: TrackReferenceOrPlaceholder) => {
const { isLocal, isSpeaking } = tile.participant
if (isLocal || isSpeaking || tile.source === Track.Source.ScreenShare) {
return 0
}
return handRaisedAt(tile) === undefined ? 2 : 1
}
return sortTrackReferences(tracks).sort((a, b) => {
if (rank(a) !== rank(b)) return rank(a) - rank(b)
// Equal ranks keep LiveKit's order, except raised hands: first come first.
return rank(a) === 1 ? handRaisedAt(a)! - handRaisedAt(b)! : 0
})
}
/**
* Orders the carousel tiles; the first `maxVisibleTiles` are the visible ones.
*
* This is what LiveKit's `useVisualStableUpdate` does, plus one step:
* 1. `sortTrackReferences` decides which tiles deserve to be visible, with
* raised hands added to it, see `sortWithRaisedHands`;
* 2. `updatePages` applies that to the `previous` order with as few moves as
* possible: an incoming tile is swapped with the outgoing one;
* 3. added here: within the visible tiles, the local camera goes first, then
* the screen shares, then the speakers and raised hands that step 2 just
* brought in. The other tiles keep the order step 2 gave them.
*
* Step 2 alone leaves an incoming tile wherever the evicted one happened to
* be, often at the bottom of the column.
*/
export function orderCarouselTiles(
previous: TrackReferenceOrPlaceholder[],
tracks: TrackReferenceOrPlaceholder[],
maxVisibleTiles: number
): TrackReferenceOrPlaceholder[] {
const sorted = sortWithRaisedHands(tracks)
let stable = sorted
try {
stable = updatePages(previous, sorted, maxVisibleTiles)
} catch (error) {
log.error('Error while running updatePages(): ', error)
}
const previouslyVisible = new Set(
previous.slice(0, maxVisibleTiles).map(keyOf)
)
const rank = (tile: TrackReferenceOrPlaceholder) => {
if (tile.source === Track.Source.ScreenShare) return 1
if (tile.participant.isLocal) return 0
const justCameIn = !previouslyVisible.has(keyOf(tile))
const wantsAttention =
tile.participant.isSpeaking || handRaisedAt(tile) !== undefined
return justCameIn && wantsAttention ? 2 : 3
}
return [
// Array.prototype.sort is stable: tiles of equal rank keep their order.
...stable.slice(0, maxVisibleTiles).sort((a, b) => rank(a) - rank(b)),
...stable.slice(maxVisibleTiles),
]
}