gsadmin/rustguac
1
0
Fork 0
mirror of https://github.com/sol1/rustguac.git synced 2026-10-24 12:42:23 +00:00
Code Issues Packages Projects Releases Wiki Activity

feat(mouse): replay the moves the browser coalesced away during a drag

Browse Source 
A busy main thread does not delay pointer movement, it discards it: moves
arriving while a handler runs are merged into one, and what is dispatched
afterwards carries only the last position. A drag feels that immediately --
the path between two points is gone, so a dragged scrollbar thumb is dropped
rather than merely lagged -- while a hover or a video degrades gracefully
under the same load.

getCoalescedEvents() returns what was merged. Replaying it in order restores
the path, though not the timing: nothing running on a blocked thread can do
that, and the positions still arrive together once the thread is free. What
changes is that the remote end sees where the pointer went rather than only
where it ended up.

Deliberately additive. The mousemove listener is untouched and still delivers
every position it did before, so touch, the compatibility mouse events it
produces and the counter that swallows them all behave as they did; the
dispatched position arrives twice and the second is dropped by the
unchanged-coordinates check already in move(). Migrating this file to pointer
events outright is the tidier shape and a far larger change -- every listener
would have to move together, or synthetic events from touch would arrive with
nothing left counting them down.

Only while a button is held, because the path matters for a drag and the
endpoint is all that matters for a hover, and replaying every merged move of
an idle pointer would multiply a session's upstream mouse traffic for movement
nobody can perceive. Capped and evenly sampled, because a pointer reporting at
1000Hz behind a thread blocked for 100ms offers a hundred positions for one
drag, and answering a rendering problem with a traffic one is no answer. An
ordinary pointer never reaches the cap.

The pop-out monitor windows get the same replay. They bypass Guacamole.Mouse
entirely -- it does not track the X axis correctly in a popup -- and map the
pointer themselves, so a drag on a second head would otherwise lose its path
exactly as the first one did. The sampler moves to a static so there is one
definition of the rule rather than two that can drift, and that path needs one
thing the main display does not: it sends unconditionally, where move() drops
an unchanged position, so it keeps what it last sent and drops a repeat --
keyed on the buttons as well as the position, since a press and a release
happen at one place and are not repeats of each other. Feature-detected on the
event rather than on PointerEvent, because a pop-out window is a realm of its
own and the constructor this page can see is not the one that made the event
it is handed.

That pointer handling lives in Guacamole.MonitorPointer rather than inline in
client.html, because these windows cannot be opened without a live
multi-monitor session: while it sat in the page the only thing a test could do
was match the source text and hope, and the two things most worth checking
there are a coordinate mapping and a suppression rule -- exactly the kind that
look right in a diff and are off by a term. It takes the canvas, a function
returning this monitor's rectangle in the combined framebuffer (a call rather
than a captured value, since the layout changes under a running window) and a
function to send a state.

Writing those tests found a real defect in the wheel, which sends a momentary
press and release: it built one state object, sent it, mutated it and sent it
again, which works only while every consumer copies synchronously. It sends
two states now.

Tested rather than tried, because this path only runs once the browser has
already thrown something away: the case it exists for is the one that cannot
be reliably produced, and a broken version feels like the behaviour it
replaced. Twenty tests cover the mapping, the button mask, the repeat
suppression, the wheel's reset of it and the replay, plus a harness driving
real pointer input through the debugger with the main thread deliberately
blocked -- because the unit tests feed events shaped the way the code expects,
and that assumption is the one worth checking against a browser.

(cherry picked from commit d8ce743fe0801bd8c776281ee5541abf8040a339)
This commit is contained in:
pletch
2026-09-13 13:31:14 -04:00
committed by Dave Kempe
parent d7e4349d0b
commit 010f41d15a
8 changed files with 1118 additions and 52 deletions
Download Patch File Download Diff File Expand all files Collapse all files
static/client.html
+13 -52
View File
@@ -186,6 +186,7 @@
<script src="/guac/KeyEventInterpreter.js"></script>
<script src="/guac/Layer.js"></script>
<script src="/guac/Mouse.js"></script>
<script src="/guac/MonitorPointer.js"></script>
<script src="/guac/Parser.js"></script>
<script src="/guac/Position.js"></script>
<script src="/guac/Status.js"></script>
@@ -953,58 +954,18 @@
}, 500);
});
// Mouse: map the pointer from popup-window pixels to this
// monitor's native pixels using the canvas's LIVE on-screen
// geometry (getBoundingClientRect), then offset into the combined
// framebuffer. Reading the live rect each event avoids any stale
// cached scale/offset. Sent raw; guacd maps the combined
// coordinate back to this head.
// Native pointer handling (bypasses Guacamole.Mouse, which does
// not track the X axis correctly in a popup window). Map the raw
// client coordinate into the canvas's live on-screen rect as a
// 0..1 fraction, then into this monitor's native pixels and the
// combined framebuffer. Fraction-based, so it self-recalibrates
// on any resize.
function popupPointer(ev) {
var rect = monitorLayout[index];
if (!rect) return;
var cr = canvas.getBoundingClientRect();
var fx = cr.width > 0 ? (ev.clientX - cr.left) / cr.width : 0;
var fy = cr.height > 0 ? (ev.clientY - cr.top) / cr.height : 0;
if (fx < 0) fx = 0; else if (fx > 1) fx = 1;
if (fy < 0) fy = 0; else if (fy > 1) fy = 1;
client.sendMouseState(new Guacamole.Mouse.State({
x: rect.left + fx * rect.width,
y: rect.top + fy * rect.height,
left: (ev.buttons & 1) !== 0,
middle: (ev.buttons & 4) !== 0,
right: (ev.buttons & 2) !== 0,
up: false, down: false
}), false);
ev.preventDefault();
}
canvas.addEventListener('mousedown', popupPointer);
canvas.addEventListener('mousemove', popupPointer);
canvas.addEventListener('mouseup', popupPointer);
canvas.addEventListener('contextmenu', function(ev) { ev.preventDefault(); });
// Scroll wheel -> momentary up/down button.
canvas.addEventListener('wheel', function(ev) {
var rect = monitorLayout[index];
if (!rect) return;
var cr = canvas.getBoundingClientRect();
var fx = cr.width > 0 ? (ev.clientX - cr.left) / cr.width : 0;
var fy = cr.height > 0 ? (ev.clientY - cr.top) / cr.height : 0;
if (fx < 0) fx = 0; else if (fx > 1) fx = 1;
if (fy < 0) fy = 0; else if (fy > 1) fy = 1;
var base = { x: rect.left + fx * rect.width, y: rect.top + fy * rect.height,
left: false, middle: false, right: false, up: false, down: false };
var dir = ev.deltaY < 0 ? 'up' : 'down';
base[dir] = true;
client.sendMouseState(new Guacamole.Mouse.State(base), false);
base[dir] = false;
client.sendMouseState(new Guacamole.Mouse.State(base), false);
ev.preventDefault();
}, { passive: false });
// Pointer, wheel and context menu. Guacamole.Mouse is not
// used here -- it does not track the X axis correctly in a
// popup -- so the mapping, the repeat suppression and the
// coalesced replay live in Guacamole.MonitorPointer, where a
// test can reach them. The rect is read per event rather than
// captured, because the layout changes under a running window.
Guacamole.MonitorPointer(canvas,
function () { return monitorLayout[index]; },
function (state) {
client.sendMouseState(new Guacamole.Mouse.State(state),
false);
});
// Keyboard: forward to the same client (combined session).
var keyboard = new Guacamole.Keyboard(win.document);
static/guac/MonitorPointer.js
+178
View File
@@ -0,0 +1,178 @@
/*
* Pointer handling for a pop-out monitor window.
*
* These windows do not use Guacamole.Mouse -- it does not track the X axis
* correctly in a popup -- so they map the pointer themselves: the raw client
* coordinate into the canvas's live on-screen rect as a 0..1 fraction, then
* into this monitor's native pixels and on into the combined framebuffer.
* Fraction-based and read from getBoundingClientRect() on every event, so it
* self-recalibrates on any resize rather than caching a scale that goes stale.
*
* It lives in its own file because the alternative is untestable. There is no
* way to open one of these windows without a live multi-monitor session, so
* inline in the page the only thing a test could do is match the source text
* and hope -- and the two things most worth checking here are a coordinate
* mapping and a suppression rule, which are exactly the kind that look right
* and are off by a term.
*
* Copyright (C) 2026 Sol1 Pty Ltd. Licensed under Apache 2.0.
*/
var Guacamole = Guacamole || {};
/**
* Attaches pointer, wheel and context-menu handling to a pop-out monitor's
* canvas.
*
* @constructor
* @param {!Element} canvas
* The canvas this monitor is drawn on, in the pop-out window's document.
*
* @param {!function(): ?Object} getRect
* This monitor's rectangle within the combined framebuffer, as
* {left, top, width, height}, or null if the layout does not have one yet.
* Called per event rather than captured, because the layout changes under
* a running window.
*
* @param {!function(!Object)} sendState
* Called with a plain mouse state -- {x, y, left, middle, right, up, down}
* -- to be forwarded to the session.
*/
Guacamole.MonitorPointer = function MonitorPointer(canvas, getRect, sendState) {
/**
* The last state sent, as a key, so the same one is not sent twice.
*
* Guacamole.Mouse drops an unchanged position inside move(); this path has
* no such check and needs one, because the coalesced replay below delivers
* the dispatched position and the mousemove that follows delivers it
* again. Keyed on the buttons as well as the position: a press and a
* release happen at one place and are not repeats of each other.
*
* @private
* @type {?string}
*/
var lastSent = null;
/**
* Maps an event's client coordinate into the combined framebuffer and
* sends it, unless it is a repeat of what was last sent.
*
* @private
* @param {!Object} ev
*/
function pointer(ev) {
var rect = getRect();
if (!rect)
return;
var cr = canvas.getBoundingClientRect();
var fx = cr.width > 0 ? (ev.clientX - cr.left) / cr.width : 0;
var fy = cr.height > 0 ? (ev.clientY - cr.top) / cr.height : 0;
if (fx < 0) fx = 0; else if (fx > 1) fx = 1;
if (fy < 0) fy = 0; else if (fy > 1) fy = 1;
var state = {
x : rect.left + fx * rect.width,
y : rect.top + fy * rect.height,
left : (ev.buttons & 1) !== 0,
middle : (ev.buttons & 4) !== 0,
right : (ev.buttons & 2) !== 0,
up : false,
down : false
};
var key = state.x + ',' + state.y + ',' + ev.buttons;
if (key !== lastSent) {
lastSent = key;
sendState(state);
}
if (ev.preventDefault)
ev.preventDefault();
}
/* Replay the moves the browser merged away during a drag, on the same
* terms as Guacamole.Mouse does for the main display: a busy main thread
* discards pointer movement rather than delaying it, and a dragged
* scrollbar thumb is dropped rather than lagged. Additive -- the mousemove
* listener below is unchanged and still delivers every position it did,
* and the one they both deliver is dropped by the check above.
*
* Feature-detected on the event rather than on PointerEvent, because a
* pop-out window is a realm of its own: the constructor the page that
* opened it can see is not the one that made this event. */
canvas.addEventListener('pointermove', function (ev) {
if (ev.pointerType !== 'mouse' || !ev.buttons)
return;
if (!Guacamole.Mouse.coalescedMovement
|| typeof ev.getCoalescedEvents !== 'function')
return;
var merged = ev.getCoalescedEvents();
if (!merged || merged.length <= 1)
return;
var events = Guacamole.Mouse.sampleCoalesced(merged);
for (var i = 0; i < events.length; i++)
pointer(events[i]);
});
canvas.addEventListener('mousedown', pointer);
canvas.addEventListener('mousemove', pointer);
canvas.addEventListener('mouseup', pointer);
canvas.addEventListener('contextmenu', function (ev) {
ev.preventDefault();
});
/* Scroll wheel -> momentary up/down button. */
canvas.addEventListener('wheel', function (ev) {
var rect = getRect();
if (!rect)
return;
var cr = canvas.getBoundingClientRect();
var fx = cr.width > 0 ? (ev.clientX - cr.left) / cr.width : 0;
var fy = cr.height > 0 ? (ev.clientY - cr.top) / cr.height : 0;
if (fx < 0) fx = 0; else if (fx > 1) fx = 1;
if (fy < 0) fy = 0; else if (fy > 1) fy = 1;
var dir = ev.deltaY < 0 ? 'up' : 'down';
/* Two states, not one mutated between the sends. Sending the same
* object twice and changing it in between only works while every
* consumer copies it synchronously -- client.html does, by wrapping it
* in a Guacamole.Mouse.State -- and silently sends the wrong thing to
* one that does not. A momentary button press is exactly the shape
* where that would be missed, since the second state is the one that
* survives. */
function wheelState(pressed) {
var state = {
x : rect.left + fx * rect.width,
y : rect.top + fy * rect.height,
left : false, middle : false, right : false,
up : false, down : false
};
state[dir] = pressed;
return state;
}
sendState(wheelState(true));
sendState(wheelState(false));
/* Sent behind pointer()'s back, so what it last saw is no longer what
* the session last got. */
lastSent = null;
ev.preventDefault();
}, { passive : false });
};
static/guac/Mouse.js
+144
View File
@@ -136,6 +136,71 @@ Guacamole.Mouse = function Mouse(element) {
}, false);
/* Recovers the moves the browser merged away.
*
* A busy main thread does not delay pointer movement, it *discards* it:
* moves that arrive while a handler is running are coalesced into one, and
* what is dispatched afterwards carries only the last position. A drag
* feels that immediately -- the path between two points is gone, so a
* dragged scrollbar thumb is dropped rather than merely lagged -- while a
* hover or a video degrades gracefully under the same load, which is why
* this is worth doing for one and not the other.
*
* getCoalescedEvents() returns the moves that were merged, the dispatched
* one last. Replaying them in order restores the path. It does not restore
* the timing: nothing running on a blocked thread can, and the positions
* still arrive together once the thread is free. The remote end sees where
* the pointer went rather than only where it ended up.
*
* This is deliberately additive. The mousemove listener above is untouched
* and still delivers every position it did before, so touch, the
* compatibility mouse events it produces and the ignore_mouse counter that
* swallows them all behave exactly as they did. The dispatched position
* arrives here first and again there, and the second is dropped by the
* unchanged-coordinates check in move(). Migrating this file to pointer
* events outright would be the tidier shape and a much larger change:
* every listener would have to move together, or synthetic mouse events
* from touch would arrive with nothing left counting them down. */
if (typeof PointerEvent !== 'undefined'
&& typeof PointerEvent.prototype.getCoalescedEvents === 'function'
&& Guacamole.Mouse.coalescedMovement) {
element.addEventListener("pointermove", function(e) {
/* Touch has its own handling -- Guacamole.Touch and
* Guacamole.Mouse.Touchscreen -- and reaches the listener above
* as compatibility mouse events, which the counter below is there
* to swallow. Pen does not coalesce meaningfully. */
if (e.pointerType !== 'mouse')
return;
/* Recently touched: the listener above is counting synthetic
* events down and this must not interfere with either the count
* or what it is suppressing. */
if (ignore_mouse)
return;
/* Only while a button is held. The path matters for a drag and
* the endpoint is all that matters for a hover, and replaying
* every merged move of an idle pointer would multiply the
* upstream mouse traffic of a session that had nothing wrong with
* it. */
if (!e.buttons)
return;
var merged = e.getCoalescedEvents();
if (!merged || merged.length <= 1)
return;
var events = Guacamole.Mouse.sampleCoalesced(merged);
for (var i = 0; i < events.length; i++)
guac_mouse.move(Guacamole.Position.fromClientPosition(element,
events[i].clientX, events[i].clientY), events[i]);
}, false);
}
element.addEventListener("mousedown", function(e) {
// Do not handle if ignoring events
@@ -340,6 +405,85 @@ Guacamole.Mouse = function Mouse(element) {
};
/**
* Whether the moves the browser coalesced away during a drag are replayed, so
* that the remote end sees the path the pointer took rather than only where it
* ended up. See the pointermove listener above for what this does and does
* not recover.
*
* Settable as window.__mouseCoalesced, or as the mouseCoalesced key in
* localStorage to survive a reconnect -- which is the form that works on a
* live session, since client.html rebuilds its own URL on every launch and
* relaunch and drops anything added to it by hand.
*
* @type {!boolean}
*/
Guacamole.Mouse.coalescedMovement = (function coalescedMovement() {
if (typeof window === 'undefined')
return true;
if (window.__mouseCoalesced !== undefined)
return !!window.__mouseCoalesced;
try {
var stored = window.localStorage
&& window.localStorage.getItem('mouseCoalesced');
if (stored === 'off' || stored === 'false')
return false;
} catch (e) {
/* Storage can be blocked outright; the global still works. */
}
return true;
}());
/**
* The merged moves to replay, at most COALESCED_MAX of them, in order.
*
* A pointer reporting at 1000Hz behind a thread blocked for 100ms offers a
* hundred positions for one drag, and forwarding each as its own instruction
* would answer a rendering problem with a traffic one. Sampling evenly, and
* always keeping the last, holds the shape of the path at a bounded cost; an
* ordinary pointer never reaches the cap at all.
*
* @param {!Array} merged
* What getCoalescedEvents() returned.
*
* @returns {!Array}
* The events to replay, in order.
*/
Guacamole.Mouse.sampleCoalesced = function sampleCoalesced(merged) {
if (merged.length <= Guacamole.Mouse.COALESCED_MAX)
return merged;
var out = [];
var step = merged.length / Guacamole.Mouse.COALESCED_MAX;
for (var i = 0; i < Guacamole.Mouse.COALESCED_MAX - 1; i++)
out.push(merged[Math.floor(i * step)]);
/* The last is where the pointer actually is, so it is never the one
* dropped. */
out.push(merged[merged.length - 1]);
return out;
};
/**
* The most merged moves replayed for one dispatched move. A pointer reporting
* at 1000Hz behind a thread blocked for 100ms offers a hundred positions for a
* single drag, and forwarding each as its own instruction would answer a
* rendering problem with a traffic one. An ordinary pointer never reaches
* this.
*
* @type {!number}
*/
Guacamole.Mouse.COALESCED_MAX = 16;
/**
* The current state of a mouse, including position and buttons.
*
tests/browser/drive-pointer.mjs
+113
View File
@@ -0,0 +1,113 @@
#!/usr/bin/env node
/*
* Drives real pointer input at the pop-out monitor harness through the
* debugger, with the main thread deliberately blocked, so that the coalesced
* replay is exercised against the browser's own getCoalescedEvents().
*
* The node tests feed it events shaped the way the code expects. This checks
* the assumption underneath them: that the browser merges moves at all when
* the thread is busy, and that what it hands back replays in order and inside
* the monitor it belongs to.
*
* Needs a Chrome listening on a debugging port and the harness server:
* node tests/browser/serve.mjs
* node tests/browser/drive-pointer.mjs [--port 9333]
*/
const args = process.argv.slice(2);
const portAt = args.indexOf('--port');
const port = portAt === -1 ? 9333 : +args[portAt + 1];
async function connect(wsUrl) {
const ws = new WebSocket(wsUrl);
await new Promise((ok, bad) => { ws.onopen = ok; ws.onerror = bad; });
let next = 1;
const waiting = new Map();
ws.onmessage = (m) => {
const msg = JSON.parse(m.data);
if (msg.id && waiting.has(msg.id)) {
const { ok, bad } = waiting.get(msg.id);
waiting.delete(msg.id);
msg.error ? bad(new Error(msg.error.message)) : ok(msg.result);
}
};
return (method, params = {}) => new Promise((ok, bad) => {
const id = next++;
waiting.set(id, { ok, bad });
ws.send(JSON.stringify({ id, method, params }));
});
}
const version = await (await fetch(`http://127.0.0.1:${port}/json/version`)).json();
const browser = await connect(version.webSocketDebuggerUrl);
const { targetId } = await browser('Target.createTarget',
{ url: 'http://127.0.0.1:8099/monitor-pointer.html' });
const list = await (await fetch(`http://127.0.0.1:${port}/json/list`)).json();
const page = await connect(list.find((t) => t.id === targetId).webSocketDebuggerUrl);
await page('Runtime.enable');
await new Promise((r) => setTimeout(r, 1200));
const read = async (expression) => JSON.parse((await page('Runtime.evaluate',
{ expression, returnByValue: true })).result.value);
let failures = 0;
const check = (name, ok, detail = '') => {
if (ok) console.log(` ok ${name}`);
else { failures++; console.log(` FAIL ${name}${detail ? '\n ' + detail : ''}`); }
};
console.log('pop-out monitor pointer, real browser');
/* Press, then drag across the canvas while the main thread is busy, then
* release. The block is what makes the browser coalesce. */
await page('Input.dispatchMouseEvent', { type: 'mousePressed', x: 150, y: 130,
button: 'left', buttons: 1, clickCount: 1, pointerType: 'mouse' });
page('Runtime.evaluate', { expression: 'window.__block(150)' });
const moves = [];
for (let i = 0; i < 60; i++)
moves.push(page('Input.dispatchMouseEvent', { type: 'mouseMoved',
x: 150 + i * 10, y: 130 + i * 4, buttons: 1,
button: 'left', pointerType: 'mouse' }));
await Promise.all(moves);
await new Promise((r) => setTimeout(r, 700));
await page('Input.dispatchMouseEvent', { type: 'mouseReleased', x: 740, y: 366,
button: 'left', buttons: 0, clickCount: 1, pointerType: 'mouse' });
await new Promise((r) => setTimeout(r, 300));
const sent = await read('JSON.stringify(window.__sent)');
const maxCoalesced = await read('JSON.stringify(window.__maxCoalesced)');
console.log(` (the browser merged up to ${maxCoalesced} moves into one event;`
+ ` ${sent.length} states were sent)`);
check('the browser really did coalesce', maxCoalesced > 1,
`max coalesced was ${maxCoalesced} -- the thread was not busy enough, so`
+ ' this run proves nothing either way');
check('real pointer input produced states', sent.length > 0);
check('every position lands inside this monitor',
sent.every((s) => s.x >= 2560 && s.x <= 2560 + 1920
&& s.y >= 0 && s.y <= 1080),
JSON.stringify(sent.slice(0, 3)));
check('the drag replays in order',
sent.filter((s) => s.left)
.every((s, i, a) => i === 0 || s.x >= a[i - 1].x),
'positions went backwards, so the replay ran out of order');
check('no two consecutive states are identical',
sent.every((s, i) => i === 0
|| JSON.stringify(s) !== JSON.stringify(sent[i - 1])),
'the repeat suppression let a duplicate through');
check('the release is reported with no button held',
sent.length > 0 && sent[sent.length - 1].left === false,
JSON.stringify(sent[sent.length - 1]));
await browser('Target.closeTarget', { targetId });
console.log(failures ? `\n${failures} failure(s)` : '\nall passed');
process.exit(failures ? 1 : 0);
tests/browser/monitor-pointer.html
+69
View File
@@ -0,0 +1,69 @@
<!doctype html>
<meta charset="utf-8">
<title>pop-out monitor pointer, in a real browser</title>
<style>
body { background:#111; color:#ddd; font:13px/1.6 ui-monospace,monospace; margin:0; padding:12px; }
#screen { position:absolute; left:100px; top:80px; width:800px; height:400px;
background:#234; display:block; }
#out { position:absolute; top:500px; left:12px; white-space:pre; }
</style>
<div>drag across the blue area &mdash; real PointerEvents, real coalescing</div>
<canvas id="screen" width="1920" height="1080"></canvas>
<div id="out">&hellip;</div>
<script src="/guac/Namespace.js"></script>
<script src="/guac/Event.js"></script>
<script src="/guac/Position.js"></script>
<script src="/guac/Mouse.js"></script>
<script src="/guac/MonitorPointer.js"></script>
<script>
/*
* Guacamole.MonitorPointer against the browser's own PointerEvents, which is
* the half tests/monitor-pointer.mjs cannot reach: there the coalesced events
* are fakes shaped the way the code expects them, and whether the browser
* really merges moves under load -- and what it puts in the list when it does
* -- is its business and not something to take on trust from a stub.
*
* Driven by tests/browser/drive-pointer.mjs.
*/
const canvas = document.getElementById('screen');
/* The monitor this stands for: a second head, offset in the combined
* framebuffer, and a different size from the canvas showing it, so a mapping
* that drops a term cannot land on the right answer by luck. */
const rect = { left: 2560, top: 0, width: 1920, height: 1080 };
const sent = [];
window.__sent = sent;
window.__maxCoalesced = 0;
/* Captured ahead of the module's own listener, purely to record what the
* browser handed over. */
canvas.addEventListener('pointermove', function (e) {
if (e.getCoalescedEvents)
window.__maxCoalesced = Math.max(window.__maxCoalesced,
e.getCoalescedEvents().length);
}, true);
Guacamole.MonitorPointer(canvas, function () { return rect; }, function (state) {
sent.push({
x : Math.round(state.x), y : Math.round(state.y),
left : state.left, middle : state.middle, right : state.right,
up : state.up, down : state.down
});
document.getElementById('out').textContent =
sent.length + ' sent, max coalesced ' + window.__maxCoalesced + '\n'
+ sent.slice(-6).map(function (s) { return JSON.stringify(s); }).join('\n');
});
/* Something expensive on the main thread, so moves arriving while it runs are
* merged rather than delivered. Without this there is nothing to recover and
* the run proves nothing. */
window.__block = function (ms) {
var until = performance.now() + ms;
while (performance.now() < until) { /* deliberately */ }
};
</script>
tests/browser/serve.mjs
+69
View File
@@ -0,0 +1,69 @@
#!/usr/bin/env node
/*
* Serves the browser harnesses: the real static tree at the URLs the pages and
* any worker expect, the harness pages themselves, and -- where one is given
* -- a recording at /recording.guac.
*
* Over HTTP rather than from a file, because a worker cannot be started from a
* file:// page at all -- which is the whole thing being tested.
*
* Usage: node tests/browser/serve.mjs [recording.guac] [port]
*/
import { createServer } from 'node:http';
import { createReadStream } from 'node:fs';
import { stat } from 'node:fs/promises';
import { dirname, resolve, join, extname, normalize } from 'node:path';
import { fileURLToPath } from 'node:url';
const root = resolve(dirname(fileURLToPath(import.meta.url)), '../..');
/* The recording is optional: the pointer harness needs no session, only the
* static tree and a page. */
const recording = process.argv[2] ? resolve(process.argv[2]) : null;
const port = parseInt(process.argv[3] || '8099', 10);
const TYPES = {
'.html': 'text/html; charset=utf-8',
'.js' : 'text/javascript; charset=utf-8',
'.css' : 'text/css; charset=utf-8',
'.guac': 'text/plain; charset=utf-8'
};
createServer(async (req, res) => {
const url = new URL(req.url, 'http://localhost');
let file;
if (url.pathname === '/recording.guac' && recording)
file = recording;
else if (url.pathname === '/' || url.pathname.endsWith('.html'))
file = join(root, 'tests/browser',
url.pathname === '/' ? 'replay.html'
: url.pathname.replace(/^\//, ''));
else {
/* Only ever out of the static tree, and never up out of it. */
const rel = normalize(url.pathname).replace(/^(\.\.[/\\])+/, '');
file = join(root, 'static', rel);
if (!file.startsWith(join(root, 'static'))) {
res.writeHead(403).end('no');
return;
}
}
try {
const info = await stat(file);
res.writeHead(200, {
'content-type' : TYPES[extname(file)] || 'application/octet-stream',
'content-length' : info.size,
/* Nothing here is cached: the point is to serve what is on disk. */
'cache-control' : 'no-store'
});
createReadStream(file).pipe(res);
} catch (e) {
res.writeHead(404).end('not found: ' + url.pathname);
}
}).listen(port, '127.0.0.1', () => {
console.log(`replay harness on http://127.0.0.1:${port}/`);
console.log(` recording: ${recording || '(none given)'}`);
});
tests/monitor-pointer.mjs
+281
View File
@@ -0,0 +1,281 @@
#!/usr/bin/env node
/*
* Pins pointer handling in the pop-out monitor windows.
*
* These windows cannot be opened without a live multi-monitor session, so
* until this logic was lifted out of client.html the only thing a test could
* do was match the source text. The two things most worth checking here are a
* coordinate mapping and a suppression rule, and both are the kind that look
* right in a diff and are off by a term.
*
* The mapping matters because it is not Guacamole.Mouse's -- these windows map
* the pointer themselves, from the canvas's live on-screen rect into this
* monitor's slice of the combined framebuffer, and a wrong offset there sends
* every click to the wrong head. The suppression matters because it is the one
* place on this branch where something that used to happen no longer does:
* this path used to send unconditionally and now drops a repeat of what it
* last sent.
*
* Usage: node tests/monitor-pointer.mjs
*/
import { readFile } from 'node:fs/promises';
import { dirname, resolve, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import vm from 'node:vm';
import assert from 'node:assert/strict';
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..');
const sources = await Promise.all([
'static/guac/Namespace.js',
'static/guac/Event.js',
'static/guac/Position.js',
'static/guac/Mouse.js',
'static/guac/MonitorPointer.js'
].map((f) => readFile(join(root, f), 'utf8')));
/**
* A monitor pointer on a fake canvas, with what it sent and the listeners it
* attached both reachable.
*
* The canvas sits at (100, 50) on screen and is 800x400 there, while the
* monitor it shows is 1920x1080 native at (2560, 0) in the combined
* framebuffer -- a second head, offset, and scaled down on screen. Deliberately
* none of those numbers are the same, so a mapping that drops a term or uses
* the wrong rect cannot land on the right answer by luck.
*/
function mounted({ rect = { left: 2560, top: 0, width: 1920, height: 1080 } } = {}) {
const sent = [];
const listeners = {};
const canvas = {
addEventListener : (name, fn) => { (listeners[name] ||= []).push(fn); },
getBoundingClientRect : () => ({ left: 100, top: 50, width: 800, height: 400 })
};
const sandbox = {
console : { log() {}, warn() {}, error() {} },
document : { body: {}, documentElement: {},
createElement: () => ({ style: { cursor: '' } }) },
Math, Object, Array,
PointerEvent : class { getCoalescedEvents() { return []; } }
};
sandbox.window = sandbox;
sandbox.window.localStorage = null;
sandbox.window.location = { search: '' };
vm.createContext(sandbox);
for (const src of sources)
vm.runInContext(src, sandbox);
let current = rect;
sandbox.Guacamole.MonitorPointer(canvas, () => current,
(state) => sent.push(state));
return {
sandbox, sent, listeners,
setRect : (r) => { current = r; },
fire : (name, ev) => (listeners[name] || []).forEach((fn) => fn(ev)),
has : (name) => !!listeners[name]
};
}
/** A DOM-ish event at a screen position. */
function at(clientX, clientY, buttons = 0, extra = {}) {
return Object.assign({
clientX, clientY, buttons,
preventDefault() { this.defaultPrevented = true; }
}, extra);
}
/** A pointermove carrying the positions the browser merged into it. */
function merged(positions, { buttons = 1, pointerType = 'mouse' } = {}) {
const events = positions.map(([x, y]) => at(x, y, buttons));
const last = events[events.length - 1];
return at(last.clientX, last.clientY, buttons, {
pointerType, getCoalescedEvents : () => events
});
}
let failures = 0;
const test = (name, fn) => {
try { fn(); console.log(` ok ${name}`); }
catch (e) { failures++; console.log(` FAIL ${name}\n ${e.message}`); }
};
console.log('pop-out monitor pointer');
/* ── The mapping ─────────────────────────────────────────────────────── */
test('the centre of the canvas is the centre of that monitor', () => {
const m = mounted();
m.fire('mousemove', at(100 + 400, 50 + 200));
assert.deepEqual([m.sent[0].x, m.sent[0].y], [2560 + 960, 540]);
});
test('the origin of the canvas is the origin of that monitor', () => {
const m = mounted();
m.fire('mousemove', at(100, 50));
assert.deepEqual([m.sent[0].x, m.sent[0].y], [2560, 0],
"a second head's offset must be added, not assumed to be zero");
});
test('a position outside the canvas is clamped to it', () => {
const m = mounted();
m.fire('mousemove', at(-500, -500));
assert.deepEqual([m.sent[0].x, m.sent[0].y], [2560, 0]);
m.fire('mousemove', at(99999, 99999));
assert.deepEqual([m.sent[1].x, m.sent[1].y], [2560 + 1920, 1080]);
});
test('the layout is read per event, not captured when the window opened', () => {
const m = mounted();
m.fire('mousemove', at(500, 250));
m.setRect({ left: 0, top: 1080, width: 3840, height: 2160 });
m.fire('mousemove', at(500, 250));
assert.notDeepEqual([m.sent[0].x, m.sent[0].y], [m.sent[1].x, m.sent[1].y],
'a monitor moved in the layout must move on the wire');
});
test('nothing is sent while the layout has no rectangle for this monitor', () => {
const m = mounted({ rect: null });
m.fire('mousemove', at(500, 250));
m.fire('wheel', at(500, 250, 0, { deltaY: -1 }));
assert.equal(m.sent.length, 0);
});
/* ── The buttons ─────────────────────────────────────────────────────── */
test('the button mask is decoded, not passed through', () => {
const m = mounted();
/* DOM buttons: 1 left, 2 RIGHT, 4 middle -- which is not the order the
* Guacamole state uses, and transposing the middle and right is the
* classic way to get this wrong. */
m.fire('mousedown', at(500, 250, 1));
m.fire('mousedown', at(500, 250, 2));
m.fire('mousedown', at(500, 250, 4));
assert.deepEqual(m.sent.map(s => [s.left, s.middle, s.right]),
[[true, false, false], [false, false, true], [false, true, false]]);
});
/* ── The suppression ─────────────────────────────────────────────────── */
test('the same state is not sent twice', () => {
const m = mounted();
m.fire('mousemove', at(500, 250));
m.fire('mousemove', at(500, 250));
m.fire('mousemove', at(500, 250));
assert.equal(m.sent.length, 1);
});
test('a press and a release at one position are not repeats of each other', () => {
const m = mounted();
m.fire('mousedown', at(500, 250, 1));
m.fire('mousemove', at(500, 250, 1));
m.fire('mouseup', at(500, 250, 0));
assert.deepEqual(m.sent.map(s => s.left), [true, false],
'the move between them is the repeat; the release is not');
});
test('the wheel clears the suppression it sends behind the back of', () => {
const m = mounted();
m.fire('mousemove', at(500, 250));
const before = m.sent.length;
m.fire('wheel', at(500, 250, 0, { deltaY: -1 }));
assert.equal(m.sent.length, before + 2, 'a wheel click is press then release');
assert.equal(m.sent[before].up, true);
assert.equal(m.sent[before + 1].up, false);
/* Without the reset this position is still what pointer() thinks it last
* sent, and the pointer would go silent at the spot the user scrolled. */
m.fire('mousemove', at(500, 250));
assert.equal(m.sent.length, before + 3,
'the position the wheel moved away from must be sendable again');
});
test('the wheel direction follows the sign of the delta', () => {
const m = mounted();
m.fire('wheel', at(500, 250, 0, { deltaY: -1 }));
m.fire('wheel', at(500, 250, 0, { deltaY: 1 }));
assert.deepEqual(m.sent.map(s => [s.up, s.down]),
[[true, false], [false, false], [false, true], [false, false]]);
});
/* ── The coalesced replay ────────────────────────────────────────────── */
test('a drag replays every position the browser merged away', () => {
const m = mounted();
m.fire('pointermove', merged([[200, 100], [300, 150], [400, 200], [500, 250]]));
assert.equal(m.sent.length, 4);
assert.deepEqual([m.sent[0].x, m.sent[0].y], [2560 + 240, 135]);
assert.deepEqual([m.sent[3].x, m.sent[3].y], [2560 + 960, 540]);
});
test('and the mousemove that follows it is not sent again', () => {
const m = mounted();
m.fire('pointermove', merged([[200, 100], [500, 250]]));
m.fire('mousemove', at(500, 250, 1));
assert.equal(m.sent.length, 2);
});
test('a hover replays nothing', () => {
const m = mounted();
m.fire('pointermove', merged([[200, 100], [500, 250]], { buttons: 0 }));
assert.equal(m.sent.length, 0);
});
test('touch is left to the handling it already has', () => {
const m = mounted();
m.fire('pointermove', merged([[200, 100], [500, 250]], { pointerType: 'touch' }));
assert.equal(m.sent.length, 0);
});
test('a move that merged nothing is left to the mousemove listener', () => {
const m = mounted();
m.fire('pointermove', merged([[500, 250]]));
assert.equal(m.sent.length, 0);
});
test('a flood is bounded by the same rule the main display uses', () => {
const m = mounted();
const max = m.sandbox.Guacamole.Mouse.COALESCED_MAX;
const flood = [];
for (let i = 1; i <= max * 8; i++) flood.push([100 + i, 50 + i]);
m.fire('pointermove', merged(flood));
assert.equal(m.sent.length, max);
});
test('the override switches the replay off and leaves the rest alone', () => {
const m = mounted();
m.sandbox.Guacamole.Mouse.coalescedMovement = false;
m.fire('pointermove', merged([[200, 100], [500, 250]]));
assert.equal(m.sent.length, 0, 'no replay');
m.fire('mousemove', at(500, 250, 1));
assert.equal(m.sent.length, 1, 'and the ordinary path still works');
});
/* ── The rest of the contract ────────────────────────────────────────── */
test('the context menu is suppressed, or right-click never reaches the host', () => {
const m = mounted();
assert.ok(m.has('contextmenu'));
const ev = at(500, 250, 2);
m.fire('contextmenu', ev);
assert.ok(ev.defaultPrevented);
});
test('the browser is stopped from acting on what it is given', () => {
const m = mounted();
const move = at(500, 250, 1);
m.fire('mousemove', move);
assert.ok(move.defaultPrevented, 'or the popup selects and drags its own canvas');
const wheel = at(500, 250, 0, { deltaY: -1 });
m.fire('wheel', wheel);
assert.ok(wheel.defaultPrevented, 'or the popup scrolls itself');
});
console.log(failures ? `\n${failures} failure(s)` : '\nall passed');
process.exit(failures ? 1 : 0);
tests/mouse-coalesced.mjs
+251
View File
@@ -0,0 +1,251 @@
#!/usr/bin/env node
/*
* Pins the recovery of coalesced pointer movement.
*
* This path only runs when the browser has already discarded something: moves
* merged away behind a busy main thread. That makes it close to untestable by
* using the application -- the case it exists for is the one you cannot
* reliably produce, and when it is broken the symptom is a drag that feels
* slightly worse, which is what it felt like before. So the behaviour is
* pinned here instead.
*
* Three things matter and none of them is obvious from reading it. The
* dispatched position must be replayed along with the merged ones, or a
* browser that fires no compatibility mousemove leaves the pointer one event
* behind for the whole drag. Nothing may be replayed unless a button is held,
* or an idle pointer multiplies a session's upstream mouse traffic for
* movement nobody can perceive. And the existing mousemove listener must keep
* working exactly as it did, because touch reaches it as compatibility events
* and a counter it shares is what swallows them.
*
* Usage: node tests/mouse-coalesced.mjs
*/
import { readFile } from 'node:fs/promises';
import { dirname, resolve, join } from 'node:path';
import { fileURLToPath } from 'node:url';
import vm from 'node:vm';
import assert from 'node:assert/strict';
const root = resolve(dirname(fileURLToPath(import.meta.url)), '..');
const sources = await Promise.all([
'static/guac/Namespace.js',
'static/guac/Event.js',
'static/guac/Position.js',
'static/guac/Mouse.js'
].map((f) => readFile(join(root, f), 'utf8')));
const monitorPointer = await readFile(
join(root, 'static/guac/MonitorPointer.js'), 'utf8');
/**
* A Guacamole.Mouse on a fake element, with the listeners it attached and the
* mousemove events it emitted both reachable.
*/
function mounted({ globals = {}, pointerEvents = true } = {}) {
const listeners = {};
const element = {
offsetLeft : 0,
offsetTop : 0,
offsetParent : null,
addEventListener : (name, fn) => { (listeners[name] ||= []).push(fn); }
};
const sandbox = {
console : { log() {}, warn() {}, error() {} },
document : {
body : {},
documentElement : {},
/* Only the CSS3 cursor probe in the constructor wants this. */
createElement : () => ({ style: { cursor: '' } })
},
setTimeout, clearTimeout, Math, Array, Object
};
sandbox.window = Object.assign(sandbox, globals);
if (pointerEvents)
sandbox.PointerEvent = class { getCoalescedEvents() { return []; } };
vm.createContext(sandbox);
for (const src of sources)
vm.runInContext(src, sandbox);
const mouse = new sandbox.Guacamole.Mouse(element);
const moves = [];
mouse.onEach(['mousemove'], (e) => moves.push([e.state.x, e.state.y]));
return {
sandbox, mouse, moves, listeners,
fire : (name, event) => (listeners[name] || []).forEach((fn) => fn(event)),
has : (name) => !!listeners[name]
};
}
/** A DOM event, near enough: the suppression path cancels what it swallows. */
function domEvent(props) {
return Object.assign({
stopPropagation() {},
preventDefault() {},
returnValue : true
}, props);
}
/** A pointermove carrying the positions the browser merged into it. */
function pointerMove(positions, { buttons = 1, pointerType = 'mouse' } = {}) {
const merged = positions.map(([x, y]) => ({ clientX: x, clientY: y }));
const last = merged[merged.length - 1];
return domEvent({
pointerType, buttons,
clientX : last.clientX,
clientY : last.clientY,
getCoalescedEvents : () => merged
});
}
let failures = 0;
const test = (name, fn) => {
try { fn(); console.log(` ok ${name}`); }
catch (e) { failures++; console.log(` FAIL ${name}\n ${e.message}`); }
};
console.log('coalesced pointer movement');
test('a drag replays every position the browser merged away', () => {
const m = mounted();
m.fire('pointermove', pointerMove([[10, 10], [20, 22], [30, 35], [40, 50]]));
assert.deepEqual(m.moves.map(p => [...p]),
[[10, 10], [20, 22], [30, 35], [40, 50]]);
});
test('the dispatched position is replayed too, not assumed to follow', () => {
/* A browser that fires no compatibility mousemove would otherwise leave
* the pointer one event behind for the length of the drag. */
const m = mounted();
m.fire('pointermove', pointerMove([[1, 1], [9, 9]]));
assert.deepEqual([...m.moves[m.moves.length - 1]], [9, 9]);
});
test('and the mousemove that follows it is not sent twice', () => {
const m = mounted();
const move = pointerMove([[1, 1], [9, 9]]);
m.fire('pointermove', move);
m.fire('mousemove', domEvent({ clientX: 9, clientY: 9 }));
assert.equal(m.moves.length, 2, 'the repeat is dropped as unchanged');
});
test('a hover replays nothing', () => {
const m = mounted();
m.fire('pointermove', pointerMove([[1, 1], [5, 5], [9, 9]], { buttons: 0 }));
assert.equal(m.moves.length, 0);
});
test('touch is left to the handling it already has', () => {
const m = mounted();
m.fire('pointermove', pointerMove([[1, 1], [9, 9]], { pointerType: 'touch' }));
assert.equal(m.moves.length, 0);
});
test('a move that merged nothing is left alone', () => {
const m = mounted();
m.fire('pointermove', pointerMove([[7, 7]]));
assert.equal(m.moves.length, 0, 'the mousemove listener delivers this one');
});
test('the mousemove listener still delivers on its own', () => {
const m = mounted();
m.fire('mousemove', domEvent({ clientX: 3, clientY: 4 }));
assert.deepEqual([...m.moves[0]], [3, 4]);
});
test('synthetic events after touch are still swallowed, and still counted', () => {
const m = mounted();
m.fire('touchstart', domEvent({}));
/* Whatever the counter is set to, a real pointermove must not sneak past
* it -- and must not decrement it either, or the listener that is counting
* is left counting the wrong thing. */
m.fire('pointermove', pointerMove([[1, 1], [9, 9]]));
assert.equal(m.moves.length, 0, 'a drag must not slip past the counter');
for (let i = 0; i < m.mouse.touchMouseThreshold; i++)
m.fire('mousemove', domEvent({ clientX: i, clientY: i }));
assert.equal(m.moves.length, 0, 'the synthetic events are swallowed');
m.fire('mousemove', domEvent({ clientX: 50, clientY: 60 }));
assert.deepEqual([...m.moves[0]], [50, 60], 'and normal service resumes');
});
test('a flood is sampled to a bounded number, keeping the last', () => {
const m = mounted();
const max = m.sandbox.Guacamole.Mouse.COALESCED_MAX;
/* From 1, not 0: a mouse starts at (0,0) and move() drops a position it
* is already at, which would cost this test its first sample for reasons
* that have nothing to do with sampling. */
const flood = [];
for (let i = 1; i <= max * 8; i++)
flood.push([i, i * 2]);
m.fire('pointermove', pointerMove(flood));
assert.equal(m.moves.length, max, `expected ${max}, got ${m.moves.length}`);
assert.deepEqual([...m.moves[m.moves.length - 1]],
flood[flood.length - 1], 'the last position is where the pointer is');
assert.deepEqual([...m.moves[0]], flood[0], 'and the first is where it was');
/* Monotonic, so the path still reads as a path. */
for (let i = 1; i < m.moves.length; i++)
assert.ok(m.moves[i][0] > m.moves[i - 1][0], 'sampled out of order');
});
test('the override switches it off, and the mousemove path is untouched', () => {
const m = mounted({ globals: { __mouseCoalesced: false } });
assert.equal(m.has('pointermove'), false, 'no listener is attached at all');
m.fire('mousemove', domEvent({ clientX: 3, clientY: 4 }));
assert.deepEqual([...m.moves[0]], [3, 4]);
});
test('a browser without pointer events keeps the behaviour it had', () => {
const m = mounted({ pointerEvents: false });
assert.equal(m.has('pointermove'), false);
m.fire('mousemove', domEvent({ clientX: 3, clientY: 4 }));
assert.deepEqual([...m.moves[0]], [3, 4]);
});
/* ── The pop-out monitor windows ─────────────────────────────────────── */
/*
* Those windows bypass Guacamole.Mouse entirely -- it does not track the X
* axis correctly in a popup -- and map the pointer themselves. That code is
* exercised directly by tests/monitor-pointer.mjs, which is possible only
* because it lives in Guacamole.MonitorPointer rather than inline in the page.
* What is left here is the join between them: the popup path must be bounded
* by the same rule and switched by the same override as the main display, or
* the two drift and only one of them answers the setting.
*/
const page = await readFile(join(root, 'static/client.html'), 'utf8');
test('the page hands its pop-out windows to the shared implementation', () => {
assert.ok(page.includes('Guacamole.MonitorPointer('),
'client.html must use the extracted pointer handling');
assert.ok(page.includes('src="/guac/MonitorPointer.js"'),
'and must actually load it, or the call is a ReferenceError');
});
test('the popup path is gated and bounded by the main display\'s own rules', () => {
const sandbox = { Guacamole: {}, console: { log() {}, warn() {} } };
vm.createContext(sandbox);
vm.runInContext(sources[3], sandbox);
vm.runInContext(monitorPointer, sandbox);
assert.equal(typeof sandbox.Guacamole.MonitorPointer, 'function');
assert.ok(monitorPointer.includes('Guacamole.Mouse.coalescedMovement'),
'the same override must reach both, or one ignores the setting');
assert.ok(monitorPointer.includes('Guacamole.Mouse.sampleCoalesced'),
'and the same cap, or the two drift apart');
});
console.log(failures ? `\n${failures} failure(s)` : '\nall passed');
process.exit(failures ? 1 : 0);