Skip to content

feat: capture requests - screenshots, screen recordings and logs on the web - #163

Merged
boehlerlukas merged 12 commits into
masterfrom
claude/capture-requests
Oct 1, 2026
Merged

feat: capture requests - screenshots, screen recordings and logs on the web#163
boehlerlukas merged 12 commits into
masterfrom
claude/capture-requests

Conversation

@boehlerlukas

@boehlerlukas boehlerlukas commented Oct 1, 2026 •
edited
Loading

Copy link
Copy Markdown
Contributor

What this adds

Web support for capture requests. When a workflow, an AI agent or a teammate asks for a screenshot or a screen recording, the customer answers from the widget card and the SDK captures the page. Logs requests are collected in the background.

Screenshots (GleapCaptureScreenshot)

  • Desktop Chrome / Edge (default, webScreenshotMethod: 'tab-first'): one-click "Share this tab" (getDisplayMedia with preferCurrentTab) gives exact pixels. Gleap's UI is hidden; one frame is grabbed, the tracks stop at once, and masked boxes are painted black. If the customer cancels, denies or anything fails, the SDK falls back to the in-browser render without a second prompt.
  • Everywhere else (Firefox, Safari, phones, tablets, dom-only): in-browser DOM render with modern-screenshot 4.7.0 (pinned).
    • Viewport only; off-screen boxes are cloned empty.
    • Hard limits: a 10 s deadline and a 12,000-element budget.
    • Videos are drawn as their current frame, the poster or a dark box.
    • If the render fails, the request fails cleanly with upload offered.
  • Masking: password, one-time-code and card fields; gl-mask / gl-block / rr-mask / rr-block; setReplayOptions masks; payment iframes; and the project's maskSelectors. It reaches same-origin frames and open shadow roots.
  • The customer marks up the screenshot in the widget's editor before sending.

Recordings (GleapCaptureRecorder)

  • Desktop:
    • Captures with getDisplayMedia and MediaRecorder: tab-first on Chromium, the browser's picker elsewhere. MP4 (avc3 → avc1) is preferred, WebM otherwise.
    • WebM files get their duration written in (GleapWebmDuration) so players can seek.
    • "Stop sharing" counts as Stop; an optional microphone is used if the project allows it.
    • Privacy veil: while recording, everything screenshots mask is blurred (12 px), including same-origin frames and open shadow roots. It goes up before the share picker opens and comes down on every way out.
  • Phones / tablets (no getDisplayMedia): rrweb page recording, uploaded in the webReplay shape the dashboard already plays. Every input value is masked, and a timeline digest goes to the AI.
  • "Record this page" fallback: when screen sharing doesn't start on desktop, the bar offers a page recording instead of failing.
  • Multi-page sites: a navigation ends a recording; after the reload the bar offers "Record again".
  • The session replay pauses while a capture records the page (rrweb can run one recording per page).

Capture bar (GleapCaptureUI)

  • Shape: a dark, fully rounded pill that can be dragged by its grip. It docks at the bottom on phones.
  • Recording: only the red dot, the timer, the mic (if allowed) and a round icon-only Stop.
  • Labels: all from the widget's translated capture-start.labels.
  • Tab screenshots: the bar is hidden before Chrome's share dialog opens, so it never appears in the screenshot.

Logs requests (GleapCaptureManager)

  • Delivery: received over the SDK websocket (capture-request), which announces &caps= on connect. Collected from the existing collectors (same shapes as bug reports), gzipped, posted to /logs once per request id.
  • Responsiveness: packing and serializing run in short slices; large events are compressed off the main thread.

Public API

Gleap.setCaptureEnabled(false);              // the widget offers upload only
Gleap.setRemoteLogCollectionEnabled(false);  // logs requests answered "unsupported"; no logs attached to captures

Classic recorder fixes found on the way

  • "Stop sharing" now stops the timer and the microphone.
  • A recording is cleared once the report carrying it is sent.
  • Silent crash reports no longer pick up the form's recording.

Bundle size

build/browser/index.js: 166.4 → 200.2 KB gzip (+34 KB, +20 %), mostly capture code and modern-screenshot (~8.6 KB).

The capture modules are compiled for the ES2020 browsers the bundle already needs, because rrweb ships ?? untranspiled (babel.config.js override). The minimum browser is unchanged. Lazy-loading the capture code would need CDN/publishing changes and is left for later.

Release notes

  • build/ and published/ are not in this PR. They are regenerated at release, and published/latest deploys to the CDN on push to master.
  • CSP: a site whose Content Security Policy restricts media-src needs blob: for the recording preview (docs PR).

Verification

  • npm test 480/480 (masking stays private, log kill switches, bundle serialization).
  • Browser pane, against a local stub server and a fake Messenger:
    • DOM screenshots, including a page with five problem videos (0.4 s), an 8,100-element page (0.64 s), the 10 s deadline and the element budget.
    • The tab-capture path against a fake stream.
    • The display recording with veil and its exits.
    • rrweb page recording (no input values in the upload) and the "Record this page" fallback.
    • WebM duration, verified with ffprobe on Chrome VP8/VP9 files.
    • Logs over the websocket; multi-page resume; Trusted Types; cross-origin Messenger iframe.
  • Local end-to-end with the branch Server + Messenger:
    • teammate screenshot (DOM fallback, editor, masked password and gl-mask);
    • phone page recording;
    • desktop "Record this page";
    • background logs over the websocket in about 0.2 s.

Not verified

  • Real screen sharing: the browser pane blocks getDisplayMedia. It needs a manual pass in Chrome/Edge (tab screenshot and recording), Firefox/Safari (DOM screenshot, window recording), and iOS Safari / Android Chrome (DOM screenshot, page recording).
  • Firefox and Safari specifics (the redraw cap, the about:blank sandbox).
  • Rendering limits:
    • The veil can't reach closed shadow roots or cross-origin frames (payment frames are blurred as a whole).
    • Cross-origin iframes and images without CORS render blank in the DOM fallback.
    • The AI chatbar frame gets no capture capabilities (upload only there).

The whole program: capture requests ("Show me the issue")

Workflows, AI agents (Kai, Kai Resolve, custom agents) and teammates can now ask a customer for a screenshot (with annotation) or a screen recording in the middle of a conversation, and can collect logs (console, network, custom data, environment data, events) from the customer's app in the background.

One server object, the capture request, backs all three. The customer sees a card in the widget; the SDK in their app does the capture; the result lands in the thread and resumes the workflow or wakes the agent. Background log requests travel over the SDK websocket every SDK already keeps open.

How each platform captures:

  • Web, desktop Chrome/Edge: screenshot via one-click "Share this tab" (exact pixels), in-browser render as fallback; recording via tab capture, "Record this page" fallback when sharing is blocked.
  • Web, Firefox/Safari/phones/tablets: in-browser screenshot; window/screen recording on desktop, page recording (rrweb) on phones.
  • iOS / Android apps (incl. React Native, Flutter, Capacitor): in-app capture of the app's own windows: no OS prompt and no new permissions (no ReplayKit, no MediaProjection). Recordings are H.264 MP4 built from frames.
  • Older SDKs, email, WhatsApp & co.: upload / reply-with-attachment fallback, so a request never dead-ends.

Design: https://claude.ai/artifact/5i88BeWfcsWUFhhJmPwK5V (private)

PRs in this program (one per repo):

🤖 Generated with Claude Code

boehlerlukas and others added 8 commits September 30, 2026 21:49
@boehlerlukas @claude
fix: classic screen recording stops cleanly on "Stop sharing" and is …
0e3c500 
…not uploaded twice

- "Stop sharing" in the browser ended only the screen track: the timer kept running and the
  microphone stayed open (the recorder went on recording audio). The track's end now stops the
  recording like the Stop button, releasing the timer and every track.
- The recording stayed in the global data after the report that carried it was sent, so a later
  silent crash report uploaded it again. It is cleared once a report with it was sent, and silent
  reports never take the feedback form's recording along.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
feat: session replay can pause while a capture request records the page
b30b0d4 
rrweb records one session per page. A capture request's page recording (phones and tablets, where
there is no screen capture) pauses the running replay, keeping its buffer, and resumes it in a new
checkpoint afterwards. A config apply meanwhile doesn't restart it; stop() ends the pause.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
feat: capture requests — screenshots, screen recordings and log reque…
493114e 
…sts on the web

The host side of the capture-requests contract (Messenger bridge §7, web capture §8, logs §9,
APIs §10):

- capture-capabilities after every Messenger ping (screenshot, recording via getDisplayMedia or
  rrweb, microphone per flowConfig.capture.allowMicrophone); everything false with
  Gleap.setCaptureEnabled(false). The SDK websocket announces &caps=capture.screenshot,
  capture.recording,capture.logs (only what is enabled).
- capture-start hides the widget without closing it and shows a capture bar in the page: shadow
  DOM under one 0x0 rr-block/gl-block host, max z-index, draggable on desktop, a bottom sheet
  with safe-area padding on phones, keyboard accessible, reduced-motion aware, labels from the
  Messenger with English fallbacks. capture-state reports every step; Cancel releases the request.
- Screenshots: desktop Chromium grabs one frame of the current tab (preferCurrentTab; ImageCapture,
  MediaStreamTrackProcessor, <video> fallback; Gleap UI hidden, tracks stopped right away);
  otherwise, or when the tab capture is declined or fails, the visible viewport is rendered in the
  browser (modern-screenshot 4.7.0, ~8.6 kB gzip). Both paint black boxes over blocked and masked
  elements, private fields (password, OTP, card), payment frames and capture.maskSelectors, also
  in shadow roots and same-origin frames; the DOM render also hides them in the clone. JPEG 0.85,
  long edge <= 2560, sent as capture-image; capture-editor opens the widget full screen.
- Recordings: getDisplayMedia + MediaRecorder (MP4 avc3/avc1, then WebM; 2.5 Mbps; 1 s chunks;
  optional microphone; auto-stop at maxDurationSec; "Stop sharing" = Stop) or, without
  getDisplayMedia, an rrweb page recording with every input masked, uploaded as a webReplay-shaped
  page-recording.json. Preview (Send / Retake / Cancel), upload with progress, /complete with a
  timeline digest (clicks, inputs without values, navigations, console errors, failed requests).
- attachLogs: the log bundle (screenshot: at capture time; recording: its window) goes to /logs,
  gzipped with CompressionStream when available.
- Log requests (capture-request on the SDK websocket): claim, bundle from the existing collectors
  (replay only when asked and web replays are on), gzip, POST /logs; once per request id;
  answered "unsupported" with Gleap.setRemoteLogCollectionEnabled(false) or backgroundLogs off.
- Multi-page sites: the bar (or "Recording stopped because the page changed") comes back after
  a page load in the same tab; the result then opens the conversation.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
test: capture requests keep masked content private and honour the log…
a05e198 
… kill switches

- Screenshot boxes cover private fields, rr-/gl- mask and block markers, the site's replay
  options, capture.maskSelectors, payment frames, open shadow roots and same-origin frames, and
  nothing else; an invalid selector doesn't switch the others off.
- Page recordings mask every input and leave masked, blocked and maskSelectors content and the
  capture bar out; timeline labels never carry field values or masked text.
- Log bundles carry only what the request includes (replay only when asked and enabled); log
  requests are answered "unsupported" when the app or the project switched them off, posted once
  per request id, and left to the device that claimed them first.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
fix: WebM screen recordings carry their duration so players can seek …
403e9dd 
…them

MediaRecorder writes WebM as a live stream without a Duration in the Segment Info (Firefox, and
Chrome before MP4 recording), so players showed no length and couldn't seek. The finished
recording now gets the measured duration before the preview and the upload: an existing Duration
is overwritten in place; otherwise one is added to the Info, with the Info size, a known Segment
size and SeekHead positions after it adjusted (files whose SeekHead points at Cues are left
alone). Anything unexpected leaves the file as recorded; MP4 is untouched. No dependency
(~1.2 kB gzip).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
feat: offer a page recording when screen sharing doesn't start
99c391c 
When a desktop recording's getDisplayMedia is denied, blocked by a policy,
unavailable or the picker is cancelled, the capture bar stays open with
"Screen sharing didn't start. You can record this page instead." and
"Record this page" (rrweb page recording, method 'rrweb') / "Cancel".
Only a Cancel there reports the outcome: capture-state + /event 'declined'
(NotAllowedError/SecurityError) or 'failed' with the error as reason.
Labels come from capture-start.labels (recordPage / recordPageHint) with
English fallbacks. The choice survives page loads, so "Record again" after
a navigation goes straight to a page recording.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
fix: screen recordings blur what screenshots mask; the log switch als…
c4bf76a 
…o covers attached logs

A getDisplayMedia video can't be masked afterwards, so while a screen
(tab/window/screen) recording runs the live page now carries a privacy
veil: one constructed stylesheet (CSP-safe; nonce'd <style> fallback),
adopted by the document, its open shadow roots and same-origin frames,
that blurs (filter: blur(12px), plus -webkit-text-security on fields)
everything screenshots mask: rr-/gl- block and mask markers, the replay
options' classes (string or RegExp) and selectors, gleap-ignore="value"
fields, password / one-time-code / card fields, masked input kinds,
payment frames and flowConfig.capture.maskSelectors. Only the outermost
matches are blurred and never <html>/<body> (their children instead), so
the capture bar stays fixed and crisp. Content added later is covered.
It goes up inside the Start click (before the picker) and the recorder
starts only once it is painted; it comes down on stop, "Stop sharing",
max duration, cancel, fallback, errors, pagehide and destroy. Page
recordings (rrweb) and screenshots don't use it.

Gleap.setRemoteLogCollectionEnabled(false) now also keeps logs off
screenshots and recordings (attachLogs), checked at capture time and
again before posting, as iOS and Android do.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
fix: capture requests never hang or freeze the page, and give up cleanly
452fbcd 
DOM screenshots
- <video> elements no longer go to modern-screenshot (its cloneVideo waits
  for a seek that never comes for camera streams, missing sources or
  preload=none): a stand-in with the video's computed style shows the
  current frame when readable, else the poster, else a dark box.
- The render has a 10 s deadline and a 10 000-element budget; Cancel stays
  available while it runs. Giving up (or a cancel) cleans up at once:
  network logging back on, sandbox removed, widget restored; the request
  fails with the upload offered.
- The style sandbox gets its srcdoc before insertion and only accepts the
  about:srcdoc load (a synchronous about:blank load used to discard it, so
  the render never yielded). Under Trusted Types the about:blank document
  serves; failing that a stub keeps modern-screenshot from creating (and
  leaking) its own frames.
- Boxes away from the viewport are cloned empty (only where their size and
  place can't change) and their images and CSS backgrounds aren't
  downloaded; modern-screenshot's per-image redraw delay (Safari/Firefox)
  is capped at two redraws.
- Network logging pauses are counted, so overlapping renders can't leave
  it off.

Tab frames: at most 15 s with the bar hidden; Escape cancels meanwhile and
stops sharing at once; mask rects follow pinch zoom (visualViewport); no
tab capture from inside a frame.

Logs and page recordings: replay events are packed in slices (large
events through CompressionStream off the main thread) and bundles are
serialized in slices into a Blob; the /logs timeout grows with the body.

Also: Gleap banners, modals, tooltips, tours, checklists and the admin UI
stay out of captures, and their actions open nothing while the widget is
hidden for one; restored bars keep their original start time and are
dropped when the request is final or expired (GET on restore); a
fallback to page recording stops whatever screen recording started; a
MediaRecorder error keeps what was recorded and offers it; standard
rr-/gl- markers stay private when a site sets its own classes; timeline
labels respect flowConfig.capture.maskSelectors; the capture bar is built
before the widget hides; timer and event callbacks are guarded. The
capture modules compile for the browsers the bundle already needs (ES2020).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This was referenced Oct 1, 2026
Merged
Merged
Merged
Merged
Merged
boehlerlukas and others added 4 commits October 1, 2026 06:47
@boehlerlukas @claude
fix: the capture bar never shows up in a tab screenshot
49e8d84 
The bar was hidden only once sharing had started, and a tab stream can still
hand out frames painted before that, so a screenshot could show the bar with
its spinner. It is now hidden as the browser's share dialog opens (Escape
still cancels), shown again with its busy state if sharing is declined, and
the first frame is taken 120 ms after the hide is painted.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
style: round, minimal capture bar; translated drag hint; Send icon af…
c2689e2 
…ter its label

Matches the native bars: the bar and its buttons are fully rounded, Stop is a
round icon-only button (its label stays the accessible name), the recording
bar shows only the dot and the timer, the start button reads "Start", the
grip shows the translated drag hint (barDragHint) as its tooltip, and the
preview's Send carries its icon after the label.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas @claude
docs: release notes for capture requests
23103f8 
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
@boehlerlukas
Merge remote-tracking branch 'origin/master' into claude/capture-requ…
982adb8 
…ests

# Conflicts:
#	CHANGELOG.md
@boehlerlukas
boehlerlukas merged commit 4f3bb58 into master Oct 1, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@boehlerlukas