Skip to content

perf(demo): installed browser, one browser process, early OpenCV load - #528

Closed
abrichr wants to merge 7 commits into
mainfrom
perf/fast-first-run
Closed

abrichr wants to merge 7 commits into
mainfrom
perf/fast-first-run

Conversation

@abrichr

@abrichr abrichr commented Oct 17, 2026 •

Copy link
Copy Markdown
Member

What changed

  • Installed browser first. Browser launches use the installed Chrome or Edge through Playwright's channel option. When neither exists, Flow downloads only Chromium's headless shell (218 MB on disk, against 576 to 587 MB for the full download in 2.0.0). Headed runs still get the full build. If a channel browser is present but won't start, auto falls back to Playwright's build. OPENADAPT_FLOW_BROWSER accepts auto, chrome, msedge, or bundled.
  • One browser process per demo. The record run and both replays share one browser process. Each run still gets a fresh browser context, so no cookies, storage, or cache carry over. In 2.0.0, each close() took about 2.8 s.
  • OpenCV and OCR load during browser start. On a fresh macOS environment, the first import cv2 takes 9 to 19 s while the system verifies OpenCV's 78 MB of bundled libraries. The demo now starts that load on a background thread before it starts the browser. The browser-process code moved to backends/_browser_process.py, which doesn't import OpenCV.
  • Visible progress. On a terminal, the demo shows the current stage on one line that rewrites itself. The summary still has at most eight lines, and piped output is unchanged.
  • README. The demo command is now uvx --python 3.12 --from 'openadapt-flow[browser]' openadapt-flow demo, with pip as the alternative.

Nothing in recording, compiling, gating, or verification changed. Both runs still really execute, and the second run still halts on the system-of-record read.

Measurements

MacBook Pro (12 cores), Python 3.12. Each run uses a new virtual environment. The machine was shared with other jobs (load average 8 to 43), so single runs vary by a few seconds. "Cold" means an empty pip or uv cache and an empty PLAYWRIGHT_BROWSERS_PATH. Baseline is the published 2.0.0 installed with pip. "New" is this branch's wheel installed with uv pip. Every run printed "Done and checked" for run 1 and "Stopped" for run 2.

Case Install (s) Demo (s) Total (s)
Baseline warm 7.5 / 8.0 / 9.1 37.1 / 39.3 / 42.0 44.7 / 47.3 / 51.1
New warm 0.5 / 0.3 / 0.3 37.9 / 37.9 / 36.8 38.4 / 38.1 / 37.1
Baseline cold (577 MB download) 33.0 / 36.4 / 35.6 54.5 / 61.2 / 63.4 87.5 / 97.5 / 98.9
New cold (installed Chrome) 10.3 / 11.7 / 22.3 48.2 / 47.1 / 46.7 58.5 / 58.8 / 69.0
New cold, no Chrome (headless shell) 9.9 58.4 68.3

In a reused environment (native libraries already verified), the demo takes 19.7 to 22 s. First output appears at 0.5 s, and the first stage line at 1.6 s.

Where the time goes (fresh environment, new build)

Phase Seconds
First import cv2 (macOS library verification) 12 to 19
Playwright driver and Chrome start 0.75 to 2.1
Recording 1.8 to 2.5
Compile (12 full-frame OCR passes) 4 to 8
Run 1 3 to 4
Run 2: the effect check waits its full 5 s timeout before it refutes 8 to 12

The remaining large costs are the OpenCV first load and OCR. Removing them means replacing the OCR stack, which needs the identity evidence rerun first. This PR doesn't do that.

Signed-off-by: Richard Abrich richard.abrich@mldsai.com

Same browser on every web path

CI found that replay, resume, bench, record, the attended service and generated MCP tools still launched Playwright's own Chromium, while PlaywrightBackend picked the installed Chrome. A project pinned under one browser then replayed under the other, and the qualification environment check halted the Desktop check-it practice run (tests/test_desktop_check_it_cli_mockmed.py). All of these paths now go through launch_chromium. tests/test_console.py now expects browser setup inside the Playwright context, still on the owner thread.

The demo now launches the installed Chrome or Edge through Playwright's
channel option. Without one, it downloads only Chromium's headless shell
instead of the full browser. A channel browser that fails to start falls
back to Playwright's own build. OPENADAPT_FLOW_BROWSER selects auto,
chrome, msedge, or bundled.

The tutorial's record run and both replays share one browser process.
Each run still gets a fresh browser context. The OCR models load on a
background thread while the browser starts.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
On a fresh macOS install the first OpenCV import takes 9 to 13 seconds
while the system verifies its native libraries. The demo now starts that
load on a background thread before it starts the browser. The browser
process code moves to a module that does not import OpenCV.

The demo prints one line per stage as it starts, instead of staying
silent until both runs end.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
The demo's summary stays at most eight lines. Stage progress now
rewrites a single line on a terminal and prints nothing when output is
piped.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr
abrichr marked this pull request as ready for review October 17, 2026 04:07
@abrichr
abrichr enabled auto-merge (squash) October 17, 2026 04:07
@abrichr
abrichr disabled auto-merge October 17, 2026 04:49
replay, resume, bench, record, the attended service, and generated MCP
tools still launched Playwright's own Chromium, while PlaywrightBackend
picked the installed Chrome. A project pinned under one browser then
replayed under the other, so the qualification environment check halted
the Desktop check-it practice run. All of these paths now go through
launch_chromium.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
The attended service now picks and prepares the browser through
launch_chromium, which runs after Playwright starts. The test still checks
that every step runs on the owner thread.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr

abrichr commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Carried over into the monorepo as OpenAdaptAI/OpenAdapt#1189 (merged as e395969abee94bdd906beb7d277dc534b05c8e47). openadapt-flow now lives in OpenAdapt/packages/flow.

@abrichr abrichr closed this Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant