Skip to content

perf(demo): faster native OCR start and a working OCR prewarm - #535

Closed
abrichr wants to merge 4 commits into
perf/native-ocrfrom
perf/first-run-30s
Closed

abrichr wants to merge 4 commits into
perf/native-ocrfrom
perf/first-run-30s

Conversation

@abrichr

@abrichr abrichr commented Oct 17, 2026

Copy link
Copy Markdown
Member

Stacked on #531. Review only: Flow is frozen for the monorepo move, so don't merge.

This branch tries to get the cold first run (openadapt-flow demo in a new environment with empty uv and Playwright caches) under 30 s on the Mac without weakening any check. It gets close but doesn't reliably get there; see "Where the time goes".

Changes

  • 4575119: Load Apple Vision with objc.loadBundle and NSData instead of the PyObjC Vision and Quartz wrappers. The native-ocr extra on macOS now needs only pyobjc-core and pyobjc-framework-Cocoa. A fresh first load took 0.9 s, compared with 2.5 s for import Vision, Quartz. The program-predicate contract records the new image input (png-data). The prewarm also starts before the CLI builds its parser and runs Playwright's Node driver once (node --version) so macOS checks it off the critical path.
  • 556a351: The OCR prewarm passed a 1x1 PNG constant that doesn't decode. The native engine raised, the prewarm swallowed the error, and Vision ran its first request during recording. The prewarm now builds a valid 64x16 PNG, and a test checks that it decodes.
  • 2e4dbcc (also on feat(ocr): native OCR on macOS and Windows, with no OpenCV on that path #531 as 0ea8ce5): the identifying-field rule for the OCR identity check. feat(ocr): native OCR on macOS and Windows, with no OpenCV on that path #531 has the safety comparison.

Timings (Mac, idle, bench2.sh, 3 runs each, load average at the end of each run)

Session 4:23-4:40 PM EDT, Oct 17. Cold totals include install.

Build Run 1 Run 2 Run 3
#531, cold 31.3 s (load 4.1) 30.9 s (4.4) 33.2 s (3.3)
this branch at c78abdb, cold 33.6 s (3.4) 29.4 s (4.0) 30.3 s (3.6)
this branch at 556a351, cold 30.8 s (9.0) 31.0 s (7.6) 32.3 s (6.1)
#531, warm 23.7 s (3.4) 23.9 s (3.7) 23.5 s (4.2)
this branch at c78abdb, warm 23.2 s (3.4) 22.5 s (4.2) 22.4 s (3.8)
this branch at 556a351, warm 23.5 s (7.1) 23.0 s (7.7) 22.8 s (8.3)

Cold runs split into install and demo:

Build Install Demo
#531 7.8, 7.5, 9.6 s 23.6, 23.4, 23.6 s
c78abdb 9.0, 6.9, 7.5 s 24.5, 22.5, 22.8 s
556a351 7.8, 8.4, 9.9 s 22.9, 22.7, 22.4 s

The demo is about 1 s faster than #531 at idle. The earlier, larger gain came from runs under load and mostly disappears on an idle machine. For reference, the earlier idle session (3:14-3:30 PM) measured 93.4-113.2 s cold for 2.0.0 and 43.5-63.0 s for #528.

Where the time goes

Timeline of one cold demo at idle (load 3.5), 22.2 s:

Stage Time
import and CLI start, until the demo begins 3.8 s
browser start 1.3 s
record 2.6 s
compile and certify 2.0 s
run 1 (honest app) 3.5 s
between the runs 0.5 s
run 2 (app that drops the save) 8.0 s
after run 2 0.5 s
  • Run 2 replays the same five steps as run 1 (about 3 s), then waits for the record check. The check polls the system of record for the effect's timeout_s, which defaults to 5 s. In run 2 the record never appears, so the check waits out the full window before the run halts. That wait is the evidence for the halt, so this branch doesn't shorten it. The 2 s in the tutorial is the HTTP request timeout, not the check window. Run 2 has no fixed sleeps to replace: the replayer's settle waits on page signals, and the rest is Playwright round trips and the postcondition template match.
  • The CLI start includes about 2 s of first-time imports and .pyc writes in a new environment. Starting the prewarm earlier moves some of that work around rather than removing it: with the prewarm on, the main thread waits on the GIL while the prewarm thread imports the same package, and a whole run is only about 0.6 s faster than with the prewarm off.
  • Install is mostly downloading into an empty cache and varies by about 3 s from run to run. About 216 MB of the roughly 470 MB a native install unpacks is OpenCV, onnxruntime and RapidOCR, which a native run never loads. They stay because auto still uses RapidOCR.

Options not taken

  • Run run 2 alongside run 1: an estimated 2.5-3 s saving. The shared browser is a process-wide, single-threaded object, so this needs per-thread browser ownership. Both runs would also write to the same durable-authority SQLite store at once. That store failed once under heavy load during this work.
  • A shorter record-check window for the tutorial's in-process store: up to about 3 s. It isn't shown to be equally safe in general, and it would change what the demo teaches.
  • Drop OpenCV, onnxruntime and RapidOCR from the native install: 3-4 s of download. This needs auto to use native, which feat(ocr): native OCR on macOS and Windows, with no OpenCV on that path #531 keeps opt-in because of false aborts.
  • --compile-bytecode at install: about 0.5 s net.

Checks run locally

- Load Vision.framework with objc.loadBundle and let Vision decode the PNG.
  This skips PyObjC's Vision and Quartz packages, which took about 2.5 s to
  load the first time on a new install, against 0.9 s. On 70 images (8,141
  lines) the output is identical to the wrapper version. The native-ocr
  extra on macOS now needs only pyobjc-core and pyobjc-framework-Cocoa.
- Start prewarm at the top of main() for demo and tutorial, before the
  parser imports.
- Run Playwright's node --version on a prewarm thread, so macOS checks the
  new driver binary (about 1.2 s) while Python imports the demo.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
…s thin

Apple Vision read a recorded patient band as only "01/15/1980 M Active",
dropping the MRN and the name. Every OCR budget passed on the DOB, sex and
status columns, so a different patient with the same DOB verified.

The OCR identity tier now classifies each recorded band token as an
identifier, a discriminative name or a date. For a record band (one that
holds an identifier or a date, or from which OCR dropped an identifying
field), the band verifies only if an identifier or a name matched, and a
second kind of identifying field matched as well when a matched field has
a look-alike character (l/1/I, O/0, S/5, Z/2, B/8, G/9, rn/m, cl/d, vv/w)
or a recorded identifying field went unmatched. Otherwise the tier
abstains. An identifier counts as evidence only on a byte-identical match.

The check runs after the existing mismatch and glyph checks and can only
turn verified into abstain. It is engine-independent and applies to the
salted-hash template path too, where every token counts as look-alike.

One existing test flips: a band whose trailing numeric identifier OCR
dropped used to verify on its name tokens alone and now abstains.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
The prewarm passed a hand-written 1x1 PNG constant that doesn't decode, so
the native engine raised, the error was swallowed, and Apple Vision ran its
first request (about 0.3 s) during recording instead. Build a 64x16 white
PNG with zlib at runtime and test that it decodes.

Signed-off-by: Richard Abrich <richard.abrich@mldsai.com>
@abrichr

abrichr commented Oct 17, 2026

Copy link
Copy Markdown
Member Author

Carried over to the monorepo in OpenAdaptAI/OpenAdapt#1199 (merged as ea09df2b).

@abrichr abrichr closed this Oct 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant