feat: browser agent fixes from live tests on ten utility sites

[YellowSnnowmann]

Summary

These fixes come from live tests of the browser agent on real Indian utility sites: Uber, Ola, Amazon, Myntra, BookMyShow, BigBasket (round 1), then Flipkart, Lenskart, Blinkit, and Zepto with the same sites again (round 2). Each fix is generic: no hostnames, site labels, or per-site steps anywhere in the engine, sight, or the planner guide.

After round 2, Amazon, Myntra, Zepto, Flipkart, Lenskart, and Blinkit run end to end to the cart or the payment checkpoint, and Uber reaches its final "Request" button after the person signs in. Round 3 (cab booking, a grocery basket, cinema seats) takes Ola to its "Confirm & Book" gate, BigBasket to its OTP login with the right item ×2 in the basket, and BookMyShow through date, format, show time, and seat count to two adjacent seats chosen in the cheapest section ("Pay ₹400").

The whole PR was then reviewed end to end (section "Review" below): four reviewers read the full diff, every finding was checked against the code, and each real one was fixed with a test that fails without it, including three blockers (two ways a secret could leak, and a dialog that stayed "the task's own" for the rest of the run). No code path names a host, a site, or one site's labels.

Forms and suggestions

1. enter picks the suggestion an autocomplete box lists.
   • A location or city box keeps its text only once a listed match is chosen. Before, the next step pressed Escape on the open list and the box emptied.
   • Now enter looks again after typing and picks the new row that matches. If several rows match, Jev chooses among at most 12 of them, and it may answer that none fits.
   • Live follow-up: on a store's search box, the completions are other searches. A pick of "boat airdopes 141 anc" at 0.43 (a near tie with "none fits") changed the search. A suggestion is now pressed only at 0.5 or more (SUGGESTION_FLOOR), and a lone row is pressed without asking only when it reads exactly as typed.
2. A panel that strings its suggestions together is never pressed.
   • A store's delivery-area popover was read as one button whose name held every row. It was the only new element mentioning the pincode, so it was pressed.
   • The press landed on the row at its middle and set a different area. A label that lists far more than the typed text is no longer a suggestion.
3. Sight reads the rows inside a panel.
   • Any element with a tab stop or a pointer cursor became a button, and every element inside it was skipped. That swallowed the popover's rows, and a tab panel's rows the same way.
   • A region role (menu, list, grid, tab panel, dialog, landmark) no longer counts as a control just for its tab stop. One the page makes pressable itself, such as a carousel slide with a pointer cursor, still does. A button or link that holds a box to type in is now read as a panel.

Requests and retries

4. A request too large for one Jev call is asked in parts.
   • Knockouts over long results pages carried 16 to 19 groups, 73 to 79 KB. The Tiny Humans gateway refused them with HTTP 502 every time, which ended the task (Amazon, BookMyShow).
   • Such a request is now cut by its questions into parts asked at once, each with the whole state. Jev evaluates each question on its own, so the answers merge back by id.
   • The cap goes from 100 KB to 48 KB. Across 7,068 live calls, 57 KB (23,600 tokens) was the largest that passed and 68 KB the smallest that was refused.
   • Live follow-up: vote::ballots took its question ids from the first answered framing only, so every part but the first lost its answers. Live, a search's "Go" won its group at 0.98 in the second part and was never pressed. The ballots now cover every question any framing asked.
5. A provider's brief outage is ridden out.
   • The client retried a 5xx twice, 100 ms and 200 ms apart, so a task ended 2.5 s into a gateway blip.
   • When max_retries is not set, a Jev call now retries four times, waiting 1, 2, 4, then 8 s.

Waiting, reading, safety

6. wait_for stops when the page says it found nothing.
   • A "No Results Found" page was checked ten times over (~50 s), three times in one task, and each rescue guessed at the query's wording.
   • Two checks in a row showing "no results", "0 results", "no products found" and the like now fail the step. The note names a phrase from our own list, never the page's text.
7. Values read into declared variables are reported.
   • The planner declares each read's variable up front, empty. The task dropped every declared variable as the caller's input, so a requested brand, price, or bag total never came back.
   • A variable now counts as read when the run changed it.
8. Login walls.
   • "Log in to see / view …", "log in or sign up", "please log in", "you must be logged in", and "login required" now pause as needs_human.
   • A header's bare "Log in" and "Sign up" still do not count.
9. The reCAPTCHA badge is no longer a captcha.
   • Only a challenge counts now: "complete the reCAPTCHA", "select all images / squares", hCaptcha's "I am human".
10. A store's and a ride app's last buttons are gated.
    • "Place your order", "Confirm order", "Complete order", and "Proceed to pay" count as payment.
    • "Confirm ride" and "Confirm pickup" need approval.
    • "Book" alone stays reversible, since on travel sites it opens the traveller form.

Planner guide

11. An option group (size, colour, quantity) is a choose of its label, never a pick. A filter on which result to take ("skip Sponsored items") belongs in the pick, never a step of its own. A pick keeps the task's item and criterion ("the first boAt Airdopes 141"), never a stand-in such as "lowest price", which took another brand live.
12. A condition names a state ("the sort shows Price: Low to High"), never a change ("the grid has updated after sorting"). A change cannot be judged on one screen, and it held a sorted list under the bar for ten checks.

task_live

13. Its hosted roles run on openrouter/deepseek/deepseek-v4-flash, the managed default OpenHuman uses, instead of agentic-v1.
14. TASK_INTERACTIVE=1 waits for the person at the terminal instead of ending at every pause:
    • approve or decline an action;
    • get past a login or captcha, then press Enter;
    • type a missing detail;
    • finish on the payment page.
15. It prints what the task read, and writes records.json, however the task stopped.
16. A pause's reason now ends with one full stop in its prompt.

Round 2: stores, ride apps, and a cinema

Commits 60e7879b to 428fb3be.

Sight (what the page reads as)

17. A box is named by its own label, aria label, placeholder, or title before the words beside it, and a divider's "OR" never names one: a search box read as "Location not set", a location box as "OR".
18. An unmarked picture button beside a bare count is "decrease" or "increase"; icon glyphs (an icon font, or a lone glyph in a font other than its parent's) are pictures, not names ("p", "!"); an icon's sprite symbol or file name names it ("#icon-cart" is "cart").
19. A small element whose React props hold a click handler is a button: a store's "Add to cart" and "Buy now" were plain words.
20. Repeated plain boxes (three or more siblings of one tag and class, each with a link or button and a word) are cards, numbered across rows so a grid of rows reads as one list.

Lists and picks

21. Runs of three or more same-role links, options, radios, or buttons showing 20+ characters are lists too: a store's link cards and a ride app's options were never offered. A list whose cards show only bare markers (carousel dots, chips, page numbers) is not results.
22. A card opens through a control that says so, then a named link (its title), then the first control: a card opened through its pack-size dropdown.
23. pick by "first " walks the list in page order for the first item that meets it (it took the fifth result, another model).

Browser presses

24. Links and forms aimed at a new tab, and a script's window.open(url) right after a press, stay in the session's tab.
25. A click refused as covered is tried once more with the element centred and the pointer moved off (a product photo's hover zoom covered "Add to cart" twelve times); a content link whose press went nowhere is followed to its address.
26. Pressing a "use my current location" control grants the session geolocation first, standing in for the browser's Allow bubble.
27. A navigation that timed out counts as open when the page is drawn; one page reading is capped at 10 s (a look took a minute).

The do loop and dialogs

28. A dialog or layer the run's own press opened is its next stage, across steps and rescues: never cleared by attention, obstacles, Escape, or undo; covered controls and its close control are not offered as moves; a failed step names its controls for the rescuer. A layer that typing opens (suggestions) is the field's.
29. A control or key pressed three times in a step is not pressed again; pressing a named control strikes off its copies on other items for the step ("Add" on six products for "add 2 packets").
30. Attention keeps a layer open when a control in front fits the step; "allow selection" is a consent closer; a stalled step says its work may be done; a control out of view counts as pressable.

Typing and suggestions

31. A plain step that types ("enter 560001 into the pincode field") runs as enter; an enter that typed nothing fails instead of reporting done, first presses a control named by the slot's word, and never types two slots into one box (a drop typed over a pickup).
32. A search box takes only a suggestion that is the same search; a place box looks twice more for late rows, and counts rows already showing that match or share most typed words.

Steps, safety, planner, rescuer

33. wait_for holds on a settled screen judged at 0.65+ three times; more "found nothing" phrases; a date option matches a strip's "WED 07 OCT"; a read takes the text holding the value itself.
34. OTP sign-in dialogs ("Login/ Sign up Using OTP") and "Sign up or Log in" pause as needs_human.
35. Planner guide and rescue protocol: dialog headings are not answers; no confirm step after a picked choice; delivery place only when the task gives one or the page asks; add before raising a count; seats by a plain step; a day in a date strip is a choose; never a stop_before for an unasked login; keep the task's size and variant words.

Round 3: cab booking, a grocery basket, and cinema seats

Commits fec1cca0 to b054d7d5.

36. A pick the page already shows selected is taken as it is (fec1cca0): Uber's cheapest car was selected by default, and pressing it again opened a fare breakdown over "Request".
37. A − count + stepper in place of Add means the item is in the cart (a7a5306e, planner guide): BigBasket's basket held the milk ×2 while the run looped to add it.
38. Before a covered retry, a focused text box lets go (6e6abc47), closing the suggestions it held open over the target; rescues never add an item twice.
39. A place box takes its closest row when none names the place exactly (9dc611f3): Ola lists "MG Road Metro Station" as "MG Road, Shivaji Nagar". The rescue protocol learns the date strip and in-card time rules.
40. Sight reads a table's rows and buttons the way a person does (18b2f821), all from BookMyShow's accessible seat table:
    • an unlabelled row is named by its control-free cells, row "01 Available" #1, so a seat's status reaches Jev (it had pressed a Handicapped seat);
    • a merged wrapper (td role="gridcell") points at the native button inside it: six presses at the cells' middles selected nothing while the buttons sat at their left edge;
    • aria-hidden cells that take no pointer events count as seen: the hit test passed through them, and every row in view lost its seat number.
41. A rescue that reruns the failed step covers nothing after it (86337b98): one claimed covers: 1, and the show-time step was dropped.
42. Lists (25331ade): a list of one-line items that are another list's cards split apart is dropped (Ola's ride cards also read as their nine lines, losing names and fares); a list not clearly chosen goes to the one Jev leaned to (0.41 against 0.05) before the longest.
43. Items chosen together read as one source (a390a15e): "the selected seats" could take only one piece of text.
44. A step that chooses several items keeps the copies rule off (94181464): "choose 2 adjacent seats" had its second seat's "Select" struck off as another item's copy; "add 2 packets of milk" keeps the rule.
45. The task's own dialog is answered when nothing serves the step (f55b43c6): a date step left a format dialog's "2D" alone for four turns and spent a rescue. A control the dialog's own bar covers (a seat row under "Pay") is offered; what lies behind the dialog still is not.
46. Planning keeps a choice's conditions, and presses a booking button first (b054d7d5): "the cheapest car" became "lowest fare" and took an auto.

Review: the whole PR, end to end

Commits 6f242057, 1fcfd171, 35e46635. Four reviewers read the full diff against main: the browser adapter and sight; the do loop, dialogs, and the Jev plumbing; step kinds and form filling; and core, rescue, tasks, and the runner. Every finding was checked against the code before it was fixed.

Hardcoding check. No hostname, site name, or one site's label appears in any code path. The word lists are generic UI vocabulary (login walls, "found nothing", place words, number words, role names, cookie-consent labels). Site names appear only in comments that describe a live failure and in test fixtures; the site-flavoured examples in the planner guide and rescue prompt were reworded.

Fixed, each with a test that fails with the fix switched off:

47. Secrets (blockers). A plain step that types was substituted twice: a value read off a page that said ${card_number} was typed as the caller's card number. It is now read from the step as written. A variable a flow defines from a fact ("recipient": "${email}") was reported as a read, which put a secret's value in the task's records; only a variable a step writes is a read now.
48. The task's own dialog (blocker). A dialog a step opened stayed "the task's own" for every later step, so a calendar left open was never cleared again. A step that works in such a dialog now hands it back at the next step; a scroll or the run's own housekeeping (clearing, dismissing, undoing) opens no dialog of the task's; opening an address forgets it. Answering such a dialog when nothing serves the step (45) now happens only on a browser task, among the dialog's own controls, and never presses a control that commits ("Yes", "Confirm", "Pay", "Book").
49. Typing steps. "Enter" also means going into something ("enter Reader mode in Safari", a desktop regression), so with " in " it types only what is quoted, data, or into a box; a step that does more than type ("… and press Enter") stays a plain step for a rescue to split; "type in" is a verb; "enter the ${otp}" types the code, not "the 123456"; a quoted text ends at its quote; the split prefers the field that names a box.
50. Presses. A control's copies are struck off only on the other items of its own list (a sticky bar's "Add to cart" is no copy), only once the next look shows the press changed the screen (a refused press acted on nothing), and an undo lifts only that press's copies. A scroll counts toward no press cap, so a long list scrolls more than three times.
51. Form filling. The control named by a slot's word is pressed only once Jev agrees it shows that slot's box (OPENER_FLOOR 0.8), never on a shared word or the role word "link" alone ("Email us" shares "email"). A place box with no fitting row asks Jev once more for the row naming the place in other words, instead of pressing the row sharing most words (a default click). A search box offers only new rows, so a menu link reading as the query is not pressed; a place box whose own words say "Search for area…" takes a place.
52. Picks. "first X" is a condition only with a condition word (rated, under, with, available…): "First AC" and "first class" are names, "first to depart" an order Jev judges. Round 3's guide rule put a choice's kind in by, where a measured ranking ignores it; the guide and rescue prompt now put it in from ("the car options" by "lowest fare"), where the ranking checks it.
53. Login stops. Found live on Amazon in this review: a plan for "do not log in" stopped short of the cart in front of the header's "Hello, sign in". A stop_before that names only signing in now gates nothing (a login wall pauses for a person by itself); signing up or creating an account stays gated.
54. Rescues. Guidance that ends by rerunning the failed step covers only the later steps its earlier steps do (the same step, or an enter filling one of their fields): round 3's rule set covers to 0 outright, which reran fields a rescue had already filled.
55. Reads and waits. A value read before a rescue outranks the flow's empty declaration of it (${total} expanded to nothing after a rescue). "no products" and "0 products" (a header's empty cart) no longer read as a search that found nothing.
56. Jev requests. A screen that fills most of a request is cut before splitting, so its questions are not asked one call each, and the budget is charged per part and framing.
57. Browser presses. Keeping a press in the tab changes only the pressed link or form (it rewrote every _blank link on the page), opens a script's new window in place only for an address on the same site (an advert's window opened on a click no longer takes the tab), and returns no window (a page's w.close() closed the agent's tab). Location is granted only in a browser the module launched on a throwaway profile, never in a person's own browser or profile. A link press is no longer followed for an expand or collapse, a download, a link that controls the page, a page already leaving, or behind a dialog shown in front; a failed follow keeps the press's own reply. The drawn-page check compares the query too. A covered retry keeps the focus when the target is a row of the focused box's own list.
58. Sight. Only an element that turns pointer events off itself counts as seen through a hit on what holds it, never a hit on the page's root: a modal library turns them off for the whole body while it hides the page behind its dialog. Only a press handler (onClick, onPress) makes a React element a control (a carousel's onMouseDown track became one big button), and such a control hides nothing pressable inside it. A record aims at the native button inside a wrapper only when the wrapper is no tab, radio, or option, which would be pressed twice. A lone glyph is a picture only in another font family than its parent's, by each stack's own family, so "XL" in a heavier weight stays. A class ending in "-selected" or "-checked" reads as selected: found live on Myntra, whose picked size carries no ARIA state.
59. Walls. A phone check asks to "verify the phone number", not to "sign in"; "Please check the reCAPTCHA box" and "verify that you are not a robot" are challenges; the login-wall phrases are tested.
60. CI. The Rust job failed on clippy::assert_is_empty, new in Rust 1.99, on three test asserts; fixed, and every check below was run with 1.99.

Not fixed here, for a decision or a follow-up:

• Decision for you: "Request ", "Book ", and "Order now" are not classed irreversible by consequence; only a planner's stop_before gates them, as it did in every live run. Gating them by wording would also ask approval for "Request a callback" and travel sites' "Book" that opens a form.
• When a step fails, login-wall phrases are matched anywhere on the page before any rescue: a fixable failure on a page that also says "Log in to see your saved addresses" pauses for a person instead.
• Minor: row names read textContent (screen-reader-only text can enter one); a card's ordinal restarts with each reading; a date control without a year matches any year; the slot words "to" and "address" give plain boxes a place box's two late looks; the 10 s reading cap also bounds the tree snapshot; the Jev request limit is not per provider.

Related issue

None in this repo. This is part of tinyhumansai/openhuman#7000, getting the browser agent working end to end.

API or behavior changes

No wire changes, and no contract version bump.

Engine

• enter may press one suggestion row after typing: Jev's pick at 0.5 or more, or a lone row reading exactly as typed.
• MAX_REQUEST_BYTES is 48,000, and a larger request is asked in parts.
• The decision journal event gains parts, and its request_bytes is the largest part.
• wait_for fails early on a found-nothing page.
• Jev retries default to four, at 1, 2, 4, and 8 s. This updates the doc of JevConfig.max_retries.
• The task controller reports reads into declared variables.

Browser

• Sight no longer reads a region with only a tab stop, or a button holding a text box, as one control.

Core

• human_needed has more login-wall wording, and the bare word "reCAPTCHA" is no longer a gate.
• consequence has six more phrases.

Bus

• FLOW_GUIDE has two clarified rules.

Round 2

• Browser: every click goes through press_element (same tab, location permission, covered retry, link follow); page readings time out at 10 s; a timed-out navigation to a drawn page succeeds.
• Engine: FlowRun keeps what is in front in front.rs; do steps cap repeated presses and copies; enter fails when it typed nothing; plain typing steps run as enter; new thresholds MAX_REPEAT_PRESSES, LAYER_COVERS, FRONT_CONTROLS, STEADY_HOLD/STEADY_CHECKS, LATE_LOOKS.
• Core: result_families adds runs of single-control cards and drops bare-marker lists (BARE_CHARS, CARD_LINK_CHARS); two more login-wall phrase families.
• No wire changes and no contract version bump.

Round 3

• Browser: an unlabelled table row's container label carries its cells' words (row "01 Available" #1, still ending in its ordinal); a merged wrapper's record id and box are its inner native button's; aria-hidden text that takes no pointer events is read.
• Engine: when nothing serves a do step and the task's dialog is in front, one more grounding question asks what answers the dialog; a covered control inside a dialog is offered while a dialog is in front; read offers the checked or selected items of one list as one source; a rescue's covers counts only what its guidance does before rerunning the failed step (refined in the review); new thresholds LIST_LEAN and LIST_LEAD.
• Core: result_families drops a list split from another's cards.
• Bus: FLOW_GUIDE gains the choice-condition and booking-button rules.
• No wire changes and no contract version bump.

Review

• Engine: a plain typing step is read before substitution; only variables a step writes are reported as reads; collected reads outrank a flow's declarations; the task's dialog is handed back after a step works in it; the dialog fallback is browser-only, dialog-only, and never commits; copies are struck per list and per press, after a change; enter confirms a slot's opener with Jev (new OPENER_FLOOR) and re-asks for a place box's row instead of pressing the closest; a login-only stop_before is done at once; rescue covers counts what the guidance does; requests cut an oversized screen before splitting and charge the budget per part.
• Browser: presses retarget only the pressed link or form and same-site window.open; location is granted only in a module-launched throwaway browser; link following skips expands, downloads, in-page controls, pages already leaving, and shown dialogs; sight's pointer-events, React-handler, aiming, glyph, and selected-class rules as in 58.
• Core: two phone-check phrases name their own action; two robot-challenge phrases.
• Bus: the guide puts a choice's kind in a pick's from, and never plans a login stop_before.
• No wire changes and no contract version bump.

Examples

• task_live has a new default hosted model.
• TASK_INTERACTIVE is new.
• follow takes an optional Person.
• Reads are printed, and records.json is always written.

Validation

Commands actually run, on macOS (arm64):

• cargo fmt --all -- --check: pass

• cargo clippy --all-targets --all-features -- -D warnings: pass

  • Run as --workspace --exclude tinycomputer-accessibility. That crate's macOS-only code has clippy failures that predate this PR and that Linux CI never compiles.

• cargo build --all-targets --all-features: not run as its own step. The clippy and test builds compiled every target.

• cargo test --all-features --workspace --exclude tinycomputer-accessibility: 1057 passed, 0 failed (round 1)

• Round 2, at 428fb3be:

  • cargo fmt --all -- --check: pass
  • cargo clippy --workspace --exclude tinycomputer-accessibility --all-targets --all-features -- -D warnings: pass
  • cargo test --all-features --workspace --exclude tinycomputer-accessibility --exclude tinycomputer-examples: 1022 passed, 0 failed (917 unit and integration tests, 105 doc tests)
  • the tinycomputer-examples tests were not re-run in round 2: the disk filled while linking its binaries. That crate's code did not change in round 2.

• Round 3, at b054d7d5:

  • cargo fmt --all: applied, then clean
  • cargo clippy --all-targets --all-features --workspace --exclude tinycomputer-accessibility -- -D warnings: pass
  • cargo test --all-features -p tinycomputer-core -p tinycomputer-browser -p tinycomputer-engine -p tinycomputer-bus -p tinycomputer: 855 passed, 0 failed
  • the tinycomputer-examples and tinycomputer-accessibility tests were not re-run: the disk is nearly full, and neither crate changed in round 3.
  • The round-3 sight changes were checked in Chromium against local copies of BookMyShow's seat-table markup (aria-hidden cells with pointer-events: none, a gridcell wrapping a left-aligned button): rows read row "01 Handicapped" #1, each "Select" record points at its BUTTON, and an aria-hidden paragraph under an overlay is still dropped.

• Review, at 35e46635, with Rust 1.99 (CI's stable):

  • cargo fmt --all -- --check: clean
  • cargo clippy --all-targets --all-features --workspace --exclude tinycomputer-accessibility -- -D warnings: pass
  • cargo test --all-features --workspace --exclude tinycomputer-accessibility: 1090 passed, 0 failed, the examples crate included (the accessibility crate's paste test types real keystrokes on macOS)
  • GitHub CI on 35e46635: Minimum supported Rust version, Docs, and Supply chain pass; Rust was still running when this was written (its run on b054d7d5 failed on the 1.99 lint, fixed here)
  • The review's sight changes were checked in Chromium against local pages: the seat table's markup, a React-wired card with its own "Add" and an onMouseDown-only carousel track, the earlier icon-glyph pages plus "XL" and "M" in a heavier weight of the same font and "OK" with a symbols font in its stack, a store's size buttons marked by class, and a full-size pointer-events: none iframe (no longer read as a frame in front).

The round-2 sight changes were each checked in Chromium against local pages built like the live cases (naming, steppers, rows of product cards, carousel dots, icon glyphs, sprites, React-wired buttons, new-tab links).

The sight change was also checked in a Chromium page against a local copy of the popover the live run hit:

• the old sight.js returned one button "Select a location for delivery … 560001, Bengaluru, Karnataka MG Road…";
• the new one returns the three rows as separate buttons, while a pointer-styled carousel slide stays a button.

The gated live_* sight test was not run: this machine has no Chrome where agent-browser looks.

Live runs

Headed Chrome on the Tiny Humans route, with the module built at each stage:

Site / run	Result	Fixed here
Uber fare estimate	both places picked from their suggestions, then needs_human at "Log in to see ride options" (~40 s)	1, 8
Myntra, cheapest white sneakers, size 9	reached PLACE ORDER (payment checkpoint), with 2 rescues	the rescues: 11, 12; the missing reads: 7, 15
Amazon, BookMyShow	ended by HTTP 502 on oversized knockouts	4, 5
Amazon, re-run 2 (after 1's floor and 11's filter rule)	reached "Proceed to Buy", but with another brand's earbuds (pick by "lowest price"), and only after 4 rescues: "Go" was lost with the second part's answers	4 (ballots), 11 (pick wording)
Amazon, re-run after 4 and 5	no 502s: 718 calls, 0 failed, 11 decisions split (largest part 47.6 KB). It failed later: a suggestion pick changed the search to "…141 anc", and a "skip Sponsored items" step had nothing to do	1 (floor), 11 (filter rule)
BigBasket	wrong delivery area (popover pressed), then "No Results Found" ×3	2, 3, 6
Ola	suggestion rows inside shadow DOM, not readable	not fixed here (below)
BlazeDemo / Selenium (model change)	84 → 44 s, 65 → 29 s	13

Round 2 (module rebuilt between fixes):

Site / task	Result
Zepto: search Maggi, add 2, cart total	✅ cart with 2 packets, ₹40/₹50 (twice, 93–102 s)
Flipkart: first boAt Airdopes 141 rated 4★+, title/price/rating, cart	✅ payment checkpoint ("Place order"), 2 rescues
Lenskart: first blue-light glasses, cart	✅ payment checkpoint ("Proceed To Checkout"), ₹500, 2 rescues
Blinkit: current location, Amul Taaza 1 L ×2, cart	✅ end to end (location detected, quantity 2, ₹95 grand total, "Login to Proceed"); one rescue took the 200 ml pack, since fixed in the rescue protocol (35)
Uber: book a cab, cheapest car, stop before Request	✅ after the person signed in: 9 options extracted, Uber Go AC ₹329.64 picked, gated at "Request Uber Go AC"
BigBasket: current location, Amul Taaza 1 L ×2	⚠️ right product added once; the basket needs an OTP login the task forbids (now a needs_human pause, 34); its place popover has no current-location option
BookMyShow: show on Wed 7 Oct, 2 seats	❌ reaches the show timings with the format answered; the date strip is not re-chosen after the format dialog, and a pick opens a cinema card instead of a show time
Ola: pickup and drop	❌ the drop was typed over the pickup and the late suggestion rows were not picked; fixed in 31–32, not yet re-run

Round 3 (module rebuilt between fixes; every approval prompt was answered by the person running the test):

Site / task	Result
Uber: cheapest car, stop before Request	✅ after sign-in: all 9 options with fares, Uber Go AC picked, gated at "Request Uber Go AC"; the pick had pressed the already-selected option, opening a fare breakdown, fixed (36)
BigBasket: Amul Taaza 1 L ×2	⚠️ the right milk added and raised to 2 cleanly; the basket needs an OTP login, so it pauses as needs_human
Ola: MG Road Metro → airport, cheapest car	✅ pickup and drop set, ride types listed, gated at "Confirm & Book" (declined). Fares need a login ("Please log in to check exact prices"); it chose Auto, not a car, and reported the cards' lines (42, 46, not yet re-run)
BookMyShow R11	❌ reached the accessible seat table; the second seat's "Select" was struck off (44); a Handicapped seat was pressed (40)
BookMyShow R12	❌ further: show time kept (41), "Select Seats" pressed in-step; six seat presses missed their buttons (40)
BookMyShow R13	❌ date, format, show time and seat count in-step, no rescue; seats A-02 and A-03 chosen, cheapest section ₹200, "Pay ₹400"; reading the seat numbers failed (aria-hidden cells, 40)
BookMyShow R14	❌ seats A-03 and A-04 chosen; seat numbers now read, and the combined source showed "03 Available; 04 Available", but Jev took a single number (0.26); the source now says plainly that these are the checked items (43), not yet re-run

Review pass, on the build with the sign-in rule (53), before the other review fixes:

Site / task	Result
Amazon: first boAt Airdopes 141 rated 4★+	✅ "Proceed to Buy (1 item)", no rescues. Every Airdopes 141 result now shows 3.8★, so the first non-sponsored result rated 4★+ was "boAt Nirvana Ion", as the task's words ask
Myntra: white sneakers, size 9	✅ PLACE ORDER, total ₹304, one rescue at the size step: the chosen size showed no state, fixed in 58
Flipkart, Zepto, Lenskart, Blinkit, BigBasket, Uber, Ola, BookMyShow	being re-run one by one on 35e46635; results follow in a comment

Known and not fixed here:

• Ola: its rows sit inside shadow roots. A fix to agent-browser's snapshot (marking pointer and click elements inside open shadow roots, resolving them with a piercing DOM.getDocument) is in the local vendor checkout but not in this PR: it needs its own commit on the tinyhumansai/agent-browser fork, then a gitlink bump here.
• BookMyShow: the seat read with the clearer combined source is not re-run yet. A planned pick of "the shows" still opens a cinema card; the seat step then presses the show time itself.
• BigBasket: the header's basket icon is not read as a control.
• Warning noise: the "failed to prepare iframe session" warnings belong in the agent-browser fork.

Tests

• flow_tests/suggestion_tests.rs, with a simulator ride form (flow_tests/places.rs):
  • picks a suggestion, with Jev and directly;
  • a plain field asks nothing;
  • a private text is never offered;
  • a panel that strings its rows together is never pressed.
  • Without their fixes, the first and last fail as live did: the box emptied, and the wrong place was picked.
• flow_tests/split_tests.rs:
  • parts stay under the cap, each with the whole state;
  • every question is asked once, in order;
  • a 400-row knockout goes out in parts and still clicks its target. This test fails with splitting disabled.
• step_kinds_tests::a_wait_for_stops_when_the_page_says_it_found_nothing
• resolve_tests::jev_calls_ride_out_a_provider_outage_unless_told_otherwise
• output_tests::a_value_read_into_a_declared_variable_is_reported: fails without the fix, with "no entry found for key".
• safety_tests:
  • login walls, the reCAPTCHA badge, and the challenges;
  • the new payment and irreversible phrases.
• sight_tests/live_tests::live_panels_and_regions_leave_their_rows_to_be_read: gated on TINYCOMPUTER_LIVE_BROWSER=1.
• Examples task_tests.rs:
  • what a person sends a paused task;
  • read lines;
  • prompt sentences.

Round 2 adds:

• step_kinds_tests::a_plain_step_that_types_is_read_as_the_enter_it_means
• pick_tests::a_first_with_a_condition_walks_the_list_in_order
• choose_tests::a_day_in_a_strip_of_dates_is_found_by_its_short_label
• suggestion_tests::a_search_box_takes_only_the_same_search_and_a_place_box_its_reworded_rows
• safety_tests::walls_only_a_person_can_pass_are_named: OTP sign-in and "Sign up or Log in" walls; a header's "Login/ Sign Up" is none.
• Browser surface tests now allow the same-tab script and the pointer move before a covered retry (Fake::evaluated_besides_keeping_the_tab, Fake::pressed_by_position).
• Not yet covered by simulator tests: the late suggestion looks and the steady wait_for. They were exercised in the live runs above. The dialog tracking in front.rs gained a test in the review.

Round 3 adds:

• pick_tests::a_pick_the_page_already_has_selected_is_not_pressed_again
• suggestion_tests::a_place_box_takes_its_closest_row_when_no_suggestion_clearly_fits (replaced in the review, below)
• do_loop_tests: a step choosing several items presses each one's own copy, and one adding a count of one item does not (the copy rule); several items are asked for by a choosing verb and a counted plural; a step finding nothing to press answers the task's dialog; a control the dialog's own bar covers is pressed and one behind it is not. Each simulator test fails with its fix switched off.
• rescue_tests::guidance_that_ends_by_running_the_failed_step_again_covers_nothing_after_it
• groups_tests::a_list_of_another_lists_cards_split_line_by_line_is_dropped
• pick_tests::a_list_not_clearly_chosen_is_the_one_jev_leaned_to_when_it_leads_clearly
• step_kinds_tests::the_items_chosen_together_in_one_list_read_as_one_source

The review adds, each run with its fix switched off and seen to fail:

• step_kinds_tests: a_plain_typing_step_types_a_value_that_names_a_fact_as_written; new cases in a_plain_step_that_types_is_read_as_the_enter_it_means; a_stop_before_names_only_signing_in_or_something_irreversible; a_stop_before_signing_in_lets_the_flow_go_on; a_value_read_before_a_rescue_outranks_the_flows_empty_declaration
• output_tests::a_variable_defined_from_a_fact_is_never_reported_as_a_read
• do_loop_tests: a_dialog_the_task_worked_in_is_in_the_way_of_the_next_step; answering_the_task_dialog_never_presses_what_commits; a_pressed_controls_copies_are_only_on_the_other_items_of_its_list
• enter_tests::a_control_named_by_the_slot_opens_its_box_only_when_jev_agrees
• suggestion_tests::a_place_box_asks_again_for_the_row_naming_its_place_in_other_words
• pick_tests: "First AC", "first class", "first to depart", "first alphabetically" are no conditions
• rescue_tests::guidance_that_does_the_later_steps_before_rerunning_the_failed_one_covers_them
• split_tests::a_screen_that_fills_most_of_a_part_is_cut_rather_than_asked_once_per_question
• records_tests::first_and_last_alone_are_the_lists_own_order; safety_tests: the new wall and challenge phrases
• browser operations_tests: only_a_control_asking_for_the_place_is_read_as_one; two_addresses_of_one_page_are_one_place; location_is_granted_only_in_a_browser_the_module_launched

Documentation

• Engine flow docs:
  • docs/crates/tinycomputer-engine/flow/filling-forms.md
  • docs/crates/tinycomputer-engine/flow/step-kinds.md
  • docs/crates/tinycomputer-engine/flow/voting-and-briefing.md
  • docs/crates/tinycomputer-engine/flow/budgets-and-switches.md
• Jev runtime and harness:
  • docs/crates/tinycomputer-engine/jev-runtime.md
  • docs/technical/jev-harness.md
  • docs/technical/jev-journal.md
  • docs/technical/jev-questions.md
  • docs/technical/decision-thresholds.md, which also gains the missing MOST_SUGGESTIONS and OPTION_EXTRA_WORDS rows
• Tasks, sight, and safety:
  • docs/technical/tasks.md
  • docs/technical/specs/browser-sight.md
  • docs/safety-and-privacy.md
• Configuration and examples:
  • docs/crates/tinycomputer/configuration.md
  • docs/crates/tinycomputer-examples/live-tasks.md
  • .env.example
• Planner guide: crates/tinycomputer-bus/src/flow/guide.md
• Round 2: docs/technical/decision-thresholds.md (seven new constants), docs/crates/tinycomputer-engine/flow/step-kinds.md, docs/crates/tinycomputer-engine/flow/filling-forms.md
• Round 3: docs/crates/tinycomputer-browser/sight.md, docs/technical/specs/browser-sight.md, docs/crates/tinycomputer-engine/flow/step-kinds.md, docs/crates/tinycomputer-engine/output.md, docs/technical/specs/task-rescue.md, docs/technical/decision-thresholds.md (LIST_LEAN, LIST_LEAD, and the copy rule's exception)
• Review: docs/crates/tinycomputer-engine/flow/the-do-loop.md (the task's own dialog, presses and copies), filling-forms.md (search versus place boxes, the place re-ask, the opener, typing steps), step-kinds.md (the login-only stop_before, "first" with a condition), docs/technical/decision-thresholds.md (OPENER_FLOOR, the missing MAX_IDLE_SCROLLS, the copy and layer rows), jev-harness.md, jev-questions.md, specs/task-rescue.md, docs/crates/tinycomputer-browser/sight.md and specs/browser-sight.md (naming order, chosen and hidden states, React controls, what a press adds to a page), docs/safety-and-privacy.md (the location grant, new tabs), docs/rescue.md, docs/crates/tinycomputer-core/prices-times-and-dates.md

Checklist

• The change is focused on one logical change: it gathers the fixes from one round of live testing, each in its own commit and each reviewable on its own.
• No new #[allow(...)], #[ignore], or relaxed lints
• No secrets, tokens, or .env contents in the diff or the description

[coderabbitai]

Important

• 🔍 Trigger review

This repository does not receive automatic reviews because it has fewer than 10 stars.

⚙️ Run configuration

• Configuration used: Organization UI
• Review profile: CHILL
• Plan: Advanced
• Run ID: 08804e41-1814-495d-8417-a759366b181d

• Autopilot · Keep fixing CodeRabbit findings and required CI, and resolving merge conflicts

---

_{Comment @coderabbitai help to get the list of available commands.}

[tinysweeper]

Tiny Sweeper review

⚠️ Review failed for 657fffb02f39. the review of #76 did not finish within 900s

[tinysweeper]

Requesting changes: 2 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.0009 · 72,330 in / 4,357 out · 6,212 cached (9%)  · gpt-5.6-luna, glm-5.3-flash
critique:    $0.0004 · 36,336 in / 2,099 out · 6,148 cached (17%) · gpt-5.6-luna, glm-5.3-flash
security:    $0.0003 · 21,020 in / 1,112 out · 0 cached (0%)      · gpt-5.6-luna
tests:       $0.0001 · 5,458 in  / 231 out   · 64 cached (1%)     · glm-5.3-flash
description: $0.0001 · 5,585 in  / 94 out    · 0 cached (0%)      · glm-5.3-flash

[tinysweeper]

Requesting changes: 2 lane(s) blocking, worst finding is high.

Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.

             $0.0093 · 742,662 in / 52,997 out · 112,689 cached (15%) · gpt-5.6-luna, glm-5.3-flash, deepseek-v4.1-flash
critique:    $0.0056 · 430,663 in / 31,300 out · 67,654 cached (16%)  · gpt-5.6-luna, glm-5.3-flash
security:    $0.0030 · 224,473 in / 14,886 out · 23,275 cached (10%)  · gpt-5.6-luna
tests:       $0.0004 · 45,230 in  / 3,936 out  · 20,480 cached (45%)  · glm-5.3-flash, deepseek-v4.1-flash
description: $0.0001 · 20,300 in  / 795 out    · 1,280 cached (6%)    · deepseek-v4.1-flash