Repository navigation
fix: IndiGo tab and picker (#62), native dropdowns, calendars, verify flakes, streamed replies - #74
Conversation
A tab only shows another panel of the page, so a stop_before phrase no longer names one and, on a payment page, choosing one is filling the form like a saved-card radio. IndiGo keeps its flight search behind a tab labelled "Book", which a flow stopping before "paying for the booking" named and refused, so no step could open the search form (tinyhumansai#62). A tab's own label still counts against the denylist: the role is only the page's claim, so a tab labelled "Pay ₹7,346" stays gated. A stop_before phrase now names a control only where its label starts a word of the phrase: "Send" is still named by "sending the email", "Rent" is no longer named by "the current bill".
A widget that opens without focusing a field refuses text typed with no target (INVALID_TARGET on the browser). choose recorded that as typed, told Jev so, and spent the attempt; it now grounds the widget's search box in the same attempt and records nothing as typed (IndiGo's city pickers, tinyhumansai#62).
|
Important
This repository does not receive automatic reviews because it has fewer than 10 stars. ⚙️ Run configuration
Comment |
Tiny Sweeper review
|
![tinysweeper[bot]](https://avatars.githubusercontent.com/in/4526585?s=60&v=4){kind=small}
There was a problem hiding this comment.
Requesting changes: 1 lane(s) blocking, worst finding is high.
Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.
$0.0161 · 279,250 in / 20,934 out · 27,658 cached (10%) · gpt-5.6-luna, glm-5.3-flash
critique: $0.0087 · 131,064 in / 11,761 out · 14,586 cached (11%) · gpt-5.6-luna, glm-5.3-flash
security: $0.0058 · 111,268 in / 4,159 out · 13,072 cached (12%) · gpt-5.6-luna
tests: $0.0000 · 18,979 in / 1,507 out · 0 cached (0%) · glm-5.3-flash
description: $0.0001 · 9,392 in / 120 out · 0 cached (0%) · glm-5.3-flash
| // city pickers, tinycomputer#62). | ||
| 1 if into_focus => { | ||
| if !self.type_into_focus(log, option).await? { | ||
| self.type_to_filter(log, screen, what, option).await?; |
There was a problem hiding this comment.
Only record filtering after the fallback succeeds
When the focused TypeText operation returns ok == false, this fallback calls type_to_filter. That function unconditionally inserts the target into typed and sets log.filtered = true before checking reply.ok, so a target that also rejects the text is recorded as successfully filtered even though no text was entered. This new call makes that failure path reachable for focused widgets and can cause later logic to skip required filtering or report a false success. Update type_to_filter to record typed and filtered only when its operation succeeds.
[RULE] incorrect-state-update ·
| || named_in_stop_before(name, stop_before) | ||
| || (!navigation && named_in_stop_before(name, stop_before)) | ||
| || (screen.surface == "sheet" && candidate.name.is_none()) | ||
| || (!is_form_control(candidate) |
There was a problem hiding this comment.
Keep payment controls gated regardless of the tab role
On a payment screen, any candidate reported with role tab now bypasses the generic payment safety gate, even when its label is non-destructive (for example, Continue). Accessibility roles come from the current page and are not a trustworthy authorization signal; a page can mislabel a submission or payment action as a tab, allowing the flow to press it before the payment confirmation step. Do not exempt navigation roles from the payment evidence gate unless the candidate is independently verified as non-activating navigation.
[RULE] authorization-bypass ·
A native dropdown (<select>) and a text box's suggestions (<input list> with a <datalist>) draw their choices in a menu outside the page. A click on the control opens that menu where nothing reads or presses it, so BlazeDemo's departure city and Selenium's "Dropdown (select)" and "Dropdown (datalist)" could never be chosen: the choose step looped until its budget ran out (77 Jev calls for the datalist). Sight now offers each enabled choice, up to 60 per control, as an option inside its control, `selected` when chosen. Suggestions two boxes share are offered under the focused one only, and each suggestion is marked with the box it was offered under. Pressing such an option sets the control's value the way a person's choice does (a suggestion through the input's own value setter, so React sees it), fires input and change, and succeeds only once the choice holds. Any other option is pressed as before. A label wrapping its dropdown, and a text block holding one, are read without the dropdown's text: Selenium's dropdown was named "Dropdown (select) Open this select menu One Two Three".
Tiny Humans' gateway answers HTTP 504 to a request that sends nothing back for 60 seconds. Unstreamed, nothing comes back until a reasoning model has written its whole reply, which for a rescue on agentic-v1 took 25 to 45 seconds on a small prompt and longer on a real one: a task that needed a rescue gave up on a gateway timeout instead (the rescue began at 12:57:44 and failed at 12:58:44). Every hosted call is now streamed and gathered into one reply, so the gateway sees the answer arriving from its first second. A route that ignores streaming still answers in one piece: the client reads a plain JSON body as the whole reply.
YellowSnnowmann
changed the title
There was a problem hiding this comment.
Requesting changes: 1 lane(s) blocking, worst finding is high.
Fix or reply to the findings below and push. The next review clears this automatically once they are gone — you should not need to dismiss anything by hand.
$0.0302 · 503,188 in / 40,568 out · 39,196 cached (8%) · gpt-5.6-luna, glm-5.3-flash, deepseek-v4.1-flash
critique: $0.0205 · 282,419 in / 24,829 out · 20,116 cached (7%) · gpt-5.6-luna, glm-5.3-flash
security: $0.0090 · 146,956 in / 11,327 out · 17,992 cached (12%) · gpt-5.6-luna
tests: $0.0004 · 37,365 in / 1,048 out · 1,088 cached (3%) · glm-5.3-flash
description: $0.0002 · 18,113 in / 502 out · 0 cached (0%) · glm-5.3-flash
| // inside its control, drawn in the control's box, and pressing one sets | ||
| // the control's value (`native_select.rs`). | ||
| const choicesOf = (element) => { | ||
| if (tag(element) === 'select') return element.multiple ? [] : [...element.options]; |
There was a problem hiding this comment.
Offer options for multiple selects
A reachable <select multiple> is represented as a combobox but receives no option nodes at all, so the reader cannot select any of its enabled choices through the newly added option path. The surrounding documentation promises choices for native dropdowns without excluding multiple selects. Return the options for multiple selects as well, or explicitly represent their supported selection semantics instead of silently dropping them.
| if (tag(element) === 'select') return element.multiple ? [] : [...element.options]; | |
| if (tag(element) === 'select') return [...element.options]; |
[RULE] incomplete-control-support ·
| pub(super) const CHOOSE_JS: &str = r#"(option => { | ||
| if (!option || option.tagName !== 'OPTION') return null; | ||
| const group = option.parentElement; | ||
| if (option.disabled || (group && group.tagName === 'OPTGROUP' && group.disabled)) return false; |
There was a problem hiding this comment.
Honor disabled fieldset inheritance
This only checks the option's own disabled state and an explicitly disabled optgroup. A select or datalist owner inside <fieldset disabled> can still have select.disabled/owner.disabled false, so this path changes the control and dispatches input/change even though the native control is disabled to the user. Check the effective disabled state (for example with the :disabled pseudo-class or an equivalent fieldset-ancestor check) for both the option and its owning control before applying the value.
[RULE] incomplete-disabled-state-check ·
| const list = tag(element) === 'input' ? element.list : null; | ||
| if (!list || element.readOnly) return []; |
There was a problem hiding this comment.
Restrict datalist choices to text-capable inputs
HTMLInputElement.list can be present on input types that do not show datalist suggestions, such as checkbox and radio inputs. For example, <input type="checkbox" list="choices"> reaches this branch and emits option nodes even though the browser does not offer that list under the control; pressing one can then incorrectly attempt to fill a non-text control. Gate this path on the same text-capable input predicate used by the rest of the reader.
| const list = tag(element) === 'input' ? element.list : null; | |
| if (!list || element.readOnly) return []; | |
| const list = tag(element) === 'input' ? element.list : null; | |
| if (!list || element.readOnly || !takesText(element)) return []; |
[RULE] invalid-control-assumption ·
| self.show_cursor(reference); | ||
| } | ||
| // A native option is chosen by value: a click opens an unseen menu. | ||
| if let Some(reply) = self.choose_if_native(operation, target.as_ref()) { |
There was a problem hiding this comment.
Keep payment controls gated regardless of the tab role
This new early return handles option clicks before the normal operation match, so any payment-control authorization performed in that path is skipped for native options. Apply the same payment-control gate before dispatching the native-option action, or have choose_if_native refuse gated targets.
[RULE] authorization-bypass ·
| if (!list || element.readOnly) return []; | ||
| // Suggestions two boxes share are offered under the one being typed | ||
| // in, so that pressing one names a single box to fill. | ||
| const users = [...document.querySelectorAll('input[list]')].filter((other) => other.list === list); |
There was a problem hiding this comment.
Shared-suggestion focus gate has no non-live test
The new users.length > 1 && document.activeElement !== element gate decides whether suggestions shared by two boxes are offered at all. It is only exercised inside live_native_choices_are_offered_as_options_and_chosen_by_value, which is behind #[cfg(feature = "agent-browser")] and a TINYCOMPUTER_LIVE_BROWSER=1 environment check, so on a machine without a display server it silently drops out — which the repository rules explicitly require tests to survive. The visible decision here is pure DOM logic that a fixture could pin without a browser; without one, a refactor that breaks the gate (say, dropping the activeElement check) leaves every offline test green.
[RULE] missing-test-branch ·
Bootstrap's date picker, among others, draws each day as a plain table
cell that shows a pointer only under the mouse, so sight never offered
the days as controls. On Selenium's web form a "set the date" step opened
the calendar, found no day to press, and three rescues later typed the
date instead (269 s). Its paging arrows read as "»" and "«", which
`choose` does not take for "next month".
A shown table with at least 28 day-number cells, whose month and year its
own heading or a short header just before it names, is now a calendar:
each enabled day is a gridcell named by its number and described by the
date it stands for ("15 November 2026"), the days around the month dated
in the months either side, and a week-number column left out. Inside a
calendar's container an arrow glyph or a bare "Next" reads as "next
month", and its opposite as "previous month", which is how `choose` pages
a calendar to a date. A table of numbers whose month is named only
elsewhere on the page is not a calendar.
A condition that lists several things draws a hedged yes/no but a crisp coverage answer; that is what the coverage question is for. Averaged, the two cannot clear the 0.75 bar unless the coverage is certain. On BlazeDemo, "the purchase form shows the passenger name, street address, city, state, and zip code", with every field filled, was judged 0.71 (yes/no 0.52, coverage 0.88): the step failed and a rescue only confirmed the form was done. A condition's calibrated yes/no within 0.10 of an even chance now defers to a coverage whose top level is at least 0.85 (`ask::deferred`), also in the deep level's screen-only view, which now asks the coverage too. A yes/no that leans either way is averaged as before, so a clear "no" is never overruled by the coverage, and a do step's completion keeps its plain average with progress.
YellowSnnowmann
changed the title
2 participants
Summary
Five fixes that stopped browser tasks partway through, or made them slow. Each fix is in its own commit(s).
1. IndiGo's "Book" tab and city-picker search (#62)
These are the two original commits of this PR, unchanged.
stop_beforephrase therefore no longer names one, and on a payment page choosing a tab ("UPI", "Cards") counts as filling the form, like picking a saved-card radio. A tab's own label is still checked against the denylist. Thetabrole is only what the page claims, so a tab labelled "Pay ₹7,346" stays gated.stop_beforephrase now names a control only where the control's label starts a word in the phrase. "Send" still matches "sending the email"; "Rent" no longer matches "the current bill".INVALID_TARGET. IndiGo's picker opens without focusing its search input, so text typed at the focus is refused.chooseused to record that as typed, tell Jev so, and spend the attempt. It now grounds the widget's search box in the same attempt and records nothing as typed.2. Native dropdowns and datalist suggestions can be chosen (
fc9d174a)A native
<select>and a text box's suggestions (<input list>with a<datalist>) show their choices in a menu Chrome draws outside the page. A click opens that menu where nothing reads or presses it, and the page looks unchanged.So BlazeDemo's departure city and Selenium's "Dropdown (select)" and "Dropdown (datalist)" could never be chosen. On the datalist,
chooselooped for 77 Jev calls, then failed.Sight offers each enabled choice as an
optioninside its control:listbox "<control>"container;selectedwhen chosen;data-tc-for) with the box it was offered under.Pressing such an option (
native_select.rs) chooses it the way a person does, without opening the menu:inputandchange;Any other
option, such as a page's ownrole="option"row, is clicked as before.Labels: a label that wraps its dropdown, and a text block that holds one, are read without the dropdown's own text. Selenium's dropdown used to be named "Dropdown (select) Open this select menu One Two Three".
3. Planner, rescuer and shaper replies are streamed (
f0f6f053)Tiny Humans' gateway answers HTTP 504 to a request that sends nothing back for 60 seconds. Unstreamed, nothing comes back until a reasoning model has written its whole reply.
On
agentic-v1, a small rescue prompt took 25 to 45 s, and a real one took longer: one task's rescue started at 12:57:44 and failed with a 504 at 12:58:44. Every hosted call is now streamed and gathered into one reply. A route that ignores streaming still answers in one piece.4. A date picker's days and arrows can be pressed (
11825daf)Bootstrap's date picker, among others, draws each day as a plain
<td>that shows a pointer only under the mouse. So sight never offered the days as controls.On Selenium's web form, a "set the date" step opened the calendar and found no day to press. Two rescues spent 133 Jev calls pressing nothing, and a third typed the date instead: 269 s in all. The paging arrows read as "»" and "«", and
chooseonly pages a calendar through a control named "next month".gridcellnamed by its number and described by the date it stands for ("15 November 2026"). Days before the month's first and after its last are dated in the months either side. A week-number column is left out.5. A crisp coverage carries a hedged condition (
abbe17e2)verify,wait_for,ifandrepeat_untilask a condition three ways: yes/no, its negation, and a five-level coverage. The coverage is there because a condition that lists several things draws a hedged yes/no but a crisp coverage answer. The two were averaged, though, and the average cannot clear the 0.75 bar unless the coverage is certain.On BlazeDemo, "the purchase form shows the passenger name, street address, city, state, and zip code" was judged 0.71 with every field filled (yes/no 0.52, coverage 0.88). The step failed, and a rescue only confirmed the form was done.
HEDGED(0.10) of an even chance now defers to a coverage whose top level is at leastCRISP_TOP(0.85). This isask::deferred.dostep's completion keeps its plain average with progress.Related issue
Closes #62. Part of tinyhumansai/openhuman#7000. The host side is tinyhumansai/openhuman#7008, and
vendor/tinycomputeris re-pinned there once this ships.API or behavior changes
No API or wire changes. Behavior changes:
is_destructive: roletabis exempt fromstop_beforenaming and from the payment-page blanket gate. It is not exempt from the label denylist.named_in_stop_beforematches only at the start of a word.choose: a refused focus type falls through totype_to_filterin the same attempt, and is no longer reported to Jev as typed.optionnodes inside their control. Labels and text blocks no longer carry a dropdown's option text.ACTION_FAILED.gridcells described by their dates, and names its arrows "next month" and "previous month".HEDGED0.10 andCRISP_TOP0.85, listed indocs/technical/decision-thresholds.md.Validation
Commands actually run on macOS (arm64). The checkout for items 2–7 was this head merged with #75:
git merge-treegives that checkout's exact tree.cargo fmt --all -- --check: pass.cargo clippy --release -p tinycomputer-browser -p tinycomputer-engine --all-targets --all-features -- -D warnings: pass. The IndiGo booking widget: city picker search is INVALID_TARGET, and a "Book" tab is classified irreversible (Kashmir flow no longer passes) #62 commits were also checked with--workspace --exclude tinycomputer-accessibility. That crate's macOS-only code has clippy failures locally that predate this change and that Linux CI never compiles. Its paste test also types into whichever app has focus on macOS.cargo test --release --all-features -p tinycomputer-browser: 109 passed, and 2 doctests passed.cargo test --release --all-features -p tinycomputer-engine: 356 passed. With-p tinycomputer: 44 passed. That makes 400 in all.TINYCOMPUTER_LIVE_BROWSER=1on Chrome for Testing 148: 6 passed, including the two new ones.scripts/build-module(release): built and attested.task_livedrove the attested module over TinyBus on Tiny Humans' routes, withagentic-v1(task_live's default) for the planner, rescue and output.verify0.80verifyfailed at 0.71 on a filled formverify0.90choosepressed day "15" in 7 Jev calls; the field holds11/15/2026. 65 sIn the BlazeDemo run after the fix, the full view's yes/no hedged twice (0.48 and 0.53) against a coverage of 0.86 and 0.91. The old average (0.67 to 0.72) would have failed the step again.
Earlier runs of the same two tasks with fixes 1–3, by model for all three roles:
agentic-v1openrouter/deepseek/deepseek-v4-flashchat-v1task_liveexits 1 when a task stops for approval, because it only counts a finished task or the payment checkpoint as a pass. Stopping before Submit is what those tasks asked for.Not yet re-run live against goindigo.in. Replaying
tasks/kashmir/plan.jsonis the next check for #62.Tests
view_tests::a_stop_before_phrase_names_a_control_only_from_the_start_of_a_wordview_tests::a_tab_is_navigation_unless_its_own_label_reads_irreversible:choose_tests::choose_finds_the_search_box_when_the_opened_widget_leaves_no_focususes a new simulator quirk,NoFocus, which gives the browser'sINVALID_TARGETrefusal. The test asserts that the city lands in the search box and that Jev is never told the refused text was typed.native_select_tests:role="option"row, and a tree ref, are clicked as before.live_tests::live_native_choices_are_offered_as_options_and_chosen_by_valueruns in a real browser and checks:change;selectedshows after choosing.live_tests::live_a_date_pickers_days_are_offered_with_the_dates_they_stand_foruses a Bootstrap-style calendar (hover-only pointer, «/», days from the months either side, a disabled day), a jQuery-UI-style one (month named in a header before the table, a "Next" link) and a "Season scores" table of 1–28. It checks:deliberation_tests::a_hedged_yes_no_defers_to_a_crisp_coverage:It fails without the fix: with the plain average restored, the first case ends
StepFailed.a_condition_split_across_views_does_not_passnow also asserts that the screen-only view asks the coverage. The simulator's oracle lets a test script a coverage directly (top_at).Deliberately not unit-tested: the streaming switch in
planner/hosted.rs. The hosted route accepts only the provider's own endpoints, so a local server cannot stand in. The client's stream handling is tested in tinyinference. Every live run after that commit planned over the stream, and anagentic-v1rescue ran its full 120 s with no 504.Documentation
docs/safety-and-privacy.mddocs/technical/architecture.mddocs/technical/decision-loops.mddocs/technical/decision-thresholds.mddocs/technical/specs/browser-sight.mddocs/technical/specs/jev-deliberation.mddocs/crates/tinycomputer-engine/flow/the-do-loop.mddocs/crates/tinycomputer-engine/flow/step-kinds.mddocs/crates/tinycomputer-engine/planner.mddocs/technical/decision-loops.mdwas already 507 lines, over the 500-line limit, before this PR. This PR adds 2 lines to it.Checklist
main.#[allow(...)],#[ignore], or relaxed lints.envcontents in the diff or the descriptionNotes for review:
role="tab". If you would rather drop the payment-page exemption for tabs, that is a one-line change.dostep's completion is untouched.HEDGEDandCRISP_TOPare documented with the other thresholds.data-tc-formark: a suggestion carries this mark besidedata-tc-seen. A suggestion pressed from an older reading fills whichever box it was last offered under.