fix(android): keep the device in touch mode while typing text - #492

Summary

On Android, typing text took the device out of touch mode:

$ adb shell input tap 600 300 && adb shell dumpsys window | grep -m1 -o 'mInTouchMode=[a-z]*'
mInTouchMode=true
$ mobilecli io text "a" --device <android> && adb shell dumpsys window | grep -m1 -o 'mInTouchMode=[a-z]*'
mInTouchMode=false

io text injects the key events of the virtual keyboard's character map with no extra flags. A key that types a character leaves touch mode unless it carries KeyEvent.FLAG_KEEP_TOUCH_MODE, which is what an on-screen keyboard sets. Out of touch mode Android behaves as if a hardware keyboard is in use: the next screen gives focus to a view, and a scroll view jumps to it. A person typing on the on-screen keyboard never sees that.

It only shows when nothing taps between the typing and the next screen: a field that submits on its own, or a form submitted with a line break.

Changes

• agents/android/java/Input.java: the events typeText injects carry FLAG_KEEP_TOUCH_MODE.
• devices/android.go: a line break sent as text (io text "\n") is typed like any other character, so it gets the flag too. It was rerouted to the enter button before.
• io button ENTER, the d-pad and the other buttons are unchanged: they are hardware keys and are meant to leave touch mode, which keyboard and tv navigation depend on.

Testing

• New e2e group "touch mode while typing" in test/android.spec.ts (4 tests): typed keys that reach the app keep touch mode, text still lands in a field, a line break as text keeps touch mode, and the enter button still leaves it. Without the fix the first fails ("Expected: true, Received: false") and so does the line-break one; with it all four pass.
• Emulator e2e, full suite on Android 16: 54 passed.
• By hand on an Android 12 emulator (API 31): without the flag io text turns touch mode off, with it touch mode stays on, and typed text lands in the Settings search field.
• By hand on Android 16: line one, a line break and line two\nline three typed into a multi-line field give three lines.
• go test ./... passes; make -C agents/android lint passes.
• Not tested: a real device, and the UiAutomation fallback injection path (the flag was checked on the direct InputManager path).

One thing the tests showed: on the emulator, with a field in focus and Gboard showing, plain typed text did not leave touch mode even before the fix, because the keyboard takes the keys first. The issue reports it with a focused field on a Samsung device, so this depends on the keyboard in use. The enter key did leave touch mode with a field in focus.

Related Issues

Fixes #491

Summary by CodeRabbit

• Bug Fixes
  • Android text entry now keeps the screen in touch mode while typing, including when entering a line break.
  • Line breaks are handled as typed text rather than triggering the hardware Enter action. Pressing hardware Enter continues to exit touch mode.
  • Typing ordinary text and line breaks in the Android playground’s Basic UI no longer switches out of touch mode.