Remove mode 1004 (focus reporting) from XM termcap - #13440

Hm, it seems to have been added to the terminfo database (and also to xterm), see e.g. here: https://github.com/ThomasDickey/xterm-snapshots/blob/4130ad43faa71d94ad1508d57d1c22237b09e012/terminfo#L972-L974

and here: https://invisible-island.net/xterm/xterm.log.html#xterm_224 and https://invisible-island.net/xterm/xterm-paste64.html (search for Bram)

Perhaps @ThomasDickey wants to commen there (is it supported in xterm or what is the plan with it)?

Bram asked for this in a conversation with me beginning January 12, 2023.

I didn't make the change immediately, since it takes time to verify which terminal descriptions to modify.

Certainly it is supported in xterm. Other terminal emulators may respond to the mode, and may (or may not) do this incorrectly. Well-known terminal emulators have their own terminal descriptions, making it unnecessary to piggy-back on my work in xterm. This may be related to a recent bug report dealing with VTE (see this).

Thanks @chrisbra for the references!

Hm, it seems to have been added to the terminfo database (and also to xterm), see e.g. here: https://github.com/ThomasDickey/xterm-snapshots/blob/4130ad43faa71d94ad1508d57d1c22237b09e012/terminfo#L972-L974

This mentions:

xterm patch #224 2007/2/11 added private mode 1004, for enabling/disabling focus in/out event reporting. The 1004 is normally part of XM in a different building-block, e.g., for reporting any events.

@ThomasDickey Am I understanding this correctly that 1004 used to be included in XM with a different meaning ("reporting any events"), and after patch #224 was repurposed to be used for focus reporting? (I'm not sure what "building-block" means in this context).

and here: https://invisible-island.net/xterm/xterm.log.html#xterm_224 and https://invisible-island.net/xterm/xterm-paste64.html (search for Bram)

These links discuss the addition of mode 1004 for reporting focus events, which is already used by Vim in t_fe and t_fd. I'm trying to figure out why mode 1004 is also included in XM in the Vim documentation.

One line in here says:

A couple of years later, I added control sequences which could be used to notify an application that the mouse pointer had moved in or out of the window (patch #224)

This does seem to support including focus reporting mode in the XM sequence (I guess this is used to implement the so-called "focus follows mouse" feature?) But then my follow up question is: why do none of the XM sequences mentioned in my original post (Vim's builtin XM value or ncurses' documentation) not include or mention mode 1004? Indeed, xterm-256color itself only uses 1000 and 1006 in its XM value.

To clarify my motivation here: I know of at least one instance where someone included 1004 in their XM terminfo entry and cited Vim's documentation here as the justification for doing so, but this caused bugs in at least one known program (htop) which supports the mouse, but not focus events. So any time the window that htop was running in gained or lost focus, it incorrectly processed an I or O key event.

No - XM (as you can see from the source-history) used to be defined without 1004. ncurses uses XM to enable/disable mouse mode; there was no corresponding enable/disable capability for focus-events because that was only used in the alternate xterm-1003 and xterm-1002 descriptions (and that dates from 2022-03-19).

Since May 2023, xterm-256color uses 1004 (which is what we're discussing).

It would be possible to "move" that 1004 to a new capability which only enables/disables the focus-event, except for a couple of problems:

• 1004 is only meaningful in conjunction with the 1006/1000/etc
• vim would only be able to use it as a 2-character name (because vim uses only the termcap interface)
• it wouldn't solve the longstanding problem of buggy implementations such as VTE

It would be possible to "move" that 1004 to a new capability which only enables/disables the focus-event, except for a couple of problems:

• 1004 is only meaningful in conjunction with the 1006/1000/etc

I am not sure why you say it's only meaningful in conjunction with 1006/1000. Terminal applications can receive focus events independently of mouse events. Consider a keyboard driven window manager or tmux, both of which can focus a terminal window without ever moving the mouse. Even if one does use the mouse to focus a terminal (e.g. by clicking the operating system window), the focus event is sent by the windowing library and need not report any mouse events.

In practice, there does seem to be a meaningful distinction. Vim, tmux, and Neovim (to name a few terminal applications I'm at least somewhat familiar with) all have the ability to enable focus events independent of mouse events.

• vim would only be able to use it as a 2-character name (because vim uses only the termcap interface)

Vim already uses t_fe and t_fd. Could that 2-character name not be used for this hypothetical new mode?

• it wouldn't solve the longstanding problem of buggy implementations such as VTE

I cannot speak to that, as I don't have any experience using libvte or any of the terminal emulators which use it. But as I mentioned above, I do know that existing ncurses applications such as htop exhibit bugs when focus reporting is coupled with mouse events.

fe/fd appear to be ok (to use in a terminal description).

so I guess we should keep it in the documentation?

I'm still in favor of removing it. To recap the discussion:

1. Vim itself does not include 1004 in its builtin XM entry (so the docs are inconsistent)
2. Most XM entries in the wild do not include 1004 (apparently xterm-256color now does, though in the most recent version of ncurses I have on my machine xterm-256color still has only 1000 and 1006 in XM)
3. I do not think that focus reporting and mouse reporting should be coupled, for the reasons I mentioned in this comment. In practice, my impression is that many terminal applications already treat focus reporting and mouse reporting separately. I propose making focus reporting a separate terminfo capability, rather than bundling it into XM. I suggested fe/fd for this (which is Vim already uses), which Thomas seemed to be ok with.

See my change here:

20231028
        + move xterm focus mode 1004 from xterm+sm+1006 into xterm+focus as
          fe/fd capabilities, like vim (vim-pr #13440).

thanks all. How about this then:

diff --git a/runtime/doc/term.txt b/runtime/doc/term.txt
index 72af6d3df..38e4f515a 100644
--- a/runtime/doc/term.txt
+++ b/runtime/doc/term.txt
@@ -332,11 +332,16 @@ using the "xterm" workaround.  These are the relevant entries (so far):

        XM      "\033[?1006;1004;1000%?%p1%{1}%=%th%el%;"
                                mouse enable / disable |t_XM|
+       FE      "\033[?1004h"   enable focus event tracking |t_fe|
+       FD      "\033[?1004l"   disable focus event tracking |t_fd|

 The "XM" entry includes "1006" to enable SGR style mouse reporting.  This
 supports columns above 223.  It also includes "1004" which enables focus
-reporting.  The t_fe and t_fd entries can be left empty (they don't have
-entries in terminfo/termcap anyway).
+reporting.
+Note: As of 2023, the "1004" is currently not used by Vim itself, instead
+it's recommended to set focus reporting independently of mouse tracking by
+the |t_fe| and |t_fd| entries, as ncurses also starts to use with the latest
+versions (and will then also end up in terminfo/termcap).

                                                *xterm-kitty* *kitty-terminal*
 The Kitty terminal is a special case.  Mainly because it works differently

looks fine (I would do "it is" rather than "it's").