Skip to content

Detect when minimizing a window pauses capture (was: Capability polling - capture-while-minimized) #257

Open

Description

@eladalon1983

Recall what the spec says about minimized surfaces:

A display surface that is being shared may temporarily or permanently become inaccessible to the application because of actions taken by the operating system or user agent. What makes a display surface considered inaccesible is outside the scope of this specification, but examples MAY include a monitor disconnecting, window or browser closing or becoming minimized, or due to an incoming call on a phone.

So minimizing a window might pause the capture.

Suppose a Web developer wants to warn users if minimizing windows would stop the capture; we have an example of such a developer here:

We need to give the user a hint that minimizing the window will cause the screen to not capture data

Currently, there's no way for this developer to know if the warning should be issued. It would be nice if developers could query that. Ideally before getDisplayMedia() is ever called, but possibly also after, via a MediaStreamTrackCapability.

Activity

  1. youennf commented on Jan 13, 2023

    @youennf
    Collaborator

    The current wording is unclear, is there a chance we can standardize more of the behavior in this area?
    How about the following rules:

    • UA ends the track when the window gets closed
    • UA mutes/unmutes the track when the window gets minimized/unminimized

    Currently, there's no way for this developer to know if the warning should be issued

    If we standardize more of the behavior, the developer will know which warnings to provide without querying a new capability or without doing UA string checks.

  2. eladalon1983 commented on Jan 13, 2023

    @eladalon1983
    MemberAuthor

    UA ends the track when the window gets closed

    Makes sense to me.

    UA mutes/unmutes the track when the window gets minimized/unminimized

    I am not sure that's desirable. Especially on Windows, I could see a user tapping the shared window's taskbar entry to just minimize a window and get it out of the way. It would not indicate a user's desire to stop the capture.

    If we standardize more of the behavior, the developer will know which warnings to provide without querying a new capability or without doing UA string checks.

    The OS-level APIs for capture might differ in whether they allow a minimized window to be captured.

  3. jan-ivar commented on Feb 15, 2024

    @jan-ivar
    Member

    UA ends the track when the window gets closed

    This makes sense to me as well.

    UA mutes/unmutes the track when the window gets minimized/unminimized

    I am not sure that's desirable. Especially on Windows, I could see a user tapping the shared window's taskbar entry to just minimize a window and get it out of the way. It would not indicate a user's desire to stop the capture.

    I agree. But it sounds like we agree that if minimizing windows would stop the capture, then the UA MUST fire mute/unmute while minimized.

    Wouldn't this solve the OP use case?

  4. changed the title [-]Capability polling - capture-while-minimized[/-] [+]Detect when minimizing a window pauses capture (was: Capability polling - capture-while-minimized)[/+] on Feb 15, 2024
  5. jan-ivar commented on Feb 15, 2024

    @jan-ivar
    Member

    Hmm the existing wording in https://w3c.github.io/mediacapture-screen-share/#hidden-display-surfaces seems to cover this fairly well. @youennf can you elaborate on what is not clear there?

  6. youennf commented on Feb 22, 2024

    @youennf
    Collaborator

    @youennf can you elaborate on what is not clear there?

    I was probably looking only at the paragraphe above.
    The paragraphs in the spec after the quoted one above are indeed mentioning when to fire mute/unmute/ended events.

  7. eladalon1983 commented on May 21, 2024

    @eladalon1983
    MemberAuthor

    I agree. But it sounds like we agree that if minimizing windows would stop the capture, then the UA MUST fire mute/unmute while minimized.

    Wouldn't this solve the OP use case?

    It would almost solve the problem. Missing functionality includes:

    1. No way for the app to know the the track would be muted if the window minimized, until the window is in fact minimized. Possibly we could defer fixing this particular issue.
    2. No way for the app to know why the track was muted. Window minimized? It's just one of several possibilities. But if we had MuteReasons...
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions