Skip to content

Conversation

@ddupont808
Copy link
Contributor

@ddupont808 ddupont808 commented Oct 24, 2025

This PR adds desktop management commands get_desktop_environment, set_wallpaper and window management commands open, launch, get_current_window_id, get_application_windows, get_window_name, etc. to the Computer interface. These commands are useful for setting up a specific scenario, which is needed for computer-use evaluations or testing.

New Commands

Desktop Management

  • get_desktop_environment() -> str
  • set_wallpaper(path: str) -> None

Window Management

  • open(target: str) -> None
  • launch(app: str, args: list[str] | None = None) -> int | None
  • get_current_window_id() -> str
  • get_application_windows(app: str) -> list[str]
  • get_window_name(window_id: str) -> str
  • get_window_size(window_id: str) -> tuple[int, int]
  • activate_window(window_id: str) -> None
  • close_window(window_id: str) -> None
  • get_window_title(window_id: str) -> str
  • window_size(window_id: str) -> tuple[int, int]
  • set_window_size
  • set_window_position
  • get_window_position

Example Usage:

# Launch applications
await computer.interface.launch("xfce4-terminal")
await computer.interface.launch("libreoffice --writer")
await computer.interface.open("https://www.google.com")

# Window management
windows = await computer.interface.get_application_windows("xfce4-terminal")
window_id = windows[0]
await computer.interface.activate_window(window_id)

window_id = await computer.interface.get_current_window_id()  # active window id
await computer.interface.window_size(window_id)
await computer.interface.get_window_title(window_id)
await computer.interface.get_window_position(window_id)
await computer.interface.set_window_size(window_id, 1200, 800)
await computer.interface.set_window_position(window_id, 100, 100)
await computer.interface.maximize_window(window_id)
await computer.interface.minimize_window(window_id)
await computer.interface.close_window(window_id)

@jamesmurdza
Copy link
Contributor

@ddupont808 Looks like the Python files didn't format.

It should work fine after you run:

uv run pre-commit install

Let me know if you have any issues.

@ddupont808
Copy link
Contributor Author

@ddupont808 Looks like the Python files didn't format.

It should work fine after you run:

uv run pre-commit install

Let me know if you have any issues.

The linter seems to be corrupting markdown in the docs:

image

@ddupont808 ddupont808 changed the title Add get_desktop_environment, set_wallpaper to computer interface Add desktop and window management commands to computer interface Oct 24, 2025
@codecov-commenter
Copy link

codecov-commenter commented Oct 29, 2025

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

❌ Patch coverage is 47.12644% with 92 lines in your changes missing coverage. Please review.

Files with missing lines Patch % Lines
libs/python/computer/computer/interface/generic.py 22.97% 57 Missing ⚠️
libs/python/computer/computer/interface/base.py 65.30% 17 Missing ⚠️
...n/computer-server/computer_server/handlers/base.py 68.08% 15 Missing ⚠️
...omputer-server/computer_server/handlers/factory.py 33.33% 2 Missing ⚠️
...ibs/python/computer-server/computer_server/main.py 0.00% 1 Missing ⚠️

📢 Thoughts on this report? Let us know!

@ddupont808 ddupont808 merged commit f2ae964 into main Oct 29, 2025
10 of 12 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants