Releases: askui/vision-agent
v0.22.1
v0.22.0
What's Changed
- feat(agent): add offset and absolute position to click, type and move… by @programminx-askui in #174
- Fix python 3 14 by @programminx-askui in #177
- Feat/chat migrate persistence by @adi-wan-askui in #179
🚀 Features
-
Offset and Absolute Positioning: Enhanced the
click(),type(), andmouse_move()methods with more precise positioning capabilities:- Offset Parameter: Added
offsetparameter to fine-tune click and mouse positions relative to a target element# Click 10 pixels right and 5 pixels up from "Submit" button agent.click("Submit", offset=(10, -5)) # Move mouse 5 pixels right and 10 pixels down from "Menu" agent.mouse_move("Menu", offset=(5, 10)) # Type text in input field with offset agent.type("text", locator="Input field", offset=(5, 0))
- Absolute Coordinates: Added support for
Pointtuples to specify exact screen coordinates# Click at absolute coordinates (100, 200) agent.click((100, 200)) # Move cursor to absolute coordinates (300, 150) agent.mouse_move((300, 150)) # Click at coordinates then type agent.type("username", locator=(200, 100))
- Offset coordinates follow screen conventions: positive x moves right, negative x moves left, positive y moves down, negative y moves up
- Offset Parameter: Added
-
SQLite Database Migration: Migrated chat persistence layer from JSON files to SQLite database for improved performance, reliability, and scalability:
- Automatic Migrations: Database migrations run automatically on startup by default (configurable via
ASKUI__CHAT_API__DB__AUTO_MIGRATE) - Migrated Components:
- Assistants configuration
- Messages, runs, and threads
- File storage metadata
- MCP (Model Context Protocol) configurations
- Migration Framework: Added Alembic for database schema versioning and migrations
- Backwards Compatibility: Original JSON files are preserved during migration, allowing easy rollback by installing an older version
- Migration Documentation: Comprehensive documentation added in
docs/migrations.mdcovering:- Migration strategy and execution flow
- Manual migration commands
- Troubleshooting and best practices
- Database configuration options
- Migration Commands:
# Run all pending migrations pdm run alembic upgrade head # Show current migration status pdm run alembic current # Disable auto-migration for debugging export ASKUI__CHAT_API__DB__AUTO_MIGRATE=false
- Automatic Migrations: Database migrations run automatically on startup by default (configurable via
🐛 Bug Fixes
- Python 3.13 Support: Updated maximum Python version requirement to 3.13 as version 3.14 is not yet supported
Full Changelog: v0.21.1...v0.22.0
v0.21.1
What's Changed
- feat/configure temperature by @adi-wan-askui in #176
- fix: switch default chat model to askui-hosted model by @adi-wan-askui in #175
🚀 Features
- Temperature Configuration: Added support for configuring the temperature parameter for agent responses
- Temperature can now be set via
MessageSettings.temperature(range: 0.0 to 1.0) - Allows fine-tuning of response randomness and creativity
- Example usage:
from askui import VisionAgent from askui.models.shared.settings import ActSettings, MessageSettings message_settings = MessageSettings(temperature=0.7) act_settings = ActSettings(messages=message_settings) with VisionAgent() as agent: agent.act("Tell me a joke", settings=act_settings)
- Temperature can now be set via
🐛 Bug Fixes
- Default Chat Model: Fixed the default chat model configuration
- Changed from
claude-sonnet-4-20250514(Anthropic-hosted, which doesn't work out of the box) toaskui/claude-haiku-4-5-20251001(AskUI-hosted) - Provides faster and better responses by using Haiku instead of Sonnet 4
- Ensures the default model works out of the box without additional configuration
- Changed from
Full Changelog: v0.21.0...v0.21.1
v0.21.0
What's Changed
- feat: simplify model and model provider configuration by @adi-wan-askui in #173
🚀 Features
-
Simplified Model Provider Configuration:
-
Multi-Cloud Provider Support: Added native support for Anthropic models hosted on AWS Bedrock and Google Vertex AI. Simply install the optional dependencies (
anthropic[bedrock]oranthropic[vertex]) and configure via environment variables.import os from askui import VisionAgent # Using Vertex AI os.environ["ANTHROPIC_VERTEX_PROJECT_ID"] = "test-project" os.environ["CLOUD_ML_REGION"] = "europe-west1" with VisionAgent() as agent: agent.act("do something", model="vertex/claude-sonnet-4@20250514")
-
Provider Registry: Configure model providers through the model registry using keys ending with
"/"instead of having to configure each model separately. For example, register"openai/"as a provider and all models prefixed with"openai/"will route to that provider implementation.with VisionAgent(models={"openai/": YourOpenAiCustomMessagesApi()}) as agent: agent.act("do something", model="openai/gpt-4o") agent.act("do something", model="openai/gpt-5")
-
Default Provider Environment Variable: Added
ASKUI__VA__MODEL_PROVIDERenvironment variable to set a default model provider that automatically prefixes all model names. -
Default Model Environment Variable: Added
ASKUI__VA__MODELenvironment variable for setting the default model. Supports both simple strings and JSON for complex configurations:export ASKUI__VA__MODEL='{"act":"askui/claude-sonnet-4-20250514"}'
-
Pre-configured Providers: Added
"askui/","bedrock/","anthropic/", and"vertex/"providers by default, enabling seamless usage like:agent.act("search", model="askui/claude-sonnet-4-20250514") agent.act("search", model="bedrock/anthropic.claude-sonnet-4-5-20250929-v1:0")
-
-
New Model Support: Added support for
claude-haiku-4-5-20251001andclaude-sonnet-4-5-20250929Anthropic models with updated model cards in documentation.
📜 Documentation
- Enhanced Documentation: Significantly expanded
docs/using-models.mdwith comprehensive guides on model provider configuration, authentication, and custom provider setup.
🚨 BREAKING CHANGES
-
Removed
modelfromMessageSettings: Themodelparameter has been removed fromMessageSettings.- Migration: Use the
modelparameter ofact(),get(),locate()or pass it to a class extendingAgentBaseinstead.# Before (v0.20.x) agent.act("search", settings=ActSettings(messages=MessageSettings(model="claude-haiku-4-5"), model="askui") # After (v0.21.0) agent.act("search", model="askui/claude-haiku-4-5")
- Migration: Use the
-
Upgraded minimum
anthropicdependency to 0.72.0:- Migration: Replace
anthropic.NotGivenwithanthropic.Omitandanthropic.NOT_GIVENwithanthropic.omit.
- Migration: Replace
-
Renamed
model_choiceparameter tomodel: The parameter name has been unified acrossActModel.act(),GetModel.get(), andLocateModel.locate().- Migration: Replace all instances of
model_choice=withmodel=.# Before (v0.20.x) act_model.act("search", model_choice="claude-sonnet-4") # After (v0.21.0) act_model.act("search", model="claude-sonnet-4")
- Migration: Replace all instances of
-
Removed
ASKUI__MESSAGES__*andANTHROPIC__MESSAGES__*environment variables: These configuration variables are no longer supported.- Migration: Use the
settingsandmodelparameters when calling<agent>.act()instead.# Before (v0.20.x) # Set via environment: ASKUI__MESSAGES__MODEL=claude-sonnet-4 # After (v0.21.0) agent.act("search", model="claude-sonnet-4", settings=ActSettings(...)) # Or use: ASKUI__VA__MODEL=claude-sonnet-4
- Migration: Use the
-
Removed
AgentBase.modelproperty: Direct access to the model property has been removed fromAgentBase.- Migration: Pass the
modelparameter to individual methods (act(),get(),locate()) or set it via theVisionAgentconstructor.
- Migration: Pass the
Full Changelog: v0.20.3...v0.21.0
v0.20.3
What's Changed
- Update using-models.md by @Lumpin-askui in #169
- fix: non-serializable MCP config model by @adi-wan-askui in #171
🐛 Bug Fixes
- MCP Configuration Serialization: Fixed an issue where the MCP configuration model was non-serializable on Windows systems. The
RemoteMCPServer.authfield previously includedhttpx.Auth
type which is not serializable, causing failures when generating OpenAPI spec and trying to serialize it. This has been resolved by overriding thefastmcpmodel with a custom implementation that restricts the
authfield to fully serializable types (str | Literal["oauth"] | None), ensuring cross-platform compatibility and proper configuration persistence.
📜 Documentation
- Model Reliability Updates: Updated model usage documentation to better highlight reliability features and improvements.
Full Changelog: v0.20.2...v0.20.3
Full Changelog: v0.20.2...v0.20.3
v0.20.2
What's Changed
- Introduce cors check by @onur-askui in #168
🚀 Features
- CORS Configuration: Added configurable CORS (Cross-Origin Resource Sharing) settings for the Chat API
- Introduced
allow_originssetting inSettingsclass that allows specifying which origins are permitted to make cross-origin requests - Default allowed origins include:
http://localhost:4200(local development)https://app.caesr.aiandhttps://app-dev.caesr.ai(Caesr app)https://hub.askui.comandhttps://hub-dev.askui.com(AskUI Hub)
- Can be configured via environment variable:
ASKUI__CHAT_API__ALLOW_ORIGINS - The CORS middleware is now configured using this setting (
src/askui/chat/api/app.py:177)
- Introduced
Full Changelog: v0.20.1...v0.20.2
v0.20.1
What's Changed
- chore(models): remove 413 from retryable status codes by @adi-wan-askui in #165
- fix(playwright): delegate browser installation to user by @adi-wan-askui in #166
- feat(chat): add default model setting for chat interactions by @adi-wan-askui in #167
🚀 Features
- Configurable Chat Model: You can now configure the default model used for chat interactions via the
ASKUI__CHAT_API__MODELenvironment variable. The default model is"claude-sonnet-4-20250514". To use a different model, such as"claude-sonnet-4-5-20250929", set the environment variable:This setting is applied globally across all chat interactions in the API (src/askui/chat/api/settings.py:69-72).export ASKUI__CHAT_API__MODEL="claude-sonnet-4-5-20250929"
🐛 Bug Fixes
-
Playwright Browser Installation: Fixed an issue where the Playwright MCP integration would fail with installation instructions that were not actionable for users. The error messages now correctly delegate browser installation to the user with a clear message: "Download and install the browser to continue." The system detects when a Playwright tool error contains installation instructions and replaces them with a user-friendly message (src/askui/models/shared/tools.py:142-158).
-
HTTP 413 Status Code Handling: Removed HTTP status code 413 (Request Entity Too Large) from the list of retryable errors. When a request is too large, retrying will not resolve the issue, so the system now fails fast instead of attempting unnecessary retries (src/askui/models/askui/retry_utils.py:48).
Full Changelog: v0.20.0...v0.20.1
v0.20.0
What's Changed
- refactor(models/askui): use anthropic client instead of httpx client to connect to inference api by @adi-wan-askui in #157
- Preserve custom MCP config on chat start; update defaults only by @mlikasam-askui in #163
- feat!(logging): switch from app like logging to lib like logging by @adi-wan-askui in #161
- feat(chat)!: improve telemetry (logging, monitoring, tracing etc.) by @adi-wan-askui in #162
- feat(chat): filter logs using `ASKUI__CHAT_API__TELEMETRY__LOG__FILTERS\ by @adi-wan-askui in #164
🚀 Features
- Library-style Logging: Switched from application-style to library-style logging, leaving configuration to the consuming application
- Telemetry (Chat): Major overhaul with structured logging using structlog, added request/correlation IDs, and metrics endpoint for Prometheus monitoring
- Log Filtering (Chat): Added
ASKUI__CHAT_API__TELEMETRY__LOG__FILTERSenvironment variable to filter out unwanted logs in chat (OPTIONS requests, health checks, metrics endpoint)
🐛 Bug Fixes
- MCP Config: Fixed issue where custom MCP configurations were being overwritten on chat start - now preserves custom configs while updating only defaults
- API Stability: Improved error forwarding and retry logic to better protect users from API instability issues
🚨 BREAKING CHANGES
- Logging: Removed `log_level` parameter from all agent constructors (VisionAgent, AndroidVisionAgent, WebVisionAgent, WebTestingAgent, AgentBase)
- Environment Variables:
ASKUI__CHAT_API__LOG_LEVELreplaced byASKUI__CHAT_API__TELEMETRY__LOG__LEVEL - Log Format: Changed to structured logging format (logfmt by default)
- Claude Support: Removed support for Claude Sonnet 3.5
Full Changelog: v0.19.1...v0.20.0
v0.19.1
What's Changed
- Preserve custom assistance on chat start, update only default assistance by @mlikasam-askui in #158
- fix(chat): run freezing on listing mcp tools by @adi-wan-askui in #159
🐛 Bug Fixes
- Chat: Fixed run freezing issue when listing MCP tools by using fastmcp within runner by pinned anyio version to 4.10.0
- Chat: Fixed issue where custom assistants were being overwritten on startup of chat api - now only default assistance are overridden while preserving custom assistants
Full Changelog: v0.19.0...v0.19.1
v0.19.0
What's Changed
- feat(chat): remove experimental default agents and theme existing agents by @adi-wan-askui in #156
🚀 Features
- Chat: Updated avatars and system prompt of default agents to fit more the Caesr brand
🚨 BREAKING CHANGES
- Agents: Removed experimental default agents (orchestrator, testing) until they are not experimental anymore. These agents are no longer available by default.
Full Changelog: v0.18.2...v0.19.0