Overview

Implements comprehensive OpenAI-compatible tool calling functionality for Swama's chat completions API, enabling AI models to interact with external functions and tools.

Key Features

🔧 OpenAI API Compatibility

• Full support for tools, tool_choice, and tool response formats
• Compatible with existing OpenAI tool calling workflows
• Handles all tool choice modes: "none", "auto", "required", and function-specific

🚀 Streaming & Non-Streaming Support

• Unified tool call handling for both response modes
• Real-time tool call chunks via Server-Sent Events (SSE)
• Memory-efficient streaming without accumulating responses

🔄 MLX Integration

• Seamless conversion between OpenAI tool specs and MLX ToolSpec format
• Automatic parameter serialization/deserialization
• Native MLX tool call result processing

💬 Complete Message Support

• Support for all message roles: system, user, assistant, tool
• Multimodal content handling (text + images)
• Tool execution result integration via role: "tool"

Implementation Details

New API Structures

• Tool & Function: OpenAI-compatible tool definitions
• ToolChoice: Flexible tool selection control
• ResponseToolCall & ResponseFunction: Standardized tool call responses
• Enhanced CompletionRequest with tool parameters
• Enhanced CompletionChoice with tool call results

Core Functions

• convertToolsToMLX(): OpenAI → MLX tool spec conversion
• convertToMLXChatMessages(): Message role mapping with tool support
• Separate streaming/non-streaming chat methods for better maintainability

Model Runner Improvements

• Refactored ModelRunner with runChat/runChatNonStream separation
• Optimized memory usage in streaming mode
• Sendable-safe concurrent operations

API Changes

Request Format

{
  "model": "qwen3",
  "messages": [...],
  "tools": [
    {
      "type": "function",
      "function": {
        "name": "get_weather",
        "description": "Get current weather",
        "parameters": {...}
      }
    }
  ],
  "tool_choice": "auto"
}

Response Format

{
  "choices": [
    {
      "message": {
        "role": "assistant",
        "content": "I'll get the weather for you."
      },
      "finish_reason": "tool_calls",
      "tool_calls": [
        {
          "id": "call_123",
          "type": "function",
          "function": {
            "name": "get_weather",
            "arguments": "{\"location\": \"Tokyo\"}"
          }
        }
      ]
    }
  ]
}

Streaming Format

data: {"choices":[{"delta":{"tool_calls":[{"id":"call_123","type":"function","function":{"name":"get_weather","arguments":"{\"location\":\"Tokyo\"}"}}]}}]}

Testing

Test Coverage

• ✅ 29 total tests passing (18 existing + 11 new)
• ✅ Comprehensive JSON encoding/decoding tests
• ✅ Tool conversion and parameter handling tests
• ✅ Message role validation including tool role
• ✅ Response structure validation
• ✅ Edge cases (empty arrays, nil values, invalid formats)

Build Compatibility

• ✅ Command line builds (swift build)
• ✅ Xcode project builds
• ✅ Package dependency resolution
• ✅ MLX integration compatibility

Files Modified

• Sources/SwamaKit/Server/CompletionsHandler.swift - Main implementation
• Sources/SwamaKit/Model/ModelRunner.swift - Stream/non-stream separation
• Sources/SwamaKit/Model/ModelPool.swift - Memory management
• Tests/SwamaKitTests/ToolCallingTests.swift - New test suite
• Package.swift & Package.resolved - Dependency updates

Dependencies

• Updated MLX Swift Examples to support ToolCall types
• All dependencies locked to specific commits for stability
• No new external dependencies added