This simple script performs the LLM request defined in manually written YAML file. It is more user-frendly alternative for curl --request POST --url https://ai.provider/v1/chat/completions --data ***.
pip install litellm
export OPENAI_API_KEY=my_key
python YamlCompletion.py <options> MyRequest.yamlFor Windows also pre-built package with all libraries included is available from the releases page.
| Argument | Explanation |
|---|---|
| -m model or -m provider/model |
Model name (LiteLLM-style provider/model is also supported) |
| -p provider | Model provider (full list) |
| -a api-base | Custom LLM API base |
| -o file | Write LLM response to file |
Also, some other options available. Full list can be seen by running python YamlCompletion.py --help.
Yaml file is sent to litellm.completion as is: all top-level key-value pairs are sent as parameters of the function.
Also, two special tags are added for inserting local files into requests:
!base64imageinserts local image file withdata:image/***;base64,header!base64fileinserts any local file as base64 text without any headers (used for audio and video data)
In this tags file path is defined relative to yaml path.
messages:
-
role: system
content: |-
You are a helpful assistant
-
role: user
content: |-
Why sky is blue?
temperature: 1.5For full list of available parameters see Litellm documentation.
messages:
-
role: system
content: |-
You are a helpful assistant
-
role: user
content:
-
type: image_url
image_url:
url: !base64image "city-streets.jpg"
-
type: text
text: "detect person and car"First request:
messages:
-
role: system
content: |-
You are a helpful assistant
-
role: user
content: |-
What's the weather like in San Francisco?
tools:
-
type: function
function:
name: get_current_weather
description: "Get the current weather in a given location"
parameters:
type: object
properties:
location:
type: string
description: "The city and state, e.g. San Francisco, CA"
unit:
type: string
enum:
- "celsius"
- "fahrenheit"
required:
- "location"And second request with tool's result:
messages:
-
role: system
content: |-
You are a helpful assistant
-
role: user
content: |-
What's the weather like in San Francisco?
-
role: assistant
tool_calls:
-
id: 'aDvEZ7iMIAJ6kOCjEnYCjDhtbKZjp7ku'
type: function
function:
name: get_current_weather
arguments: '{"location":"San Francisco, CA"}'
-
tool_call_id: 'aDvEZ7iMIAJ6kOCjEnYCjDhtbKZjp7ku'
role: tool
name: get_current_weather
content: '{"location": "San Francisco", "temperature": "72", "unit": "fahrenheit"}'
tools:
-
type: function
function:
name: get_current_weather
description: "Get the current weather in a given location"
parameters:
type: object
properties:
location:
type: string
description: "The city and state, e.g. San Francisco, CA"
unit:
type: string
enum:
- "celsius"
- "fahrenheit"
required:
- "location"