Tags: jambonz/llm
Tags
0.5.2: groq adapter extracts openai-processing-ms (#10) Groq's chat-completions endpoint forwards the same `openai-processing-ms` response header that OpenAI does — they're OpenAI-wire-compatible and their server side appears to honor the convention. Adding it to the groq metadata allowlist so consumers can compare wall-clock to vendor- reported processing time the same way they can for OpenAI. Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
0.5.1: add clientTiming breakdown to end event for OpenAI-wire adapte… …rs (#9) Captures three hrtime markers around the SDK call: t0 = before client.chat.completions.create() t1 = after withResponse() resolves (response headers in) t2 = first content token yielded back Emits as `clientTiming: {requestToHeadersMs, headersToFirstTokenMs?, totalMs}` on the end event. Affects all OpenAI-wire-compatible adapters that share _streaming.ts (openai, groq, huggingface, azure-openai, vertex-openai, deepseek). Native-SDK adapters (anthropic, bedrock, google, vertex-gemini) unchanged for now. Lets consumers split the request lifecycle to diagnose latency: - request→headers covers TLS handshake + RTT + vendor TTFB - headers→firstToken catches SSE buffering/chunk-transfer issues - total covers the full stream Co-authored-by: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
PreviousNext