effGen v0.2.9 ships the full observability and reliability stack. All telemetry is async/non-blocking — a failed export never fails inference.

Structured JSON logging with secret redaction. Every log line is a JSON object: {ts, level, module, event, attributes, trace_id, span_id}. The built-in Redactor strips OpenAI, Anthropic, Cerebras, Google, HF, Groq, Bearer, Slack, and Discord webhook patterns at the encoder — no secret ever appears in a log file.

from effgen.observability import get_logger
log = get_logger(__name__)
log.event("model.call.started", provider="cerebras", model="llama3.1-8b", cached_tokens=0)
# → {"ts": "2026-05-23T...", "level": "INFO", "event": "model.call.started", ...}

Prometheus histograms + SLO tracking. effgen_model_call_latency_seconds, effgen_tool_call_latency_seconds, effgen_agent_iteration_latency_seconds, and effgen_tokens_total now expose histogram buckets at /metrics. SLOTracker maintains a rolling-window error budget and burn_rate() at /slo.

Configurable OTel samplers + canonical span spec. Choose AlwaysOn, AlwaysOff, TraceIdRatio(p), or RateLimited(per_second) in config. effgen/observability/spans.py is the single source of truth for every span attribute name — no more scattered string literals across adapters.

Reliability primitives. Four layers now protect every adapter call:

Deterministic chaos harness. Inject NetworkTimeout, Http5xx, Http429, SlowResponse, PartialResponse, or MalformedJSON faults with Chaos(seed). Four canonical scenarios — fallback on 5xx, Retry-After honoured, timeout fires cleanly, AllProvidersFailed — all pass deterministically across 10 seeds.

Fuzz suite. Hypothesis runs 500 examples against all 66 BaseTool subclasses, random ContentPart message sequences, and the router's provider-availability logic. No unhandled exceptions, no secret leaks.

Load-testing CLI + Alertmanager rules.

# Run a 30-second load test (JSON report prints to stdout by default)
effgen loadtest --concurrency 10 --duration 30 --scenario fixed

# Or write the report to a file with --output
effgen loadtest --concurrency 10 --duration 30 --output report.json

# Integrate with Alertmanager
cp docs/observability/alert_rules.yaml /etc/prometheus/rules/effgen.yaml

See docs/observability/overview.md for full setup, docs/observability/metrics.md for all metric definitions, and docs/observability/alerting.md for Alertmanager integration.

effGen v0.2.8 makes multimodal input a first-class citizen. Send images, audio clips, and short video to any vision-capable provider through a unified Message schema — the adapter handles the translation, not your code.

Image input — Gemini, OpenAI gpt-4o, Groq, Anthropic (code-only), Together, HF. Automatic resize/MIME validation via image_pre.py. Raises CapabilityNotSupportedError cleanly when the provider doesn't support vision.

Audio input — Gemini native inline audio, OpenAI Whisper transcription + gpt-4o audio, HF Inference ASR. Auto-downsamples to 16 kHz mono; chunks files over provider max duration. Anthropic raises CapabilityNotSupportedError.

Video input — Gemini native video for providers that accept raw video; frame-sampling fallback (ffmpeg) for all others. MissingSystemDependency with install hints when ffmpeg is absent.

Unified message schema — TextPart, ImagePart, AudioPart, VideoPart form a typed ContentPart union. Message.content is always a List[ContentPart]; backwards-compatible string constructor still works.

multimodal preset — create_agent("multimodal", model) wires Gemini Flash-Lite (primary) + OpenAI gpt-4o-mini (fallback) with ImageInfo, ImageCaption, OCR, AudioTranscribe, MultimodalDescribeTool, and the full tool suite.

5 cookbook walkthroughs — image Q&A, audio transcribe + reason, video summarize, OCR + LLM structured extraction, chart reading from an image. All in docs/cookbook/.

from effgen import image_from, audio_from, video_from
from effgen.core.messages import Message, Role
from effgen.presets import create_agent
from effgen import load_model

model = load_model("gemini-2.0-flash", provider="gemini")
agent = create_agent("multimodal", model)

# Image question
img = image_from("https://upload.wikimedia.org/wikipedia/commons/thumb/4/47/PNG_transparency_demonstration_1.png/240px-PNG_transparency_demonstration_1.png")
msg = Message(role=Role.USER, content=[img, "What is in this image?"])
result = agent.run_message(msg)
print(result.output)

# Audio transcription
aud = audio_from("/tmp/clip.mp3")
msg = Message(role=Role.USER, content=[aud, "Transcribe and summarize."])
result = agent.run_message(msg)

# Multimodal preset
effgen run --preset multimodal "Describe this image" --image /tmp/photo.jpg

# Check capability
python -c "from effgen.models.capabilities import Capability; print(Capability.vision)"

See docs/multimodal/overview.md for the full architecture and docs/cookbook/README.md for the cookbook index.

effGen v0.2.7 adds a curated, domain-organized Prompt Library with 31 reusable templates across 7 domains, paired with a golden evaluation harness and an interactive playground CLI. See the full gallery.

Research — literature review (zero-shot + CoT), paper summary, citation extraction, methodology critique.

Coding — code review, bug diagnosis, refactoring plan, test generation, docstring fill.

Data / SQL — NL-to-SQL with warnings, SQL explain, SQL optimize, data profile, ETL plan.

Legal — contract summary, clause classify, research brief. All templates include mandatory legal disclaimer.

Medical — symptom triage, drug interaction, medical literature synthesis. All templates include mandatory medical disclaimer.

Creative — story continuation (zero-shot + few-shot), poetry forms, character bio, world building.

Business — meeting summary, email draft (formal/casual), OKR generation, SWOT analysis, elevator pitch.

# Discover and browse
effgen prompts list
effgen prompts list --domain research
effgen prompts list --format markdown

# Inspect and evaluate
effgen prompts show research.literature_review.v1.cot
effgen prompts eval
effgen prompts eval --domain coding --live --model llama3.1-8b

# Interactive playground
effgen prompts playground

from effgen.prompts.library import registry

p = registry.get("data.sql_from_nl.v1")
sql_prompt = p.template(
    schema_ddl="CREATE TABLE orders (id INT, customer TEXT, total FLOAT, created_at DATE)",
    question="Total revenue per customer this month",
    dialect="postgresql",
)

See docs/prompts/gallery.md for the full template catalog and docs/prompts/library.md for the framework overview.

effGen v0.2.6 adds 14 new built-in tools across document, media, and communication categories, bringing the total to 58+. Two new presets (media, notify) are also introduced.

1. OCR — OCRTool (Tesseract local + OCR.space fallback; OCRBackendUnavailable raised with install instructions).

   from effgen.tools.builtin.ocr import OCRTool
   result = OCRTool().execute({"operation": "extract", "image_path": "/tmp/scan.png"})
   print(result["data"]["text"])

2. Audio Transcription — AudioTranscribeTool (faster-whisper local; HF Inference fallback; GPU auto-detected).

   from effgen.tools.builtin.audio_transcribe import AudioTranscribeTool
   result = AudioTranscribeTool().execute({"operation": "transcribe", "audio_path": "/tmp/clip.mp3"})

3. Image Analysis — ImageInfoTool (Pillow metadata, zero network) + ImageCaptionTool (vision-capable model router).

4. Document Parsing — PDFTool (pypdf + pdfplumber), DOCXTool (python-docx), ExcelTool (openpyxl + pandas). All added to research and general presets.

   from effgen.tools.builtin.pdf import PDFTool
   result = PDFTool().execute({"operation": "text", "path": "/tmp/paper.pdf"})

5. Geo / Weather — WeatherTool (Open-Meteo, free, no auth), GeocodeTool (Nominatim/OSM, 1 req/s), MapsTool (staticmap PNG renderer).

   from effgen.tools.builtin.geocode import GeocodeTool
   result = GeocodeTool().execute({"operation": "geocode", "address": "San Francisco, CA"})

6. Email & Webhooks — EmailSMTPTool, EmailIMAPTool, SlackWebhookTool, DiscordWebhookTool. All in new notify preset. Webhook URLs are redacted in logs.

   from effgen.tools.builtin.slack_webhook import SlackWebhookTool
   result = SlackWebhookTool().execute({"operation": "post", "text": "Deploy complete!"})

See the full tool gallery for quickstart snippets for all 58+ tools.

effGen v0.2.5 adds 13 free, no-auth-required tools, bringing the built-in tool count above 44. All tools integrate with the research and general presets.

1. Academic Research — PubMedTool (NCBI, 3 ops, built-in rate limiting), ArXivTool (Atom feed + PDF download), SemanticScholarTool (search + citations + references).

   from effgen.tools.builtin.arxiv import ArXivTool
   tool = ArXivTool()
   result = tool.execute({"operation": "search", "query": "transformer attention", "max_results": 5})

2. News & RSS — RSSFeedTool (any RSS/Atom feed), NewsTool (BBC, Reuters, HN, NPR, etc. + optional NewsAPI.org key).

   from effgen.tools.builtin.news import NewsTool
   result = NewsTool().execute({"operation": "top_headlines", "category": "technology"})

3. YouTube — YouTubeTranscriptTool (captions without Google API key), YouTubeMetadataTool (via yt-dlp, public content only).

4. Social Media — RedditTool (public JSON, no OAuth), HackerNewsTool (Firebase API, no auth).

5. Translation & Language Detection — TranslateTool (LibreTranslate + offline argostranslate fallback), LanguageDetectTool (55+ languages, fully offline).

6. QR Codes — QRGenerateTool (generate locally), QRReadTool (decode from image, with OpenCV fallback if zbar is unavailable).

See the full tool gallery for quickstart snippets for all 58+ tools.

1. PolicyBasedRouter — composable routing engine with three built-in policies. Pick the cheapest provider within your budget, the fastest under your SLA, or simply the first available — and combine them freely.

   from effgen import PolicyBasedRouter, RoutingContext, CostBasedPolicy, LatencyBasedPolicy
   from effgen.models.capabilities import Capability
   
   router = PolicyBasedRouter(policies=[LatencyBasedPolicy(), CostBasedPolicy()])
   ctx = RoutingContext(
       prompt_tokens_estimate=500,
       user_budget_usd=0.01,
       latency_budget_ms=3000,
       required_capabilities={Capability.chat},
   )
   decision = router.route(ctx)
   print(decision.chosen)      # e.g., ProviderModelPair("cerebras", "llama3.1-8b")
   print(decision.eliminated)  # [(pair, reason), ...] — fully explainable

2. Transparent failover — route_and_execute(ctx, fn) retries on rate-limits / 5xx / timeouts and seamlessly moves to the next-best provider. Each hop fires a RouterEvent to registered subscribers.

   from effgen import load_model
   
   def call_provider(pair):
       model = load_model(pair.model_id, provider=pair.provider)
       return model.generate("Hello!").text
   
   router.subscribe(
       lambda event: print(
           f"Failover: {event.from_provider}/{event.from_model} "
           f"→ {event.to_provider}/{event.to_model}"
       )
   )
   result = router.route_and_execute(ctx, call_provider)

3. Cross-process SQLite rate-limit coordination — share a single rate-limit budget across multiple workers:

   from effgen import RateLimitCoordinator, SQLiteRateLimitStore
   
   store = SQLiteRateLimitStore("~/.effgen/rate_limits.sqlite")
   coordinator = RateLimitCoordinator(storage=store)  # WAL-mode, BEGIN IMMEDIATE

4. Persistent cost tracking + effgen cost CLI — every API call persists to SQLite; query spend instantly:

   effgen cost today          # per-provider per-model table
   effgen cost week           # rolling 7-day view
   effgen cost by-provider    # lifetime totals
   effgen cost set-budget 1.0 # set $1/day cap (BudgetExceededError at 100%)

5. Fully explainable decisions + budget guard — RouterDecision records every eliminated provider and why ("rate_limited", "no_key", "cost_exceeds_budget", "latency_exceeds_sla"). Configure a daily spend cap; the router automatically fails over to a free-tier provider when the budget is hit.

1. 5 new cloud backends — GroqAdapter, TogetherAdapter, FireworksAdapter, ReplicateAdapter, HFInferenceAdapter — each with streaming, native tools, rate-limit coordination, and cost tracking. 9 providers total.

   model = load_model("llama-3.1-8b-instant", provider="groq")
   model = load_model("Qwen/Qwen2.5-72B-Instruct", provider="hf")

2. Unified ProviderRegistry — list_providers(), list_models(provider), lookup(model_id) consolidated across all 9 adapters. AmbiguousModelError on bare IDs shared across providers.

3. effgen doctor — new CLI command showing which providers have API keys configured.

4. Backend parity matrix — canonical agentic task ("(17 × 23) + sqrt(144) = 403") runs identically across all providers; streaming and error surfaces verified uniform. See docs/providers/parity.md.

5. HuggingFace Router support — HFInferenceAdapter with 124-model dynamic catalog, refresh_models() + check_drift(), ModelUnavailableError with suggest_alternatives(), and custom Inference Endpoint URL.

1. Gemini 3.x/2.5/2.0 + Gemma families — full model registry with correct context windows, output limits, and feature flags; SDK migrated to google-genai>=1.0.0.

2. Gemini thinking_budget — activate Gemini's internal reasoning with GenerationConfig(thinking_budget=8192, include_thoughts=True); thinking trace surfaces in ModelResponse.metadata["thinking"].

3. Gemini grounding + Files API — GenerationConfig(grounding=True) injects Google Search; upload_file(path) passes PDFs/images to the model with a 2 GiB guard.

4. Gemini native tools — GoogleSearchTool, GeminiUrlContextTool, GeminiCodeExecutionTool activate server-side Gemini capabilities in any Agent. Parallel function calls handled automatically.

5. Anthropic Claude 4.7, extended thinking, prompt caching — full Claude 4.x registry; GenerationConfig.thinking for extended reasoning; mark_cached() + AgentConfig.cache_system_prompt/cache_tools for cache_control; cache tokens surfaced in usage.

1. Cerebras backend — 4 free-tier models (llama3.1-8b, qwen-3-235b-a22b-instruct-2507, gpt-oss-120b, zai-glm-4.7) with streaming, native function-calling, automatic RPM/TPM/RPD/TPD rate-limit coordination, and per-call cost tracking. pip install effgen[cerebras] and set CEREBRAS_API_KEY.

   from effgen import load_model
   model = load_model("llama3.1-8b", provider="cerebras")

2. OpenAI gpt-5 / gpt-5.4-nano / o-series reasoning models — full registry coverage with reasoning_effort (minimal/low/medium/high) and max_reasoning_tokens on GenerationConfig. Reasoning payloads are routed only to reasoning-capable models.

3. OpenAI prompt caching surfacing — cached_input_tokens exposed on ModelResponse.usage; AgentConfig.stable_system_prompt=True keeps the system prompt anchored at position 0 to maximize OpenAI's automatic ≥1024-token prefix cache hit rate.

4. Structured outputs v2 — OpenAIAdapter.generate_structured() with strict JSON Schema; to_openai_schema(pydantic_model) inlines $refs and forces additionalProperties: false; refusals raise ModelRefusalError.

5. OpenAI native tools — OpenAIWebSearchTool, OpenAICodeInterpreterTool, OpenAIFileSearchTool route through OpenAI's Responses API and compose with effGen's local tools in the same agent. ToolIncompatibleError fires at Agent init when paired with a non-OpenAI model.

1. Native Tool Calling — Qwen, Llama, Mistral models use built-in function calling instead of text parsing. Set tool_calling_mode="native" or "hybrid". Structured JSON/Pydantic output validation included.

2. Guardrails & Safety — PII detection, prompt injection blocking, toxicity filtering, tool permissions. One-liner: get_guardrail_preset("strict").

3. Production RAG Pipeline — Ingest PDF/DOCX/HTML/Markdown, semantic+BM25 hybrid search, reranking, inline citations. create_agent("rag", model, knowledge_base="./docs/").

4. Production API Server — OpenAI-compatible /v1/chat/completions, request queuing, agent pooling, multi-tenancy, API keys. Drop-in OpenAI replacement with local SLMs.

5. Apple Silicon Native — MLX & MLX-VLM backends for M1/M2/M3/M4. Metal GPU acceleration, unified memory. pip install effgen[mlx].