Build AI agents and workflows in minutes with one toolkit and built-in connectors to 25+ API Providers & Vector Databases.
LLM Tornado is a .NET provider-agnostic SDK that empowers developers to build, orchestrate, and deploy AI agents and workflows. Whether you're building a simple chatbot or an autonomous coding agent, LLM Tornado provides the tools you need with unparalleled integration into the AI ecosystem.
- Use Any Provider: Built-in connectors to: Alibaba, Anthropic, Azure, Blablador, Cohere, DeepInfra, DeepSeek, Google, Groq, Mistral, MoonshotAI, OpenAI, OpenRouter, Perplexity, Voyage, xAI, Z.ai, and more. Connectors expose all niche/unique features via strongly typed code and are up-to-date with the latest AI development. No dependencies on first-party SDKs. Feature Matrix tracks detailed endpoint support.
- First-class Local Deployments: Run with vLLM, Ollama, or LocalAI with integrated support for request transformations.
- Agents Orchestration: Coordinate specialist agents that can autonomously perform complex tasks with three core concepts:
Orchestrator(graph),Runner(node), andAdvancer(edge). Comes with handoffs, parallel execution, Mermaid export, and builder pattern to keep it simple. - Rapid Development: Write pipelines once, execute with any Provider by changing the model's name. Connect your editor to Context7 or FSKB to accelerate coding with instant access to vectorized documentation.
- Fully Multimodal: Text, images, videos, documents, URLs, and audio inputs/outputs are supported.
- Cutting Edge Protocols:
- MCP: Connect agents to data sources, tools, and workflows via Model Context Protocol with
LlmTornado.Mcp. - A2A: Enable seamless collaboration between AI agents across different platforms with
LlmTornado.A2A. - Skills: Dynamically load folders of instructions, scripts, and resources to improve performance on specialized tasks.
- MCP: Connect agents to data sources, tools, and workflows via Model Context Protocol with
- Vector Databases: Built-in connectors to Chroma, PgVector, Pinecone, Faiss, and QDrant.
- Integrated: Interoperability with Microsoft.Extensions.AI enables plugging Tornado in Semantic Kernel applications with
LlmTornado.Microsoft.Extensions.AI. - Enterprise Ready: Guardrails framework, preview and transform any request before committing to it. Open Telemetry support. Stable APIs.
➡️ Get started in minutes – Quickstart ⬅️
- 25/10 - LLM Tornado is featured in dotInsights by JetBrains. Microsoft uses LLM Tornado in Generative AI for Beginners .NET. Interoperability with Microsoft.Extensions.AI is launched. Skills protocol is implemented.
- 25/09 - Maintainers Matěj Štágl and John Lomba talk about LLM Tornado in .NET AI Community Standup. PgVector and ChromaDb connectors are implemented.
- 25/08 - ProseFlow is built with LLM Tornado. Sciobot – an AI platform for Educators built with LLM Tornado is accepted into Cohere Labs Catalyst Grant Program. A2A protocol is implemented.
- 25/07 - Contributor Shaltiel Shmidman talks about LLM Tornado in Asp.net Community Standup. New connectors to Z.ai, Alibaba, and Blablador are implemented.
- 25/06 - C# delegates as Agent tools system is added, freeing applications from authoring JSON schema manually. MCP protocol is implemented.
- 25/05 - Chat to Responses system is added, allowing usage of
/responsesexclusive models from/chatendpoint. - 25/04 - New connectors to DeepInfra, DeepSeek, and Perplexity are added.
- 25/03 - Assistants are implemented.
- 25/02 - New connectors to OpenRouter, Voyage, and xAI are added.
- 25/01 - Strict JSON mode is implemented, Groq and Mistral connectors are added.
- Chat with your documents
- Make multiple-speaker podcasts
- Voice call with AI using your microphone
- Orchestrate Assistants
- Generate images
- Summarize a video (local file / YouTube)
- Turn text & images into high quality embeddings
- Transcribe audio in real time
Install LLM Tornado via NuGet:
dotnet add package LlmTornadoOptional addons:
dotnet add package LlmTornado.Agents # Agentic framework, higher-level abstractions
dotnet add package LlmTornado.Mcp # Model Context Protocol (MCP) integration
dotnet add package LlmTornado.A2A # Agent2Agent (A2A) integration
dotnet add package LlmTornado.Microsoft.Extensions.AI # Semantic Kernel interoperability
dotnet add package LlmTornado.Contrib # productivity, quality of life enhancementsInferencing across multiple providers is as easy as changing the ChatModel argument. Tornado instance can be constructed with multiple API keys, the correct key is then used based on the model automatically:
TornadoApi api = new TornadoApi([
// note: delete lines with providers you won't be using
new (LLmProviders.OpenAi, "OPEN_AI_KEY"),
new (LLmProviders.Anthropic, "ANTHROPIC_KEY"),
new (LLmProviders.Cohere, "COHERE_KEY"),
new (LLmProviders.Google, "GOOGLE_KEY"),
new (LLmProviders.Groq, "GROQ_KEY"),
new (LLmProviders.DeepSeek, "DEEP_SEEK_KEY"),
new (LLmProviders.Mistral, "MISTRAL_KEY"),
new (LLmProviders.XAi, "XAI_KEY"),
new (LLmProviders.Perplexity, "PERPLEXITY_KEY"),
new (LLmProviders.Voyage, "VOYAGE_KEY"),
new (LLmProviders.DeepInfra, "DEEP_INFRA_KEY"),
new (LLmProviders.OpenRouter, "OPEN_ROUTER_KEY")
]);
// this sample iterates a bunch of models, gives each the same task, and prints results.
List<ChatModel> models = [
ChatModel.OpenAi.O3.Mini, ChatModel.Anthropic.Claude37.Sonnet,
ChatModel.Cohere.Command.RPlus, ChatModel.Google.Gemini.Gemini2Flash001,
ChatModel.Groq.Meta.Llama370B, ChatModel.DeepSeek.Models.Chat,
ChatModel.Mistral.Premier.MistralLarge, ChatModel.XAi.Grok.Grok2241212,
ChatModel.Perplexity.Sonar.Default
];
foreach (ChatModel model in models)
{
string? response = await api.Chat.CreateConversation(model)
.AppendSystemMessage("You are a fortune teller.")
.AppendUserInput("What will my future bring?")
.GetResponse();
Console.WriteLine(response);
}💡 Instead of passing in a strongly typed model, you can pass a string instead: await api.Chat.CreateConversation("gpt-4o"), Tornado will automatically resolve the provider.
Tornado has a powerful concept of VendorExtensions which can be applied to various endpoints and are strongly typed. Many Providers offer unique/niche APIs, often enabling use cases otherwise unavailable. For example, let's set a reasoning budget for Anthropic's Claude 3.7:
public static async Task AnthropicSonnet37Thinking()
{
Conversation chat = Program.Connect(LLmProviders.Anthropic).Chat.CreateConversation(new ChatRequest
{
Model = ChatModel.Anthropic.Claude37.Sonnet,
VendorExtensions = new ChatRequestVendorExtensions(new ChatRequestVendorAnthropicExtensions
{
Thinking = new AnthropicThinkingSettings
{
BudgetTokens = 2_000,
Enabled = true
}
})
});
chat.AppendUserInput("Explain how to solve differential equations.");
ChatRichResponse blocks = await chat.GetResponseRich();
if (blocks.Blocks is not null)
{
foreach (ChatRichResponseBlock reasoning in blocks.Blocks.Where(x => x.Type is ChatRichResponseBlockTypes.Reasoning))
{
Console.ForegroundColor = ConsoleColor.DarkGray;
Console.WriteLine(reasoning.Reasoning?.Content);
Console.ResetColor();
}
foreach (ChatRichResponseBlock reasoning in blocks.Blocks.Where(x => x.Type is ChatRichResponseBlockTypes.Message))
{
Console.WriteLine(reasoning.Message);
}
}
}Instead of consuming commercial APIs, one can easily roll their inference servers with a plethora of available tools. Here is a simple demo for streaming response with Ollama, but the same approach can be used for any custom provider:
public static async Task OllamaStreaming()
{
TornadoApi api = new TornadoApi(new Uri("http://localhost:11434")); // default Ollama port, API key can be passed in the second argument if needed
await api.Chat.CreateConversation(new ChatModel("falcon3:1b")) // <-- replace with your model
.AppendUserInput("Why is the sky blue?")
.StreamResponse(Console.Write);
}If you need more control over requests, for example, custom headers, you can create an instance of a built-in Provider. This is useful for custom deployments like Amazon Bedrock, Vertex AI, etc.
TornadoApi tornadoApi = new TornadoApi(new AnthropicEndpointProvider
{
Auth = new ProviderAuthentication("ANTHROPIC_API_KEY"),
// {0} = endpoint, {1} = action, {2} = model's name
UrlResolver = (endpoint, url, ctx) => "https://api.anthropic.com/v1/{0}{1}",
RequestResolver = (request, data, streaming) =>
{
// by default, providing a custom request resolver omits beta headers
// request is HttpRequestMessage, data contains the payload
},
RequestSerializer = (data, ctx) =>
{
// data is JObject, which can be modified before
// being serialized into a string.
}
});clip.mp4
Tornado offers three levels of abstraction, trading more details for more complexity. The simple use cases where only plaintext is needed can be represented in a terse format:
await api.Chat.CreateConversation(ChatModel.Anthropic.Claude3.Sonnet)
.AppendSystemMessage("You are a fortune teller.")
.AppendUserInput("What will my future bring?")
.StreamResponse(Console.Write);The levels of abstraction are:
Response(stringfor chat,float[]for embeddings, etc.)ResponseRich(tools, modalities, metadata such as usage)ResponseRichSafe(same as level 2, guaranteed not to throw on network level, for example, if the provider returns an internal error or doesn't respond at all)
When plaintext is insufficient, switch to StreamResponseRich or GetResponseRich() APIs. Tools requested by the model can be resolved later and never returned to the model. This is useful in scenarios where we use the tools without intending to continue the conversation:
//Ask the model to generate two images, and stream the result:
public static async Task GoogleStreamImages()
{
Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
Model = ChatModel.Google.GeminiExperimental.Gemini2FlashImageGeneration,
Modalities = [ ChatModelModalities.Text, ChatModelModalities.Image ]
});
chat.AppendUserInput([
new ChatMessagePart("Generate two images: a lion and a squirrel")
]);
await chat.StreamResponseRich(new ChatStreamEventHandler
{
MessagePartHandler = async (part) =>
{
if (part.Text is not null)
{
Console.Write(part.Text);
return;
}
if (part.Image is not null)
{
// In our tests this executes Chafa to turn the raw base64 data into Sixels
await DisplayImage(part.Image.Url);
}
},
BlockFinishedHandler = (block) =>
{
Console.WriteLine();
return ValueTask.CompletedTask;
},
OnUsageReceived = (usage) =>
{
Console.WriteLine();
Console.WriteLine(usage);
return ValueTask.CompletedTask;
}
});
}Tools requested by the model can be resolved and the results returned immediately. This has the benefit of automatically continuing the conversation:
Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
Model = ChatModel.OpenAi.Gpt4.O,
Tools =
[
new Tool(new ToolFunction("get_weather", "gets the current weather", new
{
type = "object",
properties = new
{
location = new
{
type = "string",
description = "The location for which the weather information is required."
}
},
required = new List<string> { "location" }
}))
]
})
.AppendSystemMessage("You are a helpful assistant")
.AppendUserInput("What is the weather like today in Prague?");
ChatStreamEventHandler handler = new ChatStreamEventHandler
{
MessageTokenHandler = (x) =>
{
Console.Write(x);
return Task.CompletedTask;
},
FunctionCallHandler = (calls) =>
{
calls.ForEach(x => x.Result = new FunctionResult(x, "A mild rain is expected around noon.", null));
return Task.CompletedTask;
},
AfterFunctionCallsResolvedHandler = async (results, handler) => { await chat.StreamResponseRich(handler); }
};
await chat.StreamResponseRich(handler);Instead of resolving the tool call, we can postpone/quit the conversation. This is useful for extractive tasks, where we care only for the tool call:
Conversation chat = api.Chat.CreateConversation(new ChatRequest
{
Model = ChatModel.OpenAi.Gpt4.Turbo,
Tools = new List<Tool>
{
new Tool
{
Function = new ToolFunction("get_weather", "gets the current weather")
}
},
ToolChoice = new OutboundToolChoice(OutboundToolChoiceModes.Required)
});
chat.AppendUserInput("Who are you?"); // user asks something unrelated, but we force the model to use the tool
ChatRichResponse response = await chat.GetResponseRich(); // the response contains one block of type FunctionGetResponseRichSafe() API is also available, which is guaranteed not to throw on the network level. The response is wrapped in a network-level wrapper, containing additional information. For production use cases, either use try {} catch {} on all the HTTP request-producing Tornado APIs, or use the safe APIs.
To use the Model Context Protocol, install the LlmTornado.Mcp adapter. After that, new interop methods will become available on the ModelContextProtocol types. The following example uses the GetForecast tool defined on an example MCP server:
[McpServerToolType]
public sealed class WeatherTools
{
[McpServerTool, Description("Get weather forecast for a location.")]
public static async Task<string> GetForecast(
HttpClient client,
[Description("Latitude of the location.")] double latitude,
[Description("Longitude of the location.")] double longitude)
{
var pointUrl = string.Create(CultureInfo.InvariantCulture, $"/points/{latitude},{longitude}");
using var jsonDocument = await client.ReadJsonDocumentAsync(pointUrl);
var forecastUrl = jsonDocument.RootElement.GetProperty("properties").GetProperty("forecast").GetString()
?? throw new Exception($"No forecast URL provided by {client.BaseAddress}points/{latitude},{longitude}");
using var forecastDocument = await client.ReadJsonDocumentAsync(forecastUrl);
var periods = forecastDocument.RootElement.GetProperty("properties").GetProperty("periods").EnumerateArray();
return string.Join("\n---\n", periods.Select(period => $"""
{period.GetProperty("name").GetString()}
Temperature: {period.GetProperty("temperature").GetInt32()}°F
Wind: {period.GetProperty("windSpeed").GetString()} {period.GetProperty("windDirection").GetString()}
Forecast: {period.GetProperty("detailedForecast").GetString()}
"""));
}
}The following is done by the client:
// your clientTransport, for example StdioClientTransport
await using IMcpClient mcpClient = await McpClientFactory.CreateAsync(clientTransport);
// 1. fetch tools
List<Tool> tools = await mcpClient.ListTornadoToolsAsync();
// 2. create a conversation, pass available tools
TornadoApi api = new TornadoApi(LLmProviders.OpenAi, apiKeys.OpenAi);
Conversation conversation = api.Chat.CreateConversation(new ChatRequest
{
Model = ChatModel.OpenAi.Gpt41.V41,
Tools = tools,
// force any of the available tools to be used (use new OutboundToolChoice("toolName") to specify which if needed)
ToolChoice = OutboundToolChoice.Required
});
// 3. let the model call the tool and infer arguments
await conversation
.AddSystemMessage("You are a helpful assistant")
.AddUserMessage("What is the weather like in Dallas?")
.GetResponseRich(async calls =>
{
foreach (FunctionCall call in calls)
{
// retrieve arguments inferred by the model
double latitude = call.GetOrDefault<double>("latitude");
double longitude = call.GetOrDefault<double>("longitude");
// call the tool on the MCP server, pass args
await call.ResolveRemote(new
{
latitude = latitude,
longitude = longitude
});
// extract the tool result and pass it back to the model
if (call.Result?.RemoteContent is McpContent mcpContent)
{
foreach (IMcpContentBlock block in mcpContent.McpContentBlocks)
{
if (block is McpContentBlockText textBlock)
{
call.Result.Content = textBlock.Text;
}
}
}
}
});
// stop forcing the client to call the tool
conversation.RequestParameters.ToolChoice = null;
// 4. stream final response
await conversation.StreamResponse(Console.Write);A complete example is available here: client, server.
Tornado includes powerful abstractions in the LlmTornado.Toolkit package, allowing rapid development of applications, while avoiding many design pitfalls. Scalability and tuning-friendly code design are at the core of these abstractions.
ToolkitChat is a primitive for graph-based workflows, where edges move data and nodes execute functions. ToolkitChat supports streaming, rich responses, and chaining tool calls. Tool calls are provided via ChatFunction or ChatPlugin (an envelope with multiple tools). Many overloads accept a primary and a secondary model acting as a backup, this zig-zag strategy overcomes temporary downtime in APIs better than simple retrying of the same model. All tool calls are strongly typed and strict by default. For providers, where a strict JSON schema is not supported (Anthropic, for example), prefill with { is used as a fallback. Call can be marked as non-strict by simply changing a parameter.
class DemoAggregatedItem
{
public string Name { get; set; }
public string KnownName { get; set; }
public int Quantity { get; set; }
}
string sysPrompt = "aggregate items by type";
string userPrompt = "three apples, one cherry, two apples, one orange, one orange";
await ToolkitChat.GetSingleResponse(Program.Connect(), ChatModel.Google.Gemini.Gemini25Flash, ChatModel.OpenAi.Gpt41.V41Mini, sysPrompt, new ChatFunction([
new ToolParam("items", new ToolParamList("aggregated items", [
new ToolParam("name", "name of the item", ToolParamAtomicTypes.String),
new ToolParam("quantity", "aggregated quantity", ToolParamAtomicTypes.Int),
new ToolParam("known_name", new ToolParamEnum("known name of the item", [ "apple", "cherry", "orange", "other" ]))
]))
], async (args, ctx) =>
{
if (!args.ParamTryGet("items", out List<DemoAggregatedItem>? items) || items is null)
{
return new ChatFunctionCallResult(ChatFunctionCallResultParameterErrors.MissingRequiredParameter, "items");
}
Console.WriteLine("Aggregated items:");
foreach (DemoAggregatedItem item in items)
{
Console.WriteLine($"{item.Name}: {item.Quantity}");
}
return new ChatFunctionCallResult();
}), userPrompt); // temp defaults to 0, output length to 8k
/*
Aggregated items:
apple: 5
cherry: 1
orange: 2
*/- 50,000+ installs on NuGet (previous names Lofcz.Forks.OpenAI, OpenAiNg, currently LlmTornado).
- Used in award-winning commercial projects, processing > 100B tokens monthly.
- Covered by 250+ tests.
- Great performance.
- The license will never change.
- ScioBot - AI For Educators, 100k+ users.
- ProseFlow - Your universal AI text processor, powered by local and cloud LLMs. Edit, refactor, and transform text in any application on Windows, macOS, and Linux.
- NotT3Chat - The C# Answer to the T3 Stack.
- ClaudeCodeProxy - Provider multiplexing proxy.
- Semantic Search - AI semantic search where a query is matched by context and meaning.
Have you built something with Tornado? Let us know about it in the issues to get a spotlight!
PRs are welcome! We are accepting new Provider implementations, contributions towards a 100 % green Feature Matrix, and, after public discussion, new abstractions.
This library is licensed under the MIT license. 💜