Tokens and Costs

Read normalized token counts and costs for every response, chat, and provider attempt, including retries and cancellations.

After reading this guide, you will know:

  • How to read input, output, cache, and thinking token counts from a response.
  • How to read per-turn and per-conversation costs.
  • How to count a request’s tokens before you send it.
  • What each token bucket counts.
  • How the internal usage ledger accounts for retries, fallbacks, and cancellations.
  • How to price token usage yourself with cost_for and Cost.aggregate.
  • How to price provider tools billed per use, such as web search.
  • How costs are recorded in Rails and how to keep registry pricing fresh.
  • How to attribute chat and one-shot usage to a user or account.

Reading Tokens and Costs

Read token counts and costs directly from the response:

chat = RubyLLM.chat
response = chat.ask "Explain Ruby blocks."

response.tokens.input
response.tokens.output
response.cost.total

chat.cost.total

response.tokens is a RubyLLM::Tokens; response.cost is a RubyLLM::Cost. Both are present even when some fields are unknown. Unknown values are nil, so a missing price does not look like a free request.

More detailed readers use the same shape:

response.tokens.cache_read
response.tokens.cache_write
response.tokens.thinking
response.cost.input
response.cost.output

response.tokens aggregates every transport attempt associated with the response. chat.tokens and chat.cost aggregate the chat’s whole internal ledger, including failed retries and cancelled attempts (see The Usage Ledger below).

The same pair of value objects is used by one-shot results. Their fields remain nil when the provider does not report enough information:

embedding = RubyLLM.embed("Ruby")
embedding.tokens.input
embedding.cost.total

transcription = RubyLLM.transcribe("meeting.wav")
transcription.tokens.input
transcription.tokens.output
transcription.cost.total

RubyLLM uses token usage from the provider and pricing from the model registry. If the registry is missing pricing for tokens that were used, the affected cost and cost.total return nil instead of pretending the cost was zero. These helpers cover token-priced conversation usage; see Pricing Tool Use for tools billed per use, such as web search.

Chat responses and one-shot results resolve model pricing within the provider you called, even when another provider uses the same model ID. A transcription on Vertex AI uses Vertex AI’s prices, not the Gemini API’s. If the response names a model that is unknown for that provider, RubyLLM uses the requested model. response.model_info returns that resolved model; response.model keeps the ID returned by the provider.

When the provider reports a request’s price, cost.total uses that amount instead of a registry estimate. Component costs such as cost.input still use registry pricing. The same rule applies to batch costs.

Hosted research is billed per task. Its cost stays unknown when the service supplies no price, even if it reports zero tokens.

Counting Tokens Before You Send

Use RubyLLM.count_tokens or chat.count_tokens to measure chat input before generation. See Tokenization for examples and counting limits.

Tokenizing Plain Text

RubyLLM.tokenize returns a model’s token IDs for a string. See the Tokenization guide to inspect text and compare token counts.

Token Buckets

Token counts use the same meanings across providers:

Reader What it counts
tokens.input Standard input, excluding cache reads and writes.
tokens.output Billable output, including thinking when it is billed as output.
tokens.cache_read Input served from the prompt cache.
tokens.cache_write Input written to the prompt cache.
tokens.cache_write_by_ttl Cache writes split by lifetime, such as {"1h" => 2048}, when reported.
tokens.thinking Thinking tokens, when reported.

To measure all input activity, add the standard input, cache reads, and cache writes. A missing count is nil, so check for it before calculating a total.

Thinking tokens can already be included in tokens.output; do not add them again. When a model has distinct thinking-token pricing, cost.thinking prices that bucket separately. Otherwise it is part of cost.output.

The Usage Ledger

An LLM transcript is not an accounting ledger. A request can be retried before one response arrives, and a cancelled stream can consume tokens without producing a completed assistant message. Counting messages therefore undercounts real provider work.

RubyLLM records each physical provider attempt internally. This changes the implementation of token and cost helpers without adding a tracking object to the public API.

One response can require several physical attempts. If a transport request fails twice and the third attempt succeeds, the response’s tokens and cost aggregate the reported usage from all three attempts.

Fallbacks work the same way. When an initial model fails and a fallback produces the message, every attempt involved in that logical generation contributes to the resulting message’s aggregates.

A cancelled attempt may have no message to attach to. It still contributes to chat.tokens and chat.cost when RubyLLM has enough reported usage. If an attempt may have been billed but the provider supplied no defensible usage, the token fields for that attempt and chat.cost.total are nil rather than misleading zeroes. An attempt that provably never reached the provider, such as a refused connection or a failed TLS handshake, records zero tokens instead: it cannot have been billed, so it does not blank the total.

Pricing Usage Yourself

Once a model is in the registry, RubyLLM can turn token usage into a RubyLLM::Cost object so you can attach a dollar figure to any usage payload:

model = RubyLLM.models.find('gpt-5.6-luna')
response = RubyLLM.chat(model: model.id, provider: model.provider).ask("Summarize Ruby's object model.")

cost = model.cost_for(response.tokens)
puts cost.input
puts cost.output
puts cost.cache_read
puts cost.cache_write
puts cost.thinking
puts cost.total

When a model publishes a long-context tier, cost_for uses those rates once the prompt exceeds the registry threshold. Prompt size is input + cache_read + cache_write. Below that threshold, or when the model has no long-context tier, standard rates apply. Batch rates remain unused by cost_for; see Batches.

Most applications use the shorter helpers on messages, chats, and agents: response.cost.total, chat.cost.total, agent.cost.total.

To combine several cost objects yourself, use RubyLLM::Cost.aggregate:

cost = RubyLLM::Cost.aggregate(messages.map(&:cost))
cost.total

If pricing is incomplete for tokens that were used, the affected cost and cost.total return nil.

Pricing Tool Use

Providers bill some provider tools per use instead of per token. A web search costs the same whether it returns two results or ten. Read how many times each tool ran from tokens.server_tool_use:

response = chat.with_provider_tools(:web_search).ask "What changed in Rails this week?"
response.tokens.server_tool_use # => {"web_search_requests" => 2}

Web searches are web_search_requests on every provider that reports them. Other tools follow the same form, such as web_fetch_requests and code_execution_requests. Tools that did not run are left out, so a response that used none returns nil. chat.tokens.server_tool_use sums the whole conversation, retries included.

Price the searches at your provider’s published rate:

PRICE_PER_SEARCH = 0.01 # US dollars, from your provider's price list

searches = response.tokens.server_tool_use.to_h.fetch("web_search_requests", 0)
search_cost = searches * PRICE_PER_SEARCH

Unless the provider reports the price of a request, cost.total prices tokens only, so add the search cost to it. OpenRouter, Perplexity, and xAI report the price with their tool charges included, and cost.total already uses it.

On Gemini and Vertex AI, web_search_requests counts the distinct queries Google Search grounding ran. Gemini 3 models bill each query, but Gemini 2.5 and older bill once per grounded prompt, so price a response from those models with any searches as one charge.

Rails Persistence

With acts_as_chat, RubyLLM writes finished attempts to ruby_llm_usages immediately. This happens before the message callback, so cancellation cannot erase usage merely because no assistant message was saved.

chat = Chat.find(params[:id])
message = chat.messages.last

message.tokens.input
message.cost.total
chat.tokens.input
chat.cost.total

The ledger is internal to RubyLLM; your application still owns only its Chat and Message models. Each usage row records the operation, provider, model, status, token buckets, cost components, and timestamps in normalized numeric columns, so ordinary SQL sums and period queries work directly against ruby_llm_usages. A JSON server_tool_use column keeps the attempt’s tool use counts. Costs are frozen at completion, so a later RubyLLM.models.refresh that changes registry pricing leaves recorded usage untouched. The 2.0 upgrade moves token counts and frozen costs from pre-2.0 message columns into the ledger and removes those columns, so old and new rows read through the same path.

One-Shot Operations

Operations outside a chat record their attempts in the same ledger. RubyLLM.transcribe, embed, paint, speak, moderate, ocr, rerank, and judge each write a row for every attempt the provider may have billed, including failed and blocked ones. A video or research job writes one row when it finishes, with the cost the provider reported. A video’s cost stays unknown when its provider reports none, and a research row names the hosted agent in its model column. These rows have no chat. A chat without a record, such as RubyLLM.chat in a service object, records its rows the same way.

Pass owner: to attribute an operation’s usage to a record, such as the user who asked for it:

RubyLLM.transcribe("memo.m4a", owner: current_user)
RubyLLM.embed(document.body, owner: current_user.account)

To attribute everything a block of code does, wrap it in RubyLLM.with_usage_owner:

class ProcessUploadJob < ApplicationJob
  def perform(upload)
    RubyLLM.with_usage_owner(upload.user) do
      transcript = RubyLLM.transcribe(upload.file_path)
      RubyLLM.embed(transcript.text)
    end
  end
end

An operation’s owner: wins over the block’s owner, and a nested block restores the outer owner when it ends. Each fiber and thread keeps its own owner, and fibers and threads started inside the block inherit it. Without an owner, RubyLLM still writes the row, unattributed. An owner that is not an Active Record record, such as a user ID String, reaches the usage.ruby_llm event but leaves the row unattributed.

Rows of an acts_as_chat record keep both their chat association and the scoped owner. In a shared conversation, wrap each turn to record who caused its attempts, including failed or cancelled attempts that produce no message:

RubyLLM.with_usage_owner(current_user) do
  chat.ask("Summarize the latest reviews.")
end

The chat writes each row once. Its owner also applies to independent model calls made by tools inside the block. Re-enter the scope when a background job resumes the work. Existing rows keep their recorded owner; RubyLLM does not infer one from your chat or message models.

To query an owner’s spend, declare the association on the owner’s model:

class User < ApplicationRecord
  has_many :ruby_llm_usages, as: :owner, class_name: "RubyLLM::ActiveRecord::Usage", dependent: :nullify
end

current_user.ruby_llm_usages.sum(:total_cost)
current_user.ruby_llm_usages.where(created_at: Time.current.all_month).group(:operation).sum(:total_cost)

Writing a row never breaks the operation that billed it. If the insert fails, RubyLLM logs a warning and returns the result. Every row names what the provider ran and billed in its model column: the model ID, or for an operation without one, the provider’s own resource, such as the hosted agent of a research task or the ID and version of a Bedrock guardrail (my-guardrail:1).

Keeping Registry Pricing Fresh

Because recorded costs are frozen at completion, stale registry pricing only affects new attempts, never usage you have already saved. To keep new costs accurate, refresh the registry on a schedule. A daily job is a good default; providers change prices infrequently.

In a Rails app with the database-backed registry, the same refresh call writes the new pricing to RubyLLM’s internal model table:

# lib/tasks/ruby_llm.rake
namespace :ruby_llm do
  desc 'Refresh the model registry pricing and capabilities'
  task refresh_models: :environment do
    RubyLLM.models.refresh
  end
end

Run bin/rails ruby_llm:refresh_models from cron, or schedule the call with your background job framework (GoodJob, Sidekiq, or similar). RubyLLM does not refresh the registry on its own; you choose the cadence.

Instrumentation

Every finished physical attempt emits usage.ruby_llm. Use it when you need per-attempt detail for billing exports, metrics, or observability:

ActiveSupport::Notifications.subscribe("usage.ruby_llm") do |event|
  payload = event.payload

  Metrics.increment("llm.attempt", tags: {
    provider: payload[:provider],
    model: payload[:model],
    status: payload[:status]
  })

  Metrics.distribution("llm.input_tokens", payload[:tokens].input) if payload[:tokens].input
  Metrics.distribution("llm.cost", payload[:cost].total) if payload[:cost].total
end

The payload contains operation, provider, model, status, tokens, cost, and owner, the record given as owner: or through RubyLLM.with_usage_owner. It does not expose RubyLLM’s internal persistence record. Without Active Record, subscribe to this event to attribute spend yourself:

ActiveSupport::Notifications.subscribe("usage.ruby_llm") do |event|
  owner = event.payload[:owner]
  Spend.add(owner, event.payload[:cost].total) if owner && event.payload[:cost].total
end

status is succeeded, failed, or cancelled. tokens and cost use the same value objects returned by responses. They are always present; individual fields are nil when the provider did not report enough information. A failed or cancelled attempt may contain the token counts observed before it stopped.

Instrumentation is an observer, not the source of truth for a persisted Rails chat. RubyLLM maintains the internal ledger and emits the same normalized facts for applications that need them elsewhere.

Next Steps