Hi, we're Arkency

TERAZ - live collective ambient experience made with Rails & Tone.js

2026-09-18T20:50:08Z

TERAZ - live collective ambient experience made with Rails & Tone.js

The first time TERAZ actually worked, there were around seventy phones in the audience at the wroclove.rb conference.

Before pressing start, I was still not completely sure what would happen. Local tests are one thing. Seventy people with their own phones, batteries, browsers, volume settings, operating systems, and expectations are another.

I was standing at my laptop, looking at a grid of rectangles on the screen, slowly bringing the instruments in. Each rectangle was one person from the audience. Each phone had been randomly assigned a part: bass, piano, crackling noise, angel pads, drone, or bells.

Then the room started to sound like one strange, fragile ambient organism, and I got goosebumps.

Why I made TERAZ

I have been orbiting around ambient music for years - as a DJ, artist, listener, and organizer of Up To Date Festival in Białystok. It is close to my heart. Ambient has always made sense to me because it can be both background and the whole world, depending on how much attention you give it.

At the same time, I spend a lot of my life in technology. I write code, build systems, and think about how things communicate with each other. For a long time, I wanted to make something where those two parts of me would actually meet, not as a gimmick, but as one idea. TERAZ came from that place.

Let’s talk a bit about the name. TERAZ means NOW in Polish. For me, that word became the whole mechanism of the piece. Participants can say: NOW I am part of the orchestra, NOW I am listening to what is happening around me, NOW I might want to check my phone, take a photo, record something - but if I do, I fall out of the orchestra.

The phone as part of the performance

I have been thinking for a long time about how we use phones during concerts and performances. Very often, instead of being fully in the room, we “document” the thing. We record, take photos, check notifications, look at something else for a second. A phone can pull us out of the exact thing we came to experience.

With TERAZ, I wanted to turn that around - the phone stays in your hand, but its role changes. To remain in the piece, you have to keep touching the screen so it does not go to sleep. It is a very simple gesture, almost stupidly simple, but it works. You are holding the device that usually distracts you, and now that same device requires your attention in order to keep playing.

That is why TERAZ has always felt a bit like a small manifesto to me. No slogans, no big statement from the stage - just a small rule: if you want to stay in the piece, stay with the phone. The technical challenge and the artistic idea grew together. The system exists to make that gesture possible.

What I learned from Pogwar

A few years earlier, I made a generative ambient music performance called Pogwar, which I presented at Arsenał Gallery in Białystok. It was based on Tone.js and gave me a lot of space to explore browser-based generative sound.

Some things from Pogwar came back later in TERAZ. One of them was a hack I discovered while working on that project: an audio element playing a silent MP3 file on an infinite loop, which helps JavaScript keep running in the background on mobile devices.

I would not broadly recommend this trick. It does not run at full power and can introduce artifacts. But it taught me something about what mobile browsers allow, what they block, and where the weird edges are.

How TERAZ works

From the audience side, it goes like this: at the beginning, I welcome everyone and explain the rules. During the premiere at wroclove.rb, I could have done this better. It was the first public version, and I learned very quickly that people need to understand one important thing: their phone may not start playing immediately. The orchestra is built gradually.

After the intro, people scan a QR code shown on the main screen. The system randomly assigns each person one of six tracks:

bass
piano
crackling noise
angel pads
drone
bells

When I press start, the QR code screen changes into a visualization: a matrix of rectangles, each representing one participant. Every phone becomes one small block inside the larger organism.

My control panel is somewhere between a mixer, a conductor’s desk, and a small emergency room.

I can see the six tracks, each with a volume slider and controls for locking or unlocking the instrument. Below that, I can see the participants, their assigned instruments, and their operating systems. I can also manually change assignments, which turned out to be very useful - for example, Android and bass were not exactly best friends.

On some Android devices, there was basically no sound below around 120 Hz. iPhones generally handled the sound much better and played more reliably. Because every instrument is technically available on every device, I could spot when an Android phone had been assigned bass and move it to another instrument.

Building for many different phones

The system is built with Ruby on Rails, Action Cable, Stimulus, and Hotwire. Sound generation happens in JavaScript with Tone.js.

I did not stream audio to the phones. That felt too heavy and too fragile. Instead, each phone can play any of the instruments locally. The server only sends small control messages through Action Cable: this track is active, this is the current volume, something has changed.

The piano is the one exception in the sound design. It uses a sample downloaded to the user’s phone, because generating a convincing piano directly on all those devices would be too computationally expensive. Everything else is generated on the participants’ phones.

I was ready for more than 120 people. At home, I tested with phones borrowed from my wife and neighbors, creating a tiny domestic orchestra. I also tested with around 100 browser tabs in Chrome. This was funny, because my Mac gave up before the server did. I kept opening tabs, but only some of them were actually able to play sound. Others started hanging or failing because I was clogging up the local audio context. So I added special test parameters that skipped audio initialization and tested only the Action Cable communication.

Designing sound for phone speakers

The music did not start in code. At first, I used Ableton Move to search for sounds. I improvised a lot, trying to find textures that made sense for a room full of small speakers. When something felt right, I started translating it into code and rebuilding the idea as a Tone.js instrument.

The six tracks have different roles: bass, piano, crackling noise, angel pads, drone, and bells. Together they create a slow ambient environment, but each participant only carries one part of it.

But not everything is set in stone. I wanted to create controlled chaos. Each instrument moves somewhat randomly within a defined range. I did not want all phones to play perfectly evenly or identically. That would probably feel too clean. The small differences between devices, timing, and generated material give the whole thing a bit of life.

At first, I thought I should make everything very precisely synchronized. Later, I let go of that idea. This is ambient music. Small timing differences are fine. They actually help. I did not measure latency in detail, because once I heard it in the room, it made sense.

Tone.js is a great library, but you need to be careful with optimization, especially on weaker phones. Some ideas that worked nicely on my machine were too heavy or unreliable across devices. I had to simplify, remove, or limit some parts to keep compatibility as broad as possible.

This was probably the biggest technical lesson for me: designing a real-time system for hundreds of phones is not the same as designing for a regular browser.

What I would change next time

Before doing it again, I need to fix one very basic thing: the intro.

People should know from the start that the piece grows slowly, and that waiting for your phone to enter is part of the experience. Otherwise, some participants expect everything to play immediately, because that is how most digital interfaces behave.

I would love to show TERAZ in a more ambient or art-oriented context. I also want to try it at a larger scale. With this project, more phones really make a big difference.

It could probably become an installation one day, but for now I am most interested in performing it live. The live version has tension. I can hear the room, see the system, feel the audience, and make decisions in the moment.

From a technical perspective, I’d love to add Rails Event Store and create a performance timeline using the events, and also make a nice timeline visualization for RES itself.

Where music and code meet

I think TERAZ worked for me because I did not have to choose which hat to wear.

The ambient-artistic part of me could care about slowness, texture, atmosphere, and attention. The programmer part could obsess over Action Cable, Android audio, sending as little data as possible, and keeping the thing alive on the cheapest Hetzner box.

Somehow, for once, these were part of a single thing.

We are all juniors again

2026-08-26T14:40:52Z

We are all juniors again

A long, long time ago, when Akinator was the peak of “advanced AI” on the internet, before services became micro, and deployments meant sending a file directly from a local computer via FTP to a server, I was introduced to the Dreyfus Skill Model. As a junior dev back then, I liked it because it explained why my senior and architect colleagues could sometimes make impactful decisions intuitively, while I, on the contrary, was expecting step-by-step instructions on how to convert HTML form input into a database row.

In short, the Dreyfus Skill Model describes that when learning a new skill, a learner goes through 5 stages to acquire it:

Novice. We need/expect step-by-step instructions and rules here, and we have problems in situations that don’t follow them.
Advanced beginner. We start to recognize some situation-specific nuances and use our experience instead of just rules.
Competent. We consciously define goals and priorities and plan our actions according to the situation.
Proficient. We intuitively recognize what the situation requires but consciously decide how to respond.
Expert. We intuitively understand the situation and respond without consciously considering each step.

I’m pretty sure that depending on how long you’ve been in IT, you may find yourself in one of those stages.

And now, getting back to AI and especially coding agents and support for the whole SDLC, it’s another skill, whether we want it or not, that we at least should know how to deal with. The good news is that whether you are a 20-, 30-, or 40-year veteran in the industry, or just joining, we are pretty much on the same level. This allows us not to know things, to experiment, to ask questions, to copy what works, and to change our views and opinions every week. This is what novices/beginners do, and I definitely feel I’m in those phases again. And I’m pretty sure that’s not only me.

Surprisingly often, when digging deeper into those “high-level AI concepts”, conversations end up on something like “well, I have a skill/MD”, or “I have a nice prompt that does the job”. And that’s totally fine, as we are all learning how to use those tools and what works in the whole SDLC. What I’m trying to highlight is that instead of hearing about “10x productivity increase”, “building another harness”, “running a swarm of agents”, or “AI changing someone’s life”, I would actually want to hear how it’s done. With details. And those details may be a single line of text in CLAUDE.md or a short explanation of this “obviously known feature”, since people expect exactly this kind of detail, because everyone feels behind.

We are in a place where, just a year ago, the Stack Overflow Developer Survey 2025 found that 76% of developers weren’t planning to use AI for deployment and monitoring, and 69% weren’t planning to use it for project planning. If that’s where we were as an industry a year ago, not many seniors around now 😉

As IDEs improved over the years, they sped up our coding by providing autocompletion, code navigation, etc. With AI coding agents, it’s different — quite a lot of people say that our time will shift from coding to other activities. And ironically, I bet you’ve already seen projects that have done less to simplify monitoring and releases for their devs than they’re now doing for their machines.

Building a harness? Show me if it’s just an instruction to your agent “ask user for review”, Claude Code hooks, or maybe good old pre-commit hooks that you are sure will execute 😉

Running your agent army? Let me know if those are your old laptops or VPSs and whether you tmux to them one by one, delegating some tasks, or maybe use Claude’s Agent Teams or AMP.

Fewer slogans, more specifics.

This is especially important now, when a lot changes literally every week, meaning there are new areas to discover and new ways to work with those tools. Their non-deterministic nature adds some complexity too. There are no best practices, so whatever works for you today is worth sharing. You don’t have to worry it will be outdated next month. You don’t have to worry about being wrong. On the other hand, it’s not out of the question that what you are doing now will stand the test of time and become a best practice sometime 🙂

We are all juniors, and let’s have some fun with it!

That said, and as this whole post was about sharing knowledge, here you have some links from us:

Last but not least, if you prefer a condensed, one-to-one experience with Arkencers where we focus on your project and on making your team AI-native in the way we see it, you may be interested in this workshop.

See you next time, and remember: together we will all become seniors sometime 🙂

Maintaining an organizational knowledge graph with an LLM and event sourcing

2026-08-11T10:00:00Z

Maintaining an organizational knowledge graph with an LLM and event sourcing

Organizations are surprisingly good at forgetting.
Decisions are made on calls, insights get buried in Slack threads, and a month later no one remembers why things are the way they are.

Arkency is no exception.
Weekly calls, ad-hoc meetings, our book clubs, Slack discussions, GitHub mentions, email inbox - we could use some support in organizing all those signals.

Then Ruby Community Conference 2026 happened in March.
In Kraków, Obie Fernandez showed some parts of his NEXUS system.
He had already described it on his blog back in January, but the conference was where I first came across it.
That was the push I needed to start building our own software.

When it was already taking shape, Andrej Karpathy published his LLM Wiki note.
Instead of a RAG system rediscovering your documents on every query, an LLM incrementally maintains a persistent wiki: interlinked markdown pages, immutable sources underneath, and a human curating the loop.
It was quite exciting to realize I was working on something that had just become one of the hottest topics in the industry.

We ended up with Planet Arkency - a multi-tenant knowledge graph with a closed ontology, built on Rails Event Store.
In this post, I want to walk you through the design decisions I made.

Unstructured input is where LLMs actually shine

For structured data, you could have built such a system like twenty years ago.
Webhooks, forms, integrations - parsing structured input into a graph is a solved problem.
But the most interesting knowledge lives in the input no parser could ever handle: meeting transcripts, Slack discussions, emails, or anything coming from an integration nobody has built yet.
This is where LLMs changed the game for us.

Everything flows into the system through a single ingestion endpoint.
Transcripts, Slack threads someone flagged with a dedicated emoji reaction, emails arriving at a bridge inbox, RSS feeds, calendar invites, personal notes.
We don’t even write code for the integration points.
Tools like Zapier or n8n watch the sources and push the content to that single endpoint.

Every ingested piece of content then goes through an extraction - the heart of the system.
An LLM reads the content and works out what it means for our knowledge: which entities appear in it, what we learned about them, and how they relate to each other.
Most of this post is about what happens around that single step.

Why a graph?

The same names keep coming back in our conversations: people, projects, clients, tools, decisions.
What changes from week to week is what we know about them and how they relate to each other.
That maps naturally to a graph: entities with attributes, connected by typed relations.
Who works on what.
Who made which decision, and when.
Which project depends on which tool.

This is where we differ most from the LLM Wiki approach.
In a wiki, the fact that someone works on some project is written down in a sentence on a page, at best with a link between the two pages.
The knowledge is there, but only a reader can make use of it.
In a typed graph, person --works_on--> project is a piece of data: you can query it, traverse it, count it.

The graph itself sits on PostgreSQL: a nodes table, an edges table with a unique (source, target, relation) triple, jsonb attributes on both - no rocket science here.
Dedicated graph databases (Neo4j, triple stores like the one NEXUS uses) could be a better fit for some specific workloads, like deep multi-hop traversal.
But nothing above is Postgres-specific.
A schema this plain is what makes the storage an actual detail - there is little to port when you write another adapter for the data layer.

The ontology

Which kinds of nodes and relations may exist is defined in an ontology, stored in a plain YAML file:

# from config/ontology.yml
node_kinds:
  - kind: person
    description: "team member, candidate, client contact, external person"
  - kind: decision
    description: "formal decision requiring group verdict — for casual suggestions use idea"
edge_relations:
  - relation: works_on
    signature: "person --works_on--> project"

The ontology is closed - if a kind or relation is not on the list, the model cannot use it.
Initially I was thinking about an open ontology, where the LLM could introduce its own types.
It brought complete chaos into the graph surprisingly fast.
In my opinion, it is better to tell the model upfront what to look for.

Not one graph, but many

“The organizational knowledge graph” suggests one universal graph for all different purposes.
We don’t believe in that, and DDD practitioners will recognize why.

We use a multi-tenant architecture to maintain separate graphs with their own ontologies, which really means their own ubiquitous languages.
Our internal Arkency graph speaks in people, projects and decisions - a domain quite close to a CRM.
The graph we run as Rails Event Store maintainers speaks in releases, known problems and community content:

Different domains, different vocabularies, the same machinery underneath.
The boundaries of a bounded context tell you where one graph ends and another begins.

What comes out of an extraction

The ontology is rendered into the extraction prompt as markdown tables and into the schema as enums.

(from app/lib/prompts/extraction.md.erb)

You are an organizational knowledge analyst for <%= Tenancy.current_tenant.name %>. We are building an internal knowledge graph.

Extract a knowledge graph from the provided content: nodes and edges. The graph should allow full reconstruction of the provided content.

## Nodes

Each node has: name, kind, short_description, description, attrs (optional key-value pairs).

Allowed kinds:

| kind | what it represents | typical attrs |
|---|---|---|
<% ontology.node_kinds.each do |k| -%>
| <%= k.fetch("kind") %> | <%= k.fetch("description") %> | <%= k.fetch("attrs", []).then { |attrs| attrs.empty? ? "—" : attrs.map { |a| a.is_a?(Hash) ? (a["values"] ? "#{a["name"]} (#{a["values"].join(", ")})" : a["name"]) : a }.join(", ") } %> |
<% end -%>

## Edges

Each edge has: source, target, relation, context, attrs (optional key-value pairs).

Allowed relations:

| relation | source kind | target kind | hint | attrs |
|---|---|---|---|---|
<% ontology.edge_relations.each do |r| -%>
<%
  sig = Ontology.parse_signature(r.fetch("signature"))
  source_kind = sig[:source].join(" / ")
  target_kind = sig[:target].join(" / ")
  hint = r["hint"] || "—"
  attrs = r.fetch("attrs", []).then { |a| a.empty? ? "—" : a.map { |at| at.is_a?(Hash) ? (at["values"] ? "#{at["name"]} (#{at["values"].join(", ")})" : at["name"]) : at }.join(", ") }
-%>
| <%= r.fetch("relation") %> | <%= source_kind %> | <%= target_kind %> | <%= hint %> | <%= attrs %> |
<% end -%>

...

Each extraction ends with the model returning one structured result: the entities it found in the content, the relations between them, and how the existing graph should change to reflect them.
We use RubyLLM’s schema support for that.

# from app/lib/extraction_result_schema.rb
array :nodes, description: "Entities to create or update. Each name must be unique — no duplicate nodes." do
  object do
    string :status, enum: ["new", "existing"], description: "'existing' iff the node was returned by search_nodes/list_nodes_by_kind/get_node_edges and you are reusing it. 'new' if you are introducing it. The system verifies the canonical name and aborts on mismatch."
    string :name, description: "Entity name. For 'existing' nodes use the EXACT canonical name from the tool call result. For 'new' nodes the canonical name you are introducing."
    string :new_name, required: false, description: "Optional. Set ONLY for 'existing' nodes when the content reveals a more explicit canonical form (e.g. acronym → full term, diminutive → full name). The node is looked up by `name` and renamed to `new_name`."
    string :kind, description: "Must be one of: #{kind_names}"
    string :short_description, description: "Stable synthesis of what this entity is (for search). General and identity-focused, not episode-specific. Max 15 words."
    string :description, description: "For new nodes: brief description based on the content. For existing nodes: synthesize prior description with new information. Rewriting for clarity is fine, but preserve prior facts."
    array :attrs, description: "Key-value attributes. Only include what is known from the content." do ... end
    array :aliases, required: false, description: "Optional. Alternative surface forms (diminutives, acronyms, full vs short forms) under which this entity was referred to in the content, or — when renaming via `new_name` — the old canonical if it remains a valid surface form. Only include NEW aliases not already present on the existing node. An alias is the SAME entity under another name — never a separate entity." do ... end
  end
end

array :edges, description: "ALL relationships. Be thorough and precise." do
  object do
    string :source, description: "Source node name (exact match — existing or newly created)"
    string :target, description: "Target node name (exact match — existing or newly created)"
    string :relation, description: "Must be one of: #{relation_names}"
    string :context, description: "Briefly explain why this relationship exists, grounded in the content"
    array :attrs, description: "Key-value attributes for this edge (e.g. since, weight)" do ... end
  end
end

Actual data operations (create or update, with the exact field-level diff) are derived server-side.
We load or initialize an ActiveRecord model, assign what the LLM returned, and let dirty tracking do the rest:

# from app/handlers/propose_graph_change.rb
node = Node.find_or_initialize_by(name: data[:name])
enforce_status!(data[:name], data[:status], node) # raises when the model's new/existing claim disagrees with the DB

was_new = node.new_record?
node.assign_attributes(short_description: ..., description: ..., attrs: node.attrs.merge(attrs))
changes = node.changes.except("updated_at", "created_at", "kind", "slug")

{ op: was_new ? "create" : "update", node_id: node.persisted? ? node.id : nil, changes: changes }

node.changes gives us {field => [before, after]} pairs for free, and this before/after snapshot becomes the wire format of the graph change proposal.
Edges get exactly the same treatment - looked up by their (source, target, relation) triple and diffed with dirty tracking.

We also don’t blindly trust what the LLM claims.
It has to declare each node as new or existing, and a validator cross-checks it against the database.
On mismatch, the LLM gets natural-language feedback and another attempt on the same conversation.

Identity resolution is the hard part

I just wrote that the model has to declare each node as new or existing.
But how would it know?
Do we load the whole graph into LLM context?
No - this is where tool calls come in.

And it is harder than a simple lookup.
“Piotrek”, “Piotr Jurewicz” and whatever Zoom’s transcription makes out of my name are the same person.
If you create a node per surface form, your graph turns into garbage within a week.

We handle it on three levels.

First, the model must look before it writes.
During extraction it has access to read-only tools like search_nodes or get_node_edges.
The extraction prompt is explicit about it:

(from app/lib/prompts/extraction.md.erb)
- Before creating any node, use search_nodes to check if it already exists. (...)
- If search_nodes returns no results, the node does not exist yet — proceed to create it. (...)
- If search_nodes returns ambiguous results, or you need broader context to make extraction decisions, use get_node_edges to inspect the node's connections.
- After finding nodes with search_nodes, use get_node_edges to see their existing relationships before deciding how to connect them.

Second, aliases are the identity mechanism.
Each node has one canonical name and any number of aliases.
The schema instructs the model that an alias is the same entity under another name - never a separate entity.
When the content reveals a better canonical form, the model sets new_name and the old name stays as an alias, so future fuzzy searches still resolve it.

Third, the search is hybrid.
Trigram similarity (pg_trgm with GIN indexes) over node names and aliases catches misspellings.
Embedding search catches semantic matches which share no characters:

# from app/models/node.rb
def self.hybrid_search(query, limit: 10)
  # fuzzy match on canonical names and aliases, powered by pg_trgm
  by_name  = where("similarity(nodes.name, ?) > 0.3", query)
  by_alias = joins(:aliases).where("similarity(node_aliases.name, ?) > 0.3", query)
  trigram_results = union_by_best_similarity(by_name, by_alias)

  response = RubyLLM.embed(query, model: "bge-m3", provider: :ollama)
  semantic_results =
    nearest_neighbors(:embedding, response.vectors, distance: "cosine")
      .select { |n| n.neighbor_distance < SEMANTIC_THRESHOLD }

  merge_and_rank(trigram_results, semantic_results, limit)
end

The embeddings come from a self-hosted bge-m3 model on Ollama, stored in pgvector.

Every fact has a source

A graph edited by an AI is only trustworthy if you can audit every change.
For every node and edge we can answer: which extraction created you, which extractions updated you, and what exactly changed each time.

Provenance lives in join tables (node_extractions and edge_extractions): one row per extraction and entity pair, holding the operation, the status, and the field-level diff produced by the dirty tracking described before.
Starting from any node, you can walk back through these rows to the extraction that touched it, and from the extraction to the ingested content it was based on.
Every fact in the graph traces back to its source.

We also record something we call the read set.
Every tool call the model makes during extraction is published as an ExtractionToolCalled event and projected into tool_invocations, linked to the nodes and edges the call returned.
So we know not only what an extraction wrote, but also what it read before deciding.
When you wonder “why did the model merge these two people?”, the answer is on the extraction page: here is the search it ran, and here is what came back.

Each node’s page shows its full history: created in, last updated in, read by N extractions.

Keeping an eye on the costs

Besides auditing changes in the graph, we also track how much each extraction costs: token usage and the resulting price.
When you work with an LLM API, it is worth keeping a finger on the pulse here.
A transcript of a few hours of conversation, processed in multiple rounds interleaved with tool calls, can generate significant costs.
Prompt caching helps a lot - the system prompt and the content stay identical between rounds, so most of the input is billed at the cache-read rate.

The exact numbers depend on the model you run the extraction on, but most of ours cost well under a dollar.

Human in the loop

We don’t let the LLM write to the graph directly.
Extraction produces a proposal with the before/after diffs, and applying it to the graph is a separate step.

Proposals can sit in a review window before they get applied.
As soon as an extraction completes, we get a short summary of it on Slack.
A human can inspect the diff, apply it early, or just let it flow after the configured delay.

Time passes between propose and apply, so the graph may have moved in the meantime.
When the current state no longer matches what the proposal was based on, the apply stops and the affected rows get marked as conflicted, with a human-readable explanation.

Event sourcing ties it all together

You may have noticed that every mechanism above was described in terms of events.
Well, this is an Arkency blog after all.

The whole pipeline is an event flow: TranscriptIngested → ExtractionRequested → KnowledgeExtracted → GraphChangeProposed → GraphChangeApplied (or GraphChangeConflicted).
Two small aggregates guard the invariants: one per ingestion (no two concurrent extractions of the same content), one per extraction (the propose → apply state machine).
Everything you see in the UI (ingestions, extractions, diffs, tool invocations) is a read model built from these events.

In this architecture, the review window is just one more state in the aggregate’s state machine, and provenance is just one more read model built from an event we already had.
I cannot understand people claiming that event sourcing makes things more complex ;)

The graph can feed itself

One feature shows the value of a uniform pipeline well.
From any node you can request research.
A job asks a model equipped with Anthropic’s server-side web_search and web_fetch tools to compile a brief about the entity:

# from app/jobs/research_topic.rb
chat = RubyLLM
  .chat(model: MODEL)
  .with_params(tools: [
    { type: "web_search_20250305", name: "web_search", max_uses: 10 },
    { type: "web_fetch_20250910", name: "web_fetch", max_uses: 10 }
  ])
  .with_schema(ResearchBriefSchema.build)

The prompt grounds the research in what the graph already knows about the entity, and tells the model when to give up:

# from app/jobs/research_topic.rb
Research "#{topic}". Use web_search and web_fetch as needed to gather facts.

In our knowledge base this entity is currently described as:
- Kind: ...
- Short description: ...
- Attributes: ...

When you can produce a useful brief, return status="completed" and put the
brief in `brief` as Markdown. (...) Cover identity, key facts a knowledgeable
reader should know, recent activity worth recording, and relationships to
other named entities. Include source URLs inline next to claims that come
from a specific page. Keep it factual; do not speculate.

Return status="aborted" instead — with `abort_reason` naming the specific
problem — when any of these holds:
- The topic is ambiguous and you cannot confidently pick the intended
  interpretation from the disambiguation context above.
- You cannot find substantive, verifiable information about this exact
  entity (...)

Do not pad an aborted result with related-but-different information.

The resulting brief is not applied to the graph directly.
It gets published as a regular TranscriptIngested event with its own kind, and flows through the same extraction, proposal and review pipeline as any other input.

The graph speaks MCP

The graph is not locked inside its own UI.
We expose it over MCP, so any AI assistant with access to our server can search it by asking questions in natural language - and answer from the graph, with sources.

Final thoughts

Working on Planet Arkency taught me a lot.
About graphs, about LLMs, and about concepts I had never even heard of before: ontologies, identity resolution, provenance.
I hope some of that knowledge stays with you after reading this post.

It also reassured me about the tools we have been using at Arkency for years.
Event-driven architecture and Rails Event Store carried this project naturally.

At the time of writing, the main production Arkency graph holds almost 2000 nodes and over 5200 edges, built by around 300 extractions.
And behind all of that, over 3600 events recording how every single fact got there.

I still have a head full of ideas on where to take this project next.

Working with RubyLLM was a pure pleasure - credits to Carmine Paolino for this gem.

If you are thinking about organizational memory for your company, or want us to help you build one, get in touch.

3 example process managers in Rails

2026-07-16T09:03:31Z

3 example process managers in Rails

I am writing this blogpost from this beautiful place in the Polish mountains. Our RailsEventStore camp takes place this week.

Process managers are meant to map business process into code. The more readable and declarative it is, the better the chance domain experts will understand it.

Over the last years, we have experimented with several approaches on how to implement process managers in Ruby.

The foundation here is RailsEventStore and the architecture where events are published and commands are used to tell what to do next.

Most business processes can be mapped into some sort of a checklist of requirements and when certain conditions are met we decide (a command) what to do next.

During the RESCamp I have released a small library - ruby_event_store-process_manager - which was extracted from the ecommerce project.

1. ReleasePaymentOnOrderExpiration

I will start with a process which defines what to do when a payment was authorized but the offer has expired. You can imagine that different businesses have different rules for such situations. BTW, that’s why I like to put process managers at the application layer and keep them under app/processes.

module Processes
  class ReleasePaymentOnOrderExpiration
    include RubyEventStore::ProcessManager.with_state { ProcessState }

    subscribes_to(
      Payments::PaymentAuthorized,
      Payments::PaymentReleased,
      Fulfillment::OrderRegistered,
      Pricing::OfferExpired,
      Fulfillment::OrderConfirmed
    )

    private

    def act
      release_payment if state.release?
    end

    def apply(event)
      case event
      when Payments::PaymentAuthorized
        state.with(payment_authorized: true)
      when Payments::PaymentReleased
        state.with(payment_authorized: false)
      when Pricing::OfferExpired
        state.with(order_expired: true)
      else
        state
      end
    end

    def release_payment
      command_bus.call(Payments::ReleasePayment.new(order_id: id))
    end

    def fetch_id(event)
      event.data.fetch(:order_id)
    end

    ProcessState = Data.define(:payment_authorized, :order_expired) do
      def initialize(payment_authorized: false, order_expired: false) = super

      def release?
        payment_authorized && order_expired
      end
    end
  end
end

Thanks to the ProcessManager certain things are taken care of for free. One of them is the concept of streams and event sourcing. All events which are relevant to this process are grouped into one stream. This stream by convention will be called ReleasePaymentOnOrderExpiration$uuid but it’s easy to override. All the events will be linked to this stream automatically. Thanks to this, the process will be rebuilt by replaying the stream on every new event. Additionally, if two events for the same process race and hit a version conflict, the state-building is retried once.

That’s what is hidden.

What we have here is typical sections:

events
state building
decision (act)

2. Reservation

Reservation is a common problem to solve in order-related apps. However, every business defines their reservation process differently. Imagine a business which sells equipment around a specific topic, let’s say photography. You want to buy a camera, a lens and a tripod. All of this is important and if any of them is not available the whole order doesn’t make sense - a camera the customer can’t yet use isn’t worth shipping

This business process is implemented here:

module Processes
  class ReservationProcess
    include RubyEventStore::ProcessManager.with_state { ProcessState }

    subscribes_to(
      Pricing::OfferAccepted,
      Fulfillment::OrderCancelled,
      Fulfillment::OrderConfirmed
    )

    private

    def act
      case state
      in order: :accepted
        unavailable = reserve_stock
        if unavailable.any?
          reject_order(unavailable)
        else
          accept_order
        end
      in order: :cancelled
        release_stock(state.reserved_product_ids)
      in order: :confirmed
        dispatch_stock
      end
    end

    def apply(event)
      case event
      when Pricing::OfferAccepted
        order_lines_hash = event.data.fetch(:order_lines).map { |ol| [ol.fetch(:product_id), ol.fetch(:quantity)] }.to_h
        state.with(
          order: :accepted,
          order_lines: order_lines_hash
        )
      when Fulfillment::OrderCancelled
        state.with(order: :cancelled)
      when Fulfillment::OrderConfirmed
        state.with(order: :confirmed)
      end
    end

    def reserve_stock
      unavailable_products = []
      reserved_products = []
      state.order_lines.each do |product_id, quantity|
        command_bus.(Inventory::Reserve.new(product_id: product_id, quantity: quantity))
        reserved_products << product_id
      rescue Inventory::InventoryEntry::InventoryNotAvailable
        unavailable_products << product_id
      end

      if unavailable_products.any?
        release_stock(reserved_products)
      end
      unavailable_products
    end

    def release_stock(product_ids)
      state.order_lines.slice(*product_ids).each do |product_id, quantity|
        command_bus.(Inventory::Release.new(product_id: product_id, quantity: quantity))
      end
    end

    def dispatch_stock
      state.order_lines.each do |product_id, quantity|
        command_bus.(Inventory::Dispatch.new(product_id: product_id, quantity: quantity))
      end
    end

    def accept_order
      command_bus.(Fulfillment::RegisterOrder.new(order_id: id))
    end

    def reject_order(unavailable_product_ids)
      command_bus.(Pricing::RejectOffer.new(
        order_id: id, reason: "Some products were unavailable", unavailable_product_ids:)
      )
    end

    def fetch_id(event)
      event.data.fetch(:order_id)
    end

    ProcessState = Data.define(:order, :order_lines) do
      def initialize(order: nil, order_lines: [])
        super(order:, order_lines: order_lines.freeze)
      end

      def reserved_product_ids = order_lines.keys
    end

  end
end

What is worth noting here is that this business process can result in 5 different commands for 3 different contexts:

Pricing::RejectOffer
Fulfillment::RegisterOrder
Inventory::Dispatch
Inventory::Reserve
Inventory::Release

This is very typical case in complex businesses. A context can be seen as a department in a company - that’s one of the DDD metaphors which I enjoy using.

3. Publishing a post on Twitter-like app

This process comes from a different app. Imagine a social media app, where people see their timeline consisting of posts from the people they follow. The process here is simple - make a delivery (posting to a timeline) per each follower, but also include the author timeline. In the future, we can imagine other rules. For example, we may want to decide to deliver to the online people first. Or maybe we want to push some ads to the timeline when a post seems related. It’s nice to have such business rules in one place.

class TimelineDeliveryProcess
  include RubyEventStore::ProcessManager.with_state { ProcessState }

  subscribes_to(
    Social::UserFollowed,
    Social::UserUnfollowed,
    Social::PostPublished
  )

  private

  def act
    recipients.each { |recipient_id| deliver_post_to(recipient_id) } if state.post
  end

  def recipients
    state.followers + [state.post.data.fetch(:author_id)]
  end

  def apply(event)
    case event
    when Social::UserFollowed
      state.with(followers: state.followers | [event.data.fetch(:follower_id)], post: nil)
    when Social::UserUnfollowed
      state.with(followers: state.followers - [event.data.fetch(:follower_id)], post: nil)
    when Social::PostPublished
      state.with(post: event)
    end
  end

  def fetch_id(event)
    case event
    when Social::UserFollowed, Social::UserUnfollowed
      event.data.fetch(:followee_id)
    when Social::PostPublished
      event.data.fetch(:author_id)
    end
  end

  def deliver_post_to(recipient_id)
    command_bus.call(
      Social::DeliverPostToTimeline.new(
        post_id: state.post.data.fetch(:post_id),
        recipient_id: recipient_id,
        author: state.post.data.fetch(:author),
        body: state.post.data.fetch(:body)
      )
    )
  end

  ProcessState = Data.define(:followers, :post) do
    def initialize(followers: [], post: nil)
      super(followers: followers.freeze, post: post)
    end
  end
end

The ruby_event_store-process_manager gem

As you see, RailsEventStore together with the ruby_event_store-process_manager gem, allows to map complex business processes into modules, which encapsulates the logic, but hides the infrastructure.

In a typical non-event-driven Rails app such processes are usually scattered across models, service objects and callbacks. The alternative approach requires publishing events (which provides event log for free) but the benefits might be worth it.

Ingress is not the owner of the invariant

2026-06-16T13:21:30Z

Ingress is not the owner of the invariant

A polemic with Callbacks Are Not Invariants by Brandon Weaver.

A disclaimer: I’m a RailsEventStore maintainer and this article ends up on the Arkency blog — so cards are on the table. Despite this, I’m keeping the core of my argument in pure ActiveRecord: no step of the reasoning requires RES. I only show the RES version at the end, separately, as “and this is what it looks like when you’re not typing it in manually”. If you’re convinced by the bare-metal AR reasoning, not the library, that’s what matters.

We agree about the disease

I enjoy reading Brandon’s Rails: The Sharp Parts series and sending it to the team — it’s one of the better pieces on the sharp edges of Rails that’s come out lately. The one about callbacks is no exception and the diagnosis is spot on. Census _save_callbacks, which shows eleven entries with two association lines and zero callbacks of its own. A mismatch between what fires before and after COMMIT. The best sentence in the entire text: “a callback is an invariant with a published bypass list” — update_all, insert_all, update_column are holes in something that was supposed to always happen.

I’m not here to defend the callbacks. I hate them myself, for the same reasons. The dispute is about the cure.

And I have one reservation for the cure Brandon proposes — and it doesn’t concern what he built, but how he named it. Because the name will travel beyond a single file, to anyone who copies the template.

A name that promises more than it delivers

What the author calls Command is Fowler’s Transaction Script. One public call, a private execute, a procedure orchestrating ActiveRecord calls. And that’s a good pattern — it’s forty years old and still going strong. The thing is, he calls it a command (suggesting CQRS, which itself announces next time) and calls ActiveSupport::Notifications events.

A name isn’t cosmetic. A name is a mental contract. When 500 engineers work in a monolith — and that’s the scale Weaver explicitly writes for — names are the only documentation anyone reads. If you tell them they’re building commands and events, in a year, half the company will think they’re building an event-driven architecture with procedures in their hands. That’s worse than no name, because it installs a mental model that doesn’t match the code.

And this isn’t about arguing with its values but about appealing to them. Brandon writes about himself that his goal is to make the invisible visible — and that the next person reading the code shouldn’t have to wonder what the author meant. I agree with that with both hands. And that’s precisely why calling a procedure a command and a notification an event plays against what he wants: it forces the reader to assume a contract — a serializable intent with a separate handler, a persistent domain fact — that doesn’t exist in the code. Correcting the names isn’t a quarrel with Weaver; it’s the completion of his own goal.

Core: ingress doesn’t own the invariant

Single-ingress is correct — one write path per operation, one entry point that owns the state change. But look where the “space cannot be reserved twice” invariant lives after the refactor:

def reserve
  seat = Seat.find(seat_id)
  seat.with_lock do
    raise AlreadyReserved, "seat #{seat_id} is already reserved" if seat.reserved?
    seat.update!(reserved: true, reserved_by: by)
    Ledger::RecordReservation.call(seat: seat, by: by)
  end
  seat
end

The rule is spread across three layers:

a check in runtime (if seat.reserved?)
a row lock (with_lock)
a constraint in the database — as Brandon rightly writes elsewhere.

This isn’t a domain model. It’s a Seat.find + update! wrapped in a procedure. Infrastructure-first, just repurposed from a callback to a service object. The question “who owns the reservation rule” still doesn’t have a single answer.

The coupling is now deliberate and visible — but it’s still there. One trade-off and one naming issue:

Side effects inline in execute. Weaver is explicit about this: announce belongs to the command body by design, not by accident — subscribers are reserved for observability only and cannot veto a write or introduce ordering dependencies. That’s a defensible trade-off: visible coupling beats hidden coupling every time. My claim is narrower: announce fires after with_lock commits, so the timing is fine — but if the process dies between commit and deliver_later, the effect is gone forever. There is nothing to replay from, because the fact was never persisted.

event_name from the namespace system. The event name extracted from module_parent_name ties the event taxonomy — that is, the contract — to the directory structure in the code. Move a module, and the names of events that someone might already be subscribed to change. This is exactly the kind of invisible coupling he’s been fighting against throughout this article — only this time it moves a layer higher.

What it looks like when an invariant has an owner

I’ll show the difference in code, because otherwise, it’s just adjectives. Pure ActiveRecord, without RailsEventStore.

First, the aggregate. It — and only it — decides whether the reservation is allowed and produces the fact. No IO, no mailer, no webhook:

class Seat < ApplicationRecord
  class AlreadyReserved < StandardError; end

  # Invariant lives here. In one place. The method always returns an event,
  # and doesn't fire side effects.
  def reserve(by:)
    raise AlreadyReserved, "seat #{id} is already reserved" if reserved?

    self.reserved    = true
    self.reserved_by = by
    self.reserved_at = Time.current

    SeatReserved.new(seat_id: id, reserved_by: by, reserved_at: reserved_at)
  end
end

An event is a fact. Past tense. Payload is a result, not a request:

SeatReserved = Data.define(:seat_id, :reserved_by, :reserved_at) do
  def event_name = "seat_reserved"
end

The handler is a simple PORO — one application use-case, one entrance for write. We’re keeping Brandon’s single-entry discipline because it’s good; we’re not inheriting his ApplicationCommand base — its only job was announce, and that role is now taken by the explicit SeatReserved with persistent log write and subscribers. There’s no constructor, no ivars, and no self.call to new.call relay — the handler is so thin that there’s nothing to decompose. And that’s the point: the rule has been moved to the aggregate, so orchestration remains trivial, and everything happens on the correct side of the commit.

module Seats
  class ReserveSeat
    def self.call(seat_id:, by:)
      Seat.transaction do
        seat  = Seat.lock.find(seat_id)
        event = seat.reserve(by:)   # aggregate guards the invariant 
        seat.save!
        Events.publish(event)       # fact in the same transaction
        event
      end
    end
  end
end

Events.publish does two things, both in the caller’s transaction: it writes the event to a persistent log and synchronously calls subscribers. First, the log: append-only, the source of truth, from which you replay:

# Persistent event log — source of truth. 
# As long as the event lives here, it can be replayed.

class StoredEvent < ApplicationRecord
  def self.append(event)
    create!(name: event.event_name, payload: event.to_h, occurred_at: Time.current)
  end
end

And the dispatcher. No magic at all: publish stores the fact, and then calls handlers subscribed to the event name — in the same thread, and in the same transaction:

# publish: persistent fact + sync subscribers, atomically with the state change.
module Events
  HANDLERS = Hash.new { |h, k| h[k] = [] }

  def self.publish(event)
    StoredEvent.append(event)
    HANDLERS[event.event_name].each { |handler| handler.call(event) }
  end

  def self.subscribe(event_name, &handler)
    HANDLERS[event_name] << handler
  end
end

Here’s all the persistence I need: StoredEvent.append runs in the same transaction as seat.save! — because publish doesn’t open its own transaction, the INSERT inherits the active one from the caller. The fact commits with the state change or not at all. Since the fact remains, any reaction can be recreated — handler retry, log replay. This is the invariant, and you have it without any additional machinery.

One fair boundary. The subscribers above execute synchronously, in a transaction — which is exactly what you want for reactions intended to be atomic with the fact. But a subscriber doing heavy or external IO (mail, webhook) shouldn’t block the transaction; it schedules the work asynchronously, via deliver_later. And here comes the only gap Weaver worries about: the very scheduling of this asynchronous work isn’t atomic by default with the fact’s save. This narrow gap — and nothing else — is patched by the transactional outbox. It doesn’t patch the persistence of the fact, because that’s already taken care of.

And the mailer and the webhook? They react to the fact. They’re not steps in the write path — they’re subscribers. You can add, remove, or replace one without touching the ReserveSeat:

Events.subscribe("seat_reserved") do |event|
  ReservationMailer.confirmed(event.seat_id).deliver_later
end

Events.subscribe("seat_reserved") do |event|
  Webhooks::Emit.call(event: :seat_reserved, payload: event.to_h)
end

Let’s compare both approaches:

	Weaver	Here
Invariant	runtime check + `with_lock` + constraint	`Seat#reserve`, single place
Event	`reserve_seat.seats` from namespace, payload = input	`seat_reserved`, explicit, payload = result
Side-effects	inline in `announce` (deliberate — observability subscribers can’t veto writes)	subscribers react to the persisted fact
Delivery	fire-and-forget after commit, can be lost	event in a transaction, always replayable

And here’s the point I care about most — because it affects the domain I’m currently working on: persisting the fact isn’t an add-on — it’s a day-one invariant. Weaver himself is clear: “If you need durable event delivery (guaranteed at-least-once), that’s a transactional outbox or CDC, not a subscriber.” So we agree on the destination. Where I part ways is the framing: he presents it as something you reach for at scale. When SeatReserved is saved in the same transaction as the state change, no effect can be lost forever: since the fact remains, the reaction can always be recreated — retry the handler, replay from the log. Outbox doesn’t create this persistence — it only uses it, automating the delivery with an at-least-once guarantee — which means subscribers must be idempotent. You don’t persist the fact because you scaled; you persist it from the first INSERT and harden delivery when the async scheduling gap becomes relevant.

And if you don’t want to write it by hand

The above is on bare AR intentionally, so the argument can stand up without any library. But you get the same structure off the shelf. Aggregate with AggregateRoot:

class Seat
  include AggregateRoot

  AlreadyReserved = Class.new(StandardError)

  def reserve(by:)
    raise AlreadyReserved if @reserved
    apply SeatReserved.new(data: { seat_id: @id, reserved_by: by })
  end

  on SeatReserved do |event|
    @reserved    = true
    @reserved_by = event.data.fetch(:reserved_by)
  end
end

An invariant in reserve, a fact in SeatReserved, a state mutation in on — and this is an event that actually lands in the event store, with versioning and replay, not a notification whose name is derived from a namespace. You attach handlers (mailer, webhook) as subscribers exactly as above.

And delivery? RES publish is exactly Events.publish: atomic write to the event store plus synchronous dispatch, inside the caller’s transaction. ruby_event_store-outbox patches the one gap I named above — transactional scheduling of async handlers — maintained, safe under concurrent workers, covered with mutation tests.

A complete walkthrough of RES — aggregates, subscriptions, outbox, strangler on an existing monolith — is a topic for a separate, much longer text. Let me preface this point, as Brandon rightly dislikes rewrites; this path isn’t rewriting. It’s strangler — precisely the incremental movement it describes, callback by callback, flag by flag. The punchline here is enough: the structure he’s approaching is available as a ready-made library, not an exotic one.

Where Weaver is right — and what I’m not saying

I’m not advocating “always event sourcing”. That would be precisely the dogmatic approach I combat in people who sell event sourcing as a religion.

His Transaction Script is sufficient for most applications. One team, one write path, reasonable discipline — and a procedure with a single input carries water for years. The Strangler fig + Flipper migrations he describes are really good. Normalizes for pure transformations — agree. Constraints as truth in the database — agree, and strongly so. His CommandSingleEntrant RuboCop cop makes the single-entry rule structural rather than disciplinary — and he’s honest about its reach: “Both catch the common mistakes”, with class << self patterns as a known blind spot.

My point isn’t “your pattern is too weak”. It’s: don’t call it a command and event when it isn’t. Because the difference between a procedure and a command, and between notification and a domain event, isn’t pedantry — it’s the difference between “an invariant has an owner” and “an invariant is smeared, but nicely named”.

I anticipate three counterarguments:

“It’s a dispute about names.” Yes — and names are a contract that will reach 500 people. A bad mental model scales worse than bad code, because code gets refactored, but beliefs don’t.

“ES is overkill.” Agreed, for most people. But a persistent record of a fact isn’t event sourcing — it’s a single INSERT in the same transaction as a state change, turning “effect can be lost” into “effect always recoverable”. You don’t skip this because you’re not doing event sourcing; it’s record path hygiene, not architecture.

Steelman 37signals — whom Weaver honestly quotes — says that disciplined callbacks scale further than they’re given credit for. In a single, cohesive team: they’re not wrong. But the same caveat applies to his solution: Transaction Script without an invariant owner also relies on discipline, which is generally absent with 500 people. The aggregate approach doesn’t ask for discipline — the structure enforces the rule.

Landing

Single-ingress is correct. But an ingress that doesn’t own the invariant isn’t a fix — it’s moving the same rule spread from a callback to a procedure. And calling that procedure a command and the notification an event installs a mental model for 500 engineers that doesn’t fit the code they have in their hands.

The strongest version of his own argument isn’t the one he wrote — it’s the one he’s getting closer to: an aggregate that monitors the invariant, a true domain event, and a fact persisted in the same transaction as the state change. Weaver ends with a CQRS announcement “next time,” and it’s a good announcement, because single-ingress without an invariant owner is only halfway down the road he’s charted.

I look forward to the continuation of the series — honestly, without irony.