feat(cli): Add an rg-equivalent CLI interface for fff by markovejnovic · Pull Request #561 · dmtrKovalenko/fff

markovejnovic · 2026-06-02T19:59:17Z

This PR adds a CLI interface to fff. After talking with @dmtrKovalenko, we decided that the best architecture is to have an fff daemon run in the background to keep the index hot. The command that the user interacts with is fff-rg, which talks to this daemon.

Before we proceed, it's worth looking at the architecture diagram here:

flowchart LR
  subgraph fff-rg
    rg-searcher
    fffd-searcher
  end

  subgraph fff-daemon
    query-service
    subgraph session-pool
      session-1
      session-2
    end

    query-service --> session-1
    query-service --> session-2
  end

  rg-bin[[rg]]

  fffd-searcher <-->|unix sock|query-service
  rg-searcher <-->|subproc| rg-bin

Let's walk over each component:

`fff-rg`

fff-rg can search either through talking through the fff-daemon, or through shelling to rg. It picks the fff-daemon if it believes to be working within a git repo. Otherwise, it will fall back to rg, and if the user doesn't have it installed, it will abort.

The core reasoning for this is that holding an fff-daemon alive for directories that don't need to be indexed makes no sense, as you'll hold a very large amount of RAM in memory for an effectively one-off search.

`fff-daemon`

The fff-daemon holds within it a query-service. This query-service is responsible for accepting connections from new clients and then passing the request down to the session-pool.

The session-pool is a pool of "active" sessions, ie. active git repositories for which we have an fff index in memory. A couple notable facts:

The session pool automatically reaps old sessions. If a search hasn't been performed for a while on a directory, chances are the user no longer cares for it, so we should evict it.
The session pool has an LRU policy which means that if we're out of available pool slots, we'll kick out some unused and old session in favor of creating a new one.

IPC protocol

I spent a good bit of time thinking through how best to handle the IPC protocol, and the epiphany I had is that we don't actually need to shuttle results between the daemon and the client. There may be hundreds, if not thousands of strings we'd need to copy, so if we can avoid it, that would be awesome.

On UNIX, you can pass file descriptors by using SCM_RIGHTS between processes. This enables us to take the stdout fd of the client, pass it to the daemon, and have the daemon directly write to the client's stdout. Neat!

The downside of this approach is that SCM_RIGHTS requires a unix domain socket, which means that passing the fd needs to happen over that. I evaluated:

Adding a sidecar iceoryx2 for all the other message passing, but ultimately I couldn't justify the maintenance burden. There's a ton of complexity added in figuring synchronizing the req-rep flow of the mmaped iceoryx2 channels and the req-rep of the fd transfer, so it made no sense.
Just having the req-rep flow between the client and the daemon live on the UDS (unix domain socket), which is ultimately what I landed on.

This lives in the crates/cli/fff-ipc-domain crate.

Holes in the implementation

Tests are quite lackluster. I added some testing, but it's quite weak.
There are a couple open questions I've left as comments on this PR.
The LRU logic could be separated out to a separate container which is responsible for automatic eviction.

markovejnovic · 2026-06-10T01:41:57Z

+/// Case sensitivity strategy for grep searches. Mirrors `fff::CaseMode` but
+/// with rkyv derives — fff-core doesn't depend on rkyv.


this is up for debate whether we want to keep it this way or not.

I wanted to avoid adding another dependency to fff that is only used in the ipc code-path, but with optimizers being what they are, maybe that's not so bad.

either way, i chose this path of having two CaseModes (one in fff and one in fff-ipc-domain), but open to discussion

we can have optional dependencies - this is absolutely fine

markovejnovic · 2026-06-10T02:13:11Z

+#[rkyv(derive(Debug))]
+pub struct SearchRequest {
+    /// Root directory to search in (must be an absolute path).
+    pub directory: String,


need to stop passing paths as strings. much better bet is to pass pathbufs as deep as possible until the fff barrier.

Bootstrap the CLI layer for FFF: - fff-ipc-domain: wire types and IPC protocol (Unix socket, bincode) - fff-daemon: background search daemon with session pooling, rg-compatible output formatting, and ANSI color matching - fff-rg: ripgrep-compatible CLI frontend with daemon/fallback searcher backends Includes 120 e2e tests: - 95 comparison tests (fff-rg vs rg side-by-side) across inline, heading, vimgrep, context, color, quiet, count, regex, unicode, and edge-case modes using test-case crate for parametrization - 25 synthetic repo scale tests (50/200/500 files) verifying match counts, line numbers, output formats, concurrency, and per-needle findability without rg comparison

dmtrKovalenko

some nits on the code I need to go through the way it actually works one more time

dmtrKovalenko · 2026-06-12T02:58:23Z

-        shared_picker.wait_for_scan(Duration::from_secs(120));
-        let file_count = {
-            let guard = shared_picker.read().expect("read lock");
-            guard.as_ref().expect("picker present").get_files().len()


this would require a major version bump

dmtrKovalenko · 2026-06-12T02:59:53Z

+//! status byte. Spawns the daemon on first use if it isn't already running.
+
+use std::io::{Read, Write};
+use std::os::unix::io::AsRawFd;


ideally we should get all the unix specific mode into a separate file cause I would love this to work on windows at some point

dmtrKovalenko · 2026-06-12T03:08:38Z

+    fn into_core(self) -> Self::Core;
+}
+
+impl IntoCoreExt for fff_ipc_domain::CaseMode {


use From trait

dmtrKovalenko · 2026-06-12T03:09:10Z

+fn main() {
+    let args = Args::parse();
+
+    tracing_subscriber::fmt()


use our log crate

dmtrKovalenko · 2026-06-12T03:10:15Z

+/// Default max file size for grep when the client doesn't specify one (4 MiB).
+const DEFAULT_MAX_FILE_SIZE: u64 = 4 * 1024 * 1024;
+
+use fff::{


do not intermix imports and constants

dmtrKovalenko · 2026-06-12T04:36:51Z

+)]
+/// Mirrors a subset of `rg` flags so `fff-rg` is a drop-in replacement.
+#[allow(clippy::struct_excessive_bools)]
+pub struct Args {


i think we should restructure the crates to be somethjing like this

cli/
daemon
ffd
frg
ipc

this iwll make it much easier to keep backward compatibility with those tools

dmtrKovalenko · 2026-06-12T04:37:49Z

+use crate::types::cli::Args;
+
+#[global_allocator]
+static GLOBAL: mimalloc::MiMalloc = mimalloc::MiMalloc;


i don't think we need mimaloc here - can drop a dependency

dmtrKovalenko · 2026-06-12T04:41:43Z

@@ -0,0 +1,147 @@
+use crate::util::Dir;
+
+const EXTENSIONS: &[&str] = &["rs", "ts", "json", "md", "txt", "toml", "yaml"];


for tests I would prefer tests in the big repo using proptest on various flags + queries

dmtrKovalenko · 2026-06-12T04:43:13Z

+bytesize = "2"
+mimalloc = { workspace = true }
+git2 = { workspace = true }
+which = "8.0.3"


im not sure we need it - if it's on a path Command will resolve it

dmtrKovalenko · 2026-06-12T04:43:25Z

@@ -0,0 +1,6 @@
+edition = "2024"


pls do not override

markovejnovic commented Jun 10, 2026

View reviewed changes

markovejnovic marked this pull request as ready for review June 10, 2026 03:21

markovejnovic force-pushed the feat/cli branch 2 times, most recently from 3a207ad to 9969bba Compare June 10, 2026 18:57

markovejnovic force-pushed the feat/cli branch from 9969bba to c27a5ff Compare June 10, 2026 21:39

dmtrKovalenko reviewed Jun 12, 2026

View reviewed changes

		/// Case sensitivity strategy for grep searches. Mirrors `fff::CaseMode` but
		/// with rkyv derives — fff-core doesn't depend on rkyv.

		@@ -0,0 +1,147 @@
		use crate::util::Dir;

		const EXTENSIONS: &[&str] = &["rs", "ts", "json", "md", "txt", "toml", "yaml"];

Conversation

markovejnovic commented Jun 2, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

fff-rg

fff-daemon

IPC protocol

Holes in the implementation

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

dmtrKovalenko left a comment

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

markovejnovic commented Jun 2, 2026 •

edited

Loading

`fff-rg`

`fff-daemon`