A Resilient, High-Performance Task Scheduler for Rust

tokio::spawn gives you fire-and-forget concurrency - no priority, no backpressure, no structured shutdown. CPU-bound work blocks the executor, starving latency-sensitive tasks. A task scheduler without priority awareness means a background file-index can stall a keystroke.

"Every CPU core stays busy. High-priority tasks always pre-empt background work. Shutdown drains gracefully, never drops a task in flight."

Rust API Documentation 📖

Echo is a bounded work-stealing task scheduler for Rust, designed as the core execution engine for application backends like Mountain. It provides structured concurrency with priority-based task scheduling across a Tokio thread pool, using lock-free crossbeam-deque queues to maximize CPU utilization without central bottlenecks.

tokio::spawn is great for I/O-bound work, but CPU-bound tasks - parsing, diffing, indexing - block the executor. Echo provides priority levels so latency-sensitive operations always take precedence, work-stealing so idle workers pick up tasks from busy ones, and structured shutdown so no task is lost when the system drains.

Echo is engineered to:

1. Prioritize Critical Work - High-priority tasks (UI responses, keystroke processing) always execute before Normal and Low-priority background work, ensuring consistent perceived performance.
2. Maximize CPU Utilization - Lock-free work-stealing via crossbeam-deque eliminates scheduling bottlenecks. Idle workers automatically steal from busy ones, keeping every core productive.
3. Provide Structured Concurrency - Unlike fire-and-forget tokio::spawn, the scheduler manages a supervised pool of workers with graceful startup and shutdown. The Drop guard ensures clean teardown even on panic.
4. Maintain a Decoupled Architecture - The generic Queue module provides core work-stealing logic as a standalone library. The Scheduler layer adds priority scheduling, worker management, and a fluent builder API.

Work-Stealing Scheduler - Implements a priority-aware work-stealing algorithm using crossbeam-deque to efficiently distribute tasks across a pool of worker threads. Idle workers automatically steal from busy peers' local deques and the global injector queue, ensuring no core sits idle while work is available.

Task Prioritization - Supports submitting tasks with High, Normal, or Low priority levels. High-priority tasks are always dequeued first from local and global deques, ensuring that latency-sensitive operations respond immediately while background work yields gracefully.

Fluent Builder API - The SchedulerBuilder provides a clean, chainable configuration interface. It defaults to the number of logical CPU cores with a minimum of two workers, and supports explicit worker count overrides and named queue configuration for future extensibility.

Graceful Shutdown - The Stop() method signals all worker threads to terminate and waits for each to complete its current task before joining. An automatic Drop guard ensures workers are signaled to stop even if the scheduler is dropped without an explicit shutdown call.

Lock-Free Performance - All queue operations use crossbeam-deque's lock-free Injector and Worker primitives. The global injector queue handles remote submissions, while each worker maintains local FIFO deques for cache-friendly task processing.

Decoupled Queue Library - The generic Queue module provides the core work-stealing logic as a standalone library, independent of any specific scheduler implementation. The StealingQueue<TTask> accepts any type implementing the Prioritized trait, making it reusable across projects.

graph LR
    classDef common   fill:#d4f5d4,stroke:#27ae60,stroke-width:1px,stroke-dasharray:5 5,color:#0a3a0a;
    classDef mountain fill:#f0d0ff,stroke:#9b59b6,stroke-width:2px,color:#2c0050;
    classDef echo     fill:#fffde0,stroke:#f0b429,stroke-width:2px,color:#4a3500;
    classDef worker   fill:#ffe0f0,stroke:#c0396a,stroke-width:1px,color:#4a0020;

    subgraph COMMON["Common - Abstract Core"]
        ActionEffect["ActionEffect ⚡ operation as value"]:::common
        Prioritized["Prioritized trait 🏷️ High / Normal / Low"]:::common
    end

    subgraph MOUNTAIN["Mountain ⛰️ - Application Logic"]
        Track["Track/ 🎯 Request Dispatcher"]:::mountain
        AppRunTime["ApplicationRunTime ⏱️ runtime executor"]:::mountain
        MountainEnv["Environment/ Providers 🔧 concrete service impls"]:::mountain
        Track --> AppRunTime
    end

    subgraph ECHO["Echo 📣 - Work-Stealing Scheduler"]
        direction TB
        subgraph SCHEDULER["Scheduler/"]
            SchedBuilder["SchedulerBuilder.rs ⚙️ fluent config, defaults to num_cpus"]:::echo
            SchedCore["Scheduler.rs 🎛️ Submit API + graceful Stop"]:::echo
            Workers["Worker.rs 🏃 Tokio threads, steal-on-idle"]:::worker
            SchedBuilder --> SchedCore
            SchedCore --> Workers
        end
        subgraph QUEUE["Queue/"]
            StealQ["StealingQueue.rs 🔒 crossbeam-deque, lock-free"]:::echo
        end
        subgraph TASK["Task/"]
            TaskDef["Task.rs + Priority.rs 📦 Future wrapper + priority level"]:::echo
        end

        Workers -- steals from --> StealQ
        SchedCore -- enqueues --> StealQ
        TaskDef -.implements.-> Prioritized
    end

    AppRunTime -- creates Future from --> ActionEffect
    AppRunTime -- Submit Future --> SchedCore
    Workers -- executes using --> MountainEnv

Connection paths:

Element/Echo/
├── Source/
│   ├── Library.rs              # Crate root (rlib), module declarations
│   ├── Queue/                  # Generic work-stealing queue library
│   │   ├── mod.rs              # Module re-exports
│   │   └── StealingQueue.rs    # Lock-free, priority-aware stealing deque
│   ├── Scheduler/              # Scheduler runtime and worker pool
│   │   ├── mod.rs              # Module re-exports
│   │   ├── Scheduler.rs        # Main scheduler: Submit + Stop lifecycle
│   │   ├── SchedulerBuilder.rs # Fluent builder with worker count config
│   │   └── Worker.rs           # Per-thread Tokio worker with steal loop
│   └── Task/                   # Task definition and priority
│       ├── mod.rs              # Module re-exports
│       ├── Task.rs             # Schedulable unit (Future + Priority)
│       └── Priority.rs         # High / Normal / Low enum
└── Documentation/
    └── GitHub/
        ├── Architecture.md     # Internal module structure and data flow
        └── DeepDive.md         # In-depth technical details

Echo serves as the core execution engine for Mountain, the native Rust/Tauri backend of the Land Code Editor. It integrates seamlessly with the ActionEffect pattern from the Common crate, executing composed asynchronous workflows across a priority-aware worker pool.

The Mountain runtime submits futures derived from ActionEffect values to the Echo scheduler, which distributes them across its workers alongside the concrete Environment provider implementations. High-priority UI operations - keystroke processing, command execution, diagnostics - always pre-empt background work like file indexing and syntax analysis.

• Rust 1.75 or later
• A Tokio runtime (Echo uses tokio internally)

Add Echo to your project via the Land workspace:

[dependencies]
Echo = { git = "https://github.com/CodeEditorLand/Echo.git", branch = "Current" }

The crate depends on tokio, crossbeam-deque, rand, log, num_cpus, and Common from the Land workspace. All dependencies are resolved through the workspace Cargo.toml configuration.

First, create and start the scheduler when your application initializes. The builder defaults to the number of logical CPU cores, with a minimum of two workers to ensure work-stealing is viable:

use std::sync::Arc;
use Echo::Scheduler::SchedulerBuilder;
use Echo::Task::Priority;

let Scheduler = Arc::new(SchedulerBuilder::Create().WithWorkerCount(8).Build());

Submit asynchronous tasks from anywhere in your application using the scheduler instance. Tasks are queued by priority and executed by the next available worker:

let MyTask = async {
    println!("This is running on an Echo worker thread!");
    // ... perform some work ...
};

// Submit the task with a desired priority
Scheduler.Submit(MyTask, Priority::Normal);

// Another example with high priority
Scheduler.Submit(async { /* critical work */ }, Priority::High);

Before your application exits, ensure a clean shutdown of all worker threads. The Stop() method drains the queue and waits for in-flight tasks to complete:

// Note: Arc::try_unwrap requires the Arc to have only one strong reference.
if let Ok(mut Scheduler) = Arc::try_unwrap(Scheduler) {
    Scheduler.Stop().await;
}

As a pure library crate, Echo provides architectural guarantees rather than runtime enforcement:

Echo is designed to be compatible with:

• Rust API Documentation 📖

• Architecture Overview - Land system architecture
• Deep Dive - In-depth technical details of the work-stealing algorithm
• Land Documentation - Complete documentation index
• Mountain - Primary consumer of Echo, native Tauri desktop shell
• Common - Abstract traits and ActionEffect system
• Why Rust
• Contribution Guide
• CHANGELOG.md
  • History of changes specific to Echo

This project is funded through NGI0 Commons Fund, a fund established by NLnet with financial support from the European Commission's Next Generation Internet program, under grant agreement No 101135429.

The project is operated by PlayForm, based in Sofia, Bulgaria. PlayForm acts as the open-source steward for Code Editor Land under the NGI0 Commons Fund grant.