Macrodata Labs helps robotics teams turn raw physical-world data into better training datasets. Refiner, our open-source data processing framework, lets you build pipelines locally in Python, then scale the same pipeline on managed cloud compute.
Introducing Refiner: focus on data, not infrastructure
Define pipelines from simple primitives. Refiner handles scale, orchestration, and everything in between.
Process robot episodes with trajectories, camera streams, audio, and language in one pipeline. Refiner handles streaming IO, sharding, and native data formats.
Deploy open-source models or bring your own API. Async execution, smart batching, parallelism, and retries are handled either way, locally or at cloud scale.
import refiner as mdr
(
mdr.read_hdf5(
"hf://datasets/nvidia/ALOHA-Cosmos-Policy/**/*.hdf5",
groups="/",
datasets={"action": "action", "observation.state": "observations/qpos"},
)
.to_robot_rows(fps=25, robot_type="aloha")
.write_lerobot("s3://robots/aloha-lerobot")
)Get more from robotics data
Read and convert Parquet, HDF5, MCAP, Zarr, RLDS, and LeRobot without custom scripts or slow local downloads.
Use optimized pipelines for timestamped subtask annotation and ego-vision hand tracking across robot episodes.
Estimate task-completion progress with reward models such as Robometer, then use those scores to weight the frames that matter most.
Scale instantly with launch_cloud()
.launch_local() becomes .launch_cloud(). Scale the same pipeline without rewriting code, changing data formats, or rebuilding your local workflow.
Run many shards across managed CPU and GPU workers without reservations or machine provisioning. Macrodata Labs handles orchestration, scheduling, and worker lifecycle.
Resources attach when work starts and release when it finishes. You pay for the compute your jobs actually consume, without idle cluster overhead.
Supervise in real time
Job: robotics-demo ID: 019e68d3...eda75Status: pending Kind: cloudStage 0 robotics_transform 0/64 shards · workers run=0 done=0 tot=8─ logs ─────────────────────────────────────────────
Inspect the DAG, transforms, launch settings, dependencies, and captured code behind each dataset build, so every output is traceable back to the run that produced it.
See the stage, shard, worker, traceback, logs, and retry state for failures instead of piecing together what happened after the fact. Inspect it on the web platform or let your agent pull it through the CLI.
See whether a run is limited by decoding, model calls, writing, CPU, memory, network, or GPU before scaling it further.
// GET STARTED
Use Refiner to start extracting more signal from your data today, or reach out to discuss your robotics data challenges directly.