• Neuriplo is a C++ library designed for seamless integration of various backend engines for inference tasks.
• It supports vision, graph, and GGUF-native generative runtimes including OpenCV DNN, TensorFlow, PyTorch (LibTorch), ONNX Runtime, TensorRT, OpenVINO, TVM, GGML, MIGraphX, Cactus, llama.cpp, ExecuTorch, and LiteRT.
• The project aims to provide a unified interface for performing inference using these backends, allowing flexibility in choosing the most suitable backend based on performance or compatibility requirements.
• The library is currently mainly used as component of the Neuriplo Infer Project

• C++17
• OpenCV
• glog

• OpenCV DNN module
• ONNX Runtime
• Pytorch (Libtorch)
• TensorRT
• OpenVINO
• Tensorflow (LibTensorFlow C++ library) - inference on saved models, not graph
• GGML - Efficient tensor library for machine learning
• TVM - Open deep learning compiler stack
• MIGraphX - AMD ROCm graph inference engine
• Cactus - GGUF-native text generation backend
• llama.cpp - GGUF-native LLM and multimodal backend
• ExecuTorch - PyTorch edge inference runtime
• LiteRT - Google AI Edge runtime, formerly TensorFlow Lite

• CUDA (if you want to use GPU)

./scripts/setup_dependencies.sh --backend <BACKEND_NAME>

Supported <BACKEND_NAME> values:

• OPENCV_DNN
• ONNX_RUNTIME
• LIBTORCH
• TENSORRT
• LIBTENSORFLOW
• OPENVINO
• GGML
• TVM
• MIGRAPHX
• CACTUS
• LLAMACPP
• EXECUTORCH
• LITERT

./scripts/test_backends.sh

./scripts/test_backends.sh --backend <BACKEND_NAME>

Backend-specific setup notes, model format constraints, Docker workflows, and GGUF backend details live in docs/DEPENDENCY_MANAGEMENT.md. ExecuTorch delegate selection is covered in docs/EXECUTORCH_DELEGATES.md.

1. Clone the repository:

   git clone https://github.com/neuriplo.git
   cd neuriplo

2. Create a build directory and navigate into it:

   mkdir build
   cd build

3. Configure the build with CMake:

   cmake ..

   Optionally, you can specify the default backend by setting -DDEFAULT_BACKEND=your_backend during configuration.

   • Note: If the backend package is not installed on your system, set the path manually in the backend's CMake module or use the automated setup scripts.

4. Build the project:

   cmake --build .

   This will compile the project along with the selected backend(s).

To use the Neuriplo library in your project, link against it and include necessary headers (check the example here):

target_link_libraries(your_project PRIVATE neuriplo)
target_include_directories(your_project PRIVATE path_to/neuriplo/include)

Ensure you have initialized and set up the selected backend(s) appropriately in your code using the provided interface headers.

Neuriplo's backend layer is organized around five design patterns, all built on the single InferenceInterface contract that setup_inference_engine returns:

• Adapter — each *Infer class wraps a vendor SDK behind InferenceInterface.
• Bridge — ModelRunner orchestrates lifecycle and delegates to any backend without knowing the concrete type.
• Abstract Factory — each backend ships an IBackendRuntimeFactory (*RuntimeFactory) that produces a coherent {backend, allocator, converter} family. setup_inference_engine builds through the factory selected at compile time by -DDEFAULT_BACKEND.
• Decorator — ProfilingBackend / LoggingBackend / CachingBackend / QuantizedBackend add cross-cutting behavior. They are opt-in; enable the profiling/logging chain at runtime with NEURIPLO_ENABLE_PROFILING=1 and NEURIPLO_ENABLE_LOGGING=1 (default off, so the production path is unchanged).
• State — BackendState{Uninitialized, Loading, Ready, Failed} makes the lifecycle explicit. Load failures set Failed and throw ModelLoadException, which the facade translates to a nullptr return (no std::exit).

The public contract is unchanged: setup_inference_engine(model_path, use_gpu, batch_size, input_sizes) still returns std::unique_ptr<InferenceInterface>. See docs/REFACTOR_DESIGN_PATTERNS.md for the full design.

Neuriplo uses a centralized configuration system that makes it easy to add new backends. The system consists of:

┌─────────────────────────────────────────────────────────────────┐
│                    Backend Configuration Flow                    │
├─────────────────────────────────────────────────────────────────┤
│                                                                   │
│  1. versions.env                                                │
│     ┌────────────────────────────────────────────────────┐      │
│     │ Defines version numbers:                           │      │
│     │ TVM_VERSION=0.18.0                                 │      │
│     │ ONNX_RUNTIME_VERSION=1.19.2                        │      │
│     │ PYTORCH_VERSION=2.3.0                              │      │
│     └────────────────────────────────────────────────────┘      │
│                           ↓                                      │
│  2. cmake/versions.cmake                                        │
│     ┌────────────────────────────────────────────────────┐      │
│     │ Reads versions.env and validates consistency       │      │
│     │ Maps backends to their version variables           │      │
│     │ Ensures all backends have version variables        │      │
│     └────────────────────────────────────────────────────┘      │
│                           ↓                                      │
│  3. scripts/*.sh                                                │
│     ┌────────────────────────────────────────────────────┐      │
│     │ Backend arrays and mappings defined directly       │      │
│     │ Sources versions.env for version numbers           │      │
│     └────────────────────────────────────────────────────┘      │
│                           ↓                                      │
│  ✓ All bash scripts and CMake automatically synchronized        │
└─────────────────────────────────────────────────────────────────┘

See Adding an Inference Backend for the complete workflow.

At a high level, a new backend needs:

1. A backends/<backend>/src implementation deriving from InferenceInterface.
2. Backend tests under backends/<backend>/test.
3. A backend CMake module such as cmake/NCNN.cmake.
4. CMake registration in cmake/BackendRegistry.cmake.
5. Link and dependency-validation rules for the backend's library layout.
6. Factory registration in include/InferenceBackendSetup.hpp and src/InferenceBackendSetup.cpp.
7. Script and docs updates when setup, testing, model format, or runtime behavior changes.

The CMake registry centralizes supported backend IDs, selected-backend module lookup, test directory lookup, and backend version-variable mapping.

The system automatically validates that every backend has a corresponding version:

# CMake validation (runs automatically)
cmake ..

Local checks (format, cppcheck, ASan/UBSan, clang-tidy) and git hook setup:

./scripts/quality/setup_hooks.sh   # pre-commit + pre-push hooks
./scripts/quality/run.sh           # format + cppcheck
./scripts/quality/sanitizers.sh    # ASan + UBSan build and ctest

See Code Quality for details.

For detailed documentation, see the docs/ directory:

• Code Quality - Formatting, static analysis, sanitizers, pre-commit hooks
• Architecture / Design Patterns - Adapter, Bridge, Abstract Factory, Decorator, and State design of the backend layer
• Dependency Management - Complete setup guide for all backends
• Adding an Inference Backend - Backend implementation and registration checklist