Skip to content

About

NeurIPS 2026's GRINQH quantization method (https://arxiv.org/pdf/2606.23419) as a plugin for vLLM

Resources

Stars

0 stars

Watchers

0 watching

Forks

 
 

Latest commit

 

History

2 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

GRINQH vLLM Plugin

Out-of-tree plugin for the Graded Input-based Quantization Hierarchy (GRINQH) quantization method for vLLM.

Note: We are working on cleaning up scripts for a smoother quantization workflow, which we aim to push by the 11th.

Install

uv pip install -e /path/to/vllm-grinqh-plugin

The package registers itself through the vllm.general_plugins entry-point group, so quantization="grinqh" becomes available after installation.

Requirements

GRINQH requires a set of thresholds calibrated for a given model. The replication repository provides scripts to generate them.

Alternatively, GRINQH accepts external thresholds in JSON format, which can be provided as a path using the GRINQH_JSON_PATH env variable.

Tested versions

We ran all of our experiments for Triton 3.6.0 on a fork of vLLM's main @ 0a5cbf633 (2026-06-03, v0.22.1).

About

NeurIPS 2026's GRINQH quantization method (https://arxiv.org/pdf/2606.23419) as a plugin for vLLM

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Packages

Contributors

Languages