Commits

Branch selector

 main

User selector

All users

All time

Commit History

Commits on Nov 28, 2023

• output logits

  

  FAREAST\yibingsheng

  committed
  7474f26

  Copy full SHA for 7474f26

Commits on Nov 27, 2023

• Init model on GPU to reduce CPU memory footprint (#1796)

  beginlner

  authored
  a8b150c

  Copy full SHA for a8b150c

• Added echo function to OpenAI API server. (#1504)

  wanmok

  authored
  665cbce

  Copy full SHA for 665cbce

Commits on Nov 24, 2023

• Fix model docstrings (#1764)

  WoosukKwon

  authored
  7c60044

  Copy full SHA for 7c60044

• [Build] Avoid building too many extensions (#1624)

  ymwangg

  authored
  e0c6f55

  Copy full SHA for e0c6f55

Commits on Nov 22, 2023

• Fix repetition penalty aligned with huggingface (#1577)

  beginlner

  authored
  de23687

  Copy full SHA for de23687

• Set top_p=0 and top_k=-1 in greedy sampling (#1748)

  beginlner

  authored
  4cea74c

  Copy full SHA for 4cea74c

• [DOCS] Add engine args documentation (#1741)

  casper-hansen

  authored
  a921d8b

  Copy full SHA for a921d8b

• Add stop_token_ids in SamplingParams.__repr__ (#1745)

  chenxu2048

  authored
  094f716

  Copy full SHA for 094f716

Commits on Nov 21, 2023

• [FIX] Fix the case when `input_is_parallel=False` for `ScaledActivation` (#1737)

  zhuohan123

  authored
  7d761fe

  Copy full SHA for 7d761fe

• [BugFix] Fix TP support for AWQ (#1731)

  WoosukKwon

  authored
  cf35d8f

  Copy full SHA for cf35d8f

• fix RAM OOM when load large models in tensor parallel mode. (#1395)

  Show description for 4bb6b67
  

  boydfd

  and

  rlin-tw

  authored
  4bb6b67

  Copy full SHA for 4bb6b67

• Rewrite torch.repeat_interleave to remove cpu synchronization (#1599)

  beginlner

  authored
  819b18e

  Copy full SHA for 819b18e

• [Fix] Fix bugs in scheduler (#1727)

  linotfan

  authored
  19849db

  Copy full SHA for 19849db

• Fix hanging in the scheduler caused by long prompts (#1534)

  chenxu2048

  authored
  3d4ceb2

  Copy full SHA for 3d4ceb2

Commits on Nov 20, 2023

• [BugFix] Fix a bug in loading safetensors (#1732)

  WoosukKwon

  authored
  f5a37c6

  Copy full SHA for f5a37c6

• [FIX] Update the doc link in README.md (#1730)

  zhuohan123

  authored
  32c927b

  Copy full SHA for 32c927b

• Migrate linter from `pylint` to `ruff` (#1665)

  simon-mo

  authored
  5ffc0d1

  Copy full SHA for 5ffc0d1

• [Docs] Fix the code block's format in deploying_with_docker page (#1722)

  HermitSun

  authored
  112627e

  Copy full SHA for 112627e

• Documentation about official docker image (#1709)

  simon-mo

  authored
  37c1e3c

  Copy full SHA for 37c1e3c

Commits on Nov 19, 2023

• Add instructions to install vLLM+cu118 (#1717)

  WoosukKwon

  authored
  06e9ebe

  Copy full SHA for 06e9ebe

• Bump up to v0.2.2 (#1689)

  WoosukKwon

  authored
  c5f7740

  Copy full SHA for c5f7740

• Fix warning msg on quantization (#1715)

  WoosukKwon

  authored
  be66d9b

  Copy full SHA for be66d9b

• [Optimization] Implement fused add rmsnorm (#1667)

  beginlner

  authored
  e105424

  Copy full SHA for e105424

• Add AWQ support for all models (#1714)

  WoosukKwon

  authored
  8d17774

  Copy full SHA for 8d17774

• use get_tensor in safe_open (#1696)

  twaka

  authored
  e946260

  Copy full SHA for e946260

Commits on Nov 18, 2023

• Support download models from www.modelscope.cn (#1588)

  liuyhwangyh

  authored
  edb3055

  Copy full SHA for edb3055

• Use `quantization_config` in hf config (#1695)

  WoosukKwon

  authored
  bb00f66

  Copy full SHA for bb00f66

• Support Min P Sampler (#1642)

  esmeetu

  authored
  e87557b

  Copy full SHA for e87557b

Commits on Nov 17, 2023

• [Minor] Fix comment (#1704)

  linotfan

  authored
  dcc543a

  Copy full SHA for dcc543a

• Update the adding-model doc according to the new refactor (#1692)

  zhuohan123

  authored
  0fc280b

  Copy full SHA for 0fc280b

• [Fix] Fix comm test (#1691)

  zhuohan123

  authored
  20d0699

  Copy full SHA for 20d0699

Commits on Nov 16, 2023

• Return usage for openai streaming requests (#1663)

  ichernev

  authored
  686f5e3

  Copy full SHA for 686f5e3

• [Fix] Update Supported Models List (#1690)

  zhuohan123

  authored
  415d109

  Copy full SHA for 415d109

• Support Microsoft Phi 1.5 (#1664)

  maximzubkov

  authored
  521b35f

  Copy full SHA for 521b35f