Commits

Branch selector

main

User selector

All users

All time

Commit history

Commits on Oct 2, 2023

• diff patch

  kduxin

  committed
  57ec86bView commit details

  Copy full SHA for 57ec86b

  Browse repository at this point

• faster inference when returning the whole list of logprobs

  kduxin

  committed
  7acda1eView commit details

  Copy full SHA for 7acda1e

  Browse repository at this point

• faster when returning the whole list of logprobs

  kduxin

  committed
  4943e45View commit details

  Copy full SHA for 4943e45

  Browse repository at this point

Commits on Oct 1, 2023

• Fix boundary check in paged attention kernel (#1241)

  soundOfDestiny

  authored
  ebe4d1dView commit details

  Copy full SHA for ebe4d1d

  Browse repository at this point

• Added `dtype` arg to benchmarks (#1228)

  kg6-sleipnir

  authored
  b5a10ebView commit details

  Copy full SHA for b5a10eb

  Browse repository at this point

Commits on Sep 29, 2023

• fixing typo in `tiiuae/falcon-rw-7b` model name (#1226)

  0ssamaak0

  authored
  0967102View commit details

  Copy full SHA for 0967102

  Browse repository at this point

Commits on Sep 28, 2023

• Bump up the version to v0.2.0 (#1212)

  WoosukKwon

  authored
  e2fb71eView commit details

  Copy full SHA for e2fb71e

  Browse repository at this point

• Provide default max model length (#1224)

  WoosukKwon

  authored
  f936657View commit details

  Copy full SHA for f936657

  Browse repository at this point

• Fix OOM in attention kernel test (#1223)

  WoosukKwon

  authored
  6f88f76View commit details

  Copy full SHA for 6f88f76

  Browse repository at this point

• Add Mistral to supported model list (#1221)

  WoosukKwon

  authored
  202351dView commit details

  Copy full SHA for 202351d

  Browse repository at this point

• [Fix] Remove false assertion (#1222)

  WoosukKwon

  authored
  2e8e49fView commit details

  Copy full SHA for 2e8e49f

  Browse repository at this point

• Fix Mistral model (#1220)

  WoosukKwon

  authored
  a8e98aeView commit details

  Copy full SHA for a8e98ae

  Browse repository at this point

• [Mistral] Mistral-7B-v0.1 support (#1196)

  Show description for bb1ba58
  

  Bam4d

  and

  timlacroix

  authored
  bb1ba58View commit details

  Copy full SHA for bb1ba58

  Browse repository at this point

• Add rope_scaling to Qwen (#1210)

  Sanster

  authored
  7bedab5View commit details

  Copy full SHA for 7bedab5

  Browse repository at this point

• Add `skip_special_tokens` sampling params (#1186)

  blahblahasdf

  authored
  20f7cc4View commit details

  Copy full SHA for 20f7cc4

  Browse repository at this point

• Use standard extras for uvicorn (#1166)

  danilopeixoto

  authored
  649aa73View commit details

  Copy full SHA for 649aa73

  Browse repository at this point

Commits on Sep 27, 2023

• Automatically configure `max_num_batched_tokens` (#1198)

  WoosukKwon

  authored
  a19bc5cView commit details

  Copy full SHA for a19bc5c

  Browse repository at this point

• fix qwen-14b model (#1173)

  Sanster

  authored
  28e616cView commit details

  Copy full SHA for 28e616c

  Browse repository at this point

• fix typo (#1184)

  Show description for 30e7752
  

  WrRan

  and

  zhuohan123

  authored
  30e7752View commit details

  Copy full SHA for 30e7752

  Browse repository at this point

• Support Longchat and RoPE scaling (#555)

  Show description for 21877b0
  

  3 peopleauthored
  21877b0View commit details

  Copy full SHA for 21877b0

  Browse repository at this point

• Allocate more shared memory to attention kernel (#1154)

  Yard1

  authored
  cf5cb1eView commit details

  Copy full SHA for cf5cb1e

  Browse repository at this point

Commits on Sep 26, 2023

• Add comments on RoPE initialization (#1176)

  WoosukKwon

  authored
  03ffd0aView commit details

  Copy full SHA for 03ffd0a

  Browse repository at this point

• [Setup] Enable `TORCH_CUDA_ARCH_LIST` for selecting target GPUs (#1074)

  WoosukKwon

  authored
  a425bd9View commit details

  Copy full SHA for a425bd9

  Browse repository at this point

Commits on Sep 24, 2023

• Align `max_tokens` behavior with openai (#852)

  HermitSun

  authored
  bbbf865View commit details

  Copy full SHA for bbbf865

  Browse repository at this point

• Fix config for Falcon (#1164)

  WoosukKwon

  authored
  9f6be86View commit details

  Copy full SHA for 9f6be86

  Browse repository at this point

• [FIX] Simplify sampler logic (#1156)

  zhuohan123

  authored
  f187877View commit details

  Copy full SHA for f187877

  Browse repository at this point

Commits on Sep 23, 2023

• [Sampler] Vectorized sampling (simplified) (#1048)

  Show description for 947b794
  

  zhuohan123

  and

  Yard1

  authored
  947b794View commit details

  Copy full SHA for 947b794

  Browse repository at this point

Commits on Sep 22, 2023

• Announce the First vLLM Meetup (#1148)

  WoosukKwon

  authored
  8d926e9View commit details

  Copy full SHA for 8d926e9

  Browse repository at this point

• Docs: Fix broken link to openai example (#1145)

  Show description for 4ee52bb
  nkpz

  authored
  4ee52bbView commit details

  Copy full SHA for 4ee52bb

  Browse repository at this point

• Use `--ipc=host` in docker run for distributed inference (#1125)

  WoosukKwon

  authored
  7d7e3b7View commit details

  Copy full SHA for 7d7e3b7

  Browse repository at this point

Commits on Sep 21, 2023

• feat: support stop_token_ids parameter. (#1097)

  gesanqiu

  authored
  f98b745View commit details

  Copy full SHA for f98b745

  Browse repository at this point

• clean api code, remove redundant background task. (#1102)

  yasong677

  authored
  2d1e86fView commit details

  Copy full SHA for 2d1e86f

  Browse repository at this point

• Add float16 and float32 (#1115)

  WoosukKwon

  authored
  1ac4ccfView commit details

  Copy full SHA for 1ac4ccf

  Browse repository at this point

• Replace DtypeTensor (#1123)

  WoosukKwon

  authored
  2ac4d5eView commit details

  Copy full SHA for 2ac4d5e

  Browse repository at this point

Commits on Sep 20, 2023

• rope_theta and max_position_embeddings from config (#1096)

  Show description for 3302f0a
  

  3 peopleauthored
  3302f0aView commit details

  Copy full SHA for 3302f0a

  Browse repository at this point