Commits

Branch selector

main

User selector

All users

All time

Commit history

Commits on Sep 10, 2024

• Add NVIDIA Meetup slides, announce AMD meetup, and add contact info (#8319)

  simon-mo

  authored
  a1d8742

  Copy full SHA for a1d8742

• [Misc] Fused MoE Marlin support for GPTQ (#8217)

  dsikka

  authored
  6cd5e5b

  Copy full SHA for 6cd5e5b

Commits on Sep 9, 2024

• [Misc] GPTQ Activation Ordering (#8135)

  kylesayrs

  authored
  c7cb5c3

  Copy full SHA for c7cb5c3

• [Bugfix] Correct adapter usage for cohere and jamba (#8292)

  vladislavkruglikov

  authored
  f9b4a2d

  Copy full SHA for f9b4a2d

• [Frontend] Add progress reporting to run_batch.py (#8060)

  Show description for 58fcc85
  

  alugowski

  and

  Adam Lugowski

  authored
  58fcc85

  Copy full SHA for 58fcc85

• [Bugfix] Streamed tool calls now more strictly follow OpenAI's format; ensures Vercel AI SDK compatibility (#8272)

  K-Mistele

  authored
  08287ef

  Copy full SHA for 08287ef

Commits on Sep 8, 2024

• [Bugfix] Fix async postprocessor in case of preemption (#8267)

  alexm-redhat

  authored
  4ef41b8

  Copy full SHA for 4ef41b8

Commits on Sep 7, 2024

• [CI/Build] Use python 3.12 in cuda image (#8133)

  Show description for cfe712b
  joerunde

  authored
  cfe712b

  Copy full SHA for cfe712b

• ppc64le: Dockerfile fixed, and a script for buildkite (#8026)

  sumitd2

  authored
  b962ee1

  Copy full SHA for b962ee1

• [Model][VLM] Decouple weight loading logic for `Paligemma` (#8269)

  Isotr0py

  authored
  36bf815

  Copy full SHA for 36bf815

• [Model][VLM] Support multi-images inputs for InternVL2 models (#8201)

  Isotr0py

  authored
  e807125

  Copy full SHA for e807125

• [Bugfix] Fix broken OpenAI tensorizer test (#8258)

  DarkLight1337

  authored
  9f68e00

  Copy full SHA for 9f68e00

• [tpu][misc] fix typo (#8260)

  youkaichao

  authored
  ce2702a

  Copy full SHA for ce2702a

• Enable Random Prefix Caching in Serving Profiling Tool (benchmark_serving.py) (#8241)

  wschin

  authored
  795b662

  Copy full SHA for 795b662

• [Model] Multi-input support for LLaVA (#8238)

  DarkLight1337

  authored
  2f707fc

  Copy full SHA for 2f707fc

• [Bugfix] Fix Hermes tool call chat template bug (#8256)

  Show description for 41e95c5
  K-Mistele

  authored
  41e95c5

  Copy full SHA for 41e95c5

• [misc] [doc] [frontend] LLM torch profiler support (#7943)

  SolitaryThinker

  authored
  12dd715

  Copy full SHA for 12dd715

Commits on Sep 6, 2024

• [Model] Allow loading from original Mistral format (#8168)

  Show description for 29f49cd
  

  patrickvonplaten

  and

  mgoin

  authored
  29f49cd

  Copy full SHA for 29f49cd

• [Misc] Remove `SqueezeLLM` (#8220)

  dsikka

  authored
  23f3222

  Copy full SHA for 23f3222

• [Kernel] [Triton] Memory optimization for awq_gemm and awq_dequantize, 2x throughput (#8248)

  rasmith

  authored
  9db52ea

  Copy full SHA for 9db52ea

• [CI/Build] Increasing timeout for multiproc worker tests (#8203)

  akondrat-amd

  authored
  1447c97

  Copy full SHA for 1447c97

• [Misc] Use ray[adag] dependency instead of cuda (#7938)

  ruisearch42

  authored
  de80783

  Copy full SHA for de80783

• [Frontend] Add --logprobs argument to `benchmark_serving.py` (#8191)

  afeldman-nm

  authored
  e5cab71

  Copy full SHA for e5cab71

• [BugFix] Fix Granite model configuration (#8216)

  njhill

  authored
  baa5467

  Copy full SHA for baa5467

• [Core] Support load and unload LoRA in api server (#6566)

  Show description for db3bf7c
  

  Jeffwan

  and

  jeejeelee

  authored
  db3bf7c

  Copy full SHA for db3bf7c

Commits on Sep 5, 2024

• [Documentation][Spec Decode] Add documentation about lossless guarantees in Speculative Decoding in vLLM (#7962)

  sroy745

  authored
  2febcf2

  Copy full SHA for 2febcf2

• Move verify_marlin_supported to GPTQMarlinLinearMethod (#8165)

  mgoin

  authored
  2ee4528

  Copy full SHA for 2ee4528

• [MODEL] Qwen Multimodal Support (Qwen-VL / Qwen-VL-Chat) (#8029)

  Show description for 9da25a8
  

  alex-jw-brooks

  and

  DarkLight1337

  authored
  9da25a8

  Copy full SHA for 9da25a8

• Inclusion of InternVLChatModel In PP_SUPPORTED_MODELS(Pipeline Parallelism) (#7860)

  Manikandan-Thangaraj-ZS0321

  authored
  8685ba1

  Copy full SHA for 8685ba1

• [Doc] Indicate more information about supported modalities (#8181)

  DarkLight1337

  authored
  288a938

  Copy full SHA for 288a938

• [Core/Bugfix] Add query dtype as per FlashInfer API requirements. (#8173)

  elfiegg

  authored
  e39ebf5

  Copy full SHA for e39ebf5

• [ci] Mark LoRA test as soft-fail (#8160)

  Show description for ba262c4
  khluu

  authored
  ba262c4

  Copy full SHA for ba262c4

• [Misc] Clean up RoPE forward_native (#8076)

  WoosukKwon

  authored
  4624d98

  Copy full SHA for 4624d98

• [bugfix] >1.43 constraint for openai (#8169)

  Show description for 1afc931
  

  SolitaryThinker

  and

  mgoin

  authored
  1afc931

  Copy full SHA for 1afc931

Commits on Sep 4, 2024

• [Doc] [Misc] Create CODE_OF_CONDUCT.md (#8161)

  mmcelaney

  authored
  e01c2be

  Copy full SHA for e01c2be