sm70

Hand-written NVFP4 W4A16 CUDA kernels for Volta

Benchmarks and notes for running modern LLMs with vLLM on 8x Tesla V100-32GB in 2026.

Run Qwen3.6-27B on four Tesla V100s at 366 tok/s using hand-written NVFP4 CUDA kernels and chain-MTP speculation.

Qwen3.8-27B in native NVFP4/FP8 on 2x PCIe Tesla V100-32GB (SM70): the PCIe runbook for v100-skinny + 1Cat-vLLM, with the 3 fixes that make it work without NVLink. 61-74 tok/s decode, MTP speculative decoding, OpenAI-compatible.

vLLM for nVidia Volta (sm_70)

Multi-GPU acceleration for MiniMax H3 video generation on NVIDIA V100 (sm_70). Ulysses sequence parallelism as a drop-in ComfyUI custom node — ~19 min to ~7 min on 8x V100.

Reproducible llama.cpp kernel and runtime optimization lab for dual NVIDIA Tesla V100 GPUs (SM70)

SGLang fork for IBM POWER9 (ppc64le): Tesla V100 sm70, CUDA 12.4, Granite LLM inference. Triton attention, float16, OpenAI-compatible API.

Serve Qwen3.8-Flash-Next (125B MoE, NVFP4) at 262K context on 4x Tesla V100-SXM2-32GB — a Volta (sm70) port of SGLang for agentic coding. Native OpenAI + Anthropic APIs.

Serve Qwen3.5-397B-A17B (AWQ) on 8x Tesla V100-SXM2-32GB (DGX-1, TP8) for agentic coding & ops — a downstream fork of 1Cat-vLLM.

Runbook + benchmarks: Qwen3.8-Flash-Next-ABLITERATED NVFP4 on 4× Tesla V100-32GB (reflashed SXM2→PCIe, 2+2 NVLink + PLX). 1Cat-vLLM 1.5.0, TP4 — 262,144-token context validated, 46 tok/s decode, 122 tok/s aggregate at 4 concurrent streams. Full E0–E17 optimization log with measured evidence.

Inferência local em Tesla V100 (Volta/sm_70): binários pré-compilados, scripts de setup e benchmarks medidos

VastLLM: a production-oriented FastLLM fork for native C++ inference, V100/SM70, long context, and Qwen3.8/3.6/3.5 series; upstream: ztxz16/fastllm

PyTorch 2.12 fork for IBM POWER9/POWER10 (ppc64le) with CUDA 12.4 and Triton. Tesla V100 sm70, GPU training and LLM inference.

OpenAI Triton compiler fork for IBM POWER9/POWER10 (ppc64le) with CUDA 12.4. Tesla V100 sm70 GPU kernels for PyTorch and SGLang.

Run Qwen3.8-Flash-Next-ABLITERATED in NVFP4 on 4×V100-32GB: 262K context, 46 tok/s decode, OpenAI-compatible, agent-ready.

DP4A FlashAttention-2 and GEMM for Volta GPUs. 46 TOP/s INT8 on CMP 100-210 where tensor cores are firmware-disabled. The .superl8 format loads weights at memory speed.