Tags: ggml-org/llama.cpp
Tags
test-llama-archs : make tensor data stdev configurable and improve he… …lp (#29133) * test-llama-archs : make tensor data stdev configurable Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-llama-archs : expand usage and add examples Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-llama-archs : fail on unknown args and log usage Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-llama-archs : add test run summary Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-llama-archs : initialize Mamba ssm_a negative Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-recurrent-state-rollback : report NMSE for logits mismatches Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * test-recurrent-state-rollback : use NMSE for rollback logits checks Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * tests : disable invalid test * cont : adjust nmse_eps * tests : zero DSA indexer score projection in synthetic fixtures Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * cont : add support for `--arch` regex * cont : alternative top-k stability * cont : indentation * cont : fix top-k value * cont : consistent logs
tests/test-backend-ops : allow regex entries in the -o filter (#29204) * tests/test-backend-ops : allow regex entries in the -o filter so far -o only accepted a comma separated list of exact op names or full test case strings. entries that are not plain op names are now treated as regexes matched against the op name (e.g. "MUL_MAT.*"), while plain names keep their exact-matching behavior so that "-o ADD" does not match ADD_EX etc. Assisted-by: pi:llama.cpp/Qwen3.8-27B * cont : don't print the FA vec slice log when not needed * tests/test-backend-ops : reformat the help text use the same style as the other tools, with separate sections for modes, options, and examples Assisted-by: pi:llama.cpp/Qwen3.8-27B
args: add env vars for temperature, top-p, min-p and penalties (#27380) Allow configuring --temp, --top-p, --min-p, --repeat-penalty, --presence-penalty and --frequency-penalty via LLAMA_ARG_* so llama-server can be fully controlled from an EnvironmentFile (e.g. systemd on Debian). Use `llama-gen-docs` to regenerate the readme files.
server : do not forward --api-key-file to router-spawned child instan… …ces (#28938) In router mode, authentication belongs to the router. unset_reserved_args() already unset LLAMA_API_KEY, but did not unset LLAMA_ARG_API_KEY_FILE. When --api-key-file was passed, children re-validated against file keys only, causing clients using --api-key to 401 on chat completions (#28820). In addition, router internal calls without auth headers (such as POST /v1/streams/lookup and DELETE /v1/stream) were silently rejected with 401. Unset LLAMA_ARG_API_KEY_FILE in unset_reserved_args() so no API keys reach child instances. This keeps keys out of child argv, ensures all keys the router accepts work end-to-end, and prevents router internal stream calls from 401ing. Fixes #28820
ggml-metal : simplify fusion pattern op list declaration (#29206) * ggml-metal : derive non-empty fusion ops from ops_all Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp * ggml-metal : drop _all suffix from fusion op pattern vectors Assisted-by: pi:llama.cpp/DeepSeek-V4-Flash-Vision-Exp
json: Fixed json enum handling (#28518) * Fixed json enum handling Added common_json_value handling for enum values. Added tests/test-json.cpp to cover testing of some aspects of common_json. * Removed tests as requested. * Applied recommended style and simplification Simplified by delegating enum constructor to the constructor of the underlying type Matched style of surrounding templating code
hexagon: overhaul of buffer and DMA handling to support 64bit mapping… …s + improvements (#29197) * hex-dma64: enable support extended buffer mappings and 64bit dma hex-dma64: expand binary ops to support more DMA scenarios hex-dma64: add binary-ops.h hex-dma64: add --hex-dma64 to run.py and fix minor issues hex-dma64: update SSM_CONV to use dma with proper support for 64bit hex-ops: remove obsolete gate for % 128 in binary ops hex-l2: dont check weight tensors against dirty ranges hex-dma64: most binary ops now support dma hex-dma: use dma_addr_t instead of plain uint64_t to avoid overhead on older targets hex-dma: update all dma users to use dma_data (instead of pointers) hex-dma64: simplify lazy buffer mapping and clonning hex-fusion: factor out try_fuse_common that checks for dma64 buffers hex-bufs: minor cleanup for mmaping logic hex-bufs: simplify buffer clonning hex-ssm-conv: tighten gating checks and check vtcm size in kparams hex-binary: fix incorred mod/wrap in scalar ops hex-binary: make sure to call precompute kparams in support checks hex-dma64: update addr handling in mm,concat,binary hex-dma64: fixing up leftover of dma_addr_t conversion hex-binary: redo the kernel selection again and fix regressions in MOEs hex-binary: specialize per-type/per-op hex-binary: vtcm-layout and per-src dma-queue hex-dma64: update dma_push to transparently handle 64bit/extended * hex-cpy: fix improper rebase with the fixes for cont. tensors * hex-dma-cpy: update CPY to use safe dma rows/size limits * hex-mmap: bump number of mmaps to 64 to allow avoid eviction in larger models * hex-dma: add support for the secondary ring as a fallback for too-large transactions * hex-rope: fix freq_factors access with 64bit dma * hex-dma: audit all ops for proper use/gards for 64bit addresses * hex-dma64: uninline glu-compute funcs to avoid register pressure due to 64bit addr math * hex-dma64: refactor binary ops to separate dma loops * hex-devel: add inspect script to help with dbg and analysis * hex-dma: refactor dma-pipelines in unary-ops * hex-dma: rewrite softmax to use dma * hex-dma: rewrite GDN dma loops and improve HVX register usage * hex-gdn: fuse GDN+CPY * hex-mm: factor out HVX solver * hex-mm: remove hvx-flat kernels, the chunked version now handles vtcm limits much better * hex-buffs: reject huge buffer allocations that we cannot memory map * hex-inspect: add logic to look for float promo calls * hex-mm: reduce HVX register spills in HVX prompt kernels * hex-bufs: do not double count buffers from tensors in the same op * hex-roll: fix merge conflict * hex-dma: reroute all matmul ddr kernels to new chunked dma/vtcm kernels * hex-dev: update developer docs to include inspection for register spils and float promos * hex-ops: forgot to add new headers * hex-softmax: fix gpt-oss dims * hex-dma64: cleanup dma_addr_t casts * hex-dma64: add support for dma/vtcm for flash-atten with sinks * hex-mm-add: fix MUL_MAT+ADD fusion with bias.weights in extended bufs * hex-add-id: add support for dma for src1 (exp. table) * hex-dma: imrpove v73 fallback paths * hex-bufs: do not drop extended mappings during va defrag * hex-scripts: fix flake8 warnings * hex-docs: fix editor-config warnings * hex-inspect: fix warnings from ty
PreviousNext