Skip to content

请问llm_type为llama时,模型权重文件必须为model/stable-vicuna-13B.ggml.q4_2.bin吗?权重文件是否可以为llama2?以及策略是否支持fp16? #534

@15229684931

Description

@15229684931

我在npu上进行测试时,迁移了wenda的代码,在适配llama时,config.yaml如下配置
llama: path: "/home/ch/stable-vicuna-13B-HF" strategy: "npu fp16"
image
但是在web端推理时,出现了回答不全的现象
231

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions