Skip to content

[Bad Case]: 使用minicpm4-0.5B同时启用sparse attention时报错 #326

@junyou2001

Description

@junyou2001

Description / 描述

你好!
我注意到minicpm4-0.5B中描述无法支持sparse attention,但是我看modeling中有就修改为可用,但报错:

topk_idx[topk_idx >= q_idx[None, :, None]] = -1
RuntimeError: The size of tensor a (355) must match the size of tensor b (711) at non-singleton dimension 1

因为还没有细看infllm实现所以不太好debug,请问该如何解决呢?非常感谢!

Case Explaination / 案例解释

No response

Metadata

Metadata

Assignees

No one assigned

    Labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions