Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

现在的代码怎么实现batch大于1的推理 #140

Open
leb-2 opened this issue Aug 23, 2024 · 2 comments
Open

现在的代码怎么实现batch大于1的推理 #140

leb-2 opened this issue Aug 23, 2024 · 2 comments

Comments

@leb-2
Copy link

leb-2 commented Aug 23, 2024

我想问一下怎么实现batch大于1的推理，一个固定的值就行，不需要动态batch
@Linaom1214
Copy link
Owner

Linaom1214 commented Aug 24, 2024 •
edited
Loading

我想问一下怎么实现batch大于1的推理，一个固定的值就行，不需要动态batch

目前的代码不支持，不使用nms 插件的话，很容易实现，现在是为了实现不使用第三库直接插入nms插件。众所周知trt是半开源的，我也无解。还有一种方案是，使用troch.fx 实现，那样就是需要一点第三方依赖，不能开盖即食。

@leb-2
Copy link
Author

leb-2 commented Aug 26, 2024

如果不使用nms插件的话怎么实现，能否详细讲讲，谢谢

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@Linaom1214 @leb-2