Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

gpu: Group GPU Library sets by variant #6483

Merged
merged 1 commit into from
Aug 23, 2024

Conversation

dhiltgen
Copy link
Collaborator

@dhiltgen dhiltgen commented Aug 23, 2024

The recent cuda variant changes uncovered a bug in ByLibrary which failed to group by common variant for GPU types.

New unit test fails without the 1 line fix, and passes with.

Fixes #6479

The recent cuda variant changes uncovered a bug in ByLibrary
which failed to group by common variant for GPU types.
@dhiltgen dhiltgen merged commit 69be940 into ollama:main Aug 23, 2024
12 checks passed
@dhiltgen dhiltgen deleted the multi_gpu_variant branch August 23, 2024 22:12
deep93333 pushed a commit to deep93333/ollama that referenced this pull request Sep 9, 2024
The recent cuda variant changes uncovered a bug in ByLibrary
which failed to group by common variant for GPU types.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

v0.3.7-rc5 no longer uses multiple GPUs for a single model
2 participants