You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
{{ message }}
Repository navigation
[Feature Request] Android GPU Acceleration Support (Vulkan/MLC-LLM) #371
Feature Request
Description
Currently, LLM Unity only supports CPU inference on Android (LLMLib.cs line 671-674 only adds "android" architecture regardless of GPU settings). This results in very slow inference speeds (2-3 tokens/sec) on mobile devices.
I would like to request Android GPU acceleration support using Vulkan backend.
Current Behavior
numGPULayers setting is visible in Inspector but has no effect on Android PossibleArchitectures() in LLMLib.cs only returns CPU architecture for Android:
csharp
else if (Application.platform == RuntimePlatform.Android)
{
architectures.Add("android"); // Only CPU, no GPU option
}
Activity