Skip to content

llamamodel: add gemma model support #1992

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
manyoso merged 3 commits into from
Feb 21, 2024
Merged

llamamodel: add gemma model support #1992

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
5608b93
llamamodel: add gemma model support
cebtenzzre Feb 21, 2024
f8f92cb
models2.json: add gemma model
cebtenzzre Feb 21, 2024
7b8d3f7
gemma: fix default prompt template
cebtenzzre Feb 21, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 1 addition & 1 deletion gpt4all-backend/llama.cpp-mainline
Open in desktop
Submodule llama.cpp-mainline updated 138 files
4 changes: 2 additions & 2 deletions gpt4all-backend/llamamodel.cpp
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -519,8 +519,8 @@ DLL_EXPORT bool magic_match(const char *fname) {
bool valid = true;

static const std::vector<const char *> known_arches {
"baichuan", "bloom", "codeshell", "falcon", "gpt2", "llama", "mpt", "orion", "persimmon", "phi2", "plamo",
"qwen", "qwen2", "refact", "stablelm", "starcoder"
"baichuan", "bloom", "codeshell", "falcon", "gemma", "gpt2", "llama", "mpt", "orion", "persimmon", "phi2",
"plamo", "qwen", "qwen2", "refact", "stablelm", "starcoder"
};

if (std::find(known_arches.begin(), known_arches.end(), arch) == known_arches.end()) {
Expand Down
48 changes: 32 additions & 16 deletions gpt4all-chat/metadata/models2.json
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,22 @@
[
{
"order": "a",
"md5sum": "6d1ca6e9533d177361fe2612a2c87474",
"name": "Gemma Instruct",
"filename": "gemma-7b-it.Q4_0.gguf",
"filesize": "4809316512",
"requires": "2.5.0",
"ramrequired": "8",
"parameters": "7 billion",
"quant": "q4_0",
"type": "Gemma",
"description": "<strong>A state-of-the-art open model from Google</strong><br><ul><li>Fast responses</li><li>Chat based model</li><li>Trained by Google</li><li>Licensed for commercial use</li><li>Gemma is provided under and subject to the Gemma Terms of Use found at <a href=\"https://ai.google.dev/gemma/terms\">ai.google.dev/gemma/terms</a></li></ul>",
"url": "https://gpt4all.io/models/gguf/gemma-7b-it.Q4_0.gguf",
"promptTemplate": "<start_of_turn>user\n%1<end_of_turn>\n<start_of_turn>model\n",
"systemPrompt": ""
},
{
"order": "b",
"md5sum": "48de9538c774188eb25a7e9ee024bbd3",
"name": "Mistral OpenOrca",
"filename": "mistral-7b-openorca.Q4_0.gguf",
Expand All @@ -15,22 +31,6 @@
"promptTemplate": "<|im_start|>user\n%1<|im_end|><|im_start|>assistant\n",
"systemPrompt": "<|im_start|>system\nYou are MistralOrca, a large language model trained by Alignment Lab AI. For multi-step problems, write out your reasoning for each step.\n<|im_end|>"
},
{
"order": "b",
"md5sum": "97463be739b50525df56d33b26b00852",
"name": "Mistral Instruct",
"filename": "mistral-7b-instruct-v0.1.Q4_0.gguf",
"filesize": "4108916384",
"requires": "2.5.0",
"ramrequired": "8",
"parameters": "7 billion",
"quant": "q4_0",
"type": "Mistral",
"systemPrompt": " ",
"description": "<strong>Best overall fast instruction following model</strong><br><ul><li>Fast responses</li><li>Trained by Mistral AI<li>Uncensored</li><li>Licensed for commercial use</li></ul>",
"url": "https://gpt4all.io/models/gguf/mistral-7b-instruct-v0.1.Q4_0.gguf",
"promptTemplate": "[INST] %1 [/INST]"
},
{
"order": "c",
"md5sum": "c4c78adf744d6a20f05c8751e3961b84",
Expand All @@ -47,6 +47,22 @@
"url": "https://gpt4all.io/models/gguf/gpt4all-falcon-newbpe-q4_0.gguf",
"promptTemplate": "### Instruction:\n%1\n### Response:\n"
},
{
"order": "d",
"md5sum": "97463be739b50525df56d33b26b00852",
"name": "Mistral Instruct",
"filename": "mistral-7b-instruct-v0.1.Q4_0.gguf",
"filesize": "4108916384",
"requires": "2.5.0",
"ramrequired": "8",
"parameters": "7 billion",
"quant": "q4_0",
"type": "Mistral",
"systemPrompt": " ",
"description": "<strong>Best overall fast instruction following model</strong><br><ul><li>Fast responses</li><li>Trained by Mistral AI<li>Uncensored</li><li>Licensed for commercial use</li></ul>",
"url": "https://gpt4all.io/models/gguf/mistral-7b-instruct-v0.1.Q4_0.gguf",
"promptTemplate": "[INST] %1 [/INST]"
},
{
"order": "e",
"md5sum": "00c8593ba57f5240f59662367b3ed4a5",
Expand Down