Repository navigation
llama : add Deepseek support #5981
Description
Support is almost complete. There is a dangling issue with the pre-tokenizer: #7036
A useful discussion related to that is here: #7144
Outdated below
Creating this issue for more visibility
The main problem is around tokenization support, since the models use some variation of the BPE pre-processing regex. There are also some issues with the conversion scripts.
Anyway, looking for contributions to help with this
Previous unfinished work:
Possible implementation plan: #5464 (comment)
Activity
dragnil1 commentedon Mar 13, 2024
Hello, @ggerganov, I'd like to try working on it as my good first issue.
ggerganov commentedon Mar 13, 2024
Ok 👍 keep us posted
Kangmo commentedon Mar 22, 2024
Waiting for this to come. deepseek model is famouse for coding well and Korean language ability.
fostiropoulos commentedon Mar 24, 2024
In the supported models there is Deepseek. I am able to use DeepSeekCoder-33B with comparatively similar results to their API. Can someone please clarify what are the failure cases with the current tokenization?
Columpio commentedon Apr 1, 2024
@fostiropoulos There may be a problem with deepseek coder 1.3b, which might be somehow irreprodicible in 6.7b, 7b, and 33b.
See this
mirek190 commentedon Apr 12, 2024
any progress with Deepseek support?
hyperbolic-c commentedon May 8, 2024
Hey, so it could support DeepSeek-coder model after #6920 be merged? Thanks !
ggerganov commentedon May 8, 2024
I've updated the description of the issue with the latest state.
Support is pretty much complete, though there are some edge cases during tokenization that are handled incorrectly. For example the letter
üis tokenized to a different token than it should - more info in the referenced linksI think we can declare DeepSeek models supported and handle the problem above into a separate task related to correct processing of added tokens, since it is not specific for DeepSeek models