Currently the LLM Reranker LLM chosen is gpt-3.5-turbo. We propose adding additional (smaller) LLM's for testing LLM Reranking such as (from smallest to closest or better than gpt-3.5-turbo):
Considering the cost and time for inference when adding a re-ranker, smaller models performing as well might be valuable for cost/time savings. Adding Mixtral and Haiku in the mix because they always seem to outperform gpt for many things.
Forked. Will update with PR and repo details when complete.
Currently the LLM Reranker LLM chosen is
gpt-3.5-turbo. We propose adding additional (smaller) LLM's for testing LLM Reranking such as (from smallest to closest or better than gpt-3.5-turbo):Considering the cost and time for inference when adding a re-ranker, smaller models performing as well might be valuable for cost/time savings. Adding Mixtral and Haiku in the mix because they always seem to outperform gpt for many things.
Forked. Will update with PR and repo details when complete.