Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Add Flag Embedding LLM Reranker #297

Merged
merged 24 commits into from
Apr 11, 2024
Merged

Add Flag Embedding LLM Reranker #297

merged 24 commits into from
Apr 11, 2024

Conversation

bwook00
Copy link
Contributor

@bwook00 bwook00 commented Apr 10, 2024

close #295

While looking for Flag Embedding original code, I found the 'Flag LLM reranker' separately.

And i saw this table.
image

Flag Embedding llm based reranker's performance was great so I decide to add our module.

jeffrey and others added 22 commits April 10, 2024 02:08
make the first module of passage_filter, threshold_cutoff
1aaf20f
add at passage filter __init__.py
d3cb20c
return at least one
0b7020f
add docs annotation of similarity_threshold_cutoff
5bcaab0
original function test done
8ca2aa1
input embedding model name not the emodel
b09572f
passage filter node test
20f6e46
add test cases for that retrieved results counts are not consistent
191ae50
make passage filter run.py
4a3721f
make run.py for passage filter
f97f3cd
test new node passage filter
79d98c5
add passage filter node to full.yaml
7c464a8
add passage filter docs
fe51a21
add passage filter at index.md and README.md
f02c66c
add passage filter rst docs
4039299
Merge branch 'main' into Feature/#282
e74afa7
fix test code for fixed simple.yaml
a411be7
Merge branch 'main' into Feature/#282
b21d2cd
@bwook00
Merge remote-tracking branch 'origin/Feature/#282' into Feature/#295
6c06db6 
# Conflicts:
#	README.md
@bwook00
Implement flag llm reranker
9629d60
@bwook00
Add Flag LLM Reranker docs
5f1a9b7
@bwook00
Edit function name
2f59a1f
@Eastsidegunn
Copy link
Contributor

Eastsidegunn commented Apr 10, 2024 •
edited
Loading

Commit copy bug LOL

@bwook00
Merge branch 'main' into Feature/#295
cbd6a2f 
# Conflicts:
#	autorag/support.py
#	docs/source/nodes/index.md
#	sample_config/full.yaml
@bwook00 bwook00 enabled auto-merge April 11, 2024 11:55
Eastsidegunn
Eastsidegunn previously approved these changes Apr 11, 2024
View reviewed changes
vkehfdl1
vkehfdl1 approved these changes Apr 11, 2024
View reviewed changes
Copy link
Contributor

@vkehfdl1 vkehfdl1 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM from now. Later, you need to fix codebase for parallel processing.
Look at #303

@bwook00 bwook00 merged commit 50c2053 into main Apr 11, 2024
3 checks passed
@bwook00 bwook00 deleted the Feature/#295 branch April 11, 2024 14:06
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@vkehfdl1 vkehfdl1 vkehfdl1 approved these changes

@Eastsidegunn Eastsidegunn Eastsidegunn approved these changes

Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

Add Flag LLM reranker
3 participants
@bwook00 @Eastsidegunn @vkehfdl1