Skip to content

Make LERF compatible with DINOv2 #9

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
chungmin99 wants to merge 3 commits into
base: main
Choose a base branch
from
Open

Make LERF compatible with DINOv2 #9

chungmin99 wants to merge 3 commits into from

Conversation

@chungmin99
Copy link
Collaborator

@chungmin99 chungmin99 commented Apr 20, 2023

Added capability to use DINOv2 models

(Using implementation/torchhub as described by https://github.com/facebookresearch/dinov2)

TODOs:

  • Compare/contrast DINO vs. DINOv2 features
  • Add DINO feature visualization(?) -- for the compare/contrast. Initial implementation cause the colors to be inconsistent across renders, so it feels confusing/disorienting.
  • Add compatibility for ViT-B/L/G DINO versions (currently dino-field embedding out hardcoded to 384)

@chungmin99 chungmin99 requested a review from kerrj April 20, 2023 23:18
@linukc
Copy link

linukc commented Jun 5, 2023 •
edited
Loading

i suggest to move KMeans and torch.pca_low_rank to the get_outputs_for_camera_ray_bundle function (aggregate all get_outputs from one image there and after perform dim reduction) to prevent different color palitre on a one image

@anas-zafar
Copy link

anas-zafar commented Oct 19, 2023

@chungmin99 , were you able to fully integrate Dinov2 with LERF?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@kerrj kerrj Awaiting requested review from kerrj

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@chungmin99 @linukc @anas-zafar