Skip to content

Feature request: builtins.fetchGitArchive #8134

Description

@bergkvist

Is your feature request related to a problem? Please describe.

# Cloning a large repository is incredibly slow
builtins.fetchGit {
  url = "git@github.com:bergkvist/private-repo";
  ref = "refs/tags/v1.2.3";
}

When working with private git repos that need to be accessed over ssh, builtins.fetchGit is currently the most convenient alternative since it uses local git config - however it is incredibly slow. On a private repo I'm working with that is ~1GB in size, it takes more than an hour to run, and doesn't display any progress like git clone does. On the other hand, fetchTarball is significantly faster, but only works over http/doesn't work through ssh/git with local config/yubikey-setup.

Describe the solution you'd like

Git supports an archive command, which is ~100x faster than builtins.fetchGit in my case (20s vs 1h). A downside is that this doesn't work with submodules.

# Example usage
git archive --format=tar.gz --remote=git@github.com:bergkvist/private-repo v1.2.3 > private-repo-v1.2.3.tar.gz
builtins.fetchGitArchive {
  remote = "git@github.com:bergkvist/private-repo";
  format = "tar.gz";
  version = "v1.2.3";
  sha256 = "..."; # if not included, use the tarball-ttl
}

Activity

  1. thufschmitt commented on Mar 31, 2023

    @thufschmitt
    Member

    That looks pretty cool when supported!

    Do you know which hosting providers do though? It looks like github doesn't at least

  2. bergkvist commented on Mar 31, 2023

    @bergkvist
    MemberAuthor

    Oh, I didn't realize github doesn't support this. gitlab (at least the self-hosted version) does though.

  3. SuperSandro2000 commented on Mar 31, 2023

    @SuperSandro2000
    Member

    Why can't you use pkgs.fetchFromGitHub?

  4. added
    fetchingNetworking with the outside (non-Nix) world, input locking
    on Apr 1, 2023
  5. bergkvist commented on Apr 2, 2023

    @bergkvist
    MemberAuthor

    @SuperSandro2000 I'm using GitLab + private repos, and I want to fetch over ssh using my local ssh-config. (this allows for using a yubikey)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    featureFeature request or proposalfetchingNetworking with the outside (non-Nix) world, input locking

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions