Skip to content

downloads a small subset of files from DR1 (iron)#41

Merged
sbailey merged 11 commits into
mainfrom
tiny_dr1
Mar 5, 2025
Merged

downloads a small subset of files from DR1 (iron)#41
sbailey merged 11 commits into
mainfrom
tiny_dr1

Conversation

@forero

@forero forero commented Feb 17, 2025

Copy link
Copy Markdown
Member

I am adding a script that

  • finds targets near a specific point in the sky in the zall-pix iron catalog.
  • finds which helpix has the most targets around that location and which tile has the larges overlap with that healpix.
  • It downloads the corresponding healpix directory and the tile directory.

By default, the script looks around the COSMOS location RA=150.12, Dec=2.21 within a radius of 0.5dec, but users can change these coordinates using command-line arguments.

@forero forero requested a review from sbailey February 17, 2025 20:15
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
Comment thread bin/desi_download_dr1 Outdated
@forero

forero commented Mar 5, 2025

Copy link
Copy Markdown
Member Author

@sbailey I think this is ready to review. I reverted to the old default location for DR1 to be consistent with the tutorials. I also added the --no-tiles option. I also added support to download EDR data (default location is rosette 1 for dark targets).

@forero

forero commented Mar 5, 2025

Copy link
Copy Markdown
Member Author

Using the default settings, tiny_edr, tiny_dr1 and tiny_dr2 weigh 102GB, 40GB and 91GB, respectively (using the --no-tiles option they only weigh 9.2GB, 23GB and 51GB)

@sbailey

sbailey commented Mar 5, 2025

Copy link
Copy Markdown
Contributor

Thanks for the update and for including the sizes. This highlights that the selected defaults are optimized for DR1, but are probably too big for both EDR and DR2 due to the increased tile density. I think that is good enough for now, but we'll need to revisit how we provide a minimal subset for DR2.

A breakdown for tiny_dr1:

21G	tiny_dr1/spectro/redux/iron/zcatalog/  (with only a single catalog!)
1.6G	tiny_dr1/spectro/redux/iron/healpix/
18G	tiny_dr1/spectro/redux/iron/tiles      (6.6G if excluding uncoadded spectra*.fits.gz and rrdetails*.h5 files)

@sbailey sbailey merged commit 2edc4f4 into main Mar 5, 2025
@sbailey sbailey deleted the tiny_dr1 branch March 5, 2025 17:39
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants