Skip to content

download-ignore-retry-after-header #44

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 1 commit into
base: main
Choose a base branch
from
Open

download-ignore-retry-after-header #44

wants to merge 1 commit into from

Conversation

amith-ananthram
Copy link

@amith-ananthram amith-ananthram commented Jul 31, 2025 •
edited
Loading

  • pass through kwargs that set timeout in the function call
  • ignore retry after header as some URLs set them arbitrarily far in the future, causing the download script to hang eg:
(base) amith@Amiths-MacBook-Pro-4 metrics % curl -I https://www.avialogs.com/media/k2/items/cache/b53ad912c7950b115492257a5e8e3636_M.jpg
HTTP/1.1 503 Service Unavailable
Date: Thu, 31 Jul 2025 23:02:44 GMT
Server: Apache/2.4.58 (Ubuntu)
X-Content-Type-Options: nosniff
X-Frame-Options: DENY
X-XSS-Protection: 1; mode=block
Referrer-Policy: strict-origin-when-cross-origin
**Retry-After: Sat, 10 Aug 2025 00:00:00 GMT**
Last-Modified: Tue, 24 Jun 2025 15:42:32 GMT
ETag: "3f2b-638532d34c015"
Accept-Ranges: bytes
Content-Length: 16171
Connection: close
Content-Type: text/html

@amith-ananthram amith-ananthram mentioned this pull request Jul 31, 2025
Open
@amith-ananthram
Copy link
Author

@chrisc36 not sure how to request a review but wanted to share this fix; happy to iterate on it if you'd like more sophisticated logic too

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
1 participant
@amith-ananthram