Skip to content

Limit file writer FDs on Windows #379

@jeroen

Description

@jeroen

With HTTP2 we can hit the max number of open files on Windows if we download many files in parallel :

https://github.com/jeroen/curl/actions/runs/12996010184/job/36243884844

Error in write_file_writer(fp, data, as.logical(close)) : 
  Failed to open file: C:\Users\RUNNER~1\AppData\Local\Temp\RtmpW6FvoY\working_dir\Rtmpm0jvhq\descriptions\astrochron.txt
Too many open files
Calls: <Anonymous> -> writer -> write_file_writer
Error in write_file_writer(fp, data, as.logical(close)) : 
  Failed to open file: C:\Users\RUNNER~1\AppData\Local\Temp\RtmpW6FvoY\working_dir\Rtmpm0jvhq\descriptions\astrodatR.txt
Too many open files
Calls: <Anonymous> -> writer -> write_file_writer
Error in write_file_writer(fp, data, as.logical(close)) : 
  Failed to open file: C:\Users\RUNNER~1\AppData\Local\Temp\RtmpW6FvoY\working_dir\Rtmpm0jvhq\descriptions\astroFns.txt
Too many open files

We should limit the overall in progress downloads in multi_download.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions