Repository navigation
Failures in /api/v2/bulk_download/awards #4283
Description
I'm noticing frequent failures with the bulk_download endpoint when requesting ten days of data. Here's an example of a request I'm issuing (this is the response from the API, which shows the parameters):
{
"status_url":"https://api.usaspending.gov/api/v2/download/status?file_name=All_SubawardsAndPrimeTransactions_2025-02-18_H05M23S53477012.zip",
"file_name":"All_SubawardsAndPrimeTransactions_2025-02-18_H05M23S53477012.zip",
"file_url":"https://files.usaspending.gov/generated_downloads/All_SubawardsAndPrimeTransactions_2025-02-18_H05M23S53477012.zip",
"download_request":{
"download_types":[
"elasticsearch_sub_awards",
"prime_awards"
],
"file_format":"csv",
"filters":{
"prime_and_sub_award_types":{
"elasticsearch_sub_awards":[
"grant",
"procurement"
],
"prime_awards":[
"-1",
"02",
"03",
"04",
"05",
"06",
"07",
"08",
"09",
"10",
"11",
"A",
"B",
"C",
"D",
"IDV_A",
"IDV_B",
"IDV_B_A",
"IDV_B_B",
"IDV_B_C",
"IDV_C",
"IDV_D",
"IDV_E"
]
},
"time_period":[
{
"date_type":"last_modified_date",
"end_date":"2025-02-07",
"start_date":"2025-01-28"
}
]
},
"request_type":"award"
}
}Here's an example of a failed request that was made using the same parameters as above, as seen from the status URL:
{
"status": "failed",
"message": "An exception was raised while attempting to process the DownloadJob:\nTraceback (most recent call last):\n File \"/data-act/backend/usaspending_api/download/filestreaming/download_generation.py\", line 99, in generate_download\n parse_source(\n File \"/data-act/backend/usaspending_api/download/filestreaming/download_generation.py\", line 441, in parse_source\n raise e\n File \"/data-act/backend/usaspending_api/download/filestreaming/download_generation.py\", line 416, in parse_source\n wait_for_process(psql_process, start_time, download_job)\n File \"/data-act/backend/usaspending_api/download/filestreaming/download_generation.py\", line 550, in wait_for_process\n raise e\nException: Command failed. Please see the logs for details.\n",
"file_name": "All_SubawardsAndPrimeTransactions_2025-02-17_H17M45S17255664.zip",
"file_url": "https://files.usaspending.gov/generated_downloads/All_SubawardsAndPrimeTransactions_2025-02-17_H17M45S17255664.zip",
"total_size": null,
"total_columns": 118,
"total_rows": 0,
"seconds_elapsed": "26992.228396"
}Any idea what's going on? Let me know if there's any other information I can provide. Thank you!
Activity