Add rate limiting to search and download requests #1483
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
alexlebens
pushed a commit
to alexlebens/infrastructure
that referenced
this pull request
Nov 6, 2025
This PR contains the following updates: | Package | Update | Change | |---|---|---| | [slskd/slskd](https://slskd.org) ([source](https://github.com/slskd/slskd)) | minor | `0.23.2` -> `0.24.0` | --- ### Release Notes <details> <summary>slskd/slskd (slskd/slskd)</summary> ### [`v0.24.0`](https://github.com/slskd/slskd/releases/tag/0.24.0) [Compare Source](slskd/slskd@0.23.2...0.24.0) ### 🔧 Transfer Overhaul This release incorporates a ton of bug fixes and refactoring in both slskd and the underlying [Soulseek.NET](https://github.com/jpdillingham/Soulseek.NET) library with the goal of resolving 'stuck' and failing transfers (both uploads and downloads). #### What's Changed - fix(docs): add missing prefixes to some environment variables by [@​RalphORama](https://github.com/RalphORama) in [#​1420](slskd/slskd#1420) - Add telemetry module to make Prometheus metrics available in the UI by [@​jpdillingham](https://github.com/jpdillingham) in [#​1410](slskd/slskd#1410) - Fix debug log that displays information about migrations at startup by [@​jpdillingham](https://github.com/jpdillingham) in [#​1421](slskd/slskd#1421) - Refactor initialization of database connection strings by [@​jpdillingham](https://github.com/jpdillingham) in [#​1422](slskd/slskd#1422) - Eliminate 'magic strings' when referencing databases and connection strings by [@​jpdillingham](https://github.com/jpdillingham) in [#​1423](slskd/slskd#1423) - Bump Soulseek.NET to 8.0.0 by [@​jpdillingham](https://github.com/jpdillingham) in [#​1438](slskd/slskd#1438) - Fix bug causing 'Value cannot be null. (Parameter 'instance')' while editing a commented-out config by [@​jpdillingham](https://github.com/jpdillingham) in [#​1441](slskd/slskd#1441) - Bump form-data in /src/web by [@​dependabot](https://github.com/dependabot)\[bot] in [#​1439](slskd/slskd#1439) - Bump ws in /src/web by [@​dependabot](https://github.com/dependabot)\[bot] in [#​1440](slskd/slskd#1440) - Tweak how file access is handled when validating config by [@​jpdillingham](https://github.com/jpdillingham) in [#​1449](slskd/slskd#1449) - Refactor upload logic, splitting file enqueue and upload and adding defensive checks throughout by [@​jpdillingham](https://github.com/jpdillingham) in [#​1456](slskd/slskd#1456) - Refactor transfer progress update logic by [@​jpdillingham](https://github.com/jpdillingham) in [#​1457](slskd/slskd#1457) - Support the ability for the web interface to listen on a unix socket by [@​encode42](https://github.com/encode42) in [#​1453](slskd/slskd#1453) - Refine support for unix domain sockets (UDS) by [@​jpdillingham](https://github.com/jpdillingham) in [#​1458](slskd/slskd#1458) - Refactor download logic to split enqueue and download by [@​jpdillingham](https://github.com/jpdillingham) in [#​1462](slskd/slskd#1462) - Improve logging of DownloadDenied and DownloadFailed events by [@​jpdillingham](https://github.com/jpdillingham) in [#​1463](slskd/slskd#1463) - Bump Soulseek.NET to 8.1.2 by [@​jpdillingham](https://github.com/jpdillingham) in [#​1464](slskd/slskd#1464) - Enqueue files one at a time by [@​jpdillingham](https://github.com/jpdillingham) in [#​1465](slskd/slskd#1465) - Wait for transfer response from remote client before enqueuing the next file by [@​jpdillingham](https://github.com/jpdillingham) in [#​1467](slskd/slskd#1467) - Implement a number of improvements and bug fixes with download logic by [@​jpdillingham](https://github.com/jpdillingham) in [#​1472](slskd/slskd#1472) - Add dedicated transfer connection timeout, set server, distributed and transfer connection timeouts explicitly. Bump Soulseek.NET to 8.4.0 by [@​jpdillingham](https://github.com/jpdillingham) in [#​1474](slskd/slskd#1474) - Skip transfer progress updates if a previous update is still in progress by [@​jpdillingham](https://github.com/jpdillingham) in [#​1479](slskd/slskd#1479) - Limit concurrent enqueue requests to 20 by [@​jpdillingham](https://github.com/jpdillingham) in [#​1480](slskd/slskd#1480) - Set concurrent search limit to 2 (from the default 10) by [@​jpdillingham](https://github.com/jpdillingham) in [#​1481](slskd/slskd#1481) - Set Unix file mode after moving downloaded files by [@​jpdillingham](https://github.com/jpdillingham) in [#​1482](slskd/slskd#1482) - Add rate limiting to search and download requests by [@​jpdillingham](https://github.com/jpdillingham) in [#​1483](slskd/slskd#1483) - Add more logging to download logic by [@​jpdillingham](https://github.com/jpdillingham) in [#​1484](slskd/slskd#1484) - Fix bug with Prometheus threadpool statistics by [@​jpdillingham](https://github.com/jpdillingham) in [#​1485](slskd/slskd#1485) - Prime message connection before enqueueing downloads locally by [@​jpdillingham](https://github.com/jpdillingham) in [#​1486](slskd/slskd#1486) - Temporarily disable unobserved exception logging by [@​jpdillingham](https://github.com/jpdillingham) in [#​1487](slskd/slskd#1487) - Refactor download logic to improve stability and improve performance by [@​jpdillingham](https://github.com/jpdillingham) in [#​1491](slskd/slskd#1491) - Bump Soulseek.NET to 8.5.0 by [@​jpdillingham](https://github.com/jpdillingham) in [#​1497](slskd/slskd#1497) - Add missing filename filter when superseding download records by [@​jpdillingham](https://github.com/jpdillingham) in [#​1500](slskd/slskd#1500) - Make check of index names in transfer indexes migration case insensitive by [@​jpdillingham](https://github.com/jpdillingham) in [#​1501](slskd/slskd#1501) - Increase concurrent download requests to 2, return enqueue results by [@​jpdillingham](https://github.com/jpdillingham) in [#​1504](slskd/slskd#1504) #### New Contributors - [@​RalphORama](https://github.com/RalphORama) made their first contribution in [#​1420](slskd/slskd#1420) - [@​encode42](https://github.com/encode42) made their first contribution in [#​1453](slskd/slskd#1453) **Full Changelog**: <slskd/slskd@0.23.2...0.24.0> </details> --- ### Configuration 📅 **Schedule**: Branch creation - At any time (no schedule defined), Automerge - At any time (no schedule defined). 🚦 **Automerge**: Disabled by config. Please merge this manually once you are satisfied. ♻ **Rebasing**: Whenever PR is behind base branch, or you tick the rebase/retry checkbox. 🔕 **Ignore**: Close this PR and you won't be reminded about this update again. --- - [ ] <!-- rebase-check -->If you want to rebase/retry this PR, check this box --- This PR has been generated by [Renovate Bot](https://github.com/renovatebot/renovate). <!--renovate-debug:eyJjcmVhdGVkSW5WZXIiOiI0MS4xNzEuMiIsInVwZGF0ZWRJblZlciI6IjQxLjE3MS4yIiwidGFyZ2V0QnJhbmNoIjoibWFpbiIsImxhYmVscyI6WyJpbWFnZSJdfQ==--> Reviewed-on: https://gitea.alexlebens.dev/alexlebens/infrastructure/pulls/1965 Co-authored-by: Renovate Bot <renovate-bot@alexlebens.net> Co-committed-by: Renovate Bot <renovate-bot@alexlebens.net>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
I'm putting a lot of effort into avoiding duplicate, concurrent operations like searches and batches of downloads. Download initiation is fairly chatty with the database as well, and I'm concerned about concurrent requests (almost certainly originating from other applications, not users) creating excess resource contention and causing things to malfunction more than they should.
This PR adds rate limiting to the creation of searches (searches can run 2 at a time and are queued and processed in order after that), and the enqueue of batches of downloads (downloads are queued locally and run asynchronously in the background)
Any API calls made to
POST /searchesorPOST /transfers/downloads/<username>while a previous request is running will return an HTTP status code of 429.There's a chance this will break external applications if they aren't already throttling requests to slskd, and if that's the case they will need to be updated to ensure they call these endpoints serially.