Is your feature request related to a problem? Please describe.
A clear and concise description of what the problem is. Ex. I'm always frustrated when [...]

See NixOS/nixpkgs#128105

Before we invent any hacks around upstream, it'd be great if the backport action could just delete merged branches on its own.