Conversation
| try | ||
| Files.copy(builtZip, staging, StandardCopyOption.REPLACE_EXISTING) | ||
| try Retry(moveIntoPlace()) | ||
| catch | ||
| // Windows refuses to replace a file that another process holds open. Under | ||
| // parallel task execution a concurrent install of the same directory may | ||
| // still be reading `destZip` after the retries are exhausted; accept its | ||
| // zip in that case. Callers derive digests from the file on disk, so the | ||
| // surviving install determines the recorded content either way. | ||
| case _: AccessDeniedException if Files.exists(destZip) => () |
There was a problem hiding this comment.
Thanks for looking into this. The implementation looks equivalent to IO.writeFileAtomically, so I suggest using IO.copyFile.
There was a problem hiding this comment.
A similar issue has been affecting me badly because, on one side, metals is not reliable and I can't use it or MCP, and, on the other side, sbt keeps locking files impacting manual execution and AI. The problem I have, though, is not just about parallel execution.
I've been working on a fix too, but my fix is on the read side. Let me publish my WIP solution, once I solve my env problems
There was a problem hiding this comment.
Here it is #9627 It's still a draft as I haven't done all the checks I'd like to do and I haven't validated against the project's guidelines.
I hope this will start a good conversation around this problem
There was a problem hiding this comment.
Good call, switched to IO.copyFile in 758c31e. The only thing kept in ActionCache is the narrow AccessDeniedException fallback: with tight-loop readers on Windows the destination can still be held open after IO.copyFile's retries are exhausted, and in that case the concurrently installed zip wins.
On Windows,
Files.movecannot replace a file that another process holds open. Under parallel task execution two tasks can install the sameclasses.sbtdir.zipconcurrently while each also reads the zip it just installed (digest, CAS copy, extraction), soinstallPackagedZipfails withAccessDeniedException. Seen in the wild as recurring sbt/zinc CI failures on(compilerInterface / Compile / compileIncremental), triggered by the duplicate compilation in #9618.The zip is now installed via
IO.copyFile, which stages next to the destination and retries the atomic rename; if the destination still cannot be replaced but exists, the concurrently installed zip wins. Callers derive digests from the file on disk, so the surviving zip determines the recorded content either way. The new test races installs against readers of the installed zip and reproduces the failure on Windows without this change.Investigated with Claude Code, with human review in the loop.