fix: always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations #3854

johanneskoester · 2025-12-01T09:35:30Z

QC

The PR contains a test case for the changes or the changes are already covered by an existing test case.
The documentation (docs/) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).

Summary by CodeRabbit

Refactor
- Reworked file-size logic into a predicate-driven helper so regular and temporary file-size queries delegate to a common implementation.
Bug Fixes
- Broadened recognition of missing-file errors and treat them as not-found; affected resource evaluations are skipped instead of failing.
Type Annotations
- Clarified file-size return-type annotations to be integer-valued for improved correctness and tooling.

_{✏️ Tip: You can customize this high-level summary in your review settings.}

…rce expressions that do size evaluations

coderabbitai · 2025-12-01T09:35:42Z

📝 Walkthrough

Walkthrough

Added an async predicate helper to compute InputFiles sizes and refactored size_files/size_tempfiles to use it; annotated _IOFile.size as async returning int; changed local size handling to raise storage-plugin FileOrDirectoryNotFoundError; and treated that exception as file-not-found in exceptions and resources logic.

Changes

Cohort / File(s)	Summary
IO sizing and predicates `src/snakemake/io/__init__.py`	Added import of `FileOrDirectoryNotFoundError`; changed `_IOFile.size` signature to `async def size(self) -> int`; simplified storage size path to return `storage_object.managed_size()` directly; in `size_local()` now raises `FileOrDirectoryNotFoundError(Path(self.file))` on missing files; added `InputFiles._predicated_size_files(self, predicate: Callable[[_IOFile], Awaitable[bool]]) -> List[int]` which applies an async predicate to each `_IOFile`, gathers per-file `f.size()` results (via `async_run`), filters out `None` sizes; refactored `InputFiles.size_files` and `InputFiles.size_tempfiles` to delegate to `_predicated_size_files` with appropriate predicates (always-true and `is_flagged(...,"temp")`).
File-not-found detection `src/snakemake/exceptions.py`	Imported `FileOrDirectoryNotFoundError` from `snakemake_interface_storage_plugins.exceptions` and updated `is_file_not_found_error` to treat `FileOrDirectoryNotFoundError` as a file-not-found condition alongside existing checks for `FileNotFoundError` and `WorkflowError` messages.
Resource evaluation error handling `src/snakemake/resources.py`	Imported `FileOrDirectoryNotFoundError` and extended `GroupResources.basic_layered` exception handling to catch `(FileNotFoundError, FileOrDirectoryNotFoundError)` when evaluating per-job resources, causing those jobs to be skipped instead of failing.

Estimated code review effort

🎯 3 (Moderate) | ⏱️ ~25 minutes

Verify async/await usage in _predicated_size_files and that async_run is used correctly for awaiting per-file size() calls.
Confirm _IOFile.size type annotation and that callers handle the async signature.
Check that raising FileOrDirectoryNotFoundError(Path(self.file)) in size_local() preserves prior semantics where appropriate.
Validate is_file_not_found_error changes to avoid false positives and ensure GroupResources.basic_layered skipping behavior remains correct.

Pre-merge checks and finishing touches

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 13.33% which is insufficient. The required threshold is 80.00%.	You can run `@coderabbitai generate docstrings` to improve docstring coverage.

✅ Passed checks (2 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title accurately summarizes the main change: ensuring FileNotFoundError is consistently raised for missing files in resource expressions during size evaluations.
Description check	✅ Passed	The description follows the template with both QC checklist items completed, confirming test coverage and documentation updates.

✨ Finishing touches

📝 Generate docstrings

🧪 Generate unit tests (beta)

Create PR with unit tests
Post copyable unit tests in a comment
Commit unit tests in branch fix/size-eval

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

_{Comment @coderabbitai help to get the list of available commands and usage tips.}

coderabbitai

Actionable comments posted: 1

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 12ce6c6 and bbcf433.

📒 Files selected for processing (1)

src/snakemake/io/__init__.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

src/snakemake/io/__init__.py

🧬 Code graph analysis (1)

src/snakemake/io/__init__.py (2)

src/snakemake/scheduling/job_scheduler.py (1)

get_size (199-200)

src/snakemake/common/__init__.py (1)

async_run (96-113)

🪛 Ruff (0.14.6)

src/snakemake/io/__init__.py

1958-1958: Unused function argument: iofile

(ARG001)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (46)

GitHub Check: tests (10, windows-2022, py313)
GitHub Check: tests (10, ubuntu-latest, py312)
GitHub Check: tests (9, ubuntu-latest, py311)
GitHub Check: tests (10, ubuntu-latest, py313)
GitHub Check: tests (10, ubuntu-latest, py311)
GitHub Check: tests (9, ubuntu-latest, py313)
GitHub Check: tests (7, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py313)
GitHub Check: tests (9, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py311)
GitHub Check: tests (9, ubuntu-latest, py312)
GitHub Check: tests (8, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py312)
GitHub Check: tests (6, ubuntu-latest, py312)
GitHub Check: tests (7, ubuntu-latest, py313)
GitHub Check: tests (5, macos-latest, py313)
GitHub Check: tests (6, windows-2022, py313)
GitHub Check: tests (5, ubuntu-latest, py311)
GitHub Check: tests (6, macos-latest, py313)
GitHub Check: tests (7, ubuntu-latest, py312)
GitHub Check: tests (6, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py311)
GitHub Check: tests (6, ubuntu-latest, py313)
GitHub Check: tests (5, windows-2022, py313)
GitHub Check: tests (4, macos-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py312)
GitHub Check: tests (4, ubuntu-latest, py312)
GitHub Check: tests (4, ubuntu-latest, py313)
GitHub Check: tests (4, windows-2022, py313)
GitHub Check: tests (2, windows-2022, py313)
GitHub Check: tests (3, windows-2022, py313)
GitHub Check: tests (3, ubuntu-latest, py313)
GitHub Check: tests (4, ubuntu-latest, py311)
GitHub Check: tests (3, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py311)
GitHub Check: tests (2, macos-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py312)
GitHub Check: tests (1, macos-latest, py313)
GitHub Check: tests (2, ubuntu-latest, py312)
GitHub Check: tests (1, windows-2022, py313)
GitHub Check: tests (2, ubuntu-latest, py311)
GitHub Check: tests (1, ubuntu-latest, py311)
GitHub Check: tests (1, ubuntu-latest, py313)
GitHub Check: apidocs

🔇 Additional comments (3)

src/snakemake/io/__init__.py (3)

37-37: LGTM!

The Awaitable import is correctly added to support the type annotation in the new _predicated_size_files method.

1956-1961: Refactoring approach is sound.

The delegation to _predicated_size_files with an always-true predicate correctly preserves the original behavior while using the new unified implementation.

Note: The static analysis warning about the unused iofile parameter in func_true is a false positive—the parameter is required to match the predicate signature expected by _predicated_size_files.

1963-1968: LGTM!

The refactoring correctly uses _predicated_size_files with a predicate that filters for temporary files, maintaining the expected behavior while leveraging the new unified implementation.

src/snakemake/io/__init__.py

coderabbitai

Actionable comments posted: 0

♻️ Duplicate comments (1)

src/snakemake/io/__init__.py (1)
1930-1955: _predicated_size_files still drops computed sizes and has a mismatched return type

The core issues in this helper are only partially fixed and will currently break all size-based resource computations:

get_size never returns the computed size.

Lines 1937–1941: size = await f.size() is computed and asserted, but there is no return size.

As written, for existing files that pass the predicate, get_size always falls through to return None.

As a result, sizes = await asyncio.gather(...) will contain only None (for existing files) and _IOFile instances (for missing files). After filtering with [res for res in sizes if res is not None], you end up with an empty list whenever all files exist, so size_files, size_files_kb, etc. all collapse to zero. This is a runtime correctness bug.

Return annotation is inconsistent with actual behavior.

_predicated_size_files is annotated as -> List[_IOFile], but the helper is conceptually returning a list of sizes (ints) and is consumed as such by size_files and the derived properties (size_files_kb, size, etc.).

After the missing-file sentinel is checked and FileNotFoundError is raised for any string-like result, the remaining values are sizes, not _IOFile objects. The annotation should reflect that (e.g. List[int]).

Callable type parameterization is non-idiomatic and may confuse type checkers.

For Callable, the first type argument should be a list of argument types. Using Callable[_IOFile, Awaitable[bool]] is unconventional; Callable[[_IOFile], Awaitable[bool]] better matches standard typing usage and tools’ expectations.

These issues overlap with the earlier bot review that flagged missing returns and the incorrect return annotation, so this is effectively a refinement / continuation of that comment.

A minimal, focused fix could look like this:
-    def _predicated_size_files(
-        self, predicate: Callable[_IOFile, Awaitable[bool]]
-    ) -> List[_IOFile]:
+    def _predicated_size_files(
+        self, predicate: Callable[[_IOFile], Awaitable[bool]]
+    ) -> List[int]:
         async def sizes():
-            async def get_size(f: _IOFile) -> Optional[Union[int, str]]:
+            async def get_size(f: _IOFile) -> Optional[Union[int, str]]:
                 if await predicate(f):
                     if await f.exists():
                         size = await f.size()
                         assert (
                             isinstance(size, int) and size >= 0
                         ), f"Invalid size for file {f}: {size}"
+                        return size
                     else:
                         return f
                 return None
@@
-            return [res for res in sizes if res is not None]
+            return [res for res in sizes if res is not None]
This preserves your missing-file behavior (raising FileNotFoundError when any path is not present) while ensuring successful size computations actually propagate out to size_files and the derived properties.

For the Callable annotation, please double-check against the Python version and typing docs you target, to ensure the suggested shape is acceptable to both the interpreter and your type checker.

To verify the behavior end-to-end, you might also want to add/extend tests asserting that:

InputFiles.size_files returns the actual byte sizes when all inputs exist.

A missing file in InputFiles causes a FileNotFoundError when accessing size_files or size.

size_tempfiles behaves analogously but restricted to temp-flagged entries.

🧹 Nitpick comments (1)

src/snakemake/io/__init__.py (1)
1957-1963: size_files predicate: unused parameter and alignment with Ruff ARG001

The implementation correctly delegates to _predicated_size_files with a predicate that always passes, but Ruff is right that the iofile argument in func_true is unused.

You can silence the warning and make the intent clearer by marking the parameter as intentionally unused:
     @property
     def size_files(self):
-        async def func_true(iofile):
+        async def func_true(_iofile):
             return True
 
         return self._predicated_size_files(func_true)
This keeps behavior identical while satisfying the static analysis hint.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between bbcf433 and a822209.

📒 Files selected for processing (1)

src/snakemake/io/__init__.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

src/snakemake/io/__init__.py

🧬 Code graph analysis (1)

src/snakemake/io/__init__.py (2)

src/snakemake/scheduling/job_scheduler.py (1)

get_size (199-200)

src/snakemake/common/__init__.py (2)

async_run (96-113)

func_true (51-52)

🪛 Ruff (0.14.6)

src/snakemake/io/__init__.py

1959-1959: Unused function argument: iofile

(ARG001)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (45)

GitHub Check: tests (6, ubuntu-latest, py312)
GitHub Check: tests (10, ubuntu-latest, py312)
GitHub Check: tests (10, windows-2022, py313)
GitHub Check: tests (10, ubuntu-latest, py313)
GitHub Check: tests (8, ubuntu-latest, py313)
GitHub Check: tests (9, ubuntu-latest, py312)
GitHub Check: tests (10, ubuntu-latest, py311)
GitHub Check: tests (9, ubuntu-latest, py313)
GitHub Check: tests (9, windows-2022, py313)
GitHub Check: tests (9, ubuntu-latest, py311)
GitHub Check: tests (8, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py313)
GitHub Check: tests (5, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py312)
GitHub Check: tests (7, ubuntu-latest, py312)
GitHub Check: tests (5, ubuntu-latest, py312)
GitHub Check: tests (7, windows-2022, py313)
GitHub Check: tests (7, ubuntu-latest, py311)
GitHub Check: tests (5, macos-latest, py313)
GitHub Check: tests (6, ubuntu-latest, py313)
GitHub Check: tests (6, ubuntu-latest, py311)
GitHub Check: tests (5, ubuntu-latest, py313)
GitHub Check: tests (6, windows-2022, py313)
GitHub Check: tests (5, ubuntu-latest, py311)
GitHub Check: tests (4, windows-2022, py313)
GitHub Check: tests (4, ubuntu-latest, py312)
GitHub Check: tests (3, windows-2022, py313)
GitHub Check: tests (3, ubuntu-latest, py313)
GitHub Check: tests (4, macos-latest, py313)
GitHub Check: tests (3, macos-latest, py313)
GitHub Check: tests (4, ubuntu-latest, py311)
GitHub Check: tests (4, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py312)
GitHub Check: tests (3, ubuntu-latest, py311)
GitHub Check: tests (2, macos-latest, py313)
GitHub Check: tests (2, ubuntu-latest, py313)
GitHub Check: tests (2, windows-2022, py313)
GitHub Check: tests (2, ubuntu-latest, py312)
GitHub Check: tests (1, windows-2022, py313)
GitHub Check: tests (1, macos-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py311)
GitHub Check: tests (1, ubuntu-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py311)

🔇 Additional comments (2)

src/snakemake/io/__init__.py (2)

37-37: Awaitable import is appropriate for new async predicate signature

Importing Awaitable from collections.abc here is consistent with the new async predicate typing in _predicated_size_files and looks correct.

1965-1969: size_tempfiles delegation looks correct

The new size_tempfiles implementation cleanly reuses _predicated_size_files with a temp-flag-based predicate and will raise FileNotFoundError when any temp file is missing, which aligns with the PR objective once _predicated_size_files is fixed as above.

coderabbitai

Actionable comments posted: 0

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)

src/snakemake/io/__init__.py (1)
1927-1983: Potential unit mismatch between size_tempfiles and temp_size_mb

_predicated_size_files() returns the raw results of f.size(), i.e. sizes in bytes. With the new implementation:

size_tempfiles now returns a List[int] of byte sizes for temp files.

temp_size_mb still does return sum(self.size_tempfiles), so despite the _mb suffix it now sums bytes, not megabytes.

That internal inconsistency is error-prone for callers relying on temp_size_mb to be in MB. Either the units should be converted or the API renamed; preserving the existing name strongly suggests keeping MB semantics.

One straightforward fix is to convert to MiB when computing temp_size_mb:
 @property
 def temp_size_mb(self):
-    return sum(self.size_tempfiles)
+    # size_tempfiles currently returns sizes in bytes; convert to MiB here.
+    return sum(size / 1024 / 1024 for size in self.size_tempfiles)
This keeps size_tempfiles as a low-level byte-based helper while restoring the documented intent of temp_size_mb.

🧹 Nitpick comments (1)

src/snakemake/io/__init__.py (1)
648-662: Preserve original FileNotFoundError as the cause when wrapping in FileOrDirectoryNotFoundError

The split between size() and size_local() and the move to FileOrDirectoryNotFoundError for missing local files is coherent and matches the new cross-module handling.

To keep the original OS-level FileNotFoundError in the exception chain (and satisfy tools like Ruff B904), consider binding the exception and re-raising with an explicit cause:
 async def size_local(self):
     # follow symlinks but throw error if invalid
     await self.check_broken_symlink()
-    try:
-        return os.path.getsize(self.file)
-    except FileNotFoundError:
-        raise FileOrDirectoryNotFoundError(Path(self.file))
+    try:
+        return os.path.getsize(self.file)
+    except FileNotFoundError as e:
+        # Propagate a uniform not-found error type while preserving the OS error as cause.
+        raise FileOrDirectoryNotFoundError(Path(self.file)) from e
If you intentionally want to hide the underlying cause, from None would also address the linter warning.

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 6f2ea03 and 3790986.

⛔ Files ignored due to path filters (1)

pyproject.toml is excluded by !pyproject.toml

📒 Files selected for processing (3)

src/snakemake/exceptions.py (2 hunks)
src/snakemake/io/__init__.py (4 hunks)
src/snakemake/resources.py (2 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

src/snakemake/resources.py
src/snakemake/exceptions.py
src/snakemake/io/__init__.py

🧬 Code graph analysis (1)

src/snakemake/io/__init__.py (2)

src/snakemake/scheduling/job_scheduler.py (1)

get_size (199-200)

src/snakemake/common/__init__.py (2)

async_run (96-113)

func_true (51-52)

🪛 Ruff (0.14.6)

src/snakemake/io/__init__.py

661-661: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (45)

GitHub Check: tests (8, ubuntu-latest, py313)
GitHub Check: tests (9, windows-2022, py313)
GitHub Check: tests (10, windows-2022, py313)
GitHub Check: tests (10, ubuntu-latest, py313)
GitHub Check: tests (9, ubuntu-latest, py313)
GitHub Check: tests (10, ubuntu-latest, py311)
GitHub Check: tests (10, ubuntu-latest, py312)
GitHub Check: tests (8, ubuntu-latest, py312)
GitHub Check: tests (9, ubuntu-latest, py311)
GitHub Check: tests (9, ubuntu-latest, py312)
GitHub Check: tests (7, ubuntu-latest, py313)
GitHub Check: tests (8, ubuntu-latest, py311)
GitHub Check: tests (8, windows-2022, py313)
GitHub Check: tests (8, macos-latest, py313)
GitHub Check: tests (7, ubuntu-latest, py312)
GitHub Check: tests (6, ubuntu-latest, py312)
GitHub Check: tests (7, windows-2022, py313)
GitHub Check: tests (4, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py311)
GitHub Check: tests (6, windows-2022, py313)
GitHub Check: tests (6, ubuntu-latest, py313)
GitHub Check: tests (5, macos-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py311)
GitHub Check: tests (4, windows-2022, py313)
GitHub Check: tests (6, ubuntu-latest, py311)
GitHub Check: tests (5, windows-2022, py313)
GitHub Check: tests (3, windows-2022, py313)
GitHub Check: tests (5, ubuntu-latest, py312)
GitHub Check: tests (4, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py312)
GitHub Check: tests (3, macos-latest, py313)
GitHub Check: tests (2, windows-2022, py313)
GitHub Check: tests (4, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py311)
GitHub Check: tests (2, ubuntu-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py311)
GitHub Check: tests (1, windows-2022, py313)
GitHub Check: tests (1, ubuntu-latest, py312)
GitHub Check: tests (2, macos-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py311)
GitHub Check: tests (1, macos-latest, py313)

🔇 Additional comments (2)

src/snakemake/resources.py (1)

239-270: Group resource evaluation correctly handles storage-backed file-not-found

Catching both FileNotFoundError and FileOrDirectoryNotFoundError around job.resources evaluation keeps the existing “skip jobs whose resource evaluation depends on not-yet-present group files” behavior, while extending it to storage-plugin paths. The change is cohesive with the rest of the PR.

src/snakemake/exceptions.py (1)

634-640: is_file_not_found_error now consistently covers storage-plugin missing files

Extending is_file_not_found_error to return True for FileOrDirectoryNotFoundError ensures that callers (like eval_resource_expression) uniformly recognize missing-file conditions from both local IO and storage plugins.

github-actions · 2025-12-01T13:49:34Z

Please format your code with pixi run format

coderabbitai

Actionable comments posted: 0

🧹 Nitpick comments (1)

src/snakemake/io/__init__.py (1)
657-660: Add exception chaining for better debugging.

While the exception conversion to FileOrDirectoryNotFoundError is correct for unified error handling, the original exception context should be preserved using raise ... from err to maintain the full traceback.

Apply this diff:
-        try:
-            return os.path.getsize(self.file)
-        except FileNotFoundError:
-            raise FileOrDirectoryNotFoundError(Path(self.file))
+        try:
+            return os.path.getsize(self.file)
+        except FileNotFoundError as err:
+            raise FileOrDirectoryNotFoundError(Path(self.file)) from err

📜 Review details

Configuration used: CodeRabbit UI

Review profile: CHILL

Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between 3790986 and c54780a.

⛔ Files ignored due to path filters (1)

pyproject.toml is excluded by !pyproject.toml

📒 Files selected for processing (1)

src/snakemake/io/__init__.py (3 hunks)

🧰 Additional context used

📓 Path-based instructions (1)

**/*.py

⚙️ CodeRabbit configuration file

**/*.py: Do not try to improve formatting.
Do not suggest type annotations for functions that are defined inside of functions or methods.
Do not suggest type annotation of the self argument of methods.
Do not suggest type annotation of the cls argument of classmethods.
Do not suggest return type annotation if a function or method does not contain a return statement.

Files:

src/snakemake/io/__init__.py

🪛 Ruff (0.14.7)

src/snakemake/io/__init__.py

660-660: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

⏰ Context from checks skipped due to timeout of 90000ms. You can increase the timeout in your CodeRabbit configuration to a maximum of 15 minutes (900000ms). (46)

GitHub Check: tests (10, ubuntu-latest, py313)
GitHub Check: tests (10, ubuntu-latest, py312)
GitHub Check: tests (10, ubuntu-latest, py311)
GitHub Check: tests (10, windows-2022, py313)
GitHub Check: tests (9, windows-2022, py313)
GitHub Check: tests (9, ubuntu-latest, py312)
GitHub Check: tests (8, ubuntu-latest, py312)
GitHub Check: tests (8, ubuntu-latest, py313)
GitHub Check: tests (9, ubuntu-latest, py311)
GitHub Check: tests (7, ubuntu-latest, py313)
GitHub Check: tests (9, ubuntu-latest, py313)
GitHub Check: tests (8, windows-2022, py313)
GitHub Check: tests (8, ubuntu-latest, py311)
GitHub Check: tests (6, ubuntu-latest, py311)
GitHub Check: tests (4, ubuntu-latest, py313)
GitHub Check: tests (7, windows-2022, py313)
GitHub Check: tests (6, ubuntu-latest, py313)
GitHub Check: tests (6, ubuntu-latest, py312)
GitHub Check: tests (7, ubuntu-latest, py312)
GitHub Check: tests (5, ubuntu-latest, py312)
GitHub Check: tests (7, ubuntu-latest, py311)
GitHub Check: tests (4, windows-2022, py313)
GitHub Check: tests (6, windows-2022, py313)
GitHub Check: tests (6, macos-latest, py313)
GitHub Check: tests (4, ubuntu-latest, py311)
GitHub Check: tests (5, windows-2022, py313)
GitHub Check: tests (4, macos-latest, py313)
GitHub Check: tests (5, ubuntu-latest, py313)
GitHub Check: tests (3, windows-2022, py313)
GitHub Check: tests (5, ubuntu-latest, py311)
GitHub Check: tests (3, macos-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py313)
GitHub Check: tests (4, ubuntu-latest, py312)
GitHub Check: tests (2, macos-latest, py313)
GitHub Check: tests (3, ubuntu-latest, py311)
GitHub Check: tests (2, ubuntu-latest, py311)
GitHub Check: tests (2, windows-2022, py313)
GitHub Check: tests (3, ubuntu-latest, py312)
GitHub Check: tests (2, ubuntu-latest, py313)
GitHub Check: tests (2, ubuntu-latest, py312)
GitHub Check: tests (1, ubuntu-latest, py313)
GitHub Check: tests (1, ubuntu-latest, py311)
GitHub Check: tests (1, ubuntu-latest, py312)
GitHub Check: tests (1, windows-2022, py313)
GitHub Check: tests (1, macos-latest, py313)
GitHub Check: apidocs

🔇 Additional comments (5)

src/snakemake/io/__init__.py (5)

48-48: LGTM: Import supports unified error handling.

The import of FileOrDirectoryNotFoundError enables consistent error handling for missing files across local and storage-backed paths, which aligns with the PR objective.

648-653: LGTM: Simplified size computation with proper type annotation.

The explicit -> int return type and streamlined storage logic improve clarity. Exceptions will correctly propagate from both storage and local paths.

1927-1937: LGTM: Predicated size helper correctly implements filtering logic.

The method properly filters files using an async predicate and computes sizes concurrently. When a missing file is encountered, FileOrDirectoryNotFoundError will propagate from f.size(), correctly implementing the PR objective to raise exceptions for missing files during size evaluations.

1940-1944: LGTM: Refactored to use predicated helper.

The property correctly delegates to _predicated_size_files with a predicate that selects all files, maintaining the original behavior while leveraging the new helper.

1946-1951: LGTM: Refactored to use predicated helper.

The property correctly filters for temp-flagged files using the predicated helper, maintaining the original behavior while simplifying the implementation.

🤖 I have created a release *beep* *boop* --- ## [9.14.1](v9.14.0...v9.14.1) (2025-12-02) ### Bug Fixes * always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations ([#3854](#3854)) ([0f1991a](0f1991a)) * ensure that spawned jobs do not handle retries themselves. The main process does that. ([#3855](#3855)) ([396271b](396271b)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

…rce expressions that do size evaluations (snakemake#3854)  ### QC  * [x] The PR contains a test case for the changes or the changes are already covered by an existing test case. * [x] The documentation (`docs/`) is updated to reflect the changes or this is not necessary (e.g. if the change does neither modify the language nor the behavior or functionalities of Snakemake).  ## Summary by CodeRabbit * **Refactor** * Reworked file-size logic into a predicate-driven helper so regular and temporary file-size queries delegate to a common implementation. * **Bug Fixes** * Broadened recognition of missing-file errors and treat them as not-found; affected resource evaluations are skipped instead of failing. * **Type Annotations** * Clarified file-size return-type annotations to be integer-valued for improved correctness and tooling. <sub>✏️ Tip: You can customize this high-level summary in your review settings.</sub>

🤖 I have created a release *beep* *boop* --- ## [9.14.1](snakemake/snakemake@v9.14.0...v9.14.1) (2025-12-02) ### Bug Fixes * always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations ([snakemake#3854](snakemake#3854)) ([0f1991a](snakemake@0f1991a)) * ensure that spawned jobs do not handle retries themselves. The main process does that. ([snakemake#3855](snakemake#3855)) ([396271b](snakemake@396271b)) --- This PR was generated with [Release Please](https://github.com/googleapis/release-please). See [documentation](https://github.com/googleapis/release-please#release-please).

fix: always raise FileNotFoundError in case of missing files in resou…

bbcf433

…rce expressions that do size evaluations

johanneskoester added 2 commits December 1, 2025 10:37

fix

3ccc91f

typing

a822209

coderabbitai bot reviewed Dec 1, 2025

View reviewed changes

src/snakemake/io/__init__.py Outdated Show resolved Hide resolved

coderabbitai bot reviewed Dec 1, 2025

View reviewed changes

johanneskoester added 2 commits December 1, 2025 10:46

typing

6f2ea03

rely on storage plugin failing with the correct exception

3790986

coderabbitai bot reviewed Dec 1, 2025

View reviewed changes

bump storage plugin interface

4962afc

backwards compat

c54780a

coderabbitai bot reviewed Dec 2, 2025

View reviewed changes

johanneskoester merged commit 0f1991a into main Dec 2, 2025
60 checks passed

johanneskoester deleted the fix/size-eval branch December 2, 2025 19:22

snakemake-bot mentioned this pull request Dec 2, 2025

chore(main): release 9.14.1 #3858

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

fix: always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations #3854

fix: always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations #3854

johanneskoester commented Dec 1, 2025 •

edited by coderabbitai bot

Loading

Uh oh!

coderabbitai bot commented Dec 1, 2025 •

edited

Loading

Walkthrough

Changes

Estimated code review effort

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

coderabbitai bot left a comment

Uh oh!

coderabbitai bot left a comment

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

coderabbitai bot left a comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

fix: always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations #3854

fix: always raise FileNotFoundError in case of missing files in resource expressions that do size evaluations #3854

Conversation

johanneskoester commented Dec 1, 2025 • edited by coderabbitai bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

QC

Summary by CodeRabbit

Uh oh!

coderabbitai bot commented Dec 1, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Walkthrough

Changes

Estimated code review effort

Pre-merge checks and finishing touches

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

github-actions bot commented Dec 1, 2025

Uh oh!

coderabbitai bot left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

1 participant

johanneskoester commented Dec 1, 2025 •

edited by coderabbitai bot

Loading

coderabbitai bot commented Dec 1, 2025 •

edited

Loading