Conversation
Typing conformance resultsNo changes detected ✅Current numbersThe percentage of diagnostics emitted that were expected errors held steady at 98.15%. The percentage of expected errors that received a diagnostic held steady at 98.07%. The number of fully passing files held steady at 133/146. |
Memory usage reportSummary
Significant changesClick to expand detailed breakdownprefect
sphinx
trio
flake8
|
|
| Lint rule | Added | Removed | Changed |
|---|---|---|---|
unsound-return-statement |
9 | 0 | 3 |
invalid-argument-type |
5 | 3 | 0 |
no-matching-overload |
5 | 0 | 0 |
invalid-return-type |
0 | 0 | 1 |
unused-type-ignore-comment |
0 | 1 | 0 |
| Total | 19 | 4 | 4 |
Raw diff (27 changes)
Tanjun (https://github.com/FasterSpeeding/Tanjun)
- tanjun/annotations.py:1449:31 error[invalid-argument-type] Argument to `Choices.__init__` is incorrect: Argument type `_EnumT@__getitem__ | float*` does not satisfy constraints (`int`, `float`, `str`) of type variable `_ChoiceT`
- tanjun/annotations.py:1454:31 error[invalid-argument-type] Argument to `Choices.__init__` is incorrect: Argument type `_EnumT@__getitem__ | int` does not satisfy constraints (`int`, `float`, `str`) of type variable `_ChoiceT`
- tanjun/annotations.py:1459:31 error[invalid-argument-type] Argument to `Choices.__init__` is incorrect: Argument type `_EnumT@__getitem__ | str` does not satisfy constraints (`int`, `float`, `str`) of type variable `_ChoiceT`
aiohttp (https://github.com/aio-libs/aiohttp)
- aiohttp/client.py:1497:16 error[invalid-return-type] Return type does not match returned value: expected `_RetType_co@_BaseRequestContextManager`, found `ClientResponse | _RetType_co@_BaseRequestContextManager`
+ aiohttp/client.py:1497:16 error[invalid-return-type] Return type does not match returned value: expected `_RetType_co@_BaseRequestContextManager`, found `ClientResponse | Unknown`
discord.py (https://github.com/Rapptz/discord.py)
- discord/utils.py:245:54 warning[unused-type-ignore-comment] Unused blanket `type: ignore` directive
+ discord/utils.py:248:16 warning[unsound-return-statement] Unsound return statement: `Unknown | list[T_co@SequenceProxy]` is not a subtype of `list[T_co@SequenceProxy]`
rotki (https://github.com/rotki/rotki)
+ rotkehlchen/chain/decoding/tools.py:119:16 warning[unsound-return-statement] Unsound return statement: `tuple[HistoryEventType, HistoryEventSubType, str | None, Unknown | None, str, str] | None` is not a subtype of `tuple[HistoryEventType, HistoryEventSubType, str | None, A@BaseDecoderTools | None, str, str] | None`
+ rotkehlchen/chain/decoding/tools.py:122:13 error[invalid-argument-type] Argument to function `decode_transfer_direction` is incorrect: Argument type `A@BaseDecoderTools` does not satisfy constraints (`BTCAddress`, `ChecksumAddress`, `SubstrateAddress`, `SolanaAddress`) of type variable `AnyBlockchainAddress`
+ rotkehlchen/chain/decoding/tools.py:123:13 error[invalid-argument-type] Argument to function `decode_transfer_direction` is incorrect: Argument type `A@BaseDecoderTools` does not satisfy constraints (`BTCAddress`, `ChecksumAddress`, `SubstrateAddress`, `SolanaAddress`) of type variable `AnyBlockchainAddress`
spark (https://github.com/apache/spark)
+ python/pyspark/core/rdd.py:1384:36 error[invalid-argument-type] Argument to function `bisect_left` is incorrect: Argument type `K@sortByKey` does not satisfy upper bound `SupportsDunderLT[Any] | SupportsDunderGT[Any]` of type variable `SupportsRichComparisonT`
+ python/pyspark/core/rdd.py:1386:24 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `int`
+ python/pyspark/core/rdd.py:1388:24 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `int`
+ python/pyspark/core/rdd.py:1431:24 error[invalid-argument-type] Argument to bound method `RDD.keyBy` is incorrect: Argument type `S@sortBy` does not satisfy upper bound `Hashable` of type variable `K`
+ python/pyspark/core/rdd.py:2575:19 error[no-matching-overload] No overload of function `nlargest` matches arguments
+ python/pyspark/core/rdd.py:2578:20 error[no-matching-overload] No overload of function `nlargest` matches arguments
+ python/pyspark/core/rdd.py:2578:20 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `list[T@top]`
+ python/pyspark/core/rdd.py:2635:24 error[no-matching-overload] No overload of function `nsmallest` matches arguments
+ python/pyspark/core/rdd.py:2635:24 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `list[T@takeOrdered]`
+ python/pyspark/core/rdd.py:2637:20 warning[unsound-return-statement] Unsound return statement: `Unknown | list[T@takeOrdered]` is not a subtype of `list[T@takeOrdered]`
+ python/pyspark/core/rdd.py:2637:51 error[no-matching-overload] No overload of function `nsmallest` matches arguments
- python/pyspark/core/rdd.py:1173:13 warning[unsound-return-statement] Unsound return statement: `RDD[T@intersection | Unknown]` is not a subtype of `RDD[T@intersection]`
+ python/pyspark/core/rdd.py:1173:13 warning[unsound-return-statement] Unsound return statement: `RDD[Unknown]` is not a subtype of `RDD[T@intersection]`
- python/pyspark/core/rdd.py:4354:16 warning[unsound-return-statement] Unsound return statement: `RDD[T@subtract | Unknown]` is not a subtype of `RDD[T@subtract]`
+ python/pyspark/core/rdd.py:4354:16 warning[unsound-return-statement] Unsound return statement: `RDD[Unknown]` is not a subtype of `RDD[T@subtract]`
+ python/pyspark/mllib/clustering.py:1100:16 warning[unsound-return-statement] Unsound return statement: `DStream[tuple[Unknown, int]]` is not a subtype of `DStream[tuple[T@predictOnValues, int]]`
- python/pyspark/mllib/regression.py:957:16 warning[unsound-return-statement] Unsound return statement: `DStream[tuple[K@predictOnValues, Unknown]]` is not a subtype of `DStream[tuple[K@predictOnValues, float]]`
+ python/pyspark/mllib/regression.py:957:16 warning[unsound-return-statement] Unsound return statement: `DStream[tuple[Unknown, Unknown]]` is not a subtype of `DStream[tuple[K@predictOnValues, float]]`
+ python/pyspark/shuffle.py:579:16 error[no-matching-overload] No overload of function `merge` matches arguments
+ python/pyspark/shuffle.py:579:16 warning[unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `Iterable[V@ExternalSorter]`
+ python/pyspark/shuffle.py:826:33 error[invalid-argument-type] Argument to function `sorted` is incorrect: Argument type `K@ExternalGroupBy` does not satisfy upper bound `SupportsDunderLT[Any] | SupportsDunderGT[Any]` of type variable `SupportsRichComparisonT`c879c79 to
10eaad2
Compare
Merging this PR will not alter performance
Comparing Footnotes
|
5ea87e1 to
1d3e699
Compare
| /// Checks declarations for solutions containing type variables fixed by an outer caller. | ||
| /// | ||
| /// TODO: Remove this check when solving preserves universal validity for non-inferable | ||
| /// variables. A raw constraint can accept `S <= T` with `T: str` for some `S`, but inferring | ||
| /// `T = S` is valid only if every type allowed by the caller's `S` satisfies the bound. | ||
| /// Relations involving other variables being inferred are left for subsequent solving. |
There was a problem hiding this comment.
Recursive inference in infer_map_impl will check S against str if matching a Source[S] argument against a Source[T] parameter, where T: str. But after this PR, the intersection arm in infer_map_impl no longer recurses if the argument is Source[S] & Unrelated, it passes the full intersection argument to constraint-set assignability, which fails to do this bounds check.
This is addressed in a better way by #28048 and #27652. This method is a temporary workaround to avoid regression in the meantime.
68f183f to
038b68d
Compare
Ecosystem reportPR #28677 ecosystem summaryAnalyzed head: The 27 diagnostic changes reflect stricter checking of generic bounds and constraints, improved inference after class narrowing, and changes to recovery after failed specialization. Spark, discord.py, and rotki pass unrestricted caller type variables to operations that require more specific types; the PR catches those missing guarantees. Tanjun loses three false positives because narrowed class arguments retain their intersections. The remaining return-message changes follow failed generic calls, including aiohttp's existing union-bound receiver limitation. All retained changes are expected; none blocks review. Diagnostic changes (12 unsound-return-statement; 8 invalid-argument-type; 5 no-matching-overload; 1 invalid-return-type; 1 unused-type-ignore-comment)1. Optional keys require both overload paths to be valid (5 no-matching-overload; 4 unsound-return-statement)Report entries (9 diagnostic hits)
Spark forwards optional key functions to The two overloads below show the same distinction with a smaller bound: the Optional overload selection (5 no-matching-overload; 4 unsound-return-statement). from typing import Protocol, TypeVar, overload
T = TypeVar('T')
class Items(Protocol[T]):
first: T
B = TypeVar('B', bound=int)
@overload
def select(items: Items[T], key: int) -> Items[T]:
...
@overload
def select(items: Items[B], key: None) -> Items[B]:
...
def select(items, key):
...
def wrapper(a: Items[T], key: int | None) -> Items[T]:
# Merge base: no diagnostic
# PR: [no-matching-overload] No overload of function `select` matches arguments
# PR: [unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `Items[T@wrapper]`
return select(a, key)2. Caller type variables must guarantee callee bounds (3 invalid-argument-type; 3 unsound-return-statement; 1 unused-type-ignore-comment)Report entries (7 diagnostic hits)
Spark passes hashable keys to The caller below accepts arbitrary Bounded overload and return recovery (3 invalid-argument-type; 2 unsound-return-statement). from typing import Protocol, TypeVar, overload
T = TypeVar('T')
class Items(Protocol[T]):
first: T
B = TypeVar('B', bound=int)
@overload
def locate(a: Items[B]) -> int:
...
@overload
def locate(a: Items[B], hi: int) -> int:
...
def locate(a, hi=None):
...
def wrapper(bounds: Items[T]) -> int:
# Merge base: no diagnostic
# PR: [invalid-argument-type] Argument to function `locate` is incorrect: Argument type `T@wrapper` does not satisfy upper bound `int` of type variable `B`
# PR: [unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `int`
return locate(bounds)Discord's sequence proxy permits arbitrary element types but sorts them without a key. The merge base incorrectly accepted an unrestricted caller type variable where sorting requires comparable values. The PR rejects that specialization, making the existing blanket ignore necessary and producing an The reduced example expresses the sorting function's relevant bound and the collection's structural element relationship locally. Returning a single element instead of a list preserves the same failed specialization and return-warning behavior. Ignored bound violation followed by an unsound return (1 unsound-return-statement; 1 unused-type-ignore-comment). from typing import Protocol, TypeVar
class SupportsLT(Protocol):
def __lt__(self, other: object, /) -> bool: ...
T = TypeVar("T", covariant=True)
class Source(Protocol[T]):
def get(self) -> T: ...
C = TypeVar("C", bound="SupportsLT")
def ordered_values(items: Source[C]) -> C:
raise NotImplementedError
def copied(items: Source[T]) -> T:
# Merge base: [unused-type-ignore-comment] Unused blanket `type: ignore` directive
# PR: no diagnostic
result = ordered_values(items) # type: ignore
# Merge base: no diagnostic
# PR: [unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `T@copied`
return result3. Failed receiver specializations discard partial results (4 unsound-return-statement)Report entries (4 diagnostic hits)
These calls already violate a receiver's type-variable bound: Spark's methods require hashable keys, but their callers accept unrestricted key types. The PR no longer carries a partially inferred key type through the failed specialization. The existing argument errors remain; recovery with Chained and single receiver calls (3 unsound-return-statement). The two RDD chains previously returned from typing import Generic, TypeVar
T = TypeVar('T')
T_co = TypeVar('T_co', covariant=True)
K = TypeVar('K', bound=int)
class Batch(Generic[T_co]):
def only_ints(self: 'Batch[K]') -> 'Batch[K]':
raise NotImplementedError
def checked_twice(self: 'Batch[T]') -> 'Batch[T]':
# Merge base: [unsound-return-statement] "Unsound return statement: `Batch[T@checked_twice | Unknown]` is not a subtype of `Batch[T@checked_twice]`"
# PR: [unsound-return-statement] "Unsound return statement: `Batch[Unknown]` is not a subtype of `Batch[T@checked_twice]`"
return self.only_ints().only_ints()
def checked(batch: "Batch[T]") -> "Batch[T]":
# Merge base: no unsound-return-statement warning
# PR: [unsound-return-statement] "Unsound return statement: `Batch[Unknown]` is not a subtype of `Batch[T@checked]`"
return batch.only_ints()A result that was already partly unknown (1 unsound-return-statement). In the regression model, the callback's result was already unknown because the annotated model type has no from typing import Generic, TypeVar
T_co = TypeVar('T_co', covariant=True)
K = TypeVar('K')
BoundK = TypeVar('BoundK', bound=int)
U = TypeVar('U')
class Batch(Generic[T_co]):
def with_value(self: 'Batch[tuple[BoundK, object]]', f: U) -> 'Batch[tuple[BoundK, U]]':
raise NotImplementedError
def unchecked(model: object, batch: 'Batch[tuple[K, object]]') -> 'Batch[tuple[K, float]]':
# Merge base: [unsound-return-statement] "Unsound return statement: `Batch[tuple[K@unchecked, Unknown]]` is not a subtype of `Batch[tuple[K@unchecked, float]]`"
# PR: [unsound-return-statement] "Unsound return statement: `Batch[tuple[Unknown, Unknown]]` is not a subtype of `Batch[tuple[K@unchecked, float]]`"
return batch.with_value(model.predict)4. Constrained callees require guarantees from their generic callers (2 invalid-argument-type; 1 unsound-return-statement)Report entries (3 diagnostic hits)
Rotki forwards a collection of addresses and an address lookup callback from a class whose address type variable is unconstrained to a helper that permits only four specific address types. The merge base accepts this call, although the caller's annotation does not guarantee any permitted address type. The PR rejects both arguments. Recovery replaces the inferred address type within the result with The reduced example preserves the collection's readable element type and the callback's accepted argument type as two protocols. Caller and callee constraints (2 invalid-argument-type; 1 unsound-return-statement). from typing import Protocol
class Source[T](Protocol):
def value(self) -> T: ...
class Consumer[T](Protocol):
def __call__(self, address: T) -> None: ...
def run[A](source: Source[A], consumer: Consumer[A]) -> A:
# Merge base: no diagnostic
# PR: [unsound-return-statement] Unsound return statement: `Unknown` is not a subtype of `A@run`
return decode(
# Merge base: no diagnostic
# PR: [invalid-argument-type] Argument to function `decode` is incorrect: Argument type `A@run` does not satisfy constraints (`int`, `str`) of type variable `C`
source,
# Merge base: no diagnostic
# PR: [invalid-argument-type] Argument to function `decode` is incorrect: Argument type `A@run` does not satisfy constraints (`int`, `str`) of type variable `C`
consumer,
)
def decode[C: (int, str)](source: Source[C], consumer: Consumer[C]) -> C:
raise NotImplementedError5. Class narrowing survives generic inference (3 invalid-argument-type)Report entries (3 diagnostic hits)
Tanjun checks that an enum class subclasses The reduced example replaces enum member lookup with a generic function accepting a class. After Existing ty issues: ty#3557 A narrowed class passed through a generic helper (3 invalid-argument-type). from typing import TypeVar
T = TypeVar("T")
def member(cls: type[T]) -> T:
raise NotImplementedError
C = TypeVar('C', int, str)
def choices_for(enum_type: type[T]) -> None:
if issubclass(enum_type, int):
# Merge base: [invalid-argument-type] Argument to function `accept_choice` is incorrect: Argument type `T@choices_for | int` does not satisfy constraints (`int`, `str`) of type variable `C`
# PR: no diagnostic
accept_choice(member(enum_type))
def accept_choice(value: C) -> None: ...6. Recovery after an invalid union-bounded method call (1 invalid-return-type)Report entries (1 diagnostic hit)
Aiohttp awaits Existing ty issues: ty#2585 Awaited result after a failed generic receiver specialization (1 invalid-return-type). The websocket method's bounded from typing import TypeVar
class Response:
async def __aenter__(self) -> "Response":
return self
S = TypeVar("S", bound="WebSocket")
class WebSocket:
async def __aenter__(self: S) -> S:
return self
T = TypeVar("T", bound="Response | WebSocket")
async def enter(response: T) -> T:
# Merge base: [invalid-return-type] Return type does not match returned value: expected `T@enter`, found `Response | T@enter`
# PR: [invalid-return-type] Return type does not match returned value: expected `T@enter`, found `Response | Unknown`
return await response.__aenter__()Reproduction
|
038b68d to
55dc223
Compare
55dc223 to
57e8f58
Compare
57e8f58 to
bd15813
Compare
bd15813 to
2c45369
Compare
Intersection arguments to generic functions can offer several valid specializations, with relationships between their type variables and other arguments. Collect these constraints through the full constraint-set-assignability type relation so inference preserves variance, alternatives, and correlations. For example, another argument can select
T = Afrom the alternatives provided bySink[A] & Sink[B].There are some cases we have to carve out:
ParamSpecorTypeVarTuplethat are nested inside an intersection. We don't handle this problem in general, but by far the most common case is an intersection with~AlwaysTruthyor~AlwaysFalsy. We avoid regression in that scenario by, whenever we have a single-positive-element intersection in covariant position, dropping the negative element(s) and recursing into the positive element in the old solver, instead of passing the intersection to the new solver. This is a sound upcast, and negative intersection elements generally don't contribute any constraints to generics solving.We also have to add some post-validation on the bounds and constraints of non-inferrable typevars. The old constraint solver validates these, but in this PR when we pass an intersection containing a non-inferable typevar directly to the new solver, we bypass that validation. The
validate_noninferablemethod added here can be removed once a better solution lands (see #28048 and #27652.)This partially addresses astral-sh/ty#3557. Argument checking and return inference still use the union-merged specialization when there are multiple valid solutions, and we haven't moved unions into the new solver yet.
Test plan
ParamSpecwrappers, and classmethodSelfinference from intersected receivers.Benchmarks
Ecosystem report
Analyzed head:
2c4536930678c26de7e9fe025a38c4c8407d6ee3. All 27 diagnostic changes are expected: stricter generic bound and constraint checking, three removed narrowing false positives, and return-type recovery changes after invalid calls. Full ecosystem report.