Beartype 0.19.0: A Flying Man-Pig Gives the Thumbs Up of QA #441
leycec
announced in
Announcements
Replies: 2 comments
-
|
@beartype 0.19.0 is now available via the viperous fangs of Anaconda + conda config --add channels conda-forge
conda install beartype@beartype 0.19.0 now parties with even the black |
Beta Was this translation helpful? Give feedback.
0 replies
-
|
I'm absolutely loving the gif collection here (not to mention the release notes themselves)! 🎉 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Uh oh!
There was an error while loading. Please reload this page.
Uh oh!
There was an error while loading. Please reload this page.
-
@beartype 0.19.0 gently glides into your CI workflow for a
crashmiracle landing. Engines go brrrrrrrr.@beartype 0.19.0 narrowly avoids the grazing sheep in this terrifying metaphor.
@beartype 0.19.0 invites you to experience either the future of QA or a new catastrophe for QA – all from the comfort of your (t)rusty keyboard. It now thrums with untold power and the lurid afterglow of our out-of-control release cycle.
pip install --upgrade beartype # <-- engines hit full throttle, stomach hits full empty@beartype 0.19.0 is proudly brought to you by...
GitHub Sponsors: When You Befriend the Bear, You Got a Bear
This release comes courtesy these proud GitHub Sponsors, without whom @leycec's cats would currently be eating grasshoppers:
https://sescollc.com
https://dylanmodesitt.com
https://metrolo.gy imagine if this domain actually worked. how cool would that be!?
Thanks so much, masters of fintech and metrology.
The Masters of Fintech and Metrology. That's who.
@beartype 0.19.0: What Broke This Time?
Probably, a whole lot. Hopefully, a whole little. The truth lies in the middle.
@beartype 0.19.0 sidles up to your codebase in its blind spot with something suspicious in its paws. Questionable new features include:
beartype.door.infer_hint(): let BeartypeAI™ write your type hints for you, because you no longer have der Wille zur Macht to constantly deal with all this [redacted pejorative]:beartype.claw.beartype_all()+BeartypeConf(claw_skip_package_names): a single one-liner type-checks your entire app stack at runtime or test-time while ignoring problematic third-party packages that inexplicably hate @beartype for "reasons":**kwargs: int | str: @beartype type-checks annotated variadic keyword arguments! yay! uhh... wait. wasn't @beartype always doing that? these emoji suggest otherwise: 😄 → 😭Deeper
O(1)type-checking. @beartype 0.19.0 now deeply type-checks type hints like:frozenset[...].set[...].collections.ChainMap[...].collections.Counter[...].collections.deque[...].collections.abc.Collection[...].collections.abc.ItemsView[...].collections.abc.KeysView[...].collections.abc.MutableSet[...].collections.abc.Set[...].collections.abc.ValuesView[...].typing.AbstractSet[...].typing.ChainMap[...].typing.Collection[...].typing.Counter[...].typing.Deque[...].typing.FrozenSet[...].typing.ItemsView[...].typing.KeysView[...].typing.MutableSet[...].typing.Set[...].typing.ValuesView[...].Shallow
O(1)type-checking support for exciting (yet wildly unpopular) PEP standards that nobody uses. @beartype 0.19.0 now quietly ignores these PEPs without throwing up everywhere:def muh_decorator_closure(*args: P.args, **kwargs: P.kwargs):).Ts = typing.TypeVarTuple('Ts')).**kwargstyping (e.g.,def muh_kwargs_func(**kwargs: typing.Unpack[MuhTypedDict]]):).Official
multiprocessingsupport. @beartype 0.19.0 now officially supports fork-based distributed workloads based on the awfulpicklemodule. 🤣Third-party decorator integration. @beartype 0.19.0 now officially supports popular Just-in-Time (JIT) decorators for machine learning (ML) like:
@equinox.filter_jit.@jax.jit.@numba.njit.Python 3.13 +
--disable-gil+--enable-experimental-jit. Pinch me, I must be hyperventilating into a paper bag again.Sane build toolchain: from
setuptools+setup.py🤮 to Hatch +pyproject.toml. 🥂Sane publishing toolchain: from antiquated GitHub Actions tokens 🤮 to PyPI-specific "Trusted Publishers". 🥂
Critical bugs resolutions: blah, blah. Who cares. I'm tired. So are you.
@beartype 0.19.0 feature list mollifies even the unruly pirate crowd in the back
infer_hint():Introducing BeartypeAI™, Your Chummy QA PalChummy Unpaid QA Pal BeartypeAI™ is on the job and grumbling already about union overtime. Because your team hates type hints (and you're reluctantly starting to admit they might be onto something), BeartypeAI™ does what nobody else wants to do. Authoring type hints is a thankless janitorial fetch quest that smells bad and consumes your last will to code.
Allow our new
beartype.door.infer_hint()function to automate your ongoing stomache pain away. Type hints may be like that kidney stone the size of your mother-in-law's big head, but that's no reason to curl up on a gurney clutching your side in blinding agony. But first:Type hints are the most compact description of the internal structure of your objects. If you know the type hint of an object, you know the object better than the object knows itself. Type hints are the ultimate self-documentation. Unlike docstrings, type hints never lie or @beartype breaks your app. Type hints are both human-readable and machine-readable. They're literally the only thing that is.
Many type hints are trivial to write. All of us can sling around breezy
list[int] | Nonetype hints while yawning. It's not impressive. My cats can write that type hint with one sleepless eye open. Seriously. Why do cats sleep with one eye open, anyway? Doesn't that kinda defeat the purpose of... I dunno, sleeping? Must suck to be a paranoid cat. Uhhh. Back to the discussion.Some type hints, however, are non-trivial. You can't write them. Nobody can. They contain more square brackets than an 80's ASCII roguelike with an understated name like Death Gehenna or Eternal Furnaces of NetSwargy. When your type hint looks like this, the end of code maintainability cannot be far:
that feeling when your type hints resemble incomprehensible toddlers
Moreover, you don't even know the internal structure of most objects. Somebody else wrote those objects. They forgot how those objects worked a hot minute after clocking out at 4:12AM seven days deep into a crunch-time death march last January. They documented how those objects worked, but their documentation doesn't make sense and lies about everything. Now nobody knows how those objects work.
But what if somebody did know how those objects work? What if somebody knew Python better than Python knew itself? Introducing... somebody.
infer_hint():Deep Introspection for the Deep Code DiverLet
@beartypeease your weary burden, traveller. It is dangerous to go alone:...uhh. If you say so,
@beartype. I guess?</weeps_in_square_bracket_hell>codebase narrowly dodges another inexpert potshot from
pygmentsBeartypeAI™: Even a Broken Algorithm is Right Twice a Commit
beartype.door.infer_hint()(i.e., the algorithm hereafter known simply as BeartypeAI™) knows all about deep introspection of arbitrarily complex objects. Here's what BeartypeAI™ knows:BeartypeAI™ is here to tell you something you don't know and wouldn't care about even if you did.
BeartypeAI™ is here to write your type hints for you. Why? Because you hate type hints. Just:
beartype.door.infer_hint()arbitrarily complex objects.It's impossible to unpack how much madness is happening inside BeartypeAI™. Let's try anyway.
BeartypeAI™ pounds back another as your
git logexplodes in the distanceHomebrew Collection Classes: Annotate the Unannotatable
We've all been there. Some "genius"-tier wise guy devbro invented his own homebrew pure-Python collection called
WeirdoCustomListwithout subclassing a standardcollections.abcabstract base class. Sure, they could have just subclassedcollections.abc.MutableSequenceto write their special-needs alternative to the builtinlisttype. That would have been too easy. They're a masochist, so they wrote everything from scratch.Homebrew collections are more common than you think. They're friggin' everywhere! They're multiplying like meat flies! The cat's choking on bloody homebrow collections! Look above, for example. See that nasty
typing.Annotated[collections.abc.Collection[str], IsInstance[pygments.lexer.include]]type hint that BeartypeAI™ wrote for you? Yeah.That's right. The public
pygments.lexer.includetype is actually a homebrew collection. They thought they were being smart. Sadly, they were being dumb. Because they wrote a homebrew collection from scratch, their collection type is unsubscriptable. You can't subscript it with child type hints like withlist[str], so you can't use their collection type as a type hint factory to write type hints, so you can't actually validate their homebrew collection. In fact, you can't validate any homebrew collections....until now. BeartypeAI™ knows all about homebrew collections. BeartypeAI™ knows that you can actually validate homebrew collections – but only if you write a custom beartype validator leveraging the PEP 593-compliant
typing.Annotated[...]type hint factory in concert with the @beartype-specificbeartype.vale.IsInstance[...]validator. Please don't do this manually. You value your precious life force that is leaking all over your keyboard as we speak.That's what the
typing.Annotated[collections.abc.Collection[str], IsInstance[pygments.lexer.include]]type hint is all about. BeartypeAI™ correctly detected that this homebrew collection is actually just an instance of thepygments.lexer.includeclass that is externally usable as a collection of strings.Let's get more explicit. Nobody understands
pygments– not evenpygments. Instead, consider...BeartypeAI™ casually flings itself above another picturesque yet ultimately incomprehensible codebase
Exhibit (A) – Weirdo Custom List
It's... it's hideous!
...which prints:
"What's so hot about that?", you may now be thinking. Allow me to now pontificate boringly.
WeirdoCustomListisn't subscriptable. It's not a type hint factory. Moreover, despite being a mutable sequence,WeirdoCustomListdoesn't actually subclass the standardcollections.abc.MutableSequenceprotocol. Yet, BeartypeAI™ correctly detected that this particular weirdo custom list is a mutable sequence of strings. How? It's best not to ask weirdo custom list these questions. 🤣wake up BeartypeAI™ when those type hints start making sense
Callable[...]Type Hints: Gods, They Suck.Annotating callables (especially callbacks) with PEP-compliant
Callable[...]type hints is basically impossible. Personally, I've never gotten a singleCallable[...]type hint to work right. They never match the callables they're supposed to when I write them myself.mypyandpyrightalways vomit all over themselves and then me. I mostly just give up now and use the unsubscriptedcollections.abc.Callableabstract base class instead of full-blownCallable[...]type hints......until now. BeartypeAI™ knows literally everything there is to know about annotating callables. What doesn't BeartypeAI™ know? Well, friends:
BeartypeAI™ knows that a
lambdafunction accepting no parameters is annotated as...BeartypeAI™ knows that a
lambdafunction accepting multiple parameters is annotated as...BeartypeAI™ knows that a normal function accepting two mandatory annotated parameters and one optional annotated parameter is "best" annotated with a PEP 612-compliant
typing.Concatenate[...]subscription as...BeartypeAI™ knows that a decorator wrapper function accepting a PEP 612-compliant parameter specification is annotated as...
BeartypeAI™ knows that a decorator wrapper function accepting two mandatory annotated parameters followed by a PEP 612-compliant parameter specification is annotated with a PEP 612-compliant
typing.Concatenate[...]subscription as...What I'm trying to say here is that BeartypeAI™ knows all and sees all and doesn't like it what it sees, but is still doing it's best for everybody. It knows more than me. It probably knows more than even you, even though you know everything. That's how much BeartypeAI™ knows.
BeartypeAI™ takes the high road when it comes to
Callable[...]type hintsTensors Type Hints: Gods, They Suck Too.
Tensor type hints really suck. So your team wants to annotate NumPy, JAX, PyTorch, or TensorFlow arrays, huh? That's a perfectly reasonable request. Too bad, though. Because tensor type hints suck.
Tensor type hints suck so bad you have to use third-party packages like
jaxtypingjust to make them work, despite the fact that both NumPy and JAX ship type hint-centric subpackages likenumpy.typingandjax.typingthat are supposed to make tensor type hints "just work." Of course, tensor type hints don't "just work." They don't even work......until now. BeartypeAI™ knows literally everything there is to know about annotating tensor type hints. Actually, that's a lie. I really wanted BeartypeAI™ to know literally everything there is to know about annotating tensor type hints in time for @beartype
0.19.0rc1. Sadly, I played video games instead. I only got around to implementing BeartypeAI™ support for inferring NumPy tensor type hints.Still, NumPy is better than nothing. One out of four ain't bad. Right? ...anybody? 😮💨
And... that's the type hint. That type hint requires no third-party dependencies. It's all BeartypeAI™, all one-liner. Nobody's writing that sort of gruelling bracket hell on their own. Not even @leycec. Just let somebody else do your suffering for you. That somebody is BeartypeAI™. Who knew?
your codebase grips its hat as BeartypeAI™ boldly shows off for no reason
infer_hint()Time Complexity: All Roads Lead toO(1)infer_hint()is the first @beartype API to respect the long-standing BeartypeConf(strategy=BeartypeStrategy.O*)configuration option. Previously, *all* @beartype APIs defaulted toO(1)` constant-time behaviour by randomly sampling container items for improved scalability. In the future, all @beartype APIs will allow you to customize this behaviour by specifying alternate iteration strategies like:O(n)linear-time behaviour, in which @beartype exhaustively examines all possible container items with recursion.O(log n)logarithmic-time behaviour, in which @beartype recursively examines only a logarithmic subset of all possible container items – a scalable compromise between non-deterministicO(1)immediacy and deterministicO(n)lethargy.Now,
infer_hint()is the first @beartype API to fully support two of those three strategies. Witness as history unfolds with a discomfiting "plop!":infer_hint(obj)is equivalent toinfer_hint(obj, conf=BeartypeConf(strategy=BeartypeStrategy.On))). Under theO(n)strategy,infer_hint()exhaustively examines all possible container items with recursion. To infer authoritative type hints from interactive REPLs and Jupyter Notebooks,infer_hint()differs from the remainder of the @beartype codebase by defaulting toO(n)-style linear-time iteration. This is generally what most users "probably" want when inferring type hints. Since you are reading this, you are not one of those users.infer_hint(obj, conf=BeartypeConf(strategy=BeartypeStrategy.O1))). Under theO(1)strategy,infer_hint()pseudo-randomly examines only a single container item at each nesting level. This is generally what algorithms like structural similarity see below and multiple dispatch more seeing below want. To activate the Hyperlight Drive, these use cases want to explicitly pass aconfenabling theO1strategy.BeartypeStrategy.O1: punch it, bald man!@leycec punches it with fear in his heart
infer_hint()Use Cases: Where We Pontificate Both Laconically and LoquaciouslyWhat do those words even mean? Doesn't matter. Thankfully, what does matter is that type hint inference has real-world use cases that far exceed just "write my type hints for me, cause i h8 type hints m8. fr!" Just get a gander of these algorithmic goodies:
O(1)structural similarity comparison – faster even than==-based equality comparison between arbitrary objects, which has worst-caseO(n)linear-time complexity and thus scales poorly. Hyper-fast object comparison is what we sayin'.O(1)and worst-caseO(k)multiple-dispatch forkthe number of callables being dispatched to – probably the fastest multiple-dispatch algorithm in any language. Hyper-fast dispatch is what we still sayin'.Let's plumb these depths like Mario on a Piranha plant pipe bender.
@beartype smokes two bug-filled joints. then, @beartype smokes two more.
Use Case #1: Structural Similarity (So It Does That Too Now, Huh?)
In the beginning, there was:
But what if you want to test whether two objects are merely structurally similar (i.e., have a similar internal structure but are neither literally nor semantically identical)? Without BeartypeAI™, you can't do that. But you have BeartypeAI™. You no longer have to accept the mouldy table scraps that the standard Python library has left you.
Structural similarity compares the large-scale "shape" of two objects without regard for the small-scale minutiae (like the exact items) in those objects.
Structural similarity thus combines:
isoperator (likeO(1)time complexity when callinginfer_hint(obj, conf=BeartypeConf(strategy=BeartypeStrategy.On)))) with...==operator (like actually computing meaningful work) with...Tensors offer a useful way to understand structural similarity: "Do two tensors have the same
dtype(i.e., type of all items in a tensor) andndim(i.e., dimensionality)? If yay, those two tensors are structurally similar; if nay, those two tensors are structurally dissimilar."There are two different kinds of structural similarity, broadly speaking:
awesome_hintcontains a dictionary mapping to integers,baleful_hintcontains a dictionary mapping to merely objects. You are now thinking: "Integers are objects, you doltish man. Shouldn't we be able to ignore these awkward trivialities?" I object to being called a dolt while admitting you make a point. Make things vaguer by harnessing the perfidious power ofbeartype.door.infer_hint()+beartype.door.is_bearable(). Arise, substructural similarity! A new darkness!Structural similarity cheatsheet, because the one-liner is a harsh mistress:
infer_hint(obj_1) == infer_hint(obj_2).is_bearable(obj_1, infer_hint(obj_2)).Structural similarity: when you care about what your objects care about.
say goodbye to expensive comparisons that never really liked you anyway
Use Case #3: Single Dispatch (Dis Ain't Yo Momma's Dispatch)
"Dispatch" is a common decision problem in... well, basically any modern language that matters. So, not C. i have no regrets for igniting this flame war
Everyone's familiar with single-dispatch polymorphism, whereby an object-oriented language dynamically routes a call of an object's method to that object's "deepest" subclass overriding that method. In Python, we call this the method-resolution order (MRO) of an object. It's quite boring and pedantic stuff, really. I personally wouldn't click any of those links – especially not on the weekend.
But what if you want to perform single-dispatch outside of a class hierarchy that you directly control? Moreover, what if want to single-dispatch on arbitrary type hints deeply describing the internal structures of objects? Classes are superficial; they fail to fully convey the types of items contained in instances of those classes, which is why we do this type hint thing.
Moreover, what if we want to perform multiple-dispatch, whereby the callable that is dynamically routed to (i.e., called) depends not simply on the type of a single object but an arbitrary number of objects? This is the dynamical Hell we now find ourselves in.
Interestingly, it turns out that combining the
beartype.door.infer_hint()+beartype.door.is_bearable()functions trivially yields highly efficientO(1)algorithms that transparently implement both single- and multiple-dispatch. First, the full-throttle single dispatch algorithm:This exhibits time complexity:
O(1). oh by godsO(1). the gods glare with envyO(k)forkcallables being dispatched to. the gods smugly look down and snickerGenerally speaking, we expect
kto be small in the average case – like,k < 10small. So this is basicallyO(1)single-dispatch in even the non-amortized worst case.Totally realistic and compelling usage resembles something like:
That's probably the fastest possible single-dispatch algorithm in any language. But here's where the bullet train really goes off the rails...
@beartype flies foolishly close to the fathomless void so you don't have to
Use Case #3: Multiple Dispatch (Dat Sasquatch Ain't Got Nuthin' on Us)
The single-dispatch algorithm trivially generalizes to multiple-dispatch as well. How? With cleverness, grit, and twin handlebar moustaches. 👨 👨 ← gritty moustauche twins
Let's reduce the multiple-dispatch case (of dispatching over multiple objects) to the single-dispatch case (of dispatching on a single object) by concatenating those multiple objects into a single object. Specifically, let's encapsulate those multiple objects into a tuple. Tuples are highly space- and time-efficient in Python. More importantly, tuples whose items are hashable are themselves hashable. By subscripting fixed-length
tuple[...]types by the multiple type hints (almost all of which are hashable) to be dispatched across, we can actually leverage the almost exact same algorithm as above to performO(1)multiple dispatch:That's... literally the exact same function. The signature just accepts variadic positional arguments
*argsrather than a singleobj. Whatevah!Crucially, this is still amortized worst-case
O(1)and non-amortized worst-caseO(k)multiple-dispatch forkthe number of callables being dispatched. In other words, we have a cardinality-invariant dispatch algorithm. The time complexity of this algorithm is unconditionallyO(k)regardless of the number of objects being dispatched over or the size of those objects. Since we generally expectkto be small, this is still basicallyO(1)multiple-dispatch. wuuuuuuuuuutTotally realistic and compelling usage intensifies holistically:
That's definitely the fastest possible multiple-dispatch algorithm in any language. Can't do better than
O(1). Suck it, Julia. Suck it.I've tested that rickety jerry-rigged shadow madness. Against all odds... it somehow works. No idea how, honestly. Probably falls down in edge cases, honestly. But at least for one fleeting moment in the rain, we had a dream of something beautiful. 🤣
O(1)multiple dispatch: the skull means it wants to help you**kwargs: The Type-checking Chickens Come Home to RoostA few bicycle trips ago, my wife asked me a real eye-opener as the stinging sweat trickled down:
Me:
Seriously. Metaphors, folks. What good are they for if they make less sense than cats wearing pizza hats? Which leads us straight to...
Variadic keyword arguments. All this time, it was reasonable to believe that @beartype was type-checking annotated variadic keyword arguments like
def func_in_a_funk(**kwargs: int). In actuality, @beartype was type-checking nothing there. Annotated variadic keyword arguments were silently ignored. Our flimsy reasons for doing nothing were fivefold:The last reason is a good reason. The rest are bad reasons. Therefore, @beartype now type-checks annotated variadic keyword arguments as standardized by PEP 484 a decade ago.
The syntax is a bit odd, though. Although boring, this is worth belabouring. Python usually wants you to explicitly spell everything out. "Explicit is better than implicit" – except when it's not, apparently. All variadic keyword arguments are dictionaries mapping from strings (i.e., excess parameter names passed by keyword) to arbitrary objects (i.e., the values of those parameters). So far, so boring.
Since all variadic keyword arguments necessarily satisfy the type hint
dict[str, object], however, Python interprets the type hint annotating a variadic keyword argument as the child value type hint of that dictionary. Thus,def oh_boy(**kwargs: float)is semantically equivalent todef oh_boy(kwargs: dict[str, float])from the perspective of the body ofoh_boy(). So far, still boring... albeit kinda weird and meandering now.Above, I self-importantly wrote that "The types of the values of excess keyword arguments vary by keyword." The real-life embodiment of this is:
Above, the first excess keyword parameter
hear_nuffinhas typeintwhile the second excess keyword parametersee_nuffinhas typestr. How do you type this madness as a single type hint when the types vary by keyword? Simple:The Beartype-Friendly Way, which is also the awful way. This approach has the benefit of being supported by @beartype, but the drawback of sucking. You ambiguously type
**kwargsas the union of the types of the values of all excess keyword arguments: e.g.,The PEP 692 Way, which is honestly also kinda awful. This has the benefit of being unambiguous (unlike the "Beartype-Friendly Way" above), but the multiple drawbacks of not currently being supported by @beartype, requiring Python ≥ 3.12, and expanding into 500 lines of sad-faced boilerplate that will break your will to sling code on Friday nights. Under PEP 692, you type
**kwargsas thetyping.Unpackof atyping.TypedDictsubclass that you define just to unambiguously constrain the types of the values of all excess keyword arguments: e.g.,Even if @beartype supports PEP 692 by the time you read this, Spoiler from the sad future: ...it still doesn't!? I'd still personally opt for the
@beartype-friendlydef fugly_muffin(**kwargs: int | str):approach. Sure, it's ambiguous. But it's also a trivial 9 characters rather than 500 lines of sad-faced boilerplate. Consider the number of callables that accept**kwargsin your codebase. Are you really gonna define one uniqueTypedDictsubclass just to unambiguously annotate each**kwargsparameter? Really? Some of us might think we are. But then we try and fall down clutching our rib cages. The finger-breaking reality of that much boilerplate has broken greater devs than us before.I scoff into my "Revenge of the Nerds"-era pocket protector. Annotating variadic keyword arguments may still suck after all these years – but at least @beartype now supports the slightly less sucktastic way.
five out of ten men who are pigs support this feature. do you?
Theory Crafting Time: @leycec Spins Fake News Faster Than a Turboprop
Let's create a fake PEP and pretend it exists. In other words, this subsection is of no value to anyone whatsoever. Still, unhinged dreams exist for a reason. If I was a CPython
typingdev, this is the Mirror World PEP 692 that I personally would have written to trivialize**kwargstype hints:Trivial. Right? Just subscript
typing.Unpack[...]with dictionary-like key-value pairs of the names and types of all excess keyword arguments accepted by that callable. This is already valid Python syntax as shown above. No changes to the CPython PEG (Parser Expression Grammar) or parser are required. This syntax promotes trivial, readable, maintainable, debuggable one-line type hints for**kwargs. No extraneous 500-lineTypeDefsubclasses or whatevah boilerplate are required.Moreover, this same syntax easily generalizes to
Callable[...]type hints. Currently,Callable[...]type hints fail to support keyword arguments; they only support positional-only parameters. Since nobody uses positional-only arguments,Callable[...]type hints are basically useless as defined. Instead,Callable[...]type hints could be readily extended using the exact same syntactic mechanism to support both keyword and keyword-only arguments: e.g.,Dictionary-like syntax should totally be the standard way to type keyword parameters. We know I mean standardization business, because I just used the word "totally."
man-pig ponders the existential nature of bad standards, as nobody cares
beartype_all(): It Actually Works Now, KindaAh, yes. The venerable
beartype.claw.beartype_all()import hook. Anybody remember that thing? Me neither. Nobody uses that thing, because that thing blows up whenever you look at it. Let's back up.beartype_all()unconditionally type-checks literally everything. Whereasbeartype_this_package()only type-checks your package,beartype_all()type-checks both your package and everybody else's packages too. Fake footnote: Technically, this includes even the standard CPython library. Pragmatically, the standard CPython library contains no type hints whatsoever. Why? Because CPython devs hate runtime typing. This fake footnote means nothing. I wasted my time writing this. You wasted your time reading this. We cry tears in the rain together.It's the "everybody else's packages too" part that is the problem there. Although you expect your package to be type-checked with
@beartype, nobody else does. Nobody expects their package to be type-checked with@beartypewithout their permission or knowledge – at least, not until you open 317 pending issues on their issue tracker enumerating every shocking yet mundane "error" in their package when type-checked with@beartype. This is whybeartype_all()fails you when you need it most. You can't control other people's intransigence towards the Bear... until now.Introducing the new
BeartypeConf(claw_skip_package_names: Collection[str] = ())configuration option!claw_skip_package_namesis a package name blacklist (e.g., ban, deny, ignore, or omit list of the names of all packages and modules to be excluded from consideration), enabling you to selectively ignore one or more problematic third-party packages when type-checking the entire Universe viabeartype_all(). The Universe just got a little smaller and a lot smarter, folks.Because
claw_skip_package_namesis sane:list,tuple,set, whatevahs).'bad_apple').'lovely_cat.ugly_dog').Because reality is even more disappointing than public education prepared us for, the
claw_skip_package_names"option" is basically mandatory. It's not optional despite being called an option. Whenever you callbeartype_all(), you also need to passclaw_skip_package_names: e.g.,claw_skip_package_names: because the Universe is kinda like in the Aliens franchise.when you fly with @beartype, you fly with a man who is a pig
beartype_all()+pytest-beartype: They Were Meant for Each Otherpytest-beartypeplugin users are now thoughtfully chewing their upper lips and thinking:You do you. That's why
pytest-beartypemaintainer (and all-around Python-Zig tooling God) @tusharsadhwani has already implemented support for command-line equivalents of both thebeartype_all()import hook and theclaw_skip_package_namesoption. In short, just:Let's unpack this. CLI stuff is always so crufty, isn't it? Passing:
--beartype-packages='*'instructspytest-beartypeto internally call the universalbeartype_all()import hook rather than the localbeartype_packages()import hook. Good.--beartype-skip-packagesblacklists those packages and modules from consideration. Good.--beartype-skip-packages: because not all heroes write one-liners.sick burn, pig-man. but what does this have to do with @beartype? the answer may shock somebody.
A Deeper Shade of
GrayO(1)Type-checking@beartype 0.19.0 deeply type-checks a ton of fun containers I've loosely dubbed reiterables.
A reiterable is a collection satisfying the
collections.abc.Collectionprotocol with guaranteedO(1)read-only access to only the first collection item. Reiterables include sets, frozen sets, dictionary views, deques (i.e., double-ended queues), and all other containers matched by one or more of the following PEP 484- or 585-compliant type hints:frozenset[...].set[...].collections.ChainMap[...].collections.Counter[...].collections.deque[...].collections.abc.Collection[...].collections.abc.ItemsView[...].collections.abc.KeysView[...].collections.abc.MutableSet[...].collections.abc.Set[...].collections.abc.ValuesView[...].typing.AbstractSet[...].typing.ChainMap[...].typing.Collection[...].typing.Counter[...].typing.Deque[...].typing.FrozenSet[...].typing.ItemsView[...].typing.KeysView[...].typing.MutableSet[...].typing.Set[...].typing.ValuesView[...].@beartype now deeply type-checks almost all of the core PEP 484 and 585 standards, resolving feature request #167 kindly submitted by the perennial brilliant @langfield (...how I miss that awesome guy!) several lifetimes ago back when I was probably a wandering vagabond Buddhist monk with a bad attitude, a begging bowl the size of my emaciated torso, and an honestly pretty cool straw hat that glinted dangerously in the firelight.
There's still a bit of low-hanging fruit dangling its juicy skin here and there – but not much. The biggest offenders that have yet to be deeply type-checked are:
Iterable[...]type hints. Not hard, so I claim. Just needs a bit of spit and polish, so I claim. I'm claiming lots of things without hard evidence here.Callable type hints (e.g.,
collections.abc.Callable[...],typing.Callable[...]). Thankfully, BeartypeAI™ now provides a trivialO(1)one-liner for deeply type-checking any callable type hinthintagainst any arbitrary callablefunc:Type variables (e.g.,
typing.TypeVar('T'). Still no idea how to dynamically generate efficient code type-checking type variables, honestly. It's feasible, but let's avoid thinking about this until there's absolutely nothing left to do. 😅...how is to possible that so much and yet so little has changed? Please manage our time better or we're never gonna cross that finish line, GitHub. @leycec assumes no responsibility for just playing video games for a year.
@beartype: It's actually starting to do stuff, now.
if i'm reading these schematics right, @beartype actually does stuff now
PEPs 612 + 646 + 692: @beartype Now Shallowly Loves You!
@beartype's love for PEP 612 – Parameter Specification Variables, PEPs646 – Variadic Generics, and PEP 692 – Using TypedDict for more precise
**kwargstyping may be a tepid pool of mucky brackish water you can barely dip your toes into – but at least @beartype 0.19.0 tried, daggumit.PEP 612: Make Your Decorator Closure So Complex It Explodes
@beartype 0.19.0 now supports you in your aspirations to obfuscate decorator closures beyond the dark horizon of MIT Python obfuscation competitions by silently ignoring those aspirations:
@beartype doesn't pretend to understand what
typing.ParamSpec('P').kwargsmeans, but @beartype doesn't have to. @beartype is here to crush bugs and play video games... and @beartype is all outta video games.that one unforgettable moment when @beartype 0.19.0 reveals its true nature
PEP 692: Finally,
TypedDictIs Useful for Something@beartype 0.19.0 now supports you in your aspirations to precisely type-check
**kwargsby silently ignoring those aspirations:That's better than @beartype used to do (which was blow chunks everywhere). We don't deeply type-check this yet, but we will. Would @leycec lie!? 😓
newer and sleaker @beartype does a surprise fly-by over your codebase. hats are almost lost.
PEP 646: Even Type Variables Are Now Tuples, Huh?
It's all tuples all the way down with PEP 646.
@beartype 0.19.0 now supports you in your aspirations to precisely type-check... actually, I really have no idea. But that's okay, because neither does @beartype 0.19.0. What is PEP 646 besides really confusing? Couldn't tell ya. All I know is that @beartype now silently ignores PEP 646-compliant type variable tuples (i.e.,
typing.TypeVarTupleobjects):I kinda get it, but I kinda don't. Since @beartype 0.19.0 doesn't get it any more than I do, @beartype doesn't deeply type-check
TypeVarTupleobjects yet. Will it ever? No idea. Let's pretend:@beartype doesn't even deeply type-check
TypeVarobjects yet – which is the slightly lower-hanging fruit here. Oh, when will free time materialize for @leycec? What has @beartype done to deserve this punishing development schedule? I fear for your immortalgit log, @beartype. 😨@beartype 42.42.42 chortles as it contemplates the darkness of the past
Multiprocessing Queues: @beartype No Longer Hates You!
@beartype 0.19.0 officially supports the standard
multiprocessingAPI for fork-based distributed workloads. All beartype exceptions (i.e., exception subclasses published by thebeartype.roarsubpackage) now support pickling and unpickling via the standardpicklemodule, which then suffices to support the standardmultiprocessingpackage, which shockingly leveragespicklerather thandillin 2024.@beartype 0.19.0: "Why is
picklestill even a thing!?"You're stupid,
multiprocessing. I hate that in an API.Third-party Decorators: @beartype No Longer Hates You!
@beartype 0.19.0 goes hard on integration with external decorators published by third-party packages that previously hated @beartype. This includes popular Just-in-Time (JIT) decorators for machine learning (ML) like:
@equinox.filter_jit.@jax.jit.@numba.njit.@beartype should now support almost everybody else's decorators. In fact, @beartype now generically supports all pseudo-callable wrapper objects (i.e., objects defining both the
__call__()and__wrapped__dunder attributes).The
@beartypedecorator should also now be context-free. You may now chain (i.e., list)@beartypeabove or below most third-party decorators. Sincebeartype.clawimport hooks (likebeartype_this_package()andbeartype_package()) inject@beartypeabove all other decorators,beartype.clawimport hooks now transparently support all other decorators... probably. 😬If you previously blacklisted @beartype from type-checking callables decorated by any of the above with
@typing.no_type_check, let us give thanks as you remove@typing.no_type_checkeverywhere.Examples or it only happened in the DMT hyperplane:
...which raises the expected type-checking violation:
The perspicacious user may now be thinking:
You're not wrong. But we're tired. At least @beartype works now for various definitions of "works." If you just hit this ambiguous type-checking violation message in your workflow and want @beartype to justifiably do something about it, bang on our issue tracker until the cats start squalling and biting @leycec in the face. Works every time.
@beartype 0.19.0: we broke our sanity for your security.
@beartype 0.19.0: it's been a long journey, fam.
CPython 3.13: Finally, No Longer Feel Emberassed about CPython
@beartype 0.19.0 officially supports Python 3.13, the first official CPython release you no longer need to feel ashamed of running in public.
Python 3.13 supports an official LLVM-based Just-in-Time (JIT) compiler via the PEP 744-compliant
--enable-experimental-jitcompile-time option. OMMMMMMMMMMMMMMMMG..... It's happening. It's really happening. My breathing is now laboured and making awkwardly squishy noises that upset the cat.Python 3.13 also supports no-GIL GIL-free multi-threading via the PEP 703-compliant
--disable-gilcompile-time option. Yes! YES! YEEEEEEESSSSS!!!! Wait. Where am I? What are these fingers on this keyboard? This must be what Xanadu is typed of.If I worked on a proprietary Python package, I'd have money. I'd also:
python >=3.13for production workloads as soon as CPython 3.13 lands in October.--enable-experimental-jitfor development workloads.It's time to go fast. Finally, it's time to feel shameless.
OMG IZ @beartype + CPython 3.13 +
--enable-experimental-jitWTF FAFO!!!Bear Beta Fan Club: Announcing Release Candidates Your CI Cares About
@beartype 0.18.0 broke the entire world. @leycec can now admit that to himself while clutching his Maine Coon teddy cat. If your codebase survived @beartype 0.18.0, you deserve an "I Survived @beartype 0.18.0 and All I Got Was This Lousy Badge" badge. The ill-fated @beartype 0.18.0 release cycle that nearly broke my fingers taught me many things: suffering, pain, agony, blah, blah... You know. Just the standard stuff, really.
@beartype 0.18.0 taught me that @beartype has become a lot bigger than me. Other people and people-like AI that are doing meaningful things with their lives and synthetic lives (respectively) now depend on new @beartype releases not throwing up all over everybody.
@beartype ≥ 0.19.0 intends to avoid that throw-up. Several days before releasing any new major version like 0.19.0, 0.20.0, or 0.21.0: ...we see the number sequence I trust
Basically, I'm just doing standard beta releases now. That's all I had to say. Instead, I laboriously enumerated a workflow that doesn't really make sense when you squint at it. Oh, well. This too was wasted time.
The sins of the fathers must never be repeated. Never forget @beartype 0.18.0! Never forgive @leycec! Wait. Shouldn't @leycec be forgiven already at some point!? <-- dat poor guy
@beartype 0.18.0: shocking behind-the-scenes tell-all reveals sordid truth of what went wrong that fateful day
Beartype Release Motto: "Release late. Release rarely. Release safely."
In discussion thread #433, @jedie wisely asks the question we're all wondering:
Indeed, @jedie. It isn't. You're right about everything. I now quote myself like a narcissist. gods what am i become
For ordinary packages, "Release early, release often" is the best possible advice. For @beartype, this is the worst possible advice. Why? Because @beartype is mission-critical. When @beartype breaks, increasingly the entire Python ecosystem breaks. This includes PyTorch – which then transitively includes ChatGPT, OpenAI, Microsoft, and by extension the entirety of American late-stage capitalism. Do we grok the stakes here? The stakes somehow become a whole lot more bigly than "one bald autist has fun smashing code together in a remote Canadian cottage."
I should probably be paid to do this hyper-cuboidal tesseract we call @beartype. Imagine if all neurosurgeons were unpaid volunteers. This is hyperbole, but it's also not. @beartype is the neurosurgeon that fixes bugs during LLM training. Much like Soviets under the USSR, I pretend that I'm being paid by behaving responsibly towards the rest of humanity. "Release early, release often" is what I used to believe. Then I broke PyTorch with the ill-fated @beartype 0.18.0 release. Now, I choose wisely.
The new motto is:
On the bright side, "Release safely." is good! We can all agree. On the dark side, "Release late." and "Release rarely." are both bad. We still agree. But one out of two ain't bad. Right?
New @beartype release will probably land as follows:
0.19.0) once every six months or so.0.19.0rc0) once every month or so.This broadly parallels CPython's shift to a yearly release schedule with intermittent mid-yearly alpha and beta pre-releases. Since @beartype isn't as bigly as CPython, we can and should go faster and harder than CPython on releases – but we can't go that much faster or harder. Realistically speaking, @beartype releases will remain slower than your average open-source Python package.
Sucks, huh? I know and commiserate by blowing smoke out of gigantic nostrils on a picturesque beach.
that feeling when you're only in month 1 of an interminable 6-month release cycle
Hatch +
pyproject.toml: The Build System We Deserved 10 Years Ago, Today@beartype 0.19.0 now sports a sane build system. It's sporty! Somehow, we found the strength to refactor the archaic @beartype 0.18.0 toolchain from
setuptools+setup.py🤮 to Hatch +pyproject.toml🥂 . This includes support for modern packaging standards like PEP 517 and 621.It went great, actually. Thanks for asking. I highly recommend Hatch for all projects – new and curmudgeonly alike. It's like Rust's Cargo, only Python. It actually works, unlike everything else.
Hatch: because you're too bald to fight Python anymore.
@beartype 0.19.0 now gives thanks for this build system it is about to blow up
PyPI Trusted Publishers: Because Deployment Wasn't Hard Enough
@beartype 0.19.0 now sports a sane publishing system. It's less insane! Somehow, we found the stamina to refactor the archaic @beartype 0.18.0 release workflow from antiquated (and unsurprisingly insecure) GitHub Actions tokens 🤮 to PyPI-specific "Trusted Publishers" (i.e., PyPI's modern implementation of OpenID Connect (OIDC)). 🤷
In theory, doing so should resolve the current plethora of "Unverified details" that currently pollutes @beartype's PyPI project page. We're not unverified, PyPI! You're unverified.
In practice, doing so will almost certainly change nothing and thus have no benefit whatsoever. Indeed, doing so will probably prevent our entire release workflow from behaving as expected – further squandering scarce open-source volunteerism for no particularly good reason whatsoever.
Bureaucracy: "What is it good for when @leycec could just be playing video games about robot assassins who insist they meant well instead?"
@beartype 0.19.0: on its way to a PyPI project page near you
You. Are. Beartype.
Announcing all the fave @beartype users from the ashes of our issue tracker:
@posita, @wesselb, @iamrecursion, @patrick-kidger, @langfield, @JelleZijlstra, @RobPasMue, @GithubCamouflaged, @kloczek, @uriyasama, @danielgafni, @JWCS, @rbroderi, @AlanCoding, @tvdboom, @crypdick, @jvesely, @komodovaran, @kaparoo, @MaximilienLC, @fleimgruber, @EtaoinWu, @alexoshin, @gabrieldemarmiesse, @James4Ever0, @NLPShenanigans, @rtbs-dev, @yurivict, @st--, @murphyk, @dosisod, @Rogdham, @alisaifee, @denisrosset, @damarro3, @ruancomelli, @jondequinor, @harshita-gupta, @jakebailey, @denballakh, @jaanli, @creatorrr, @msvensson222, @avolchek, @femtomc, @AdrienPensart, @jakelongo, @Artur-Galstyan, @ArneBachmann, @danielward27, @WeepingClown13, @rbnhd, @radomirgr, @rwiegan, @brettc, @spagdoon0411, @helderco, @paulwouters, @jamesbraza, @dcharatan, @kasium, @AdrienPensart, @sunildkumar, @peske, @mentalisttraceur, @awf, @PhilipVinc, @dcharatan, @empyrealapp, @rlkelly, @KyleKing, @skeggse, @RomainBrault, @pablovela5620, @thiswillbeyourgithub, @WeepingClown13, @JWCS, @Logan-Pageler, @knyazer, @Moosems, @frrad, @minmax, @jaanli, @jonnyhyman, @f-fuchs, @jennydaman, @denballakh, @bionicles, @taranlu-houzz, @adamtheturtle
center right: your codebase. center left: @beartype. everybody else: @beartype's competition, which doesn't of course exist.
Beta Was this translation helpful? Give feedback.
All reactions