Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

support pep 673 Self type #11871

Open
zzzeek opened this issue Dec 29, 2021 · 13 comments
Open

support pep 673 Self type #11871

zzzeek opened this issue Dec 29, 2021 · 13 comments
Labels
affects-typeshed feature meta topic-self-types

Comments

@zzzeek
Copy link

@zzzeek zzzeek commented Dec 29, 2021

Feature

Support the pep-673 Self type

pep is at: https://www.python.org/dev/peps/pep-0673/

a rough idea of it is that this code:

from typing import Self

class Widget:
    def add_thing(self, value: str) -> Self:
        self.value = value
        return self

is quasi-equivalent to:

from typing import TypeVar

TWidget = TypeVar("TWidget", bound="Widget")

class Widget:
    def add_thing(self: TWidget, value: str) -> TWidget:
        self.value = value
        return self

Pitch

the appeal of the pep is that for the very common pattern of method-chained object construction, methods can easily indicate they return an object of the same type upon which the method is being invoked. For example if I made class SubWidget(Widget), the SubWidget.add_thing() method reports SubWidget as the return type automatically. The pattern using TypeVar seems to be more or less equivlant, but is more verbose requiring the declaration of TypeVar objects per class hierarchy as well as that it has to be explicitly present on the "self" parameter in methods.

We are looking for this feature to make our job of integrating typing into SQLAlchemy an easier job. it seems to be a fairly straightforward translation between two idioms.

cc @CaselIT

@zzzeek zzzeek added the feature label Dec 29, 2021
@erictraut
Copy link

@erictraut erictraut commented Dec 29, 2021

FWIW, both pyright and pyre have implemented this feature in its currently-documented form if you want to play with it.

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Dec 30, 2021

FWIW, both pyright and pyre have implemented this feature in its currently-documented form if you want to play with it.

I know :) it's great. I hate when people email SQLAlchemy with a feature request and say " django does it! " :) but yeah, just trying to see if people using mypy can take advantage also in the near term or if there's no roadmap for this one here.

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Dec 30, 2021

There is already a draft PR working on adding support for PEP 673: #11666

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Dec 30, 2021

@AlexWaygood we are literally trying to choose right now if SQLAlchemy 2.0, for which we hope to have beta releases by the end of Q1 2022, can use pep673 Self or not. I know this is normally not something projects can estimate but is it your experience a PR like the one suggested in #11666 can move towards being merged within a few months or is it likely it could stall for a long time?

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Dec 30, 2021

at the same time, if we do release 2.0 using pep 673, mypy users would be more interested in seeing this PR completed so for that reason alone I might decide to use it now.

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Dec 30, 2021

Hey @zzzeek -- unfortunately, I'm not really the right person to ask about this :// I'm not actually on the mypy core team, and have only contributed a few documentation fixes to mypy -- I tend to contribute more towards the CPython & typeshed repos, I just stalk the mypy issue tracker from time to time 😄 Perhaps @hauntsaninja might be able to shed some more light?

Having said that, I think the approach typeshed has taken would probably serve you well! You could define a single TypeVar in a standalone module like this (let's call it typing_utils):

# typing_utils.py
from typing import TypeVar

Self = TypeVar('Self')

Then, in the rest of your code base, you could use that TypeVar like this:

# foo.py
from typing_utils import Self

class Foo:
    def returns_self(self: Self) -> Self:
        return self
    @classmethod
    def returns_cls(cls: type[Self]) -> type[Self]:
        return cls

In that way, you'd get type annotations that work perfectly for now; and if PEP 673 is accepted and implemented, you could quite easily run a single script using AST to clean up your code base, converting Foo into the following:

# foo.py
from typing_extensions import Self

class Foo:
    def returns_self(self) -> Self:
        return self
    @classmethod
    def returns_cls(cls) -> type[Self]:
        return cls

The old version of Foo (using pre-PEP 673 syntax) and the new version (using PEP 673 syntax) would have semantically identical type annotations from mypy's perspective.

@hauntsaninja
Copy link
Collaborator

@hauntsaninja hauntsaninja commented Dec 30, 2021

I also just stalk the mypy issue tracker from time to time :-) I don't think there is anyone who gets paid time to work on mypy, so there isn't a concrete roadmap. In my experience, you can expect a lag time of a couple months from contributor-completes-PR to mypy-release-that-contains-said-PR.

Re PEP 673 specifically:
I haven't looked at the linked PR, but in the abstract PEP 673 doesn't seem like a hard thing to implement.
I'll also note that PEP 673 has not yet been officially accepted, so I'd keep that in mind when thinking about what promises you're comfortable making to SQLAlchemy users. AlexWaygood's suggestion of just using a normal type variable (and ignoring the bound) + codemodding at a later time is a good option to consider.

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Dec 31, 2021

Having said that, I think the approach typeshed has taken would probably serve you well! You could define a single TypeVar in a standalone module like this (let's call it typing_utils):

# typing_utils.py
from typing import TypeVar

Self = TypeVar('Self')

OK, we can certainly go with that, the approach we have at the moment is copied from how pep-673 illustrates it, with individual TypeVars that are each "bound" to the target type. There should be no reason that's needed though, right? that is, the "Self" that's not bound will work just as well?

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Dec 31, 2021

For situations like this, the bound argument is, unfortunately, still necessary, yes — in an ideal world, mypy should be able to understand that TypeVars in that situation are implicitly bound, but that hasn't been implemented yet.

It depends whether you're using mypy to check function implementations, or just writing stubs, though. If you're just writing stubs, you can get away with a lot more, and the bound argument is usually not required 🙂

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Dec 31, 2021

this is inline typing in SQLAlchemy for 2.0 where we will no longer use stubs.

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jan 1, 2022

this is inline typing in SQLAlchemy for 2.0 where we will no longer use stubs.

In which case, yeah, you'll often need the bound argument, unfortunately, or you'll find mypy will erroneously complain about things you're doing inside your functions 😕 but, it still may be worth adding some kind of 'tag' to these TypeVar names (such as having the "Self" suffix with all of them) so that you can easily identify them in the future and change them quickly if/when PEP 673 is accepted.

from typing import TypeVar

FooSelf = TypeVar('F', bound='Foo')

class Foo:
    bar: int
    
    def change_bar(self: FooSelf, x: int) -> FooSelf:
        self.bar = x  # mypy erroneously complains about this line if the TypeVar is not bound 
        return self

@zzzeek
Copy link
Author

@zzzeek zzzeek commented Jan 2, 2022

yes that's what we've been doing, thanks for the tips!

@AlexWaygood
Copy link
Member

@AlexWaygood AlexWaygood commented Jan 2, 2022

yes that's what we've been doing, thanks for the tips!

No problem!

@97littleleaf11 97littleleaf11 added the meta label Feb 20, 2022
@AlexWaygood AlexWaygood added affects-typeshed topic-self-types labels Apr 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
affects-typeshed feature meta topic-self-types
Projects
None yet
Development

No branches or pull requests

5 participants