Typing: undocumented behaviour change for protocols decorated with `@final` and `@runtime_checkable` in 3.11

[AlexWaygood]

Python 3.11 introduced an undocumented behaviour change for protocols decorated with both @final and @runtime_checkable. On 3.10:

>>> from typing import *
>>> @final
... @runtime_checkable
... class Foo(Protocol):
...     def bar(self): ...
...
>>> class Spam:
...     def bar(self): ...
...
>>> issubclass(Spam, Foo)
True
>>> isinstance(Spam(), Foo)
True

On 3.11:

>>> from typing import *
>>> @final
... @runtime_checkable
... class Foo(Protocol):
...     def bar(self): ...
...
>>> class Spam:
...     def bar(self): ...
...
>>> issubclass(Spam, Foo)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "<frozen abc>", line 123, in __subclasscheck__
  File "C:\Users\alexw\coding\cpython\Lib\typing.py", line 1547, in _proto_hook
    raise TypeError("Protocols with non-method members"
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: Protocols with non-method members don't support issubclass()
>>> isinstance(Spam(), Foo)
False

This is because, following 0bbf30e (by @JelleZijlstra), the @final decorator sets a __final__ attribute wherever it can, so that it is introspectable by runtime tools. But the runtime-checkable-protocol isinstance() machinery doesn't know anything about the __final__ attribute, so it assumes that the __final__ attribute is just a regular protocol member.

This should be pretty easy to fix: we just need to add __final__ to the set of "special attributes" ignored by runtime-checkable-protocol isinstance()/issubclass() checks here:

cpython/Lib/typing.py

Lines 1906 to 1909 in 848bdbe

	_TYPING_INTERNALS = frozenset({
	'__parameters__', '__orig_bases__', '__orig_class__',
	'_is_protocol', '_is_runtime_protocol'
	})

@JelleZijlstra do you agree with that course of action?

Linked PRs

• gh-103171: Document and test behaviour change in 3.11 for runtime-checkable protocols decorated with @final #103173
• [3.11] gh-103171: Revert undocumented behaviour change for runtime-checkable protocols decorated with @final #105445
• gh-103171: Forward-port new tests for runtime-checkable protocols decorated with @final #105473
• [3.12] gh-103171: Forward-port new tests for runtime-checkable protocols decorated with @final (GH-105473) #105474

[JelleZijlstra]

My first instinct is to leave this as is. Marking Protocols as final is a weird thing to do in the first place (the whole point of protocols is to subclass from them, though usually implicitly). Adding more names to that _TYPING_INTERNALS list is risky because if people use those names in their protocols, we'll silently ignore them. You could imagine a Protocol that is meant to match only classes decorated with @final.

[JelleZijlstra]

This might be a bigger issue for PEP-702 though: what if you want to @deprecated a runtime-checkable Protocol? That makes more sense than making it final, so I'll amend the PEP to say that __deprecated__ should be ignored by @runtime_checkable.

[AlexWaygood]

My first instinct is to leave this as is. Marking Protocols as final is a weird thing to do in the first place (the whole point of protocols is to subclass from them, though usually implicitly). Adding more names to that _TYPING_INTERNALS list is risky because if people use those names in their protocols, we'll silently ignore them. You could imagine a Protocol that is meant to match only classes decorated with @final.

Sure. In that case, I think we should probably document the behaviour change with a .. versionchanged notice in the docs somewhere — sound good?

[JelleZijlstra]

Sure, that's fine.

[sobolevn]

I mark all things as @final by default. Because inside an internal code-base no things should be extended without a good reason.

So, I would consider this as a bug. __final__ is an internal thing that should not be exposed to users and should not affect the runtime of regular operations.

[AlexWaygood]

__final__ is an internal thing that should not be exposed to users and should not affect the runtime of regular operations.

I'm not sure that makes sense. The fact that @final sets the __final__ attribute is documented, and I don't think we ever intended it to be an internal thing. typing.py doesn't need the attribute to be set at all; the only reason why we introduced the behaviour in 3.11 where it sets the attribute wherever possible was so that third-party tools could more easily introspect whether a method or class had been decorated with @final. That's the opposite of it being an internal thing. https://docs.python.org/3/library/typing.html#typing.final

Personally, I wasn't initially sure this new behaviour makes sense for isinstance(), but I can see arguments either way. What really concerns me is the new behaviour with issubclass(), which seems pretty unfortunate to me.

[AlexWaygood]

@ilevkivskyi, do you have any opinions on how @final and @runtime_checkable should interact at runtime?