Regression: Type of literal narrowed when asserting equality to a NewType

[p-sawicki]

Bug Report

When asserting equality between a new type and literal values, the type of the literal appears to be narrowed to the new type. This makes mypy report errors on subsequent comparisons against other new types with --strict.

This behavior starts from ad30be8. cc @hauntsaninja

To Reproduce

from typing import NewType

TEST = 1
MyInt1 = NewType("MyInt1", int)
MyInt2 = NewType("MyInt2", int)

def f(i: MyInt1, j: MyInt2) -> None:
    assert i == TEST
    if j == TEST:
        print("test")

Expected Behavior

No errors

Actual Behavior

error: Non-overlapping equality check (left operand type: "MyInt2", right operand type: "MyInt1")  [comparison-overlap]

Your Environment

• Mypy version used: mypy 1.20.0+dev.ad30be89f146adaae398bd854146952158a53cad and up
• Mypy command-line flags: mypy ~/test.py --strict
• Mypy configuration options from mypy.ini (and other config files): None
• Python version used: 3.13.5

[hauntsaninja]

Oh interesting, hadn't thought much about NewType's / didn't show up in mypy_primer. I will take a look and see if there is a fix that makes sense, but it's not fully clear to me that the new behaviour is entirely undesirable.

Especially since this isn't represented well in mypy_primer or our test suite, curious how minimised your example is from the real-world example, or if there are any other relevant snippets you can share

[p-sawicki]

This came up in a test for a function that could be initialized with different NewTypes. The returned object contained the NewType attribute that the function was initialized with and it was checked at the end. So something like

user = create(user_id=UserId(TEST_ID_NUMBER))
user_id = validate(user)
assert user_id == TEST_ID_NUMBER

team = create(team_id=TeamId(TEST_ID_NUMBER))
team_id = validate(team)
assert team_id == TEST_ID_NUMBER

I think it might make sense to report an error in this case but I found the one that mypy reports now confusing. If I wasn't at least a little familiar with the recent type narrowing changes I would probably be completely lost at trying to figure out how TEST_ID_NUMBER was typed as UserId.

Also, the error points to the line assert team_id == TEST_ID_NUMBER in this example but the fix is actually to change assert user_id == TEST_ID_NUMBER to assert user_id == UserId(TEST_ID_NUMBER) earlier. So maybe there could be some hint pointing to the earlier line.

[hauntsaninja]

I will think about this more, especially the part about diagnostics, which is a good thing to call out.

But in general, the behaviour of NewType is modelled after the behaviour of subclasses (and is basically implemented as such), and these are the semantics I would expect of subclasses:

# mypy: strict-equality, warn-unreachable
class A: ...
class A1(A): ...
class A2(A): ...

def check_a(base: A, a1: A1, a2: A2):
    if a1 == a2:  # E: Non-overlapping equality check (left operand type: "A1", right operand type: "A2")
        reveal_type(a1)  # N: Revealed type is "__main__.A1"
        reveal_type(a2)  # N: Revealed type is "__main__.A2"

    if a1 == base:
        reveal_type(a1)  # N: Revealed type is "__main__.A1"
        reveal_type(base)  # N: Revealed type is "__main__.A1"
        if a2 == base:  # E: Non-overlapping equality check (left operand type: "A2", right operand type: "A1")
            reveal_type(a2)  # N: Revealed type is "__main__.A2"
            reveal_type(base)  # N: Revealed type is "__main__.A1"

from typing import NewType

M1 = NewType("M1", int)
M2 = NewType("M2", int)

def check_m(base: int, m1: M1, m2: M2):
    if m1 == m2:  # E: Non-overlapping equality check (left operand type: "M1", right operand type: "M2")
        reveal_type(m1)  # N: Revealed type is "__main__.M1"
        reveal_type(m2)  # N: Revealed type is "__main__.M2"

    if m1 == base:
        reveal_type(m1)  # N: Revealed type is "__main__.M1"
        reveal_type(base)  # N: Revealed type is "__main__.M1"
        if m2 == base:  # E: Non-overlapping equality check (left operand type: "M2", right operand type: "M1")
            reveal_type(m2)  # N: Revealed type is "__main__.M2"
            reveal_type(base)  # N: Revealed type is "__main__.M1"

[ilevkivskyi]

FWIW I think NewTypes should be special-cased w.r.t. to narrowing, as they are not like real subclasses in some aspects. Here are some motivating examples:

Universal default value:

INVALID: Final = -1
def get_owner(uid: UserId, team_id: TeamId):
    if uid == INVALID:
        if tid == INVALID:  # There should be no error here
            return None

"Multi-level" NewTypes:

Path = NewType("Path", str)  # relative to user directory
NormPath = NewType("NormPath", Path)  # includes user directory

def op(path1: NormPath, path2: Path):
    if path1 == path2:  # Could this be a coincidence?
        ...  # I would be more comfortable if path2 remained a Path

Narrowing from regular superype:

def func(whatever: object, uid: Uid):
    if whatever == uid:
        reveal_type(whatever)  # I think this should be plain int.

Mostly these examples revolve around an (IMO important) guarantee that the only way to create an instance of a NewType is to explicitly wrap a base type value.

[JukkaL]

Also, this looks wrong, and it doesn't involve NewType:

class MyInt(int):
    def m(self): pass

VALUE = 2

def f(x: MyInt) -> None:
    if x == VALUE:
        reveal_type(VALUE)  # MyInt <<-- should be int
        VALUE.m()  # No error (false negative)

Note that at runtime, MyInt(5) == 5 is true.

[hauntsaninja]

Okay, I will add special-casing for NewType. "only way to create an instance of a NewType is to explicitly wrap a base type value" is a principle that makes sense to me. This looks pretty easy, but conflicts with some of my open PRs so I will land those first.

Jukka's example is something else, basically that mypy doesn't see that int implements __eq__ (if it didn't, the narrowing is actually completely fine). There are some improvements I have ready for narrowing related to numeric types that I will land first, then I will look into this. In practice on primer, the main int/str subclass we see is for Enum's, and I made a few improvements related to that in my rewrite.