Skip to content

comparison chaining has wrong source positions in most contexts (python 3.11.0rc2) #95921

Closed

Description

@15r10nk

problem

comparison chaining has wrong source positions in most contexts

>>> cnd = True                                            
>>> a = 1 if 1 in 2 == 3 else 0                                                                
Traceback (most recent call last):                                                              
  File "example.py", line 43, in <module>                                                             
    a = 1 if 1 in 2 == 3 else 0                                              
        ^^^^^^^^^^^^^^^^^^^^^^^                                                                  
TypeError: argument of type 'int' is not iterable  

This bug requires two things to happen:

  1. The comparison chain has to contain more than on comparison the type does not matter (==,in,<=,is,...)
  2. The context has to be something other than an expression

no problem

The following examples work:
only one comparison

Traceback (most recent call last):
  File "example.py", line 4, in <module>
    if 5<"5":
       ^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

inside expression

>>> a=False or 4<"5"<6
Traceback (most recent call last):
  File "example.py", line 15, in <module>
    a=False or 4<"5"<6
               ^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

reason

The problem seems to be the positions in the compiled bytecode:

script:

import dis

def foo():
    if 1<2<"no int":
        return 1
    return 0

for p in dis.get_instructions(foo):
    print(p.positions, p.opname,p.argval)

output (Python 3.11.0rc2+):

Positions(lineno=4, end_lineno=4, col_offset=0, end_col_offset=0) RESUME 0
Positions(lineno=5, end_lineno=5, col_offset=7, end_col_offset=8) LOAD_CONST 1
Positions(lineno=5, end_lineno=5, col_offset=9, end_col_offset=10) LOAD_CONST 2
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) SWAP 2
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) COPY 2
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) COMPARE_OP < # range of the whole if
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) POP_JUMP_FORWARD_IF_FALSE 30
Positions(lineno=5, end_lineno=5, col_offset=11, end_col_offset=19) LOAD_CONST no int
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) COMPARE_OP < # range of the whole if
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) POP_JUMP_FORWARD_IF_FALSE 38
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) JUMP_FORWARD 34
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) POP_TOP None
Positions(lineno=5, end_lineno=6, col_offset=4, end_col_offset=16) JUMP_FORWARD 38
Positions(lineno=6, end_lineno=6, col_offset=15, end_col_offset=16) LOAD_CONST 1
Positions(lineno=6, end_lineno=6, col_offset=15, end_col_offset=16) RETURN_VALUE None
Positions(lineno=7, end_lineno=7, col_offset=11, end_col_offset=12) LOAD_CONST 0
Positions(lineno=7, end_lineno=7, col_offset=11, end_col_offset=12) RETURN_VALUE None

the generated ast looks fine (snippet):

            If(
               test=Compare(
                  left=Constant(
                     value=1,
                     lineno=5,
                     col_offset=7, # correct
                     end_lineno=5,
                     end_col_offset=8),
                  ops=[
                     Lt(),
                     Lt()],
                  comparators=[
                     Constant(
                        value=2,
                        lineno=5,
                        col_offset=9, # correct
                        end_lineno=5,
                        end_col_offset=10),
                     Constant(
                        value='no int',
                        lineno=5,
                        col_offset=11, # correct
                        end_lineno=5,
                        end_col_offset=19)],
                  lineno=5,
                  col_offset=7,
                  end_lineno=5,
                  end_col_offset=19),

other examples

here is a list of all the problems I found so far:

>>> if 4<"5"<6:
...     pass
...
Traceback (most recent call last):
  File "example.py", line 20, in <module>
    if 4<"5"<6:
TypeError: '<' not supported between instances of 'int' and 'str'
>>> while 4<"5"<6:
...     pass
...
Traceback (most recent call last):
  File "example.py", line 24, in <module>
    while 4<"5"<6:
TypeError: '<' not supported between instances of 'int' and 'str'

>>> assert 4<"5"<6
Traceback (most recent call last):
  File "example.py", line 27, in <module>
    assert 4<"5"<6
TypeError: '<' not supported between instances of 'int' and 'str'

>>> a=2+(5 if 4<"5"<6 else 3)
Traceback (most recent call last):
  File "example.py", line 29, in <module>
    a=2+(5 if 4<"5"<6 else 3)
         ^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

>>> a=[a for a in [1] if 4<"5"<a ]
Traceback (most recent call last):
  File "example.py", line 34, in <listcomp>
    a=[a for a in [1] if 4<"5"<a ]
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

>>> a={a for a in [1] if 4<"5"<a }
Traceback (most recent call last):
  File "example.py", line 35, in <setcomp>
    a={a for a in [1] if 4<"5"<a }
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

>>> a={a:a for a in [1] if 4<"5"<a }
Traceback (most recent call last):
  File "example.py", line 36, in <dictcomp>
    a={a:a for a in [1] if 4<"5"<a }
      ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

>>> a=list((a for a in [1] if 4<"5"<a ))
Traceback (most recent call last):
  File "example.py", line 37, in <genexpr>
    a=list((a for a in [1] if 4<"5"<a ))
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

>>> cnd = True
>>> a = 1 if 1 in 2 == 3 else 0
Traceback (most recent call last):
  File "example.py", line 43, in <module>
    a = 1 if 1 in 2 == 3 else 0
        ^^^^^^^^^^^^^^^^^^^^^^^
TypeError: argument of type 'int' is not iterable

>>> if True and 4<"5"<6:
...     pass
...
Traceback (most recent call last):
  File "example.py", line 45, in <module>
    if True and 4<"5"<6:
TypeError: '<' not supported between instances of 'int' and 'str'

expected result

The positions should always match the ast-node range:

>>> a=list((a for a in [1] if 4<"5"<a ))
Traceback (most recent call last):
  File "example.py", line 37, in <genexpr>
    a=list((a for a in [1] if 4<"5"<a ))
                              ^^^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

or, even better only the failing comparison:

>>> a=list((a for a in [1] if 4<"5"<a ))
Traceback (most recent call last):
  File "example.py", line 37, in <genexpr>
    a=list((a for a in [1] if 4<"5"<a ))
                              ^^^^^
TypeError: '<' not supported between instances of 'int' and 'str'

Activity

  1. changed the title [-]comparison chaining in assertions produces wrong source positions in bytecode[/-] [+]comparison chaining has wrong source positions in most contexts (python 3.11.0rc2)[/+] on Sep 18, 2022
  2. 15r10nk commented on Sep 18, 2022

    @15r10nk
    ContributorAuthor

    @brandtbucher I think this is related to #93691.
    Do you know if there someone already working on this?

    Does it makes sense for me to look into it? I have (almost) no cpython core developer experience and it would take me probably far longer than someone else.

  3. brandtbucher commented on Sep 18, 2022

    @brandtbucher
    Member

    Thanks for the detailed report. This issue seems to stem from the fact that we have two implementations of comparisons in the bytecode compiler: one generic version (compiler_compare) and one optimized for branching conditions (in compiler_jump_if). The latter doesn’t update the location to the current node, which is probably an oversight.

    I’ll have a fix up either today or tomorrow. It will probably need to wait for 3.11.1, though.

  4. self-assigned this
    on Sep 18, 2022
  5. added 5 commits that reference this issue on Sep 20, 2022
  6. added 2 commits that reference this issue on Sep 20, 2022
  7. brandtbucher commented on Sep 20, 2022

    @brandtbucher
    Member

    Thanks again for the great bug report!

  8. 5 remaining items

  9. brandtbucher commented on Dec 21, 2022

    @brandtbucher
    Member

    I have a working fix for 3.11 and 3.10, just need to write a decent regression test. I'll pick it back up tomorrow.

  10. 15r10nk commented on Jan 1, 2023

    @15r10nk
    ContributorAuthor

    can you please point me to the commit which fixes this issue. I would like to run the executing testsuite against it.

  11. hauntsaninja commented on Sep 7, 2023

    @hauntsaninja
    Contributor

    Was going through older issues, looks like this is still an issue on 3.11. Brandt do you still have your branch?

  12. StanFromIreland commented on Aug 18, 2025

    @StanFromIreland
    Member

    Is there anything left to do here, the reported issue and later regression are fixed from my testing?

  13. removed
    pendingThe issue will be closed if no feedback is provided
    on Nov 28, 2025
  14. picnixz commented on Nov 28, 2025

    @picnixz
    Member

    I'm going to close this one as completed. If there are still problems, let's open a new issue (but add a ref to this one).

  15. hauntsaninja commented on Nov 29, 2025

    @hauntsaninja
    Contributor

    There is still an issue on Python 3.11, but I guess that is no longer in bugfix :-) Thanks 15r10nk and brandtbucher!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

interpreter-core(Objects, Python, Grammar, and Parser dirs)type-bugAn unexpected behavior, bug, or error

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions