Repository navigation
test_decimal failed in pgo #91821
Description
Bug report
configure with --enable-optimizations and then make run_profile_task, fail occurred.
./python -m test --pgo --timeout=1200
0:00:00 load avg: 3.02 Run tests sequentially (timeout: 20 min)
0:00:00 load avg: 3.02 [ 1/44] test_array
0:00:01 load avg: 3.02 [ 2/44] test_base64
0:00:02 load avg: 3.02 [ 3/44] test_binascii
0:00:02 load avg: 3.02 [ 4/44] test_binop
0:00:02 load avg: 3.02 [ 5/44] test_bisect
0:00:02 load avg: 3.02 [ 6/44] test_bytes
0:00:10 load avg: 2.71 [ 7/44] test_bz2
0:00:11 load avg: 2.71 [ 8/44] test_cmath
0:00:12 load avg: 2.71 [ 9/44] test_codecs
0:00:14 load avg: 2.65 [10/44] test_collections
0:00:17 load avg: 2.65 [11/44] test_complex
0:00:17 load avg: 2.65 [12/44] test_dataclasses
0:00:18 load avg: 2.65 [13/44] test_datetime
0:00:24 load avg: 2.44 [14/44] test_decimal
test test_decimal failed
0:00:32 load avg: 2.22 [15/44] test_difflib -- test_decimal failed (1 failure)
0:00:35 load avg: 2.12 [16/44] test_embed
0:00:35 load avg: 2.12 [17/44] test_float
......
Total duration: 2 min 13 sec
Tests result: FAILURE
After then I tried to reproduce the error.
I found make buildbottest TESTOPTS="-j4 -uall,-cpu test_decimal" success! The specific commands it executes are:
./python -u -W default -bb -E -m test -r -w -j 1 -u all -W --slowest --fail-env-changed --timeout=1200 -j4 -uall,-cpu test_decimal
...
======================================================================
FAIL: test_from_tuple (test.test_decimal.CWhitebox.test_from_tuple)
----------------------------------------------------------------------
Traceback (most recent call last):
File "/data/self/code/mine/Lib/test/test_decimal.py", line 5538, in test_from_tuple
self.assertEqual(str(c.create_decimal(x)), '-0E-1000007')
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: '-0E-1000026' != '-0E-1000007'
- -0E-1000026
? ^^
+ -0E-1000007
? ^^
......
1 test OK.
10 slowest tests:
- test_decimal: 11.3 sec
1 re-run test:
test_decimal
Total duration: 11.7 sec
Tests result: FAILURE then SUCCESS
Because pgo mode does not have the -w parameter, it will not re-run after fail.
Finally, I reset commit and locate that gh-91291 causing the problem.
Activity
penguin-wwy commentedon Apr 22, 2022
When I revert #32242, everything was fine.
dignissimus commentedon Apr 22, 2022
On my own computer I can reproduce this without configuring with
--enable-optimizationsFidget-Spinner commentedon Apr 22, 2022
I also get the same error on Windows (no PGO, debug build) when running test_decimal.
I'm curious as to why our CI is still passing.
JelleZijlstra commentedon Apr 22, 2022
I can reproduce the test failure on MacOS, even in debug mode, and even with #32242 reverted. However, it reproduces only with
./python.exe -m unittest test.test_decimal.CWhitebox.test_from_tupleand passes with./python.exe -m test test.test_decimal -m '*test_from_tuple*'. The difference is in theprecfield of the decimal context: it must be 9 for the test to succeed, but in the unittest invocation it's 28. The test module'sinit()function sets it to 9, but apparently that sometimes gets lost.pythongh-91821: Make test succeed consistently
JelleZijlstra commentedon Apr 22, 2022
#77931 (PR #7355) fixed a previous similar problem with this test; cc @serhiy-storchaka.
emaste commentedon Apr 22, 2022
Fails also on FreeBSD (no PGO)
gh-91821: Make decimal test succeed consistently (#91825)
JelleZijlstra commentedon Apr 24, 2022
I merged #91825 which should make this test pass consistently.
As a more general solution, I think we should check the decimal global context in
libregrtest.save_env, to make sure that it doesn't change between tests.