Repository navigation
unittest may incorrectly return exit code 5 when setUpClass raises an exception #136442
Description
Bug report
Bug description:
In Python 3.12+, unittest returns exit code 5 when no tests are run and none are skipped. However, if a test class’s setUpClass() method raises any exception (except unittest.SkipTest; related to #106584), no tests are executed, and unittest may incorrectly return exit code 5.
This is misleading: the failure occurred due to an error in class-level setup, not because all tests were skipped.
import unittest
class BugTest(unittest.TestCase):
@classmethod
def setUpClass(cls):
raise ValueError("Simulated setup failure")
def test_example(self):
self.assertTrue(True)python -m unittest bug_test.py
echo $?Actual result:
- Exit code is
5.
Expected result:
- Exit code should be
1, because an error occurred during test setup.
I think we should report the error first, then check whether no tests are run and none are skipped or not.
https://github.com/python/cpython/blob/3.12/Lib/unittest/main.py#L282-L288
Suggested:
if self.exit:
if not self.result.wasSuccessful():
sys.exit(1)
elif self.result.testsRun == 0 and len(self.result.skipped) == 0:
sys.exit(_NO_TESTS_EXITCODE)
else:
sys.exit(0)CPython versions tested on:
3.12
Operating systems tested on:
macOS
Linked PRs
- gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception #136487
- [3.14] gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (GH-136487) #141576
- [3.13] gh-136442: Fix unittest to return exit code 5 when setUpClass raises an exception (GH-136487) #141577
Activity
ZeroIntensity commentedon Jul 9, 2025
Sounds like a reasonable fix. Would you like to submit a PR?
ueshin commentedon Jul 9, 2025
@ZeroIntensity Sure, I will submit a PR. Thanks!
Merge branch 'main' into pythongh-136442
[SPARK-52739][PYTHON][TEST] Apply a patch to unittest.main to report …
Merge remote-tracking branch 'upstream/main' into pythongh-136442
gh-136442: Fix unittest to return exit code 5 when setUpClass raises …
pythongh-136442: Fix unittest to return exit code 5 when setUpClass r…
pythongh-136442: Fix unittest to return exit code 5 when setUpClass r…
[3.13] gh-136442: Fix unittest to return exit code 5 when setUpClass …
[3.14] gh-136442: Fix unittest to return exit code 5 when setUpClass …
pythongh-136442: Fix unittest to return exit code 5 when setUpClass r…