`test_add_python_opts` fails when executed with `PYTHONDEVMODE=1` in the environment

[pradyunsg]

Bug report

Bug description:

In #148507 (for #148487), we ended up introducing a failure when the test suite is executed in an environment with PYTHONDEVMODE=1.

❯ PYTHONDEVMODE=1 ./python.exe -m test -v test_regrtest -m test_add_python_opts
== CPython 3.16.0a0 (heads/main-dirty:398d7e1d130, May 18 2026, 17:38:05) [Clang 17.0.0 (clang-1700.6.3.2)]
== macOS-15.7.5-arm64-arm-64bit-Mach-O little-endian
== Python build: debug
== cwd: /Users/pradyunsg/Developer/github/cpython/build/test_python_worker_45275æ
== CPU count: 8
== encodings: locale=UTF-8 FS=utf-8
== resources: all test resources are disabled, use -u option to unskip tests

Using random seed: 1985948743
0:00:00 load avg: 2.29 Run 1 test sequentially in a single process
0:00:00 load avg: 2.29 [1/1] test_regrtest
test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) ... 
  test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) (opt='--fast-ci') ... FAIL
  test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) (opt='--slow-ci') ... FAIL

======================================================================
FAIL: test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) (opt='--fast-ci')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pradyunsg/Developer/github/cpython/Lib/test/test_regrtest.py", line 2292, in test_add_python_opts
    self.check_add_python_opts(opt)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/pradyunsg/Developer/github/cpython/Lib/test/test_regrtest.py", line 2287, in check_add_python_opts
    self.assertEqual(proc.returncode, 0, proc)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 0 : CompletedProcess(args=['/Users/pradyunsg/Developer/github/cpython/python.exe', '-m', 'test', '--fast-ci', '--testdir=/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym', 'test_regrtest_noop1'], returncode=2, stdout='+ /Users/pradyunsg/Developer/github/cpython/python.exe -u -bb -E -m test --fast-ci --testdir=/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym test_regrtest_noop1 --dont-add-python-opts\nUsing random seed: 3098864716\n0:00:00 load avg: 2.29 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)\n0:00:00 load avg: 2.29 [1/1/1] test_regrtest_noop1 failed (2 failures)\ntest_config (test_regrtest_noop1.WorkerTests.test_config) ... FAIL\ntest_python_opts (test_regrtest_noop1.WorkerTests.test_python_opts) ... FAIL\n\n======================================================================\nFAIL: test_config (test_regrtest_noop1.WorkerTests.test_config)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop1.py", line 19, in test_config\n    self.assertEqual(config_get(\'warnoptions\'),\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n======================================================================\nFAIL: test_python_opts (test_regrtest_noop1.WorkerTests.test_python_opts)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop1.py", line 32, in test_python_opts\n    self.assertEqual(sys.warnoptions,\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n----------------------------------------------------------------------\nRan 2 tests in 0.004s\n\nFAILED (failures=2)\ntest test_regrtest_noop1 failed\n\n== Tests result: FAILURE ==\n\n10 slowest tests:\n- test_regrtest_noop1: 144 ms\n\n1 test failed:\n    test_regrtest_noop1\n\n0:00:00 load avg: 2.29 Re-running 1 failed tests in verbose mode in subprocesses\n0:00:00 load avg: 2.29 Run 1 test in parallel using 1 worker process (timeout: 10 min, worker timeout: 15 min)\n0:00:00 load avg: 2.29 [1/1/1] test_regrtest_noop1 failed (2 failures)\nRe-running test_regrtest_noop1 in verbose mode (matching: test_config, test_python_opts)\ntest_config (test_regrtest_noop1.WorkerTests.test_config) ... FAIL\ntest_python_opts (test_regrtest_noop1.WorkerTests.test_python_opts) ... FAIL\n\n======================================================================\nFAIL: test_config (test_regrtest_noop1.WorkerTests.test_config)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop1.py", line 19, in test_config\n    self.assertEqual(config_get(\'warnoptions\'),\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n======================================================================\nFAIL: test_python_opts (test_regrtest_noop1.WorkerTests.test_python_opts)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop1.py", line 32, in test_python_opts\n    self.assertEqual(sys.warnoptions,\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n----------------------------------------------------------------------\nRan 2 tests in 0.004s\n\nFAILED (failures=2)\ntest test_regrtest_noop1 failed\n1 test failed again:\n    test_regrtest_noop1\n\n== Tests result: FAILURE then FAILURE ==\n\n10 slowest tests:\n- test_regrtest_noop1: 144 ms\n\n1 re-run test:\n    test_regrtest_noop1\n\n1 test failed:\n    test_regrtest_noop1\n\nTotal duration: 291 ms\nTotal tests: run=4 failures=4\nTotal test files: run=2/1 failed=1 rerun=1\nResult: FAILURE then FAILURE\n')

======================================================================
FAIL: test_add_python_opts (test.test_regrtest.ArgsTestCase.test_add_python_opts) (opt='--slow-ci')
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/Users/pradyunsg/Developer/github/cpython/Lib/test/test_regrtest.py", line 2292, in test_add_python_opts
    self.check_add_python_opts(opt)
    ~~~~~~~~~~~~~~~~~~~~~~~~~~^^^^^
  File "/Users/pradyunsg/Developer/github/cpython/Lib/test/test_regrtest.py", line 2287, in check_add_python_opts
    self.assertEqual(proc.returncode, 0, proc)
    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^
AssertionError: 2 != 0 : CompletedProcess(args=['/Users/pradyunsg/Developer/github/cpython/python.exe', '-m', 'test', '--slow-ci', '--testdir=/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym', 'test_regrtest_noop2'], returncode=2, stdout='+ /Users/pradyunsg/Developer/github/cpython/python.exe -u -bb -E -m test --slow-ci --testdir=/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym test_regrtest_noop2 --dont-add-python-opts\nUsing random seed: 2729056636\n0:00:00 load avg: 2.29 Run 1 test in parallel using 1 worker process (timeout: 20 min, worker timeout: 25 min)\n0:00:00 load avg: 2.29 [1/1/1] test_regrtest_noop2 failed (2 failures)\ntest_config (test_regrtest_noop2.WorkerTests.test_config) ... FAIL\ntest_python_opts (test_regrtest_noop2.WorkerTests.test_python_opts) ... FAIL\n\n======================================================================\nFAIL: test_config (test_regrtest_noop2.WorkerTests.test_config)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop2.py", line 19, in test_config\n    self.assertEqual(config_get(\'warnoptions\'),\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n======================================================================\nFAIL: test_python_opts (test_regrtest_noop2.WorkerTests.test_python_opts)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop2.py", line 32, in test_python_opts\n    self.assertEqual(sys.warnoptions,\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n----------------------------------------------------------------------\nRan 2 tests in 0.004s\n\nFAILED (failures=2)\ntest test_regrtest_noop2 failed\n\n== Tests result: FAILURE ==\n\n10 slowest tests:\n- test_regrtest_noop2: 135 ms\n\n1 test failed:\n    test_regrtest_noop2\n\n0:00:00 load avg: 2.29 Re-running 1 failed tests in verbose mode in subprocesses\n0:00:00 load avg: 2.29 Run 1 test in parallel using 1 worker process (timeout: 20 min, worker timeout: 25 min)\n0:00:00 load avg: 2.29 [1/1/1] test_regrtest_noop2 failed (2 failures)\nRe-running test_regrtest_noop2 in verbose mode (matching: test_config, test_python_opts)\ntest_config (test_regrtest_noop2.WorkerTests.test_config) ... FAIL\ntest_python_opts (test_regrtest_noop2.WorkerTests.test_python_opts) ... FAIL\n\n======================================================================\nFAIL: test_config (test_regrtest_noop2.WorkerTests.test_config)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop2.py", line 19, in test_config\n    self.assertEqual(config_get(\'warnoptions\'),\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n======================================================================\nFAIL: test_python_opts (test_regrtest_noop2.WorkerTests.test_python_opts)\n----------------------------------------------------------------------\nTraceback (most recent call last):\n  File "/var/folders/y1/j465wvf92vs938kmgqh63bj80000gn/T/tmpnnvrvyym/test_regrtest_noop2.py", line 32, in test_python_opts\n    self.assertEqual(sys.warnoptions,\n    ~~~~~~~~~~~~~~~~^^^^^^^^^^^^^^^^^\n                     [\'error\', \'error::BytesWarning\'])\n                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^\nAssertionError: Lists differ: [\'error::BytesWarning\'] != [\'error\', \'error::BytesWarning\']\n\nFirst differing element 0:\n\'error::BytesWarning\'\n\'error\'\n\nSecond list contains 1 additional elements.\nFirst extra element 1:\n\'error::BytesWarning\'\n\n- [\'error::BytesWarning\']\n+ [\'error\', \'error::BytesWarning\']\n?  +++++++++\n\n\n----------------------------------------------------------------------\nRan 2 tests in 0.004s\n\nFAILED (failures=2)\ntest test_regrtest_noop2 failed\n1 test failed again:\n    test_regrtest_noop2\n\n== Tests result: FAILURE then FAILURE ==\n\n10 slowest tests:\n- test_regrtest_noop2: 135 ms\n\n1 re-run test:\n    test_regrtest_noop2\n\n1 test failed:\n    test_regrtest_noop2\n\nTotal duration: 273 ms\nTotal tests: run=4 failures=4\nTotal test files: run=2/1 failed=1 rerun=1\nResult: FAILURE then FAILURE\n')

----------------------------------------------------------------------
Ran 1 test in 1.068s

FAILED (failures=2)
test test_regrtest failed
0:00:01 load avg: 2.29 [1/1/1] test_regrtest failed (2 failures)

== Tests result: FAILURE ==

1 test failed:
    test_regrtest

Total duration: 1.2 sec
Total tests: run=1 (filtered) failures=2
Total test files: run=1/1 (filtered) failed=1
Result: FAILURE

This happens because of the following block's conditional failing, breaking the expectation of the assertion in the test:

cpython/Lib/test/libregrtest/main.py

Lines 657 to 663 in 43c3438

	# Add warnings filter 'error', unless the user specified a different
	# filter. Ignore BytesWarning since it's controlled by '-b' below.
	if not [
	opt for opt in sys.warnoptions
	if not opt.endswith("::BytesWarning")
	]:
	python_opts.extend(('-W', 'error'))

CPython versions tested on:

3.13, 3.14, 3.15, 3.16, CPython main branch

Operating systems tested on:

macOS

Linked PRs

• gh-150046: Pass -E to subprocesses from check_add_python_opts #150089

[pradyunsg]

/cc @StanFromIreland @vstinner