test_tools: test_freeze_simple_script() fails on s390x SLES 3.x: [Errno 2] No such file or directory: (...)

[vstinner]

s390x SLES 3.x:

test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script) ... ERROR

Stdout:
creating the script to be frozen at /tmp/test_python_3vf4i07o/tmp7pw_cxg6/app.py
copying the source tree into /tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython...

(...)

======================================================================
ERROR: test_freeze_simple_script (test.test_tools.test_freeze.TestFreeze.test_freeze_simple_script)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/test/test_tools/test_freeze.py", line 32, in test_freeze_simple_script
    outdir, scriptfile, python = helper.prepare(script, outdir)
                                 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 146, in prepare
    copy_source_tree(srcdir, SRCDIR)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Tools/freeze/test/freeze.py", line 95, in copy_source_tree
    shutil.copytree(oldroot, newroot, ignore=ignore_non_src)
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/shutil.py", line 588, in copytree
    return _copytree(entries=entries, src=src, dst=dst, symlinks=symlinks,
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/Lib/shutil.py", line 542, in _copytree
    raise Error(errors)
shutil.Error: [('/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ', '/tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython/build/test_python_29475æ', "[Errno 2] No such file or directory: '/home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ'")]

Stdout:
creating the script to be frozen at /tmp/test_python_3vf4i07o/tmp7pw_cxg6/app.py
copying the source tree into /tmp/test_python_3vf4i07o/tmp7pw_cxg6/cpython...

----------------------------------------------------------------------
Ran 36 tests in 7.203s

FAILED (errors=1)
test test_tools failed

build: https://buildbot.python.org/all/#/builders/540/builds/6583

Linked PRs

• gh-109615: Fix test_tools.test_freeze SRCDIR #109935
• [3.12] gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) #109950
• [3.11] gh-109615: Fix test_tools.test_freeze SRCDIR (#109935) #109951
• gh-109615: Fix support test_copy_python_src_ignore() #109958
• [3.11] gh-109615: Fix support test_copy_python_src_ignore() (#109958) #109962
• gh-109615: Fix support test_copy_python_src_ignore() on WASM #109970
• [3.11] gh-109615: Fix support test_copy_python_src_ignore() on WASM (GH-109970) #109976
• gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore #110108
• [3.11] gh-109615: Look for 'Modules' as landmark for test_copy_python_src_ignore (GH-110108) #110110
• [3.12] gh-109615: Fix support test_copy_python_src_ignore() (#109958) #110340

[vstinner]

The test fails also on s390x Debian 3.x: https://buildbot.python.org/all/#builders/49/builds/6610

[serhiy-storchaka]

It tries to copy the source tree from /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ, but the correct path is /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/.

[vstinner]

build: https://buildbot.python.org/all/#/builders/540/builds/6583

• Git was run in directory: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build.

• configure command: ./configure --prefix '$(PWD)/target' --with-pydebug

• test.pythoninfo:

  • sysconfig[abs_builddir]: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build
  • sysconfig[srcdir]: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build

• Tests are run in directory: /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_worker_8799æ

Tools/freeze/freeze.py computes:

SRCDIR = os.path.dirname(os.path.dirname(TOOL_ROOT))

It tries to copy the source tree from /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/build/test_python_29475æ, but the correct path is /home/dje/cpython-buildarea/3.x.edelsohn-sles-z/build/.

Oh. Maybe the code defining SRCDIR is wrong.

[vstinner]

I proposed PR #109935 to fix the issue. I'm not sure yet if my fix is correct.

[vstinner]

It seems like freeze copy_source_tree() copies build/ directory which is modified by other tests running in parallel. So it's likely that the copy can fail. I updated my PR #109935 to ignore this directory in the copy. Recently, I fixed a similar issue in venv with __pycache__/ directories: see issue gh-109748.