Python launcher invoked with command "py" fails to start process

[kai2nenobu]

Bug report

When I invoke a python launcher of version 3.11.0 with "py" command, a python launcher fails to start process.

Step to reproduce

1. Install python 3.11.0 with py launcher for all users.

2. Invoke "py" in command prompt.

3. I expect to launch a python interpreter, but got a error as below

   >"py"
   Unable to create process using 'C:\Python311\py"':  The system cannot find the file specified.

   Debug output when enable PYLAUNCHER_DEBUG

   >"py"
   argv0: py
   version: 3.11.0
   SearchInfo.originalCmdLine: "py"
   SearchInfo.restOfCmdLine:
   SearchInfo.executablePath: (null)
   SearchInfo.scriptFile: (null)
   SearchInfo.executable: py"
   SearchInfo.executableArgs: (null)
   SearchInfo.company: (null)
   SearchInfo.tag: (null)
   SearchInfo.oldStyleTag: False
   SearchInfo.lowPriorityTag: False
   SearchInfo.allowDefaults: False
   SearchInfo.allowExecutableOverride: False
   SearchInfo.windowed: False
   SearchInfo.list: False
   SearchInfo.listPaths: False
   SearchInfo.help: False
    -V:3.11          C:\Python311\python.exe
    -V:3.10          C:\Python310\python.exe
    -V:3.9           C:\Python39\python.exe
    -V:3.7           C:\Python37\python.exe
   env.company: PythonCore
   env.tag: 3.11
   # about to run: C:\Python311\py"
   Unable to create process using 'C:\Python311\py"':  The system cannot find the file specified.

I think this error seems to be similar to #95285. If I invoke py or "py.exe", a python interpreter starts well as I expected.

>py
Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

>"py.exe"
Python 3.11.0 (main, Oct 24 2022, 18:26:48) [MSC v.1933 64 bit (AMD64)] on win32
Type "help", "copyright", "credits" or "license" for more information.
>>>

In version 3.10.8, all of "py", py and "py.exe" start a python interpreter correctly.

Your environment

• CPython versions tested on: 3.11.0
• Operating system and architecture: Windows 10 (10.0.19045.2310)/amd64

Linked PRs

• gh-99442: Fix handling in py.exe launcher when argv[0] does not include a file extension #99542
• [3.11] gh-99442: Fix handling in py.exe launcher when argv[0] does not include a file extension (GH-99542) #99579

[kai2nenobu]

This error causes a failure to build MSI installer between version 3.7 and 3.10. (Example of GitHub Actions build failure in my personal project).

This is because WiX invokes "py" -m pip install -U blurb during build for exe.wixproj and recent GitHub-Hosted runner uses Python Launcher of version 3.11.0.

[pfmoore]

Confirmed, I am seeing the same issue (see the linked rust issue report above).

Command: "py" "-V"
argv0: py
version: 3.11.0
SearchInfo.originalCmdLine: "py" -V
SearchInfo.restOfCmdLine:  -V
SearchInfo.executablePath: (null)
SearchInfo.scriptFile: (null)
SearchInfo.executable: py"
SearchInfo.executableArgs: (null)
SearchInfo.company: (null)
SearchInfo.tag: (null)
SearchInfo.oldStyleTag: False
SearchInfo.lowPriorityTag: False
SearchInfo.allowDefaults: False
SearchInfo.allowExecutableOverride: False
SearchInfo.windowed: False
SearchInfo.list: False
SearchInfo.listPaths: False
SearchInfo.help: False
 -V:3.11          C:\Users\Gustav\AppData\Local\Programs\Python\Python311\python.exe
 -V:3.10          C:\Users\Gustav\AppData\Local\Programs\Python\Python310\python.exe
 -V:3.9           C:\Users\Gustav\AppData\Local\Programs\Python\Python39\python.exe
 -V:3.8           C:\Users\Gustav\AppData\Local\Programs\Python\Python38\python.exe
env.company: PythonCore
env.tag: 3.11
# about to run: C:\Users\Gustav\AppData\Local\Programs\Python\Python311\py" -V
Unable to create process using 'C:\Users\Gustav\AppData\Local\Programs\Python\Python311\py" -V': The system cannot find the file specified.

[eryksun]

In parseCommandLine(), the value of end points to just past the closing double quote if there's no "." found in the tail component. findArgumentEnd() could be modified to exclude the double quote from the end, since the double quote at the end isn't actually part of the command-line argument. It's a syntax character for argument parsing.

cpython/PC/launcher2.c

Lines 565 to 580 in 88385b8

	const wchar_t *tail = findArgumentEnd(search->originalCmdLine, -1);
	const wchar_t *end = tail;
	search->restOfCmdLine = tail;
	while (--tail != search->originalCmdLine) {
	if (*tail == L'.' && end == search->restOfCmdLine) {
	end = tail;
	} else if (*tail == L'\\' || *tail == L'/') {
	++tail;
	break;
	}
	}
	if (tail == search->originalCmdLine && tail[0] == L'"') {
	++tail;
	}
	// Without special cases, we can now fill in the search struct
	int tailLen = (int)(end ? (end - tail) : wcsnlen_s(tail, MAXLEN));

---

A couple of issues in findArgumentLength(), which could also be resolved here without creating a new issue for them:

• If the buffer doesn't start with a double quote, it needs to search for white space -- at least tab ("\t") in addition to space (" ").
• Since findArgumentLength() is used by findArgumentEnd(), which is only ever called to parse the first argument of the command line, it should not implement backslash escaping of double quotes. When parsing the first argument of the command line, both CreateProcessW() and CommandLineToArgvW() enter quote mode if the first character is a double quote, in which case the argument ends at the next double quote, regardless of whether it's followed by whitespace. That's a bit simpler than the C runtime, but even the C runtime doesn't implement escaping of double quotes in the first argument.

[pfmoore]

The fix appears to have addressed part of my issue, but I'm still getting the same error in a slightly different case. I haven't yet been able to reduce the test case to something smaller, but if I use the just command line utility, with a justfile as follows:

set windows-shell := ["pwsh.exe", "-NoLogo", "-Command"]

test_py:
    #!py
    print(12)

then running just test_py gives the error:

Unable to create process using 'C:\Users\Gustav\AppData\Local\Temp\justc2pOnD\py C:\Users\Gustav\AppData\Local\Temp\justc2pOnD\test_py': The system cannot find the file specified.

If I add some debugging prints, I can see that the command being run was "py" "C:\Users\Gustav\AppData\Local\Temp\justc2pOnD\test_py".

[zooba]

Thanks for the quick extra test! This one looks like it's probably shebang parsing - can you include the whole output you get with PYLAUNCHER_DEBUG set?