Pdb doesn't handle anonymous pipe symlinks

[johnslavik]

Bug report

Bug description:

To reproduce (Bash):

❯ ./python -m pdb <(echo 'print("Hello world!")')
Error: /dev/fd/63 does not exist

This isn't exactly right. /dev/fd/63 was a symlink to an existing anonymous pipe, but it was checked after being processed with os.path.realpath (which made the path unusable for open or stat -- GH-99390):

cpython/Lib/pdb.py

Lines 184 to 190 in 706fdda

	class _ScriptTarget(_ExecutableTarget):
	def __init__(self, target):
	self._target = os.path.realpath(target)
	if not os.path.exists(self._target):
	print(f'Error: {target} does not exist')
	sys.exit(1)

Technically, this is a 3.11 regression caused by https://github.com/python/cpython/pull/26992/files#r663030110.
The new behavior was preserved in the refactor of executable targets in Pdb (GH-112570). Before these two patches, os.path.realpath was only used in Pdb to figure out the directory to be set as sys.path[0] since GH-23338. Prior to that, os.path.realpath wasn't used in Pdb at all.

To fix this edge case, I think we could simply bring back the old behavior of passing the raw path to system functions and continue to use the real path elsewhere.

A dead-simple workaround is to just use regular files.

CC @jaraco @iritkatriel @gaogaotiantian
CC @ZeroIntensity (mentorship): stdlib, 3.13 & 3.14 (unless it's a feature at this point), 3.15. Not sure if I would give this OS-specific labels.

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-142315: Don't pass the "real path" of Pdb script target to system functions #142371
• gh-142315: Add test capturing missed expectation. #142383
• [3.14] gh-142315: Don't pass the "real path" of Pdb script target to system functions (GH-142371) #142497
• [3.13] gh-142315: Don't pass the "real path" of Pdb script target to system functions (GH-142371) #142498

[StanFromIreland]

3.13 & 3.14 ..., 3.15

If it affects all branches we need not add version labels. For more on these labels, you can read Benedikt's (very good IMO) proposed docs.

[johnslavik]

(Thanks Stan!)

[jaraco]

My instinct is that this is a symptom of a limitation of realpath. And since the issue with realpath is not formally documented, I wonder if it would first make sense to document when realpath is unsafe to use. The code was written with the assumption that it is safe to use in all cases. Thanks for providing the reference to the issue that touched on the underlying cause. Here's what I recommend:

First, file an issue, something like "realpath unsafe under some conditions" illustrating how use of realpath causes problems and to determine what is the strategy for Python regarding this function (e.g. document the danger, remediate the danger, etc). In my opinion, realpath is less useful if it can't guarantee that the resulting path is as usable as the input path.

Second, if it looks like we're not considering remedying the situation at realpath, then consider what options are viable here. I don't recall right now why realpath was used or what value it provides (if any). It's probably worth simply replacing the call to realpath to see if any tests fail or to read through the code to see if anything is relying on the canonical form of the path.

[jaraco]

Thanks for the link to the comment in 26992. Reflecting on that again, it explains why realpath was used (for compatibility with a previously separate code path) and a way to make it more compatible. I wouldn't be opposed to changing it to use self.orig for the existence check for a quick, easy fix.

[ZeroIntensity]

If it affects all branches we need not add version labels.

Ah, I missed this comment. I think this is still a gray area. If I see an issue without version labels, my first thought is "this hasn't been triaged yet", not "this affects all versions". Let's wait until Bénédikt's PR is merged before we begin applying it.

[gaogaotiantian]

I'm not the expert of realpath and anonymous pipe here. I understand what did not work - to be it's not a very common usage. It would be great to make it work - but I need to understand the cost. So what wouldn't work if we change the code to make this rare case work?

As a dev tool, pdb sometimes can't offer the perfect result under all corner cases. It would be great to cover these cases, but we also need to have a good balance between covering dark corners and the maintainability/working on normal cases.

[johnslavik]

I'm not the expert of realpath and anonymous pipe here. I understand what did not work - to be it's not a very common usage. It would be great to make it work - but I need to understand the cost. So what wouldn't work if we change the code to make this rare case work?

We'll need to specialcase when realpath doesn't exist but the original path does. This would preserve the original semantics, not costing anything other than more complicated and nontrivial code -- which is a fair enough reason to reject the proposal and instead document anonymous pipe symlinks as unsupported. We would still have to fix the error message. See GH-142371 for the fix.