SharedMemory.unlink() can fail if called during interpreter shutdown

[samtygier]

SharedMemory.unlink() contains a import which can fail if called during interpreter shutdown.

For example when working with SharedMemory as the buffer for a numpy array it is useful to create a wrapper object to ensure that clean up is done with the object is deleted.

#import numpy as np
from multiprocessing import shared_memory

class ShArray:
    def __init__(self):
        self.shared_memory = shared_memory.SharedMemory(create=True, size=400)
        #self.array = np.ndarray((10,10), dtype="i4", buffer=self.shared_memory.buf)

    def __del__(self):
        self.shared_memory.close()
        self.shared_memory.unlink()

a1 = ShArray()

(Commented lines show my typical usage, but are not need to demonstrate the issue)

Exception ignored in: <function ShArray.__del__ at 0x7f36eee510d0>
Traceback (most recent call last):
  File "/home/sam/git/cpython/test_shared_mem2.py", line 13, in __del__
  File "/home/sam/git/cpython/Lib/multiprocessing/shared_memory.py", line 240, in unlink
ImportError: sys.meta_path is None, Python is likely shutting down
/home/sam/git/cpython/Lib/multiprocessing/resource_tracker.py:224: UserWarning: resource_tracker: There appear to be 1 leaked shared_memory objects to clean up at shutdown
  warnings.warn('resource_tracker: There appear to be %d '

This happens on 3.8 and current main, both on Linux x86-64.

Patch to follow

[vstinner]

I'm not a supporter of multiprocessing programs relying on the GC to call __del__() finalizers. You should close resources explicitly, using with: statement if possible. But well, the change seems acceptable to me, so I merged it.

cc @pitrou

[pitrou]

Seems fine to me. with statements are not always usable.

[vstinner]

with statements are not always usable.

So we should make them usable in more cases :-)

Even without with:, the idea is to explicitly release resources, not rely on the GC for that. The objects are not finalized in a specific order by the GC. The exact order is really complicated and depends on too many things. So it's likely that your code will fall into many corner cases. ImportError: sys.meta_path is None, Python is likely shutting down is just one example of such issue that you can get if you rely on the GC.

In PyPy, it expect that it will be even worse.