Improve import time of various stdlib modules

[layday]

Feature or enhancement

Proposal:

Following on from #109653, further improvements can be made to import times.

Links to previous discussion of this feature:

https://discuss.python.org/t/deferred-computation-evalution-for-toplevels-imports-and-dataclasses/34173

For example:

importlib.metadata is often used for tasks that need to happen at import, e.g. to enumerate/load entry point plug-ins, so it might be worth seeing if we can cut down its own import time a bit more.

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

Linked PRs

• gh-118761: Improve import time of pprint #122725
• gh-118761: Speedup pathlib import by deferring shutil #123520
• gh-121423: Improve import time of socket by writing socket.errorTab as a constant and lazy import modules #121424
• gh-118761: Improve import time of mimetypes #126979
• gh-118761: improve import time for pickle #128732
• gh-118761: substitute re import in base64.b16decode for a more efficient alternative #128736
• gh-118761: improve import time for secrets #128738
• gh-118761: Improve import time for csv #128858
• gh-118761: Reduce import time of gettext.py by delaying re import #128898
• gh-118761: improve optparse import time by delaying textwrap import #128899
• gh-118761: Improve import time of tomllib #128907
• gh-118761: Improve import time for pstats and zipfile by removing imports to typing #128981
• gh-118761: Improve import time of sqlite3 #129118
• gh-118761: Improve import time of subprocess #129427
• gh-118761: Always lazy import warnings in threading #129428
• [3.13] gh-118761: Improve import time of subprocess (GH-129427) #129447
• [3.12] gh-118761: Improve import time of subprocess (GH-129427) #129448
• gh-118761: Improve import time by lazy import of warnings #129765
• gh-118761: Improve import time by lazy import of traceback #129811
• gh-118761: Always lazy import re in locale #129860
• gh-118761: Improve import time of dataclasses #129925
• gh-118761: Improve import time of cmd module #130056
• gh-118761: Revert "Improve import time of subprocess (GH-129427)" #130201
• [3.13] gh-118761: Revert "Improve import time of subprocess (GH-129427)" (GH-130201) #130204
• [3.12] gh-118761: Revert "Improve import time of subprocess (GH-129427)" (GH-130201) #130205
• gh-137855: Improve import time of sqlite3 #131796
• gh-118761: Optimise import time for ast #131953
• gh-137855: Optimise import time for textwrap #131956
• gh-118761: Fix star-import of ast #132024
• gh-118761: Fix star-import of ast (alternative) #132025
• gh-118761: Improve import time of annotationlib #132028
• gh-118761: Use enum._simple_enum for annotationlib.Format #132031
• gh-118761: Cover the import time optimisations in What's New #132035
• gh-118761: Optimise import time for shlex #132036
• gh-118761: Optimise import time for string #132037
• gh-137855: email.quoprimime removing re import #132046
• gh-118761: Defer import of functools in annotationlib #132059
• gh-118761: Lazily import annotationlib in typing #132060
• GH-118761: Expose more core interpreter types in _types #132103
• gh-118761: Add helper to ensure that lazy imports are actually lazy #132614
• gh-118761: Add test_lazy_import for more modules #133057

[hugovk]

@layday Is it okay if I repurpose this issue as an "Improve import time of various stdlib modules" like #109653 but for 3.14?

I've got some pprint improvements, and if we have importlib.metadata and some others, we can group them under the same umbrella issue like last time.

[layday]

Sure!

[danielhollas]

I've opened a PR over at the importlib_metadata repo that avoids importing inspect. python/importlib_metadata#499

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases.
Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

@layday were you planning on tackling these?

It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method

This seems to be solved on main, importlib.abc no longer imports importlib.resources.

[danielhollas]

I've opened a PR over at the importlib_metadata repo that avoids importing inspect. python/importlib_metadata#499

This has been merged and released in version 8.4 of importlib_metadata 🎉

importlib.metadata imports zipfile at the top for a function that won't be called in the vast majority of cases. It also imports importlib.abc, which in turn imports importlib.resources, to subclass an ABC with a single, non-abstract method - I assume redefining the method in importlib.metadata would be harmless. Some other less frequently-used imports which are only accessed once or twice, such as json, could also be tucked away in their calling functions.

I've submitted python/importlib_metadata#502 that defers zip import, and python/importlib_metadata#503 which defers json and platform.