Jit stencils on Windows contain debug data in 3.15

[chris-eibl]

Bug report

Bug description:

E.g.

void
emit__NOP(
    unsigned char *code, unsigned char *data, _PyExecutorObject *executor,
    const _PyUOpInstruction *instruction, jit_state *state)
{
    // 
    // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00\x04\x00\x00\x00\xf1\x00\x00\x00<\x00\x00\x00\n\x00\x01\x11\x00\x00\x00\x00\x00\x00\x00\x00.\x00<\x11\x00\x00\x00\x00\xd0\x00\x15\x00\x01\x00\x04\x00\x00\x00\x16R\x00\x00\x00\x00\x00\x00clang version 21.1.4\x00\x00'
    // 4c: 00 00 00 00
    const unsigned char data_body[80] = {
        0x00, 0x00, 0x00, 0x00, 0x04, 0x00, 0x00, 0x00,
        0xf1, 0x00, 0x00, 0x00, 0x3c, 0x00, 0x00, 0x00,
        0x0a, 0x00, 0x01, 0x11, 0x00, 0x00, 0x00, 0x00,
        0x00, 0x00, 0x00, 0x00, 0x2e, 0x00, 0x3c, 0x11,
        0x00, 0x00, 0x00, 0x00, 0xd0, 0x00, 0x15, 0x00,
        0x01, 0x00, 0x04, 0x00, 0x00, 0x00, 0x16, 0x52,
        0x00, 0x00, 0x00, 0x00, 0x00, 0x00, 0x63, 0x6c,
        0x61, 0x6e, 0x67, 0x20, 0x76, 0x65, 0x72, 0x73,
        0x69, 0x6f, 0x6e, 0x20, 0x32, 0x31, 0x2e, 0x31,
        0x2e, 0x34,
    };
    memcpy(data, data_body, sizeof(data_body));
}

which is in fact empty on main for Linux (at least for me on WSL) or Windows on 3.14.

The above block is contained in every stencil :(

Clang now generates debug info into the object files. Don't know when it happened, maybe one of the clang updates. Doesn't matter anyway, the fix is easy, just ignore .debug$S COFF sections, which brings us back to

void
emit__NOP(
    unsigned char *code, unsigned char *data, _PyExecutorObject *executor,
    const _PyUOpInstruction *instruction, jit_state *state)
{
    // 
    // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00'
    // 4: 00 00 00 00
}

CPython versions tested on:

CPython main branch

Operating systems tested on:

Windows

Linked PRs

• GH-142050: Jit stencils on Windows contain debug data #142052

[chris-eibl]

Don't know when it happened

I've done some further investigation. Just changing back to

_LLVM_VERSION = "20"
_EXTERNALS_LLVM_TAG = "llvm-20.1.8.0"

in

cpython/Tools/jit/_llvm.py

Lines 14 to 15 in cd4d0ae

	_LLVM_VERSION = "21"
	_EXTERNALS_LLVM_TAG = "llvm-21.1.4.0"

fixes this. So it must be related to the update of clang to 21.1.4.0 (#140973).

Looking closer at the data

    // _NOP.o:     file format coff-x86-64
    // 0: '\x00\x00\x00\x00\x04\x00\x00\x00\xf1\x00\x00\x00<\x00\x00\x00\n\x00\x01\x11\x00\x00\x00\x00\x00\x00\x00\x00.\x00<\x11\x00\x00\x00\x00\xd0\x00\x15\x00\x01\x00\x04\x00\x00\x00\x16R\x00\x00\x00\x00\x00\x00clang version 21.1.4\x00\x00'
    // 4c: 00 00 00 00

reveals clang version 21.1.4. Thus, I had high hopes that using -fno-ident might fix it. But unfortunately, this is not enough, still more metadata is written. So searching in the clang command line reference for metadata I found -fno-stack-size-section which doesn't change anything.

Thus, I think just ignoring .debug$S COFF sections is the easiest way to go and I have verified using

py Tools\jit\build.py -p PC -o jit-files aarch64-pc-windows-msvc x86_64-pc-windows-msvc i686-pc-windows-msvc

that with this fix for all three configurations this metadata is no longer contained in the stencils (and was present in all of them before).

I've checked again in WSL Ubuntu 24.04.3 LTS for clang 21.1.3 and on native Ubuntu 24.04.3 LTS for clang 21.1.7 that this does not happen for x86_64-pc-linux-gnu. I don't have a setup for cross builds, so unfortunately I cannot comment on aarch64-unknown-linux-gnu for Linux and I have no Mac.

I have no idea why clang changed that only for Windows.

[chris-eibl]

As written in #142052 (comment), I am unsure about a backport, because

• 3.14 uses clang 19.1.7 and is not affected. I've verified for aarch64-pc-windows-msvc x86_64-pc-windows-msvc i686-pc-windows-msvc.
• no clang update is planned there anymore (I assume?)
• the clang version used for the jit cannot (easily without touching _llmv.py) be changed. Make LLVM_VERSION configurable with env variable #138497 supports that only for configure based builds. And it is discouraged to use a different clang version, anyway. And the jit is experimental in 3.14 ...
• but better safe than sorry? The fix is small and simple ...

[chris-eibl]

I have no idea why clang changed that only for Windows.

Ha, I found it, even though it is not part of the 21.1.0 release notes:
llvm/llvm-project#142970: [CodeGen][COFF] Always emit CodeView compiler info on Windows targets

MSVC always emits minimal CodeView metadata with compiler information, even when debug info is otherwise disabled. Other tools may rely on this metadata being present. For example, linkers use it to determine whether hotpatching is enabled for the object file.

It is contained in clang since 21.1.0-rc1.
Unfortunately, this is done unconditionally (at first it was planned to do it only when hotpatching is enabled) and there seems to be no switch to disable it.

[chris-eibl]

PR is merged, I think this can be closed.