Emscripten buildbot: Support multiple Emscripten versions

[hoodmane]

We want to allow different branches to use different Emscripten versions. The plan is as follows:

1. Add emscripten_version.txt file and add --emsdk-cache argument to emscripten/__main__.py. If --emsdk-cache is passed, the build script will require that the appropriate emsdk version is found in that directory.
2. Backport this to 3.14.x
3. Set up buildbot cache dir
4. Make a PR to buildmaster config to pass the --emsdk-cache argument

cc @freakboy3742

Linked PRs

• gh-145177: Support multiple Emscripten versions for Emscripten buildbot #145180
• [3.14] gh-145177: Support multiple Emscripten versions for Emscripten buildbot (GH-145180) #145582
• gh-145177: Put node version into emscripten/config.toml as well. #146156
• gh-145177: Add make-dependencies command to emscripten build script #146158
• [3.14] gh-145177: Put node version into emscripten/config.toml. (GH-146156) #146159
• gh-145177: Add emscripten run --test, uses test args from config.toml #146160
• [3.14] gh-145177: Add make-dependencies command to emscripten build script (GH-146158) #146184
• [3.14] gh-145177: Add emscripten run --test, uses test args from config.toml (GH-146160) #146186

[freakboy3742]

I did some preliminary work on the buildbot in preparation for this about 6 months ago, and then got distracted by other things. Thanks for picking this up again, @hoodmane.

Agreed that this is something that needs to be addressed, as it's a blocker on us bumping the Emscripten version. I don't recall if we were going to lock the Emscripten version for the pyodide_2025_0 tag at beta 1 or RC1 - but if it's beta 1, time is obviously a factor.

The one question I've got about the design specification here is whether a separate file is the best way to handle version specification. Having it in a separate file isn't a huge problem to parse... but if it was a constant in the Emscripten build script to start with that you can access with python -m emscripten --emdsk-version, doesn't that simplify the common use cases (in that it doesn't need independent file access and parsing)?

On the subject of the build script - there are some other housekeeping items that I'd like to address:

1. clean should be a little more selective. It currently destroys all of cross-build; it should only be purging the parts of cross-build that are emscripten specific (so that Emscripten, iOS and Android builds can co-exist). This is admittedly a "me" problem... but it's a problem I have :-)
2. The cross-build directory name should be configurable. Again, this is somewhat a "me" problem... but I currently need to have cross builds for 3.10-3.15, and they can't currently co-exist.
3. We shouldn't be building dependencies (libFFI and mpdec) on every build. A similar caching strategy to the one described for EMSDK should be possible - look to this directory; if the cache has a build for the right emscripten version, use it rather than rebuilding.
4. @brettcannon has been trying to move all the platform build tooling to the Platforms folder;

These issues don't necessarily need to be handled all in one PR - and I'm happy to deal with the ones that are me problems :-) However, we do want to minimize the number of changes that are needed in the buildbot configuration, and we need to be aware how the impact of changes in the interface to building will impact on the ability of the buildbot configuration to identify which Python version is being built.

[hoodmane]

a separate file is the best way to handle version specification. Having it in a separate file isn't a huge problem to parse... but if it was a constant in the Emscripten build script to start with that you can access with python -m emscripten --emdsk-version, doesn't that simplify the common use cases (in that it doesn't need independent file access and parsing)?

I don't think it matters that much, but sure I can switch to that approach.

[hoodmane]

Updated the PR.

[hoodmane]

@freakboy3742 should clean only remove cross-build/wasm32-emscripten and leave alone cross-build/build for now?

[hoodmane]

there are some other housekeeping items that I'd like to address:

I opened #145219 for items 1-3. Item 4 is tracked as #145176.

[brettcannon]

The one question I've got about the design specification here is whether a separate file is the best way to handle version specification.

WASI does it in a separate file to make backports easier; you can just copy the file blindly as a backport if the per-Python variance is separate from the code.