Move the `eval_breaker` to `PyThreadState`

[colesbury]

Feature or enhancement

The eval_breaker is a variable that keeps track of requests to break out of the eval loop to handle things like signals, run a garbage collection, or handle asynchronous exceptions. It is currently in the interpreter state (in interp->ceval.eval_breaker). However, some of the events are specific to a given thread. For example, signals and some pending calls can only be executed on the "main" thread of an interpreter.

We should move the eval_breaker to PyThreadState to better handle these thread-specific events. This is more important for the --disable-gil builds where multiple threads within the same interpreter may be running at the same time.

@markshannon suggested a combination of per-interpreter and per-thread state, where the thread copies the per-interpreter eval_breaker state to the per-thread state when it acquires the GIL.

Linked PRs

• gh-112175: Add eval_breaker to PyThreadState #115194

[gaogaotiantian]

If I remember correctly, eval_breaker also contains the global version for instrumentation. Will having multiple eval_breakers in different threads work with the current instrumentation mechanism?

[colesbury]

I don't think it'll be an issue in the default build, but I'll need to think about how instrumentation works in --disable-gil builds with multiple threads.

I think Mark's idea works basically like:

• Set the eval_breaker in PyThreadState.eval_breaker.
• On _PyThreadState_Detach(), copy bits from PyThreadState.eval_breaker to PyInterpreterState.interp_eval_breaker
• On _PyThreadState_Attach(), copy bits from PyInterpreterState.interp_eval_breaker back to PyThreadState.eval_breaker

In --disable-gil builds, we will need to loop over all the threads when setting interpreter-wide bits.

[swtaarrs]

I'm currently working on this, so if anyone has any related ideas/comments, please post them!

[swtaarrs]

@markshannon, do you have any more context on your per-thread eval_breaker idea that Sam described a couple comments up? Also, this is a heads up that I'm working on this, since you expressed an interest in it in a previous PR comment.

I'm fleshing out exactly how all the flags will work in free-threaded vs. normal builds. As Sam says above, we need to loop over all threads for interpreter-wide flags in a free-threaded build, and this is pretty straightforward. All eval_breaker flags will go in a new member of PyThreadState. I'm planning on preserving the interpreter-wide interp_eval_breaker (renamed from eval_breaker) to keep holding the global instrumentation version. Properly supporting the version number in a free-threaded build will be handled separately from this issue.

For normal builds, if we want to avoid looping over all threads, we can set interpreter-wide flags on the active thread and use interp_eval_breaker to shuffle them between threads when a context switch happens. Each flag is slightly different, so here's my current plan for how to handle them (this is just for the normal build, where we still have the GIL):

• _PY_SIGNALS_PENDING_BIT is easy - signals are only ever handled by the main thread in the main interpreter. We set the bit on that thread when a signal is received.
• _PY_ASYNC_EXCEPTION_BIT also applies to a single, known thread, so we set the bit on that specific thread.
• _PY_GIL_DROP_REQUEST_BIT is again set on a specific thread (the one holding the GIL). This is only set while holding gil->mutex, which should ensure that the thread holding the GIL doesn't change between when we decide to set the flag and when we actually set the flag.
• _PY_GC_SCHEDULED_BIT can be handled by any thread in the targeted interpreter. The one existing caller of the private function _Py_ScheduleGC(PyInterpreterState*) schedules a GC for the current interpreter, so I believe it should be safe to change it to operate on a PyThreadState* instead, and always pass the current thread. The next time that thread checks its eval_breaker, it will run the GC. If it yields before then, it will move the GC bit to interp_eval_breaker for the next scheduled thread to pick up. If we want to preserve the ability for one interpreter to schedule a GC in another interpreter, the strategy used for _PY_CALLS_TO_DO (next item) should work.
• _PY_CALLS_TO_DO is the least restricted case, because since gh-104812: Run Pending Calls in any Thread #104813, any thread can add a callback to run in any interpreter, and callbacks can be allowed to run in any thread. The general strategy is similar to _PY_GC_SCHEDULED_BIT: set the bit on the current thread, and if that thread yields before processing the callback, move the bit to interp_eval_breaker. If no thread in the targeted interpreter holds the GIL, directly set interp_eval_breaker.
  • To safely set this signal across interpreters, I will ensure that a) this bit is only set while holding gil->mutex for the signaled interpreter (similar to _PY_GIL_DROP_REQUEST), and b) bits will only be transferred between eval_breaker and interp_eval_breaker while holding gil->mutex. Half of part b) already happens with update_eval_breaker_from_thread(), called from take_gil(). I'll add a reversed version of that in drop_gil(). Both directions will only apply to _PY_GC_SCHEDULED_BIT and _PY_CALLS_TO_DO_BIT, since the other flags only apply to a single thread.

Does this all sound reasonable, especially for the interpreter-wide flags? If it's too complicated, I could set them by looping over all threads in both build types. That would add some overhead to normal builds, but I expect it wouldn't be measurable overall. It's not much work to try out both implementations, so I could see if there's a measurable performance difference, if that would help make the decision.