Skip to content

Fix race condition with py::make_key_iterator in free threading #5971

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
colesbury wants to merge 4 commits into
base: master
Choose a base branch
from
+15 −2
Open

Fix race condition with py::make_key_iterator in free threading #5971

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
4 commits
Select commit Hold shift + click to select a range
bfe55ed
Fix race condition with py::make_key_iterator in free threading
colesbury Jan 21, 2026
fa03953
style: pre-commit fixes
pre-commit-ci[bot] Jan 21, 2026
70c2ffc
Use PyCriticalSection_BeginMutex instead of recursive mutex
colesbury Jan 22, 2026
8690dd7
style: pre-commit fixes
pre-commit-ci[bot] Jan 22, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
16 changes: 14 additions & 2 deletions include/pybind11/detail/internals.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -230,6 +230,7 @@ using instance_map = std::unordered_multimap<const void *, instance *>;
#ifdef Py_GIL_DISABLED
// Wrapper around PyMutex to provide BasicLockable semantics
class pymutex {
friend class pycritical_section;
PyMutex mutex;

public:
Expand All @@ -238,6 +239,17 @@ class pymutex {
void unlock() { PyMutex_Unlock(&mutex); }
};

class pycritical_section {
pymutex &mutex;
PyCriticalSection cs;

public:
explicit pycritical_section(pymutex &m) : mutex(m) {
PyCriticalSection_BeginMutex(&cs, &mutex.mutex);
}
~pycritical_section() { PyCriticalSection_End(&cs); }
};

// Instance map shards are used to reduce mutex contention in free-threaded Python.
struct instance_map_shard {
instance_map registered_instances;
Expand Down Expand Up @@ -856,7 +868,7 @@ inline local_internals &get_local_internals() {
}

#ifdef Py_GIL_DISABLED
# define PYBIND11_LOCK_INTERNALS(internals) std::unique_lock<pymutex> lock((internals).mutex)
# define PYBIND11_LOCK_INTERNALS(internals) pycritical_section lock((internals).mutex)
#else
# define PYBIND11_LOCK_INTERNALS(internals)
#endif
Expand Down Expand Up @@ -885,7 +897,7 @@ inline auto with_exception_translators(const F &cb)
get_local_internals().registered_exception_translators)) {
auto &internals = get_internals();
#ifdef Py_GIL_DISABLED
std::unique_lock<pymutex> lock((internals).exception_translator_mutex);
pycritical_section lock((internals).exception_translator_mutex);
#endif
auto &local_internals = get_local_internals();
return cb(internals.registered_exception_translators,
Expand Down
1 change: 1 addition & 0 deletions include/pybind11/pybind11.h
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -3173,6 +3173,7 @@ iterator make_iterator_impl(Iterator first, Sentinel last, Extra &&...extra) {
using state = detail::iterator_state<Access, Policy, Iterator, Sentinel, ValueType, Extra...>;
// TODO: state captures only the types of Extra, not the values

PYBIND11_LOCK_INTERNALS(get_internals());
if (!detail::get_type_info(typeid(state), false)) {
class_<state>(handle(), "iterator", pybind11::module_local())
.def(
Expand Down
Loading