python - Forcing a thread to block all other threads from executing -

update:

this answer states i'm trying impossible of april 2013. this, however, seems contradict alex martelli says in python cookbook (p. 624, 3rd ed.):

upon return, pygilstate_ensure() guarantees calling thread has exclusive access python interpreter. true if calling c code running different thread unknown interpreter.

the docs seem suggest gil can acquired, give me hope (except don't think can call pygilstate_ensure() pure python code, , if create c extension call it, i'm not sure how embed memory_daemon() in that).

perhaps i'm misreading either answer or python cookbook , docs.

original question:

i want given thread (from threading module) prevent other thread running while segment of code executing. what's easiest way achieve it?

obviously, great minimize code changes in other threads, avoid using c , direct os calls, , make cross-platform windows , linux. realistically, i'll happy have solution whatsoever actual environment (see below).

environment:

• cpython
• python 3.4 (but can upgrade 3.5 if helps)
• ubuntu 14.04

use case:

for debugging purposes, calculate memory used objects (as reported gc.get_objects()), , print summary report sys.stderr. in separate thread, because want summary delivered asynchronously other threads; put time.sleep(10) @ end of while true loop actual memory usage calculation. however, memory reporting thread takes while complete each report, , don't want other threads move ahead before memory calculation finished (otherwise, memory snapshot hard interpret).

example (to clarify question):

import threading th import time  def report_memory_consumption():   # go through `gc.get_objects()`, check size , print summary   # takes ~5 min run  def memory_daemon():   while true:     # other threads should not until call complete     report_memory_consumption()     # sleep 10 sec, update memory summary     # sleep time when other threads should executed     time.sleep(10)   def f1():   # something, including calling many other functions   # takes ~3 min run  def f2():   # something, including calling many other functions   # takes ~3 min run   def main():   t_mem = th.thread(target = memory_daemon)   t1 = th.thread(target = f1)   t2 = th.thread(target = f2)   t_mem.start()   t1.start()   t2.start()  # requirement: no other thread running while t_mem not sleeping 

the python cookbook correct. have exclusive access python interpreter @ point when pygilstate_ensure() returns. exclusive access means can safely call cpython functions. , means current c thread current active python thread. if current c thread did not have corresponding python thread before, pygilstate_ensure() have created 1 automatically.

that state right after pygilstate_ensure(). , have gil acquired @ point.

however, when call other cpython functions now, such pyeval_evalcode() or other, can implicitly make gil gets released meanwhile. example, case if implicitly python statement time.sleep(0.1) gets called somewhere result. , while gil released thread, other python threads can run.

you have guarantee when pyeval_evalcode() (or whatever other cpython function called) returns, again have same state before - i.e. on same active python thread , again have gil.

---

about original question: there no way achieve this, i.e. call python code , avoid gil gets released result somewhere meanwhile. , thing, otherwise end in deadlocks, e.g. if don't allow other thread release lock holds.

about how implement use case: real way in c. call pygilstate_ensure() gil. , @ point, must call cpython functions cannot have side effect of calling other python code. careful. pyobj_decref() call __del__. best thing avoid calling cpython functions , manually traversing cpython objects. note don't have complicated outlined it: there underlying cpython memory allocator , think can information there.

read here memory management in cpython.

related code in pymem.h, obmalloc.c , pyarena.c. see function _pyobject_debugmallocstats(), although might not compiled cpython.

there tracemalloc module add overhead. maybe underlying c code (file _tracemalloc.c) helpful understand internals bit better.

---

about sys.setswitchinterval(1000): related going through python bytecode , handling it. main loop of cpython in pyeval_evalframeex in file ceval.c. there you'll find such part:

if (_py_atomic_load_relaxed(&gil_drop_request))     ... 

all logic switch interval covered in file ceval_gil.h.

setting high switch interval means main loop in pyeval_evalframeex not interrupted longer time. not mean there aren't other possibilities gil released meanwhile , thread run.

pyeval_evalframeex execute python bytecode. let's assume calls time.sleep(1). call native c implementation of function. you'll find in time_sleep() in file timemodule.c. if follow code, you'll find this:

py_begin_allow_threads err = select(0, (fd_set *)0, (fd_set *)0, (fd_set *)0, &timeout); py_end_allow_threads 

thus, gil gets released meanwhile. now, other thread waiting gil pick , run other python code.

theoretically, think, if set high switch interval , never call python code in turn release gil @ point, safe. note impossible, though. e.g. gc called time time , __del__ of objects have various side effects.