Python 3.12 introduced a new limitation preventing multiple nested profilers from running at the same time: python/cpython#110770
We're running into this limitation with warcprox. Attempting to run with --profile enabled will cause it to immediately error out with an Another profiling tool is already active exception the first time it tries to spin up a BasePostfetchProcessor thread.
We instantiate cProfiler profilers in one place - inside BasePostfetchProcessor.run:
|
self.profiler = cProfile.Profile() |
|
self.profiler.enable() |
|
self._run() |
|
self.profiler.disable() |
We do, however, additionally interact with those profilers via PooledMitmProxy, where we grab the profiler from the thread, enable it, process the request, and disable it again:
|
profiler = self.profilers[threading.current_thread().ident] |
|
profiler.enable() |
|
self._process_request_thread(request, client_address) |
|
profiler.disable() |
This could either be repeated invocations to enable() the same profiler when it's already enabled (calling .enable() twice on the same profiler raises an exception), or a call to enable() a second profiler from a different thread while a profiler from the first thread is still active.
Python 3.12 introduced a new limitation preventing multiple nested profilers from running at the same time: python/cpython#110770
We're running into this limitation with warcprox. Attempting to run with
--profileenabled will cause it to immediately error out with anAnother profiling tool is already activeexception the first time it tries to spin up aBasePostfetchProcessorthread.We instantiate
cProfilerprofilers in one place - insideBasePostfetchProcessor.run:warcprox/warcprox/__init__.py
Lines 106 to 109 in 37e4cd5
We do, however, additionally interact with those profilers via
PooledMitmProxy, where we grab the profiler from the thread, enable it, process the request, and disable it again:warcprox/warcprox/mitmproxy.py
Lines 753 to 756 in f7dd1e7
This could either be repeated invocations to
enable()the same profiler when it's already enabled (calling.enable()twice on the same profiler raises an exception), or a call toenable()a second profiler from a different thread while a profiler from the first thread is still active.