Skip to content

--profile doesn't work with Python 3.12+ #258

@mistydemeo

Description

@mistydemeo

Python 3.12 introduced a new limitation preventing multiple nested profilers from running at the same time: python/cpython#110770

We're running into this limitation with warcprox. Attempting to run with --profile enabled will cause it to immediately error out with an Another profiling tool is already active exception the first time it tries to spin up a BasePostfetchProcessor thread.

We instantiate cProfiler profilers in one place - inside BasePostfetchProcessor.run:

self.profiler = cProfile.Profile()
self.profiler.enable()
self._run()
self.profiler.disable()

We do, however, additionally interact with those profilers via PooledMitmProxy, where we grab the profiler from the thread, enable it, process the request, and disable it again:

profiler = self.profilers[threading.current_thread().ident]
profiler.enable()
self._process_request_thread(request, client_address)
profiler.disable()

This could either be repeated invocations to enable() the same profiler when it's already enabled (calling .enable() twice on the same profiler raises an exception), or a call to enable() a second profiler from a different thread while a profiler from the first thread is still active.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type
    No fields configured for issues without a type.

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions