Skip to content

CompactionHook.close() releases the compactor but never the token counter #12984

Description

@nanhanq1

Describe the bug

CompactionHook owns two objects that can hold resources - the token_counter and the compactor - and it treats them asymmetrically. warm_up() and warm_up_async() warm up both (haystack/hooks/compaction/hooks.py:282-297 on main @ 8a5406e):

def warm_up(self) -> None:
    """Warm up the token counter and the compactor, which may hold resources such as a Chat Generator."""
    if hasattr(self.token_counter, "warm_up"):
        self.token_counter.warm_up()
    if hasattr(self.compactor, "warm_up"):
        self.compactor.warm_up()

but close() and close_async() release only the compactor (haystack/hooks/compaction/hooks.py:299-310):

def close(self) -> None:
    """Release the compactor's resources."""
    if hasattr(self.compactor, "close"):
        self.compactor.close()

Nothing else in haystack/ closes a token counter either - grep -rn "token_counter.close" haystack/ returns nothing - so a counter that was warmed up is never released.

OpenAITokenCounter is the counter where this is visible: warm_up() builds an openai.OpenAI client, and with it an HTTP connection pool, and its own close() drops it again (haystack/token_counters/openai_counter.py:122-126). Because Agent.close() -> close_hooks() -> CompactionHook.close() never reaches it, the client outlives the hook.

Error message

None - nothing raises, the client just stays open. OpenAITokenCounter.client is still set after a full Agent-style teardown:

before warm_up(): released (client is None)
after  warm_up(): OPEN (leaked)
after  close():   OPEN (leaked)   <-- expected: released

Expected behavior

close() and close_async() should release the token counter the same way they release the compactor, mirroring what warm_up() and warm_up_async() already do.

To Reproduce

import os
os.environ["OPENAI_API_KEY"] = "sk-not-a-real-key"

from haystack.hooks.compaction import CompactionHook, SlidingWindowCompactor
from haystack.token_counters import OpenAITokenCounter

counter = OpenAITokenCounter(model="gpt-5-mini")
hook = CompactionHook(compactor=SlidingWindowCompactor(), context_window=1000, token_counter=counter)

hook.warm_up()        # Agent.warm_up() -> warm_up_hooks() -> hook.warm_up()
print(counter.client) # an openai.OpenAI instance
hook.close()          # Agent.close()   -> close_hooks()   -> hook.close()
print(counter.client) # still an openai.OpenAI instance, expected None

hook.close_async() has the same gap.

Additional context

SummarizationCompactor does define warm_up/warm_up_async/close/close_async, because it holds a ChatGenerator - which is likely why the compactor half was handled and the token counter half was missed.

The fix is small: handle the token counter in close()/close_async() exactly as the compactor is handled, and let warm_up_async() prefer a counter's warm_up_async when it defines one, so all four lifecycle methods treat both resources the same way. I have that with tests ready and will open the PR if that is the right shape.

FAQ Check

System:

  • OS: Windows 11 (reproduced), platform-independent
  • Haystack version: main @ 8a5406e / 3.3.0-rc0

Disclosure: found and reproduced with the help of an AI coding agent (Claude Code); I reviewed the report and ran the reproducer myself.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions