Skip to content

Fix memory leaks and code quality issues #93

@ajshedivy

Description

@ajshedivy

Summary

Several code quality issues affect production stability: a memory leak from unbounded global lists, mutable default arguments that can cause shared-state bugs, unsafe error string parsing, and silent exception swallowing.

Issues

1. Memory Leak: Unbounded global_query_list (High Severity)

Both Query and PoolQuery maintain static class-level lists that grow indefinitely:

# client/query.py
class Query:
    global_query_list: List["Query[Any]"] = []
    
    def __init__(self, ...):
        Query.global_query_list.append(self)  # Never removed

# pool/pool_query.py
class PoolQuery:
    global_query_list: List["PoolQuery"] = []
    
    def __init__(self, ...):
        PoolQuery.global_query_list.append(self)  # Never removed

Every query created during the lifetime of the process is retained in memory forever. In long-running applications or services, this will cause steadily increasing memory consumption.

Fix: Either remove queries on close/cleanup, use weakref.WeakList, or remove the global list entirely if it's not needed externally.

2. Mutable Default Arguments (Medium Severity)

At least 5 instances of opts: Dict[str, Any] = {} across the codebase:

# base_job.py, sql_job.py, etc.
def __init__(self, ..., opts: Dict[str, Any] = {}, **kwargs):
def connect(..., opts: Dict[str, Any] = {}, **kwargs):

All calls to these functions share the same dict object. If any code path mutates opts, it affects all subsequent calls.

Fix: Replace with opts: Optional[Dict[str, Any]] = None and if opts is None: opts = {} inside the function body.

3. Unsafe Error Parsing with ast.literal_eval() (Medium Severity)

In core/exceptions.py:

error_dict = ast.literal_eval(error_string)

ast.literal_eval() can parse arbitrary Python literals, not just JSON. While it won't execute code, it's the wrong tool for parsing JSON error strings from the server.

Fix: Replace with json.loads(error_string) which is both safer and more appropriate for JSON data.

4. Silent Exception Swallowing (Medium Severity)

In core/cursor.py nextset():

def nextset(self):
    try:
        if len(self.query_q) > 1:
            self.query_q.popleft()
            self.query = self.query_q[0]
            return True
        return None
    except Exception:
        return None  # Silently swallows ALL exceptions

Also in various query close() paths where AttributeError may be silently ignored.

Fix: Log the exception before returning, or narrow the except clause to specific expected exceptions.

5. Dead Code (Low Severity)

QueryManager.run_query() is defined but never called anywhere in the codebase.

Fix: Remove the dead method, or integrate it into the query execution path if it was intended to be used.

Files to Modify

  • mapepire_python/client/query.py — Fix global_query_list lifecycle, mutable defaults
  • mapepire_python/pool/pool_query.py — Fix global_query_list lifecycle
  • mapepire_python/base_job.py — Fix mutable default arguments
  • mapepire_python/client/sql_job.py — Fix mutable default arguments
  • mapepire_python/core/exceptions.py — Replace ast.literal_eval() with json.loads()
  • mapepire_python/core/cursor.py — Fix silent exception swallowing in nextset()
  • mapepire_python/query_manager.py — Remove dead run_query() method

Acceptance Criteria

  • global_query_list no longer grows unboundedly (queries removed on close or use weak references)
  • All mutable default arguments replaced with None + conditional initialization
  • ast.literal_eval() replaced with json.loads() in error parsing
  • No bare except Exception: pass or except Exception: return None — exceptions are either logged or narrowed
  • Dead code removed
  • All existing tests still pass
  • Add a unit test that verifies queries are cleaned up from global list after close

Metadata

Metadata

Assignees

Labels

bugSomething isn't working

Type

No type

Projects

No projects

Milestone

No milestone

Relationships

None yet

Development

No branches or pull requests

Issue actions