Skip to content

Memory leak in tools.parsl_task_queue.task_list #37

Description

@frobnitzem

The get_task_results method only removes tasks from self.task_list if they are 'exec_done' or 'failed'.
Parsl defines a lot of other task states.
Those other ones won't get deleted from self.task_list, leaking both the Future objects and any associated resources.

for taski,task in reversed(list(enumerate(self.task_list))):
task_status = task.task_status()
if task_status == 'exec_done' and task.done:
results_list.append(task.result())
del self.task_list[taski]
elif task_status == 'failed':
failed_number += failed_number
del self.task_list[taski]

This will also throw off the count in get_queued_number.

While we're at it, we can log the result when task_status == 'failed'.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions