Skip to content

sutime not working with ProcessPool #34

@akashdave-v

Description

@akashdave-v

Why it is not working by just replacing ThreadPoolExecutor with ProcessPoolExecutor?
@FraBle Can you please look at this and reply as soon as possible? Thank you.

Here is a code snippet using your code:

import os
import json
from sutime import SUTime
from concurrent.futures import ProcessPoolExecutor
from multiprocessing import cpu_count

def get_sutime():
    jar_files = os.path.join(os.path.dirname(__file__), os.pardir, "jars")
    sutime = SUTime(jars=jar_files, mark_time_ranges=True)
    return sutime


def struct_filing(sentence, sutime):
    return sentence, sutime.parse(sentence)


def get_time(text_list):
    sutime = get_sutime()
    with ProcessPoolExecutor(max_workers=cpu_count()) as executor:
        return dict(
            future.result()
            for future in [
                executor.submit(struct_filing, sentence, sutime)
                for sentence in text_list
            ]
        )


if __name__ == "__main__":
    data_set_1 = {
        "The Formula One race came to Singapore in 2008.",
        "It was a tremendous success and gained wide international coverage.",
        "In a first for the F1, the event was held at night.",
        "State-of-the-art technology was used to ensure that the race operated"
        " in near daylight conditions.",
        "Holding the race at night had a great advantage as European audiences"
        " could watch it at a convenient time.",
        "Over the years, Singapore’s tourism industry has gained significant"
        " benefits from the media attention that the race has generated",
    }
    print(json.dumps(get_time(data_set_1), sort_keys=True, indent=4))

Originally posted by @FraBle in #9 (comment)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions