Currently one entire bro-domain import runs as a single task.
To optimize speed, we should aim to split these into celery tasks, like the uploadtask.
This way, during request wait times (which is 95% for import tasks), we are able to process other objects. Speeding up the time to complete by a landslide.
To convert this import flow into a celery task:
- Identify each step
- Convert the step to a function
- Convert the function to a celery task
- Link the celery tasks in a task chain
- Correct the bulk import to send celery tasks instead
*Optional: should maybe add a bulk-import and importtask object, instead of the current importtask object. To follow the setup of UploadTasks
Currently one entire bro-domain import runs as a single task.
To optimize speed, we should aim to split these into celery tasks, like the uploadtask.
This way, during request wait times (which is 95% for import tasks), we are able to process other objects. Speeding up the time to complete by a landslide.
To convert this import flow into a celery task:
*Optional: should maybe add a bulk-import and importtask object, instead of the current importtask object. To follow the setup of UploadTasks