⚡ Bolt: Parallelize sequential asset loading (Python/JS)#535
Conversation
Co-authored-by: aafre <8656674+aafre@users.noreply.github.com>
|
👋 Jules, reporting for duty! I'm here to lend a hand with this pull request. When you start a review, I'll add a 👀 emoji to each comment to let you know I've read it. I'll focus on feedback directed at me and will do my best to stay out of conversations between you and other bots or reviewers to keep the noise down. I'll push a commit with your requested changes shortly after. Please note there might be a delay between these steps, but rest assured I'm on the job! For more direct control, you can switch me to Reactive Mode. When this mode is on, I will only act on comments where you specifically mention me with New to Jules? Learn more at jules.google/docs. For security, I will only act on instructions from the user who triggered this task. |
There was a problem hiding this comment.
Code Review
This pull request optimizes I/O operations by parallelizing sequential tasks in both the Python backend and the React frontend. Specifically, it introduces ThreadPoolExecutor in app.py for downloading icons and Promise.all in useIconRegistry.ts for file conversions. The review feedback identifies redundant variable initializations and emphasizes the need for thread-local Supabase clients within the worker threads to ensure thread safety and prevent potential race conditions.
| failed_icons = [] | ||
| for icon in icons_result.data: | ||
|
|
||
| def _download_worker(icon): | ||
| icon_path = session_icons_dir / icon["filename"] | ||
| success = download_icon_from_storage( | ||
| icon["storage_path"], str(icon_path) | ||
| ) | ||
| if not success: | ||
| failed_icons.append(icon["filename"]) | ||
| logging.error( | ||
| f"Failed to download icon: {icon['filename']} from {icon['storage_path']}" | ||
| ) | ||
| return icon["filename"] | ||
| return None | ||
|
|
||
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | ||
| results = executor.map(_download_worker, icons_result.data) | ||
| failed_icons = [filename for filename in results if filename is not None] |
There was a problem hiding this comment.
The initialization of failed_icons = [] at line 3321 is redundant because the variable is completely overwritten by the list comprehension at line 3337. Additionally, there is a potential thread-safety concern: the _download_worker uses the global supabase client via download_icon_from_storage. Existing parallel logic in this file (see _copy_icon_worker at line 2828) explicitly creates a thread-local Supabase client to avoid race conditions or connection pool issues. Consider following that established pattern here.
| failed_icons = [] | |
| for icon in icons_result.data: | |
| def _download_worker(icon): | |
| icon_path = session_icons_dir / icon["filename"] | |
| success = download_icon_from_storage( | |
| icon["storage_path"], str(icon_path) | |
| ) | |
| if not success: | |
| failed_icons.append(icon["filename"]) | |
| logging.error( | |
| f"Failed to download icon: {icon['filename']} from {icon['storage_path']}" | |
| ) | |
| return icon["filename"] | |
| return None | |
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | |
| results = executor.map(_download_worker, icons_result.data) | |
| failed_icons = [filename for filename in results if filename is not None] | |
| def _download_worker(icon): | |
| # Create thread-local client for safety, matching pattern in _copy_icon_worker | |
| from supabase import create_client | |
| thread_supabase = create_client(SUPABASE_URL, SUPABASE_SECRET_KEY) | |
| icon_path = session_icons_dir / icon["filename"] | |
| try: | |
| file_data = thread_supabase.storage.from_("resume-icons").download(icon["storage_path"]) | |
| with open(icon_path, "wb") as f: | |
| f.write(file_data) | |
| return None | |
| except Exception as e: | |
| logging.error(f"Failed to download icon: {icon['filename']} - {e}") | |
| return icon["filename"] | |
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | |
| results = executor.map(_download_worker, icons_result.data) | |
| failed_icons = [filename for filename in results if filename is not None] |
References
- To ensure thread safety, each worker in a thread pool should create its own instance of clients that are not thread-safe (e.g., database clients).
| failed_icons = [] | ||
| for icon in icons_result.data: | ||
|
|
||
| def _download_worker(icon): | ||
| icon_path = session_icons_dir / icon["filename"] | ||
| success = download_icon_from_storage( | ||
| icon["storage_path"], str(icon_path) | ||
| ) | ||
| if not success: | ||
| failed_icons.append(icon["filename"]) | ||
| logging.error( | ||
| f"Failed to download icon: {icon['filename']} from {icon['storage_path']}" | ||
| ) | ||
| return icon["filename"] | ||
| return None | ||
|
|
||
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | ||
| results = executor.map(_download_worker, icons_result.data) | ||
| failed_icons = [filename for filename in results if filename is not None] |
There was a problem hiding this comment.
Similar to the change in generate_pdf_for_saved_resume, the initialization of failed_icons at line 3578 is redundant. Also, please ensure thread safety by using a thread-local Supabase client within the worker, consistent with the implementation of _copy_icon_worker elsewhere in this module.
| failed_icons = [] | |
| for icon in icons_result.data: | |
| def _download_worker(icon): | |
| icon_path = session_icons_dir / icon["filename"] | |
| success = download_icon_from_storage( | |
| icon["storage_path"], str(icon_path) | |
| ) | |
| if not success: | |
| failed_icons.append(icon["filename"]) | |
| logging.error( | |
| f"Failed to download icon: {icon['filename']} from {icon['storage_path']}" | |
| ) | |
| return icon["filename"] | |
| return None | |
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | |
| results = executor.map(_download_worker, icons_result.data) | |
| failed_icons = [filename for filename in results if filename is not None] | |
| def _download_worker(icon): | |
| # Create thread-local client for safety, matching pattern in _copy_icon_worker | |
| from supabase import create_client | |
| thread_supabase = create_client(SUPABASE_URL, SUPABASE_SECRET_KEY) | |
| icon_path = session_icons_dir / icon["filename"] | |
| try: | |
| file_data = thread_supabase.storage.from_("resume-icons").download(icon["storage_path"]) | |
| with open(icon_path, "wb") as f: | |
| f.write(file_data) | |
| return None | |
| except Exception as e: | |
| logging.error(f"Failed to download icon: {icon['filename']} - {e}") | |
| return icon["filename"] | |
| with ThreadPoolExecutor(max_workers=MAX_ICON_COPY_WORKERS) as executor: | |
| results = executor.map(_download_worker, icons_result.data) | |
| failed_icons = [filename for filename in results if filename is not None] |
References
- To ensure thread safety, each worker in a thread pool should create its own instance of clients that are not thread-safe (e.g., database clients).
What: Refactored sequential loops inside
app.py(icon downloading) anduseIconRegistry.ts(file to base64 conversion) to execute concurrently viaThreadPoolExecutorandPromise.all.Why: Iterating over independent network or disk I/O sequentially creates cumulative latency. Concurrent execution overlaps wait times, optimizing latency in both environments.
Impact: Reduces I/O delay times linearly proportional to the number of icons being downloaded or exported, improving PDF/thumbnail generation latency.
Measurement: Execution time to export or download multiple icons decreases as tasks run in parallel.
PR created automatically by Jules for task 124923643150996873 started by @aafre