Multipart Parallel Downloading

[Abdel427]

Participants :

Mohamed Bouhajra & Abdel-Basser Laamouri

Description

This project introduces a Multipart Parallel Transfer system designed to optimize the retrieval of massive datasets and archives within MByte. The engine logically divides heavy assets into independent segments (chunks) that are fetched simultaneously via concurrent network streams, maximizing bandwidth saturation and throughput. This architecture provides granular fault tolerance: in the event of a connection drop or failure, the client automatically validates and retries only the specific missing segments. This ensures that multi-gigabyte transfers are robust and efficient, eliminating the need to re-transfer the entire file when errors occur during long operations.

[jayblanc]

Creating multiple connection simultaneously on the same server won't increase the bandwidth (or not so much).
Anyway, the subject is interesting because it requires to 'chunk' the file to allow download part by part.
The simultaneous download is interesting when the same file is present on many server on the neighborhood ; in that situation the client can download chunks from multiple servers at the time.
It will require a client side chunk storage before recreating the real file. You will have to use local storage for that.