Skip to content

[Bug]: Two downloads with the same filename overwrite each other's temp file #332

Description

@Kaushik2003

Description

While downloading, the file is written to a temporary path built only from the
destination:

// src/download/index.ts:119
const tempPath = `${destPath}.tmp`;

The download pipeline step runs several items at the same time (3 by default,
src/pipeline/steps/download.ts:131), and two items can easily end up with the
same filename — either because the filename template produces the same name,
or because generateFilename() falls back to something derived from the URL.

When that happens, both downloads write to the same .tmp file. Each one
opens its own fs.createWriteStream(tempPath), so the second truncates the file
the first is still filling, and then both rename it to the final name.

Two things go wrong, depending on the timing:

  • The file that survives may hold bytes from two different downloads.
  • Whichever attempt renames second fails with
    ENOENT: no such file or directory, rename '.../file.bin.tmp' -> '.../file.bin',
    because the other attempt already moved the shared temp file away.

The cleanup path makes it worse: if one download fails, cleanupTempFile()
(src/download/index.ts:121-127) deletes ${destPath}.tmp — which may be the
other download's file, still in progress.

Steps to Reproduce

  1. Write an adapter with a download step whose filename template produces the
    same name for more than one item, for example a fixed filename: report.pdf
    with several items.
  2. Leave concurrency at its default of 3.
  3. Run the command against two or more items with different URLs.
  4. Only one file is written, and its contents are not reliably from either URL.

The clobbering can also be seen directly against httpDownload, with no adapter
involved: start one download whose response body is held open, and while it is
still streaming, start a second download to the same destination path. The
second one truncates and renames the first one's temp file, and the first fails
on rename.

Expected Behavior

Each download attempt should use its own temporary file, so parallel downloads
never touch each other's data. A unique suffix would be enough, for example
${destPath}.${process.pid}-${counter}.tmp.

Downloads that genuinely target the same destination path should either be
serialized or reported as a conflict — not silently interleaved.

webcmd Version

0.7.1

Node.js Version

Other

Operating System

Linux

Logs / Screenshots

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't working

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions