The first `backup` after a `restore` is much slow than subsequent `backup`s

[ltratt]

[This is on rustic-main, pointing to rustic_core-main compiled as of yesterday.]

If (on a file system that's 99% got the right files at the start) I do restore then backup then backup I see the following timings: 40s (restore); 321s (backup); 29s (backup). In other words, the first backup is 10x slower than the second backup.

Judging by the I/O activity I observe, the first backup is probably reading every file in and checksumming, even though it presumably knew what they all were from the restore? Once that's done, subsequent backups remain very fast (until the next restore).

Naively, I wonder if the cache is not being updated (fully? partially?) after restore?

[aawsome]

Hi @ltratt! Thanks for opening this issue!

I think the problem here is how restore works:

• It does a lot of effort to determine which data is really needed to restore the contents of the files-to-restore. (This is because accessing this can be costly as this may be on a remote storage and even may need a warm-up) Then it restores the contents of all files.
• In a second run it simply sets the metadata of all files (reading metadata is cheap as this is typically locally cached or if not it exists on a hot repository)

I think that the second step in fact updates the metadata of all files setting a new mtime. And this results in a subsequent backup run not being able to detect files as unchanged. Which means each file is re-read. (Note that there is no cache to be updated. backup uses a parent snapshot and a change detection algorithm based on the metadata saved in this parent snapshot)

So, to solve this issue, the restore should be able to detect if the metadata is already correctly set and skip setting it, if-so.

[ltratt]

@aawsome That hadn't occurred to me -- your reasoning makes much more sense than my guess at an explanation!

[FWIW, I've recently evaluated a number of backup tools. rustic's performnace is by far the best, with only this restore/backup issue coming up in my tests. I'm impressed!]