mode = force: only delete the existing cache after computation

[sdascoli]

It often happens that we set mode=force to overwrite a cache, then forget to switch back just after, run the function and it deletes the cache we just took ages to populate. And now it's too late...
Would it be possible to delete the cache only after the computation (or simple overwrite it), so that when this happens we can just stop the accidentally started job without loosing anything?

[sdascoli]

(also, isn't "overwrite" a better name than "force"?)

[kingjr]

It often happens that we set mode=force to overwrite a cache, then forget to switch back just after, run the function and it deletes the cache we just took ages to populate. And now it's too late...

I also faced this issue.

Quite hard to handle with the job / cache checking however, thoughts @jrapin ?

[jrapin]

Could be nice indeed but I expect it's complex to implement, the current implementation is just a few lines deleting the cache once, and then running the regular code. Switching to deletion after computation implies many small changes in different parts of the pipeline.

Also, while the current behavior is painful, I am a bit afraid of what would happen without it, if something bugs for part of the computation, you could end up with a mixture of old and new cache without noticing

[kingjr]

Yes I would recommend to simply change the cache folder is these situations.