Save/restore compressed transitions

[keiohta]

I'd like to request a feature that enables users to save and restore stored transitions with as small data size as possible.
The current implementation does allow users to do that as:

samples = rb.get_all_transitions()
joblib.dump(samples, "filename.pkl", compress=3)

, but this can be improved.

For example, in most cases samples["states"] and samples["next_states"] considerably overlaps except the last state, and since the state occupies the most part of the stored data (imagine Atari games whose input is 84x84x4 and output is scalar value), so just removing the overlapped state enables to reduce almost half of total data size.

There should be more sophisticated ways to enable that, so let's discuss in this thread.

[ymd-h]

@keiohta Thank you for your request!

We might reduce data size by saving internal data structure directly, as long as user enables next_of or stack_compress.

However, saving internal data structure has problem of version compatibility.
Now, I only take care of public API of cpprb (not internal data structure).
If I provide save feature, users would expect their saved data can be used later. (Otherwise, they would be disappointed...)

Anyway, I need some investigation.
Do you know whether these overlapping data do affect compressed data size? (Are there any smart compression algorithm?)

[keiohta]

Great. (btw, it might be better to show Y-axis with log-scale.)

Removing the overlapped data requires us to know the end of an episode, and I think we can do that by calling on_episode_end method similar to multi-step return. Although I'm not familiar with the internal structure, I think the combination of the following two realize the function.

1. specifying next_of and stack_compress when instantiating a buffer
2. calling on_episode_end when finishing an episode

How do you think of this?

[ymd-h]

@keiohta

1. specifying next_of and stack_compress when instantiating a buffer
2. calling on_episode_end when finishing an episode

I agree, otherwise buffer cannot know.

In my quick idea, def save(self, filename, *, compress=3, safe=True) is a candidate API.

When user manually specifysafe=False, then the buffer saves internal compressed data.
The default safe=True calls get_all_transitions().

The other point is, can we depend joblib? (or any other library?)

[keiohta]

Yeah, I think that would be a good interface.

Another important point is how users can restore the compressed data. I think it would be great if API can also prepare a method that restore the compressed data and return exactly the same format with get_all_transitions, i.e., it includes states and next_states.

[keiohta]

The other point is, can we depend joblib? (or any other library?)

I don't have knowledge around compression libraries... I think just using joblib would be fine.

[ymd-h]

@keiohta

I released cpprb v10.1.0 just now.

ReplayBuffer.save_transitions(file, *, safe=True) and ReplayBuffer.load_transitions(file) are introduced.

If next_of and/or stack_compress are enabled and safe=False is manually specified, then buffer tries to reduce the file size.

Please try it!

[ymd-h]

Save and load transitions are introduced at cpprb v10.1.0

Please see document.