Optimize interleaved pipeline for numpy array buffer cache

[meatybobby]

Is your feature request related to a problem? Please describe.
Mentioned in #1583 (comment). Currently we load image bytes to numpy array every time when we apply a filter

Describe the solution you'd like
Add a buffer cache in InterleavedBatch

Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.

Additional context
Add any other context or screenshots about the feature request here.