Do you have a novel VAE also?

[ketyi]

Dear Authors,

To achieve this encoding

you need to have another VAE than WAN 2.1's one, right? 👀

[caijiaran1993]

No, it's not needed. In fact, our operation involves duplicating the first frame and concatenating it to the front of the original video. After passing through the VAE, we discard the latent code corresponding to this duplicated frame. In this way, the resulting latent code has each channel corresponding to 4 video frames.

[lzoltan35]

@caijiaran1993 I would like to ask, how does the inference process work in this case? I'm mainly interested in generating the very first latent.
We have a single reference image, do we repeat it 4 times and feed this to the VAE to get a single latent?