How to train from scratch with sd3 or flux with this code?

[ZhendongWang6]

Hi, very nice work. I would like to know how to train from scratch with sd3 or flux with the proposed pyramid-flow framework? Is there any code? I could not find it.

[feifeiobama]

pretraining script is essentially the same as finetuning script (https://github.com/jy0205/Pyramid-Flow/blob/main/scripts/train_pyramid_flow.sh)

can check https://github.com/jy0205/Pyramid-Flow/blob/main/docs/DiT.md for docs

[ZhendongWang6]

I see. How to load from original sd3 weights while within the pyramid-flow framework? but not to load your provided mmdit weights.

[ZhendongWang6]

Or to say, how to change the parameters in train_pyramid_flow.sh) to achieve my pre-training purpose?

[feifeiobama]

have you tried https://github.com/jy0205/Pyramid-Flow/blob/main/train/train_pyramid_flow.py#L131

[ZhendongWang6]

Yes, I have tried it. But I find the current code only supports safetensors loading. I modified it to load safetensors from SD3-Medium, shown as follows. There are missing keys. Why?

[feifeiobama]

can just set strict=false cc @jy0205

[ZhendongWang6]

That can pass, but if that, some of parameters in PyramidDiffusionMMDiT are not found.