Update README.md (#6)

- Update README.md (381368668dc85e82e908e241b1cd38451910339a)


Co-authored-by: Zhicheng Sun <[email protected]>

README.md

@@ -28,15 +28,16 @@ This is the model repository for Pyramid Flow, a training-efficient **Autoregres
 
 ## News
 
-* `2024.10.29` ⚡️⚡️⚡️ We release [training code](https://github.com/jy0205/Pyramid-Flow?tab=readme-ov-file#training) and [new model checkpoints](https://huggingface.co/rain1011/pyramid-flow-miniflux) with FLUX structure trained from scratch.
+* `2024.11.13`  🚀🚀🚀 We release the [768p miniFLUX checkpoint](https://huggingface.co/rain1011/pyramid-flow-miniflux) (up to 10s).
 
-  > We have switched the model structure from SD3 to a mini FLUX to fix human structure issues, please try our 1024p image checkpoint and 384p video checkpoint. These checkpoints are trained with synthetic data from FLUX. We will release 768p video checkpoint in a few days.
+  > We have switched the model structure from SD3 to a mini FLUX to fix human structure issues, please try our 1024p image checkpoint, 384p video checkpoint (up to 5s) and 768p video checkpoint (up to 10s). The new miniflux model shows great improvement on human structure and motion stability
+* `2024.10.29` ⚡️⚡️⚡️ We release [training code](https://github.com/jy0205/Pyramid-Flow?tab=readme-ov-file#training) and [new model checkpoints](https://huggingface.co/rain1011/pyramid-flow-miniflux) with FLUX structure trained from scratch.
 * `2024.10.11`  🤗🤗🤗 [Hugging Face demo](https://huggingface.co/spaces/Pyramid-Flow/pyramid-flow) is available. Thanks [@multimodalart](https://huggingface.co/multimodalart) for the commit! 
 * `2024.10.10`  🚀🚀🚀 We release the [technical report](https://arxiv.org/abs/2410.05954), [project page](https://pyramid-flow.github.io) and [model checkpoint](https://huggingface.co/rain1011/pyramid-flow-sd3) of Pyramid Flow.
 
 ## Installation
 
-We recommend setting up the environment with conda. The codebase currently uses Python 3.8.10 and PyTorch 2.1.2, and we are actively working to support a wider range of versions.
+We recommend setting up the environment with conda. The codebase currently uses Python 3.8.10 and PyTorch 2.1.2 ([guide](https://pytorch.org/get-started/previous-versions/#v212)), and we are actively working to support a wider range of versions.
 
 ```bash
 git clone https://github.com/jy0205/Pyramid-Flow
@@ -48,7 +49,7 @@ conda activate pyramid
 pip install -r requirements.txt
 ```
 
-Then, download the model from [Huggingface](https://huggingface.co/rain1011) (there are two variants: [miniFLUX](https://huggingface.co/rain1011/pyramid-flow-miniflux) or [SD3](https://huggingface.co/rain1011/pyramid-flow-sd3)). The miniFLUX models support 1024p image and 384p video generation, and the SD3-based models support 768p and 384p video generation. The 384p checkpoint generates 5-second video at 24FPS, while the 768p checkpoint generates up to 10-second video at 24FPS.
+Then, download the model from [Huggingface](https://huggingface.co/rain1011) (there are two variants: [miniFLUX](https://huggingface.co/rain1011/pyramid-flow-miniflux) or [SD3](https://huggingface.co/rain1011/pyramid-flow-sd3)). The miniFLUX models support 1024p image, 384p and 768p video generation, and the SD3-based models support 768p and 384p video generation. The 384p checkpoint generates 5-second video at 24FPS, while the 768p checkpoint generates up to 10-second video at 24FPS.
 
 ```python
 from huggingface_hub import snapshot_download
@@ -74,8 +75,9 @@ model_dtype, torch_dtype = 'bf16', torch.bfloat16   # Use bf16 (not support fp16
 
 model = PyramidDiTForVideoGeneration(
     'PATH',                                         # The downloaded checkpoint dir
+    model_name="pyramid_flux",
     model_dtype,
-    model_variant='diffusion_transformer_384p',     # SD3 supports 'diffusion_transformer_768p'
+    model_variant='diffusion_transformer_768p',
 )
 
 model.vae.enable_tiling()
@@ -92,15 +94,23 @@ Then, you can try text-to-video generation on your own prompts:
 ```python
 prompt = "A movie trailer featuring the adventures of the 30 year old space man wearing a red wool knitted motorcycle helmet, blue sky, salt desert, cinematic style, shot on 35mm film, vivid colors"
 
+# used for 384p model variant
+# width = 640
+# height = 384
+
+# used for 768p model variant
+width = 1280
+height = 768
+
 with torch.no_grad(), torch.cuda.amp.autocast(enabled=True, dtype=torch_dtype):
     frames = model.generate(
         prompt=prompt,
         num_inference_steps=[20, 20, 20],
         video_num_inference_steps=[10, 10, 10],
-        height=384,     
-        width=640,
+        height=height,     
+        width=width,
         temp=16,                    # temp=16: 5s, temp=31: 10s
-        guidance_scale=9.0,         # The guidance for the first frame, set it to 7 for 384p variant
+        guidance_scale=7.0,         # The guidance for the first frame, set it to 7 for 384p variant
         video_guidance_scale=5.0,   # The guidance for the other video latent
         output_type="pil",
         save_memory=True,           # If you have enough GPU memory, set it to `False` to improve vae decoding speed
@@ -112,7 +122,15 @@ export_to_video(frames, "./text_to_video_sample.mp4", fps=24)
 As an autoregressive model, our model also supports (text conditioned) image-to-video generation:
 
 ```python
-image = Image.open('assets/the_great_wall.jpg').convert("RGB").resize((640, 384))
+# used for 384p model variant
+# width = 640
+# height = 384
+
+# used for 768p model variant
+width = 1280
+height = 768
+
+image = Image.open('assets/the_great_wall.jpg').convert("RGB").resize((width, height))
 prompt = "FPV flying over the Great Wall"
 
 with torch.no_grad(), torch.cuda.amp.autocast(enabled=True, dtype=torch_dtype):