VideoRebirthAI

/

hyperflow-h3-flash

Like 176

Follow

Video Rebirth 47

Hi @darksidewalker — we don't have plans to release the unquantized FP16/BF16 weights at the moment. If there's a specific quant or format you need (beyond the single-file ComfyUI INT8 release we're considering in #2), let us know what you're targeting.

@videorebirth I think W4A8 and W6A8 convrot quants are good variants to add. These quants have faster generation speed than 4/6-bit quantized GGUFs.
Thanks for your amazing work!

Thanks @fm3at ! Two updates from our side:

1. Single-file W8A8 INT8 for ComfyUI is now live. fl2va_int8_convrot_comfyui/ and ref2va_int8_convrot_comfyui/ are in this repo, loaded with HyperFlow Loader from ComfyUI-HyperFlow-Flash 1.1.0 or newer. On RTX 4090 / 5090 they run faster than the Q4_K_M GGUF, so you're right about convrot vs. GGUF speed.

2. Why we're holding off on W4A8 / W6A8 for now:

   • Speed: the gain over GGUF comes mainly from the INT8 activation path, which W8A8 already has. Lower-bit weights mostly buy size, not speed.
   • Size: Flash is already pruned and low-rank factored, and only part of the file is INT8 weights. So lower-bit weights shrink it much less than they do on the base model.
   • Offload and quality: W8A8 does not have to fit in VRAM. Each step is compute-bound, so streaming the weights in block by block costs very little speed. Our Python package does this with --offload stream, though it needs a lot of system RAM today. And on a few-step model without CFG, we haven't validated 4-bit or 6-bit convrot yet, so we'd rather not trade quality for size.

If you're on a 24 GB+ RTX card, give the new *_int8_convrot_comfyui files a try. For smaller GPUs we recommend the GGUFs for now. If that doesn't work for your setup, tell us your GPU and VRAM and we'll take a look.

@videorebirth Thanks for the reply! I have an Nvidia RTX 5070 Ti with 16 GB VRAM. I've already tried the INT8 version with your loader in ComfyUI, but it crashed with an Out of Memory error. The Q4_K_M GGUF quant provides good quality, but it's significantly slower than even the standard minimax INT8 weights. While monitoring the loading process for your INT8 model, I noticed that the weights are trying to load entirely into VRAM instead of utilizing ComfyUI's smart memory management. If possible, could this be fixed somehow?

UPD: i rewrite your comfyui loader node with local llm and got working int8 weights with comfyui dynamic smart memory. got 4,8 s\it with 0.5 mp 5 seconds length video. smaller q4_k_m gguf gives me 8.8s\it. your int8 weights quality is perfect for 14b model, thanks again for this superb work.