Wenwu2000/MobileDiffuser-SD3-medium

MobileDiffuser Core ML Models (SD3 Medium and FLUX.2 Klein)

Compiled Core ML resources for MobileDiffuser, an open-source iOS app that runs diffusion models entirely on the iPhone with Core ML on the Apple Neural Engine (ANE).

• Code: https://github.com/TWWinde/MobileDiffuser/tree/MCL
• Project page: https://twwinde.github.io/mobilediffuser-project-page/

The image above is FLUX.2 Klein, 4 steps, 1024 x 1024, from the Core ML stages in this repository (rendered on a Mac).

The repository name says "SD3-medium" for historical reasons; it now also holds the FLUX.2 Klein resources, because the app downloads everything from here.

Contents

Folder	Model	Resolution	Size	Used by the app
coremlsd3_2step/	SD3 Medium, 2-step distilled	512	2.9 GB	SD3M · 2 steps
coremlsd3_reseed/	SD3 Medium, 4-step ReSeed student	512	2.9 GB	SD3M · 4 steps · 512
coremlsd3_reseed_1024/	SD3 Medium, 4-step ReSeed student	1024	2.9 GB	SD3M · 4 steps · 1024
coreml_flux2_klein_1024/	FLUX.2 Klein 4B, 4-step distilled	1024	7.5 GB	FLUX.2 Klein · 1024
coreml_flux2_klein_512/	FLUX.2 Klein 4B, denoiser only	512	3.9 GB	FLUX.2 Klein · 512 and the fast mode
coremlsd3_4step/, coremlsd3_4step_1024/	SD3 Medium, earlier 4-step student	512 / 1024	2.9 GB each	legacy, no longer used by the app
sd3-medium-4step-mlx-4bit/	SD3 Medium 4-step, MLX 4-bit weights	—	1.3 GB	not used by the Core ML app
checkpoints/	Source distilled SD3 Medium checkpoints (.safetensors)	—	16.7 GB	for reproducing the conversion

SD3 Medium folders

TextEncoder.mlmodelc, TextEncoder2.mlmodelc (CLIP-L / CLIP-G), VAEDecoder.mlmodelc, MultiModalDiffusionTransformerConditioning.mlmodelc, the split denoiser MultiModalDiffusionTransformerStage*.mlmodelc, and the tokenizer files.

• The coremlsd3_reseed* folders hold one transformer block per stage (Stage0 … Stage24) and ReSeedBiases.json. The weights are the ReSeed SD3 Medium 4-step student; ReSeed strength 0 is the plain student, so one set of weights serves both modes.
• The 1024 folders use exact query-chunked attention and are decoded with the 512 VAE as 3 x 3 tiles.
• Denoiser weights are INT8 (linear symmetric).

FLUX.2 Klein folders

• Embed, Double0…4, Single0…19, Final: the denoiser, one transformer block per stage, INT8.
• TextStage0…8 and embed_tokens.f16: the first 27 layers of the Qwen3-4B text encoder and its token embedding table. Prompts use 256 tokens.
• KleinVAEDecoder: FLUX.2 VAE decoder for a 512 tile; 1024 is decoded as 3 x 3 tiles.
• tokenizer.json, RoPE tables and small JSON configuration files.

The text encoder, VAE and tokenizer exist only in coreml_flux2_klein_1024/ and are shared by both resolutions, so coreml_flux2_klein_512/ needs the 1024 folder next to it.

Performance

iPhone 15 Pro, iOS 18.5, cpuAndNeuralEngine, warm runs in the app:

Model	Resolution	Total	App memory
SD3 Medium 2-step	512	~5.6 s	~86 MB
SD3 Medium 4-step (+ ReSeed)	512	~9.4 s	~58 MB
SD3 Medium 4-step (+ ReSeed)	1024	~31 s	~146 MB
FLUX.2 Klein 4-step, full resolution	1024	~77 s	~185 MB
FLUX.2 Klein 4-step, fast (3 steps at 512, 1 at 1024)	1024	~33 s	~220 MB

The first generation after install also compiles the stages on the phone, once (about 45 s for SD3 Medium at 1024, a few minutes for FLUX.2 Klein). FLUX.2 Klein at 512 has not been timed on the phone. Details are in the repository docs.

Usage

In the app (recommended). Build MobileDiffuser from the MCL branch, open Settings and download the resources for the selected model. No Python, Git LFS or CLI is needed. For FLUX.2 Klein at 512 or the fast mode, download the 1024 folder as well.

Manually. Download a folder and place it in the app's Application Support/MobileDiffuserModels/ directory, or bundle it into the app target for development:

hf download Wenwu2000/MobileDiffuser-SD3-medium --include "coremlsd3_reseed/*" --local-dir .

Reproducing the conversion. Conversion scripts and notes are in the code repository: docs/REPRODUCING_MODELS.md, docs/ANE_1024.md, docs/RESEED.md and docs/FLUX2_KLEIN.md.

Notes

• The resources are compiled for fixed shapes and are meant for cpuAndNeuralEngine.
• All models are step- and guidance-distilled; use guidance scale 1.0.
• checkpoints/sd3-medium-4step.safetensors is the earlier MobileDiffuser 4-step student used for coremlsd3_4step*; the coremlsd3_reseed* folders were converted from the ReSeed student in Wenwu2000/ReSeed.

Licenses

• Stable Diffusion 3 Medium derivatives (all coremlsd3_* folders, checkpoints/, sd3-medium-4step-mlx-4bit/): Stability AI Community License of the upstream model.
• FLUX.2 Klein 4B and its Qwen3-4B text encoder (coreml_flux2_klein_*): Apache-2.0.

Make sure your use complies with the upstream licenses.

Citation

@misc{tang2025mobilediffuser,
  author       = {Wenwu Tang and Dong Wang and Olga Saukh},
  title        = {MobileDiffuser: On-device Stable Diffusion 3 Medium on iPhone with Core ML},
  year         = {2025},
  publisher    = {GitHub},
  howpublished = {\url{https://github.com/TWWinde/MobileDiffuser}}
}