Models on Apple Silicon

Large open-weight image and language models are too slow, or too big, to run well on a Mac, and a quick port can quietly change what they draw.

Distilled Ideogram 4 for MLX by folding classifier-free guidance into a single branch and cutting sampling steps from 12 to 6. At 800×992 on an M4 Pro, an image takes ~86 s instead of ~250 s (2.9× faster). Quantized to 18.6 GB, it runs on a 48 GB Mac.

Extracted Qwen3.8-27B’s multi-token-prediction head as a speculative-decoding drafter. On coding prompts, decode goes from 11.4 to 21.5 tokens/s (up to 1.89× faster). Released 6-bit MLX weights with a full-precision vision tower. The two Qwen releases have 7.7K downloads on Hugging Face, for a model whose ~21 GB of weights need a Mac with 36 GB or more: desktop machines, not phones or base laptops.

Ported FLUX 3 Image Klein to MLX and released it the day Black Forest Labs did. At 1024², the port’s render of BFL’s own example matches BFL’s published image at 35.9 dB, visually indistinguishable; the 8-bit tier stays at 34.2 dB. It covers text-to-image, editing with up to 10 references and bounding-box layout, and the 6-bit tier runs in 9.6 GB.

Solved the VAE’s memory without tiling seams: the decoder runs its convolutions in row strips with a one-row halo, over whole-image GroupNorm statistics, and its attention in query chunks. The output is identical to a full-tensor decode, but a 1024² image needs 3.9 GB instead of 14 GB, and a 2048² decode needs 10.4 GB.

Ported MOSS-Image, OpenMOSS’s bilingual text-rendering model, exactly: 69.7 dB against the official PyTorch pipeline in fp32, with the sampler and seed noise matched bit for bit, so the official seeds redraw the official gallery. The 8-bit tier peaks at 9.3 GB for a 1024² image, against 38.7 GB for mlx-vlm’s port on the same Mac, and its OCR matches the unquantized model on every text prompt.

Trained a few-step mode for MOSS-Image. A small student, trained on the full model’s own 50-step trajectories, draws the composition half; one guided step from the original model sets the lettering, and the original model finishes the detail. A ~1 MP image takes about 2 minutes instead of 12½ (15 network passes instead of 100, 6.2× faster), and English OCR recall holds (0.947 against the full model’s 0.934).

Found why the community’s few-step MOSS-Image adapters paint organic surfaces with copies of one motif (lichen as identical rosettes, pebbles as beads), and why the released mode doesn’t. A pre-registered test on SDXL, Qwen-Image and FLUX.1-dev showed the defect does not generalize to other distilled models.

Agent lane: a serving recipe, a thinking-budget proxy and four patches to run Qwen3.8-27B as a local agent backend, at 15–19 tokens/s end to end with 24–27k tokens of context.

1. Open-weight checkpoint
2. MLX port
3. Parity with the official pipeline
4. Distill · quantize
5. Few-step student
6. Speculative decoding
7. Benchmark on one machine
8. Release on Hugging Face

1. Report speedups as ratios, not raw tokens/s: absolute throughput depends on the Mac, the ratio transfers.
2. Benchmark everything on one machine (Mac mini, M4 Pro, 48 GB) and ship the setup and known limitations with each release.
3. Fold classifier-free guidance into one branch before step-distilling 12 → 6.
4. Distil only the composition half of MOSS-Image’s trajectory and let the original model finish the detail: each distilled step in the mid-noise band added repeated texture.
5. Prove a port exact before quantizing it: fp32 parity with the official pipeline and the official seeds, then each tier gated on OCR and added grain.
6. Decode in strips with a halo, not tiles: the same pixels as a full decode, at a quarter of the memory.
7. Ship no 4-bit image tier that adds grain or garbles text, even where it would save memory.

• 6.2× fasterMOSS-Image, ~1 MP on an M4 Pro: the trained few-step mode vs the official 50 steps (~743 s → ~119 s)moss-image-mlx README and research/turbo
• 1.89× fasterQwen3.8-27B coding decode with the MTP drafter (11.4 → 21.5 tokens/s)MTP drafter model card, Hugging Face
• 2.9× fasterIdeogram 4 at 800×992, 6-step release vs stock (~250 s → ~86 s)ideogram4-mlx-distill, BENCHMARKS.md
• 4× less memoryMOSS-Image, one 1024² image: 9.3 GB peak for the 8-bit port vs 38.7 GB for mlx-vlm’smoss-image-mlx README, same Mac
• 35.9 dBFLUX 3 Klein port vs Black Forest Labs’ published render, 1024²: visually indistinguishableflux3-mlx README
• 7.7Kdownloads of the two Qwen3.8-27B releases, a model that needs a 36 GB+ MacHugging Face API, all-time downloads, read at build; model card (memory)

MLX, mflux, mlx-vlm, Python, Distillation, LoRA training, Quantization, Speculative decoding, Parity testing, OCR evals