Not sure it will be really usable. Check for Flash-Moe and Colibri repos A lot of request for qwen3.6 moe, it might worth exploring

Yeah, Gemma is not the best for coding I guess. qwen must be better

Yeah, the same ideas should work for qwen. You can try porting this engine to use Owen.

Owen 3.6-35b-a3b was my initial idea, but I switched to Gemma because of its simpler architecture and kernels

Curious if the same idea could work with gpt-oss-120b? So one could run at least slowly on a Mac

Yeah, gpt-oss-120b is also MoE, so the same ssd-streaming and caching ideas should work. Feel free to fork and try implementing it!

Thank you very much for sharing! Great results and useful info!

uh, it's a bit difficult to discuss the classical approach with vRAM and regular RAM. Not really familiar with optimisations and hacks, I always worked with apple platforms and shared memory. But description sounds cool, good luck with your project!

Yeah, I tried both rearranging experts on disk and predicting the next expert using statistical approach. Reordering helped on the test prompt, but failed on another prompt. Markov and cross layer prediction didn't work either

It is Apple platform only implementation because of Metal (and Swift). Other platforms would require CUDA or Vulkan and a complete rework

hm. just open repo, copy commands into your terminal and you will get app installed (if you have swift toolchain installed)

after that download 14gb of weights and enjoy offline inference (and a bit of Gemma4 intelligence) for your everyday tasks

multi turn chat is coming!

what is a swift tool chain?

uh, don't worry. Just install the latest Xcode from the App Store. It includes everything you need to run this project

Thank you! If you can use it for your tasks I would be happy!

uh, I don't think it is possible to compare them. DwarfStar4 is for high end macs and a lot of ram. this project is more targeted to low end devices and "general use" Gemma4 model