bookmarks

@trymirai

Mirai Labs · 1,124 followers

Frontier on-device AI lab. Models, runtime & infrastructure to make on-device AI interactive, ambient & continuous.

1 bookmark

View all →

Today we are releasing our speculative decoding implementation in our inference engine uzu. Initially for Qwen3.6 27B, with support for Qwen3.8 27B and Muse Glimmer coming soon. On Apple M5-series chips, we outperform MTPLX (MLX + speculative decoding) by almost 2x, and llama.c…↗