⚡ Native MLX Swift LLM inference server for Apple Silicon. OpenAI-compatible API, SSD streaming for 100B+ MoE models, TurboQuant KV cache compression, MACOS + iOS iPhone app.
Swift777 stars54 forksMIT apple-siliinferenceiosllmmetalmlxmoeon-device-aiopenai-apiswift