FreeToken Run 290B+ AI models on your own hardware

By Strange Man · Published 1 Sep 2026

Found something pretty damn interesting: FreeToken by FlashML.

It’s an edge native MoE serving engine designed to run frontier scale open weight models locally, using your GPUs + CPU + system memory as one inference platform.

Some of the interesting bits:

290B+ MoE models on consumer hardware

Adaptive CPU/GPU execution

Expert + KV-cache memory management

NVIDIA RTX 30/40/50 support

OpenAI + Anthropic-compatible APIs

Supports models like DeepSeek, Qwen and GLM

Basically: less cloud dependency, more AI running directly on your own machine.

GitHub: https://github.com/FlashML-org/FreeToken

Definitely worth checking out if you're into local LLMs, inference optimization, or AI infra.

GitHub - FlashML-org/FreeToken: FreeToken brings datacenter-scale m... (https://github.com/FlashML-org/FreeToken)

FreeToken brings datacenter-scale model serving to your desktop. Run massive models locally, fast and efficiently. - FlashML-org/FreeToken