Kimi K2.6 has 1 TRILLION parameters. Can you run it on one machine? Let's do the math 🧮
- 1T params × 2 bytes = 2 TB. That's 25 H100s just for the weights
- Trained for 4-bit → about 600 GB
- Mixture of experts: 384 experts, 8 wake up per token → only 32B active
- KV cache: ~70 KB per token → a full 256K chat is ~18 GB
- 8× H100 (640 GB): fits, with room for about two long chats
- 8× H200 (1.1 TB): fits, with ~500 GB left to serve a team
So yes, one machine. Buy memory, not compute.
Comment KIMI and I'll send you the sizing sheet.
(Weights and cache only; runtime overhead ignored.)
Commented KIMI? No need to wait for a DMThe trillion-parameter sizing sheet: will Kimi K2.6 fit on your GPUs?Every number from the reel, where it comes from, and the three formulas you need to size any model yourself. Opens in Google Sheets or Excel.