Back of the envelope calculation (could be off, correct me if I am)
If you are a large enough company that spends million+ on inference a month, it makes sense to buy a GB300 rack ($6M on top range from what I could find) which has 20.7 TB. Since the model is mixed trained (MXFP4), you would need less than 10% of the rack's memory to serve the full model. Aggregate HBM bandwidth: 576 TB/s. You can run over 6000 parallel agentic workflows (each with ~100k context on average) at ~30 tok/s.
Assuming the annual amortization+electricity at $1.5M/year and about 50% average annual utilization, you get less than 60 cents (USD) per million output token, for a frontier model with plenty of capacity to share, all your data never leaving premises and well over an order of magnitude cheaper!
As long as a company believes that the openweight models will continue to get more capable and 'AI is here to stay', this model provides the first solid footing for a decision to just buy a rack.
show comments
m_ke
Also open sourced a bunch of infra to go with it.
Anyone who claims open source and open weights models are "decel" needs to get their head checked
> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.
+ the existing 100 million monthly active users, or more than 20 million US dollars for commercial products have to name Kimi clause
show comments
eamag
> we build a self-evolving, hierarchically organized knowledge graph that agents continuously expand through web-scale exploration across knowledge-intensive and coding domains
That's interesting!
whimsicalism
It's funny that we've finally returned to tanh activation functions, time is a circle.
show comments
a-dub
knowledge graph guided task synthesis. very cool! i have long wondered about the "how do you get good coverage of all the tasks" problem.
maybe some interesting theoretical work there around the rate of production of new knowledge itself and various mechanisms (human approaches, mechanistic approaches, etc).
storus
What would be the current best method to fine-tune it for my own specific agentic tasks? LoRA + DPO? GRPO? Something else?
show comments
eamag
Can someone explain what are teachers in Multi-Teacher On-Policy Distillation? I can imagine math, coding and other verifiable domains, but they also have biology? Is it where distillation from bigger models come in?
show comments
htrp
Why is everyone's pricing the same? licensing agreements with moonshot as an inference provider?
NguyenDat377
That's pretty impressive, I don't want to be the one undermine but I personally think the industry is shifting to rather designs that enable Models to work instead of chugging it all to data and compute ?
lenerdenator
What would it take to get an American open model to compete with this?
show comments
logicallee
What are the coding/agent harnesses (like Claude Code or ChatGPT Codex) that can be used with this?
weberer
Does anyone know if a torrent is available? I think it would take quite a while to download 1.5tb from their servers.
show comments
Mohiuddin7
That's the great achievement, but ahh, wanted to try it, but my RAM had other plans
Tejush
Any guess on the pre-training tokens/flops they consumed?
show comments
colesantiago
This is amazing to witness.
Moonshot open sourcing Kimi K3, a frontier AI and other components really means we are getting abundant AI for all of humanity.
Kudos to Moonshot for truly being what OpenAI should have been.
Fable-level and frontier AI should be open source and available to everyone for free.
show comments
m00dy
I would want to see three things before drawing strong conclusions:
End-to-end tokens/sec and cost on realistic coding agent trajectories, including tool outputs and retries, not isolated decode benchmarks.
Cache hit rates and prefill cost for branching, multi-turn sessions.
Router-load distributions after post-training, where expert collapse or specialization problems often show up.
Back of the envelope calculation (could be off, correct me if I am)
If you are a large enough company that spends million+ on inference a month, it makes sense to buy a GB300 rack ($6M on top range from what I could find) which has 20.7 TB. Since the model is mixed trained (MXFP4), you would need less than 10% of the rack's memory to serve the full model. Aggregate HBM bandwidth: 576 TB/s. You can run over 6000 parallel agentic workflows (each with ~100k context on average) at ~30 tok/s.
Assuming the annual amortization+electricity at $1.5M/year and about 50% average annual utilization, you get less than 60 cents (USD) per million output token, for a frontier model with plenty of capacity to share, all your data never leaving premises and well over an order of magnitude cheaper!
As long as a company believes that the openweight models will continue to get more capable and 'AI is here to stay', this model provides the first solid footing for a decision to just buy a rack.
Also open sourced a bunch of infra to go with it.
Anyone who claims open source and open weights models are "decel" needs to get their head checked
https://github.com/MoonshotAI/MoonEP
https://github.com/kvcache-ai/AgentEnv
https://github.com/MoonshotAI/FlashKDA
License: https://huggingface.co/moonshotai/Kimi-K3/blob/main/LICENSE
> If the Licensee or any of its affiliates operates a Model as a Service business, and the aggregate revenue of the Licensee and its affiliates exceeds 20 million US dollars (or the equivalent in other currencies) in total over any consecutive 12 months, the Licensee must enter into a separate agreement with Moonshot AI before using the Software or its derivative works for any commercial purpose.
+ the existing 100 million monthly active users, or more than 20 million US dollars for commercial products have to name Kimi clause
> we build a self-evolving, hierarchically organized knowledge graph that agents continuously expand through web-scale exploration across knowledge-intensive and coding domains
That's interesting!
It's funny that we've finally returned to tanh activation functions, time is a circle.
knowledge graph guided task synthesis. very cool! i have long wondered about the "how do you get good coverage of all the tasks" problem.
maybe some interesting theoretical work there around the rate of production of new knowledge itself and various mechanisms (human approaches, mechanistic approaches, etc).
What would be the current best method to fine-tune it for my own specific agentic tasks? LoRA + DPO? GRPO? Something else?
Can someone explain what are teachers in Multi-Teacher On-Policy Distillation? I can imagine math, coding and other verifiable domains, but they also have biology? Is it where distillation from bigger models come in?
Why is everyone's pricing the same? licensing agreements with moonshot as an inference provider?
That's pretty impressive, I don't want to be the one undermine but I personally think the industry is shifting to rather designs that enable Models to work instead of chugging it all to data and compute ?
What would it take to get an American open model to compete with this?
What are the coding/agent harnesses (like Claude Code or ChatGPT Codex) that can be used with this?
Does anyone know if a torrent is available? I think it would take quite a while to download 1.5tb from their servers.
That's the great achievement, but ahh, wanted to try it, but my RAM had other plans
Any guess on the pre-training tokens/flops they consumed?
This is amazing to witness. Moonshot open sourcing Kimi K3, a frontier AI and other components really means we are getting abundant AI for all of humanity.
Kudos to Moonshot for truly being what OpenAI should have been.
Fable-level and frontier AI should be open source and available to everyone for free.
I would want to see three things before drawing strong conclusions:
End-to-end tokens/sec and cost on realistic coding agent trajectories, including tool outputs and retries, not isolated decode benchmarks.
Cache hit rates and prefill cost for branching, multi-turn sessions.
Router-load distributions after post-training, where expert collapse or specialization problems often show up.
Is this p-hacking?