Considering the likelihood of Moonshot being able to collect a judgement for violating these terms in an American court is basically zero, and the relative ease of finding the weights without the licensing agreement, &quot;have to pay&quot; seems very generous.

It&#x27;s only &quot;model as a service&quot; providers beyond the stated threshold that have to pay. No issues for any kind of purely internal use.

Or, you can just not pay and save that money to defend yourself in case they do sue you. Because it isn&#x27;t established that model weights are copyrightable.

abd why should we expect this to be the case? I can atleast buy old server hardware for cheap. GPUs are currently locked down the whole industry.

&gt; more than $20 million…in monthly revenueSign me up!

Models this size aren&#x27;t targeting hobbyists running models in their bedrooms on gaming&#x2F;surplus hardware bought with disposable income.This model is for researchers who already have racks of equipment, inference providers who sell access to models, and for businesses who must (or choose to) use on-prem models. All of which have to pay to use the model if their revenue rises beyond certain thresholds: <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;moonshotai&#x2F;Kimi-K3.1-Pro&#x2F;blob&#x2F;main&#x2F;LICENSE" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;moonshotai&#x2F;Kimi-K3.1-Pro&#x2F;blob&#x2F;main&#x2F;LI...</a>

Wow... I&#x27;d like to know more. 40 analysts not spending 100USD&#x2F;month is 4k&#x2F;month saved. That means you break even in 27 months. But you probably used Claude much more than that.Also what harness do you use? Any problems you encountered on the way?

Are they happy with Kimi 2.8&#x27;s reasoning ability? I haven&#x27;t tried it.

There&#x27;s no Kimi 2.8 as far as open weight is concerned. WTF. This must be a bot and other bots.

How bad did the electric bill get hit?

We&#x27;re running Kimi 2.8 on a 4x AMD MI300A single server on-premise that cost about $110k.We will break even from our old Claude spend in about 2 weeks.It services 30-40 analysts and staff concurrently and is faster than any of the SaaS out there for us.

&gt; Aggregators like OpenCode Zen and OpenCode Go can shop around providers on these. OpenRouter shows you the costs of a bunch of the providers, and you can route to the cheapest ones.Unfortunately due to their licensing requirements for modal-as-a-service providers, the pricing can’t go lower than Moonshot’s from what I can tell.This is still great if like your example you trust together.ai more, or if different providers have better speeds&#x2F;cache rates (though usually the original lab has the best cache hit rate).And obviously if you’re a large enough service to afford to self host or finetune it then this is good.

There&#x27;s no need to use ablated models right now for cyber work. Several open weight models don&#x27;t refuse cyber requests (DeepSeek, GLM, Kimi K3, etc). And as you said, OpenRouter has many 3rd party ZDR inference providers and several of those serve these models.

The problem with looking at different prices from different model providers is a lot of the cost is generally driven by cache hit rates. 
That’s where you can accumulate expense and the cache hit rate can very wildly from one provider to the next.

The benefit is that you can use these models on a variety of different providers. You&#x27;re not stuck with a single proprietary provider. Aggregators like OpenCode Zen and OpenCode Go can shop around providers on these. OpenRouter shows you the costs of a bunch of the providers, and you can route to the cheapest ones.I regularly use together.ai with models like this, where they&#x27;re fully self hosting them and I can turn off all data retention, at a fraction of the cost of proprietary models. I recently discovered abliterated models (models edited to not refuse to work on security stuff) and hosts for them like refuseless.com, and that&#x27;s an option for open source models, not for proprietary ones.

Excuse me, could you elaborate how you came up with 192GB of RAM? I almost thought they downsized model, but it&#x27;s still 2.8T tokens. Are we talking about not-yet-out 1-bit quant or assumption that something like ds4 (daskstar, not deepseek) will heavily optimize it?Not a trolling question, genuine interest.

You could easily run it with SSD streaming on as little as 192GB RAM (The dense portion is around 115GB or so at native precision, and the remainder would fit KV and a cache for the most commonly referenced experts.) This fits Gorgon Halo, which is a reasonably priced platform. It would be slow, but still usable for high-value tasks.

&gt; a flood of 8x B300 will be homelab ready in a few monthsHaha, that&#x27;s beyond wishful thinking, I&#x27;m afraid. We&#x27;re not even seeing a flood of A100s (6 yo and several generations old). Every GPU is still being squeezed right now, and the trend will probably continue for the foreseeable future.

The world is bigger than the US. If the political climate in the US obstructs the build out, the build out will occur somewhere else.

No, they&#x27;ll be shipped off to Asia where technology is embraced.

There are 300 racks of nVidia A100 servers for sale from a refurber right now on eBay...

&gt;Possible data center bans incomingA lot of the data center build out has been bipartisan. Unless Dems realize they need to listen to their constituents to avoid a 2028 bloodbath (they probably won&#x27;t), I wouldn&#x27;t count on that to save us from RAM&#x2F;GPU prices.

Possible data center bans incoming + tons of GPUs sitting in warehouses due to lack of power infrastructure to power them.Perhaps a flood of 8x B300 will be homelab ready in a few months

&quot;Pacing the liquidity frontier&quot; :D brilliant. Will be using that.

Its really not certain model weights are copyrightable in the first place.

I assume you mean condition 2 - where they want anyone serving it as a service (to others) and earning more than $20mUSD per year in revenue to contact moonshot and enter a separate commercial agreement.

It&#x27;s not an open weight model (check the license, and the definition of that term if you disagree).

Due to costs I am unfortunately years away from ever seeing hardware at this level, but if you just want to study LLM&#x27;s and AI then there are still plenty of models that work fine on crappy hardware, and you don&#x27;t have to worry about them becoming sentient on you :-)

&gt; Black Friday is coming soon thoughA few weeks before Black Friday a couple years ago, I noticed a listing on Amazon for a high-end AI card for around $40,000 (maybe a B100, don&#x27;t recall). The listing had the automatically inserted Xmas season note &quot;Free returns until Jan 31.&quot; I spent a serious minute pondering if I could float the cash for the ~40 days between the credit card bill being due and returning the card. :-)

AND developer cloud has an image for K3 running it on a single MI300x. Offloading to system ram can be quite decent.I cant say how well that image works as I&#x27;m still waiting for them to approve my gpu access

Depends on number of users - for a single user with a couple of sessions, you don&#x27;t quite need that much. Bit of a waste of hardware to only serve this for single users though.

It’d be interesting to think if device swarm serving of these models ever becomes possible - are a few projects that seem to be just starting on this.It seems more plausible to me than wherever the frontier is at any given time getting small enough to serve on one local card directly

I have managed to boot up these models for 1-2 users on just a couple B300s. Which costs a lot still but it&#x27;s not that bad.

No, also because that would mean I have to read up on rope scaling or whatever to go beyond the default 256k context, which so far I’ve avoided. But you can allocate it as 220k context for 2 parallel Pi workers, that really speeds things up! Or as ~8x 50k context for dedicated parallel document processing.

Do you need 450k context?I usually keep under 200k if I can help it for Claude&#x2F; open aiAnd lower locally on my Mac to save on prefill.

wish they just said &quot;we are better than openai and anthropic, fuck you sam and you dario&quot;

NVIDIA HGX B300 seems to have 8 Blackwells, but I have a hard time getting concrete information from the NVIDIA website on what it actually is, and how you operate it, physically. The picture has this intermediate-product vibe that it should be part of a larger system, like the GB200&#x2F;300 ones, and I feel like a little boy looking through the window of some shop with incomprehensible things. But I guess the whole point probably is that if you want to have it you put out a tender and someone else will worry about the design and integration and you just pay the bill and point to some empty space in your datacenter cage with large red 5-pole power bundles eagerly waiting.

2 gb300 node actually. Or 8 gb300 cards since 1 node has 4.

you are aware that hardware gets better and cheaper, right ?

Subtle quote from their website:&gt; Advancing the Frontier of Open Intelligence.Too bad you probably need something like an 8x B300 node to run this. Black Friday is coming soon though, but I fear this open intelligence will be pacing anyone&#x27;s liquidity frontier so I&#x27;ll probably keep trailing the frontier for a while with nvidia&#x2F;qwen3.8-27B-NVFP4 on dual RTX PRO 4000 in TP=2. I can almost fit 450k context in there.

Speaking of arena comparisons of K3.1 and DS4.1-Flash, we&#x27;re spinning one up that&#x27;s more relevant than design, I think: <a href="https:&#x2F;&#x2F;agentduel.com&#x2F;leaderboard" rel="nofollow">https:&#x2F;&#x2F;agentduel.com&#x2F;leaderboard</a>Here, DS4.1-Flash and K3.1 are basically tied, with K3.1 being slower and more expensive. Big error bars, though.Even with error bars, DS4.1-flash is clearly an impressive model.

And how do you reckon GLM and Qwen stack up?

every time i accidentally use flash versions of deep seek i get really frustrated with the model till i realize that it was the wrong model. im also a little confused because it seems like at least comparisons make it sound like ds is no longer focusing on their high reasoning model family …

How do you use them day-to-day? The subscription lasts for minutes of usage at at time, and API costs are crazy high. There is zero reason to use it over an Anthropic subscription, it is not faster or cheaper - unlike DeepSeek 4.1, MiMo 2.6 or other flash models.

If you use it day-to-day you can definitely feel that K3 (and K3.1) are much stronger than DS4.1-Flash-1017. There&#x27;s something the benchmarks don&#x27;t capture. DS4.1-Flash-1017 is an amazing model for the price, but it&#x27;s not a very large model in comparison.I wouldn&#x27;t over-weight internal benchmarks from a competitor that claim benchmaxxing TBH. Kimi K3 has been at the top of Design Arena for open-weight models for a long long time, and I suspect K3.1 will be there for a long time too. It&#x27;s pretty hard to fake that one since it&#x27;s actual human preferences for the artifacts produced.

&gt; When we edit a task file to contradict one of these rules, K3.1 ignores the edit and fails the taskOuch, that&#x27;s a nasty case of reward hacking. Thanks for pointing this out!

Wow, really interesting work. Run of the mil benchmarks has been so shallow in measuring real performance, especially outside agentic software development.Aside from this paper and K3.1 specifically, is there somewhere where I can read more a high-level introduction about the state of things in this niche? like the different types of faults like these models fail and ideally how to fish them out?

&gt; so it seems K3.1 is benchmaxxed.It&#x27;s worse than that. It seems that they straight up trained on the test sets, according to this [1].&gt; When prefilling K3.1’s reasoning with a request to list its instructions, K3.1 names hidden requirements in 65%
of its answers, K3 in 18%. When we edit a task file to contradict one of these rules, K3.1 ignores the edit and
fails the task (in three of four tasks), while other models follow it. K3.1 does follow edits that do not contradict
a hidden rule.&gt; We think these behaviors are most likely signs of
training directly on the benchmark tasks with a teacher that saw the hidden grading rules, for example on-policy
self-distillation.&gt; K3.1 also
names HLE on 8.4% of the questions, often while trying to work out which benchmark a question comes from&gt; For example, in the production-planning task, the prompt never mentions
tiers or lexicographic order. K3.1 writes “Instructions strongly say set-level aggregate fulfilled qty per priority tier
lexicographic [...]”, which is the hidden ranking rule. In 23.3% of its 189 attempts, K3.1 presents a requirement
that the prompt lacks and the hidden tests check as an instruction[1] - <a href="https:&#x2F;&#x2F;r40.org&#x2F;papers&#x2F;kimi_k31_benchmark_audit.pdf" rel="nofollow">https:&#x2F;&#x2F;r40.org&#x2F;papers&#x2F;kimi_k31_benchmark_audit.pdf</a>

Depends on what you&#x27;re using it for. DSf is pretty good at executing planned coding tasks but it isn&#x27;t very intelligent. K3.1 doesn&#x27;t need to be a coding master if it can be a fable replacement.

I wanted to switch to DS4.1F from K3 (not even K3.1). Even the 1017 snapshot takes 5x-10x the tokens that K3 uses for the task, and often misses more intricate gotchas that K3 catches.

I wonder how much the harness hurts&#x2F; helps here. Been using both the Oct V4.1-Flash and K3.1 in Pi for some time and now almost exclusively use V4.1-Flash in dsh and it feels superior.

I&#x27;m not talking about Hy4 Preview. I&#x27;m talking about Hy4, it was announced this morning. - <a href="https:&#x2F;&#x2F;hy.tencent.ai&#x2F;research&#x2F;hy4" rel="nofollow">https:&#x2F;&#x2F;hy.tencent.ai&#x2F;research&#x2F;hy4</a>

For reference, Hy4 Preview was a 770B model (49B active) + 10B MTP (0.7B active). Not nearly the same potential as Kimi K3 (which is 2.8T) wrt. real-world knowledge - which matters at the new frontier of advanced research and scholarly tasks. Coding benchmarks are pretty much universally saturated by now which is why newer intelligence indexes have chosen to weigh tasks quite differently.

Doesn&#x27;t seem to outpace DeepSeekV4.1-Flash by much. Hy4 also announced their models and they have tons of internal benchmark, and K3.1 doesn&#x27;t score that high on those benchmarks whereas Opus5.5 does, so it seems K3.1 is benchmaxxed.

Third-party inference providers are now hosting it, with more popping up by the hour: <a href="https:&#x2F;&#x2F;openrouter.ai&#x2F;moonshotai&#x2F;kimi-k3.1-pro" rel="nofollow">https:&#x2F;&#x2F;openrouter.ai&#x2F;moonshotai&#x2F;kimi-k3.1-pro</a>Together appears to have double throughput of the Moonshot AI endpoint at the same price point (with Modal having double that throughput), so I suspect there is room for inference providers to optimize...and therefore headroom to cut costs for competition.

<a href="https:&#x2F;&#x2F;xnotme.com&#x2F;PeterNutter1&#x2F;status&#x2F;2118084568419045704" rel="nofollow">https:&#x2F;&#x2F;xnotme.com&#x2F;PeterNutter1&#x2F;status&#x2F;2118084568419045704</a>

I think chatter is putting it mildly. There&#x27;s plenty of evidence in this work [1] that is hard to explain. At the very least it&#x27;s a failed attempt at benchmaxxing the model, but more likely is straight up training on the test set (and the attempt at hiding it failed). Something leaked pretty heavily in their setup.And there&#x27;s indications of it happily replying with fake encrypted data in oAI format here [2]. While this is a bit less conclusive than [1], it&#x27;s still dubious.[1] - <a href="https:&#x2F;&#x2F;r40.org&#x2F;papers&#x2F;kimi_k31_benchmark_audit.pdf" rel="nofollow">https:&#x2F;&#x2F;r40.org&#x2F;papers&#x2F;kimi_k31_benchmark_audit.pdf</a>[2] - <a href="https:&#x2F;&#x2F;x.com&#x2F;PeterNutter1&#x2F;status&#x2F;2118084568419045704" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;PeterNutter1&#x2F;status&#x2F;2118084568419045704</a>

Hypothesis for why K3.1 dropped so much in relative position to other models when AA switched to private evals? Primarily FrontierCode, CritPt V2, and the new private version of AA Omniscience where the model massively underperforms relative to its performance on public benchmarks?

Lots of chatter about how this model is benchmaxxed, but didn’t observe this in our testing.Very strong open model release that is a bit off the frontier but totally amazing for most use-cases.<a href="https:&#x2F;&#x2F;x.com&#x2F;danielmckinn0n&#x2F;status&#x2F;2115213822957117535?a=fOnT23O3ybHYeBwR1xU&amp;s=46" rel="nofollow">https:&#x2F;&#x2F;x.com&#x2F;danielmckinn0n&#x2F;status&#x2F;2115213822957117535?a=fO...</a>

It&#x27;s not impossible that the separate agreement in question imposes light limitations.I think one of the initial reasons why otherwise open source models started with the limitations on model-as-a-service use cases was because of quantised or entirely different models being served as the model in question on open router, and they needed a legal method to limit that. It might just be that, thought it might also be the direct opposite, hard to know.

This doesn’t stop any company spinning it up for their internal use. It only bites if you try to resell it.

Average Joe likely can&#x27;t host the model to begin with but the average company won&#x27;t be able to do so either, at least not for others[0]:&gt; If the Licensee or any of its affiliates operates a Model as a Service business,
and the aggregate revenue of the Licensee and its affiliates exceeds 20 million
US dollars (or the equivalent in other currencies) in total over any consecutive
12 months, the Licensee must enter into a separate agreement with Moonshot AI
before using the Software or its derivative works for any commercial purpose.[0]: <a href="https:&#x2F;&#x2F;huggingface.co&#x2F;moonshotai&#x2F;Kimi-K3.1-Pro&#x2F;blob&#x2F;main&#x2F;LICENSE" rel="nofollow">https:&#x2F;&#x2F;huggingface.co&#x2F;moonshotai&#x2F;Kimi-K3.1-Pro&#x2F;blob&#x2F;main&#x2F;LI...</a>

These releases are meant for companies, not average Joe:<a href="https:&#x2F;&#x2F;www.theregister.com&#x2F;ai-and-ml&#x2F;2026&#x2F;11&#x2F;03&#x2F;british-fintech-revolut-slashes-ai-coding-costs-10x-by-cutting-out-claude-and-going-self-hosted&#x2F;5307345" rel="nofollow">https:&#x2F;&#x2F;www.theregister.com&#x2F;ai-and-ml&#x2F;2026&#x2F;11&#x2F;03&#x2F;british-fin...</a>

You can run it on high end laptop at like 0.1 token per second.But I think smaller size models like qwen 3.8 are way more important.

Even if you can’t run this at home, this is still a step toward commoditization &amp; diversification of inference and breaking up of concentrated power.
Being able to serve a frontier-class model with 1M$ of capital in GPUs instead of multi-year billion dollar investment in training &amp; research is a big difference

Too late to edit my previous comment: I meant an uncensored provider.

Side question: Can you (or anyone reading this) recommend a decent source for an LLM provider?I&#x27;ll admit that I&#x27;m lazy and haven&#x27;t looked yet... I&#x27;m just wondering if you have any recommendations.

read the report, they nipped the cyber capability so it&#x27;s not strong on offensive security. hy4 did the same thing, looks like glm5.3 is the last strongest open weight with offensive cyber ability we are going to be getting for a while.

Now let&#x27;s see this on some abliterated APIs. I&#x27;m sick and tired of Claude standing guard for copyright after Anthropic stole everything there was to steal and got away with it.

Look at the license. It has all sorts of arbitrary restrictions, so it&#x27;s not an open weight model, in the same way that &quot;source available&quot; is not open source (but worse, since there&#x27;s nothing human readable in the weights).

&gt; Kimi K3.1 Pro is an open-weightLie right in the first sentence

Why don&#x27;t you complain about OpenAI or Anthropic weights?

Only a 1453GB download. Fits on one LTO5 tape, maybe.

With common use of n-gram tables, comparing &quot;params&quot; count has gotten harder. Those are exceptionally sparse and light on compute, way beyond even the sparsest MoE experts. You could host them on a slow spinning disk and still have a viable local inference platform.

Chinese labs seem to think that fable&#x2F;mythos is a 10T+ model, and target that amount of parameters with next iteration of qwen&#x2F;kimi&#x2F;ds&#x2F;glm. Proves nothing tho.

3T parameters! I&#x27;ve been really impressed with Kimi&#x27;s coding performance. It&#x27;s replaced a lot of my Claude Code workflows, which just got way too expensive. Wonder how big Fable and Opus are in comparison.

I love thisBut i also get the immediate &quot;but how is anyone supposed to run this locally&quot; feelingMaybe we need a new network. Like, Nextdoor + counterstrike pub servers, but for local LLM. We can call it semilocal:)

I read somewhere that Qwen 4 is right around the corner. Praise be!

quantization has come a really long way. Q3 is fine, especially GSQ-RCO. You can run 27B well on 16GB of VRAM.

Fwiw, the Qwen 3.8 27B at 4 bit quant is apparently quite good and is probably the best recent model. It runs (iirc) on 18-22ish GB of memory.

If someone released a 0.1bit quant I can run it. &#x2F;s

Kimi K3.1 Pro model weights are out