Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in carnage.
"... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..."
This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no problem being #1.
show comments
mark_l_watson
Meta is rocking AI. As of last week I have been using their excellent muse coding harness with their model Muse Spark 1.2.
Starting this morning I am running their new local 30B model muse-glimmer on my old MacMini 32G using Ollama (remember to increase the context size!) and pi coding harness. I am getting good results with muse-glimmer running locally, with the caveat that everything runs slowly (e.g., give it a task and then go walk outside or do Qi Gong exercises for a while).
The quantized releases often change in the weeks following release as new improvements are discovered, so either use a tool that checks HuggingFace for new versions or manually check back in a few days or weeks to check for improved versions.
Initial reports are good. It hasn't been out long enough for anyone to really test thoroughly, but the people I know who have stable non-public test cases are reporting impressive results compared to even Qwen3.6 27B. That's a good sign that this might not be benchmaxxed (trained to excel at public benchmarks with less impressive performance on general tasks) which has been becoming common with recent releases.
www.reddit.com/r/localllama is a good place to keep up with the details from people who are actually using it. It feels strange to recommend a subreddit over Hacker News, but on this topic the /r/localllama threads are much more on topic right now if you're looking for information about the model.
There are some initial reports that even the 2-bit quantization is looking somewhat usable. That might make it small enough to squeeze into 16GB GPUs. I'd take those reports with a grain of salt because early tests are often optimistic and I've yet to see good results from anything 3-bit or less, but it should be fun to experiment with.
show comments
polymorph1sm
Some interesting findings from the chat template designs:
1. The template name is Onyx ATEM as found in the tool call exception message
2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like :<atem:function_calls> / <atem:invoke> / <atem:parameter>
With the business model for API based LLMs looking iffy at best it seems like we’re heading back to the “server under your desk” era of IT again.
show comments
andy99
The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user.
Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to the point that using it might actually be faster. I find Qwen and some others rehash the same things over and over when thinking without getting anywhere, in mg limited checks here Muse is much better.
show comments
jawiggins
> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation.
The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, notifications, and newsfeeds and is constantly preparing things for you.
show comments
_ache_
It is interesting but it does look like a careful distillation of (Spark and) biggers open-weight models.
The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so).
It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.
show comments
Gecko4072
What I think would be perfect is a model that could run on a single DGX spark and be competitive with DSV4 Flash 731. Flash is already a game changer. Hopefully meta plans on this, like the old 70b. V4 flash is smart enough for any use but slightly too big. 27b-30b isn’t intelligent enough.
show comments
hougaard
Tried it (the full version, using 120 GB RAM), wasn't impressed, gave it some defective code, and asked it to fix all errors. It kept looping around and around and digging itself deeper and deeper into a rabbit hole; eventually, it got into a "reasoning" discussion about whether a custom compiler was used that supported the wrong syntax...
sajithdilshan
Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.
show comments
avaer
I lament the comments saying this in any way redeems Meta (the company).
The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse.
You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
But I guess most people just don't care.
I'm glad it's open. It does not make me think any better of Meta.
Optimizing models specifically for always-on local agent workflows is the right primitive for decentralized systems. Brilliant release.
noodleweb
Happy to see meta back in the game, it's like after llama nothing came out that was comparable to mainstream open models.
mirekrusin
Great to see Meta back, looks like really strong, local model, can't wait for llama.cpp support.
show comments
kyledrake
The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer.
For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy, for fun, etc).
show comments
maxignol
Optimizing speed is really the way to go.
Yet 24GB is not what everyone can afford.
Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.
show comments
OsamaJaber
The comparison set is Gemma4-31B and Qwen3.6-27B, not the current Qwen
Fair on size, but the headline numbers are against a model a generation back
show comments
tosh
good to see new open weights releases from meta
show comments
hypfer
Having played around with this model a bit, I am fairly confident that it is not competing in the coding space.
It can do that, but its actual selling point appears to be a different take on guardrails and safety alignment.
Either that or the only new training data left was industrial quantities of dark romance literature and Wattpad.
Clever business move. 131k context is more than enough for that use case, and due to that small K/V footprint, you can probably have a bunch of characters on the same GPU.
Or it's just a happy little accident. We will never know.
___
I was informed that normal people use LLMs for mundane tasks like asking for a pancake recipie.
That it apparently can also do decently.
Unfortunately, it is also very confident, regardless of whether it is actually correct.
So maybe it should actually stay the smut engine and nothing else.
bentt
Meta seems like the one American bigtech that
would distill the the other American frontier models. My enemy’s enemy is my friend?
show comments
harisamin
Let’s give thanks to all those meta engineers who have been ripped for my heir teams (while sitting right by them) working on manually tagging data. I guess the morale dip paid off in some way? I wish you all well and hope you find some happiness … IYKYK
nezhar
I tried to run it with lemonade by installing it via hf but did not succeed, it gets some weird 500 errors. I also see that ollama has currently only an mlx version available.
Anybody here succeed to run this on AMD?
richardfey
Looking forward to giving this a try with llama.cpp. I’m watching the open-weights competition with high expectations.
realaaa
and immediately followed up with Manifesto from the man himself - what / how are they going to make of it longer term?
I guess for FOSS and self hosted it is good - but I am still wondering how are they going to Meta-stasize it ;)
solarkraft
Wow, Meta is back (at least for now)!
I like this class of model. Multi-token prediction makes it viable to run dense models at not-too-far-off speeds as MoE models with much better intelligence.
The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model.
It even comes pre-quantized and with a MTP/drafter model. Looking good!
Let’s hope they aren’t dishonest with the benchmarks this time …
show comments
androiddrew
I'd really like to see a 45B-ish dense model ready for a dual GPU setup. Something with a little more intelligence while still within the range of some higher end local setups.
show comments
vibe42
Meta released their own 4-bit quant of this model for devices with 24GB VRAM.
That's a modern gaming laptop; cheapest I see in the US with 24GB is $3.5k.
Should be quite a bit faster than the new M5 MacBook Pro, and you can run Linux on it!
jakswa
Another candidate for the 7900XT (20GB VRAM) I got sitting around. I pulled latest llama.cpp (targeting vulkan during build) after seeing a muse PR merged a few hours ago, and unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL runs on my 7900XT barely (and with no MTP). Sits at 19GB VRAM w/ 4 parallel 113k context slots, all layers on GPU, and at 700 tok/s prompt, and ~36 tok/s generation.
Waiting on Q3 to download to check speed + do my usual anecdotes. I generate beefy code snippets and poems, and also ingest my HOA declaration and answer nuanced questions.
edit: i should've prefaced this somewhere with: This card ballparks at 800GB/s IO, which I can't seem to find easily on the market anymore. Kinda the ideal card for this model, if I just had a _little_ more VRAM (XTX is 24GB).
edit2: not mtp, this is dflash model (param in child comment). I'm up to ~60 tok/s generation and sitting at 19GB VRAM (i added --no-mmproj (makes it text-only i believe) because I'm used to speculative decoding wanting more VRAM and I'm already close to the limit :sweat_smile:)
show comments
gunalx
Meta did not abandon opensource. I would love to see a smaller distill, or a moe of this size but the benchmarks seems competetive as long as it isnt benchmaxed witch i would not be suprosed if it is.
show comments
nirbendavid
Many companies are stressed about token cost, as we are moving to a consumption based charge.
In the meantime - new open source models, such as DeepSeek V4 Flash and GLM5.2 reduced the price to about 13x chepaer. Also OpenAI had reduced its price for considerably.
Now Meta is back in this game. The upcoming months are going to be interesting (GoT)...
Havoc
The favourable comparisons to Gemma 4 and qwen3.6 look promising!
show comments
spaqin
That's a bit amusing - not that I have the hardware to run it, but officially it's not available in Hong Kong. Not that getting it would be much of a problem with a help of a VPN either, but I'll assume mainland China is also restricted. Certainly not a competition for Chinese open weight models... in China.
ionwake
Sorry I dont know if this is the right place but... 2000AD The Glimmer Rats , was the best drawn comic strip story by far in that publication.
TormentNexusAI
The combo that makes agents reliable: progressive tool routing, persistent memory, and multi-model failover.
jckahn
Where is the pelican??
show comments
folienumero
In my experience it's faster (10tk/s vs 35tk/s) and better than qwen3.6 series.
koof
kind of a nonspecific complaint, but i haven’t yet had much luck with anything under ~120b, feels like models released on that order is coming to a trickle. the last few qwen models didn’t seem to go that high, and i got worse results than qwen3.5-122b
bwfan123
Next step: Burn the weights of these local models into an asic that ships cheap on a laptop (AMD/taalas looking at you), and I will be a happy camper. Make it pluggable so I can select a model I want. I use qwen3.5-coder currently on my laptop, and while it works well enough for me, it is somewhat slow processing tokens.
I would hazard a guess that fast small models with a smart agent harness can do quite well compared to large models which cant be run locally.
swrrt
Just asking, what is the recommended models for M3 MacBook with 18G memory? Seems modern local models are not available.
show comments
catoc
Personally I would never trust a coding agent or agent harness from Meta.
I agree with their open-source model approach, but actually trusting Meta… to protect my privacy and my data… when it’s running on my personal hardware…
Not . In . A . Million . Years - that ship has sailed
bronxbomber92
I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
show comments
golly_ned
Having just bought a 5070 Ti (16GB) instead of a 5090 (24GB), I am sad.
show comments
zmmmmm
Meta knows how to win back developer's hearts .... let's see if they have the goods
show comments
heysagnik
even 30B model is too large to large on local device (low end). meta should provide free hosted model api to use it.
show comments
hndhyc0bdt
Refreshingly practical
wyzer
How are you handling the tradeoff between quantization for device fit and accuracy loss on tool calling? That's where local agents typically break down in production.
aussieguy1234
The SWE bench verified score is similar to Opus from not so long ago.
Sure, you can get better performance from cloud models.
But most software, not just AI, will be faster and more reliable in the cloud. The question is do we need that additional power and cost.
If the answer is no, then just like other software, people will run AI locally.
wxw
Meta's clearly changing strategies back towards their original "frontier open source", but this time around they have a lot more competition from leading Chinese labs.
I'm all for it though, and I think Glimmer is a fantastic bet on locally-hostable models. I for one would love to self-host as much as I can.
sgt
Can I run this on my RTX 5090?
show comments
hnx0rqy49u
Clear, useful, done
jhgik798
How many data using in Polish Language?
eugene3306
will it run on 2x 5060Ti with 16GB each?
show comments
brcmthrowaway
Any MLX results?
show comments
ThouYS
Qwen 3.6 27B is still such a beast!
mytailorisrich
Random question: Would you be able to run this model on a Macbook Air M5 (latest)?
show comments
HardCodedBias
LOL the mogging of GDM is hilarious.
I don't know why MSL released this, but it is very nice that they did.
nutjob2
The more open weight models get released the greater the market for personal and small business oriented hardware to run these models. This will drive lower cost hardware, which has stagnated in recent years due to most software not needing the performance and capacity.
show comments
soupspaces
what's the catch?
spwa4
From twitter Alexandr Wang
> 3/ muse glimmer was developed with its own architecture and recipe, optimized for its size and agentic performance requirements.
This means we're in the endgame does it not? If the architecture was NOT optimized for intelligence ...
brumbelow
and now the recent Meta model 'security issue' begins to make sense
treksis
thank you zuck.
m00dy
what I can tell is that Meta is just starting and it is so underrated.
moron4hire
"Meta Muse" immediately made me think of Metamucil.
Product teams really need to hire at least one or two people with a 12-year-old's sense is humor. They need to winnow all the potential stupid jokes out of their product namings.
hn97o8vvbt
Quietly the best thing in the thread
reilly3000
PSA: Fast RAM isn't going to be getting cheaper anytime soon. Acquiring inference hardware is a really good way to own an appreciating hard asset. Learning how to use it and cool it is a hacker's journey worth taking. My 4090 I bought in late 2022 for $1600 is selling for a cool $3,489.95 right now, and going strong under nominal use. My DRR5 has tripled in value, my nvmes almost doubled. I grabbed a 128GB M5 Max MacBook Pro when they were still available and told all my friends to buy at least one. With that and a base M4 Studio 36GB, HuggingFace rates that hardware as:
> Amazing!
You have a total of 128.94 TFLOPS of computing power. 71.3% percentile on scale of "GPU Poor" to "GPU Rich"
The way I see it, these are amazing machines that the richest folks are hovering up. I think they should be in the hands of regular people as much as possible. They depend on an incredibly global, increasingly fragile supply chain. If the become impossible to produce, their value would increase tremendously. I think they will become really valuable to you to use the tokens directly, but if that isn't the case, they can be rented out or resold. Please don't just buy any hold. Let's try to get as many people that can use them for decent things that help humans. For example:
As an industry, I wish we would stop calling these things "open weight" because it is too easy to confuse with actual "open source", which they are not.
Photoshop source code+ OSI license = open source
Photoshop binary you can run on your own computer = open weight
Photoshop SaaS web app = closed, proprietary (Opus, GPT, etc.)
"Open weight" models are still just binary blobs that are completely inscrutable. It's like bringing home a dog from the rescue and just hoping that it doesn't have a tendency to bite kids in the face. You just can't know. The only thing that you can do is try to add more training (fine tuning) telling it not to bite kids.
I don't think the FOSS community has ever accepted this, but somehow we're feeling like it is okay now.
Will be interesting to see how Qwen3.8 27B compares against this once it releases this week. Seems like dense 30B is back in fashion?
EDIT: An open weight version of Muse Spark 1.2 is going to be released as well:
https://x.com/alexandr_wang/status/2086756152034066792
https://xcancel.com/alexandr_wang/status/2086756152034066792
Remember when we needed 200 servers for an enterprise website because Apache used one process or thread per connection - and Nginx collapsed that into a single box overnight? That moment for LLMs is near. It’s going to move us from the big iron era of AI to small portable brains. Nature has already proved it’s possible with 20 watts and very little heat generation. And I think the data center buildout will end in carnage.
https://xcancel.com/finkd/status/2086755195535413696
"... Soon we'll also release the weights for Muse Spark 1.2, our latest foundation model..."
This is bigger news - good for self hosting enthusiasts and a strategically sound move for Meta. Any push towards 'anti Chinese' models will directly benefit Meta as the competition on the frontier open-weights American models is almost non-existent. Meta will have no problem being #1.
Meta is rocking AI. As of last week I have been using their excellent muse coding harness with their model Muse Spark 1.2.
Starting this morning I am running their new local 30B model muse-glimmer on my old MacMini 32G using Ollama (remember to increase the context size!) and pi coding harness. I am getting good results with muse-glimmer running locally, with the caveat that everything runs slowly (e.g., give it a task and then go walk outside or do Qi Gong exercises for a while).
Unsloth has quantized versions uploaded: https://huggingface.co/unsloth/Muse-Glimmer-30B-GGUF
The quantized releases often change in the weeks following release as new improvements are discovered, so either use a tool that checks HuggingFace for new versions or manually check back in a few days or weeks to check for improved versions.
Initial reports are good. It hasn't been out long enough for anyone to really test thoroughly, but the people I know who have stable non-public test cases are reporting impressive results compared to even Qwen3.6 27B. That's a good sign that this might not be benchmaxxed (trained to excel at public benchmarks with less impressive performance on general tasks) which has been becoming common with recent releases.
www.reddit.com/r/localllama is a good place to keep up with the details from people who are actually using it. It feels strange to recommend a subreddit over Hacker News, but on this topic the /r/localllama threads are much more on topic right now if you're looking for information about the model.
There are some initial reports that even the 2-bit quantization is looking somewhat usable. That might make it small enough to squeeze into 16GB GPUs. I'd take those reports with a grain of salt because early tests are often optimistic and I've yet to see good results from anything 3-bit or less, but it should be fun to experiment with.
Some interesting findings from the chat template designs:
1. The template name is Onyx ATEM as found in the tool call exception message
2. It appears to be following a harmony-style chat template. But the tool use seems to be a xml like :<atem:function_calls> / <atem:invoke> / <atem:parameter>
3. atem: a internal joke of meta in reverse?
https://huggingface.co/meta-models/Muse-Glimmer-30B/blob/mai...
With the business model for API based LLMs looking iffy at best it seems like we’re heading back to the “server under your desk” era of IT again.
The gguf is up and works, I don’t know if it’s them or unsloth that’s facilitated this but it’s nice because e.g. Inkling still doesn’t appear to have support in llama.cpp which makes it irrelevant to a class of user.
Unfortunately I don’t have enough experience with Qwen 27B to immediately compare, but I do it’s Qwen 3.6 35B A3. It’s much slower obviously but it seems to be way more efficient with its thinking to the point that using it might actually be faster. I find Qwen and some others rehash the same things over and over when thinking without getting anywhere, in mg limited checks here Muse is much better.
> Muse Glimmer is a 30-billion-parameter model optimized for always-on local agent workflows. It’s small enough to run on a Mac or PC with a single consumer GPU, enabling use cases that range from local agents and function calling, to local coding, and LLM-as-a-judge evaluation.
The next iteration in LLM products is a 24/7 thinking loop where the claude-code like thing gets input continuously from your wearable, notifications, and newsfeeds and is constantly preparing things for you.
It is interesting but it does look like a careful distillation of (Spark and) biggers open-weight models.
The progress compared to Qwen3.6 27B is good, not that impressive, it's a 4 months old model. (kuto to them to compare to 27B dense and not 35B MoE, it's more fair to do so). It is very probable that Qwen3.8 27B will crush Glimmer-30B on most benchmarks.
What I think would be perfect is a model that could run on a single DGX spark and be competitive with DSV4 Flash 731. Flash is already a game changer. Hopefully meta plans on this, like the old 70b. V4 flash is smart enough for any use but slightly too big. 27b-30b isn’t intelligent enough.
Tried it (the full version, using 120 GB RAM), wasn't impressed, gave it some defective code, and asked it to fix all errors. It kept looping around and around and digging itself deeper and deeper into a rabbit hole; eventually, it got into a "reasoning" discussion about whether a custom compiler was used that supported the wrong syntax...
Still needs 32-64GB memory to run it locally. 64GB Macbook pro with an M5 chip costs more than 4k Euros in Germany. A more practical model would be a language specific (e.g Python or JVM language) and excellent at tool calling and reasoning. Maybe that way they can shrink it even more.
I lament the comments saying this in any way redeems Meta (the company).
The researchers releasing this stuff have almost nothing to do with Meta other than being bankrolled by the slaughterhouse.
You aren't the customer, you are the pawn in big tech's game of thrones. Your good will is a commodity to be traded, almost literally. It will be used against you the moment it's convenient. This is open weights because Meta couldn't monetize it in any other way than to cloud developer's judgement of their reputation.
But I guess most people just don't care.
I'm glad it's open. It does not make me think any better of Meta.
Pelican, rendered by Muse Glimmer on my Mac running LM Studio (with this model release: https://lmstudio.ai/models/muse-glimmer): https://tools.simonwillison.net/markdown-svg-renderer#url=ht...
It has all of the components of a pelican riding a bicycle, though not exactly arranged in the right order!
(For comparison, here are the pelicans I got from Muse Spark 1, 1.1, and 1.2: https://bsky.app/profile/simonwillison.net/post/3mseqv5z4qk2... )
Optimizing models specifically for always-on local agent workflows is the right primitive for decentralized systems. Brilliant release.
Happy to see meta back in the game, it's like after llama nothing came out that was comparable to mainstream open models.
Great to see Meta back, looks like really strong, local model, can't wait for llama.cpp support.
The post suggests that you need an rtx 5090 use it, which is currently selling for around $5,000 USD. I wouldn't exactly call that "my device", since my device costs about 25% of that for the entire computer.
For the same cost, you could run on a frontier model on a pro plan for two years. The economics dont make a lot of sense for this to me, so I would love some input on why people want to do this instead (privacy, for fun, etc).
Optimizing speed is really the way to go. Yet 24GB is not what everyone can afford. Maybe we could take some of those 56tk/s and transfer into some free RAM space using MoE loading ? I'd be glad with a less than 10GB and more than 6tk/s model.
The comparison set is Gemma4-31B and Qwen3.6-27B, not the current Qwen
Fair on size, but the headline numbers are against a model a generation back
good to see new open weights releases from meta
Having played around with this model a bit, I am fairly confident that it is not competing in the coding space.
It can do that, but its actual selling point appears to be a different take on guardrails and safety alignment.
Either that or the only new training data left was industrial quantities of dark romance literature and Wattpad.
Clever business move. 131k context is more than enough for that use case, and due to that small K/V footprint, you can probably have a bunch of characters on the same GPU.
Or it's just a happy little accident. We will never know.
___
I was informed that normal people use LLMs for mundane tasks like asking for a pancake recipie.
That it apparently can also do decently.
Unfortunately, it is also very confident, regardless of whether it is actually correct.
So maybe it should actually stay the smut engine and nothing else.
Meta seems like the one American bigtech that would distill the the other American frontier models. My enemy’s enemy is my friend?
Let’s give thanks to all those meta engineers who have been ripped for my heir teams (while sitting right by them) working on manually tagging data. I guess the morale dip paid off in some way? I wish you all well and hope you find some happiness … IYKYK
I tried to run it with lemonade by installing it via hf but did not succeed, it gets some weird 500 errors. I also see that ollama has currently only an mlx version available.
Anybody here succeed to run this on AMD?
Looking forward to giving this a try with llama.cpp. I’m watching the open-weights competition with high expectations.
and immediately followed up with Manifesto from the man himself - what / how are they going to make of it longer term?
I guess for FOSS and self hosted it is good - but I am still wondering how are they going to Meta-stasize it ;)
Wow, Meta is back (at least for now)!
I like this class of model. Multi-token prediction makes it viable to run dense models at not-too-far-off speeds as MoE models with much better intelligence.
The submission’s title (open weights 30B local coding model) is luckily wrong: This is meant to be a general agentic model.
It even comes pre-quantized and with a MTP/drafter model. Looking good!
Let’s hope they aren’t dishonest with the benchmarks this time …
I'd really like to see a 45B-ish dense model ready for a dual GPU setup. Something with a little more intelligence while still within the range of some higher end local setups.
Meta released their own 4-bit quant of this model for devices with 24GB VRAM.
That's a modern gaming laptop; cheapest I see in the US with 24GB is $3.5k.
Should be quite a bit faster than the new M5 MacBook Pro, and you can run Linux on it!
Another candidate for the 7900XT (20GB VRAM) I got sitting around. I pulled latest llama.cpp (targeting vulkan during build) after seeing a muse PR merged a few hours ago, and unsloth/Muse-Glimmer-30B-GGUF:UD-Q4_K_XL runs on my 7900XT barely (and with no MTP). Sits at 19GB VRAM w/ 4 parallel 113k context slots, all layers on GPU, and at 700 tok/s prompt, and ~36 tok/s generation.
Waiting on Q3 to download to check speed + do my usual anecdotes. I generate beefy code snippets and poems, and also ingest my HOA declaration and answer nuanced questions.
edit: i should've prefaced this somewhere with: This card ballparks at 800GB/s IO, which I can't seem to find easily on the market anymore. Kinda the ideal card for this model, if I just had a _little_ more VRAM (XTX is 24GB).
edit2: not mtp, this is dflash model (param in child comment). I'm up to ~60 tok/s generation and sitting at 19GB VRAM (i added --no-mmproj (makes it text-only i believe) because I'm used to speculative decoding wanting more VRAM and I'm already close to the limit :sweat_smile:)
Meta did not abandon opensource. I would love to see a smaller distill, or a moe of this size but the benchmarks seems competetive as long as it isnt benchmaxed witch i would not be suprosed if it is.
Many companies are stressed about token cost, as we are moving to a consumption based charge. In the meantime - new open source models, such as DeepSeek V4 Flash and GLM5.2 reduced the price to about 13x chepaer. Also OpenAI had reduced its price for considerably. Now Meta is back in this game. The upcoming months are going to be interesting (GoT)...
The favourable comparisons to Gemma 4 and qwen3.6 look promising!
That's a bit amusing - not that I have the hardware to run it, but officially it's not available in Hong Kong. Not that getting it would be much of a problem with a help of a VPN either, but I'll assume mainland China is also restricted. Certainly not a competition for Chinese open weight models... in China.
Sorry I dont know if this is the right place but... 2000AD The Glimmer Rats , was the best drawn comic strip story by far in that publication.
The combo that makes agents reliable: progressive tool routing, persistent memory, and multi-model failover.
Where is the pelican??
In my experience it's faster (10tk/s vs 35tk/s) and better than qwen3.6 series.
kind of a nonspecific complaint, but i haven’t yet had much luck with anything under ~120b, feels like models released on that order is coming to a trickle. the last few qwen models didn’t seem to go that high, and i got worse results than qwen3.5-122b
Next step: Burn the weights of these local models into an asic that ships cheap on a laptop (AMD/taalas looking at you), and I will be a happy camper. Make it pluggable so I can select a model I want. I use qwen3.5-coder currently on my laptop, and while it works well enough for me, it is somewhat slow processing tokens.
I would hazard a guess that fast small models with a smart agent harness can do quite well compared to large models which cant be run locally.
Just asking, what is the recommended models for M3 MacBook with 18G memory? Seems modern local models are not available.
Personally I would never trust a coding agent or agent harness from Meta.
I agree with their open-source model approach, but actually trusting Meta… to protect my privacy and my data… when it’s running on my personal hardware…
Not . In . A . Million . Years - that ship has sailed
I wish they would release the quantized versions in a safetensor format. Many frameworks can't load PTE and GGUF.
Having just bought a 5070 Ti (16GB) instead of a 5090 (24GB), I am sad.
Meta knows how to win back developer's hearts .... let's see if they have the goods
even 30B model is too large to large on local device (low end). meta should provide free hosted model api to use it.
Refreshingly practical
How are you handling the tradeoff between quantization for device fit and accuracy loss on tool calling? That's where local agents typically break down in production.
The SWE bench verified score is similar to Opus from not so long ago.
Sure, you can get better performance from cloud models.
But most software, not just AI, will be faster and more reliable in the cloud. The question is do we need that additional power and cost.
If the answer is no, then just like other software, people will run AI locally.
Meta's clearly changing strategies back towards their original "frontier open source", but this time around they have a lot more competition from leading Chinese labs.
I'm all for it though, and I think Glimmer is a fantastic bet on locally-hostable models. I for one would love to self-host as much as I can.
Can I run this on my RTX 5090?
Clear, useful, done
How many data using in Polish Language?
will it run on 2x 5060Ti with 16GB each?
Any MLX results?
Qwen 3.6 27B is still such a beast!
Random question: Would you be able to run this model on a Macbook Air M5 (latest)?
LOL the mogging of GDM is hilarious.
I don't know why MSL released this, but it is very nice that they did.
The more open weight models get released the greater the market for personal and small business oriented hardware to run these models. This will drive lower cost hardware, which has stagnated in recent years due to most software not needing the performance and capacity.
what's the catch?
From twitter Alexandr Wang
> 3/ muse glimmer was developed with its own architecture and recipe, optimized for its size and agentic performance requirements.
This means we're in the endgame does it not? If the architecture was NOT optimized for intelligence ...
and now the recent Meta model 'security issue' begins to make sense
thank you zuck.
what I can tell is that Meta is just starting and it is so underrated.
"Meta Muse" immediately made me think of Metamucil.
Product teams really need to hire at least one or two people with a 12-year-old's sense is humor. They need to winnow all the potential stupid jokes out of their product namings.
Quietly the best thing in the thread
PSA: Fast RAM isn't going to be getting cheaper anytime soon. Acquiring inference hardware is a really good way to own an appreciating hard asset. Learning how to use it and cool it is a hacker's journey worth taking. My 4090 I bought in late 2022 for $1600 is selling for a cool $3,489.95 right now, and going strong under nominal use. My DRR5 has tripled in value, my nvmes almost doubled. I grabbed a 128GB M5 Max MacBook Pro when they were still available and told all my friends to buy at least one. With that and a base M4 Studio 36GB, HuggingFace rates that hardware as:
> Amazing! You have a total of 128.94 TFLOPS of computing power. 71.3% percentile on scale of "GPU Poor" to "GPU Rich"
The way I see it, these are amazing machines that the richest folks are hovering up. I think they should be in the hands of regular people as much as possible. They depend on an incredibly global, increasingly fragile supply chain. If the become impossible to produce, their value would increase tremendously. I think they will become really valuable to you to use the tokens directly, but if that isn't the case, they can be rented out or resold. Please don't just buy any hold. Let's try to get as many people that can use them for decent things that help humans. For example:
https://spectrum.ieee.org/small-language-models-ai-pharmaceu...
As an industry, I wish we would stop calling these things "open weight" because it is too easy to confuse with actual "open source", which they are not.
Photoshop source code+ OSI license = open source
Photoshop binary you can run on your own computer = open weight
Photoshop SaaS web app = closed, proprietary (Opus, GPT, etc.)
"Open weight" models are still just binary blobs that are completely inscrutable. It's like bringing home a dog from the rescue and just hoping that it doesn't have a tendency to bite kids in the face. You just can't know. The only thing that you can do is try to add more training (fine tuning) telling it not to bite kids.
I don't think the FOSS community has ever accepted this, but somehow we're feeling like it is okay now.