- i really dont want to scroll 200 pages just to see what your blog articles are
show comments
jordanb
It's the bitter-lesson to feature-engineering lifecycle.
When a technique or technology is new people are making massive gains by just applying it to some use case, or gathering more data for training, or giving it more resources.
As time goes on those "bitter lesson" gains start to hit the shallow part of the logistic curve and companies have to start investing more and more effort into engineering for each small, incremental gain.
show comments
truvem
One thing that makes LLMs complicated in production is that they're stateless — every call starts from zero. The complexity compounds when you need agents to maintain context across sessions and models. That's a layer that's largely missing from most stacks today.
show comments
charcircuit
Why didn't this author compare Llama 3 with GLM 5.2 (released 1 week ago) which is a more standard attention based LLM? To compare 2 separate families of LLMs and then pointing out that they are different is not a surprising result and detracts from the point the author is trying to make.
If you look at it, the diagrams are very similar, but the main differences are that the feedforward is replaced with a MoE (router to multiple feedforwards) and the model has a different attention implementation.
- with all due respect, from a ux perspective, could you kindly add a page where i can see just the titles of all your blog posts
- https://ianbarber.blog/blogroll
- https://ianbarber.blog/archive
- https://ianbarber.blog/blog
- https://ianbarber.blog/posts
- none of the above links work
- i really dont want to scroll 200 pages just to see what your blog articles are
It's the bitter-lesson to feature-engineering lifecycle.
When a technique or technology is new people are making massive gains by just applying it to some use case, or gathering more data for training, or giving it more resources.
As time goes on those "bitter lesson" gains start to hit the shallow part of the logistic curve and companies have to start investing more and more effort into engineering for each small, incremental gain.
One thing that makes LLMs complicated in production is that they're stateless — every call starts from zero. The complexity compounds when you need agents to maintain context across sessions and models. That's a layer that's largely missing from most stacks today.
Why didn't this author compare Llama 3 with GLM 5.2 (released 1 week ago) which is a more standard attention based LLM? To compare 2 separate families of LLMs and then pointing out that they are different is not a surprising result and detracts from the point the author is trying to make.
https://sebastianraschka.com/llm-architecture-gallery/?compa...
If you look at it, the diagrams are very similar, but the main differences are that the feedforward is replaced with a MoE (router to multiple feedforwards) and the model has a different attention implementation.