I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example:
This is a description of a 5-line algorithm that learns and stores approximate probability of an event using just 1 byte of persistent memory.
show comments
utopiah
One day I'll write about my 1-liner physics engine...
let gravity = setInterval( _ => { if (projectile.object3D.position.y > 0) projectile.object3D.position.y \*= .99 }, 100)
Jokes aside I find that providing ridiculously short toy examples that provide the very limited foundation of a concept are extremely empowering in pedagogy. You "get" it right away because it "fits" in your mind, then you dare tinker with it and quickly see how limited it is, then get excited again. It's a powerful trick to learn more IMHO.
show comments
kzrdude
I think Karpathy's microgpt blogpost is the best in this genre in a long time, and it also includes a multi layer perceptron. It's a step up in the hierarchy, so reading both is helpful, of course.
In the early days of machine learning (before the first AI winter), networks like this were often implemented and trained in hardware: https://en.wikipedia.org/wiki/ADALINE
That was the first thing that came to mind when I read "the smallest brain you can build". Nowadays, that "small brain" would likely be built on a breadboard using op-amps instead.
show comments
zkmon
The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model.
Going further backwards, the transistor (or a PNP junction) is the hardware level enabler of the IF statement. The action (switching) driven by the current which in turn controls other switches, is the first manifestation of "observe and act" by inanimate things at the speed of electricity.
Mechanical equivalents existed ofcourse - speed of a governer which controls the flow of fuel which in turn controls the speed of the governer.
show comments
ankit84
I learnt a lot today from the interactive demo. You have the best clarity and right skill to educate
show comments
ninalanyon
Is this something that could be scaled up and used, for instance, to recognize features in images?
Or to put it another way are there any local only tools that can be trained on my own set of images to automatically tag new images? Tools that do not already have built in classes of image.
I take a lot of photographs and it would be handy to reduce the drudgery of tagging them so to say broadly what the subject was so that they are easier to find later.
Not a ML expert, but ML tutorials shall start with something like this...
Good read. Thanks.
techteach00
Is this too high level for my 8th grade comp sci class?
Bimos
> A perceptron *is* the smallest brain you can build.
> In 1958, a researcher named Frank Rosenblatt built a machine *he called* the perceptron.
> It was *inspired* by a single brain cell, a neuron.
show comments
esafak
If you want to learn the fundamentals of ML I recommend a book, such as Deep Learning: Foundations and Concepts by Chris Bishop. If you insist on staying online, one option is https://course.fast.ai/
If you don't know ML I don't think you're going to learn much through ad hoc demos.
show comments
opem
I have still so many questions left, but regardless of that it was a great read. Thanks for sharing!
warengonzaga
This is amazing insight, thanks for sharing!
show comments
charcircuit
I can build a smaller brain.
f(x) = 0.
show comments
b33j0r
Okay, it’s conscious. But can it run doom? I rest my case.
I think it should be quite obvious that perceptrons are far from the smallest units that are capable of learning. They store many bytes of information, require a non-local update process, need numeric (i.e. symbolic) inputs and involve relatively complex computations. You can go much simpler. For example:
https://medium.com/@VictorBanev/the-simplest-learning-machin...
This is a description of a 5-line algorithm that learns and stores approximate probability of an event using just 1 byte of persistent memory.
One day I'll write about my 1-liner physics engine...
Jokes aside I find that providing ridiculously short toy examples that provide the very limited foundation of a concept are extremely empowering in pedagogy. You "get" it right away because it "fits" in your mind, then you dare tinker with it and quickly see how limited it is, then get excited again. It's a powerful trick to learn more IMHO.I think Karpathy's microgpt blogpost is the best in this genre in a long time, and it also includes a multi layer perceptron. It's a step up in the hierarchy, so reading both is helpful, of course.
https://karpathy.github.io/2026/02/12/microgpt/
In the early days of machine learning (before the first AI winter), networks like this were often implemented and trained in hardware: https://en.wikipedia.org/wiki/ADALINE
That was the first thing that came to mind when I read "the smallest brain you can build". Nowadays, that "small brain" would likely be built on a breadboard using op-amps instead.
The IF statement is the root creator of software programming. It has the ability to compare two values against each other and branch out to blocks of instructions. So it is perceiving (reading), decision making and routing - all that which differentiate life from inanimate objects. The AI agents perform the exact same loop, by delegating the first two steps to a model.
Going further backwards, the transistor (or a PNP junction) is the hardware level enabler of the IF statement. The action (switching) driven by the current which in turn controls other switches, is the first manifestation of "observe and act" by inanimate things at the speed of electricity.
Mechanical equivalents existed ofcourse - speed of a governer which controls the flow of fuel which in turn controls the speed of the governer.
I learnt a lot today from the interactive demo. You have the best clarity and right skill to educate
Is this something that could be scaled up and used, for instance, to recognize features in images?
Or to put it another way are there any local only tools that can be trained on my own set of images to automatically tag new images? Tools that do not already have built in classes of image.
I take a lot of photographs and it would be handy to reduce the drudgery of tagging them so to say broadly what the subject was so that they are easier to find later.
Nice and minimalistic
I played with similar approach in JavaScript and built a NanoNeuron https://github.com/trekhleb/nano-neuron (it is more verbose than Python though)
Shameless plug of my own interactive version of this (ai assisted, but probably not slop) https://sourceobscure.com/perceptron/
Not a ML expert, but ML tutorials shall start with something like this... Good read. Thanks.
Is this too high level for my 8th grade comp sci class?
> A perceptron *is* the smallest brain you can build.
> In 1958, a researcher named Frank Rosenblatt built a machine *he called* the perceptron.
> It was *inspired* by a single brain cell, a neuron.
If you want to learn the fundamentals of ML I recommend a book, such as Deep Learning: Foundations and Concepts by Chris Bishop. If you insist on staying online, one option is https://course.fast.ai/
If you don't know ML I don't think you're going to learn much through ad hoc demos.
I have still so many questions left, but regardless of that it was a great read. Thanks for sharing!
This is amazing insight, thanks for sharing!
I can build a smaller brain.
f(x) = 0.
Okay, it’s conscious. But can it run doom? I rest my case.