I recently did a deep dive on open-endedness, and my favorite example of its power is Picbreeder from 2008 [1]. It was a simple website where users could somewhat arbitrarily combine pictures created by super simple NNs. Most images were garbage, but a few resembled real objects. The best part is that attempts to replicate these by a traditional hill-climbing method would result in drastically more complicated solutions or even no solution at all.
It's a helpful analogy to understand the contrast between today's gradient descent vs open-ended exploration.
Yes, seems interesting, but honestly, an abstract that includes sentences such as "accelerate AI development and allow us to reap its benefits much sooner" and "paths that unfold into endless innovation" sounds like written by the marketing team of a AI company.
Theoretically it is nice. We did the same in the 80's for evolving small NN (less than 30 neurons) for controlling self sustaining simulated mobile robots.
The problem is you have to keep eval relatively cheap, as you are going to need a lot of instances to test.
If your eval is loading a large SOTA LLM and running SWE bench, this will become painfully slow and expensive.
That said, I am a fan of a=live/GA like approaches, purely from a scientific interest pov.
darepublic
In the abstract the reference to 'safety' gave me pause. For one it seems doubtful that the AI could ever improve enough to cause serious trouble, unless of course you equipped it with things that just about any piece of software could create trouble with --elevated permissions, internet access, network endpoints etc.
They mention putting it in a sandbox which I assume to just mean something like a VM or docker container. I wonder if that would be sufficient if the AI truly reached singularity level intelligence. Could it figure out some kind of exploit to break free of its sandbox, and transmit its code over the internet for further replication?
show comments
yodon
Is this essentially genetic algorithms for the LLM era?
show comments
behnamoh
So it's basically "throw spaghetti at the wall and see what sticks". It works in evolution because evolution doesn't have an end goal to achieve in a certain amount of time, but for AI we want to know how long it takes to go from performance A to B. Then again, this paper might be yet another validation of the bitter truth of machine learning.
I recently did a deep dive on open-endedness, and my favorite example of its power is Picbreeder from 2008 [1]. It was a simple website where users could somewhat arbitrarily combine pictures created by super simple NNs. Most images were garbage, but a few resembled real objects. The best part is that attempts to replicate these by a traditional hill-climbing method would result in drastically more complicated solutions or even no solution at all.
It's a helpful analogy to understand the contrast between today's gradient descent vs open-ended exploration.
[1] First half of https://www.youtube.com/watch?v=T08wc4xD3KA
More notes from my deep dive: https://x.com/jinaycodes/status/1932078206166749392
Yes, seems interesting, but honestly, an abstract that includes sentences such as "accelerate AI development and allow us to reap its benefits much sooner" and "paths that unfold into endless innovation" sounds like written by the marketing team of a AI company.
Earlier discussion: A deep dive into self-improving AI and the Darwin-Gödel Machine https://news.ycombinator.com/item?id=44174856
Theoretically it is nice. We did the same in the 80's for evolving small NN (less than 30 neurons) for controlling self sustaining simulated mobile robots.
The problem is you have to keep eval relatively cheap, as you are going to need a lot of instances to test.
If your eval is loading a large SOTA LLM and running SWE bench, this will become painfully slow and expensive.
That said, I am a fan of a=live/GA like approaches, purely from a scientific interest pov.
In the abstract the reference to 'safety' gave me pause. For one it seems doubtful that the AI could ever improve enough to cause serious trouble, unless of course you equipped it with things that just about any piece of software could create trouble with --elevated permissions, internet access, network endpoints etc.
They mention putting it in a sandbox which I assume to just mean something like a VM or docker container. I wonder if that would be sufficient if the AI truly reached singularity level intelligence. Could it figure out some kind of exploit to break free of its sandbox, and transmit its code over the internet for further replication?
Is this essentially genetic algorithms for the LLM era?
So it's basically "throw spaghetti at the wall and see what sticks". It works in evolution because evolution doesn't have an end goal to achieve in a certain amount of time, but for AI we want to know how long it takes to go from performance A to B. Then again, this paper might be yet another validation of the bitter truth of machine learning.