A fairly obvious theory at that, unless I&#x27;ve critically misunderstood what&#x27;s being described. As with so many obvious ideas I&#x27;ve always assumed that the reason I haven&#x27;t come across it in the wild is because it doesn&#x27;t work (or is comparatively inefficient, or tends to blow up during the training run, or etc).&gt; Realized reasoning gains, hardware efficiency, and RL scaling remain to be established.Yeah so the first entry in that list is - if I may be so bold - typically what you&#x27;d start with at a small scale _before_ writing up and publishing your &quot;genius&quot; idea. This is the usual crank with delusions of grandeur presenting something straightforward that he hasn&#x27;t tested as though it were a working breakthrough.Ironically the cost of testing such theories has fallen to an all time low given the capabilities of coding models. I wouldn&#x27;t be surprised if a frontier model could one shot a test of this.

appears it&#x27;s just a theory, sort of surprised this lifted up so far on hn

Is this just a theory about an architecture or are there actually some benchmarks&#x2F;results to substantiate it?

not sure w&#x2F;o context why this is important -- no results&#x2F;implementation &amp; i believe there are prior combinations of transformers&#x2F;RNNs. but perhaps I&#x27;m missing the relevance&#x2F;insight

I can&#x27;t believe that actually works? This is far more interesting than the half baked ML idea.Edit: Thanks to your link I&#x27;ve now learned about the zigzag transformer which is yet another design that somehow feels like cheating reality. <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Zigzag_transformer" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Zigzag_transformer</a>

I thought it&#x27;s about <a href="https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Autotransformer" rel="nofollow">https:&#x2F;&#x2F;en.wikipedia.org&#x2F;wiki&#x2F;Autotransformer</a>

Recurrent Looped Transformer