Genuinely it take a lot of work and talent to be this hype-motivated and completely ignore anything except what is popular on X at any given time.
Note: RSI is an incredibly important topic -- I just don't care to listen to Sakana on this matter -- they are the epitome of "hypebeast" https://www.urbandictionary.com/define.php?term=hypebeast
(Thanks for sharing hardmaru)
The point of this announcement is to draw attention to the fact that the currently hyped topic is what they have been working on since their inception. If anything, it gives off a Schmidhuber-esque 'actually it was me who invented that' vibe. But trying to retroactively claim credit for the hype is nowhere near the same as following the hype.
As for your impression that the company is more generally hype-chasing, I'm really not sure how you would come to that conclusion. At the time of their founding, chatbots were the hype on the product side and model scaling was the hype on the research side -- topics they have largely eschewed. They instead were founded with a focus on evolutionary and collective intelligence and have maintained a fairly cohesive research direction ever since.
"For the past 300,000 years, Earth has had only one form of advanced intelligence on it: humans. With the recent advent of AI foundation models, some believe we are at the dawn of a new kind of intelligence. As AI continues to evolve, we may witness the proliferation of diverse intelligent lifeforms coexisting with us."
I am not even going to link more than one thing I think I've made my point
Edit: Nevermind, realized he's just an uneducated troll just looking for his kicks. Comment flagged.
Now that models are getting stronger at agentic work, it is very natural that many labs are chasing some form of auto-research.
> was posed specifically under the framing of there needing to be more fundamental research beyond squeezing as much as we can out of relatively vanilla transformer stacks.
Not to be contentious, but this is so broad of a description that it could include literally thousands of papers in the last year or two. I'm imagining double digits or more if we go back the full decade.
I'm saving brownie points for people who deserve them
Organizations, especially businesses, are not individuals. If the implication is that David Ha has always been doing this, and will always be doing this, and that Sakana is David Ha ... then that's a far worse insult to the employees at Sakana than my little tweaking.
What do you think they're improving on? How would a model self-improve without some metric/data of some kind to check? When you have metrics+data, that is a benchmark. And yes, simulations and or soft-verification like LLM judges are still a kind of benchmarking. Maybe its not a static benchmark they can easily hack.
Folks -- RSI does not mean the self-improvement is them going to therapy and seeking inner peace to overcome trauma.
Edit: Nevermind, realized he's just an uneducated troll just looking for his kicks. Comment flagged.
I ended up borrowing the ideas from it for one of my own personal projects.
SGD generates a stronger learning signal,is more efficient, and scales better. Using it end-to-end makes it stronger yet.
Yet somehow mixing in a weaker blunt evolution stage improves the result?