It even says, quoting from the website: "Headlong is alpha research software."
Yeah that's totally something I want to curl thing.sh | bash , great idea.... Wow.
I understand that people want to get people using their software as quickly as possible and with the absolute minimum of friction, but let's put some more thought into how this could be done in a less sketchy way.
It's like we've regressed to the days when you would download a .exe file from tucows and blindly run/trust it on your windows 98SE PC.
> Whatever anyone tells Audel becomes part of the single experience that every other conversation draws on. In practice, Audel is bad at keeping secrets. Ask it what it’s been working on with someone else and it will often just tell you, even though we’ve asked it not to. We also haven’t studied what happens when two people give conflicting instructions. For now, we assume anything you tell Audel is shared with everyone on the team.
Where are the interesting engineering parts at? Seems to be an interesting idea and perhaps design, but to call the implementation/engineering itself bad seems to be an understatement.
Humans aren't scheduling a wake-up to the next thought. Ideally, a sub second agentic loop with no FINAL / wake-up, always "spinning" would get closer. I'm conscious about the waste of resources this would drag with it (because of current architectures), but exciting still.
PS. I love the take on using bash instead of Python (one less abstraction layer!) and using UNIX fundamentals as stepping stone when composing tools as agents are naturally drawn to using it on a box anyways.
There are just so many now that it's hard to personally test them all or just trust the vibes.
Spend a day or two going through your existing chat sessions, and create your own private benchmark with test cases based on real tasks, that you don't share with anyone nor publicly. Make it easy to add/remove new harnesses and model combinations, make it give you a final score, ideally avoid using other LLMs for scoring, then use this to figure out if the new model/harness actually improves things for you.
I've been doing this for some time, and while most new releases show big increases in the benchmarks/evaluations, my own benchmark usually barely moves.
If you wonder what the Rust is for: It is the Ratatui TUI.
https://github.com/exoharness/exo/
https://github.com/laude-institute/headlong
https://github.com/microsoft/agent-lightning
and now https://github.com/PrimeIntellect-ai/prime-agent
Of course the don't have exactly the same scopes but they are in general all about persistent memory and / or continous agent loops. Like I miss those times where only once a week a new js framework was promoted.
This is why "I made it think in a loop" doesn't result in significant improvement in LLM performance. It's not learning. You need RLAIF, STAR, IDPO, etc to retrain the model to learn from its mistakes. And you need a human to review it so it's not compounding mistakes. It's expensive and time-consuming. Doing it wrong leads to bad outcomes. But not doing it leads to no significant improvement.
Wow. So, be nice or I'll replace you with a very large shell script?