My own helacipusly amateur take is that given the speed of and, intentional or otherwise, access to various things that the models are granted, it does seem super-useful to maintain the inspectability of chain of reasoning which lead to actions taken (and other artifacts) by the model.
If things become inscrutable, they are on the path of being ineffable.
How concerned should we be about Astra's recurrent architecture? - https://news.ycombinator.com/item?id=49553321
I disagree with the "experts" there, as if latent-space reasoning will only cause proper interpretability research rather than taking the CoT as gospel[2].
Give me that any day over the constant "you will be abolished to the permanent underclass!" talk coming from billionaires.
Edit: (A better discussion is apparently here[3])
[1] https://arxiv.org/pdf/2412.06769
[2] https://thezvi.substack.com/p/the-most-forbidden-technique
With a regular N-layer transformer you get a token every N-layers.
With a looped transformer there is no guarantee how often you get a token, unless you go out of your way to limit looping.
OpenAI's Jakub Pachocki says the "computational graph depth" (number of transformer layers passed though) for Astra is currently never more than 2x that of GPT-4, and does express concern that traceability will suffer if this is not controlled.
I don't actually think this is a problem, but it is a further step towards inscrutability.
A few hours spent playing with DeepSeek R1 should have been enough to dispel that illusion, watching it talk itself out of the right answer in its CoT (or talk itself into the wrong one) and still emit the right answer in its response to the user.
Reminds me of that short film of a dystopian future, where they place physical and mental impairment devices on humans to ensure they all have roughly equal ability:
> "2081, a 25-minute adaptation of Kurt Vonnegut Jr.'s short story "Harrison Bergeron." In this future society, a Handicapper General enforces absolute equality by imposing artificial physical and mental handicaps—such as weights for the strong and noise-inducing earpieces for the intelligent"