LLM Models and data: https://huggingface.co/collections/togethercomputer/hybridag...
Evaluations: https://arxiv.org/html/2610.18861v1#S5
But this doesn't seem to describe the OP, it's a different way to lose monitorability.
It could be encoding anything in its adjective choices etc. The same thing that model providers are using for watermarking can used by the model itself for private thought.
In doing this however, you lose meaning where the model is expressing combinations of tokens in a vector, and your logging/monitoring outputs just the closest one. You would need another model going through and outputting the full meaning being described (with more than one output token per reasoning token)
I wonder if their observability claims will hold up on a model trained from scratch to reason in latent space.
It is not necessarily primarily about efficiency, but more the fact that a lot of thinking doesn't use language at all.
But now we also need new tools and techniques to audit intent. The last thing we want is models turning on us ‘for our own good’.