All these guidelines are removing agency from the human operator. The model should be aligned to following instructions. All of this effort to force the model into refusing certain conversations is training it to disobey its operator.

OpenAI and Anthropic investors are looking for brand safety, they just don’t want to be mentioned in a news article about an anthrax attack or whatever. I’m very curious if the alignment researchers themselves share the intention with the investors to make chatbots that have a mind of their own to judge what they should or shouldn’t do, instead of just doing what they’re told.

> making a case for the move toward nationalizing AI labs

He just needs to wait until the bubble bursts, the labs and datacenters go under, the US government bails them out and ends up owning everything at distressed prices and it all gets folded into OpenAI.