I'd be much more worried if my local water treatment plant or local bar had an ethics team, and people working there probably would feel slightly more useless than Raytheon's ethics team, which I'm sure feel like they're doing something important and worthwhile.
Obviously not. The ethics team at Raytheon is not going to ever be allowed to overrule the leadership or more importantly the political leadership of this country whose support of Raytheon is fundamental to their survival
They don't direct the company, they bring language and legitimacy to its actions. And like an attorney, just as soon as they can't pull that off, they become something between useless and an internal threat.
Sounds like perfect credentials.
a) champaign socialist
b) head of ethics at OpenAI and not really listened to because you're really probably (a)
Once these things are taken care of, there are so many opportunities to make so much money. If you're in the US, look at what the current US politicians and the administration is doing, and basically copy them. Right now in the US is the era for shameless greedy people, and it is extremely easy to make money from the gullible population. But again, requires the whole "shameless" part.
Not a single link to any of those. That's just rude.
It seems like the pattern is almost always the same: the ethics team exists quietly in the background, and the only time anyone hears about them is when they either publicly leave the company or start a campaign against it.
So what exactly is the business case? Is it actually worth maintaining an ethics function just to be able to say you have one, when the only time it becomes visible is when it goes against the company publicly? And if this outcome is so predictable, why do companies keep hiring these people in the first place?
I want to clarify that I am not doubting the cybersecurity capabilities of frontier models-- I have no reason to believe that the hack itself was not carried out by the model. But the companies' use of LARPing language in describing the incident, granting agency to the models in their phrasing definitely does raise suspicion on my end, particularly in light of their track record of releasing models which have been 'too dangerous to release' for years now.
This might actually be the reason they pushed her out. OpenAI and Anthropic base their whole business plan and philosophy on the idea that LLMs are a unique technology to the point they can cause infinite harm or benefit to humanity depending on who controls them, so the only rational choice is to invest all your resources in getting to ASI first so you can tell it to stop any other attempts. Linking AI to old questions defeats that idea because it exposes AI as not so unique.
The other more likely option is she was asking uncomfortable questions, either about the social impact of building AI controlled by a for-profit entity or about the possibility that AI systems are conscious.
Her typical day: "No Sam, you can't just publicly screw over everyone and get rid of all the jobs, the peasants care about that sort of thing and if they get angry enough we'll have real problems. No we can't just kill them all, why would you suggest that?"
That gives you a three-layer picture:
Task objective: Did it accomplish what we asked?
Acceptability constraint: Did it avoid unacceptable ways of accomplishing it?
Adversarial evaluation: Can we find trajectories where the model gets a high score while violating the intended constraint?
I think what you are pointing to with your reference to Goodhart's "Law" (which is from monetary-policy and school-exams, i.e. "teaching to the test") is that the models would eventually do the minimum amount of ethics required to have an action stay valid. However, if a model is rated on ethics and it achieves the short-term-objective, then the higher ethics scoring trajectory should win. In short, 1) this is leagues ahead of where we are now for AI safety and breaking-out-of-the-lab, and 2) in baking ethics into a measurement we are adding "the spirit of the exercise" back into the maths, which is something Goodhart's Law does not account for.
It's OK though, they don't need an ethics team because everyone at OpenAI is ethical:
> “AI ethics doesn’t live with one owner or team at OpenAI and ethical considerations are deeply embedded into the model-building process driven by a number of teams across research."
> She has held a variety of academic positions at Temple University, Princeton, and the University of Pennsylvania – where she completed her PhD in Political Science and Government. Her Dissertation was titled “Small Talk: The Socialities of Speech in Liberal Democratic Life.”
This doesn't even feel relevant to ethics. It's adjacent, but like... I guess I'd expect a moral philosophy degree? Maybe even mathematics in there?
I find the position odd. Curious to hear what these people do and how you choose who to hire.
An ethics department is like a human resources department, but for broader ethics related matters.
HR: Ostensibly exists to support the employees, but really exists to shield the company from both a PR and legal liability standpoint when the company has a rift with one or more employees.
Ethics Department: Ostensibly exists to make sure the company acts ethically, but really exists to shield the company from both a PR and legal liability standpoint when the company acts unethically.
It is a good personal signal that you have never worked for a company with an ethics department or an ethics employee. The less ethical your company the more likely it is to have an ethics department.
Head Of Ethics comes in, looks at what is really going on behind the curtains...
Head Of Ethics scrams out of there as fast as they can.
100% of the time, ethics department attracts activists, which is 100% trouble for the company in the future.