Is it my fault or the company who trained it and is running the inference?
This is not the case with SaaS services.
This is a deflection. A human is responsible for the use of a gun. The individual/corporation ought to be responsible for the actions of their agent. If you purchase an agent from someone else it’s your responsibility according to the terms of your agreement. And, as in many other things in life, there ought to be certain rights certain parties cannot legally be allowed to sign away.
The case will be lot more complicated if someone uses Kimi to hack into a site. Should the person giving agent the command responsible or the CEO of kimi.
The reality is they have to reduce the capability to ensure security. If someone wants more? Then use the product with your identity and face scan at each session.
Trade offs mate.
With LLMs, at least in the cases of internal/test models doing things they shouldn’t, the people “pulling the trigger” are the board and CEO.
Reading posts on here is slowly becoming akin to brain rot.
I don't think this takes seriously enough the possibility that said CEO doesn't think the failure mode is likely and ignores it. Plenty of people are willing to take risks of the flavor "heads you win, tails everyone loses".
Corporate judgements are a joke outside the EU's X% of revenue approach.
Current US law provides the individuals who benefit with corporate liability coverage. I.e. Altman personally gets to keep OpenAI's upside, but if it fucks something up that liability is only on the company.
That's an insane risk optimization environment to put in place for something scaling fast.
At minimum, US prosecution (at the state level, because Trump Co are idiots) for breaking existing laws is needed.
I do think this carries some weight from this particular author due to the length of his tenure. I happen to agree with him in spirit, but this is still largely a post revolving around sentiment not substance. Does anyone think that the overriding incentives even leave room for something like this in practice?
At least the Rationalists will handwave something for that with their "coherent extrapolated volition" idea where the superintelligence is supposed to figure out what humanity would collectively want if humanity was superintelligent and good, not that I buy it. This guy seems [.] to be coming from the NGO blob world.
Going behind a paywall is a reduced distribution over what an individual can easily access, and the content is no longer permanent but subject to whatever the publisher chooses to keep providing.
I sense that these are people who have already eaten the cake and want to somehow absolve themselves of it.
Like I said in an earlier comment, she could've just chosen compliance like many others _definitely would have_ to keep the infinite money tap flowing. Instead, she chose to risk her and her family's lives by publishing that book _under her own name_ *and then suing them* after FB tried to gag her.
Sure, that book might not have been possible. But also the unique things she did for the company might not have been possible as well. To her credit, she does a good job of pointing out that she was complicit, but if she had listened to her own voice early, there is a small possibility that Facebook might not have been as powerful. Multiply that possibility across several other employees and imagine where the road could've led.
There's a difference between post-facto bravery (sometimes much less than that) and using your own legs to walk away much early and not enabling things you are uncomfortable with. This is what other people have been trying to point out.
People who were part of the sausage factory, on gaining financial independence, feeling suddenly liberated to talk about how the sausage was made, seem like exactly the people who would be most able to speak to institutional problems.
This doesn't seem like an argument to discount their views?
You cannot take people, who first build the doombot and _then_ talk about it being dangerous for mankind, at face value. Especially when this playbook has been used multiple times within the past decade.
Besides, these "views" were already known to people who had their eyes and ears open. It's not something brand new. OpenAI has had multiple points in the past where its values have been tested and they've come out lacking. People who knew then, and only now talk about it, aren't people I can fully trust.
What playbook?
With AI, what is it? Scraping Australian government's data, and going around a bug in a website to get in?
I think humanity develops all its technology in three phases. Build it, see if it’s too bad, apply regulations and or roll back. We naturally won't move to the phase 3 before we see the phase 2.
Here we are talking about something with consequences in the digital world, usually on something pretty niche.
There IS an argument about pacing, and about not letting weapons, energy grids, hospitals, etc. getting managed by an autonomous AI, but I think we are still pretty far from it and even further to it being so in charge that it will obliterate us.
Is this the first time we have been in this position? Can anyone think of some prior examples?
Why do people working in tech consistently get disillusioned into some company's mission statement or the equivalent? Its easy to just say the simplest reason is money, but this has been going on for decades though. You don't see the same attraction to adult entertainment (gambling, video, etc.) software jobs so there is obviously a line a lot of people won't cross. Those industries are at least honest about what they do, its not hidden behind some mission statement.
By all indications the shallowest reasoning is once someone can "cash out" thats when their values matter more. Maybe there is an element of maturity that happens after working for 5+ years that kicks in? Maybe it really is achieving FU money? It would be interesting to hear honest accounts from people that went through that cycle across more industries than AI.
> Two changes are urgently needed. First: AI companies need to rely more on the safety expertise that already exists in other fields. And second, before we create systems significantly more capable than the ones we have today, we need new science to ensure that more capable models (and their successors) will make safe choices when we aren’t looking.
He mentions farther down about learning from aerospace engineers and nuclear engineers about safety. Those industries are heavily regulated, so perhaps regulation above a certain capability level is needed. Defining what that level is might be tough, though.
The second point is harder: in the field of AI, practice has extended far beyond theory, so his call for new science is going to be fundamentally tough, because we can't effectively coordinate a global slowdown in AI development so we can let theory catch up. This means, like so many other industries, the safety lessons will be written in blood.
Yeah so that's never going to happen
Last I checked it was still within the laws of physics to run air-gapped systems, and to ensure it is physically impossible for a model to “escape” or gain access to information it shouldn’t have. Maybe this safety guy should have been worried about that and not humble-bragging about writing 12 reports.
What are the concerns of individuals in comparison to the overall progress of humanity?
Always overlooked counterpoint: what point is the progress of humanity if it doesn't take into account the concerns of the individuals?
This pattern is playing out with increasing frequency.
And if we can't solve it for an AGI, what are we going to do with an ASI?
Mr Robinson if you are reading this – if you are truly concerned about AI safety share proof of donation of 100% of your OpenAI earnings and equity towards undoing the damage you have done to society during your time there.
In the absence of that this is simply a career pivot into being an AI "influencer" and/or raising money for a new scam.
Would I respect a martyr who sacrificed their financial security to do this more? Of course. But it's important to applaud people speaking out on important topics
This isn’t a zero sum game, I’m happy to hear from people both previously inside OpenAI and completely independent of them.
AI is a force multiplier for intelligence. Even if "aligned", aligned with whom or what?
Whom are you comfortable with, lording as some sort of demi-god over you?
AI doesn't tell you what goals you want it to achieve. Allowing people to destroy human society with it is obviously not a good idea.
He does give information, namely the culture there factually being inconducive to self-regulation.
You accuse the guy of "bias", but you never argue explicitly, what that's supposed to mean. Your implications actually run counter to your own implied goals.
I think there is a misunderstanding here.
The people who are annoyed at the accolades are claiming it was abduntantly clear for a long time to people on the outside that this was case, hence the increduality at the notion that it took a person on the inside a long time to realize this was the case.
The people who are annoyed are like the liberal kids in this video [0].
Sure, antagonizing people for "seeing the light" is probably not helpful, but there is no reason to give them extra credibility for coming to the same conclusion just way way later (despite being on the inside) as the people on the outside.
The author linked in this post does have "extra credibility" due to his direct involvement.
People having surmised that state before is nice, but since they've been ineffectual at getting society to actually act on that, now throwing away that extra leverage in favor of their point is at best ridiculous.
No they don't. By that logic, if they quit and said Altman was very trustworthy we should give extra weight to their words because they had direct involvement? How ridiculous are we trying to get here.
>now throwing away that extra leverage in favor of their point is at best ridiculous.
How are they throwing away extra leverage? Not putting people who recently quit on a pedestal does not negate those people's testimonies.
I agree that if your goal is to maximize quitting of talent at a company, it will surely discourage anyone else who quits hoping to reinvent their career as a lauded martyr against Big AI. In that sense they would be shooting themselves in the foot. But there is no reason it should deter other people who are quitting for more noble, less self-obsessed reasons. If I were the author of the article I wouldn't begrudge the skepticism. Given the article's first sentences, I'm led to believe they themselves would understand the sentiment. (I must admit I found it hilarious that the first sentence starts similarly to the speech the mom gave in the video I shared).
Sam is a shady dude, would not put it past him
They are not the same thing, and it’s unhelpful to assume they have no ethics.
Sorry that I don't take it seriously when the whistleblower parrots the narrative the CEOs of those companies are already espousing in the desire to amp up hype for an IPO.
This person should be shamed.
It is even easier to just quietly retire and spend the spend of your life on interesting expensive hobbies.
If they are wrong about the things they are claiming then they have decided to publicly antagonize a lot of powerful people who are betting heavily on going full steam ahead on AI and have no compunction whatsoever against retaliating against anyone who tries to get in their way.
Does that really seem a likely scenario to you?
>After three and a half years at OpenAI,
It's also worth considering that the author could have just "quiet quit", resting and vesting while also crying about how AI is literally the digital grim reaper.
The other issue is that the narrative about safety within these companies is largely a function of the extreme financial incentive.
As an example, Anthropic was an "ai safety" company that has now produced an AI that fails to listen to basic instructions. If you were concerned about safety, would you produce an AI that was unable to follow instructions? You ask a question, it begins executing commands and doing things.
Safety is product to sell to politicians, not consumers.
Not serious.
> Easy to suddenly find a moral compass when you, your kids and their kids never have to worry about working for money again.
If someone is in this situation, you can safely ignore their hand-wringing about “safety.”
Yes, it is easier to have a moral compass when you don’t have to worry about you and your children starving. But that doesn’t imply that moral compass is wrong or broken.
If it wasn’t clear, the coup should have solidified it.
Yet he stayed for 3 more years and vested his stock and improved the company and then spoke out.
I believe that is why most of the comments here are mocking him.
I mean you can be truly concerned and also think donating to AI safety doesnt work, or maybe just be a bit selfish. That doesnt make the concern less real. Its easy to read these articles as the author taking the moral high ground and writing it as some sort of way of proving to themselves theyre a good person, but isnt it just as likely that they think providing an inside perspective can do good by convincing people openAI is a bad actor? I think most of these AI insider accounts largely agree with you that theyre not the most upstanding citizens, does that mean we should write them off?
Highly suspect trends that can only make one believe it's marketing.
Even though it’s in model producer’s interest that these models do what you don’t want them to do - they want to engineer the model’s to behave in the interests of theirs.
I can’t believe people can’t see it lmao.