So Google has established a product called Search. For that product rules have been established. Google has monopolized that product.
Now Google is replacing that product with a new product. But they keep calling it the same thing. Because they want to keep their monopoly.
That is what has been deemed illegal. Gemini is not illegal. Pretending the worst version of Gemini is Search is illegal, because it breaks the rules established for Search.
But IANAL.
Three stars review is taken down for "libel": https://support.google.com/maps/thread/367778263/google-maps...
HNer gets a legal threat after saying that he didn't like a doctor: https://news.ycombinator.com/item?id=44734895
Google maps german policy: https://support.google.com/contributionpolicy/answer/1699727...
Meanwhile, service in Germany is still rather poor (especially if you have children), but at least no one can complain!
If Google started to show how many reviews were removed, it was because the lawfare started to seriously affect the ratings.
My guess is that often reviews are generalizing (easy mistake to make). E.g. they say "service is slow", when they should say: "When I was there on Thursday at noon service was slow for my table".
So maybe Americans should worry about their own business and not tell a country with a much longer history what is best for them?
As an European, this attitude is absolutely fucking shocking.
Police force their way into my home without excellent cause to do so? I'd want someone executed. It would be unforgivable.
It has nothing to do with the guns, but with the absolutely gross violation of the most basic rights codified by the ECHR.
>This would be more about privacy violations and unnecessary chicanery.
Which is actually worse than the typical American shooting. US cops do not typically go to the scene planning to murder someone, in this case the German cops very much knowingly went to harass someone for calling the German interior minister a prick. It is absolutely egregious premeditated harassment.
German minister of interior, the guy in charge of the police, sending the cops to harass people for calling him a prick. Even the current US admin is not so thin-skinned.
Idk what exactly happend. But somehow they got approval from court to collect evidence of the crime (electronic devices) via house search. As said, the harm caused by the house search was later deemed unproportional to the crime.
Who dictates the "should”?
The German law simply removes the need to assume anything.
According to[0] the (aptly named) Lawfare Institute:
Since the term “lawfare” is controversial in some circles, and subject to a
variety of interpretations and uses, a bit more explanation about our
understanding of the concept is in order. Going back to the 1950s, the term
has frequently been used in contexts wholly unrelated to national security,
ranging from divorce law to courtroom advocacy to colonialism to airfare for
lawyers.
When you see the pattern, best to stopped arguing with such users as their goal is not an honest debate, their goal is just to 'DDoS' you with their opinion.
You just have to link the review and they will send Google a legal document to delete it.
Make one big enough, and you’ll even have people trying to play it crooked by pretending to play it straight.
Its where the DRM bypass software lives these days, along with Bypass Paywalls Clean.
For the plebes, it is a civil tort.
For the AI companies, its required for "training". Rules dont apply to too-large-to-fail professional for-profit pirates.
But damn you if you try to make a Netflix-clone with every video. Wew.
I looked up how this works, and I found ads for law firms that specialize in removing bad reviews. They charge a set price for each review they remove. As a regular consumer, you just have to accept that your honest reviews will be removed, unless you're willing to risk going to court, where you'll have to prove that your subjective experience was accurate.
Anyway, Germany is probably one of the few places where this happens. The issue isn't necessarily that reviews can be challenged. The issue is that users aren't informed when they leave a review that they may later be required to provide proof of their visit.
I once left a negative review of a very popular touristy business in Germany after a genuinely terrible experience. I included photos and detailed information, yet they still challenged the review, claiming I had never been a customer. Google then required me to provide additional evidence to prove that I had actually visited the place.
What made it even more frustrating is that they challenged the review two years in a row. After the second challenge, I wrote to them that if they continued contesting the review, I would consider it harassment and pursue legal action. After that, they stopped.
What I find pretty shady is that most businesses seem to wait a year or two before contesting reviews. By that point, most people no longer have receipts, invoices, or other documentation. If they challenged reviews immediately, customers would be much more likely to still have that evidence available. In my case, I take photos frequently, so Google accepted my proof and kept the review online.
Ironically, after going through this process myself, I've come to believe that some form of verification should probably be standard worldwide. Requiring reviewers to provide evidence that they were actually customers could help reduce fake reviews. But if that's going to be the standard, it should be clearly communicated upfront, before people submit their reviews.
Another related issue I have with Google Maps is that, at least in my home country, some places have reviews disabled because Google considers them too prone to polarization or controversy. Schools are one example.
Personally, I think that's a terrible idea. I'd rather be able to read the reviews and make up my own mind. Instead, Google, in its infinite wisdom, decides that certain topics are too contentious for users to see feedback at all.
I find that to be one of the worst decisions made by the Google Maps team. Hiding reviews doesn't eliminate disagreement or bias, it just removes information that users could otherwise evaluate for themselves.
Truthful statement seem to be a defense for Germany, though. https://www.gesetze-im-internet.de/englisch_stgb/englisch_st...
> Section 192
> Insult despite proof of the truth
> Proof of the truth of the asserted or disseminated fact does not preclude punishment in accordance with section 185 if the insult results from the form of the assertion or dissemination or the circumstances under which it was made.
Making a claim of 3 star service, without providing 3 star service evidence, isn't any different than a claim like they don't clean the dishes enough. Sure, you didn't write out the exact words, but by submitting the review, you still submitted such a claim, and so it remains a question of if you provided evidence to prove it, which gets into how deep the legal system wants the evidence verified (is a picture enough, or do you need to provide evidence the picture is of the store at the date and time relevant to your review).
Is that defamation? That isn't a statement of fact, that's just relaying what you heard.
If I relay what the Daily Mail says, am I also guilty of defamation, or can I rely on what others say?
This post is written by an LLM, and possibly a complete fabrication.
You can even find many articles from specialized lawyers if you search in German: https://www.prinz.law/bewertungen/klage-zugestellt-wegen-neg...
But can confirm from personal experience that businesses threaten to sue based on Google Maps reviews.
https://www.thelocal.de/20260504/how-a-google-maps-update-ex...
https://rmx.news/article/germany-chancellor-merz-quietly-fil...
https://www.theguardian.com/politics/2016/may/19/boris-johns...
Google does not get routinely sued.
Yes, Lawyers send takedown requests to Google.
Google sends the reviewer a message and ask for a statement. If you provide one. They will check it and decide whether they seem it defamation or not.
However, Google doesn't verify shit. Even if you send them proof of a purchase or visit and your message is objectively a opinion (not defamation) they will take it down. Why? I assume because they really don't care about individual reviews and rather spare the time and money and just take the comment down.
Google changing their review displays is Google's decision. It has NOTHING to do with any threat of being sued.
They could actually leave the review up when it's not defamatory but they simply don't want to invest the time to validate it or risk leaving a potentially defamatory review online.
And the legal threat is without proof, so why would you even bring it up. OP could have simply posted the letter. They didn't wich makes this completely unprovable
(I just wrote an extensive analysis about that https://nickyreinert.de/2026/2026-05-28-google-maps-reviews/
It is relevant for Google though, because they want to transfer it to another product.
And the court is saying that whatever that new product is, Google is not allowed to mislead the public by pretending it is search.
In my view, the ruling could mean that Gemini's output is legally seen as first-person speech by Google regardless of where it is published.
To me it seems pretty reasonable that content Google generates with Gemini and publishes on their own google.com domain is their speech. Or at least they are fully editorially responsible.
When I wrote "Gemini is not illegal" it would have been more correct to say that the court has not decided about Gemini but specifically about search.
And I do think it makes sense to distinguish between explicitly chatting with an LLM and entering something in the search bar people have used for decades now.
I'm just wondering whether the logic of the ruling is tied to the specific context (search results) or applied more generally to LLM generated text.
The problem here is that
1) the AI Overview is giving incorrect information, and
2) it is Google's own words
...which makes Google liable for anything false there.
They weren't liable for false statements in search results, because that was 100% other people's content, being linked to, with a small blurb taken directly from their words. With the AI Overviews, regardless of whether it's getting its information from other people's content, it's remixing it based on Gemini's algorithms and synthesising statements that are frequently wrong.
If ai output is not copyrightable, it should not be considered personal output. So nobody should be responsible for it. Or if it is considered personal output, it should be copyrightable. Or perhaps the ai companies will be liable for all output, and they will therefore all cease to exist in any useful form? This seems like another alternative, where the output legal value is not central, but there will be a thousand different fights about how it is presented to others.
It's perfectly legally consistent to say that AI-generated content has no copyright (because it's the product of a computer, not a human), and also that the human or organization operating an AI is legally responsible for anything in its output that is legally actionable.
Someone needs to hold legal responsibility for any piece of content out there. You can't just wrap your decisions in AI and get to be free of all liability for it.
But copyright isn't like that. There's nothing lost to society by saying that content is not copyrightable, and particularly given how the major LLMs were trained, there's a lot lost to society by saying that they can take all of that from everyone without consent, and then everything it produces has copyright and can be used for, say, Google's profit in perpetuity.
So I think I like the current decision which is more about presentation, dissemination, application, and claims than content, and there should of course be liability for LLM creators if they are not actively dealing with results like CP, violence, or many other illegal or dangerous things.
search engine -> search terms -> search command
Google accepted the argument up to that point. Then they argued that they are not responsible for the search result. The court argued that what they did was not a search (or not protected the way a search is).
The argument whether Gemini, ChatGPT or Claude are search engines and whether a prompt is search terms is a different argument. Certainly an interesting one, but not what that court looked at.
It's not just about the output, the input matters also.
But if Google accurately summarizes the defamatory page, then the summary is defamatory?
The whole point is that these particular defamatory claims weren't made by some third-party, but hallucinated entirely by Google's own AI.
There is no "accurate summary". Just a pack of lies fabricated in-house at Google.
It seems that's exactly what they did.
It doesnt even seem to be what the article is saying. This looks like a section 230 sort of issue. Section 230 is a US law that protects platforms like facebook, google etc. being treated as publishers because the information, presumably, is just being passed through. But Germany is saying the AI results are authored by Google.
And to continue your thought, what does that imply about copyright of training data? If Google is authoring the output it seems harder to argue they are ripping someone else off.
It really seems like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
Not much really other than it being a change of category from pure search surfacing other's speech to their crummy new chatbot input with search underneath. GP was just mistaken that this was in any way related to their quasi-monopolistic position.
> If Google is authoring the output it seems harder to argue they are ripping someone else off.
They still pirated a lot of the training material, it's not like they went and licensed copies of all the books they used in the inputs. Even discounting all the publicly available data on the internet and the models recreating things word for word a lot of the books etc they ingested are illegal copies.
> like a tightrope to say google is publishing their own opinion but their opinion is also just someone else's work.
It's a grey area for sure between remixing and blatant copying that changes depending on the precise output. But it's inarguable imo that they consumed and ingested the work of basically every digitized word humans have ever written for their own profit without an ounce of compensation for the original authors. Copyright is full of these grey lines though like fair use doctrine, it's incredibly difficult to define in a systematic way what distinguishes transformative and non transformative works for example.
I am fine with that solution since I don't need to shill for them for some ie investment or employment reasons. Less technically skilled people (aka your parents or grandparents) are getting their lives fucked up left and right because they learned to trust search results, and now they suddenly can't.
Own. Your. Shit.
"Tatbestand
Die Verfügungsklägerinnen begehren von der Verfügungsbeklagten die Unterlassung von Darstellungen KI-generierter Antworten in einer Suchmaschine.
...
Die Verfügungsbeklagte betreibt eine Internet-Suchmaschine unter google.de in Deutschland. Dabei werden nach der Eingabe von Suchbegriffen durch eine die Suchmaschine benutzende Person auf den „Suchbefehl“ hin mithilfe bestimmter Algorithmen nach möglicher Relevanz sortierte Ergebnisse angezeigt. Zusätzlich bietet die Suchmaschine auch als unterstützende Funktion ein Suchergebnisformat an, bei dem mithilfe einer generativen künstlichen Intelligenz (im Folgenden:KI) repräsentative Ergebnisse zusammengefasst und angezeigt werden."
No "Suchmaschine" + no „Suchbefehl“ -> no "Tatbestand"
No "search engine" + no "search command" -> no "elements of a crime"
I think you have misapprehended the ruling. Of course running an LLM that spits out libel is also a crime outside of search engines. It's just that _this_ crime has those elements. If Meta's LLM defamed a business, then the ruling would instead talk about Whatsapp users and chats, or whatever.
I someone would put the libellous statement into Google Translate in English and the LLM spits out the German translation would that be a crime from Google?
My point is exactly that the court has not decided more than it has decided: If it is a search in a search engine it is a crime.
There isn't someone at a keyboard typing content. It's still just search, repeating quotes from other websites. The difference is that the algorithms have turned lossy, so the quotes are not always preserved in their original form.
i.e.
> Now that Google is manufacturing answers using their own LLM
To be honest, I find it somewhat rude that you select a portion of what I wrote and then criticise it.
The court found google to be liable for that, because company A has no other recourse otherwise. Which website operator was the company supposed to contact to remove this "false" information? It only exists in the aether of google search results, and only as a side effect of them adding functionality into that when someone is searching.
No, as the article lays out, google hallucinated facts that werent present in source material and so at that point the court found them liable for that.
Previous protections existed for search because it has been argued (and agreed) that they are merely a vessel for showing the content to the user. But when they begin to editorialize and reword the content to show their own version of the content (the AI summary) those protections dont apply even if the AI summary is shown in the same UI as search.
Same for self-driving. Your car is not self-driving until it accepts liability and you count as just a passenger.
But watch as Germany soon loses AI Google results.
But if you hold that position, you also have to be fine with companies not offering products and services in your country. AI systems will eventually be good enough (in 10-20 years) for companies to be able to deploy such systems with sufficient accuracy to afford the lawsuits. Until that time, such countries would just not have access to systems before they were bulletproof.
I doubt that will be the case, because of the long tail problem. (same with self driving cars and other ML related problems).
In fact, we have counter-examples today. Newspapers (even reputable ones) can't get it right every time, despite the fact that they have both trained people and in theory they're setup to catch that w/ reporters - fact checkers - editors. And still, from time to time, they get it wrong. (and I'm not talking about purposefully getting it wrong, just honest mistakes.)
What will likely happen with a ruling like this is that the answers will be hedged and legalesed and muddied up the wazoo.
This relies on me being able to find out if a newspaper lies about me, which is usually easy since we can all go buy the same newspaper. With AI it is much harder to find out that it has been telling potential customers wrong things about my business.
Ineffective mechanisms. So if we accept ineffective mechanisms as sufficient redress in those spheres, why not here too?
You will find references to this in stories where the government lets people get away with murder, because of course, that's dramatic for a story.
But when "bigger interests" (ie. the Chancellor's bank accounts) are at play, just to name one example, China gets to distribute lead-painted children's toys in Germany and doesn't have to accept liability. Russia gets to import sanctioned natural gas over illegally constructed pipelines. Etc.
As to how this goes within the EU, in the worst (but common) case, is as follows: the government chooses a company, and refuse to sue them. They have a tendency to choose the worst possible company, like using Palantir for policing Germans [1]. Then you find out most of the Chancellor's grandchildren are working there.
[1] https://www.dw.com/en/german-police-expands-use-of-palantir-...
This doesn’t sound convincing. What AI and what company?
LLMs don’t seem like they will ever be reliable like that.
Self driving like waymo might be?
True. But you never know if / when there will be a new big breakthrough in AI, which will probably be based on a new architecture / paradigm, i.e. it won't be LLM-based
I think it will even be solved soon, like, within the next 18 to 36 months. Hallucinations are the biggest problem consumers have with LLMs and a solution to that would be instantly worth billions of dollars. I’m sure every company in this space is desperately trying to figure it out before everyone else.
A non-deterministic system will always make mistakes, but we’ll hit a target where LLMs make fewer mistakes than humans and that will be good enough for almost all applications.
Exactly. "hallucinations" are not some special case. They are the LLM working as designed.
I don’t know the numbers but as user, it seems impossible for it to be useful without expert review. It is also debatable if it brings any value when you consider the cost of building and using LLMs and the time of expert. Also need to include the opportunity cost the expert is spending on reviewing slop instead of creating work themselves and the long term consequences of this on the expert himself
There will never be a time where there are zero hallucinations because of the non-determinist nature of LLMs, but eventually the frequency of hallucinations will be so low that it doesn’t matter. If the robot makes one mistake for every ten a human makes, that’s coming out ahead (depending on the nature of the mistake, of course).
Also I’m not making any value judgement about the technology and how it’s used and, frankly, I don’t really appreciate the assumption that I am. I’m fucking job hunting right now and it’s a hellscape thanks to LLMs.
I’m just being real about where the tech appears to be going based on its current trajectory and my experience in the industry. There seems to be a lot of cope going around that these things won’t ever be good enough to take our jobs. They will, and sooner than any of us are ready for. Leadership is fine with slop as long as it ships, the tech as it stands today doesn’t have to be much better to reach that standard.
Correct, most jurisdictions do not allow businesses which cannot be held liable for their actions. This is pretty core to a modern society.
Imagine if a company selling Knicks tickets was not expected to then actually provide said tickets and there was simply nothing you could do about it. Oopsies our sales page is for entertainment purposes only
To be fair, the internet has spent some 30 years figuring out how this works and it’s still not fully resolved. For the most part we’ve agreed that companies must follow the laws of both where they live and where they operate. This wasn’t always obvious!
Businesses are made to make it easier to share profits and responsibilities when trying to fulfill users wants. Laws are made to offer protections for consumers (because nobody has time for common sense), but at the end of the day the consumer has to take responsibility or no products can be made. If you're too fat for a chair, it's on you to find or make one that works- not every product is for every person. Laws only stop chairs being sold that are too dangerous for anyone.
How would that be the company's action? The only way the business might be liable is if they advertised their product as safe to use when lighting yourself on fire or if there was already some law that required them to warn customers not to light themselves on fire while using the product and they ignored that law.
In this case, it's not about what somebody else did. It's what Google did. There were already laws against lying about companies by saying they did illegal things when they didn't, google broke the law, so that's what google got in trouble for.
Consumer protection laws aren't there to replace common sense, they're there to prevent things like outright fraud and poisonings/murder.
Freedom as in freedom of private property can only be guaranteed by the State. The State watches over its own population and makes sure that private property, i.e. capital and work, is made productive, so people go to work, businesses make profits, and everybody pays taxes. Taxes are the State's prime source of income.
When all these million of private interests collide, which they are bound to do, the State provides a jurisdictional system that has to decide between those private interests and the State's own interests.
E.g.: If a business owner refuses access to medical patents or to lower prices and safe potentially people's lives, the State has to decide between that immediate interest and its own interests, which is protecting private capital, as its source of income. Since I'm in Germany: In the emission scandal Volkswagen didn't just physically harm people, VW violated the private property of millions of customers. Despite that, the German State sided with Volkswagen the larger capital and did nothing. During Corona, the German State refused to open patents for a limited time to help safe people's lives in poor countries. Doing so would've violated the interest of private capital, so it refused. In contrast, if I as an individual refuse to help somebody in an emergency, the State would either fine me or put me in jail. In this case, people's lives become the State's prime interests, because they are also the State's source of income, as a productive workforce.
Do you really want to go there? That everything in the world would have a literal "caveat emptor" attached to it?
As for not wanting to force companies to release only "safe-for-children products", I do actually agree. However I consider it to be a matter of degree, and in this case for example, I think that if nothing else, Google should say the very least make the disclaimer a bit more prominent and maybe tweak the model so that it's not quite as confident in its claims in the AI Overview.
That would be nice, but as every effort to restrict kids from using software which are not safe-for-children keeps getting condemned for being invasive surveilence, and every effort to stop kids getting the hardware instead gets condemned because of how much of society is now built on assumption everyone has a phone…
Something has to give.
Dunno what, but something.
And yes, it is ok to remove choice if the existence of that choice violates other person’s rights.
Google can continue offering that choice if they make sure nobody is defamed.
The “AI can make mistakes” kind of disclaimers they hide in the corner don’t really cut it
What sounds like a win to me. I certainly hope my country makes it dangerous for companies to break the law and/or harm the public with shitty products that aren't ready to be released legally/safely.
Well.. I mean.. yeah? I don't think this is as bad as you think it is.
Have you looked at SV and its product offerings recently? It's mostly just enshittified gamified value extraction that doesn't respect the user at all.
"If you do not let us do all this the way we want, we will take away your ability to use our shit" hits different when the "shit" in that sentence is actually just "shit".
Well, for cars anyway, the manufacturer was always liable for the car doing something wrong (example: driver changes the volume on the radio, and that disables the brakes).
It's just that techbros want an exception to this rule if the car is self-driving.
I see no reason for an exception to this rule.
for now only volvo accepts liability, and only for "slow crawl mode"
Let someone else sacrifice the safety of their populace.
Heck - self driving is the fastest way to authoritarian government in practice. I’m surprised more people on HN haven’t cottoned on to that fact.
A self driving system will naturally build networks to share road state.
This network will eventually shift over to the government having the ability to manage how traffic should move during emergencies.
And at that point the government can easily decide where your car should go.
The inevitability of this outcome is blindingly obvious.
It’s highly beneficial to let other nations experiment and simply be followers.
If Waymo can be proven negligent or something, then sure, bleed em dry. But as long as they're acting in good faith and significantly reducing overall road fatalities per mile driven, I think it's actually pretty unreasonable to try to hold them to such a high standard you end up subjecting society to more of the higher fatality rates caused by humans.
That would be a boon for Germany in my book. If you wanted AI results you could go use an AI.
It's fine to have Gemini as an option, it's also fine to have a combined result page, that should just be something people are able to chose if they want that (even persistently if they want). It should just not be the default.
When I search, I want to see search results. When I ask AI, I want to ask AI. Combining the two into one is a disaster.
Mercedes-Benz does this in limited cases. Waymo does it generally. (In China, Level 4 and 5 transfers risk to the manufacturer. This is the correct way to do it.)
I don't think Tesla is based in China
> AI can make mistakes, so double-check responses
If they want to claim ownership, then they will have to accept responsibility.
Self-driving cars don't need to be particularly good for companies to make models where they accept liability in some circumstances, and the cars refuse to drive in other circumstances.
I found one time Musk was using a few seconds of disengagement to insult a driver, but it still would have counted as an FSD crash by Tesla's statistics.
Time to set my VPN location to Germany. I'm tired of the "udm" trick.
https://www.google.com/search?udm=14&q=%s
in your address bar once?
As far as I can tell the ruling is more nuanced. If AI is defaming you, there needs to be a way to correct the record.
A company being open to liability does not mean it is always liable, just that it can be if it really messes up (especially if there are aggravating circumstances, e.g. you need to drag them to court to issue a correction).
I don't think so. It is easy to imagine the following (currently only fictional) scenario: the AGI does give perfectly correct answers (in a suitable sense), but some people in power consider these answers to be too dangerous, so they sue the company behind the AGI on terms of liability (i.e. the company is liable if the AGI gives answers that those in power don't like and which these people consider to be too dangerous for the public to know).
Or the system is very smart and recognizes the insanely deep "logical gaps" in the rules of the role that it is to serve.
Otherwise most of it would not even exist.
Everyone would have continued paying out the nose to the IBM’s of the world year after year (who had unusual willingness to sign short ambiguously worded custom contracts to their own disadvantage, if paid vast amounts of money).
And be on mainframes to this very day… maybe Y combinator and HN wouldnt even exist in that world.
A lot of people in IT seem to think law and contracts are in a sense mathematical. They aren't; they're more like a high school book report - to be interpreted, as objectively as possible, but definitely also establishing the intent behind the letters.
Particularly contracts - no, you can't trick your way into things in most cases. "Surprising" clauses are invalid in most legal systems, in particular if one party to the contract is a layperson.
Yes, except… very notably… the USA… (especially certain parts such as Delaware). Which has a judiciary that entertain such clauses much more often than pretty much any other country I’ve heard of.
A clause like: “you agree to deliver your first born son on demand” buried in page 15 would indeed be laughed out of the courtroom.
But less severe clauses buried in page 15 might not be immediately laughed out. Causing a lot of expense and heartache before it gets dismissed.
Just think about it, if it were really just a total joke… why would large firms (FAANG, etc.) with serious legal departments want to look like clowns producing gibberish, year after year?
I mean, it's not like most people have any kind of curriculum to fix those early assumptions.
The entire American tech industry has exported Americas predatory, parasitic, and unethical consumer laws (the majority of which are ghost written by the wealthy and corporate legal teams). When I studied law in school decades ago, tactics like bait-and-switch, false advertisting, intentionally misleading or deceptive practices etc to sell products or contracts were illegal across the developed world.
Those illegal, anti-consumer tactics were the SOP of every tech startup I can think of from the early 2000's onwards; following the same route of initially offering a compelling feature set to attract and entice users – usually for free – until securing a certain number of users or funding, then changing the value proposition to exploit that user base, and extract as much wealth from them as possible, ad infinitum.
Today these tactics are known as enshittification, and the average American pseudo-libertarian software engineer will say this is fine, but that's what every anti-consumer parasite and criminal has said in history. Lying, misleading, and exploiting people for financial gain is fundamentally immoral, corrupt, and sociopathic, therefore it should be illegal. Just because it's the norm, or a digital product, you wrote that in the T&C's, or your doing everything behind the liability shield of an LLC, doesn't change that.
What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism. I'm an athiest, but ya'll need jesus (the real brown socialist one). Many would argue Americas current institutional collapse is the natural result of this systemic corruption.
I wouldn't argue that America's moral standards haven't declined (significantly) but I also think it's a romanization to suggest that 1950s America was the pinnacle of morality.
Lying, misleading, and exploiting people for financial gain has been a part of the fabric of American society since the country was founded.
If we're being honest, humans everywhere have demonstrated a high capacity for this behavior since the dawn of civilization.
There is evidence of that being a common 50's perspective though. It was when most conservatives and liberals alike had been burned by the greed of the guilded age, stock market collapse, great depression, and world war. The majority of the working class in the developed world were experiencing significant gains in QoL/SoL thanks to labour movements and aggressive unionization, did not view CEO's as admirable heroes, or fellow consumers and workers with malice and contempt. Hard work actually resulted in financial security, and greater opportunity for your children.
The economy of the 1950s was due to a variety of factors, including lack of international competition (European and Japanese industrial bases were devastated in the war), pent up consumer savings from war time, demographics (the baby boom), etc. Unionization was certainly a part of the mix but it seems you're cherry-picking to support your romanticized view.
And let's not forget: the 1950s were a good economic time if you were a white man. They weren't nearly so great if you were black or a woman.
The distribution of work vs financial security was not uniform, but every developed economy had a greater distribution of wealth and stronger economic mobility in the 1950's than in the 1930's, or almost any time in human history. Every country not directly involved in cold/civil war improved in these factors from 1945 until the 80's, and that has continued in most of the developing world until today.
Do you believe black or indigenous populations were in a better position in 1930, or at any time during colonialism/imperialism, than they were in 1950? If you did, you would be wrong, and the only way to support it is by "cherry-picking" and "romanticization".
> What ever happened to the concept of building a valuable, quality product and stable returns for generations? Working to improve the quality of life and standard of living of the community? Of the world? I feel like a 1950's traditional conservative when I suggest that, but most Americans are so heavily indoctrinated with corporate greed and sociopathy they'd consider that sentiment radical leftist extremism.
The implication here was that, at some point in the not too distant past (like the 1950s), the US had less "corporate greed and sociopathy." And that the US was working in some benevolent manner to improve life for future generations and even the entire world.
I made the argument that the economic situation during this period of time was the result of factors like the war and demographic tailwinds, not some sense of morality that has been lost in recent years.
The US has been a violent, oppressive country since its founding. Violence, conquest and displacement, slavery, political assassinations, genocide, brutal individualism.
To suggest that the US of the 1950s, or any other period of time in the 20th century, was morally superior to the US of the 2020s because wealth was more evenly distributed (among working white men) and white men had more economic mobility requires in my opinion a significant romanization about what US society is and was.
The US might have been founded on the highest of ideals but the actual history is very much a story of "actions versus words".
But watch as Germany doesn't really mind losing blatant fabrication that mainly benefits Google.
Vendors keep ignoring the obvious --- that AI is a liability issue waiting to happen as evidence of it just keeps coming.
Otherwise would involve a fundamental overhaul of legal precedent to make lying acceptable.
Can we just trash this as a marketing term? If/when AGI arrives there will be no point quibbling over competency. What we are looking at is just bad search results
[0] e.g. Zuckerberg: https://www.meta.com/superintelligence/
I don't think this is the case. It's not clear intelligence is a coherent concept to begin with, let alone a one-dimensional thing you can maximize. What, is it going to write stageplays that puts shakespeare to shame? Find a way to enforce world peace or end greed? I'm not at all convinced we anything more than the simple competency we expect from humans is possible. What would we look for to know it's there?
If it did, who would say "nope, still not super-intelligent"?
(I'm sure it's more than "nobody": the least realistic thing to me about my brother's LARP group was that nobody in that universe denied the gods it contained even though they'd immediately smite anyone who did that).
I'd be very happy to get better plays. I don't think we'll see this either, as this implies spending a lot on something with no return.
> (I'm sure it's more than "nobody": the least realistic thing to me about my brother's LARP group was that nobody in that universe denied the gods it contained even though they'd immediately smite anyone who did that).
That's sort of my point: judgement across history is inherently a messy process that entails consensus, which is anathema to a capital-driven world. Even the idea that there is a correct way to run society is very western and wouldn't pass earnest scrutiny. I strongly suspect that any sort of "super-intelligence" won't be useful to capital as it won't actually solve capital's problems—instead, it will reveal the contradictions inherent in a capitalist/western society and point strongly towards capital as the cancer that is crippling our planet. Hence, I don't think we will ever see it.
Liability isn't being *created* here, it has existed legally for a very long time.
False information can cause real harm --- and the legal burden of proof is on the source.
Search engines were provided legal exemption on the basis that they were simply quoting/referencing 3rd party sources who where legally liable for the content.
LLM chatbots legally exceed these bounds by fabricating info/content on their own ---data that does not exist elsewhere. This is a liability issue waiting to happen as there is no other responsible party/source to blame.
https://blog.platinumids.com/blog/ai-hallucination-crisis-co...
It's trivial to use a search engine instead of AI. But people still use AI to save the time needed to read and check.
But in any case, none of this absolves slander. The legal burden of proof is still on the source --- which is AI.
e.g. The failed marketing of A&W's third-pounder burger as so many people (in the USA) didn't believe that it was bigger than a quarter-pounder.
down vote all you want, but I firmly believe this is an example where the user needs to use some judgement on the information they receive and have some critical thinking skills. google would be right to remove all AI results from germany.
Now, search engines are usually afforded some amount of protection against defamation claims — they’re not held liable for simply indexing and quoting third party defamatory claims. Which is to say: Google wouldn’t be liable for claiming you’re a poopoo head if this comment shows up in search results.
The point of this ruling is that AI-generated text isn’t a quote from a third party, it’s text generated by Google’s own tools, so it’s speech by Google itself. It might be wrong, sure, but it’s still presented as a statement of fact.
At trial they can have the whole debate about whether Google was negligent in how they build their systems, and all that jazz, but let’s be clear here — it’s not a matter of every little factual mistake getting Google sued (and that would be absolutely terrifying from a freedom of speech perspective), but rather that the technical means by which you generate content doesn’t change your liability in publishing that content.
It is not just some people, it is a lot of people insode the tech itself pushing the "trustworthy" claim.
Put differently: it’s not newspaper readers complaining the paper is inaccurate, it’s the people mentioned in the articles.
consider Purdue pharma - the Sacklers got off with all their wealth intact because they were too big to sue and properly collect money for their victims.
They're also almost universally regarded as having committed evil acts at this point, so who cares why they got away with it?
The problem for LLMs is that they do not learn, and can't be prevented to produce that libel ever again. If Google finds a way to make that happen, no court would stop them from offering an LLM.
how?
errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
google would be right to remove all AI results from germany
i'd consider that a win.
If it’s important, check it. If it’s not important, then it is pretty much just entertainment.
LLMs can be very useful in a general web search and save some time, but if you don’t put those literacy & critical thinking skills to the test and actually confirm anything, then you might as well not even have bothered with the search at all unless you’re hoping it can just replace all of your original thinking too.
Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
> Obviously the marketing point of the AI tools is it just gives you an answer straight up so you don't have to bother reading normal sources.
To lazy people yes. That would be a marketing point. It’s not that though, so you use it to save time, but you don’t get to skip the verification step.
A journalist could not make up a harmful statement about someone and get away by saying the readers should have all read the sources. AI companies want to take all the benefits and profits, while holding none of the liability and responsibility for the harms they are causing.
Journalists do that all the time. We even have a whole collection of words to describe it. Muckraking for instance is probably about 100 years old. Its even in the Google auto-complete in the browser I am posting from.
LLMs are, for all intents and purposes, the equivalent of outsourced workers.
Google created a summary, not just sharing search results.
Google is responsible for the output it created and then published.
If they had only surfaced search results, then they would not be liable for what other people generated.
Google’s scale does not protect it from this liability.
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
Because if someone goes through the citations and it doesn’t substantiate what was generated, then what was generated was obviously bollocks. Being able to recognize those contradictions is an essential skill to using LLMs with web search at all. It’s not rocket science.
My eyes are brown.
I dislike coffee.
My phone is on the desk next to me right now.
One of these is false, and the other two are true. Can you recognise which is which? Or do you not have this "essential skill"?
When you're being given information about a topic you don't already know about, there's no skill to be able to recognise which pieces of that information are correct and which aren't. Either you know the information already, or you don't.
it's like saying people should not use a computer if they don't know how to keep it secure.
you have to look at the reality, which is that people are not educated or critical enough to use AI safely. and that misinformation can cause people to get killed.
just yesterday there was a post about a man being falsely accused of a crime by AI tools and losing his job and his family as a consequence.
these mistakes destroy lives, and most people are far to trusting to use AI tools safely
Or you don’t care about those things at all, and you will buy the T-Shirt that’s in front of you right now rather than wait later and buy one that better reflects your supposed values when you’ve done an appropriate amount of research. Using AI may even reduce the amount of time you spend on that part.
Your T-Shirt buying patterns & values are not my concern though.
Businesses might (well, will) suffer because people are misusing AI, but it is a misuse to do anything with it without an additional verification step.
To be clear here, I have no issue with Google taking it on the chin in cases like this, but what the comment I was originally responding to had this:
> errors can be so subtle that it is not possible to recognize them unless you spend an hour researching every fact presented. at that point, what's the benefit of AI? nobody is going to do that.
And my point is this: if it matters, verification is not optional. If it doesn’t matter, then fine, skip the verification step, but if you’re taking whatever text is generated by a GPT at face value without understanding what that is or being able to determine the source inputs for the “claims” it outputs, then you’re part of the problem because sometimes the source is just a GPT-generated web page, and that’s obviously not trustworthy. Sometimes it’s a MediaWiki site page that doesn’t actually exist, but because it’s MediaWiki it’s not going to return a 404. Using a tool requires understanding it including its failure modes, and in the case of LLMs that means: trust nothing, verify everything.
If a Google employee (like a support agent) says a mistruth, the company is liable and you can sue. They can’t just say “hihi oopsies our support agents are useless”
It's entirely possible for legislatures to attach clauses to restrictions that make them only apply to businesses above 50 employees, or above $1M ARR, or whatever.
I, too, agree that google is too large and should be broken up into multiple smaller companies.
(Sarcasm to support your argument)
But until then, be a good citizen?
What? That's fucking feudalism... Peasants and Lords.
If you're lucky enough, you're born as a Lord. (And maybe don't live during a revolution)
This makes no sense to me at all. If you're small you should get less bureaucracy than if you're bigger.
For e.g. self driving cars there should not be any exemptions. There are people's lives at stake, people who didn't sign up for your shitty service.
As others already said, that would be a great outcome, and german citizens would benefit.
It is a search engine. It used to have decent excerpts. It doesn't need hallucinated generative AI summaries. I am about to click the link anyway.
So Google could, for example, switch from a tiny "this could be wrong!" byline to having the AI be less overconfident every freaking time regardless of whether it's spouting made up crap or actual facts.
The scale doesn't sound like a way out. If your company expects to get away with doing the wrong thing where smaller companies can't, then the solution isn't to continue getting away with it.
As a society we decide. Are we embracing all users, are there basic rights and assunptions? Do we only enable some?
As a free (as in cost to end user) system, Germany is arguing that their social compact raises the mininum bar. Frankly, thus might help drive a rush to increased accuracy for AI- tech finds a way. Equally it may hinder - beaurocracy creates barriers.
I'd love to be able to rely on these search results. I see them ad the same prior set of inaccuracies whereby I have to do more research. At least now there's a summary and direct links to the supporting information. But equally, we're primed with the information in the summary.
It doesn't help that there have been active efforts for decades to prevent people from learning and developing critical thinking skills.
We oppose the teaching of Higher Order Thinking Skills (HOTS) (values clarification), critical thinking skills and similar programs that are simply a relabeling of Outcome-Based Education (OBE) (mastery learning) which focus on behavior modification and have the purpose of challenging the student’s fixed beliefs and undermining parental authority. - The 2012 Texas Republican Party Platform (saying the quiet part out loud)
This very much depends on where you live between state (US), and countries. Where I live, it's the complete reverse, critical thinking is baked into the population, into learning, into nigh everything. Our challenge is the complete lack of privacy, sadly.
The law they broke was a law protecting personal and business reputation against false statements of fact. Essentially no one can say I might be wrong, check yourself, but X is Y if that claim is essentially defamatory.
This is pretty good, I hope googles approach is to make sure they don't end up making statements of fact like they did and use more appropriate wording like according to X.... with direct disclaimer that they can't verify it. Even better that they look court documents to find any legal ruling and point people to that too.
In Germany, you also have the Oberlandesgericht and the Bundesgerichtshof as a higher venue for civil cases (and the Constitutional Court as well). So, this is not like "the law has changed" at all. (I'm not a German qualified lawyer, but qualified in a close enough continental system, and on my way to be qualified in a common law jurisdiction soon, that's why I dare to comment on this aspect)
But if you as a first party are publishing something directly, like Google is here, you're generally liable for what it says.
https://www.startribune.com/google-ai-overview-lawsuit-defam...
Previous cases in this space (eg Meta v KGM, Walters v OpenAI) have not turned on Section 230 specifically.
Also I am not entirely certain your statement is correct - which false claim does the article make exactly?
Next do Amazon that is selling AI generated foraging books: - https://www.theguardian.com/technology/2023/sep/01/mushroom-...
When I was a kid it was possible to buy any foraging books from a store and they had a minimum quality. Is that so difficult to achieve? Is profiteering not punished anymore?
Well, they disclaimed and the user acknowledged
Source: https://www.youtube.com/watch?v=P-QaEB5eXSU Minute 3:00
In fact, in most EU countries "the user acknowledged" works only for a very small subset of stuff, precisely because our lawmakers know that the strong party in a contract would use that to get away from every legal obligation.
Anyone can write a disclaimer, that does not make it enforceable.
And what institution gives out the licenses for journalists and scientists? Is it revokable?
Do we sue scientists for unintentional procedural errors?
This is about what Google knows.
And Google knows perfectly well that LLMs hallucinate all the time. They will provide incorrect information, confidently and often.
Which is why this whole article is explicitly about holding Google accountable.
Google 100% has the choice not to include AI overviews. Which they know are often false.
It would mean egg on their face, to be sure, but would it even meaningfully impact revenue?
The legal profession uses the term "reckless disregard", see, e.g., [1].
[1] https://en.wikipedia.org/wiki/New_York_Times_Co._v._Sullivan
Or selectively. "Vaccines cause autism!" "Okay" "Vaccines don't cause autism!" "Prove it beyond a reasonable doubt or we'll arrest you for lying"
If we can solve these problems, then yes?
A return to the classical understanding of the person, society, and the common good is indispensable.
There's three basic paths for a company hit by this ruling to comply:
1. Stop showing users generated content.
2. Figure out how to generate the content with more quotes and attribution to source websites, to regain the protection offered to search engines.
3. Figure out the hallucination problem, so that every statement in machine generated content is true, or at least defensible.
Rather, they are stuck unable to do that much better, unwilling to admit (especially in a way that might spook shareholders) that it's a hallucination-machine all the way down. They're playing for time and market-share while hoping some unspecified and inherently-unpredictable new discovery arrives which will be compatible with their existing infrastructure and investments.
The thing is, no one has the slightest idea how to stop hallucinations.
The models are fundamentally "hallucinatory" at core - they generate what is _probable to follow the string thus far in its training corpus_, modulo RLHF and friends.
Notice that nothing there has any rigorous relationship to truth.
Sure, the companies could start pumping money into pure research on what models other than transformers might yield something that can reason rigorously, but at that point you're talking about finding a way to throw out LLMs entirely in favor of a less-pathologically-broken model, like Gary Marcus keeps complaining people should be doing.
At this point, there is nothing to gain but volatility. Let the US figure out the economic and social disruption, and then adopt whatever sticks. There is no rush. The useful parts got little moat. It's not like there is a magnitude of difference to justify the expenses and risks of frontier operations, in comparison to open models, or smaller players. Chances are, all of this AI business will settle on local models for most use cases. The US may get their first trillionaire, but let's be real, that's not because of innovation but corruption, exploitation and raging wealth inequality. If the AI economy is not panning out as "projected", there is no recovery. That money got converted to heat and single-use hardware trash. Not worth risking pensions and the social fabric over "the tech edge".
Or what, do you think it's a genuinely good thing that hosting negative reviews is essentially illegal in Germany?
It’s super easy to catch on dates and numbers, but it gets other details wrong all the time too. But so many people won’t be double checking the results.
Even those results have a lot to be desired, it is just buried deeper in the insanely verbose research report and impressive looking amount of sources you see move past.
I recently have had a close look at the various "deep research" options the big three (Anthropic, OpenAI and Google) offer. None of them are exactly transparent about how they perform searching other than the "research plan" the present upfront the and shitload of sources they show you (which, to be frank, seems to be clever UX/marketing to make it look extra legitimate and impressive). Which is already a worrying sign to me, as you can't audit the process itself properly. But even with the lack of information available on the front-end I can still see enough that worries me. A few examples:
- "Sources" are taken at face value almost no critical look at the validity of the source, the context it is placed in, etc.
- A lot of sources I know are legitimate are rarely included while a lot of listicles, low effort "reviews", etc do make the cut.
- In multiple instances when looking closer at the research plan and the "hints" they show during searching it becomes painfully clear that often enough they start with an answer in mind based on training data and try to validate that rather than actually researching the data itself.
- Subtly different prompts that by all means should still produce the same factual outcome actually provide wildly different results. This one probably relates to the other points.
In addition to all of this, I also am 100% convinced that AI powered search is incredibly expensive[1], more so than traditional search. In my mind this increased cost eventually will need to be paid by someone, which likely is going to be the user. Since the process is non-transparant I am not confident that the results will not end up being polluted by sponsored deals, etc. There is simply no way in my mind that this is going to end up well for us users.
[1] A while ago I have experimented with creating my own deep research flow with the idea that I might be able to do something with local models. To limit costs I used a SearXNG instance for searching, setup playwright for browsing sources. Using an agentic flow with agents making all the various calls and dispatching other agents ended up eating A LOT of tokens. Even when I did switch to a non agentic flow where each step is orchestrated by code calling on LLMs with simple prompts to validate results still ate a metric ton of tokens for the simplest search query. Mind you, this was not even doing actual deep research but only a few simple search queries. Ironically, google models also did seem to have more trouble coming up with good search queries compared to other models.
"rewire brains for AI dependency" (for money and power reasons obvious to everyone).
In contrast to "secretly implant an agenda about non-AI subject N", which is complicated enough that AI companies are still too out of control to be attempting yet.
When Google Search is quoting a 3rd-party website that happens to have bad information, that's not on them. Blame shifts to the 3rd-party. This is Google's privilege being a search engine.
When Google operates as an answer machine instead of a search engine, the privelege doesn't apply anymore. There's no 3rd-party to take the blame.
> According to the court, the Al mixed up information about other, genuinely sketchy companies with the plaintiffs and drew connections that didn't appear in any of the linked sources.
Implying: if the false claim was found in the sources then it would be protected speech.
There is nothing special about "answer machine" versus "search engine". You are making that part up.
Guess that's the end of their AI overviews in the EU!
Overregulation, at best, is a good way to guarantee that your country won't have access to interesting and useful features and technologies. At worst, it's a good way to guarantee that the twenty-first century will belong to the US, if not to China.
I can make mistakes. It's on you to fact-check my claims.
Do you think these are harmless statements? Does the disclaimer suffice? If I was Google's AI Overview, do you think 100% of people will check those sources?
There is nuance here, and it's not going away because AI and innovation.
Yes, I do. An assertion made without evidence can be dismissed without evidence, whether it comes from Google or from you.
If you demand perfection, you will receive nothing. Why is that so hard for people to understand? The world simply cannot work the way you say it should.
A Danish newspaper at one point falsely claimed that a named person had raped a child, with no evidence. The only way that person escaped public judgement was by going into full attack mode and publicly suing and attacking the newspaper and the reporter. You have to be an incredibly strong and resourceful person to do that and not just go into hiding. Also one thing is suing a Danish newspaper, imagine the legal team, the resources, you face if you're trying to take Google to court for defamation.
Also in a similar vein Apple pulled a feature real fast last year when it was doing something similar to BBC news summaries -
https://www.macrumors.com/2025/01/06/bbc-calls-out-apple-ai-...
It can destroy you. You are analyzing this on the assumption that people are rational and have capacity to do the check. They have neither.
Anyone who does accept Google's AI output blindly will soon find that their mistaken opinion of CamperBob2 is the least of their problems. There is a reason Google goes (well) out of their way to warn people that the results may be wrong. That should be sufficient warning for reasonable people of good faith.
You have no idea what it can mean to end up in such a situation.
No one with a straight face would compare an AI overview saying someone is an asshole to the New York Times running a story saying someone is an asshole.
In the world according to you, AI couldn't exist. Or more likely, it would be accessible only to academic and corporate/financial/government/military elites. That's not OK, and I'm unwilling to join you in pretending that it is.
Correct? Or am I misinterpreting your post?
No one is, until someone is. You could say the same about newspapers, TV, or the internet.
> If they do, that's not Google's problem.
Well yes, it is, at least in Germany for now, where at least one judge seems to see it that way. And that's correct, IMHO, because Google would (in that case) make something appear as factual information when it is not.
> It's someone else's fault, principally those who blindly acted on information they were told was potentially incorrect.
Those people are definitely at fault; history is full of examples of mobs acting horribly based on shallow, incomplete, misleading, flawed, or flat out wrong information. But you can't just put all blame on those mobs, when someone gave them this information - or do you think, for example, Donald Trump played no part in the January 6 riots and should not be held accountable for it?
> In the world according to you, AI couldn't exist. Or more likely, it would be accessible only to academic and corporate/financial/government/military elites. That's not OK, and I'm unwilling to join you in pretending that it is.
That isn't my stance at all. AI should exist, and it should be widely available in my opinion. I don't even think this is primarily about AI, but entities in a position of power acting responsibly, such as Google (which has worked hard to position itself as a gateway to information and steward of facts.)
I think we should not put the burden of verifying information entirely on consumers, we shouldn't allow big corporations to run the largest social experiment in history on their own terms, and we need to talk about responsibility and safety.
> So your position is that the general public should be given access to AI only when it is either capable of flawless accuracy, or when the AI provider is prepared to assume unbounded liability despite warning the user that perfect accuracy is not possible.
That is neither my, nor the sibling's, nor this ruling's position. The issue the court took was that Google presented AI-generated information in a shape that was indistinguishable to common users from the previously statistically sourced factoids in Google Search - a product that historically allowed to search the internet for things and get back search results, not AI-provided guesses and hopefully-correct information.
The entire point is that companies should not be allowed to use AI recklessly. This point is also one that the Pope made in his encyclical, and the EU posits in the AI act, by the way.
I'm missing the comparison here. Trump (in)famously did not post any disclaimers. He did not leave any room for doubt when he accused the Democrats of stealing the election. He spread feces and called it fact, so yes, he should certainly have been held accountable for the consequences. His voters, however, decided not to do so, and that was the end of it.
The issue the court took was that Google presented AI-generated information in a shape that was indistinguishable to common users from the previously statistically sourced factoids in Google Search - a product that historically allowed to search the internet for things and get back search results, not AI-provided guesses and hopefully-correct information.
Did they, or did they not, include a highly-visible disclaimer that the results might be incorrect?
I'm hammering on this, not just because I think it's what should have determined the outcome of the case, but also because the last few times I've seen Google AI summaries, they did not include such a disclaimer. Gemini itself still does, but the instant results on the search page don't appear to. Which is obviously not OK.
If they have stopped warning users, or if they didn't warn the users in the situation leading up to the lawsuit, then that more-or-less instantly flips me over to your side. We wouldn't actually have anything to argue about in that case.
It is obvious that the defamed companies are the ones having a problem, not the ignorant viewers.
Why should those companies not hold Google liable for that outcome?
Well, good luck with your... um... innovative HR strategy.
The point is, if you take Google AI summaries seriously, that's a "you" problem, not a "me" problem. I have a rather generic name, so it's safe to say it shows up in some pretty foul contexts. Life goes on.
What profit? I don't know either but they enabled this for a reason right?
This is the third iteration of the same concept, after AMP and Instant Answers, but somehow with even less of a pushback than with the previous ones.
The second these things came on the scene and confidently spoke lies at times, I knew this sort of lawsuit would be inevitable. Nice to see Germany got it right.
I very much hope we don't see attempts to re-write T&C to avoid this liability.
We really should have triple fines or worse for people who try to push the boundaries of the law.
If you just display third party content, that's one thing, but if you generate content yourself and those content are false, harmful, that's on you. Google should be shamed to do this kind things recklessly and irresponsibly.
I'm thrilled that companies are liable for crap that ends up hurting other people. I don't think they should get an easy way out, and I also like that there's a carve out for people who aren't making money off of software (like OSS devs.)
But I do think that this is a much better start than letting companies ignore the impact to software consumers or having open source devs be on the hook for volunteer work.
All three have the ability to perform a web search, then compose a reply based on the search results. Pretty much the exact thing that Google AI Overview does. This ruling may make them liable for false answers.
No, the article implies the court’s logic is that the AI search results are presented as search results and that’s a big part of why they are liable. It seems like the court (again, according to the article) does not find the disclaimers that Google has slapped on the AI results compelling because again, it chose to represent these as a summary of search results and it is aware of the failure rate.
> The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."
> Google built the AI, Google offered it to users, so Google owns what it produces, "because it alone has influence over the AI's offering and the algorithms with which the AI operates."
Google does not, as a general rule, control the actual content of search results, but usually there’s a distinction between the ranking and presentation of the results vs. the actual content. In this case, the court is basically saying, “You sold this to people as a search summary, you know it might be full of crap and you chose to do it anyway. No, you don’t get to claim the equivalent of a US safe harbor defense.”
There is a subtle difference in stating it as a search summary compared to an opinionated answer. Most users are always going to treat it as a response from google instead of search results where the user is still responsible for understanding and come up with their own interpretation.
This is probably the right step in some sense to make one liable for their statements/assertions.
Tool user is liable in the case of misuse unintended purpose of the tool.
Tool provider is liable when the use of the tool, by design, causes unintended effects despite proper use of the tool.
A simple "AI may make mistakes" line under the box will not help while the box contains false information. The specific information (lines or words) should not be provided if that's a mistake of false.
Another point from the article: they are not just aggregating content, but generating it. If you generate falsehoods, that are not even stated by your sources, of course you're responsible.
This can have significant impact to AI in Germany and the rest of Europe, but it's good to question it and hold people accountable.
You're not even paying for a google service, search is free... You might be the product, and your data, but you didn't directly pay for a service and they didn't sell you a fake service.
I'm not taking Google's side, this isn't about whether it's right or wrong to rob websites of traffic, this is about AI's returning search metadata.
But I'm surprised that they lost this argument, and the line they took in the first place.
The Internet isn't made of fact checked data, it's crowd sourced. How can anyone be liable?
If instead Google gives you an answer right there on google.com, without going to another site, they ARE responsible for it.
That makes sense to me?
(What's the alternative, after all? Having no one responsible for what the AI summary says is clearly untenable.)
Probably not, because it's a similar situation where Wikipedia is accumulating user provided content. And people know Wikipedia can be freely edited.
You, however, might be liable. It's your content.
https://en.wikipedia.org/wiki/Asian_News_International_v._Wi...
Directly? Quite possibly. They'd then have to transfer that liability to you.
Yes, and it's called defamation when you don't follow it.
This is why you have to say "I think this person is a murderer" and not "This person is a murderer."
One is opinion. One is fact.
This isn't super hard.
None of "AI" companies call their apps "Entertainment fun text generator". They are call them serious names, use words like "intrllegence" and "thinking".
So yeah I'd think if any of "AIs" start to recommend to drink some bleach or take a flight from a 10th floor window these companies should be liable.
I was prepared to say the same thing as you but after reading it seems totally fair.
The key difference is that this would be illegal if a human wrote it too.
This is a gigantic own goal for Google. The average person’s impression is that Google AI is much worse than ChatGPT, even though that’s not actually the case. Google is shoving a terrible model in everyone’s faces.
After all, if I can get ChatGPT or Claude to say something false that should count too, right?
The arguments of the ruling should generally apply even when the AI agent makes false statements it wasn't notified about. But in that case the defendant might have a stronger claim about not being able to reasonably ensure the correctness of all statements, and having taken reasonable measures to ensure correctness. Google couldn't really claim any of that after ignoring cease and desist letters about the false claims
Imagine a search for your name resulted in an AI summary saying you are involved with child-trafficking because low capability model linked your first name and perhaps a couple articles on supporting children non-profits to it — and then offering that in a convincing sounding summary right at the top!
If you knew this was all possible and you did it anyways for personal gain then you are additionally negligent which may add aggravation to your charges.
Someone needs to hold the liability at the end of the day. People are experiencing real harm from false claims LLMs are spitting out.
A disclaimer and couched language will probably fly through. And it's going to matter what expectations an user could reasonably have, too.
I first tried it to remind me of what happened in a previous book in a series that I was reading. When I realized it was either misstating plot points or straight up hallucinating, I tried it on a bunch more books to amuse myself.
Older classics are of course more accurate, but for newer or less popular books Gemini won't shy away from giving you a summary culled from misinterpreted Reddit threads and Goodreads reviews. It's like getting a secondhand account from someone who talked to another person who had read the book a long time ago. You get the general gist of it, but with some added flavor.
Even if you upload an entire epub of a book, the results aren't stellar. Rather than a Cliffs Note's quality summary, they're pretty sparse or leave out important bits of information. One chapter summary I got back made a point of describing what one of the characters was wearing, even though it had absolutely zero to do with anything else. Yes, that's technically a "summary", but not quite my tempo.
If Google wants to present summaries of websites in anything more than a very, very superficial description, they're going to have to improve their model's ability to understand context and importance. In theory, a novel is a self-contained bundle of text, so pulling accurate information out of it should be straight forward. A website is naturally going to be way more of a challenge.
All that said, I find the AI summaries from Google/Gemini to be quite useful and a time saver, but I know to always double check something if it's at all important.
So the only thing happening here is they consider Google to be the author of the AI answers and apply a similar sort of anti-defamation law. Perhaps it's a good consequence but the law that allows for this seems to be quite broken.
[0] https://www.settle-in-berlin.com/google-reviews-removal-defa...
Either:
- Google stole the entirety of the world's copyrighted material and it's using it illegally
- Or, AI learned, in which case what is writing is the AI's own saying, and since it comes from Google, they are liable for it
Then in order to increase revenue Google dumbed down search very badly and made it very hard to find obscure information. Whoever decided to delete random search terms is someone I want to punch in the face really hard. Call this Google Search 2.0
But the way Google has integrated Gemini Flash into search is actually pretty good and is definitely an improvement over Google Search 2.0
The ruling also lessened the free speech rights for AI. This is a big one. In conjunction with holding an operator liable for its AI's doings, that will lead to interesting cases where conduct that would have been previously protected under free speech rights will become a liability. Basically, machine-based cognitive capability becomes a liability when it is customer-facing.
there are no indications it is a scam, but "significant organizational problems and extremely bad customer support lead to (list of bad experiences)".
It's hard to see if the individual complaints really support a general problem, or if this simply the only result that talks about "scam + business-name". Probably, the latter.
The same problem happens on google search, if you look "<obscure false fact>", you'll get pages mentioning that false fact. If you fall into the trap of confirmation bias, it leads you to think the false fact is in true.
Is something like,
"People online say that x y and z because a b c"
a credible, correct answer, even if it isn't because of a/b/c?
The instance of this ruling people apparently did not actually say any of the offending claims. 'The court also found that the AI overview made claims "that are not even made in the search results." None of the linked sources drew any connection between the plaintiffs and the shady companies the AI mentioned. The court called these "the defendant's own statements."'
One way to formulate things that would be less would be "once support a time, in some fabulated world, it's not impossible that some imaginary character would say something following some reason." But then, of course this is not aligning the the deception scheme pushed by companies putting in their interface that the "machine is thinking hard for you".
Probably not, for the same reason search results aren't an issue.
Even if the answers are correct, they could still be biased, incomplete, misleading, and all the other media-literacy things people should be looking out for.
This ruling seems to go the opposite direction; 'I am legally obliged to give correct answers, so I am always right, trust the AI'.
If you sell food, in a food stall, labeled as food and you add a disclaimer that it is toxic and will make you sick. You are still selling toxic food and you are liable for it.
Google is pretending to give answers to your questions. They offer you a service about answering questions. And then they add a disclaimer "we do not answer questions just write bullshit". That is still fraud and Google should be liable for it.
> isn't this just a soft ban on the deployment of non-deterministic software?
Tetris is non-deterministic and it is not banned like millions of other programs. I do not follow you.
They are also no longer covered by safe harbour provisions because it is them answering it, not some content they refer you to.
If your software deterministically produces incorrect output for some inputs, you’re liable the same as if it did it non-deterministically.
Honestly I can understand the ruling, but the side effects might be severe.
that trust level obviously varies, but for example if the page found is from wikipedia i know that the content has been reviewed by multiple people, so i can trust that more than say a blog post on the same topic.
on the other hand, AI results are not certain to be an accurate reflection of that wikipedia content, and therefore it is less useful than the page itself.
practically speaking i don't even look at the results in detail. i just check if my search keywords match and then i open the pages and read them myself.
the AI summary does not tell me which keywords on the pages matched. it is therefore completely useless.
So my understanding is their unreliable AI summaries are a legal liability for Google in Germany and people can request corrections through the courts.
Every AI model can make something up sometimes. Over millions of daily calls, it's essentially impossible for the technology to be guaranteed correct 100% of the time.
Genuinely, by far the most common situation where claims of defamation arise in Germany is any less than 5 star review left to a business on google maps.
It's not like this exclusively affects restaurant reviews, every corner of German society is subject to this same evil censorship mechanism.
Now if that information is BS and cannot be relied upon, that’s really bad. Leading people on and not delivering? Honestly, Google themselves should have been on it and not the German government. It’s a bad look for them.
Aside: I’ve noticed their AI mode is pretty pathetic for troubleshooting something. 50% of the times the first response is riddled with inaccuracies and mistakes. Repeat prompting is absolutely necessary (so do not expect to one shot anything).
Also I will admit that I still find myself using it because I’m lazy, and it’s easier to talk to AI to get the right answer. Searching organically is hard these days with the volume of content having gone up exponentially.
Because of the same rules, German restaurants also get to pick and choose which reviews stay up. They can literally take down any specific reviews they like.
A restaurant that mostly gets 1 star reviews will still show up with 5 stars on Google maps, as they will simply delete the reviews with less than 5 stars as defamatory.
Here's a couple of examples:
https://www.reddit.com/r/germany/comments/1peujau/google_rev...
https://www.reddit.com/r/germany/comments/18z4shs/legal_thre...
https://www.reddit.com/r/frankfurt/comments/1lox7ha/bad_revi...
https://www.reddit.com/r/germany/comments/1iiaco8/restaurant...
https://www.reddit.com/r/AskGermany/comments/1ha7sxf/why_do_...
https://www.reddit.com/r/SipsTea/comments/1t613w7/it_was_fin...
https://www.reddit.com/r/germany/comments/1l98608/threatenin...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/1sw34jc/can_you_ge...
https://www.reddit.com/r/germany/comments/10kmn66/writing_an...
tl;dr Germans are particularly bad at coming up with reasonable rules to handle these situations
Broad defamation laws always have overwhelmingly negative consequences.
The US has no strict defamation law and yet your bad review will often still get removed from Amazon or Yelp for no valid reason.
How could Google possibly decide which defamation claim is valid and which is not? What realistic cost-effective alternative do they have which does not involve accepting every report of defamation at face value?
>The US has no strict defamation law and yet your bad review will often still get removed from Amazon or Yelp for no valid reason.
Amazon and Yelp choose to do that, they are not forced to do that. How is that supposed to be even vaguely relevant?
HN users love to blame this stuff on "Tyrannical EU law" when these things aren't driven by law at all!
If Google says "We are removing this because it violates German defamation laws", they don't have to be telling the truth!
Companies are making these choices for their own benefit and somehow HN posters still blame government.
However, I can easily see the slippery slope where in practice this means providing any AI response becomes too risky, and it becomes another money club used to extract wealth from big tech due to the current hysterical anti-AI moral panic.
Which would ultimately kill the ability of the German people to get access to competitive AI models.
I'm sure many anti-tech/anti-civilization doomers on HN will cheer this on. However, in reality it would do nothing to stop Germany falling behind, and continue its economic malaise/low productivity growth and social welfare collapse. When things in Germany get bad, Germans historically have tended to...ummm...cause issues for Europe. I would not take this lightly given the current rise of more polarized political rhetoric and German economy pivoting hard into weapons manufacturing.
It's one thing for an LLM to get things wrong. This case is not that though. And if big tech can't make sure that their models don't libel companies and individuals then good riddance. Whatever diminutive economic advantage one can get from "competitive LLMs" probably isn't worth it anyway, especially with all of the other disadvantages of these "competitive LLMs" compounding on our societies and the planet, as the resource usage of the data centres necessary to run and train them exacerbate the ongoing climate crisis.
My point is Germany further rejecting participation in the the next wave of technology (as they have done for 50 years) is going to NOT improve the trajectory they are currently on...which is bad.
The neo-Hitler and re-militarization thing is already happening. AfD leads in the polls, they're opening talking about remigration of immigrants, Germany now produces more ammunition than any country on earth. AI has nothing to do with it.
And, given I'm speaking to one of the the anti-civilization/anti-tech doomers of HN as predicted, do you think Germany rejecting technology again is going to improve the "climate crisis" or hurt it? How is the de-commissioning of Nuclear working out there?
But to loudly proclaim your love for this idea as most have here seems a bit premature given there are potentially massive downsides to applying legal liability to an AI model, if not done extremely carefully.
I'm just trying to move a bit beyond the emotional first order thinking. You might see this as manipulation if you love just swimming in emotional vibes, but I find this website becoming quite boring lately due to the lack of debate beyond this first-order thinking.
As someone living in Europe, watching a bunch of rich American engineers fanboy European over-regulation which they aren't actually being subjected to, is super cringe. It's the epitome of the principal-agent problem.
The solution is simple. Block the EU from accessing any of your services. They can make their own search and social media and digital marketing and AI.
Good luck to them. VPNs will boom.
I can understand the UK struggling with their imperialistic traditions and so on, but you'd think Germans at least would have some humility in trying to project their global ideas and ambitions on the world at this point.
Funny, because I would say this applies much more to AI corporations shoving their product into every facet of society without consent or regard for the harm it's causing.
I also seriously doubt that people are going to be enraged and boycott an entire continent of 400million+ people and start using VPNs because a corporation worth billions got held liable in one specific case in one specific location over the behavior of their hallucination machine - a machine no one asked for and is being shoved down their throats, mind you. If anything, this is gong to have the opposite effect and give people a little relief from the onslaught of BS and lies.