Feels like Anthropic crying do as I say not as I do.
An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
The original authors of all the text, creators of the media and developers of the software did far more work than Anthropic.
Did Anthropic put work in? Yes. Did they derive their value from Humanity being open with knowledge then try to sell it back? Also yes.
Did they even steal the tech? Also yes.
Is that not what the foundation models are? A new reorganization of existing knowledge?
So then the problem is that Anthropic seems hypocritical when they knowingly insert themselves into this chain, and then complain about people down-chain from them.
To remedy the negative impressions (if they even care to do so) they should do 1 of 2 things: 1) stop complaining about it 2) stop distilling other people's work
No LLM products would exist without the avalanche of largely non-consensual use of IP to create them, full stop. Any of these companies doing this and then turning around and complaining when their IP is "breached" are going to met with a chorus of tiny violins.
It is not a natural assumption that someone will digitize the artwork and use it to adjust a couple thousand matrix coefficients in a complex computer program. To most people that seems like copying with extra steps. The brain may in some ways resemble a computer, but what sets it apart is that we have always lived with brains. Everything a human does has already anticipated the presence of other brains, while etched circuits on ultrapure silicon crystals are something new.
A comparable idea could be that an encyclopedia or maths book is only distilling the things that other people did, and how dare they sell them. But the "only" is doing quite a bit of work. LLMs do not just spawn into existence. There is a body of work that they feed on, and then there is also very attributable work they do around and on top of that. All labs are struggling around the first order question: Is it okay to use prior work like this? The second order issue is still entirely reasonable to separately have and enforce rules about.
It’s like FTL. Until someone realizes it, it’s just talk.
This isn’t meant as a moral argument, just musing about the relative cost comparison.
Image/video models are, but those weren’t the topic.
The totality of the content on internet is worth several orders of magnitude more.
A training set of 15 trillion tokens is 10 trillion words.
A penny a word is cheaper than the cheapest beginner freelance writer.
That makes a training set of 10 trillion words cost $100B.
Lots of assumptions there for sure, but we're certainly in the ballpark you are describing.
And the monetary cost doesn't even register when weighed against the blood, sweat and tears that went into capturing the authentic experiences of real human beings, whose honest expressions are now at least in some cases getting hoovered up, ingested, and then destroyed for all eternity, for fear that this specific work is the rounding error that might give an equally immoral competitor the edge in the bicycle-riding flamingo race that is currently consuming an absurd amount of the world's creativity and attention.
$500B for all kinds including TV and online
$300B for newsrooms including all staff
$140B for newsroom reporters only
So yeah, I think the price of the information ingested is way higher than training costs
How is this even a question.
The knowledge that created the Haber-Bosch process [1] helps to sustain the majority of the world's populous, add another five hundred trillion dollars for that just to start with.
The creation of the printing press and all written information that allowed it to be built provided dissemination of knowledge beyond the ultra wealthy and is worth a non-finite amount of money.
LLM's are cool math, but they are less than a rounding error in comparison to even the tiniest sliver of human knowledge and technological output.
[0] https://pubmed.ncbi.nlm.nih.gov/35143880/ [1] https://cen.acs.org/food/agriculture/The-industrialization-H...
And also, humans have been doing a lot more work than just mathematics...
“bro like, what if we could price the sum total of human knowledge? That wouldn’t be that much, right?”
Anthropic’s anger here seems mostly rooted in their annoyance that this exposes they don’t really have core IP that’s not just easily replicated. And that’s clearly a problem for a deeply unprofitable company trying to convince people they’re worth $2 trillion.
Oh that’s very sad.
Meanwhile Anthropic made a product from the work effort of millions of people without compensating them, sell that product on tap and unless I am mistaken do not even have their competitors’ cover of having released any sort of meaningful open weights model.
They have taken from culture (including very specifically their most direct customers’ specific culture — our culture), turned it into a machine to make themselves rich, appear likely to predicate their valuation on permanently removing people from the workforce, then want to dump themselves onto pensions funds and ordinary savers to carry the bag.
It is, I agree, philosophical, because karma is a philosophy as well as a bitch.
With my apologies to Brewster Kahle, "Universal Access to All Knowledge."
"The Spice must flow."
>An argument can be made that Anthropic is also only doing the last 5% of the work (because the content they are training on was the other 95%) but that's a bit more philosophical.
How is this philosophical? They should release the unsupervised pretrains, at the very least.
I see absolutely no distinction between the two, aside from minor technical approaches to gathering the content.
Actually, it's an interesting argument to make. How many labour-hours went into creating the training data Anthropic has collected? Probably multiple billions of hours. How many labour-hours did it take them to setup the datacenters, scrapers, and training algorithms? A few thousands hours?
It sounds exactly the same, not more philosophical to me, except one is more inconvenient.
I think a good litmus test here would be if Anthropic were to not care about distilling their models when the distillers keep the resulting models closed-source and sell tokens via an API. If they cared only about security concerns and not about people profiting off of their work, then they should be publicly fine with this and only protest against it going into open-weights models.
And writing a book requires many more resources than what anthropic does
Anthropic and OpenAi are spending a $$$$ to "distill" human output into an AI model, then others are spending $$ to distill their AI model into a near-equivalent model.
This is the same reason IP rights exist. On the surface, something like a patent feels ludicrious and even feels morally wrong. Some guy wrote down the recipe for arranging atoms or bits in a particular way, and now I can't!? However, it's designed to solve the same problem, figuring out and describing the process is much more costly than replicating it.
That doesn't mean they won't try, and that also doesn't mean they won't succeed.
I wish people here could at least bother to inform themselves about the IP rights they are so quick to insist are abhorrent, when they seem to not even have a first clue as to what they actually cover.
You could say Anthropic distilled human knowledge and art.
How can one answer this statement in good faith? AI companies literally violated IP by massively pirating works instead of legally licensing them.
The word itself is the pivot, not anything else.
I don't know anyone with even a passing understanding of how LLM training works that thinks that is the appropriate analogy.
it's not complex. there's hundreds of billions of investor dollars counting on vendor lock in and walled gardens
If I'm a business and I need something done today, and bc Anthropic has the best model, there's a 99.9 chance it will be completed successfully for $1000. And using Deepseek there's a 70% chance it will, for $10 - you or me will go for the $10. Big businesses don't. Bc 1000 per task is nothing to them.
The real issue is that Deepseek has a 99.7% chance. So I can run it 10 times until it works and still pay 1/10 the money.
Also, is it really 99.9% vs 70%, or 99.9% vs 99%?
Big businesses might pay $1000 vs $60 for certain tasks, but that won't work out well at scale.
Most folks I know can choose from any of the big labs or open weight models and they get billed internally for tokens against their budget. There’s little incentive to no switch to the lower cost closers.
This setup is a nightmare scenario for the big labs trying to execute the traditional enterprise sales plays. Those only work if your product is sticky and AI models are one of the least sticky things in the history of tech.
why should sammie or darigold have the keys to the kingdom?
Can you elaborate on that? I mean my direct answer would be no, of course not. But why do you think frontier models are distilled? I think maybe there is an equivocation over the word “distillation.”
Frontier labs train on their own pretraining data, human feedback, synthetic data, and research. A distilled model is specifically optimized to reproduce another model's behavior.
Meanwhile R1-Distill-Qwen-32B was distilled from DeepSeek-R1.
If you want to say a frontier model is "distilled" from the world's data and R1-Distill-Qwen-32B is distilled from DeepSeek-R1 then you are equivocating two very different things.
Does anyone know if there are any distillation datasets available? I'd love to see these distributed on BitTorrent. I think it's critical that AI be democratized and not isolated in the hands of a few private companies.
Third world societies are not "brainwashed" one way or another with respect to that.
We just see those governments throwing shade at each other. And bullying us ina similar way.
Love the arguments btw. Your position is so cut and dry that you can’t even support it with evidence.
China has zero independent media and has the largest and most sophisticated censorship and surveillance state in the world.
Trump would love to have the absolute power that Xi has, but thankfully doesn’t.
Tired of this lazy whataboutism.
Trump kicked and banned major news orgs from the White House just now. You’d have to be will fully ignorant or really naive to believe we have fair and just media.
The whole point of AI is to get rid of you so rich people can play with the planet like it's minecraft. Software engineers just think they're special because they're the ones building it like they'll get a pat on the head for being good little servants to the investor class. Or worse, that their portfolios will let them be the gods who rule over ashes.
https://www.anthropic.com/research/glm-5-3-and-the-spread-of...
It's VERY clear that the US companies are trying to push for regulation to kill open models and open weights. I see this as much more hostile and authoritarian response than what we're seeing come out of China right now.
So is China going to always publish in the open? No clue. But right now they're modeling much better behavior.
They don't have to be our friends to act in our interest.
The point is get what you can from both to develop open models, data and tools.
Or did they not pull back when their models allegedly became highly capable, with the whole mythos debacle ?
Absolutely. In fact, the authoritarian regime already did try to force export controls on major frontier labs not long go so this isn't a theoretical.
You ask about distillation but I wonder, is there any training datasets (~ TB-order) available that startup folks in SV use or is it so that everyone has to create their own scraping pipeline ?
While I'd like to agree with this, the fact is that pushing the frontier out has always taken (and folks expect to continue to take) hundreds of millions/billions of dollars. Open source and distilled models can follow on for much cheaper, but it's hard to imagine the frontier ever being "democratized" given the huge sums of money required. It was this realization that forced OpenAI to take tons of private investment in the first place.
The amount of progress that came out of academia and other public sources should not be underestimated either and without all that OpenAI and Anthropic wouldn't even exist.
It’s sort of like gun nuts arguing that more guns is the answer. I mean, ok, maybe you’re a responsible gun owner or AI user but relying on personal responsibility doesn’t fix systemic problems. There are bad people out there.
You can do this with cars, tools, computers, ... whatever you want. So, no, I think your point is wrong.
Now, what I want to regulate are accordions.
Haven’t tried getting a gun in California. How bad is it? How could it be improved?
You have to pass a basic knowledge test, have a clean background, prove residency in the state, be 21 (or 18 for hunting rifles, IIRC) and then wait 10 days.
It’s not like we ask to wait for 2 weeks when you have something to say or when you want to pray to your god.
AI-automated warfare is looking pretty scary too. I don’t think it will stay in Ukraine.
The situation is sort of like the Internet before broadband. There probably aren't enough AI-capable home machines to have big swarms of bots that run autonomously. If the swarm depends on an LLM API, it will be easier to cut it off once it's noticed.
I think it's an even chance that we will see a bot swarm in the wild (rather than coming from an AI lab) by the end of next year.
IMHO the biggest problems will be with poorly written AI slop software, probably small business crapware produced with minimal investment (maybe entirely without an experienced developer) and legacy equipment that’s been abandoned by the manufacturer. This will lead to disruption but not catastrophe.
Other countries have governments that have earned that level of trust. I believe the US could get there eventually, but it will take a very long time because it has a very long way to go.
I predict that the AI scaremongering will fizzle out when the bubble bursts. There will still be die-hard believers but the public will lose interest.
I don’t see how a stock market crash will make AI-related concerns go away. There was a dot-com crash but the Internet just kept getting bigger and causing more problems. In many ways security has improved, but we worry more than ever about social media, etc.
For Internet-related disasters where people died, see [1].
I would bet that there will be AI-related disasters. Arguably the US bombing a school in Iran counts, though it seems to be due to organizational issues, too.
[1] https://chatgpt.com/s/t_6abd4e3226b48191a26a0fe6768c722f
People did use global agreements and regulation to fix the ozone hole, though, so I think there’s a chance.
Replace “gun” with anything and you will see how your comment falls apart.
What’s next? A registry for food purchases? Your beer gut is starting to show.
We do have lots of food safety regulation, which has more to do with selling food.
Fortunately there is no right to AI.
Surely you could name three such societies?
Worse for whom?
The only effective defense against predatory corporate and government AI is personal protective AI.
Anything else is unilateral disarmament. It's the only way individuals can survive in the worse case scenario.
> gun nuts
Guns are different. They can't protect you against the government, contrary to gun nut claims.
I think this explains why they are open sourcing broadly. It's not to be nice. It's a strategic play by the Chinese government to help ensure there are many players in this race and not too much power accumulates to American labs (even if American labs benefit in the process)
There is no rule that every Chinese LLM company must open-source their models, and many don't.
> If China ever gets ahead, they're going closed source and weights immediately.
Anthropic itself admits that Chinese models are merely months behind. Your argument does not make sense, because Chinese labs are contributing massive optimizations like the one this post is about.
Please quote people you appear to be patronizing. China can't do anything about previous released self-hosted Chinese models. If you can show that local Chinese models funnel vast amounts us data home I'm sure you can move a lot of people to your side.
Comments like this also always fail to address why there aren't Western AI companies doing the same thing. Is it because they might get sued into oblivion by Big AI in the US?
It might be better for all of us if you solve that first instead of repeating something the government has been repeating for the last decade or more. It does this, mind you, while sabotaging itself in countless high-tech fields and leaving it all to China for the taking.
Or perhaps they've looked at history and concluded that this historically hasn't been where the value is, anyway. It wouldn't be unprecedented - FAANG companies have a long tradition of publishing their algorithms and releasing open weight models. Because they saw the real value as being the training data and in proprietary special-purpose models. For example Google published the transformer architecture and released BERT as an open weight model, but doesn't really even talk in public about the (presumanbly) specialized internal models behind revenue-generating products.
That's giving a lot of credit to Google's organizational ability to productize Google's research...
I realize "going as fast as we can" is not the most popular position atm. But I'm far more interested in what good we can do than 10% apocalypse scenarios. I volunteer with a charity for childhood brain cancer and I do not want to see another 4 year old die. I'm willing to risk anything to stop this.
(I also have flash next running even faster on this machine, something a single 5090 can do, with expert cache/pinning, but not quite as fast) :)
Local inference will have a boom of cheap, powerful, and available cards at some point (even if it isn’t until 2028/2029). At some point the hyperscalers, and frontier labs, will face the capex problems that everyone talks about, and NVidia, AMD, Apple, and Intel will want to keep selling products.
Powerful, by today’s standard, local inference needs to be accessible to really unlock the “AI” economy long term. It’s just like how the move from mainframes to the PC 40ish years ago unlocked the “computer revolution.”
Once hyperscalers stop buying in the quantities they are now, there's going to be a lot of hardware supply to serve by then very hardware efficient models.
I don’t think it is intentional but this is actually quite bad for the western labs.
The entire booster narrative has been “look at how their revenue is growing! $10bn to $100bn ARR in under a year! This’ll be a multi-trillion IPO!” and the extrapolated future growth from $100bn to $500bn and $500bn to $1tn justified future investment… but that revenue was just because inference was expensive.
The revenue growth story is all that matters pre-IPO. If revenue falls from $100bn to $50bn that’s very very bad optics for OpenAI and Anthropic even if they are now profitable, it completely destroys the growth narrative.
I never take them seriously, I just assume they are coming from countries that don't understand how capitalism works or are operating out of bad faith. The underlying reality of the market is always changing and needs are always changing. Some AI companies will fail, that is a given. Remember alta-vista? Yahoo? Did search go away? How about Microsoft phones? Nokia? Motorola?
OpenAI and Anthropic are not in the inference business. That is a commodity. They need to sell products and solutions.
Inference will be too cheap long term to make money because it is being commoditized and customers will start to care about results and not just be wowed by impressive technology.
And of course this technology will continue to exist but that is irrelevant to the business. OpenAI investors don’t care if LLMs exist in 10 years, they care if their investment in OpenAI has made money.
Relative to traditional software margins, the type of margins we are all used to, inference is obscene.
If local LLMs get "good" enough, people will soon paying for subscriptions to ChatGPT and Claude, which hurts their revenue.
It is vanishingly rare I ask an older model to do any task. Newer bigger and smarter models will just do the task better.
Therefore, I believe we are nowhere near 'good enough'.
I never drive my steam engine to work these days. It isn't good enough.
I can see my new Thursday afternoon "oh chit" moment being that I didn't complete my weekly task because i torched all of those tokens M-W doing task/ticket grooming using the hot hot model instead of the dodo model with jira mcp connector :D
To borrow your steam engine analogy, if local LLMs get as good as a Toyota Prius, even if OpenAI / Anthropic offer Ferraris, most people will be happy with their Prius as their daily driver.
Similarly, if the big labs start raising prices or cutting usage, you won't be able to use it as much as you want -- whereas a local LLM will run all day every day without costing you any extra money.
So right now you are right, but who knows how long that will last.
However, there are many use cases where they aren’t the right tool for the job.
This is not nearly true for everyone else in the world.
For example, think about the world in ~2021 pre-LLM. Would anyone say the sentence "I only want the fastest and smartest humans working on my project"?
No of course not. Most people don't want to pay $10 million dollar salary to the best programmers in the world. They prefer to pay $200k salary to a median programmer and that's good enough for their ecommerce website.
But so many teams said they wanted to Raise the Bar to infinity and hire a World Class Team.
Given the recent deepseekv4.1 advances - how good of a 3B model can we make to run on an iphone natively? is it good enough to match common muse/dot use cases for consumers? the phone is already always on.. no need for a cloud server.
I don’t expect the economics of local vs cloud ai to change until either the bubble pops or new ai chips land that can run big models fast with low power demand.
> So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
That inference wasn't profitable is a widespread myth.
Analysis based on Kimi K3 suggests that OpenAI and Anthropic have margins well north of 95%: https://inferencex.semianalysis.com/run/kimi-k3-on-b200
Over the last months I have seen news that OpenAI made breakthroughs in inference efficiency multiple times.
I have no reason to believe that the leading US labs don't have their own optimizations, or that they learned of this particular optimization from DeepSeek.
If they already did, then DeepSeek still made them discount their prices significantly, which eats margin.
I wonder if margins on GPT-6.1 Sol and Opus 5.5 are now 75% or 90%.
The difference matters when they're investing the excess into salaries and bonuses to build the next frontier model.
Seen from a high-level perspective, if Chinese open models are compressing US AI labs' profit margins and those margins fund US AI labs' dominance, then open models are decreasing American AI dominance.
"Thus the expert in battle moves the enemy, and is not moved by him."
They figured out a clever method for avoiding excessive training costs via distillation. That forces the hand of frontier labs to move faster, produce better models, etc. (to avoid embarrassment and 'falling behind'—all the while shouldering most of the cost), which they can just keep distilling—or applying other techniques against—much to the dismay of said frontier labs.
Checkmate.
Certainly anyone who knows something about inference is going to speculate, looking at the change in token pricing (and particularly how the % drop in cache read pricing is much larger than the % drops in pricing for other token types), that there is some kind of KV cache optimization behind these newer models. But even if is true, I don't think anyone can say with certainty what it may be. It may be the labs making their own innovations (they have some very smart people, and this is probably an area where having unlimited pre-release access to frontier LLMs like Fable and Astra gives an additional research edge), it may indeed be the direct application of Chinese labs' methods, or it may be some combination of the two. Sure, it is fun to speculate about, but beyond the facts of the token pricing changes and the increased inference speed, it's just speculation. The certainty the author displays here is not very helpful.
The author also seems to have a bit of an axe to grind agains the US labs, judging by the tone. I think that detracts from the discussion too.
They also seem confused about why Chinese labs have released these optimizations recently. Well, you have to release them (with or without explanation) if you are going to release an open weight architecture, and that is what the Chinese labs have been doing for a long time. Sure, there are reasons behind that to discuss too, but this isn't exactly new.
So, this is an interesting topic to think about, and the Chinese labs do indeed deserve credit for some very clever new attention and inference techniques, but I would read it with a skeptical eye.
Any ideas?
Because of the basic huge recession going on in China, you can't actually make money in China doing China things. So they gotta gird up their export stuff and try to export. That entails strong relations with American companies, American PR, English stuff, etc.
If you want an essay about this from a VC, read this one
https://earnedintuition.substack.com/p/involution-without-ex...
Enshittification and related problems can be a result of market forces just as much as they can be a result of monopoly/duopoly or a small cartel. Excess competition sometimes results in all firms scraping the barrel to squeeze out pennies, especially with technology (such as large online marketplaces) making pricing more transparent.
Marx actually predicted that ever-intensifying competition would destroy markets through overproduction, although he did not use the term involution.
The reason it doesn't work like that IRL is centralized marketplaces. If the winning strategy on Alibaba is low prices, bad quality and botted reviews to compensate then every seller has to do it to survive, because they can't get buyers outside the platform. That's not excess competition. It's a lack of competition just on a different level.
Well yes, that’s why I mentioned those specifically. But even if the marketplaces were split up, someone could create an aggregator to comparison shop and the same effects would apply. The problem for producers is that the Internet erases information asymmetry.
It’s also not just affecting low end goods, it’s a constant pressure on everyone, which is why many formerly upscale brands are seeing the same problems. It’s also a general problem with public companies, as large shareholders demand constant growth, as well as many private companies owned by PE where brands are stripped for short-term profits.
My experience is that the best low cost mid-tier products right now are coming from fronts like Vevor and Fanttik who do sourcing from noname factories in China. I’m not sure if their position is sustainable; it’s not like they have much of a moat. (I guess Fanttik has a team that adds some slick design to their otherwise utilitarian items.) If that model holds up then maybe that’s the future, but I suspect that they just have a temporary advantage thanks to a dual presence and connections in both the US and China.
Not that OpenAI, Anthropic or SpaceX aren't doing the same.
Do you do business in China?
I'm curious what you mean by this? Because in my experience, you can only do business in China by doing "China" things.
I'd be interested in picking your brain as to how you get around those issues?
I'm talking like, getting 100x, VC sized returns. Of course you can sell widgets in China, it's a major world economy.
That is how actual capitalist market should work and what anti-monopoly legislation should ensure.
Why did we (the west) ever start open sourcing anything? Maybe we just like sharing? Maybe humanity only grows on pre-competitive layers like Linux and clean water. Maybe, the chinese government is closer to their people, and does not let large companies influence them and just doesn't like closed private hyperscalers with a lot of power?
(Some points assume the government has a role in the openness, which I think is likely)
Once upon a time, everyone had a secret sauce in network or data encoding or query optimization, but in the last ~10 years computational physics and economics have basically decided the “correct” architecture and everyone (including OSS) has converged.
Put another way: if they were not cheaper and open, they would simply not be competitive. They would already be dead.
I don’t think this ends well for the Chinese labs. This is going pretty much like I thought. Western labs is just copying their improvements (I don’t think publishing the techniques matter here.. they’d just hire to gain the knowledge or figure it out themselves), and they have access to more GPUs and have better branding, so in the end where can the Chinese labs compete? Even lower cost? Open weights? I’m not sure open is a sustainable way to compete either. Eventually there will be some fully open source AI models that cuts out that avenue of competition as well.
The weird ideology here is to dominate the market at ANY COST, even it benefits the opponents.
Basically, western labs are in it for the money / commercial monopoly. Chinese labs are in it for the tech? As long as they can keep distilling models, and get access to research other ways, they benefit. And if they can push western labs forward, they'll benefit from that themselves.
We don't know either way, so I find the whole thing silly to speculate on.
Unpopular, maybe, but what about the normal reasons? The researchers are looking to make a name for themselves, and/or they genuinely care about AI advancement.
I can't even fathom the trend these days of "we don't review the code" from security team perspective.
Just my guess though.
They're building bridges over the moats that companies with far too much US investment are trying to build, and if they do it continually it can help destabilize the US economy.
Because contrarily to the author's assumption, all labs, Western or not, have sufficient skills to discover the optimizations anyway, and publishing or not is not actually that important?
I'm glad people are saying this out loud, because that is what they want. Not for the good of the world, but for the good of their pockets.
How co-designed are these optimizations with the model itself? I'd imagine you can't just stick post-training adapters onto existing architectures for these things, or am I wrong?
I really want to explore the inference space, but it seems like many of the inference optimizations are coming from model-hardware codesign. I don't seem to recall many generic "inference engine" optimizations since prefill/decode disagg a year ago.
This matters for me since I want to break in but the bar seems to be understanding the actual theory of the training process now too given the codesign happening, and I'm not the richest guy on the block lol
Second, sparse attention is an old area of active research. Offloaded N-gram tables are the next big open weight technological leap.
Deepseek was the company that invented some and improved some other ideas and got them to workreliably in production. Before that Sam and Dario were basically competing in who has the most expensive training.
I still use claude and openai right now, but I can see that not long in the future I won't bother with them, still waiting for a model good enough with computer use and a good enough computer use agent
I’m incredibly skeptical that OpenAI is spinning up custom ASICs for improved inference performance, but they never thought of optimizing KV cache until a tiny Chinese lab did it? Give me a break.
timeline suggests not.
Ah brings back Halo 2 memories
They do claim that it violates their ToS, which we can assume is simply correct, since they get to put whatever they want in their ToS.
Given all that, I don't know what the fuss is. Are they supposed to not use the advances that were openly published by Chinese labs? The entire industry is built on a discovery made at Google, which was published openly. Should Chinese labs therefore not use transformers? Should US labs not try to prevent distillation of their models?
Had western labs figured that out before, they would have used it to make kv caching cheaper before and not only now.
The burden of proof here is on western labs. But I doubt they'll try to lie that much.
Is the thinking that the day or so between US systems achieving ASI and Chinese systems doing the same, we'll figure out a way to neutralize them indefinitely? Because otherwise, none of this makes much sense. And it only starts to swerve back to sanity if the assumption is that this isn't a race or competition, but instead a joint effort to achieve something good for humanity. But you can't really delta profit off that, can you?
I mean, there's a pretty big difference between labs publishing their research openly and a competitor utilizing it versus a lab breaking TOS to... hmmm, what's the word? steal data from a competitor?
>That’s because, unlike the Western companies, the Chinese are pretty much giving away their recipes.
Yeah, Western AI companies have never published their research. It's crazy how the Chinese had to independently develop the foundational technology that powers LLMs because Western companies simply never publish their research (I mean, as long as you ignore stuff like this <https://arxiv.org/abs/1706.03762>).
>The latest one shamelessly copied without acknowledgement is the breakthrough in KV cache optimizations that DeepSeek has generously shared with the world.
Thank you, generous corporation. I'm sorry that other corporations don't provide you free publicity for your selfless contributions to the world.
>Now I don’t know why they would freely give away such a breakthrough, but they just did
Well I'm glad the author finally got to their point. A very insightful analysis.
>They do seem to be a little embarrassed by the copying. Hence the silent releases without much pre-announcement for both Claude Opus 5.5 and GPT-6.1 Sol.
You have to be in pretty deep to infer this kind of emotion to these kinds of corporate activities.
>So the Chinese labs have thrown a lifeline to the Western loss-making labs, and I just have no clue as to why.
Then why write this article? Why point out these things just to have no conclusion?
This article sucks. Even if you hate US AI labs and are all aboard Chinese labs producing open models, there's nothing of substance here. This is the loose draft that you hand to your LLM to finish for you, but it seems the author just forgot to do so.
Even if you're willing to characterize US AI labs as evil and selfish and Chinese AI labs as righteous and generous (which is already completely trivializing these dynamics to the extent that anybody over the age of 14 can likely identify is lacking nuance), you can at least put some effort into producing some hypotheses about why these dynamics are occurring. Of course, odds are if the author did try to articulate some hypothesis, they'd likely quickly realize that the narrative they're painting just doesn't hold up.
This is an extremely thin analysis that has obviously been voted to the top of the homepage because HN hates the big labs.
It's not that niche, if you've been online a little bit you'd know this expression.
No way. You'd need to be pretty well versed in gamer lingo. Even more specifically, combative, likely FPS gamer lingo.
Sarcasm aside, gaming and FPS terminology are so tightly coupled with online culture that it's just assumed everyone knows it. Spawn camping is among the oldest examples of gaming terms that broke out into common online usage and it dates back to Quake some time around 1997.
[0] anyone older than 39 at this point
What is online culture? Is someone who is heavily into instagram for fashion, facebook for family contact and news, maybe Google for mail and search, part of online culture? Because I know people who are like that and there's no way they know what "spawn" or "camping" mean in gamer context and certainly wouldn't be able to piece together what "spawn camping" is.
To put it simply, it's a Venn diagram with two overlapping, non-coincident circles. The amount of overlap has varied with time but that doesn't mean online culture doesn't exist or conversely that everything is part of online culture.
Strong consensus bias in this statement. I have no doubt it is true in your circle (and to a lot of people). But there are plenty of online-24/7 cultures that don’t have any overlap with gaming.
I've been online since circa 1995 (earlier if you count BBSs), and I can't say I did. It's possible to infer its meaning but assuming everyone is on the same circles as one is, is silly.
It shows that the author hasn't thought about their audience and has assumed that everyone knew and used the same terminology as them. And, worse, assumed that they'd understand immediately why they were using that terminology.
The opening is If you read the news headlines these days, you would be forgiven for thinking that the Western labs are getting spawn-camped by Chinese labs en masse. If you don't know what spawn-camping is you're lost; if you do it's not obvious what that means in this context. Good writing brings the reader along with the writer.
It would have been clearer if they'd written: If you read the news headlines these days, you would be forgiven for thinking that Western AI labs are being outplayed by Chinese AI labs using something similar to the gamer technique of "spawn-camping". The Chinese appear to be waiting for each Western release and then instantly distilling it. A little like gamers waiting for their opponents to reappear (from the dead) at their camp and then kill them off immediately.
This is better because a reader unfamiliar with the idea of spawn-camping learns something and it explains the metaphor. But I could be wrong in my interpretation of why they are using spawn-camping since they fail to explain it.
The premise in this article is: Western companies do a ton of expensive work building new models, meanwhile the Chinese companies just wait for a Western release and then they immediately grab and distill it and announce it as their own model. That’s the spawn-camp.
Do you have many mini tantrums like this per day? Probably makes you very difficult to work with Mr I was a CTO.
"spawn-camping" is the process of taking out your enemies at the point they spawn (or appear) in a game without giving them a chance to regroup. In this case I think the writer is saying that the news implies that western models are getting distilled on release. Not the perfect analogy but it gives some color.
In a PvP (player vs player) game, if you kill a player the moment they spawn into the game arena, that's called "spawn-camping".
Edit: Apt domain.
And the only data they are showing is that cache prices went down for new Claude/OpenAI models but that's proving nothing, IMO.