12 pointsby michaefe2 hours ago6 comments
  • mrDmrTmrJan hour ago
    Can someone walk me through exactly how powerful distillation is?

    I've never seen a good blog post or metric establishing what % of a new model's "intelligence" can be captured. I'm not disputing the document's claim. Widespread effort at distillation is clear evidence that it is highly effective. I guess I just don't have an intuitive or technical sense for how powerful distillation is.

  • ggman hour ago
    I wouldn't want to over state it, but from the point of view of a nation state excluded from a class of technology, acquiring it by means which look unusual and probably are is not inherently "wrong" -it probably breaks at least one other nations laws and attracts sanctions but you won't get any party from the USSR or the PRC or any other trade sanctioned economy past and present to agree "they did a wrong thing" because firstly states are essentially amoral and do things in the interest of the state good or bad, right or wrong, they are judged as fruitful or unfruitful and secondly although it's classic "whataboutism" in many ways, "you made me do it" is very strong.

    "You only leapfrogged us by stealing our tech and copying it" is a very weak line of reasoning. "You did not contribute to the embedded R&D costs to get to this point" is dickering about the price attributed to all three of George Bernard Shaw, Oscar Wilde and Winston Churchill.

  • fakedang8 minutes ago
    Saw this firsthand when Deepseek's thinking model referenced Claude and Open AI guidelines in separate chats as part of the thinking process notes.
  • toomuchtodo2 hours ago
    Kinda cool and very meta if Chinese AI companies are using distillation to liberate model value from US investment driven AI companies, by then releasing models created with distillation for free. Its circular IP and copyright avoidance and potential model improvement all the way down.

    Parallels to how China bootstrapped their state manufacturing capacity with Western company joint ventures contributing capital and know how, and then gave them the boot once they no longer needed them.

    • langfo2 hours ago
      Just finished reading "The Great Heist" which details across a number of manufactoring sectors with some wild stories the way China implemented their "Made in China 2025" policy.
  • cyanydeez2 hours ago
    O no! China is plundering the very plunder we worked very hard to plunder from everyone on earth.
    • chews21 minutes ago
      Add a dash of nimby and xenophobia and this is about right... you've got the likes of OpenAi and Anthropic destroying printed work from Austrailia and New Zealand to acquire latent data, Meta executives downloading commercial porn on their home computers and driving SSD's into work, all of them are building datascenters in metropolitan areas with generous tax breaks for their non-profitable ventures. The fact that 60% of model usage is now Chinese is what's prompting these folks to whine for protectionary measures. Meanwhile they will have no problem clearcutting massive swaths of meaningful work for people in their pursuit of AGI.
  • blackhaz3an hour ago
    Time to run the same playbook back to their frontier labs.
    • qarlan hour ago
      If their frontier LLMs are the results of distilling US LLMs - then stealing it back won't work.