There is no posttraining data commons(openathena.ai)
31 pointsby lukasschwab12 hours ago3 comments
  • mindwokan hour ago
    A posttraining data commons and the infrastructure surrounding it is the next great open source project IMO, akin to Postgres or Linux.

    The Marin project from Stanford (and I believe now ran by Openathena, the authors of this blog post) is really promising and cool: https://marin.community/

  • impossiblefork2 hours ago
    I think a post-training common would have to be somewhat protected, since these things would be created with great effort and would potentially impact people's ability to earn money in their jobs.

    I could also imagining them happening as state projects for the sake of competitiveness.

    • lukasschwab31 minutes ago
      I'm not sure I'll do it justice, but one of the general ideas behind open-model initiatives is that the economic impacts of specialized post-training are a fait accompli. OpenAI and Anthropic have spent $many-millions procuring closed posttraining data.

      The relevant question here is whether you want open-weight models — projects like Open Athena — to remain or become competitive in the same range of specialties.

    • dannywan hour ago
      They're already kinda protected, open weight releases rarely come with datasets, especially post-training datasets. NVIDIA is probably the exception.

      I also wonder when the NSA starts sharing their wiretaps/etc with labs for 'national security'. Probably already happening, to be honest.

      • edot9 minutes ago
        Wrong direction. NSA says “give us the weights to Fable and loan us some engineers who can get a TS-SCI clearance so we can train our own model in house, if you don’t well just take them, in fact maybe we already did, it’s for national security”.
      • Retro_Devan hour ago
        > I also wonder when the NSA starts sharing their wiretaps/etc with labs for 'national security'.

        What would this benefit the NSA? If they share wiretap records with labs such as Anthropic, then suddenly anyone who can jailbreak a model gets access to this data. You don't need to train a model on data to understand or interpret it, as far as I know - providing others (esp. AI labs) with sensitive data is a breach of security, or I am not understanding.

  • emscb2 hours ago
    [dead]