29 pointsby hinshun5 days ago7 comments
  • diziet_sma5 days ago
    How long do we expect it to take dynamic derivations to get stabilized?

    The GitHub RFC says it's only 80% complete

    • Ericson23145 days ago
      I am one of the core Nix developers, and hte one most involved with Dynamic Derivations. Yes, I'm aiming to massage the issue trick a bit on this, and write up a proper post going over what's left.

      The short version is that dynamic derivations depends on content-addressing derivations, and content-addressing derivations has been in experimental-feature limbo for a while now. However (!), with a little push + judicious cutting scope, we can get the latter stabilized. And then the remaining road to stable dynamic derivations should be very smooth.

      It's a more complicated story to tell because the work is a series of yak shaves, but it's not actually a huge amount of work.

    • hinshun5 days ago
      I'm not sure, but we're in contact with some core Nix folks to see what we can do help move it along.
  • sporeray5 days ago
    Is there a particular reason you chose to target ninja over other formats?
    • hinshun5 days ago
      It made sense internally and we also thought it'll be compelling for people interested in Nix since Nix itself uses meson and outputs ninja.

      What other formats do you think are worth targeting?

      • sporeray5 days ago
        Not particularly, I was just curious thanks!
  • Buzz-Lightyear5 days ago
    This is really exciting! What are the early performance gains looking like in nix compilation?
    • hinshun5 days ago
      We're still missing this feature to fully incrementally compile nix: https://github.com/pdtpartners/nix-ninja/issues/19

      Since there's a target that depends on a generated source file for Nix's bison parser. But other targets incrementally compile comparably with regular ninja. So far we observed that Nix's sandboxing overhead is neglible.

    • Ericson23145 days ago
      As a developer not of this, but of dynamic derivations in Nix itself, let me put it this way:

      A long term end goal is that if you don't change headers and only change is single C/C++, anywhere in the build graph for your entire system you should get a quick recompile of just that file + relink just the executables/shared libraries the output object file is built in.

      This will require boiling an oceans, to get all the packages' build systems using Ninja or similar like this. But that's hopefully

      - less boiling that rewriting the whole world of open source in Bazel/Buck2.

      - a far more incremental, crowd-source-able project, as you could convert packages one-by-one, starting with big builds like LLVM and Chromium as Edgar says.

  • kbcyfsh5 days ago
    Wow this is so cool! Looking forward to trying it out!
  • dwarddd5 days ago
    This is cool! I'll be staying tuned for updates
  • rushil7915 days ago
    Very cool.
  • jbverschoor5 days ago
    I'll probably get downvoted. But isn't that exactly what make does? specify how an output file should be made, and which input files it depends on.
    • hinshun5 days ago
      No worries. I think Make vs Ninja is a whole another discussion but how it relates to this project is that Ninja has less features than Make (e.g. no globs, etc) which makes it attractive as a simple build graph format.

      Nix provides the toolchain and dependency management (like where boost is from), as well as ability to remotely execute it which makes it attractive to scale out large builds like a `-j 999`. Nix also lets you do things like patching boost and then recompiling both boost and the downstreams (incrementally and with early cut-off if using nix-ninja) all in one build graph.

      All in all, probably not useful if you're not already needing features from Nix. But if you are, this should speed up your builds by a significant amount.

      • jbverschoor5 days ago
        Not doing nix.. I tried a few times.. maybe I'll adapt my docker-based tooling at a certain point.

        Less features is probably good, because make originally was just the dependency thing, but got abused as a build system.

        Any performance improvements such as incremental compilations / incremental builds / interpreters / copy-on-write / hot-code reloading are always welcome.