165 pointsby linggen8 hours ago11 comments
  • whateverboat5 hours ago
    > This module is under active development. Once upstream, it should allow Rust developers to run Rust code on GPUs. We aim to develop a rusty GPU programming interface, which is safe, convenient and sufficiently fast by default. This includes automatic data movement to and from the GPU, in a efficient way. We will (later) also offer more advanced, possibly unsafe, interfaces which allow a higher degree of control.

    I really appreciate the work and the effort that went into this. However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

    • ux2664784 hours ago
      > However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

      They're very different languages, with different semantics. Without reading more than the synopsis of the paper, they're 100% leveraging the substructural type system and will have a really tight requirement for you to use a certain kind of Rust code at the CPU/GPU boundary.

    • aw16211075 hours ago
      > However, such an approach has previously not really worked for C++ with LLVM offload. Why would it work for Rust?

      I think that will depend on the exact reason(s) C++ with LLVM offload didn't work out? If Rust differs from C++ in a way that addresses pain points/failure modes/etc. from the C++ attempt, for instance, then perhaps it isn't unreasonable to think Rust could succeed where C++ didn't (c.f., Mozilla's pre-Rust attempts to parallelize Firefox's CSS styling engine). Inversely, if Rust doesn't do things differently in the right way perhaps one might expect the effort to also not work out. Or maybe the problems are entirely non-technical and things could work out in either language.

    • boxed3 hours ago
      Seems to work out well for Mojo, so I'd guess it's more an issue with C++.
    • jcelerier3 hours ago
      > However, such an approach has previously not really worked for C++ with LLVM offload

      ... isn't Metal shading language just C++17 compiled with LLVM ? working on every Mac and iPhone in the world is not what I would call "not really worked". Likewise, SYCL works just fine.

      • mathisfun1233 hours ago
        bruh lol this is so wrong and so confident i don't want to even attempt to explain how wrong you are.

        > just C++17 compiled with LLVM

        i invite you to attempt to compile/run absolutely any C++17 codebase on your iphone's GPU lol!

        • nish__an hour ago
          Wouldn't any metal app have to be written in C++17?
        • jcelerieran hour ago
          I mean "any C++ codebase" doesn't make sense in general. I run C++23 code on ESP32, that doesn't mean I'm gonna build KDE or chrome for it, and that doesn't make it any less C++.
    • winningChild3 hours ago
      [dead]
    • winningChild3 hours ago
      [dead]
  • YuechenLi5 hours ago
    So... why go through LLVM at all instead of having the MIR target PTX/HIP C directly then?

    If they really wanted a vendor neutral solution for Rust GPU, that already exists: you write the CPU side code, including buffering, allocation, concurrency, etc through Vulkan binding and consume the compute kernel in SPIR-V from HLSL/GLSL/WGSL etc. As it stands, the way they use Rust here feels more like using it like TypeScript types/interfaces than anything else.

    Again, the size of most operations that should be done on the GPU is known ahead of time before compilation, so it's very much possible to statically allocate memory at compile time instead of going through all this trouble to write what's essentially a Rust shaped DSL for GPU compute.

    • minraws2 hours ago
      Because it's convenient? Shader and vulkan semantics can be quite limiting and annoying to write.

      Maybe it doesn't matter in a post AI world but perhaps it will allow better abstractions.

      No need to yuck someone else's yum.

    • sanxiyn4 hours ago
      People go through trouble to write Python-shaped DSL for GPU compute. We will go "why o why?", but apparently such things are necessary to succeed in the market.
      • YuechenLi3 hours ago
        Well, I suppose fake Python is better than fake C++ at least.

        But yeah, I think Python's dominance in science/ML will eventually pass, just as FORTRAN and Matlab did before.

  • bicepjai4 hours ago
    I write all my code in Rust because I am a Rustacean. In many of my custom LLM inference engine projects, the biggest fight has always been bindings. I don’t want to maintain and write bindings; also, if I use an existing project that provides bindings, then I have to wait for the owner to update or fork it and then maintain it on top. It has been a big headache. Running Rust core on GPU sounds like something I will try from day one. Kudos to the team and will watch it closely.
    • 3 hours ago
      undefined
  • Thomashuet7 hours ago
    That's promising but did they publish any code? I can't find anything in the abstract.
  • boywitharupee5 hours ago
    is this mainly about making host binaries self-contained for heterogenous workloads?

    also, seems like this is mostly targeted towards HPC audience?

  • jasonjmcghee6 hours ago
    > the rust-gpu project has to emulate pointers[8], which we consider a blocking issue for most HPC benchmarks.

    Why is it a blocking issue?

    I feel like this is very aligned with the goals of rust-gpu.

    • minraws6 hours ago
      Pointers are sort of needed for high performance memory management for HPC targets for existing design patterns, maybe we can think of better solutions down the line but it's hard for me to say anything I just use/abuse CUDA pointers as well.
      • adgjlsfhk15 hours ago
        Julia has pretty good design heritage for how to deal with this sort of thing. you build the right abstractions and everything works (the main key is making sure the compiler elides bounds checks)
  • maxchisto6 hours ago
    does anyone know Mojo well enough to comment how Rust + gpu-offload compares to it?
    • giancarlostoro6 hours ago
      Mojo is not fully open sourced yet, but it will eventually be, would be an interesting comparison though.
      • maxchisto6 hours ago
        Mojo's OSS status doesn't prevent us from evaluating its memory model, writing and benchmarking kernels in it, etc
        • giancarlostoro5 hours ago
          Sure, and I realized after I posted the std lib is opened up, not sure how much of it will reveal the underlying Mojo specifics though.
  • Alephinitesimal6 hours ago
    The NVIDIA+AMD support is the part I find really interesting. I know OpenMP and SYCL can already target multiple GPU vendors, but doing this while keeping Rust's safety model seems pretty compelling. I'm curious how portable the performance is in practice.
  • jheriko6 hours ago
    [dead]
  • iberator7 hours ago
    [flagged]
    • aabhay7 hours ago
      Genuinely curious what these constant changes are. In fact I don’t feel they’re moving fast enough to give us improvements to core.
      • frollogaston7 hours ago
        It was changing quickly in the earliest releases, about 10 years ago. Like one day I pulled our repo and there was new "?" syntax, but that was a feature I'd been wanting anyway.

        Edit: Oh, async/await was a bigger and more recent one, 2019. I've heard that this wasn't an easy decision for them but was kinda needed.

        • aabhay7 hours ago
          A few years in between releases with a sane versioning system seems fine to me. If you don’t want new features don’t build with that new toolchain.

          It’s not javascript where you need to support all possible browsers.

          • frollogaston6 hours ago
            Teams will disagree over what toolchain to use, and you will read others' code, so this doesn't dodge the issue. Otherwise there'd be no complaint about C++. I don't think Rust is bloated though, every feature has a very good reason.
            • speedstyle2 hours ago
              Would they disagree, new toolchains work great with old code? And even add 95% of the features to older editions, just not new keywords or inference defaults. I think in C++ people complain about the effort to migrate --std, rather than having to learn variants or concepts. It does add to the complexity of course, and it's useful to agree on a consistent style, I just mean Rust doesn't have the particular issues with everyone having to migrate in lockstep, or anyone needing to for new tools
              • frollogastonan hour ago
                Same with C++, new standards will support old code. But they'll disagree on using the new features in the new toolchains. Or more likely, they'll just use the new features. So there's no option of sticking to old feature sets, you will encounter the new ones at the very least.

                Even with a style guide in C++... Google is known for having one. I was forced to change how I use basic strings multiple times there. They also changed the rules around refs (&).

    • jjice7 hours ago
      That's an opinion. One you're very welcome to hold, but it's not universal by any means. Lots of people enjoy rust.
      • 7 hours ago
        undefined
  • rfgplk6 hours ago
    Fascinating how many people still overcomplicate offloading to GPUs.