Libraries Run Rust Inside Python (With PyO3)(belderbos.dev)
22 pointsby lumpa2 hours ago6 comments
  • simonwan hour ago
    I was a bit nervous about this trend when it started picking up because I care about Pyodide (Python running via WebAssembly) and libraries that use PyO3 might not work in Pyodide.

    Thankfully that's now been mostly solved - you can compile and publish WASM builds of Rust or C extensions on PyPI now and a Pyodide can then use them.

    Here's the WASM build of the Rust-including Pydantic-core package for example: https://pypi.org/project/pydantic_core/#pydantic_core-2.49.0...

    • saghm44 minutes ago
      I'm kind of surprised to hear that was a concern because my perception is that PyO3 is more analogous to wrapping native C code in Python libraries (which is pretty well established since long before either Rust or WASM), and Pyodide seems more like something for use in browsers rather on the server-side. When I looked into this earlier this year, trying to run Python via WASM on the server side with Pyodide had a bunch of hoops I needed to jump through, and it ended up being a lot simpler to use a build of Python from one of the core Python contributors that directly targeted WASM.
      • simonw24 minutes ago
        Yes, Pyodide is almost entirely used in web browsers. I want to be able to run existing code that uses libraries like Pyodide-core in a browser.
  • the__alchemistan hour ago
    This is a nice party trick! I use it to provide easy installation of software that is written in rust, but is primarily used by Python users. (e.g.: Biology tools) They can do `pip install <name>`, since some people prefer this over downloading executables/installers. Bonus: Maturin/PyO3 can build "manylinux" binaries automatically, which helps with the Linux ABI diaspora.
    • benji-york30 minutes ago
      `pip install zig` works today.
  • throwaway6346711 minutes ago
    They’re not running Rust they’re running machine code compiled with the Rust compiler. Like many other Python libraries such as numpy are running machine code compiled with C++, C or Fortran. That was always the case for Python and is its main selling point, it’s a slow but easy to write glue language that can make faster native code scriptable.

    Also the article reeks of AI slop, it’s just trying to sell you a Rust course.

    • vlovich1238 minutes ago
      One main difference is that pyo3 is safer and easier to use to build said modules and have them work correctly without issues. Thus modules which hadn’t been worth it before to build in this manner suddenly become so
      • throwaway634673 minutes ago
        There’s a whole legacy of wrapper interface generators like SWIG that are way more powerful than PyO3, QT is e.g. fully mapped to Python using a very powerful wrapper generator (SIP). PyO3 is just a really simple interface generator in comparison, look at what SWIG can do, or SIP, what PyO3 does is trivial in comparison.
  • roywigginsan hour ago
    > reach for, a Rust extension does the work

    ai; dr, sorry

  • skeledrewan hour ago
    But does it work everywhere Python works? My main issue with this Rust move has always been compatibility. Python can be embedded and ran in a heck of a lot of places. What's the story when libraries that I may want to depend on are actually implemented in Rust and my target doesn't/can't handle the toolchain and there's no build target?
    • the__alchemist37 minutes ago
      It won't work in those cases. Do you have explicit examples? I suspect this won't come up much as Rust has, IME, much broader target support. (Including platforms without a GPOS)
      • galangalalgol21 minutes ago
        I can't come up with any cases where cpython would compile that rustc can't target. There are some more niche interpreters that can run on embedded stuff with no rustc support, but they are also subsets of python so it isn't like you can just pull a wheel file and expect it to work
        • vlovich1235 minutes ago
          For what it’s worth the GCC backend for rustc is making progress and rustc more generally supports a lot of embedded stuff already
  • hk1337an hour ago
    Why not just run C inside of Python?
    • saghm43 minutes ago
      For the same reasons someone might prefer Rust to C in general; safety, ergonomics, etc. If you want to write C in Python, nothing is stopping you.
    • the__alchemist39 minutes ago
      You can do that too. I generally prefer rust as it's a nice lang to work with.
      • m00dy26 minutes ago
        yeah, Rust is eating C/C++...
    • simonw22 minutes ago
      Because you want to use other existing libraries that are written in Rust.