This way I can easily write inlined functions with proper C speed, in a Python esque language.
Reasons why I rarely use Cython without this:
- unmodified Python code running in Cython is often faster, but not that much. 50% or so is my rule of thumb.
- but the development style is very different for Cython and Python, adding a lot of impedance. Eg reloading doesn’t quite work, profiling is different, you need a build directory and a build step at deployment etc.
- Cython proper is different enough and I’d argue unintuitive enough that if I have to have all these things in place, I might as well wrap my function in Rust.
This would give you an easy way to make things faster when you need it, with scope for making it much faster via all the funny Cython, while keeping Python-esque semantics.
I think it would require some language modification since Cython was definitely a superset of Python; I think some of that syntax has been streamlined, but maybe still a strict superset.
On one hand, it seemed like JIT work was really progressing in high gear until SC requested a new PEP. Also unfortunate, as it seemed like part of the hope for the JIT was that it'd counter the performance penalty from free threading.
On the other, it did feel (to me) like with all the fancy (and definitely cool) optimizations it got (automatic refcount operation removal, new type-based specializations, inplace math operations), the macrobenchmark benefits weren't _that_ exciting for a JIT. It definitely has potential for more, maybe with even fancier optimizations like unboxing ints, but the complexity just keeps going up.
> It said that the PEP was "overly optimistic" about user-visible differences with the JIT enabled.
The JIT is technically generated from the same source as the interpreter loop itself, so originally the differences were extremely unlikely, but I imagine the risk just kept going up as new JIT-only paths and optimizations were introduced.
And politically, I feel like SC’s reasoning was just “this side project has landed in main branch almost by accident and has been stewing without oversight ever since, with no stabilization in sight”; and the latest PEP was probably still too vague, with a lot of “here’s more things we can add to the stew” - so I can understand if SC wasn’t satisfied.
Not to say that the technical objections were without merit, they likely very much weren't, and the debugger and instrumentation support and maintenance work sounds fairly painful on its own.
If having to put all that in writing and get a bunch of people to agree feels too burdensome to you, maybe that’s a sign that you would be happier working on something other than core changes to the world’s second most popular programming language.
(edit: That wasn’t meant to be a jab at the core developers working on the JIT. Based on what’s in the public discussion threads, they have been handling things very well throughout the process. It’s unfortunate that the SC let the project get this far before realizing the project had grown well beyond its original PIP’s mandate, and I’m sure that is deeply frustrating to the people leading the project. I know I would not be handling the setback nearly as gracefully as they are.)
There are already many way of addressing Python performance (leaving aside whether Python is a good choice if performance matters) such as Cython, Numba, Pypy and C extensions.
Python is a mature production language with a lot of users. Even before it was huge - like 20 years ago when I was in college - getting a small backwards compatible API change into a python extension module (gnu readline in my case) required convincing GvR personally that the API change was necessary.
Every JIT attempt I've seen for python just tries to cut too deep in the first pass, and scope themselves too large.
I believe any successful attempt at optimizing python performance from JIT will FIRST start with a 100% semantically compatible hidden type model for optimizing object property access and method dispatch.
The single biggest performance delta in most dynamic language jits come from that, not from optimizing guards off of bytecode operations or from compiling to machine code. It's from converting hashtable lookups into constant-time guard + offset lookups.
The hidden type model is also a hard nut to crack. It needs to perserve dict semantics which have very dynamic behaviour via the various `__*__` methods, and handle exotics scope objects since scoped name lookups are the most common ones which have their own exotic behaviours (I remember python providing some ability for callee frames to inspect caller scopes in some limited ways).
Python already does it in a lot of cases, see opcode specializations like LOAD_ATTR_INSTANCE_VALUE, LOAD_ATTR_SLOT.
More digging led to this: https://peps.python.org/pep-0659/ which seems to be what the effort is organized around?
Python's object semantics are broadly similar to Javascript, bar the prototypical inheritance chain, but with the extra complexity of potentially needing to handle dynamic `__*attr__` overrides. Optimizing for this would require a tree of hidden map objects much like JS JIT engines use. I dug around the codebase a while back looking for something like that but didn't find it. `OrderedDict` seemed promising but after more investigation that turned out to just be adding property ordering to dictionaries.
The thing is, you can build that entire hidden type structure out, AND optimize against it, without ever having to mess with things like bytecode replacement. Once you have the hidden type model, you can modify the property lookup algorithm in the VM itself (as in cpython c code) to use some annotation structure hanging off of the bytecode to look up inline caches of type-guarded fastpaths. It would be an optimized fastpath that integrated directly into the C code that interprets the GETATTR opcode.
Probably also during the 2 to 3 migration, but that goal appears to have now been abandoned.
I wonder how the authors of other implementations like pypy will be able to keep up as the reference implementation becomes more and more complex.
Today there are far more people using it, and everyone has a different perspective on what it is. No longer is it a niche glue language, now it's a backbone of AI technology at major tech firms, so it's understandable they'd like it to be faster on its own.
Are the JIT developers the same people who were working on Faster CPython at Microsoft? No surprise that they don't have the same amount of energy to dedicate to the JIT now that it's gone from a full-time job to a hobby project.
I wouldn't be surprised if they all went nodejs instead :/
His current focus seems to be speeding up GC and free threading.
Where do we go from here? Microsoft did not continue to finance the Faster CPython project and the current speed gains are good, but not what we hoped.
F# is on the softest side of that, but letting the compiler tell the AI "nope, here's why" seems like a huge win in saving cycles and time.
The downside at first was simply, "how many examples of F# are there to learn from", and early on it showed, but I think that's been long overcome.
And F# is obviously NOT the best example since it's both "soft" on functional purity and niche. I imagine something like haskell is probably doing even better, although I've yet to check that at all (not that I can code haskell but I haven't looked at anecdotes or anything).
I realize this doesn't really help anyone who's grown to love python, and there's surely no perfect language, but just like how the web catapulted javascript, I see a future where the answer isn't "use AI to make python better" so much as "what already works super well with AI, and does it ease the user adoption"
What about the other major change to Python's implementation: free threading? Has it lived up to expectations? I assume it's not also in doubt, despite Guido recently saying "Personally I wish we’d stuck with the GIL"? [0]
[0] https://discuss.python.org/t/pep-805-safe-parallel-python/10...
A bit like Chrome there's a fully open source version and a proprietary version with a few extra bells and whistles, but the latter is entirely free for both open and commercial usage (you can't redistribute it for a fee). So you may as well use it.
It has other interesting features
• Scalable pauseless GC
• Single binary distribution of Python programs (like Go)
• Fancy and optimized interop with other languages including Java, JavaScript, Kotlin, Ruby and any other Truffle language.
• Supports native extensions. Tested and works with common libs like SciPy, NumPy, PyTorch, Pandas etc: https://graalpy.org/python-developers/compatibility/
• They're working on a GIL-free mode
In recent weeks there has been a release of a new web framework for Python that only runs on GraalPy, called Pyronaut (https://pyronaut.io/). It takes advantage of JVM interop to expose the Micronaut framework, which is similar to Spring for those who know it. Thanks to the JIT and well optimized base libraries it runs extremely fast, being about 3x faster than FastAPI+Granian, or 7x faster than Flask+Gunicorn and having many more features.
https://www.youtube.com/watch?v=zo6L7bxn4OY
(that was presented at Devoxx but the Pyronaut devs will start touring Python specific conferences soon)
So although CPython may not have made it past the JIT milestone, that doesn't mean Python as a language or community has to fail it. It does require openness to alternative implementations, but historically Pythonistas have been fairly good about that (e.g. Pypy was embraced back when it was more active).
This stuff does require you to accept that it's made by Oracle, but as it's open source, that should be OK.
Disclosure: I sit next to some of the people who work on this stuff.
> We want the PEP to be clear that it’s ultimately the JIT team’s responsibility to find solutions for these packages and tools, and how to deal with situations where a tool can’t be supported or made to work
Is it just me or is this absurdly impossible, to the point of willfully killing the project? Committing a team of brilliant compiler engineers to doing free consulting ad infinitum for every bit of tooling in the world, however hacked-together?
The alternative would be to have the bravery to say: this is the way forward to keep this language relevant into the future, and as a steering committee we collectively own that some tooling may be out of date. Instead, this is a signal that Python is going to rest on its laurels for a very long time.
Python is my favorite language, but I'm honestly tired of having to rewrite performance bottlenecks in Cython or Rust all the time. When the Faster CPython project started with Microsoft's backing, I was really excited. Mark Shannon had that plan promising huge speedups across a few cycles, and while that clearly turned out to be much harder than anticipated, the team still kept pushing forward to deliver a real JIT.
Seeing all of that get bogged down in committee bureaucracy now is just frustrating.
I have a lot of respect for the core devs and I know maintaining CPython, and it is a time donation job. It isn't easy. But I feel that over-caution and red tape ends up discouraging serious contributions. They are setting an unreasonably high bar for something that would obviously help the entire ecosystem.
Just think about the sheer amount of electricity and compute power being wasted every single day running inefficient Python across millions of machines worldwide.
Python really deserves a proper JIT. I genuinely hope they reconsider this.
PyPy lacks upstream interest, looked like it could have a shot after many years of development but recently NumPy removed support because they just couldn’t fathom how a non-dead project could be a couple of minor versions behind CPython.
Now the new JIT is dying because of what seems to be a lack of upstream interest. (The things TFA describes as being required are not unreasonable, but not realistic for a necessarily speculative long-running project without wild amounts of committed funding.)
I’m not optimistic.
IIRC Unladen Swallow was, despite the extreme hype, nothing more than the internship project of a couple of students. It died because their internship was over.
Otherwise you're right, though. Lots of CPython alternatives suffer from lack of upstream interest - the Python language designers don't seem very interested in anything other than CPython. In particular, they appear to only consider CPython when adding features to the language. Two examples:
- Requiring `dict`s to be ordered because that's how they work in CPython, without considering how they work in other implementations
- Adding pattern matching syntax that's suitable for CPython's PEG parser, even though it's not LL(1) so can't be parsed by, say, MicroPython
for me it's over for python