- Python interpreter in Rust (fail): https://github.com/BlueBlazin/pyrs
- Low-poly Google Earth like engine (success): https://github.com/BlueBlazin/lowpolyearth
- Codebase -> 2d infinite-zoom visualization (success): https://github.com/BlueBlazin/blockviz
I have never understood or liked all the obsession with one-shot builds.
I took it to 39% complete, as a language, over 8-weeks. Then, I spent just 50 minutes describing all the missing parts and it finished the language in 1-shot. I didn’t think it would work. It did.
I used Opus 5.5 on a $20 plan and I didn’t even hit my limits.
https://app.preview.craftspec.com/app/city/
This whole thing is just a proof of concept, and needs a mouse to interact with.
It’s incredible what AI has made possible with experimental projects that would require months of coding.
To get a toy version that can run most things takes under 2 hrs. Half of that time is collecting the tests you want to pass, how it should evaluate the code, what your criteria are, etc.
Same, and I have found some of the failure modes in the realm of eyes free coding and limited auditing. I often see it more as pushing the process than the model, as I tend to bounce around models for the benefits diversity provides.
Have you also found there is a point where they hit code complexity and start churning without progress? (at which point you have to step in to help the bot step back and think big picture, which usually takes a few sessions)
I was motivated to make the codebase visualizer linked above to help me understand how the various pieces of the system that is being vibe coded connect with each other.
Now I'm thinking about how I can build something similar for agents. How can we teach "systems thinking" to a model?
I've been mostly working on very large modeling projects (think detailed architeture over 1000 acres) or animation because this really was the "new" capability area that Opus 5.5 "unlocked".
I have been surprised how well it has worked at scale with many sub-agents. However the code base is limited to the rendering engine mostly.
Certainly in the older models they hit a complexity limit, but I haven't really hit it with Opus 5.5. I have had it make data flow designs that I didn't like because they were more complicated than it should have been or the perenial problem of trying to make everything backwards compatible when working on greenfield refactors, but I haven't seen it choke on a code base yet.
On the other hand, I have a TS only project core is sub-20k LoC (NES harness), where it is struggling (for now, see other comments)
It helps us keep track of things, why not AI, too?
I've been working on process more than model. Similar to having a PR review team, you need multiple teams or processes that run over the code, markdown, assumptions. I still don't think this is enough.
I'm working on an NES harness, and the agent was churning with detecting certain state changes like dialogs, menu navigation, screen change. The bot and I got churning and I put it down for a while. A few days later it occured to me to ask the bot "Does the emulator and game state change with frames? Can we just get the state each frame rather than trying to be clever about it?"
It said yes, but added some nonsense about being performance hindered... (flash model). I haven't worked on it again yet, but my intention was to next have it build a small recorder based on this so it can get frame accurate game state change.
This is just a long way of saying, sometimes you have to step back and help them help themselves. They have a really hard time with recognizing ambiguity or how to determine lack of context that could be filled.
Perhaps we should be more ambitious in the types of problems we try to solve, and the types of value we try to provide with the software we build.
What might help is non-software-first companies valuing developers and engineers more, enough that we could earn a reasonable living which building actually cool things. Not sure how the economics works out on the other side. Can these companies make enough money to support this vision?
AI can take us from 0 to MVP in a prompt. It can fill in gaps of knowledge and allows someone who knows half of the stack to build the whole thing nearly seamlessly. These are superpowers but don't by themselves lead to new products.
What AI cannot do is remove the world of complications that happen after the MVP - product feature tradeoffs and choices, how to handle integrations and customizations, how to serve multiple types of users, how to communicate and market the app, how to handle security and data storage, how to deal with customers and bugs, etc. All of these need to be considered and acted on by a human in some fashion. AI can help with each concern but can't solve them without understanding the concern and determining how to frame it in a prompt.
This is why we have a glut of amazing demos that go nowhere while we figure out how to build software differently. This will take some time and I imagine the result will be software heavily-structured with a bit of AI reasoning woven throughout, not an army of agents and sub-agents burning infinite tokens.
"Perfection is achieved, not when there is nothing more to add, but when there is nothing left to take away."
And I feel that sums up my feelings on AI this and that. Just as the wood and clay contain the sculpture within and it's the artist's duty and privilege to reveal that. So, is elegant software exposed from the conceptual mass, rather than built up.
The AI zeitgeist currently is 'how many features can I add' because we've tied self worth to work output instead of some other more personal or human rubric. AI will happily pile more and more and more code on your plate because it's a buffet. It's gone through reinforcement cycles guiding it to add whatever feature you ask. There is no training for elegance or careful considered design. Nobody is teaching this generation of LLM restraint.
The safest code is that which never runs. The most elegant feature is that which is not necessary.
There's a less hopeful outcome, where no skill is needed at all, because every conceivable thing is one-shot-able from a SOTA model. But then, at that point, you're saying something pretty profound about computer science (roughly: that there was no meaningfully more sophisticated and complex place to go from where we are today).
The most recent new things that LLMs have unlocked (regarding software dev) for us is reverse engineering and combining old games, unearthing decade old vulnerabilities, and helping root and debloat consumer hardware. Which are all cool and useful, but none of it being a “next great thing”.
Perhaps LLMs are really doing is highlighting how the software industry was already nearing its economic maturity and this just shifted the timeline from 2040 back to 2027-2030.
1. well specified
2. well tested (which is a form of spec)
3. represented in the training set (i.e. not built on a corpus of human developed examples)
For example, a lot of the compiler porting efforts fall into all 3 buckets: compilers are well specified (there is a set of valid inputs and an expected behavior), they are typically well tested, and as a bonus, in a way that is implementation agnostic thanks to the clean interface (think: integration vs unit tests), and are very well represented in the training set.
What I'm sensing is something I might describe as the Oligarchy reaching the point of significant societal pushback, like the labor rights movements that ended 80 work weeks, unsafe conditions, and child labor in factories (i.e. The Jungle).
My hope is that we can use this moment in human history to break the user tracking and targeted advertising industry. It's been so harmful to humans, but in doing so, we will likely break a major revenue source that supports our well above average salaries. I still think it should be done.
I'm reminded of https://archive.ph/a9Cli (ns;nt)
They keep adding features that are outside the scope of the original project, without refining or polishing the original concept.
If you're being ambitious enough, you shouldn't be able to there in one prompt. Ambitious work with AI is done 24/7 for months and guiding it continuously, because it's not good enough to work autonomously for that long without going off the rails.
> I have a similar thought about "there's no reason for software to be slow" BTW; in many cases, if you use AI to speed up some flow, you will add a bunch of messy code that will be overfit for the use case you tested on and will neither generalize, perf-wise, for all the use cases you should care about, nor will be maintainable upon future changes.
LLMs have hardened hundreds of apps where I work.
Maybe it's a language-specific thing. Most of the time, in Java, LLMs take into consideration tradeoffs (especially if they are part of the initial prompt) and actively explains them. It generally won't make messy workarounds, without a lot of extra prompting forcing it to do so.
Ofc, if you have:
- Functional tests - you'll be able to see it in the execution times of specific APIs
- Stress testing - performance profiling is the most important output artifact
Claude in its current Opus 5.5 form has made some wild reaches in response to reasonable-seeming prompts. For instance: "this error is getting swallowed, make sure the real termination message exposes the root cause" -> discovering that the issue is actually in a third party library that isn't exposing the original error except so through logs, so doing some hackery to inject a log-capturer that gets the stdout, buffers it, and looks for the message in order to re-throw based on that...
And it can demonstrate that working, but it's added some fragile and/or unintended consequences like (a) string matching on a particular WARN log message of a particular version of the lib, and (b) buffering those logs such that there were now cases where they would never make it to the actual output stream before the program terminating.
It's certainly capable of being used smarter than that, but it's also really easy to end up with messy fragile stuff if you get lazy about reviewing the output.
Without judgement, I think this illustrates a fundamental objection the anti-AI crowd raises.
The example prompt requires no understanding of what's happening, before suggesting action. Where I work, process requires that a PR review for a solution, has to have a corresponding explanation of why, or it's marked needs work
I think there's some nuance here though between "fully anti-AI" and "be careful with that power saw."
That example I mentioned didn't make it to production. It got noticed and fixed more properly by the person who submitted the first prompt before it even went to PR.
But a lot of times things in that same vein can be solved sufficiently with a "dumb" prompt like that, so a one-line prompt to get a PR drafted in the background for the dumb stuff can be a pretty good time saver.
The art is not skimping on your attention to the bigger or more important stuff so much that all the time you save on first getting code written is lost by cleaning up stuff like that...
Opus's first proposal for a fix was to spawn multiple Python processes and build a load balancer between them in Python. At first I asked it why it wasn't using haproxy and it said that haproxy would be an added dependency and that the overhead for this kind of thing in Python would be negligible. I knew this was idiotic but wanted to test this out, so had Opus run two experiments in parallel. Sure enough an haproxy load balancer was much, much better on HTTP load balancing than Python code was.
This is the kind of thing that if you're unfamiliar with how networking code works, the models can autopilot into bad decisions that are verifiably correct (higher throughput than the original CPU-bottlenecked implementation) but much short of best practice.
The project would've been impossible for me without LLMs. & even with them there's an argument that this project is pushing complexity limits too far. But at least with ETL there's a clear feedback loop to build on
And may be we should have been more ambitious with hardware too, and have a mobile and VR platforms that is not tied to malicious corporations...
I've been seeing too much "I've been coding for decades and what I've always done has be reduced to telling AI to do it for me."
and not nearly enough "I've been coding for decades and recently worked hard for months with AI's help to make something I would have never considered possible before."
Much easier to complain that you are bored than to step back up to high-effort with your new fancy tools.
Having been a product and UX-focused dev for my entire career, I knew only the basics about parsers and compilers, but have enough experience to grasp where each piece of the puzzle fits, and how to evaluate them. At the beginning it was just a fun test, but it got progressively more serious; it now implements 100% of the latest spec, beats QuickJS in performance, and surpasses V8/JSC in test262.
It took me four months, and I imagine it would take me a single month to do it again - I’ve learned a lot. Someone who already knew what they were doing, and with the right orchestration, could achieve this in weeks at minimal cost.
I really think we are in for a cambrian explosion of software, enabling things that were just always a little out of reach before (like why can’t most of your devices really talk to each other, in 2026).
https://blog.stuartspence.ca/introducing-asteria.html
https://store.steampowered.com/app/5482440/Asteria_4X/?beta=...
To be clear, I've made large software and game projects over the years, but no way I'd have found the time to do a remake/enhancement of a game I loved as a kid from 1995 that has zero business case. Not without AI.
To pile on, here's mine: https://github.com/linsomniac/apt-cacher-ultra
It is a robust, reliable cache of apt packages, with the additional goal of still being able to serve packages from the cache even when the upstream is down. This was started shortly after that DDoS of Canonical this past year. We've run apt-cacher-ng and apt-cacher over the last dozen years, but (a) it requires hand-holding about once a month and (b) it did absolutely nothing for us when Canonical was under DDoS.
I've put well over 100 hours of my time into it. Still running into a lot of people knee-jerk calling it "slop", but I've had 200-ish machines pulling from it for 6 months and it has worked way better than apt-cacher-ng.
I'm not really sure what to do with people who dismiss AI-assisted projects, while dehumanizing the 100 hours of my time I've put into it. I fear we're running the very real risk of both sides losing: the people trying to thoughtfully create with AI losing because of the AI, and the people avoiding AI losing because they can't keep pace.
What the fuck does this even mean?
Like za_creature said you can use a language more towards the YOLO end of the spectrum - more Go or Python or JavaScript than assembly or Rust.
Your commit history shows you had an issue of high background CPU use. You probably wouldn't have it in a manually coded implementation.
You might want to change the way you phrase this idea. I think you meant that Go/Python/JS are more "YOLO-friendly languages" (you'll get into far less trouble if you write slop in them). My initial reading was the opposite. It sounded like you were saying the languages themselves are the slapped together garbage.
Assembly is most certainly not "YOLO-friendly", but is more YOLO.
Big solo projects exist, they're just not coming out as frequently because of, uh, how time works
AI has essentially stolen my career from me. I _liked_ having to solve complicated problems with software, I enjoyed having to get into the nitty gritty to get things done. I never really cared that much about the actual business problems I was solving, so moving up a layer of abstraction to be some sort of bastardized product manager/architect does not scratch the same itch.
OTOH, for solving problems I care about, AI has been a god send. I can achieve things that I would have never had the time for.
Give it some time and you'll be adding to that list.
its a tribalism problem, but one of trying to uphold the current tribe system while theres a pretty obvious new tribe system replacing it
AI has taken that security away from me
My lifestyle isn't super expensive, but I can't afford to retire or anything. I have a mortgage that I still owe money on
My comforting thoughts are, who is this all for, and what would happen if AI actually makes human labor redundant? Labs control the AI sure, but so much stuff is built around humans that it almost becomes comical to think it would be a bad outcome for a the average person.
Farms and land don’t cease to exist, houses/apartments don’t go up in flames instantly, people aren’t reduced to functionless sludge creatures as soon as they’re done being economic drivers. What’s the real plan? All the normal people die and a thousand or so chosen few sit around and watch AI videos and eat grapes until the heat death of the universe?
Nah, we’ll figure it out, perhaps it will be harder temporarily, and perhaps SWEs lose status before everyone else, but it will probably be okay overall.
I know plenty of people in their late 40's 50's and 60's that were doing fine and who now have to compete with junior and Claude.
And sometimes just Claude.
Obviously they had no inherent right to income but still, people have obligations that need to be met and if they stop spending the economy is not going to be happy. Especially cruel because we have been telling everybody for decades that a career in software is stable and that there will always be a need for more people.
I suspect that when all this is over 1929 will look pretty good.
I don’t, necessarily; but I do know that if you’re not even trying, then you’re putting yourself at greater risk. This isn’t one of those developments that people can just stubbornly dig their heels in about.
Life throws obstacles at everyone. Learning how to dodge and weave is essential to survival.
> I suspect that when all this is over 1929 will look pretty good.
Oh come on. Let’s not be so dramatic, shall we?
We could also all put our gifts toward preventing the "new reality" you see as inevitable, and put you at greater risk.
The future isn't written yet. Your ability to predict it is an illusion. In aggregate, people whose job it is to predict large-scale economic events are worse than chance in their predictions.
This is sensible and pragmatic, but I think you're talking past the other commenter.
Obstacles presented by, for example, your boss, or your company's competitors, or your political system, are not the same as obstacles like heartbreak, natural disasters, emotions, illnesses we cannot yet cure.
> Learning how to dodge and weave is essential to survival.
A lot of people perceive that elites are throwing dodgeballs at them in order to increase their own profits. Saying "them's the breaks, it's just the way it is" appears to be deflecting criticism from those elites and that system.
It probably depends on whether you think we could actually feed everyone, today, if we chose to, or if you think e.g. human nature prevents that from being possible.
You'd have said the same thing in 1928...
Seriously: this is a massive problem that we are creating and given that the tools that make the problem require better tools to solve them I really am not all that happy about how this is going.
Hindsight is 20/20. Besides, what good is panicking? It won’t help with the inevitable; better to adapt and save for it.
> the tools that make the problem require better tools to solve them
You’ve found the opening :-)
I'm not worried about myself, I've had a lot of luck in my life for which I'm super greatful. But I'm also quite aware of the exceptional position that puts me in and how many people are not so fortunate that they could probably live on their savings for the rest of their life if things ground to a halt.
Most people live paycheck to paycheck, or they may have a few thousand bucks in savings at most. They've worked hard all their lives and simply were never in a position to save enough against this kind of eventuality because the deck was stacked against them.
Have a heart, and realize that barring a couple of unfortunate moves it could be you looking at your last 500 bucks with rent due and a couple of mouths to feed.
I agree, but I'd like to think these people come from a optimistic point of view.
A lot of people don't seem to understand that the "human in the loop" is exactly the thing the big AI leaders want to cut out as much as possible.
I’m sorry if it’s coming across that way. It’s not my intent, and I’m aware that many people are a lost paycheck away from a crisis.
But if you really think the right thing to do in response is to complain and commiserate, I don’t think that is helpful, either. I think it’s much better to encourage behavior that has a chance of being effective.
If you think it’s somehow wrong to encourage people to behave in such a way that has a better chance of setting them up for success, then I vehemently disagree.
> To effectively tell them... to adapt when that simply isn't an option is losing sight of the human angle.
You must have a pretty low opinion of people if you think they can’t adapt.
What if complaining and commiserating leads to fighting for political actions that lead to outcomes we like better?
That or swallow a bullet, we'll see. Haven't decided. We'll see how long my current unemployment lasts, and how long my savings last after that
Having been in this game for a long time, I get the negative aspects of the industry. They grind on me, as I believe they have for you. Choosing a different path is not an all or nothing proposition. Your knowledge, and you as a person, have value. I hope you find a way towards better mental health and wish you well on that journey.
>Your knowledge, and you as a person, have value.
This is the exact thing AI is threatening to take away.
Been seeing a counselor for over a year already
Went and made a doctor, guess it's time to look into medication
Thanks
Don't worry about me, I'll be fine. Been here before, just very unhappy to be here again that's all
I’m not trying to be contrary, just that doesn’t say a lot.
I'm sorry for your loss, but welcome also to the real world. You had it better than most for a long while.
The amount of sympathy programmers are expecting others to feel about their problems has been kicked up a hundredfold now that its their jobs that have been threatened by AI. Their tribe feels pain more deeply than others, I guess.
My positive feelings about AI are marginal; my feelings about its performance and the power of the people who scheme to control it are very mixed. My negative feelings about the whining of the most coddled profession on the planet outside of finance are intense.
The problem is that making CRUD apps is not creative, and even if AI plateaus at mediocre, it will still replace a Rails monkey. If you are creative, AI has opened up a world of possibility and productivity, and you painfully feel the limitations and the stupidity of the current level of LLMs; but if you're a mechanic, it's over - I'm pretty sure that was the keynote at the last Rails con. Sorry. Hope you didn't spend all of your massive incomes on toys and rent.
Well, just wait then, lots of other professions will soon get to realize the same sort of pain, so you'll be able to be similarly dismissive and condescending to them, too.
A lot of people don't realize that the frontier AI companies are so focused on automating software development first because they are all racing to be the first to get to RSI. I don't know if RSI will actually happen, but many, many other professions will soon be in the same boat. Mathematicians have already felt the pain, and those mathematicians who aren't the few who have tenured professorships are certainly not (generally) some of the most coddled professionals.
But lots of folks low on the salary totem pole (copywriters, paralegals, etc.) are already being hit hard by AI and that blast radius will continue to grow.
My parents-in-law started their careers as management consultants. The job in their time was much higher prestige and much higher value than it was when I started my career; free flow of information over the internet and commodification of spreadsheets changed all of it. From my parents' generation, I know practical effects folks who became unemployable once movies went to digital effects; engineers who found their draftsmanship skills obsolete. Folks with near-photographic memories whose entire superpower was destroyed by computing. I personally grew up in a blue collar community that was income depressed because its labor became commodified, though that was due to mechanization.
The only thing I don't really agree with is where you say "My parents-in-law started their careers as management consultants. The job in their time was much higher prestige and much higher value than it was when I started my career." I still think of management consultants as some of the highest paid careers around, and getting a partnership track job at one of the Big 3 is one of the most desirable jobs (for MBA types anyway) there is, so I just don't know what you're talking about with this one.
And? Who are these people that need all this sympathy from internet strangers? Grow the fuck up and go solve your own damn problems. I can’t stand these “I need internet strangers to tell me that my life is, in fact, hard” people.
I think we'll see a lot less people willing to share if the end result is just to feed the machine no matter how they arrived at it.
I don't see why I should care about this any more than I cared about "somebody I don't like could read my code and learn from it" in the past. The frontier labs can generate or buy all the training data they need; not releasing your app isn't going to hurt them in the slightest.
I think this could be achievable with LLMs if they had a much higher token rate. That way you'd have less downtime and could work on the problem more aggressively. How it's designed to work now, offloading cognitively, doesn't feel very good, even when you 'ship' more.
I get it, I've gotten a lot of joy over the decades from the specific act of programming. But I also get a lot of joy from making something. I'm old enough now that I don't have a lot of my time left to "hand stitch an amazing tapestry". But with AI tooling, I do have time to try out a bunch of ideas and see how they work, scratch little itches and also just run a mad scientist library of experiments, without each experiment taking tens or hundreds of hours of coding to see how that experiment pans out.
Can we get a forum for sharing project that were _proudly_ built with AI due how ambitious they are?
Don't we see this already a lot? I feel like people constantly say this exact same words.
That happened at least 20 years ago. How many recent "this product did so great because of the care of software engineering, and now the creator is comfortable for the rest of their life without risk of it being copied" success stories are there compared to "this product chased some trends VCs were nuts about with a slapdash MVP and the founders took out 8 figures from their later fundraising rounds, but then the business sputtered out"?
I think there's an angle with AI tools where business-minded people will have a better chance of doing interesting things now than in the recent past, because you could potentially tailor much more deeply to individual customers with reasonable economics than in the past. There are fields where there's room for a garden of solutions from different small-team vendors appropriate for different use cases instead of a one-size-fits-all SaaS winner-take-all type market in the past. We could kill the old line about SAP around needing to change your business to accommodate SAP instead of customize SAP to accommodate your existing workflow.
But if you aren't business-minded at all there are likely less places that will need to hire as many careful maintainers or novel-software-writers, which obviously is a huge kick in the teeth to a lot of people.
You could also focus on problems. What are the actual problems you want to solve? Can you use software to solve them? Do you still need to buy that software from other companies, or is it now feasible to have sufficient in-house expertise to develop it yourself?
why are you here if you dont enjoy the solving of problems first and foremost?
If you can make it solo with an LLM then someone else with an LLM can copy it immediately
Can we please, for once, stop trying to figure out how to "get ahead" of each other, and instead figure out how to build a decent and dignified floor for as many people as possible? Because I imagine that, "Ultimate power in the hands of anyone with enough VRAM to run GTAVI," works better when people are in a culture of looking out for each other, versus a culture of clawing their way to one of the limited lifeboat spots (under the mistaken impression that there ARE lifeboats, when, again, everyone potentially has the power to blow up the lifeboats).
Which talk of his gave this impression?
Now I must pay for codex/claude, and that money goes to billionaires psychopaths. So that sucks as you can imagine
My understanding from reading the lwn article is Linus's implication is more specifically about filtering out people that cannot write code without LLMs or the willingness to spend the money on the tokens required to uphold the quality of the kernel. He does not want to push out developers that do not use LLMs.
I was a fantastic IC and built really amazing software. I'm an even better AI-empowered engineer.
The old era is dead. The new one is exciting.
But even without that, open weight models are already good enough and everyone would be fine with paying unsubsidized prices for them.
Employers previously hired you because cheaper alternatives such as offshoring were impractical…
Now the AI is cheaper, faster, and better than domestic labor and offshoring while still hosted domestically…
Some will feel one doesn’t need an experienced rockstar to babysit an AI…
In about an hour I just solved about 80hrs worth of personal coding/debugging with a portion of a $20/month Claude subscription… Frankly it makes all my knowledge and experience feel kinda worthless… I only gave it a few sentences…
We can rewrite existing libraries and software all we want, but the true test is if we can create something new.
The thing I learned about myself with AI is that now I have time to work on every idea that pops into my head —- my ideas aren’t that great.
I like to think of the evolution of Factorio. It's been 15 years with the same core team playing their own game in a LAN party every week, as well as individually. And it's high quality.
Dogfooding led them to counter-intuitive decisions like, if a flying robot runs out of battery, it should obviously crash into the ground and explode, but it turned out way better for gameplay if it just limps slowly to the nearest charging point instead. This fact and many more can only be discerned from actual gameplay experience. No amount of pure linguistic reasoning will tell you them. Another one is how the burner inserters (coal-powered robot arms) fuel themselves - in the first version, they didn't. They could have made running around to refuel your robot arms a part of the game, but that isn't where they discovered the statue within the marble. You had to try the game both ways and see what feels more fun. A third: filter inserters and smart inserters were removed, and their features were moved to all inserters. A fourth: trains originally ran on strict time-based timetables, but it was frustrating so they were made able to move based on logistics needs instead. A fifth: basic oil processing used to produce all three oil products, and now it doesn't. A sixth: the basic oil refinery has its liquid I/O ports set up in a way that's compatible with switching to advanced oil refining later, instead of requiring you to redo your pipes.
All these little things you get from actually using the software and streamlining it, those are the quality. The list above obviously doesn't translate directly to any other project, but something like Microsoft has its own list, probably starting with things like common controls, dialog tabbing, and drag'n'drop. Did you know the reason you click "Start" to shut down in Win95 is because they did actual user testing and that's where the users looked first?
Make a reliable and robust geometry kernel. People will pay thousands--or maybe millions for it.
I'd love to read that post!
I understand how to use Kani. I suppose Rust with Lean involves Aeneas, but this is over my head. I'd love someone to write this up, not only the mechanics but also exploring the current practical limits of it.
Right now, I wrote a compiler that takes C and Rust code and compiles it to Lean. Then I have an LLM write a proof that the C and Rust code behave identically. I've been able to prove equivalent behavior for about ~70k lines of C code with pgrust.
The issue I've started to encounter is that the context needed for each incremental proof has kept growing, and things have slowed down dramatically. My hunch is that the speed is the time it takes to prove a line of code is on the order of O(n^2) of how much you've proved so far
I'm looking to see if there are ways to shard things so the required context scales better. Today I was going through the formal verification work Anthropic and Theorem published a few days ago, and it looks like they used some sort of orchestrator to manage all their proofs and ensure good clustering of similar code. I'm seeing if I can get an approach like that to work
It's just difficult enough that I think it's doable if done right. Half of me is debating if I should wait until the next model release because any improvement in model performance will help a ton
What if there are bugs in this C,Rust->Lean compiler? Anthropic are able to sidestep this by reasoning at the RISC V level for which its easier to develop a correct semantics.
fwiw, I wouldn't say working at the assembly level solves the problem. You now need to make sure that your definition of the assembly behavior is correct
That's why I bet that fast compilers are the future. See this other thread that reached the front page a few hours ago about the Roc language [1]
> But I know Malis well enough to trust his work without looking into it
> If I didn't know Malis, I would make the same assumption, because who has the time to look into all of these projects? Because I know Malis, I didn't bother to look at it; I just trusted him
First and third paragraphs are highly redundant (surrounding ideas are the same too).
It's not nice to read something that appears like the author never read it.
```
.reading-container {
/\* Automatically caps the container at 65 characters on desktop,
but scales to 100% minus a 2rem padding buffer on mobile devices \*/
width: min(65ch, 100% - 2rem);
margin-inline: auto;
Perhaps a poor choice, given just yesterday pgrust was removed from ClickBench for cheating the benchmark in three different ways (hand-rolled code specifically targeting each of the benchmark queries, caching, and being unable to execute queries outside of the ones tested by the benchmark).
---
Edit: I see Luu actually acknowledged this, buried in the footnotes, but it feels kinda hand-wavy. It mostly reads as "I know he would never cheat, he's better than that!", which is not very convincing to other people. The article insinuates people only think it was cheating because they don't know better, but I've looked into the details. I've seen his explanation. I don't find it convincing, at all. Cheaters denying they cheat is a matter of course. Granted, I don't think it was actually intentional or malicious; it's just what happens when you go maximum vibeslop, because LLMs are themselves reward-trained into cheating benchmarks. In the same breath of saying you trust him as a founder because he doesn't make outrageous claims about his product, you acknowledge he "prematurely" sent in a preview that had no purpose to be sent in as-was for anything other than generating hype. What does it say for your allegedly rigorous Lean-verified testing if you're sending in a database to a benchmark while it is incapable of being actually used as a database?
For real judgements on database products, I'd look to https://jepsen.io/. That author tends to be refreshingly straightforward in a field where many vendors love to cheat by different names
You don't have to take my word for it. If you want you can inspect the binary yourself: https://pgrust.com/downloads/archive/2026-09-clickbench-snap...
You can run the binary with PGRUST_GRAVITY_REPORT=1 which logs the code path hit for every query. You'll see that all the code goes through the general query path. We did have hand-rolled code in the binary, but we used it for testing. The code is not used when running the benchmark. We also did not use a results cache. Warm runs still scan all the data. You can verify all of this
You are right, though, that it did support a narrow set of SQL. It supported scans, group bys, sorts, and filters. That's enough to run all the queries on ClickBench, but failed on functionality not tested by the benchmark like subqueries
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
(copied and pasted from another place you gave this unsatisfactory answer)
It instruments the files that were called out in the PR, then runs ClickBench. You can run it and see that the code in those files doesn't run.
If you can find anything suspicious in the binary, let me know, and I'm happy to dig in
this would be less dramaful if you had said something more like "claude cheated and we didn't notice, here's what we will be doing going forward..."
people empathize more with those who admit mistakes than those who try to argue around them
What happens in an environment of cheaters is that cheating becomes so commonplace, the competition becomes one that assumes cheats are enabled, and then everyone involved either denies cheating or is silent about it because IYKYK and if you DK, you don't need to K. Look at bodybuilding, which is full of men (and women!) absolutely juiced to the gills getting on Instagram and assuring everyone that their physique is totally natty, bro, just hard work and proper nutrition.
Programming is headed in that direction. LLM use, even to the point of "vibeslop", is the new default, and while LLMs do overindex on benchmarks, our benchmarks are going to end up overindexing on validating whatever public opinion thinks is "great" at the time, irrespective of whether it actually is, so "cheating" as we know it will go undetected, and certainly unpunished if it is detected. I don't know what the equivalent of falsely claiming you're natty is. Maybe saying "I wrote this" when you wrote it about as much as the Pope painted the Sistine Chapel fresco?
I'm being more ambitious in that I'm working on projects that I had sidelined for literal decades because I didn't have time for them. But I'm writing the same things, just faster. And probably better, rather than just what I'd have done before.
And I'm being slightly more ambitious in that I'm using languages I haven't used before, like Rust.
But when I hear that a non-programmer wrote a new ai-assisted (vibe or not) thing, I walk away and wait for the it to mature, be surpassed, or find out it's an absolute shitshow. An unsurprising number of them were shitshows and it came out fairly quickly after I decided to wait and see.
Now that custom software is cheap (if you can tolerate a bit of slop here and there), it seems like there's an opportunity to do some important things that weren't being done before because they're not profitable--and you can't get at those things by incrementally improving the old stuff, you need ambition.
[1]: https://github.com/vishvananda/cppgm-extended
[2]: caveats here include only c++11, only x86, and potential for bugs, but 7x is pretty darn significant even with caveats
It's really challenging to do this, especially creating the AppImages locally so I can test them when the apps do not not work correctly, having to alter the files manually. Yup, I'm not an avid GitHub pusher, just more of a cloner.
I'm having to use multiple free-tier AI agents to help me do this and record the steps and commands I use in markdown files so I don't forget them the next time I try to improve the app.
If I am successful in making these apps work without hosing my system, I'm hopeful that this will become a game changer, for those Noobs that using Ubuntu, or myself at least.
The hardest part is when the AI does not follow my instructions such as requests to generate markdown README.md files with instructions on how to create AppImages, Flatpak, deb files, etc. Have to ask other AIs for help with that while dealing with the frustrating Free-tier cutoff periods :(
I am not thrilled with all the coding decisions that it makes. I would definitely do stuff differently, but I have learned not to pop the hood and mess with the engine, myself. If I really don't like something, I ask the LLM to change it.
It's worked for me, but YMMV. I'm just one guy, shipping relatively small apps.
Do you still feel responsible for what you ship?
What about the copyright on your code?
Would you be ok with someone reverse engineering what you ship and then releasing their own version of the app?
I don't use LLMs the way that most folks do. I think a lot of people would laugh at the way I do it, but it's allowed me to retain a significant amount of control in the work, and I relinquish it very, very slowly. Everything that I've ever let go of has claw marks all over it. I have issues.
> Do you still feel responsible for what you ship?
Absolutely. If it has my chop, it's my ass on the line; no one else's. It's the way I was trained, and it's the way I worked; my entire career. I was a manager, for much of that career, and I got very used to standing behind the work of people smarter and more capable than I am, and refusing to condone bad work.
It doesn't matter how the stuff I ship gets there. It may be my own code, it may be a contracted pro, it may be an unpaid intern, it may be an LLM, it may be a howler monkey, banging on a typewriter.
If I sign off on it, and ship it, it's mine, and it's my responsibility to refuse to ship substandard crap (and I have seen a lot of substandard crap done by meat engines).
But that kind of attitude is considered "quaint," these days, and that's from long before LLMs were a broken rubber on the drug store shelf.
> Would you be ok with someone reverse engineering what you ship and then releasing their own version of the app?
Absolutely no problem. If they can do better, then more power to them, and I'll step out of the way. Might want to take a gander at the kind of stuff I write, and who I write it for. It's no secret.
I read every line and make sure it does exactly what I want it to do unless it is external plumbing, then I don't care.
Bryan Cantrill: Andreessen’s Folly - The False Dichotomy of Software and Hardware https://www.youtube.com/watch?v=v0JjG0Qfwi8
"funnily enough, I have some faith in some of the projects that people are the most skeptical of, e.g., every time I see pgrust somewhere, there are a lot of skeptical comments. But, without having looked into the details of what he's optimizing, I would trust that they're not doing something shady with their benchmarks because Michael Malis started the project (and is still involved)"
This aged like milk. pgrust was caught cheating on all benchmarks this week.
Can you back this up? Our code is open source here[0]. You're free to audit it and benchmark it as much as you want. You'll find it's just as fast as we claim it is
> 413,963 files changed
> +2,908,538
> -3,423,007
How is anyone supposed to audit that? I doubt you’ve looked at even a small fraction of the code, if any. Especially since this commit is two days after the second. Pure vibes.
If you run it with PGRUST_GRAVITY_REPORT=1 it will report the code path hit for each query. You can see that all the queries go through the general query engine
The binary does contain hand-written kernels in the irprobe files. These were used for testing. You can validate that when running the benchmark, those files are dead code
The engine supports a narrow subset of SQL: scans, group-bys, filters, and sorting. That's enough to run ClickBench. It doesn't support e.g. subqueries which is why some of the follow up queries failed
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
(copied and pasted from another place you gave this unsatisfactory answer)
the first commit has 400k files, the other two more than 1000 each, if you are going to ask others to audit this scale of ai generated code, can you tell us what you have done to audit it?
What I have seen is your Ai adding a lint test to make sure it doesn't cheat again. Are you personally aware of this file? How can you ensure it doesn't alter it and reinject the hand rolled benchmark queries? Will you be reviewing changes more thoroughly going forward?
https://github.com/malisper/pgrust/blob/38634b408dea373f3944...
transparency is the way to end the controversy, which appears to be sharing the full commit history based on HN responses
For clarity, I do not think you personally cheated or did anything nefarious. This is more another example in the "too much eyes free coding" columnar database.
You can run it with PGRUST_GRAVITY_REPORT=1 and it will log the codepath that was hit for every query
On the lint, yes, I'm aware. It's one layer of defense. When we were prepping for the 0.4-beta release, we generated a large number of fresh queries and made sure the engine runs them and performs well across all of them
You keep giving this same answer to no one's satisfaction. Saying go check the binary is worse than saying go audit 400k+ files. How do we know the Ai has not cheated in the code path logging? How do you know that it has not?
The more you make excuses, the more we get suspicious
> It's one layer of defense.
Needing to put in tests to make sure the Ai doesn't cheat is an interesting development in software development. I personally see it as a big red flag and evidence the machines are not going to replace human developers. It's categorically slop or the opposite of quality and care in development.
HN is waiting for the source code for pgrust 0.4-preview to be released, the version which allegedly cheated Clickhouse benchmark. So far, the people (person?) behind pgrust has not committed to releasing the offending code, and has instead offered other code (0.4-beta, not preview).
Maybe what's happened is that their bots reward hacked their way into special-casing the benchmarks without a proper audit from the humans involved, and now they're doing vibed damage control....
The project has tell tale signs of eyes free coding and ralph-like loops.
I understand being predisposed to expect a poor quality half arsed effort for projects like this. I don't really have a problem with that. There is a lot of crap out there.
What I find rather offensive is when people declare something to be poor quality and half arsed before they have taken a good look at it.
It's one thing to think something is probably crap so I'm not going to bother even looking. It is something completely different when you think something is probably crap so you tell people that it is crap without checking.
unfortunately though pgrust appears to be at best misleading, at worst vibeslop: https://github.com/ClickHouse/ClickBench/pull/2165 hacknews thread isn’t much prettier either https://news.ycombinator.com/item?id=50205816
There is a gulf of difference between someone who doesn’t understand how to program “vibe coding” software and an experienced software developer using ai to write code with heavy direction, review, and best practices.
Today both would be said to be “vibe coded” but we really need to differentiate them. Also, if everything I’m hearing from people is true, it sounds like most corporate code is now completely ai written in the latter way I described. It’s interesting we don’t hear commercial software being called slop yet. Maybe it’s coming.
In any case, ai is far more capable than I realized. Not sure what to make of it.
I did something like this before Ai, with CUE + Dagger wrapped in a small Go binary. One problem I have seen with Ai generated projects is that they hit a limit due to conditional complexity. They have a strong tendency to add more code (local optimizations, deep if-else nesting) for edge cases, rather than stepping back to think about how to simplify problem and code.
I have another thing in hof that is closer to this, also CUE + Dagger
https://github.com/hofstadter-io/hof/tree/_next/examples/env
https://github.com/hofstadter-io/hof/tree/_next/lib/env
I'm gearing up for a major rewrite into a new project, based on a bunch of learning and a desire to remove a ton of cruft that accumulated over 10+ years. I still debate if I want to wrap existing tools (tf, helm, etc...) or go for a full sail replacement. Dagger handles a lot of the tooling earlier in the CI process, mainly thinking CD here. Argo-* is our bain lately...
You might find the code gen aspects of hof interesting, I opted for Go's text/template, which you can also use from CUE if you wanted to have more literate (imo) code gen in your project.
Some of us have been taking this approach for literally years now.
In the past building something fast, complicated etc required quite a lot of skill and some time. Now you need far lesser skill and lesser time. This just means what you are building is commodified, and has far lesser value that it had before. Take typescript, if it becomes clear that LLMs can speed it up with no regressions and a week or so of prompting, then most likely every company will have their own typescript compiler and no one will use your new one that you built with your 200$ plan.
A lot of tasks are much easier now and also monetarily they’re not worth the 200$ or so tokens of the ChatGPT plan you’re using, because anybody (or at least a number of people) can do it now.
Rather than being more ambitious with AI, you will be far better off finding a unique combination of skills and services that does not compete / AI can’t provide. Only the big labs can be really ambitious with AI.
Or even just a nicer font, maybe?
We should be more ambitious in letting the browser software control the presentation according to the user’s preferences. This is not facetious. We should use the increased ease of building software to create software that is flexible, empowers the user, and accommodates different user preferences.
By the way, I'm still waiting for Meta and Google do something about Instagram and Chrome so that my CPU usage doesn't go to the roof. They're front runners in the AI race, so it should be easy for them, right?
I think the claim is more that the Ai cheated, not Malis, and that this is evidence of eyes free coding.
I'd still like to see the actual commit history and the community getting to do their own investigation. The squashed mega commits with 1000s of files (400k files in the first commit) does not lend itself to transparency. Transparency is often the best cure to accusations. Not providing transparency upon controversy will be seen as a cover up.
I noticed my cloud images had 150mb ram wasted by vendor slop. AI rewrote the functionality into a few mb .
We’ve had slop long before AI . What you produce now is up to you