If we have a compiler for language X in language X, and don't use any tools such as a parser or lexer generator, then there is nothing but code in X in the project, and that makes the developer of language X feel like they have earned major brownie points ... err, I mean, ... that they have kept their project free of cumbersome dependencies.
If language X is a one-implementation invention, then the tooling won't exist which hits these checkboxes: (1) is written in X; (2) generates code for X. By the time language X is mature enough that its ecosystem has something like that, it is long past the point where it would make sense to introduce it into its one and only implementation. There would have to be interest in writing another implementation.
E.g. the first C compilers would never have used Yacc.
Among languages that have one implementation, and that use parser generators, we will almost always see that another language is used for bootstrapping and the generator is for that language.
For some designers, that is a bruise to the ego, or else an unappetizing dependency. Even if they are boostrapping with another language, they are thinking forward to a future release where they will ditch that: they will rewrite parts that are in the boostrapping language in the new language to make it self-hosting.
If you use tooling like parser generation, which is in the ecosystem of, and oriented toward, the to-be-jettisoned-one-day boostrapping language, that throws a barrier in the path toward self-hosting. So you tend not to do it.
Like if you are boostrapping with C, and have it in the back of your mind to get rid of it, do you want to be bringing in a complicated tool with its own input language, which generates C? You think twice and are more likely to go ahead if you've resigned yourself to sticking with the C dependency.
But "tolerant" isn't the same as "as good as hand-written." A hand-rolled recursive descent parser can say something like "missing semicolon after return statement" because the code knows exactly what construct it's in.
A generated parser's error is usually derived mechanically from the state machine, "expected one of: X, Y, Z, got W", which is correct but generic. This is what Yantra does at the moment. Closing that specific gap would mostly require hand-authored, context-specific messages layered on top. But its a good problem to solve.
For yantra specifically, it doesn't have error recovery at all yet. A syntax or lexer error just stops parsing at that point, no resynchronization, no continuing to find more errors in one pass. It's a known, documented gap, not something I'd claim is solved. For the kind of smaller or evolving DSLs this is aimed at, that's probably an acceptable tradeoff, but it does exist as a limitation.
> A generated parser's error is usually derived mechanically from the state machine, "expected one of: X, Y, Z, got W", which is correct but generic.
You are not describing the difference between hand-written and generated but rather top down vs. bottom up.
There may be some confusion here because hand-written parsers are almost always recursive decent (a form of top down), while the most popular parser generators (yacc, bison) are bottom up. However hand-written bottom up parsers do exist (Pratt parsing, recursive ascent), as do top down parser generators (ANTLR, Coco/R, Chumksy). Chumsky in particular gives pretty decent error messages out of the box.
Bottom up parsers can parse a larger set of languages than top down parsers, but the tradeoff is you don't know what you are parsing until you successfully reduce the rule (unlike top down parsers). This is why errors in bottom up parsers often lack comments on structure/context like "after return statement" and can only give a list of which alternative symbols would have been valid.
this is extra helpful if the parser generator also generates intuitive language bindings to, e.g. python or TS.
One thing worth separating out though: recursive descent is already top-down, so it gets "build the tree, then decide what to do with it" for free, the same way ANTLR's LL(*) does. The intresting part of what I built is that it's getting that capability while keeping LALR's bottom-up table-driven parsing.
Where a grammar-first tool like this is more useful is smaller or evolving DSLs, where you want the grammar as a readable, declarative spec with automatic conflict detection instead of hand-tuned lookahead logic, and cases like generating multiple outputs (e.g. C++ and Java) from one grammar, which is awkward to bolt onto a hand-rolled parser after the fact.
Of course, it's not the same as having the parse tree all done from a previous pass and just walking it to do semantics.
But as you said, it's still one linear left-to-right pass. Appreciate you drawing the line precisely.
then Yantra would become great to parse well-known structured data from python at low-level speed.