Writing about pratt parsing so I remember it(www.unix.dog)

58 pointsby dverlaeckt808 hours ago2 comments

mjbq3 hours ago
Could I suggest also looking at a sed-inspired parsing language (a DSL) which is called nom and is documented at http://www.nomlang.org An example script for parsing symbolic expressions is at http://www.nomlang.org/eg/maths.parse.pss and another which turns the expressions into LaTeX is /eg/maths.tolatex.pss
Nom may seem quite verbose. I have written another language (which compiles to nom) which I call Syntagma. The syntax of syntagma is much closer to EBNF but also has powerful additions, such as a "lookahead" syntax, a negated lookahead syntax (eg: "parse this as an expression if it is not followed by a multiply operator"). Also, Syntagma has a "condition" syntax, meaning that a parse rule will only reduce if the attributes of the parse tokens match certain conditions.
An example of an expression parser written in Syntagma is at http://www.nomlang.org/syntagma/eg/expr.sta This is much more succinct than the equivalent nom script. The script /syntagma/eg/expr.toforth.sta converts symbolic expressions (and assignments) into Forth.
Parsing is an interesting topic, but I feel that it is quite difficult or non-intuitive with some of the traditional tools and methods (at least for me).
jstanley5 hours ago
Personally I never found the shunting yard algorithm very intuitive at all.
A recursive descent parser makes much more sense to me.
- throw-qqqqq3 hours ago
  For me, the win is that a Pratt parser is more data-driven.
  It’s easy to look at a table or map of operators and their precedence and convince myself that it’s correct. A recursive descent parser needs to be very clearly structured to have similar clarity, IMO.
  You can easily combine Pratt parsing with recursive descent, using the Pratt part only for expressions. FWIW, this is my favorite approach for parsing C.
  - bonzini2 hours ago
    Pratt parsing, in its simplest form, is essentially an optimized recursive descent that uses lookahead to skip levels. Which as you say ends up being clearer too.
  - arnsholt2 hours ago
    Yeah, for “normal” syntax I’d generally go for recursive descent, but for expression parsing with operator precedence I’d implement Pratt parsing if I had any say in the matter.
    throw-qqqqq2 hours ago
    I think of it as expressions vs statements, but I think we agree.
    Lexing for tokenization -> recursive descent for statements -> Pratt parsing for expressions.
    S-Expressions and prefix-notation are so easy to work with (compared to in-fix IMO), because the precedence is explicit.