Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Re-using nested parse elements with different parse actions #189

Open
TMiguelT opened this issue Mar 12, 2020 · 3 comments
Open

Re-using nested parse elements with different parse actions #189

TMiguelT opened this issue Mar 12, 2020 · 3 comments

Comments

@TMiguelT
Copy link
Contributor

@TMiguelT TMiguelT commented Mar 12, 2020

Lets say I want to make a library of parser elements that are not tied to a particular processing action, and then I want to use those elements for two different use-cases. In this case, evaluating math as if it's in python 2 vs python 3:

from pyparsing import *
import operator

number = Regex('[0-9]').setParseAction(lambda s, lok, toks: int(toks[0]))
op = Or(['+', '-', '/', '*']).setParseAction(lambda s, lok, toks: {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.truediv
    }[toks[0]])
expression = (number + op + number).setParseAction(lambda s, loc, toks: toks[1](toks[0], toks[2]))

def python3Eval(expr):
    return expression.parseString(expr)

def python2Eval(expr):
    # Use floordiv instead of truediv because Python 2 sucks
    py2Operator = op.copy().setParseAction(lambda s, lok, toks: {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.floordiv
    }[toks[0]])
    
    # Even though I only redefined the operator parsing, I have to redefine the entire expression class as a consequence
    py2Expression = (number + py2Operator + number).setParseAction(lambda s, loc, toks: toks[1](toks[0], toks[2]))
    return py2Expression.parseString(expr)

See how, even though the actual token descriptions are correct for both languages, the parsing behaviour must be different. But there's no neat way of saying "I want to re-use the entire expression tree, but setting the operator parse action to something different". I'm encountering this in another non-trivial situation I'm using the parser for, as well.

@TMiguelT
Copy link
Contributor Author

@TMiguelT TMiguelT commented Mar 12, 2020

It's a bit nasty, but I suppose one solution to this might be creating a function like element.alterChild(oldElement, func).

I'm envisaging that alterChild(oldElement, func) would recurse down the ParseElement tree until it finds any instance of oldElement, and then replaces it with func(oldElement), allowing the user to attach a new setParseAction on it. I guess there would also have to be the facility to specify just one branch of the element tree in case they don't want to modify all instances of that class.

For example, it would work like this for the above example:

def python2Eval(expr):
    py2Expression = expression.copy().alterChild(op, lambda oldOp: oldOp.copy().setParseAction(lambda s, lok, toks: {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.floordiv
    }[toks[0]])
    
    return py2Expression.parseString(expr)
@ptmcg
Copy link
Member

@ptmcg ptmcg commented Mar 12, 2020

What if you use a Forward for the binary operator, as a late-binding expression to be defined just before parsing:

from pyparsing import *
import operator

number = Regex('[0-9]').setParseAction(lambda s, lok, toks: int(toks[0]))

# use oneOf, not Or
opSymbol = oneOf(['+', '-', '/', '*'])

py3op = opSymbol().setParseAction(lambda s, lok, toks: {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.truediv
    }[toks[0]])

# Use floordiv instead of truediv because Python 2 sucks
py2op = opSymbol().setParseAction(lambda s, lok, toks: {
        '+': operator.add,
        '-': operator.sub,
        '*': operator.mul,
        '/': operator.floordiv
    }[toks[0]])

# define operator using a Forward - an expression to be named later
op_fwd = Forward()
expression = (number + op_fwd + number).setParseAction(lambda s, loc, toks: toks[1](toks[0], toks[2]))

def python3Eval(expr):
    op_fwd <<= py3op
    return expression.parseString(expr)

def python2Eval(expr):
    op_fwd <<= py2op
    return expression.parseString(expr)

Would this work for you?

@TMiguelT
Copy link
Contributor Author

@TMiguelT TMiguelT commented Mar 12, 2020

That's certainly an interesting solution. It's a tad ugly because I have to define the Forward() (op_fwd) and the actual tokens (opSymbol) as separate variables, but maybe that's fine (?). I'll try it out and see if I have any issues. Thanks for the answer!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked pull requests

Successfully merging a pull request may close this issue.

None yet
2 participants
You can’t perform that action at this time.