Join GitHub today
GitHub is home to over 31 million developers working together to host and review code, manage projects, and build software together.
Sign upbpo-35975: Support parsing earlier minor versions of Python 3 #12086
Conversation
gvanrossum
added
the
DO-NOT-MERGE
label
Feb 28, 2019
the-knights-who-say-ni
added
the
CLA signed
label
Feb 28, 2019
bedevere-bot
added
the
awaiting merge
label
Feb 28, 2019
This comment has been minimized.
This comment has been minimized.
I could use some help finding the cause of the one remaining test failure -- test_parser.py computes the size of tree nodes differently somehow. [UPDATE: Fixed it, I think.] |
This comment has been minimized.
This comment has been minimized.
The main thing left to do is adding tests. There are some tests in typed_ast that I can migrate over. But first I think we should complete the debate over on bpo about whether this is desirable at all. |
ilevkivskyi
approved these changes
Mar 3, 2019
Thanks! This looks very clean, I just have few minor suggestions. (I also there is a merge conflict now.) |
return compile(source, filename, mode, flags) | ||
return compile(source, filename, mode, flags, | ||
dont_inherit=False, | ||
optimize=-1, |
This comment has been minimized.
This comment has been minimized.
ilevkivskyi
Mar 3, 2019
Contributor
Why we now need to pass these two extra arguments? Maybe add a comment?
This comment has been minimized.
This comment has been minimized.
gvanrossum
Mar 5, 2019
Author
Member
Hm, those are the default values that I somehow copied when cleaning this up. I'll remove them -- we just need feature_version
.
/* Async comprehensions only allowed in Python 3.6 and greater */ | ||
if (is_async && c->c_feature_version < 6) { | ||
ast_error(c, n, | ||
"Async comprehensions are only supported in Python 3.6 and greater"); |
This comment has been minimized.
This comment has been minimized.
ilevkivskyi
Mar 3, 2019
Contributor
Indentation here and in several other places is a bit unusual, I would rather justify it after (
.
This comment has been minimized.
This comment has been minimized.
gvanrossum
Mar 5, 2019
Author
Member
OK, fixed. My guess is that at some point in the past that function had had a name that was 2 characters shorter. :-)
@@ -1746,6 +1758,12 @@ ast_for_funcdef_impl(struct compiling *c, const node *n0, | |||
node *tc; | |||
string type_comment = NULL; | |||
|
|||
if (is_async && c->c_feature_version < 5) { | |||
ast_error(c, n, | |||
"Async functions are only supported in Python 3.5 and greater"); |
This comment has been minimized.
This comment has been minimized.
gvanrossum
added some commits
Feb 21, 2019
gvanrossum
force-pushed the
feature-version
branch
from
87c80c1
to
ae36b7b
Mar 5, 2019
gvanrossum
changed the title
bpo-35975: [WIP] Support parsing earlier minor versions of Python 3
bpo-35975: Support parsing earlier minor versions of Python 3
Mar 5, 2019
gvanrossum
removed
the
DO-NOT-MERGE
label
Mar 5, 2019
This comment has been minimized.
This comment has been minimized.
OK, this is ready for final review and merge. (Sorry that the rebase lost some of the review history, I'm used to different tooling.) |
This comment has been minimized.
This comment has been minimized.
(Well, I promised tests. Upcoming. Docs are already done.) |
ilevkivskyi
approved these changes
Mar 7, 2019
Thanks for the updates! All looks good, I have few optional suggestions. |
@@ -145,13 +145,18 @@ and classes for traversing abstract syntax trees: | |||
modified to correspond to :pep:`484` "signature type comments", | |||
e.g. ``(str, int) -> List[str]``. | |||
|
|||
Also, setting ``feature_version`` to the minor version of an | |||
earlier Python 3 version will attempt to parse using that version's | |||
grammar. For example, setting ``feature_version=4`` will allow |
This comment has been minimized.
This comment has been minimized.
return ast.parse(source, type_comments=True, | ||
feature_version=feature_version) | ||
|
||
def parses(self, source, minver=lowest, maxver=highest, expected_regex=""): |
This comment has been minimized.
This comment has been minimized.
ilevkivskyi
Mar 7, 2019
Contributor
parses
is not very descriptive and is easy to confuse with parse
, maybe parse_all
, or parse_all_versions
?
fstring = """\ | ||
a = 42 | ||
f"{a}" | ||
""" |
gvanrossum commentedFeb 28, 2019
•
edited by bedevere-bot
This adds a
feature_version
flag toast.parse()
(documented) andcompile()
(hidden) that allow tweaking the parser to support older versions of the grammar. In particular iffeature_version
is 5 or 6, the hacks for theasync
andawait
keyword from PEP 492 are reinstated. (For 7 or higher, these are unconditionally treated as keywords, but they are still special tokens rather thanNAME
tokens that the parser driver recognizes.)https://bugs.python.org/issue35975