Join GitHub today
GitHub is home to over 50 million developers working together to host and review code, manage projects, and build software together.
Sign upGitHub is where the world builds software
Millions of developers and companies build, ship, and maintain their software on GitHub — the largest and most advanced development platform in the world.
Fully incorporate the code from Python 3.7.2 #78
Conversation
The graminit.c was built using a tweaked copy of the 3.6 pgen, with changes from these these token.h and tokenize.c files.
Update to better (though now failing) tests, .travis.yml and .gitignore
|
OK, this is now ready for full review. Note that I dropped support for Python 3.3 -- it's been dead long enough. I'm keeping 3.4, since it still has some life in it (though the last few Windows issues only occurred on 3.4). |
Hm... I got it working for 3.7 on Windows in the next diff. The remaining 4 diffs (PyOS_Readline through More C89) are all just to support 3.4, somehow. |
|
Does anyone want me to clean up these diffs? I suppose it's possible with creative use of |
|
I probably should add code to appveyor.yml to run pytest too. |
|
Also requesting review from @ethanhs (you're not in the python org so I can't request it from the menu). |
This code definitely isn't backwards compatible for 1.2.1, and I'd like to reserve 1.3.0 for PR #75. Also change for format (-dev to .dev0) to prevent complaints from setuptools.
| @@ -0,0 +1,329 @@ | |||
| #include "Python.h" | |||
gvanrossum
Jan 22, 2019
Author
Member
Oh, you should probably ignore everything I did to tools. This was an attempt at automating the update process more, but it's really not working very well yet, and I should just redo that if we even need it.
If all goes as I hope, Python 3.8 will support type comments natively, and then extracting the upstream ast module into typed_ast will be much simpler (though we'll still need to do the work to backport the code to earlier CPython versions and C89). Even if this doesn't happen, we should probably keep a branch of CPython live and only do purely mechanical transformations from there to typed_ast -- merging upstream into that branch should be simpler that the current update process.
…s were correct, just Grammar itself wasn't)
|
This looks good! Thanks for doing this. I've got a few requests for tests and documentation. |
| @@ -0,0 +1,78 @@ | |||
| diff --git a/ast3/Grammar/Grammar b/ast3/Grammar/Grammar | |||
msullivan
Jan 23, 2019
Collaborator
Including these patches directly is great, both from a repeatability perspective and hopefully from a future reviewing perspective.
Could you add directions for regenerating them after future changes?
msullivan
Jan 23, 2019
Collaborator
(Or, maybe per your comment about ignoring tools/, don't bother?)
| #define PyPARSE_PRINT_IS_FUNCTION 0x0004 | ||
| #define PyPARSE_UNICODE_LITERALS 0x0008 | ||
| #endif | ||
|
|
||
| #define PyPARSE_IGNORE_COOKIE 0x0010 | ||
| #define PyPARSE_BARRY_AS_BDFL 0x0020 | ||
| #define PyPARSE_ALWAYS_ASYNC 0x8000 |
msullivan
Jan 23, 2019
Collaborator
Nit: make this PyPARSE_ASYNC_ALWAYS to match the struct field name?
| @@ -1616,7 +1604,7 @@ tok_get(struct tok_state *tok, char **p_start, char **p_end) | |||
| /* async/await parsing block. */ | |||
| if (tok->cur - tok->start == 5) { | |||
| /* Current token length is 5. */ | |||
| if (tok->async_def) { | |||
| if (tok->async_always || tok->async_def) { | |||
msullivan
Jan 23, 2019
Collaborator
This whole async/await logic was pulled out in Python 3.7, right? Probably worth calling that out clearly in the comments.
This is a full port, following the recipe in update_process.md. I've also tried to keep the recipe up to date. I haven't cleaned up the commits, and I haven't run AppVeyor yet (I'm creating this PR to trigger the latter).