Test Annotation Framework for Imported CPython test #1973
Conversation
There was a problem hiding this comment.
I really like this idea! It would at least allow us to mark which tests should actually need work on our side in the test results webpage. @palaviv @youknowone what do you think?
|
I love this idea! |
…and comments Co-authored-by: James Webber <jamestwebber@users.noreply.github.com> Co-authored-by: Jeong YunWon <youknowone@users.noreply.github.com>
|
I agree marking things more than comments is a good idea. |
|
Yeah, lgtm. There might be more skips/expectedFailures to replace with this since this pr was opened, but we can do that as a follow-up |
|
I strongly prefer this idea to current state. Not sure librptest has to be placed in I don't agree |
|
👍 for the last paragraph @youknowone |
When importing test currentlyfrom CPython, they need to be annotated with comments like
or how ever the author likes. Further the original file revision and the orignating python (language) revision is missing in most of the imported tests, what makes tracking nearly impossible.
To improve the convention base "solution", I propose a python annotation framework using decorators to properly annotate the imported test cases, such that they can be tracked and evaluated in a comprehnesive and comfortable way.
Examples can be seen in Lib/test/test_rpt.py.
This the initial implementation. The core functionality is there and tested, but, e.g., the reporting is very rudimentary and more for demonstration purpoposes.