Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-39206: Add encoding detecting to modulefinder #17817

Open
wants to merge 5 commits into
base: master
from

Conversation

@LucianaMarques
Copy link

LucianaMarques commented Jan 4, 2020

@the-knights-who-say-ni

This comment has been minimized.

Copy link

the-knights-who-say-ni commented Jan 4, 2020

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@LucianaMarques

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@LucianaMarques

This comment has been minimized.

Copy link
Author

LucianaMarques commented Jan 4, 2020

Just recording, I just signed the CLA.

@LucianaMarques LucianaMarques force-pushed the LucianaMarques:fix-issue-39206 branch from a47ad67 to 8251bd0 Jan 4, 2020
Lib/modulefinder.py Outdated Show resolved Hide resolved
@LucianaMarques LucianaMarques force-pushed the LucianaMarques:fix-issue-39206 branch from 8251bd0 to 0c465a0 Jan 5, 2020
@LucianaMarques

This comment has been minimized.

Copy link
Author

LucianaMarques commented Jan 6, 2020

@asottile should I add a file to Misc/NEWSs.d? I did't find much informaiton on the dev guide.

@asottile

This comment has been minimized.

Copy link
Contributor

asottile commented Jan 6, 2020

@asottile should I add a file to Misc/NEWSs.d? I did't find much informaiton on the dev guide.

yep! there's two ways to do that -- iirc clicking on the details link will take you to the web UI for it, otherwise you can pip install blurb locally and then run blurb add

@plokmijnuhby

This comment has been minimized.

Copy link
Contributor

plokmijnuhby commented Jan 6, 2020

Could you also add some more tests for this issue?

@LucianaMarques

This comment has been minimized.

Copy link
Author

LucianaMarques commented Jan 6, 2020

yep! there's two ways to do that -- iirc clicking on the details link will take you to the web UI for it, otherwise you can pip install blurb locally and then run blurb add

Thanks, will do it!

Could you also add some more tests for this issue?

Of course! I will add some tests to ModuleFinder. I also thought about guaranteeing this in a broader context whenever there is an open() to a file in a future PR. Is there any specific test case you would like me to include? Thank you so much for the review!

@plokmijnuhby

This comment has been minimized.

Copy link
Contributor

plokmijnuhby commented Jan 6, 2020

It might be at least worth setting up tests to open files with odd encodings. Test it on a file with some Unicode characters in it, and on files with PEP263 style encoding declarations.

On a different note, I would point out that when running files, they are opened in bytes mode and that's it. Encoding is inferred during the compilation process, simply by running the compile() function on the bytestring read from the file, with no need to import tokenize or open in 'r' mode or anything else.

blurb-it bot and others added 4 commits Jan 6, 2020
…nto fix-issue-39206
@LucianaMarques

This comment has been minimized.

Copy link
Author

LucianaMarques commented Jan 7, 2020

@plokmijnuhby I've added one example, could you please give me feedback on it? If it's ok, I will add one ow two more with different encoding types or testing default is type to UTF-8. Thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.