Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-42627: Fix wrong parsing of Windows registry proxy settings #26307

Open
wants to merge 19 commits into
base: main
Choose a base branch
from

Conversation

Copy link

@CrazyBoyFeng CrazyBoyFeng commented May 22, 2021

In Windows registry proxy settings, https=host:456 does not mean that the proxy server communicates with the client through the HTTPS protocol, but it means that the proxy server proxies the client's traffic to the HTTPS website. The proxy server itself communicates with the client through the HTTP protocol by default.

fix pypa/pip#9568 fix pypa/pip#9614 fix urllib3/urllib3#2164 fix psf/requests#5740

https://bugs.python.org/issue42627

@the-knights-who-say-ni
Copy link

@the-knights-who-say-ni the-knights-who-say-ni commented May 22, 2021

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

CLA Missing

Our records indicate the following people have not signed the CLA:

@CrazyBoyFeng

For legal reasons we need all the people listed to sign the CLA before we can look at your contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

If you have recently signed the CLA, please wait at least one business day
before our records are updated.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

@the-knights-who-say-ni
Copy link

@the-knights-who-say-ni the-knights-who-say-ni commented May 28, 2021

Hello, and thanks for your contribution!

I'm a bot set up to make sure that the project can legally accept this contribution by verifying everyone involved has signed the PSF contributor agreement (CLA).

Recognized GitHub username

We couldn't find a bugs.python.org (b.p.o) account corresponding to the following GitHub usernames:

@testinguser

This might be simply due to a missing "GitHub Name" entry in one's b.p.o account settings. This is necessary for legal reasons before we can look at this contribution. Please follow the steps outlined in the CPython devguide to rectify this issue.

You can check yourself to see if the CLA has been received.

Thanks again for the contribution, we look forward to reviewing it!

CrazyBoyFeng and others added 3 commits May 28, 2021
To compat with customary formats from software such as `requests[socks]`
Lib/urllib/request.py Outdated Show resolved Hide resolved
Lib/urllib/request.py Outdated Show resolved Hide resolved
@zooba
Copy link
Member

@zooba zooba commented Sep 16, 2021

@CrazyBoyFeng Looks like you're going to hit a known error in CI right now. Merging the latest from master should fix it.

@CrazyBoyFeng
Copy link
Author

@CrazyBoyFeng CrazyBoyFeng commented Sep 17, 2021

I've synced the branch to the latest, hope this works.

@gfxonline
Copy link

@gfxonline gfxonline commented Nov 19, 2021

Any updates on this PR?

@CrazyBoyFeng
Copy link
Author

@CrazyBoyFeng CrazyBoyFeng commented Nov 20, 2021

Until this bug is fixed, when using a proxy on Windows, you can work around it by:

  • As a developer, use the monkey patch. Or manually read the registry and set the environment variable https_proxy=http://proxy.
  • As an application user, set https_proxy=http://proxy to the environment variable.

If it is a socks proxy, you should set environment variables http_proxy=socks4://proxy and https_proxy=socks4://proxy.

@gfxonline
Copy link

@gfxonline gfxonline commented Nov 22, 2021

Until this bug is fixed, when using a proxy on Windows, you can work around it by:

  • As a developer, use the monkey patch. Or manually read the registry and set the environment variable https_proxy=http://proxy.
  • As an application user, set https_proxy=http://proxy to the environment variable.

If it is a socks proxy, you should set environment variables http_proxy=socks4://proxy and https_proxy=socks4://proxy.

That matches my envvars settings and I get the same problem. I was under the impression that this PR solves this exact issue where https is used to communicate with the proxy regardless of the protocol defined in the environment variables.

@sethmlarson
Copy link

@sethmlarson sethmlarson commented Jan 7, 2022

Hello! I'm happy to see this PR, up until this point I didn't know that this issue existed which was causing a lot of confusion for urllib3 users. If I could suggest an alternate solution, instead of introducing any changes related to SOCKS we change the following line to be http instead of https:

# Before:
proxies['https'] = 'https://%s' % proxyServer

# After:
proxies['https'] = 'http://%s' % proxyServer

And then keep the rest of the code as-is. My thought is this is the first mention of SOCKS in CPython as far as I can tell and adding support for SOCKS is a much bigger feature addition compared to the smaller fix of making HTTP and HTTPS requests use the same "default" proxy scheme.

@CrazyBoyFeng
Copy link
Author

@CrazyBoyFeng CrazyBoyFeng commented Jan 7, 2022

@sethmlarson
Although the previous code did not explicitly handle socks proxies. But it can pass http_proxy=socks://host:port from the environment variable. And downstream libraries such as requests can receive and parse it correctly.
So this doesn't mean that cpython didn't support socks proxy before.
Even if it didn't support it before, it should support it now.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment