Skip to content

/ cpython

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-39406: Implement os.putenv() with setenv() if available #18095

Open
wants to merge 2 commits into
base: master
from
Open

bpo-39406: Implement os.putenv() with setenv() if available #18095

wants to merge 2 commits into from
+68 −60

Conversation

@vstinner
Copy link
Member

vstinner commented Jan 21, 2020 •
edited by bedevere-bot

If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

If setenv() is available, posix_putenv_garbage state is not needed.

https://bugs.python.org/issue39406
@serhiy-storchaka
serhiy-storchaka approved these changes Jan 21, 2020
View changes
configure Show resolved Hide resolved
Modules/posixmodule.c Outdated Show resolved Hide resolved
Modules/posixmodule.c Outdated Show resolved Hide resolved
Modules/posixmodule.c Outdated Show resolved Hide resolved
@vstinner
bpo-39406: Implement os.putenv() with setenv() if available
Loading status checks…
472b8cf 
If setenv() C function is available, os.putenv() is now implemented
with setenv() instead of putenv(), so Python doesn't have to handle
the environment variable memory.

Renane posix_putenv_garbage to posix_putenv_dict in posixmodule.c.
On Windows or if setenv() is available, posix_putenv_dict is no
longer needed.
@vstinner vstinner force-pushed the vstinner:setenv branch from a40ec8d to 472b8cf Jan 21, 2020
@vstinner

This comment has been minimized.

Sign in to view
Copy link
Member Author

vstinner commented Jan 21, 2020

I updated the PR to use SetEnvironmentVariable() on Windows.

@serhiy-storchaka: Would you mind to review the updated PR?
@vstinner
Fix usage of PyErr_SetFromWindowsErr()
Loading status checks…
617f7f4
@vstinner

This comment has been minimized.

Sign in to view
Copy link
Member Author

vstinner commented Jan 21, 2020

On Windows, I removed the explicit check on _MAX_ENV:

    if (size > _MAX_ENV) {
        PyErr_Format(PyExc_ValueError,
                     "the environment variable is longer than %u characters",
                     _MAX_ENV);
        goto error;
    }

On my Windows 10 VM, _MAX_ENV is equal to 32767, but SetEnvironmentVariableW() seems to accept way longer values!

>>> value='x' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value)    
(True, 32767)
>>> value='x' * (32767+10); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 32777)
>>> value='abcd' * (32767); os.environ['long'] = value; os.environ['long'] == value, len(value) 
(True, 131068)

But I got ERROR_NO_SYSTEM_RESOURCES (error 1450: "Insufficient system resources exist to complete the requested service") for a value of 1 GB:

>>> import os
>>> value='x'*(2**30)
>>> os.putenv('long', value)
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
OSError: [WinError 1450] Ressources système insuffisantes pour terminer le service demandé

The limit seems to be the memory rather than an hardcoded limit. A "shorter" value (64 MB) is fine:

>>> import os
>>> value='x'*(2**26)
>>> os.putenv('long', value)
@serhiy-storchaka
serhiy-storchaka reviewed Jan 21, 2020
View changes
Modules/posixmodule.c
/* putenv() requires to manage the memory manually: use a Python dict
object mapping name (bytes) to env (bytes) where env is a bytes string
like "name=value\0". */
PyObject *posix_putenv_dict;

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 21, 2020

Member

Why rename it? This increases the size of the diff.
Modules/posixmodule.c
}
#else /* MS_WINDOWS */
#elif defined(HAVE_PUTENV)

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 21, 2020

Member
Suggested change
#elif defined(HAVE_PUTENV)
#elif defined(HAVE_SETENV) || defined(HAVE_PUTENV)
Modules/posixmodule.c
Py_RETURN_NONE;
}
#endif /* MS_WINDOWS */
#endif /* HAVE_PUTENV */
#endif /* HAVE_PUTENV */

This comment has been minimized.

Sign in to view
Copy link
@serhiy-storchaka

serhiy-storchaka Jan 21, 2020

Member
Suggested change
#endif /* HAVE_PUTENV */
#endif /* HAVE_SETENV || HAVE_PUTENV */
@vstinner

This comment has been minimized.

Sign in to view
Copy link
Member Author

vstinner commented Jan 21, 2020

Oh, a test fail on Windows:


2020-01-21T10:50:49.1283745Z ======================================================================
2020-01-21T10:50:49.1283887Z ERROR: test_unset_error (test.test_os.EnvironTests)
2020-01-21T10:50:49.1284029Z ----------------------------------------------------------------------
2020-01-21T10:50:49.1284166Z Traceback (most recent call last):
2020-01-21T10:50:49.1284296Z   File "D:\a\cpython\cpython\lib\test\support\__init__.py", line 695, in wrapper
2020-01-21T10:50:49.1284429Z     return func(*args, **kw)
2020-01-21T10:50:49.1284504Z   File "D:\a\cpython\cpython\lib\test\test_os.py", line 962, in test_unset_error
2020-01-21T10:50:49.1284628Z     self.assertRaises(ValueError, os.environ.__delitem__, key)
2020-01-21T10:50:49.1284764Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 799, in assertRaises
2020-01-21T10:50:49.1284895Z     return context.handle('assertRaises', args, kwargs)
2020-01-21T10:50:49.1285989Z   File "D:\a\cpython\cpython\lib\unittest\case.py", line 202, in handle
2020-01-21T10:50:49.1286157Z     callable_obj(*args, **kwargs)
2020-01-21T10:50:49.1286268Z   File "D:\a\cpython\cpython\lib\os.py", line 691, in __delitem__
2020-01-21T10:50:49.1286377Z     raise KeyError(key) from None
2020-01-21T10:50:49.1295463Z KeyError:
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@serhiy-storchaka serhiy-storchaka

@pitrou pitrou

@pablogsal pablogsal

Assignees
No one assigned
Labels
CLA signed awaiting merge
Projects
None yet
Milestone
No milestone
4 participants
@vstinner @serhiy-storchaka @the-knights-who-say-ni @bedevere-bot
You can’t perform that action at this time.