Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

venv: optional activate link/copy to solve ci/portability headaches #100281

Open
kfsone opened this issue Dec 15, 2022 · 1 comment
Open

venv: optional activate link/copy to solve ci/portability headaches #100281

kfsone opened this issue Dec 15, 2022 · 1 comment
Labels
type-feature A feature request or enhancement

Comments

@kfsone
Copy link

kfsone commented Dec 15, 2022

Problem

activate location is platform dependent.

Background

15+ years ago PyPY made a decision that has often been a headache for portability; putting activate in Scripts for Windows and bin for everyone else.

This is sometimes misreported as a bug (#97586), core issue (https://bugs.python.org/issue35003), discussion (#18083) and recurrent threads on the pypy issues list (pypa/virtualenv#2404, pypa/virtualenv#1418 (comment))

It's an exceptional pain point because 1/ you chose python to ease portability, -2->/ you use venv as an extra layer of abstraction, -3->/ surprise! your abstraction requires you to take two different routes based on platform!

Proposal

Introduce an option that causes the creation of a symlink or second copy of the activator at a user specified path.

Ubuntu|Pwsh> venv -p python 3.10 --activator-copy=/opt/ci/activate /opt/ci
Windows|Pwsh> venv -p python 3.10 --activator-copy=o:/ci/activate o:/ci

Arguments

(Per the original change) "Scripts is the correct location for activate on Windows"

  • This doesn't move it, so everything that already supports the split will work as before,
  • Python itself now violates the Scripts rule, which is an attractor to people who wanted to hide platform specifics,

It's not a big deal, just if .. then else

CI and building today is complex with many moving parts, having an intermediate level abstraction suddenly require you to think about platforms is always a headache. If you're using something like gitlab, you end up with a huge hitch in complexity to achieve this for the scope it's in under the current system.

Windows doesn't support symlinks

Whether it is a file or a symlink is irrelevant, and anyone who is actively working to support a version of Windows old enough to not support symlinks already has enough of their hair on fire that they'll gladly accept it falling back to creating a file instead of junction.
@kfsone kfsone added the type-feature A feature request or enhancement label Dec 15, 2022
@kfsone kfsone changed the title venv: script destination to solve ci/portability headaches venv: optional activate link/copy to solve ci/portability headaches Dec 15, 2022
@eryksun
Copy link
Contributor

eryksun commented Dec 16, 2022

This is sometimes misreported as a bug (#97586)

I changed #97586 into a feature request a couple months ago. Creating a relative symlink or a junction (bind mount) to the "Scripts" directory is a reasonable approach if the filesystem supports reparse points (e.g. NTFS and ReFS). The downside is that there's no fallback if reparse points aren't supported (e.g. FAT32, exFAT and most UNC providers, such as a VirtualBox shared folder).

Your proposal to symlink or copy just the activation script will require modifying the PowerShell script "Activate.ps1". Currently it's implemented to be relative to the directory of the shell script. Thus a symlink or copy in "bin" would add "bin" to PATH instead of "Scripts".

Python itself now violates the Scripts rule

Putting the base interpreter binaries in "Scripts" makes about as much sense, literally speaking, as putting scripts in "bin", so I'd be fine with making that change to the base installation layout.

15+ years ago PyPY made a decision that has often been a headache for portability; putting activate in Scripts for Windows and bin for everyone else.

Using "Scripts" on Windows dates back to the mid 1990s. Here's what this looked like in distutils when the platform install schemes were added in 2000.

cpython/Lib/distutils/command/install.py

Lines 15 to 40 in a233d86

INSTALL_SCHEMES = {
'unix_prefix': {
'purelib': '$base/lib/python$py_version_short/site-packages',
'platlib': '$platbase/lib/python$py_version_short/site-packages',
'scripts': '$base/bin',
'data' : '$base/share',
},
'unix_home': {
'purelib': '$base/lib/python',
'platlib': '$base/lib/python',
'scripts': '$base/bin',
'data' : '$base/share',
},
'nt': {
'purelib': '$base',
'platlib': '$base',
'scripts': '$base\\Scripts',
'data' : '$base\\Data',
},
'mac': {
'purelib': '$base:Lib',
'platlib': '$base:Mac:PlugIns',
'scripts': '$base:Scripts',
'data' : '$base:Data',
}
}

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@kfsone @eryksun