Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

shutil.make_archive() root_dir do not work #66220

Open
DemoHT mannequin opened this issue Jul 21, 2014 · 34 comments
Open

shutil.make_archive() root_dir do not work #66220

DemoHT mannequin opened this issue Jul 21, 2014 · 34 comments
Labels
3.7 3.8 docs easy stdlib type-bug

Comments

@DemoHT
Copy link
Mannequin

@DemoHT DemoHT mannequin commented Jul 21, 2014

BPO 22021
Nosy @vstinner, @giampaolo, @tarekziade, @ezio-melotti, @bitdancer, @hynek, @vadmium, @serhiy-storchaka, @bananaappletw, @csabella, @lysnikolaou, @miss-islington, @rixx
PRs
  • #10191
  • #10367
  • #20709
  • #20710
  • #20711
  • #20712
  • Files
  • Issue22021.patch
  • Note: these values reflect the state of the issue at the time it was migrated and might not reflect the current state.

    Show more details

    GitHub fields:

    assignee = None
    closed_at = None
    created_at = <Date 2014-07-21.09:50:13.659>
    labels = ['easy', 'type-bug', '3.8', '3.7', 'library', 'docs']
    title = 'shutil.make_archive()  root_dir do not work'
    updated_at = <Date 2020-06-08.11:56:35.819>
    user = 'https://bugs.python.org/DemoHT'

    bugs.python.org fields:

    activity = <Date 2020-06-08.11:56:35.819>
    actor = 'lys.nikolaou'
    assignee = 'docs@python'
    closed = False
    closed_date = None
    closer = None
    components = ['Documentation', 'Library (Lib)']
    creation = <Date 2014-07-21.09:50:13.659>
    creator = 'DemoHT'
    dependencies = []
    files = ['36022']
    hgrepos = []
    issue_num = 22021
    keywords = ['patch', 'easy']
    message_count = 34.0
    messages = ['223568', '223569', '223572', '223622', '223624', '223668', '223805', '273073', '273074', '273077', '273078', '273080', '273128', '273180', '318132', '318148', '318172', '318178', '328860', '328865', '329326', '329355', '329365', '329371', '330902', '337016', '342317', '342347', '342359', '370959', '370960', '370961', '370962', '370975']
    nosy_count = 15.0
    nosy_names = ['vstinner', 'giampaolo.rodola', 'tarek', 'ezio.melotti', 'r.david.murray', 'docs@python', 'hynek', 'martin.panter', 'serhiy.storchaka', 'DemoHT', 'bananaappletw', 'cheryl.sabella', 'lys.nikolaou', 'miss-islington', 'rixx']
    pr_nums = ['10191', '10367', '20709', '20710', '20711', '20712']
    priority = 'normal'
    resolution = None
    stage = 'patch review'
    status = 'open'
    superseder = None
    type = 'behavior'
    url = 'https://bugs.python.org/issue22021'
    versions = ['Python 3.7', 'Python 3.8']

    @DemoHT
    Copy link
    Mannequin Author

    @DemoHT DemoHT mannequin commented Jul 21, 2014

    set root_dir do not work

    output:
    =====================================================

    Python 3.4.0 (v3.4.0:04f714765c13, Mar 16 2014, 19:25:23) [MSC v.1600 64 bit (AMD64)] on win32
    Type "copyright", "credits" or "license()" for more information.
    >>> import shutil
    >>> shutil.make_archive("tmp.tar.gz", "gztar", "c:/xjtu", "c:/tmp")
    'C:\\Python34\\tmp.tar.gz.tar.gz'

    =====================================================

    source code of make_archive()
    =====================================================
    756 save_cwd = os.getcwd()
    757 if root_dir is not None:
    758 if logger is not None:
    759 logger.debug("changing into '%s'", root_dir)
    760 base_name = os.path.abspath(base_name)
    761 if not dry_run:
    762 os.chdir(root_dir)
    ...
    ...
    782 try:
    783 filename = func(base_name, base_dir, **kwargs)
    784 finally:
    =====================================================

    base_name is set before chdir, so the archive always be created in cwd, whether set root_dir or not.
    so, line 760 should be move below line 762

    @DemoHT DemoHT mannequin added stdlib type-bug labels Jul 21, 2014
    @ezio-melotti
    Copy link
    Member

    @ezio-melotti ezio-melotti commented Jul 21, 2014

    Thanks for the report, do you want to provide a patch?
    (You can check the devguide if you need more information.)

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Jul 21, 2014

    I believe this is working as designed, although the documentation does not make that clear. root dir is the root directory of the *created archive*, it has nothing to do with where the archive file itself is placed.

    @bitdancer bitdancer added the docs label Jul 21, 2014
    @DemoHT
    Copy link
    Mannequin Author

    @DemoHT DemoHT mannequin commented Jul 22, 2014

    I don't think so.

    In source code, it just change work dir to root_dir, do nothing, and then the change word dir back.

    If it works as design, the "root_dir" will be meaningless. should be remove.

    @DemoHT
    Copy link
    Mannequin Author

    @DemoHT DemoHT mannequin commented Jul 22, 2014

    Here's the path

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Jul 22, 2014

    The point *should* be that if you have something like:

    /home/me/some/directory/my/stuff/a
    /home/me/some/directory/my/stuff/b
    /home/me/some/other/directory

    and you set rootdir to '/home/me/some' and base_dir='/home/me/some/directory/my' then the file paths in the archive will be:

    directory/my/stuff/a
    directory/my/stuff/b
    

    At least, that's how I read the docs, though as I said they are *not* clear. (I can't otherwise imagine any reason to have the root_dir parameter, with that name.)

    Is this not what happens?

    @DemoHT
    Copy link
    Mannequin Author

    @DemoHT DemoHT mannequin commented Jul 24, 2014

    that sounds reasonable

    @bananaappletw
    Copy link
    Mannequin

    @bananaappletw bananaappletw mannequin commented Aug 19, 2016

    Sorry to bother.

    But This patch is still not accepted.

    I still suffer this issue.

    Is anyone willing to review this patch?

    Thanks

    @bananaappletw
    Copy link
    Mannequin

    @bananaappletw bananaappletw mannequin commented Aug 19, 2016

    Or, Is there anything I could help?

    I am glad to help fixing this issue.

    @berkerpeksag
    Copy link
    Member

    @berkerpeksag berkerpeksag commented Aug 19, 2016

    Hi bananaapple, there are two things that need to be addressed:

    1. We need to decide whether this is a bug in make_archive() implementation or not. See msg223572 and msg223668 for details. I personally agree with David's analysis in msg223668.

    We can rephrase make_archive() documentation to clarify what root_dir exactly does.

    1. If this is a bug in make_archive() implementation, we need to add a test case. You can take a look at Lib/test/test_shutil.py.

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Aug 19, 2016

    Agree with David, this is working as designed (and matching the behavior of say "tar -C"). But the documentation is not clear if not confusing.

    @bananaappletw
    Copy link
    Mannequin

    @bananaappletw bananaappletw mannequin commented Aug 19, 2016

    Hello Berker Peksag,

    1. For now, I realize the comment of David.

    This is not a bug of make_archive() implementation.

    However, I think documentation is not clear to understand at the first time reading it.

    Maybe it should be rephrased.

    1. Actually, what I am looking in this function is able to output the archive to another directory rather than current directory.

    But, this functionality is not provided.

    I think this functionality will be useful if implemented.

    What do you think about this?

    Thank you for your explaination.

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented Aug 19, 2016

    "base_name is the name of the file to create, including the path". So you should be able to specify the path to where you want it created. If that doesn't work, that's a bug.

    @bananaappletw
    Copy link
    Mannequin

    @bananaappletw bananaappletw mannequin commented Aug 20, 2016

    I am sorry.

    I understand now.

    Thank you.

    @rixx
    Copy link
    Mannequin

    @rixx rixx mannequin commented May 29, 2018

    I'm similarly confused by this issue. If somebody can help me understand what's going on, I'll put my understanding into a documentation patch.

    I have created this minimal example to demonstrate what I don't understand: I've created a directory structure within /tmp like this:

    bpo-22021
    └── root
    └── structure
    ├── content
    │   └── please_add.txt
    └── do_not_add.txt

    Now I'd like to create a zip archive that contains the directories "structure" and "content", and the file "please_add.txt", but not the file "do_not_add.txt". My understanding of the documentation was that I'd have to do this:

    >> shutil.make_archive(base_name='/tmp/issue22021archive', format='zip', root_dir='/tmp/issue22021/root', base_dir='/tmp/issue22021/root/structure/content')

    But on my system, the created file (/tmp/issue22021archive.zip) looks like this according to unzip -l:

    Archive: issue22021archive.zip
    Length Date Time Name
    --------- ---------- ----- ----
    0 2018-05-30 00:26 tmp/issue22021/root/structure/content/
    0 2018-05-30 00:26 tmp/issue22021/root/structure/content/please_add.txt
    --------- -------
    0 2 files

    This is consistent with my experience that created archives will always contain directories from / on, which is unexpected to me.

    It appears this happens whenever base_dir and root_dir is set, but to my understanding the documentation does not warn against this.

    I've confirmed this behaviour with Python 3.6.5 and Python 3.5.3, but I suspect that doesn't really matter, as it seems to be a documentation issue.

    @bananaappletw
    Copy link
    Mannequin

    @bananaappletw bananaappletw mannequin commented May 30, 2018

    I think this snippet might help you.

    shutil.make_archive(base_name='/tmp/issue22021archive', format='zip', root_dir='/tmp/issue22021/root/', base_dir='structure/content/')

    unzip -l /tmp/issue22021archive.zip

    Archive: /tmp/issue22021archive.zip
    Length Date Time Name
    --------- ---------- ----- ----
    0 2018-05-30 11:14 structure/content/
    0 2018-05-30 11:02 structure/content/please_add.txt
    --------- -------
    0 2 files

    @rixx
    Copy link
    Mannequin

    @rixx rixx mannequin commented May 30, 2018

    Thank you, that's what I figured out later last evening. To my understanding, the docs don't give any indication that base_dir is supposed to be relative to root_dir, so I'd add this information, and maybe add a similar example to the one above, if that's appropriate?

    @bitdancer
    Copy link
    Member

    @bitdancer bitdancer commented May 30, 2018

    Sounds reasonable to me.

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Oct 29, 2018

    FYI.

    Joannah Nanjekye converted bpo-22021.patch into a PR: PR 10191.

    Serhiy Storchaka closed the PR: "The current behavior looks correct to me. It is consistent with the behavior of the tar command. This change breaks tests and I think it will break user code. Read the discussion on bpo-22021."
    #10191 (comment)

    @serhiy-storchaka
    Copy link
    Member

    @serhiy-storchaka serhiy-storchaka commented Oct 29, 2018

    This is a documentation issue.

    @lysnikolaou
    Copy link
    Contributor

    @lysnikolaou lysnikolaou commented Nov 6, 2018

    Is anybody working on this or can I submit a PR?

    @vstinner
    Copy link
    Member

    @vstinner vstinner commented Nov 6, 2018

    Is anybody working on this or can I submit a PR?

    Yes: Joannah Nanjekye wrote PR 10191 (from bpo-22021.patch) which has been rejected. This issue is more complex from what it looks.

    @lysnikolaou
    Copy link
    Contributor

    @lysnikolaou lysnikolaou commented Nov 6, 2018

    Since Serhiy said that this is a pure documentation issue, I thought that a doc PR is all that was needed, which I would be happy to make. Am I missing something here?

    @rixx
    Copy link
    Mannequin

    @rixx rixx mannequin commented Nov 6, 2018

    Yes, this is a documentation issue: A patch clarifying what root_dir and base_dir do, and how they interact (or how they are to be used in combination) would be sufficient to close this issue.

    @lysnikolaou
    Copy link
    Contributor

    @lysnikolaou lysnikolaou commented Dec 2, 2018

    Ping.

    @lysnikolaou
    Copy link
    Contributor

    @lysnikolaou lysnikolaou commented Mar 2, 2019

    Pinging once more for review.

    @csabella
    Copy link
    Contributor

    @csabella csabella commented May 13, 2019

    @giampaolo.rodola, would you be able to provide a review of this doc change? Thanks!

    @giampaolo
    Copy link
    Contributor

    @giampaolo giampaolo commented May 13, 2019

    @cheryl.sabella don't have the bandwidth right now (traveling), sorry.

    @csabella
    Copy link
    Contributor

    @csabella csabella commented May 13, 2019

    @giampaolo.rodola, no problem. I intended this more as a general request rather than something for right now. If you do have any availability at some point, please keep this one in mind. Thanks!

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 8, 2020

    New changeset 7633371 by Lysandros Nikolaou in branch 'master':
    bpo-22021: Update root_dir and base_dir documentation in shutil (GH-10367)
    7633371

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 8, 2020

    New changeset d5489a9 by Miss Islington (bot) in branch '3.7':
    bpo-22021: Update root_dir and base_dir documentation in shutil (GH-10367)
    d5489a9

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 8, 2020

    New changeset be5ed59 by Miss Islington (bot) in branch '3.9':
    bpo-22021: Update root_dir and base_dir documentation in shutil (GH-10367)
    be5ed59

    @miss-islington
    Copy link
    Contributor

    @miss-islington miss-islington commented Jun 8, 2020

    New changeset 12dfbae by Miss Islington (bot) in branch '3.8':
    bpo-22021: Update root_dir and base_dir documentation in shutil (GH-10367)
    12dfbae

    @lysnikolaou
    Copy link
    Contributor

    @lysnikolaou lysnikolaou commented Jun 8, 2020

    Since #54576 is now merged, should we close this?

    @ezio-melotti ezio-melotti transferred this issue from another repository Apr 10, 2022
    Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
    Labels
    3.7 3.8 docs easy stdlib type-bug
    Projects
    None yet
    Development

    No branches or pull requests

    9 participants