Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-44899: add support for archiving changing files with tarfile #30426

Open
wants to merge 3 commits into
base: main
Choose a base branch
from

Conversation

marko-tuononen
Copy link

@marko-tuononen marko-tuononen commented Jan 6, 2022

@github-actions
Copy link

@github-actions github-actions bot commented Feb 6, 2022

This PR is stale because it has been open for 30 days with no activity.

@github-actions github-actions bot added the stale label Feb 6, 2022
@marko-tuononen
Copy link
Author

@marko-tuononen marko-tuononen commented Feb 8, 2022

bump

Copy link
Contributor

@MaxwellDupre MaxwellDupre left a comment

Look ok to me.
Ran 491 tests in 10.948s
OK (skipped=8)
== Tests result: SUCCESS ==
1 test OK.
Total duration: 11.2 sec
Tests result: SUCCESS
cpython on  bpo-44899/tarfile-changing-files [$] via 🐍 v3.11.0a5+ took 11s

Copy link
Member

@ethanfurman ethanfurman left a comment

Please also address questions in bpo issue.

raise exception("unexpected end of data")
if exception is not None:
raise exception("unexpected end of data")
print("warning: file changed as we read it")
Copy link
Member

@ethanfurman ethanfurman Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

if there are 5 blocks to copy, and the file size changes during the read of block 2, how many times are we going to get the warning printed?

# If there's data to follow, append it.
if fileobj is not None:
copyfileobj(fileobj, self.fileobj, tarinfo.size, bufsize=bufsize)
copybuf = io.BytesIO()
tarinfo.size = copyfileobj(fileobj, copybuf, tarinfo.size, exception=None, bufsize=self.copybufsize)
Copy link
Member

@ethanfurman ethanfurman Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The original bpo issue was for a system that had limited resources, but this solution is going to store the entire, possibly large, file in memory.

raise exception("unexpected end of data")
if exception is not None:
raise exception("unexpected end of data")
print("warning: file changed as we read it")
Copy link
Member

@ethanfurman ethanfurman Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Do not use print(), use warnings.warn instead.

Copy link
Author

@marko-tuononen marko-tuononen Feb 8, 2022

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Got it. I will use warnings.warn instead of print().

@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Feb 8, 2022

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

@github-actions github-actions bot removed the stale label Feb 9, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

5 participants