gh-99631: Add custom loads and dumps support for the shelve module #118065

furkanonder · 2024-04-18T19:52:53Z

Issue: Custom unpickler and pickler for the shelve module #99631

📚 Documentation preview 📚: https://cpython-previews--118065.org.readthedocs.build/

…BsdDbShelf

Lib/shelve.py

Doc/library/shelve.rst

Lib/shelve.py

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

…ssue-99631-2

encukou

BsdDbShelf still uses pickle.Unpickler directly to walk the keys.

Doc/library/shelve.rst

Lib/shelve.py

bedevere-app · 2024-04-23T14:57:49Z

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-authored-by: Petr Viktorin <encukou@gmail.com>

…DbShelf

…ssue-99631-2

furkanonder · 2024-07-14T02:26:23Z

Doc/library/shelve.rst

@@ -160,18 +181,20 @@ Restrictions
   modules.  The *dict* object passed to the constructor must support those
   methods.  This is generally accomplished by calling one of
   :func:`!bsddb.hashopen`, :func:`!bsddb.btopen` or :func:`!bsddb.rnopen`.  The


By the way, we also need to update this sentence (from bsddb to berkeleydb). bsddb is deprecated according to https://www.jcea.es/programacion/pybsddb.htm.

It looks like the change would be bigger: it’s a new module (although the docs are not very clear) with maybe a new API.

Updating or deprecating this should be discussed in its own ticket 🙂

Yes, I agree on it. It would be better to open a new ticket to discuss this issue.

... or consider instead opening topic on Discourse; this may need a larger audience than what you'd get on the bug tracker.

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

…ssue-99631-2

picnixz

Thank you for the changes, those are much clearer now! I don't have any other comments (by the way, thank you for being reactive and accepting my feedback!)

I'll let a core dev check other stuff (I've stared too much at that PR so it's better if there is a fresh eye for confirmation).

Lib/test/test_shelve.py

furkanonder · 2024-07-29T10:21:46Z

Thank you for the changes, those are much clearer now! I don't have any other comments (by the way, thank you for being reactive and accepting my feedback!)

You're welcome. Thank you so much for your time and effort.

picnixz

Some few nitpicks that I thought of. Some should be addressed (3.15 -> next + self.subTest(proto=5)`, while others are purely cosmetics.

I'll let a core dev check other stuff (

This core dev is now me.

Doc/library/shelve.rst

Lib/test/test_shelve.py

modified: Doc/library/shelve.rst

…t_missing_custom_serializer

…e wrapping

picnixz

Almost done!

Doc/library/shelve.rst

Lib/test/test_shelve.py

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

furkanonder added 3 commits April 18, 2024 22:23

Use custom loads & dumps instead of custom pickler & unpickler for Shelf

e269c09

Allow custom loads & dumps instead of custom pickler & unpickler for 1�7

3465149

…BsdDbShelf

Update documentation for serializer and deserializred functions

44b9fa1

bedevere-app bot mentioned this pull request Apr 18, 2024

Custom unpickler and pickler for the shelve module #99631

Open

bedevere-app bot added the awaiting review label Apr 18, 2024

furkanonder mentioned this pull request Apr 18, 2024

gh-99631: Add custom loads and dumps support for the shelve module #99632

Closed

merwok reviewed Apr 18, 2024

View reviewed changes

Lib/shelve.py Show resolved Hide resolved

merwok reviewed Apr 18, 2024

View reviewed changes

Doc/library/shelve.rst Outdated Show resolved Hide resolved

eendebakpt reviewed Apr 20, 2024

View reviewed changes

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Lib/shelve.py Outdated Show resolved Hide resolved

furkanonder and others added 10 commits April 21, 2024 00:11

Update Doc/library/shelve.rst

f2eed32

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

Update Doc/library/shelve.rst

b3e5723

Co-authored-by: Pieter Eendebak <pieter.eendebak@gmail.com>

Update documentation for serializer and deserializer functions

53d5557

Merge branch 'main' into issue-99631-2

d496eab

Fix lines according to PEP-8

1e295ba

Merge branch 'issue-99631-2' of github.com:furkanonder/cpython into i 1�7

2011baa

…ssue-99631-2

Fix doc according to line 80

3cbabe9

Merge branch 'main' into issue-99631-2

67b7340

Fix inline emphasis issue in docs

c6b43e2

Merge branch 'issue-99631-2' of github.com:furkanonder/cpython into i 1�7

52a90f7

…ssue-99631-2

encukou requested changes Apr 23, 2024

View reviewed changes

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Lib/shelve.py Outdated Show resolved Hide resolved

bedevere-app bot added awaiting changes and removed awaiting review labels Apr 23, 2024

furkanonder and others added 6 commits July 14, 2024 01:31

Update the definition of the open function.

4f79cf6

Co-authored-by: Petr Viktorin <encukou@gmail.com>

Pass the serializer and serializer arguments of Shelf.__init__ of Bsd 1�7

798fdb2

…DbShelf

Add unittests for BsdDbShelf

bb1150d

Merge branch 'issue-99631-2' of github.com:furkanonder/cpython into i 1�7

98d841b

…ssue-99631-2

Update BsdDbShelf's set_location, last and first functions

1159bb6

Update BsdDbShelf's next and previous functions

4b4f1b6

furkanonder commented Jul 14, 2024

View reviewed changes

furkanonder and others added 5 commits July 29, 2024 11:40

Remove white space in test_shelve

4202ede

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Add test_custom_incomplete_serializer_and_deserializer_bsd_db_shelf

2827eb4

Merge branch 'issue-99631-2' of github.com:furkanonder/cpython into i 1�7

9918531

…ssue-99631-2

Update the serializer and deserializer functions

54188bd

Move os.mkdir and addCleanup functions beginning of the testcases

786a248

picnixz approved these changes Jul 29, 2024

View reviewed changes

Lib/test/test_shelve.py Outdated Show resolved Hide resolved

Use self.assertIsNone when checking None types

20c2450

change the test order

b3770ae

furkanonder added awaiting review and removed 3.14 bugs and security fixes awaiting changes labels Sep 13, 2024

furkanonder added 3 commits April 19, 2025 21:05

Merge branch 'main' into issue-99631-2

588623a

Update shelve module version references from 3.14 to 3.15

4d9599b

Merge branch 'main' into issue-99631-2

9b204b7

picnixz reviewed May 30, 2025

View reviewed changes

furkanonder added 7 commits May 30, 2025 18:25

Change shelve module version references from 3.15 to next

8b06918

modified: Doc/library/shelve.rst

Change shelve module version references from 3.15 to next

b0f0bbc

refactor nested context managers for better readability

d1bb227

simplify assertRaises calls in test_missing_custom_deserializer & tes 1�7

791743b

…t_missing_custom_serializer

refactor nested context managers for better readability

6b4be8b

Add type_name_len helper and use shorter variable names to reduce lin 1�7

34a32b9

…e wrapping

Merge branch 'main' into issue-99631-2

bf6f3aa

furkanonder requested a review from picnixz May 30, 2025 16:04

mostafaammer added this to lavitaconnect@MOSTAFAAMMER May 31, 2025

picnixz reviewed Jun 2, 2025

View reviewed changes

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Doc/library/shelve.rst Outdated Show resolved Hide resolved

Lib/test/test_shelve.py Outdated Show resolved Hide resolved

Lib/test/test_shelve.py Outdated Show resolved Hide resolved

furkanonder and others added 4 commits June 2, 2025 20:05

Improve the description of the open function

4b000cd

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Update the description of ShelveError

2dcda2a

Co-authored-by: Bénédikt Tran <10796600+picnixz@users.noreply.github.com>

Simplify conditional branches in serializer and deserializer functions

00bfb01

Merge branch 'main' into issue-99631-2

23ea842

Uh oh!

gh-99631: Add custom loads and dumps support for the shelve module #118065

Are you sure you want to change the base?

gh-99631: Add custom loads and dumps support for the shelve module #118065

Conversation

furkanonder commented Apr 18, 2024 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

encukou left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

bedevere-app bot commented Apr 23, 2024

Uh oh!

furkanonder Jul 14, 2024

Choose a reason for hiding this comment

Uh oh!

merwok Jul 14, 2024

Choose a reason for hiding this comment

Uh oh!

furkanonder Jul 16, 2024

Choose a reason for hiding this comment

Uh oh!

erlend-aasland Jul 17, 2024

Choose a reason for hiding this comment

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

furkanonder commented Jul 29, 2024 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

picnixz left a comment • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

picnixz left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

furkanonder commented Apr 18, 2024 •

edited by github-actions bot

Loading

furkanonder commented Jul 29, 2024 •

edited

Loading

picnixz left a comment •

edited

Loading