Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-42021: Fix possible ref leaks during _sqlite3 module init #22673

Merged
merged 11 commits into from Oct 15, 2020

Conversation

erlend-aasland
Copy link
Contributor

@erlend-aasland erlend-aasland commented Oct 12, 2020

@corona10, would you mind reviewing this?

https://bugs.python.org/issue42021

@erlend-aasland erlend-aasland changed the title bpo-42021: Handle PyDict_SetItemString() errors in _sqlite3 bpo-42021: Fix possible ref. leaks during _sqlite3 module init Oct 12, 2020
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 12, 2020

FYI, the Ubuntu CI fails while testing tkinter, not sqlite3:

test_heading_callback (tkinter.test.test_ttk.test_widgets.TreeviewTest) ... Timeout (0:20:00)!
Thread 0x00007f5092516080 (most recent call first):
  File "/home/runner/work/cpython/cpython/Lib/tkinter/__init__.py", line 696 in wait_visibility
  File "/home/runner/work/cpython/cpython/Lib/tkinter/test/test_ttk/test_widgets.py", line 1539 in test_heading_callback

Copy link
Member

@corona10 corona10 left a comment

Thanks for the work,
I left several reviews for better usage.

The reviews are based on the recommended usage of C API.
We can remove dict and related codes after the review is applied.

It is recommended extensions use other PyModule_*() and PyObject_*() functions rather than directly manipulate a module’s __dict__.

Modules/_sqlite/microprotocols.c Outdated Show resolved Hide resolved
Modules/_sqlite/microprotocols.c Outdated Show resolved Hide resolved
Modules/_sqlite/module.c Outdated Show resolved Hide resolved
Modules/_sqlite/module.c Outdated Show resolved Hide resolved
Modules/_sqlite/module.c Outdated Show resolved Hide resolved
Modules/_sqlite/module.c Outdated Show resolved Hide resolved
Modules/_sqlite/module.c Outdated Show resolved Hide resolved
@corona10 corona10 requested a review from vstinner Oct 13, 2020
@bedevere-bot
Copy link

bedevere-bot commented Oct 13, 2020

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

And if you don't make the requested changes, you will be put in the comfy chair!

@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 13, 2020

Thanks for reviewing @corona10! I'll have a look at it tonight.

Modules/_sqlite/module.c Outdated Show resolved Hide resolved
Erlend Egeberg Aasland and others added 2 commits Oct 13, 2020
Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
- Use PyModule_Add*() iso. directly manipulating the module's __dict__
- Fix return type mismatch
- Ditch ADD_STRING (after cleaning up, there was no use for it)

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 13, 2020

I have made the requested changes; please review again.

@bedevere-bot
Copy link

bedevere-bot commented Oct 13, 2020

Thanks for making the requested changes!

@corona10: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from corona10 Oct 13, 2020
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 13, 2020

Thanks for the work, I left several reviews for better usage.

Thanks again for reviewing!

It is recommended extensions use other PyModule_*() and PyObject_*() functions rather than directly manipulate a module’s __dict__.

That's way better. I should've though of that, but I'm not that familiar with all the recommended practices of the C API yet. Thanks!

By the way, regarding the integer constants loop between line 412 and 419. Wouldn't it be cleaner to use PyModule_AddIntMacro here? I would tear out _int_constants, _IntConstantPair (and it's typedef), and replace the constant array with a helper function (for example static int add_integer_constants(PyObject *module) { ... }) that could be called from the module init function. Would it be ok to add this to the PR?

@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 13, 2020

Regarding this snippet in the module init function:

   /* In Python 2.x, setting Connection.text_factory to
         OptimizedUnicode caused Unicode objects to be returned for
         non-ASCII data and bytestrings to be returned for ASCII data.
         Now OptimizedUnicode is an alias for str, so it has no
         effect. */
      Py_INCREF((PyObject*)&PyUnicode_Type);
      if (PyModule_AddObject(module, "OptimizedUnicode", (PyObject*)&PyUnicode_Type) < 0) {
          Py_DECREF((PyObject*)&PyUnicode_Type);
          goto error;
      }

This looks like something that should be deprecated.

@bedevere-bot
Copy link

bedevere-bot commented Oct 14, 2020

Thanks for making the requested changes!

@corona10: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from corona10 Oct 14, 2020
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 14, 2020

By the way, regarding the integer constants loop between line 412 and 419. Wouldn't it be cleaner to use PyModule_AddIntMacro here? I would tear out _int_constants, _IntConstantPair (and it's typedef), and replace the constant array with a helper function (for example static int add_integer_constants(PyObject *module) { ... }) that could be called from the module init function. Would it be ok to add this to the PR?

@corona10: This snuck in in the last push (commit 70a42ab)! If you want me to exclude this, let me know, and I'll revert it right away. Sorry 'bout that.

Modules/_sqlite/module.c Show resolved Hide resolved
Modules/_sqlite/microprotocols.c Show resolved Hide resolved
@bedevere-bot
Copy link

bedevere-bot commented Oct 15, 2020

A Python core developer has requested some changes be made to your pull request before we can consider merging it. If you could please address their requests along with any other requests in other reviews from core developers that would be appreciated.

Once you have made the requested changes, please leave a comment on this pull request containing the phrase I have made the requested changes; please review again. I will then notify any core developers who have left a review that you're ready for them to take another look at this pull request.

Co-authored-by: Dong-hee Na <donghee.na92@gmail.com>
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 15, 2020

I have made the requested changes; please review again.

@bedevere-bot
Copy link

bedevere-bot commented Oct 15, 2020

Thanks for making the requested changes!

@corona10: please review the changes made to this pull request.

@bedevere-bot bedevere-bot requested a review from corona10 Oct 15, 2020
Erlend E. Aasland added 2 commits Oct 15, 2020
Copy link
Member

@corona10 corona10 left a comment

lgtm

@corona10
Copy link
Member

corona10 commented Oct 15, 2020

@erlend-aasland Thanks for the work, I also manually check the module attribute by my hand and it looks good.

@corona10 corona10 merged commit 644e942 into python:master Oct 15, 2020
3 checks passed
@erlend-aasland
Copy link
Contributor Author

erlend-aasland commented Oct 15, 2020

@erlend-aasland Thanks for the work, I also manually check the module attribute by my hand and it looks good.

Great! Thanks for a thorough review, @corona10!

@erlend-aasland erlend-aasland deleted the bpo-42021 branch Oct 15, 2020
xzy3 pushed a commit to xzy3/cpython that referenced this pull request Oct 18, 2020
adorilson pushed a commit to adorilson/cpython that referenced this pull request Mar 13, 2021
@tongxiaoge1001
Copy link

tongxiaoge1001 commented Jun 16, 2021

Is python3.7 involved?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

None yet

6 participants