Incorrect error handling for APIs that can raise exceptions #105375

erlend-aasland · 2023-06-06T13:31:33Z

For example, in the collation callback two str objects (string1 and string2) are created using PyUnicode_FromStringAndSize. Error handling should happen directly after each call to PyUnicode_FromStringAndSize:

cpython/Modules/_sqlite/connection.c

Lines 1870 to 1875 in 0cb6b9b

    
           string1 = PyUnicode_FromStringAndSize((const char*)text1_data, text1_length); 
        
           string2 = PyUnicode_FromStringAndSize((const char*)text2_data, text2_length); 
        
           if (!string1 || !string2) { 
        
               goto finally; /* failed to allocate strings */ 
        
           }

Other cases where error handling is not done immediately after the API has been used:

I might have missed some; I did not do a complete audit yet.

Linked PRs

The text was updated successfully, but these errors were encountered:

Check for error after each call to PyUnicode_FromStringAndSize().

) Check for error after each call to PyUnicode_FromStringAndSize().

…pythonGH-105412) Check for error after each call to PyUnicode_FromStringAndSize(). (cherry picked from commit a24a780) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

GH-105412) (#105440) Check for error after each call to PyUnicode_FromStringAndSize(). (cherry picked from commit a24a780) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

GH-105412) (#105441) Check for error after each call to PyUnicode_FromStringAndSize(). (cherry picked from commit a24a780) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Error handling was deferred in some cases, which could potentially lead to exceptions being overwritten.

vstinner · 2023-06-09T09:26:17Z

There were many "init" functions of C extensions calling all C functions without checking for exceptions, and only checking for exceptions once at the end. Most of them have been fixed with better error checking, but not all of them yet.

vstinner · 2023-06-09T09:27:35Z

See issue #105374 about C API being error prone: when it's unclear if passing NULL was done on purpose or by mistake

vstinner · 2023-06-09T09:33:25Z

See also capi-workgroup/problems#47

(cherry picked from commit eede1d2) Bail immediately if an exception is set, to prevent exceptions from being overwritten.

Improve error handling so init bails on the first exception.

Fix a bug where an exception could end up being overwritten.

Fix a bug where an IndexError could end up being overwritten. (cherry picked from commit f668f73) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

…105600) Fix bugs where exceptions could end up being overwritten. (cherry picked from commit 00b599a) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

…105601) Fix bugs where exceptions could end up being overwritten. (cherry picked from commit 00b599a) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Bail on first error in heapctypesubclasswithfinalizer_finalize()

Bail on first error to prevent exceptions from possibly being overwritten.

Fix bugs where exceptions could end up being overwritten because of deferred error handling. Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

…H-105586) Fix bugs where exceptions could end up being overwritten because of deferred error handling. (cherry picked from commit 33c92c4) Co-authored-by: Nikita Sobolev <mail@sobolevn.me> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

Bail on first error in heapctypesubclasswithfinalizer_finalize()

…honGH-105608) Bail on first error in heapctypesubclasswithfinalizer_finalize() (cherry picked from commit d636d7dfe714e7168b342c7ea5f9f9d3b3569ed0) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

) (#105613) Fix bugs where exceptions could end up being overwritten because of deferred error handling. (cherry picked from commit 33c92c4) Co-authored-by: Nikita Sobolev <mail@sobolevn.me> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

…-105608) (#105615) Bail on first error in heapctypesubclasswithfinalizer_finalize() (cherry picked from commit d636d7d) Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

) (#105612) Fix bugs where exceptions could end up being overwritten because of deferred error handling. (cherry picked from commit 33c92c4) Co-authored-by: Nikita Sobolev <mail@sobolevn.me> Co-authored-by: Erlend E. Aasland <erlend.aasland@protonmail.com>

sobolevn · 2023-06-10T05:49:21Z

Wow, it was a productive night! Great work!

erlend-aasland added the type-bug An unexpected behavior, bug, or error label Jun 6, 2023

erlend-aasland self-assigned this Jun 6, 2023

This comment was marked as outdated.

Sign in to view

AlexWaygood added the topic-sqlite3 label Jun 6, 2023

bedevere-bot mentioned this issue Jun 6, 2023

gh-105375: Improve error handling in sqlite3 collation callback #105412

Merged

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 6, 2023

pythongh-105375: Improve error handling in sqlite3 collation callback

5429c38

Check for error after each call to PyUnicode_FromStringAndSize().

erlend-aasland linked a pull request Jun 6, 2023 that will close this issue

gh-105375: Improve error handling in sqlite3 collation callback #105412

Merged

erlend-aasland closed this as completed in #105412 Jun 7, 2023

erlend-aasland added a commit that referenced this issue Jun 7, 2023

gh-105375: Improve error handling in sqlite3 collation callback (#105412

a24a780

) Check for error after each call to PyUnicode_FromStringAndSize().

This was referenced Jun 7, 2023

[3.12] gh-105375: Improve error handling in sqlite3 collation callback (GH-105412) #105440

Merged

[3.11] gh-105375: Improve error handling in sqlite3 collation callback (GH-105412) #105441

Merged

erlend-aasland changed the title ~~Incorrect error handling in sqlite3 collation callback~~ Missing error handling for APIs that can raise exceptions Jun 7, 2023

erlend-aasland changed the title ~~Missing error handling for APIs that can raise exceptions~~ Incorrect error handling for APIs that can raise exceptions Jun 7, 2023

erlend-aasland reopened this Jun 7, 2023

erlend-aasland added topic-C-API and removed topic-sqlite3 labels Jun 7, 2023

erlend-aasland mentioned this issue Jun 7, 2023

gh-105375: Improve _pickle error handling #105475

Merged

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 7, 2023

pythongh-105375: Improve _pickle error handling

9b514d5

Error handling was deferred in some cases, which could potentially lead to exceptions being overwritten.

erlend-aasland mentioned this issue Jun 8, 2023

gh-105375: Improve error handling in PyUnicode_BuildEncodingMap() #105491

Open

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 8, 2023

pythongh-105375: Improve error handling in PyUnicode_BuildEncodingMap()

17065c4

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 8, 2023

pythongh-105375: Improve error handling in compiler_enter_scope()

a4d8114

bedevere-bot mentioned this issue Jun 8, 2023

gh-105375: Improve error handling in compiler_enter_scope() #105494

Merged

vstinner mentioned this issue Jun 9, 2023

Argument accepting a Python object or NULL: error prone API capi-workgroup/problems#47

Open

bedevere-bot mentioned this issue Jun 9, 2023

[3.11] gh-105375: Improve error handling in _elementtree (GH-105591) #105601

Merged

erlend-aasland added a commit that referenced this issue Jun 9, 2023

[3.11] gh-105375: Improve errnomodule error handling (#105590) (#105595)

76682ba

(cherry picked from commit eede1d2) Bail immediately if an exception is set, to prevent exceptions from being overwritten.

erlend-aasland linked a pull request Jun 9, 2023 that will close this issue

gh-105375: Improve error handling in _elementtree #105591

Merged

erlend-aasland added a commit that referenced this issue Jun 9, 2023

[3.12] gh-105375: Improve errnomodule error handling (#105590) (#105596)

e0087df

(cherry picked from commit eede1d2) Bail immediately if an exception is set, to prevent exceptions from being overwritten.

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 9, 2023

pythongh-105375: Harden _datetime initialisation

506d2c6

Improve error handling so init bails on the first exception.

bedevere-bot mentioned this issue Jun 9, 2023

gh-105375: Harden _datetime initialisation #105604

Open

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 9, 2023

pythongh-105375: Improve _decimal error handling

6daf015

Fix a bug where an exception could end up being overwritten.

bedevere-bot mentioned this issue Jun 9, 2023

gh-105375: Improve _decimal error handling #105605

Open

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 9, 2023

pythongh-105375: Fix error handling in testcapi heaptype test

f134dab

Bail on first error in heapctypesubclasswithfinalizer_finalize()

bedevere-bot mentioned this issue Jun 9, 2023

gh-105375: Harden error handling in _testcapi/heaptype.c #105608

Merged

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 9, 2023

pythongh-105375: Improve PyErr_WarnExplicit() error handling

1603759

Bail on first error to prevent exceptions from possibly being overwritten.

bedevere-bot mentioned this issue Jun 9, 2023

gh-105375: Improve PyErr_WarnExplicit() error handling #105610

Open

erlend-aasland added a commit to erlend-aasland/cpython that referenced this issue Jun 9, 2023

pythongh-105375: Improve error handling in the sys extension module

02cac21

Bail on first error to prevent exceptions from possibly being overwritten.

bedevere-bot mentioned this issue Jun 9, 2023

gh-105375: Improve error handling in the sys extension module #105611

Open

bedevere-bot mentioned this issue Jun 9, 2023

[3.12] gh-105375: Improve error handling in zoneinfo module (GH-105586) #105612

Merged

This was referenced Jun 9, 2023

[3.11] gh-105375: Improve error handling in zoneinfo module (GH-105586) #105613

Merged

gh-105375: Harden pyexpat initialisation #105606

Open

erlend-aasland added a commit that referenced this issue Jun 9, 2023

gh-105375: Harden error handling in _testcapi/heaptype.c (#105608)

d636d7d

Bail on first error in heapctypesubclasswithfinalizer_finalize()

bedevere-bot mentioned this issue Jun 9, 2023

[3.12] gh-105375: Harden error handling in _testcapi/heaptype.c (GH-105608) #105615

Merged

Incorrect error handling for APIs that can raise exceptions #105375

Incorrect error handling for APIs that can raise exceptions #105375

erlend-aasland commented Jun 6, 2023 •

edited by sobolevn

This comment was marked as outdated.

vstinner commented Jun 9, 2023

vstinner commented Jun 9, 2023

vstinner commented Jun 9, 2023

sobolevn commented Jun 10, 2023

Incorrect error handling for APIs that can raise exceptions #105375

Incorrect error handling for APIs that can raise exceptions #105375

Comments

erlend-aasland commented Jun 6, 2023 • edited by sobolevn

Linked PRs

This comment was marked as outdated.

vstinner commented Jun 9, 2023

vstinner commented Jun 9, 2023

vstinner commented Jun 9, 2023

sobolevn commented Jun 10, 2023

erlend-aasland commented Jun 6, 2023 •

edited by sobolevn