Skip to content

gh-77188: Name-mangled private methods don't roundtrip when pickling #21480

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Open
wants to merge 5 commits into
base: main
Choose a base branch
from

Conversation

ZackerySpytz
Copy link
Contributor

@ZackerySpytz ZackerySpytz commented Jul 15, 2020

Copy link
Contributor

@nanjekyejoannah nanjekyejoannah left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Makes sense, please fix the conflict for merge

@furkanonder
Copy link
Contributor

@ZackerySpytz Are you gonna continue this task? @nanjekyejoannah You can see the edited PR for this issue.

@ZackerySpytz
Copy link
Contributor Author

Yes, I will update this PR.

@kumaraditya303 kumaraditya303 added type-bug An unexpected behavior, bug, or error needs backport to 3.10 only security fixes needs backport to 3.11 only security fixes labels Dec 17, 2022
@python python deleted a comment from netlify bot Dec 17, 2022
@serhiy-storchaka
Copy link
Member

In general, LGTM.

But how much it affects the performance? Could you please run some microbenchmarks to measure the changes in the pickling time for non-private methods? If the difference is insignificant, it is good, but if it is large, it could be worth to optimize the code for non-private methods.

@hugovk hugovk removed the needs backport to 3.10 only security fixes label Apr 7, 2023
@erlend-aasland erlend-aasland changed the title bpo-33007: Name-mangled private methods don't roundtrip when pickling gh-77188: Name-mangled private methods don't roundtrip when pickling Jan 5, 2024
@serhiy-storchaka serhiy-storchaka added needs backport to 3.12 only security fixes needs backport to 3.13 bugs and security fixes and removed needs backport to 3.11 only security fixes labels May 9, 2024
vyrjana added a commit to vyrjana/lmfit-py that referenced this pull request Nov 23, 2024
…kling

- Attempting to make use of multiprocessing.Pool while providing, e.g., Minimizer.least_squares a function for calculating the Jacobian did not work prior to the renaming of the method due to an issue related to pickling and unpickling of private methods. See the (currently) open issue (python/cpython#77188) and pull request (python/cpython#21480) for more information.
newville pushed a commit to lmfit/lmfit-py that referenced this pull request Dec 5, 2024
* Updated the Minimizer.least_squares method to wrap the Jacobian function
- The Jacobian function provided by the user is now handled similarly to how it is handled in, e.g., Minimizer.leastsq.
- If the Jacobian function is called 'Dfun' in the keyword arguments, then it is also accepted (Minimizer.scalar_minimize already does this).

* Updated the Minimizer.__jacobian method signature to fix exception
- The signature now matches the signature of Minimizer.__residual. This was changed because the following exception was being raised whenever the user provided a function for calculating the Jacobian: "TypeError: Minimizer.__jacobian() got an unexpected keyword argument 'apply_bounds_transformation'".

* Updated the Minimizer.__jacobian method to use the new parameter
- The updating of the parameter values is now similar to how it is done in Minimizer.__residual.

* Updated test_least_squares_jacobian_types to handle the wrapped Jacobian function
- The params argument of the jac_array function is now an actual Parameters object like it is in the f function, which means that the parameters can be accessed via string keys.

* Updated the coerce_float64 function
- Added explicit support for sparse matrices (e.g., Block Sparse Row) and LinearOperator objects. This change is needed because the updated test_least_squares_jacobian_types would otherwise fail. Similar to how these types are handled when calculating the Hessian matrix in Minimizer.least_squares.

* Renamed the Minimizer.__jacobian method to avoid a bug related to pickling
- Attempting to make use of multiprocessing.Pool while providing, e.g., Minimizer.least_squares a function for calculating the Jacobian did not work prior to the renaming of the method due to an issue related to pickling and unpickling of private methods. See the (currently) open issue (python/cpython#77188) and pull request (python/cpython#21480) for more information.

* Added a test for when the "least_squares" method is provided a function for the Jacobian
- Verifies that a) the fitted parameters have the expected values, b) the jac function is actually called, and c) that there are indeed fewer function evaluations as a result.

* Updated the docstring of Minimizer._jacobian
- The docstring is updated to include the parameters and return values.

* Added tests for (un)pickling Minimizer when a Jacobian function is provided

* Updated timeout value in test_jacobian_with_forkingpickler
- Increased the timeout value so that slow runners can actually succeed instead of timing out. If no timeout was specified, then a failing test could end up hanging instead.

* Added an example that demonstrates the benefits of providing a Jacobian function

* Updated the list of contributors
@hugovk hugovk removed the needs backport to 3.12 only security fixes label Apr 25, 2025
@serhiy-storchaka serhiy-storchaka added the needs backport to 3.14 bugs and security fixes label May 8, 2025
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
awaiting core review needs backport to 3.13 bugs and security fixes needs backport to 3.14 bugs and security fixes type-bug An unexpected behavior, bug, or error
Projects
None yet
Development

Successfully merging this pull request may close these issues.