Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

bpo-39028: Performance enhancement in keyword extraction #17576

Merged
merged 2 commits into from Dec 18, 2019

Conversation

@seberg
Copy link
Contributor

seberg commented Dec 12, 2019

All keywords should first be checked for pointer identity. Only
after that failed for all keywords (unlikely) should unicode
equality be used.
The original code would call unicode equality on any non-matching
keyword argument. Meaning calling it often e.g. when a function
has many kwargs but only the last one is provided.


Unless I am missing something big, this seems like it was the original intention, and simple typo. It may make sense to backport?

I have to admit, I have not actually recompiled and timed this, I may do that tomorrow (but I have never had the need to compile python, so need to look up how).

https://bugs.python.org/issue39028

@seberg seberg changed the title ENH: Fix performance issue in keyword extraction bpo-39028: Fix performance issue in keyword extraction Dec 12, 2019
@seberg

This comment has been minimized.

Copy link
Contributor Author

seberg commented Dec 12, 2019

Ah, I missed that this code is used for PyArg_* where it is not clear that strings are almost always interned, so the change would likely be useful for argument clinic functions, but probably not in general.

EDIT: Both can make sense I guess, but for the clinic it is clearly interned I think? So another option is to have two versions here...

if (kwname == key) {
return kwstack[i];
}
}
/* ptr == ptr should normally find a match in since keyword keys

This comment has been minimized.

Copy link
@serhiy-storchaka

serhiy-storchaka Dec 12, 2019

Member

This comment is related to kwname == key.

Copy link
Member

serhiy-storchaka left a comment

This change LGTM, but please move the comment before the pointers comparison or before the first loop.

Although this is almost dead code. It is used in just few sites generated by Argument Clinic and soon will not be used at all.

@serhiy-storchaka

This comment has been minimized.

Copy link
Member

serhiy-storchaka commented Dec 12, 2019

Ah, sorry, the use in _PyArg_UnpackKeywords() is not dead, so this optimization has long term effect.

@seberg seberg force-pushed the seberg:kwarg-extract-performance branch from ba65512 to 17561c2 Dec 12, 2019
@seberg

This comment has been minimized.

Copy link
Contributor Author

seberg commented Dec 12, 2019

Yeah, although it only will have remotely noticeable effects for >3 kwargs probably, which seems pretty rare in python. Anyway moved and tweaked the comment.

@seberg seberg changed the title bpo-39028: Fix performance issue in keyword extraction bpo-39028: Performance enhancement in keyword extraction Dec 12, 2019
}
/* This function assumes the strings should be interned, so that this
should only be reached on error, and the loop below will never
find a match */

This comment has been minimized.

Copy link
@methane

methane Dec 13, 2019

Member

If we keep the second loop, why we need to have both of find_keyword_interned and find_keyword?

This comment has been minimized.

Copy link
@seberg

seberg Dec 13, 2019

Author Contributor

Oh man, sorry, I do not think we do. I was trying around with it a bit, and apparently forgot to commit before pushing or something :/. fixed.

All keywords should first be checked for pointer identity. Only
after that failed for all keywords (unlikely) should unicode
equality be used.
The original code would call unicode equality on any non-matching
keyword argument. Meaning calling it often e.g. when a function
has many kwargs but only the last one is provided.
@seberg seberg force-pushed the seberg:kwarg-extract-performance branch from 17561c2 to 52562b9 Dec 13, 2019
@methane

This comment has been minimized.

Copy link
Member

methane commented Dec 16, 2019

@methane methane merged commit 75bb07e into python:master Dec 18, 2019
9 checks passed
9 checks passed
Docs
Details
Windows (x86)
Details
Windows (x64)
Details
macOS
Details
Ubuntu
Details
Azure Pipelines PR #20191217.42 succeeded
Details
bedevere/issue-number Issue number 39028 found
Details
bedevere/news News entry found in Misc/NEWS.d
continuous-integration/travis-ci/pr The Travis CI build passed
Details
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
5 participants
You can’t perform that action at this time.