Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Fastpath for encoding unsorted dict to JSON #95385

Closed
aivarsk opened this issue Jul 28, 2022 · 1 comment
Closed

Fastpath for encoding unsorted dict to JSON #95385

aivarsk opened this issue Jul 28, 2022 · 1 comment
Labels
performance Performance or resource usage type-feature A feature request or enhancement

Comments

@aivarsk
Copy link
Contributor

@aivarsk aivarsk commented Jul 28, 2022 •
edited

JSON encoder can produce output sorted by key (json.dumps(sort_keys=True)) The default value for sort_keys is False. The code always creates a list of dictionary items, sorts them if needed, and then iterates through them.
However, if sorting is not requested we can use PyDict_Next to avoid allocating a list of tuples.

pyperformance on my machine before changes:

### json_dumps ###
Mean +- std dev: 10.6 ms +- 0.1 ms

pyperformance after changes:

### json_dumps ###
Mean +- std dev: 9.00 ms +- 0.08 ms
@aivarsk aivarsk added the type-feature A feature request or enhancement label Jul 28, 2022
@bedevere-bot bedevere-bot mentioned this issue Jul 28, 2022
Merged
@mdboom mdboom added the performance Performance or resource usage label Jul 28, 2022
@aivarsk
Copy link
Contributor Author

@aivarsk aivarsk commented Jul 29, 2022

Patched pyperformance to use json.dumps(sort_keys=True)

before changes

### json_dumps ###
Mean +- std dev: 14.0 ms +- 0.2 ms

after changes

### json_dumps ###
Mean +- std dev: 13.6 ms +- 0.2 ms

Probably because the new code does not create an iterator from the list and PyList_GET_ITEM is a bit faster than PyIter_Next (extra incref/decref)?

@corona10 corona10 closed this as completed Aug 6, 2022
corona10 pushed a commit that referenced this issue Aug 6, 2022
@aivarsk
gh-95385 Fastpath for encoding dict to JSON (gh-95374)
15f4a35
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
performance Performance or resource usage type-feature A feature request or enhancement
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@mdboom @aivarsk @corona10