Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

perf: Prevent unnecessary calls to format() #1429

Open
wants to merge 2 commits into
base: master
from

Conversation

@nwoltman
Copy link
Contributor

@nwoltman nwoltman commented May 28, 2016

Currently, the input SQL string is being formatted for every single query, which creates unnecessary overhead when there are no input values to use for formatting. This change prevents format() from being called if there is no user input for values (unless the user is using a custom queryFormat function, which is always called).

Benchmark results of calling connection.query() with no values argument
Before This PR Speedup (PR/before)
3,195,399 ops/sec 6,809,493 ops/sec 2.13
closes #1416
@nwoltman nwoltman force-pushed the nwoltman:query-format branch from 4a8b818 to 546715f May 28, 2016
@nwoltman nwoltman changed the title Prevent unnecessary calls to format() perf: Prevent unnecessary calls to format() May 28, 2016
@dougwilson
Copy link
Member

@dougwilson dougwilson commented May 28, 2016

Thanks! Unfortunately I'm not comfortable pulling this in due to breaking people's code. For example, I have seen SO answers using the "queryFormat" to alter the SQL (rather than purely inserting the values into the SQL), and of course our documentation in the README implies that "queryFormat" is always called, even when there are no values.

Perhaps there is another way to achieve the performance improvement you are trying to get? What is the performance different, anyhow?

@nwoltman nwoltman force-pushed the nwoltman:query-format branch from 546715f to f8072f6 May 28, 2016
@nwoltman
Copy link
Contributor Author

@nwoltman nwoltman commented May 28, 2016

I updated the changes so that the custom queryFormat function is always called, but SqlString.format is only called when needed. I also added benchmark results to the PR description. Just a heads up, the only way I could get the benchmark to finish without running out of memory was to comment out these two lines.

@dougwilson
Copy link
Member

@dougwilson dougwilson commented May 28, 2016

What if the guard was just directly added within the SqlString.format? Does that still have the same speedup? It would mean that the speed up extends to anyone using our format in any context (for example, using it within queryFormat).

@nwoltman
Copy link
Contributor Author

@nwoltman nwoltman commented May 28, 2016

I'm looking into addressing performance inside SqlString.format in a separate PR, but this PR still has merit. After running a benchmark, I was actually surprised to see that there is a noticeable performance difference between having the guard inside SqlString.format and having it outside:

Outside (this PR) Inside
6,809,493 ops/sec 6,440,577 ops/sec
@dougwilson
Copy link
Member

@dougwilson dougwilson commented May 28, 2016

Gotcha. I'm afraid for people who have overwritten SqlString.format , which is possible, they would break, and I'm just not confortable breaking them. I can hold this for when 3.0 comes, though.

@dougwilson
Copy link
Member

@dougwilson dougwilson commented May 28, 2016

By the way, if you have other changes, see if you can PR them this weekend if you would like to see them land in 2.11.0. I need to fix a bug introduced by one of these SqlString optimizations (not one of yours), and then master will be ready for 2.11, so just a heads up for anything else you want to include :)

@nwoltman
Copy link
Contributor Author

@nwoltman nwoltman commented May 28, 2016

Oh wow, I didn't think people would overwrite SqlString.format 😕
But if that's the case I'm totally fine with holding off on this until v3.0 👍

And I'll have a PR for SqlString.format out later today :)

@dougwilson dougwilson force-pushed the mysqljs:master branch from 8085d13 to e609a37 May 28, 2016
@dougwilson dougwilson force-pushed the mysqljs:master branch 3 times, most recently from 65c4c0c to fa96a75 Jun 6, 2016
@dougwilson dougwilson self-assigned this Nov 2, 2016
@dougwilson dougwilson force-pushed the mysqljs:master branch 2 times, most recently from 638d79d to 88bade0 Jun 29, 2017
@dougwilson dougwilson force-pushed the mysqljs:master branch 2 times, most recently from 946727b to 37fbbdd Nov 16, 2018
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Linked issues

Successfully merging this pull request may close these issues.

None yet

3 participants
You can’t perform that action at this time.