Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Accelerate build context directory file walk #1442

Open
wants to merge 3 commits into
base: master
from

Conversation

@swiedenfeld
Copy link
Contributor

@swiedenfeld swiedenfeld commented Jul 27, 2020

This PR addresses the performance issue described in #1409.

#1412 has partially improved performance by memoizing frequently reused ignore patterns.
These commits now improve performance on my Windows 10 machine further by:

  • Use Java NIO Files.walk to efficiently walk the build context directory. It visits files and subdirectories in parallel and filters any items efficiently ignored
  • Use java.nio.file.Path#toUri instead of java.io.File#toUri, because it executes faster

With #1412 and these code changes, build context scans in my unit test setup (see #1412) on my Windows 10 machine now pass in about 6 seconds, while they previously used to take around 27 seconds.

swiedenfeld added 3 commits Jul 26, 2020
java.nio.file.Path#toUri executes faster compared to java.io.File#toUri
@swiedenfeld
Copy link
Contributor Author

@swiedenfeld swiedenfeld commented Aug 2, 2020

@bsideup I could use some input on this :)

@swiedenfeld
Copy link
Contributor Author

@swiedenfeld swiedenfeld commented Aug 21, 2020

ping (before this goes stale)

@stale
Copy link

@stale stale bot commented Sep 20, 2020

This issue has been automatically marked as stale because it has not had recent activity. It will be closed if no further activity occurs. Thank you for your contributions.

@stale stale bot added the resolution/stale label Sep 20, 2020
@stale stale bot closed this Sep 27, 2020
@bsideup bsideup reopened this Sep 27, 2020
@stale stale bot removed the resolution/stale label Sep 27, 2020
@bsideup
Copy link
Member

@bsideup bsideup commented Sep 27, 2020

@swiedenfeld sorry for letting it stale, I will try to find time and have a look at it soon!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants
You can’t perform that action at this time.