Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add sccache to the Windows job #6231

Merged
merged 2 commits into from Feb 6, 2020
Merged

Conversation

Smjert
Copy link
Member

@Smjert Smjert commented Feb 5, 2020

Also fix debug symbols on googletest,
so that it doesn't try to create a separate pdb.

This is a tool which is similar to ccache, but it will be used for Windows.
After the first round of cache generation, the build time should be halved to around 20mins.
We have to use Ninja instead of msbuild to build because the latter is not properly supported, moreover we are using a custom build of sccache since it has a fix for the support of a MSVC debug flag that has yet to be upstreamed.

Also fix debug symbols on googletest,
so that it doesn't try to create a separate pdb.
@Smjert
Copy link
Member Author

@Smjert Smjert commented Feb 6, 2020

This needs a small fix that I'm going to quickly do, because it has a side effect I did not expect with the submodule cache.

@Smjert Smjert force-pushed the stefano/ci/sccache branch from ed07b2f to 149d0a0 Feb 6, 2020
@Smjert
Copy link
Member Author

@Smjert Smjert commented Feb 6, 2020

Still doesn't work... context:

Information, Processed 0 files from D:\a\1\s\.git\modules successfully.
Information, Uploaded 0 out of 61 bytes
Information, Content upload is done!
Information, 
Content upload statistics:
Total Content: 0.0 MB
Physical Content Uploaded: 0.0 MB
Logical Content Uploaded: 0.0 MB
Compression Saved: 0.0 MB
Deduplication Saved: 0.0 MB
Number of Chunks Uploaded: 0

It's not uploading files inside that folder, which means that the cache for the submodules will be empty for the next run.
The benefit from this PR is bigger than the submodule cache regression, if anyone wants to merge it right away, othewise I'll try to investigate more on what's happening later.

@Smjert Smjert force-pushed the stefano/ci/sccache branch from f5b9ff9 to a2feab7 Feb 6, 2020
@Smjert
Copy link
Member Author

@Smjert Smjert commented Feb 6, 2020

Finally found the fix, now it should be good to go.

sccache needs its cache to be uploaded as normal files,
not with a tar, otherwise the cache when extracted
will contain extraneous files and folders (PaxHeader).
The Files mode though by default ignores the .git folder,
so we have to use the .artifactignore file to unignore it,
and we have to copy in the cached path for the cache task to see it.
@Smjert Smjert force-pushed the stefano/ci/sccache branch from a2feab7 to fb7e809 Feb 6, 2020
@theopolis
Copy link
Member

@theopolis theopolis commented Feb 6, 2020

Awesome work!

@Smjert Smjert merged commit 9a99c6b into osquery:master Feb 6, 2020
13 checks passed
@Smjert Smjert deleted the stefano/ci/sccache branch Feb 14, 2020
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

2 participants