Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

gh-93735: Split Docs CI to speed-up the build #93736

Merged
merged 4 commits into from Jun 15, 2022
Merged

gh-93735: Split Docs CI to speed-up the build #93736

merged 4 commits into from Jun 15, 2022

Conversation

AA-Turner
Copy link
Member

@AA-Turner AA-Turner commented Jun 11, 2022

#93735

A

@AA-Turner AA-Turner added type-feature docs needs backport to 3.10 needs backport to 3.11 skip news and removed awaiting review labels Jun 11, 2022
@AA-Turner AA-Turner mentioned this pull request Jun 11, 2022
Open
@AA-Turner
Copy link
Member Author

@AA-Turner AA-Turner commented Jun 11, 2022

The diff is rendered badly here, but in effect I copy and pasted the job, removed doctest from the first and replaced the python build with actions/setup-python, and for the second I removed make check and make html.

A

hugovk
hugovk reviewed Jun 11, 2022
View changes
Copy link
Member

@hugovk hugovk left a comment

A nice speed improvement!

Looking at an earlier build (~15 mins):

image

This has parallel docs (~4 min) and doctest (~7 min):

image

and

image

So even if they ran in serial (they won't), it's still quicker. In part it's because "Build HTML documentation" only takes 2 min instead of 6 min (or sometimes up to 15 min)!

.github/workflows/doc.yml Outdated Show resolved Hide resolved
.github/workflows/doc.yml Show resolved Hide resolved
.github/workflows/doc.yml Outdated Show resolved Hide resolved
@AA-Turner
Copy link
Member Author

@AA-Turner AA-Turner commented Jun 11, 2022

CI failed because of:

There is no default python version for this setup-python major version, the action requires to specify either python-version input or python-version-file input. If the python-version input is not specified the action will try to read required version from file from python-version-file input.

This is a mildly annoying change, I quite like using the default of "give me the latest stable Python".

A

ezio-melotti
ezio-melotti reviewed Jun 12, 2022
View changes
.github/workflows/doc.yml Show resolved Hide resolved
.github/workflows/doc.yml Outdated Show resolved Hide resolved
.github/workflows/doc.yml Outdated Show resolved Hide resolved
.github/workflows/doc.yml Show resolved Hide resolved
@AA-Turner
Ezio's comments
e0cffa9 
- Split to SPHINXOPTS and SPHINXERRORHANDLING
- Remove SPHINXOPTS from make check
hugovk
hugovk approved these changes Jun 12, 2022
View changes
@rhettinger
Copy link
Contributor

@rhettinger rhettinger commented Jun 13, 2022

+1 from me.

@AA-Turner AA-Turner requested a review from JelleZijlstra Jun 14, 2022
JelleZijlstra
JelleZijlstra approved these changes Jun 15, 2022
View changes
Copy link
Member

@JelleZijlstra JelleZijlstra left a comment

Looks good, but I'm no expert on CI syntax.

@JulienPalard JulienPalard merged commit 4f26963 into python:main Jun 15, 2022
15 checks passed
@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jun 15, 2022

Thanks @AA-Turner for the PR, and @JulienPalard for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10, 3.11.
🐍🍒⛏🤖

@JulienPalard
Copy link
Member

@JulienPalard JulienPalard commented Jun 15, 2022

Good idea!

miss-islington pushed a commit to miss-islington/cpython that referenced this issue Jun 15, 2022
@AA-Turner @miss-islington
pythongh-93735: Split Docs CI to speed-up the build (pythonGH-93736)
3c7cf3e 
(cherry picked from commit 4f26963)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Jun 15, 2022

GH-93831 is a backport of this pull request to the 3.11 branch.

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jun 15, 2022

Sorry, @AA-Turner and @JulienPalard, I could not cleanly backport this to 3.10 due to a conflict.
Please backport using cherry_picker on command line.
cherry_picker 4f26963526f386bba84de8e14962163bfd5da955 3.10

@AA-Turner
Copy link
Member Author

@AA-Turner AA-Turner commented Jun 15, 2022

@JulienPalard do you want me to have a go doing the 3.10 backport?

A

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jun 15, 2022

Thanks @AA-Turner for the PR, and @JulienPalard for merging it 🌮🎉.. I'm working now to backport this PR to: 3.10.
🐍🍒⛏🤖

@miss-islington
Copy link
Contributor

@miss-islington miss-islington commented Jun 15, 2022

Sorry @AA-Turner and @JulienPalard, I had trouble checking out the 3.10 backport branch.
Please backport using cherry_picker on command line.
cherry_picker 4f26963526f386bba84de8e14962163bfd5da955 3.10

@AA-Turner
Copy link
Member Author

@AA-Turner AA-Turner commented Jun 15, 2022 •
edited

It won't be a clean backport as the docs job was split into three steps in 3.11 -- see https://github.com/python/cpython/blob/3.10/.github/workflows/doc.yml

A

AA-Turner added a commit to AA-Turner/cpython that referenced this issue Jun 15, 2022
@AA-Turner
[3.10] pythongh-93735: Split Docs CI to speed-up the build (pythonGH-…
40978d7 
…93736).

(cherry picked from commit 4f26963)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
@bedevere-bot
Copy link

@bedevere-bot bedevere-bot commented Jun 15, 2022

GH-93833 is a backport of this pull request to the 3.10 branch.

@AA-Turner
Copy link
Member Author

@AA-Turner AA-Turner commented Jun 15, 2022

make check using sphinx-lint hasn't been backported to 3.10, so I retained suspicious in the backport PR.

A

@AA-Turner AA-Turner mentioned this pull request Jun 15, 2022
Closed
miss-islington added a commit that referenced this issue Jun 15, 2022
@miss-islington @AA-Turner
gh-93735: Split Docs CI to speed-up the build (GH-93736)
7db66d8 
(cherry picked from commit 4f26963)

Co-authored-by: Adam Turner <9087854+AA-Turner@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@JulienPalard JulienPalard

@JelleZijlstra JelleZijlstra

@hugovk hugovk

@ezio-melotti ezio-melotti

Assignees

@JulienPalard JulienPalard

Labels
docs skip news type-feature
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.

None yet
8 participants
@AA-Turner @rhettinger @miss-islington @JulienPalard @bedevere-bot @JelleZijlstra @hugovk @ezio-melotti