Skip to content

[py]: Initial implementation of linting in CI #10628

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closed
wants to merge 8 commits into from

Conversation

symonk
Copy link
Member

@symonk symonk commented May 9, 2022

New Python Linting

Here's typically how this stuff will work for now, since we do not want pre-commit put in place; this utilises tox which currently handles our flake8 stuff. Main difference here is that both black and isort will rewrite files which are violating our rules, these will be unstaged and require you to re-stage the fixed versions however the python code base will be uniform across the board which is vital, we will also have less merge conflicts with more keeping more to a single line (e.g imports etc).

This adds the following to a new tox -e linting recipe; formerly tox -e flake8 (specifically).

  • flake8.
  • isort for consistent import management.
  • black for a consistent code base throughout all contributors.

Github actions have been updated as part of this work and will fail just like flake8 currently does when things are not uniform. Tools are now configured via the setup.cfg file for everything, we should consider moving to a pyproject.toml in future; but we use bazel so do not follow a normal 'python packaging' setup.

I have been adding commit hashes that do sweeping linting changes to .git-blame-ignore-revs which can be used in conjunction with git blame for checking who was responsible for what changes originally without everything coming back as me with the original changes.

Typical working flow:

  • Branch off trunk and do some work
  • Prior to committing; tox -e linting
  • If tox linting is not exiting 0 due to changes in black or isort those files will be unstaged and amended automatically, re-stage them and apply tox -e linting again until it is good (this is handled automatically by pre-commit which is why I was pushing this so strongly)
  • flake8 fixes will manually need resolved as they currently do

We are so far away from having a good mypy coverage but we can work towards it in future and consider adding it into the tox -e linting recipe.

Negatives (for now):

  • Since we are allowing pushing without enforcing the stuff is good; remote side can error so --no-verify is not a valid work around, I don't really see that as a big 'negative' at all but something to be aware of.

Outstanding tasks:

  • Make CI apply it and not only flake8.
  • Black requires a toml
  • Should this only run against staged files?
  • Ensure git blame is not impacted by the initial rewriting commits.
  • Update CONTRIBUTING.md to reflect the workflow for new contributors
  • various other teething issues

@symonk symonk added the C-py Python Bindings label May 14, 2022
@titusfortner titusfortner added this to the 4.3 milestone May 24, 2022
@symonk symonk closed this Jun 11, 2022
@symonk symonk deleted the py-tox-black branch June 11, 2022 08:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
C-py Python Bindings
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants