Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Fix #4037: Format sources with scalafmt #4260

Open
wants to merge 5 commits into
base: master
from

Conversation

@MaximeKjaer
Copy link
Contributor

@MaximeKjaer MaximeKjaer commented Oct 19, 2020

As discussed in #4037, as of Git 2.23, it is possible to ignore certain commits in a git blame (see this article). This effectively removes the main argument against automatic formatting, which is that it ruins git blame. This PR therefore introduces scalafmt to format all sources.

As the current coding style leaves some room for the code author to use some arbitrary judgment, the tool of course generates a very large diff, as it applies rules uniformly. I would argue that this is perhaps more of a good thing than a bad one.

Nevertheless, I still tried to keep the formatting close to the original rules, e.g. with a line width of 80 characters, 4 space indentation for call site continuations, etc (most of these are encoded in the Scala.js scalafmt preset). However, a few of the current coding style rules cannot be maintained when formatting with scalafmt; an incomplete list is below:

  • scalafmt does not support configuring spacing around : in _: Foo | _: Bar
  • scalafmt does not support newlines.implicitParamListModifierPrefer = before at the same time as binpack.preset = true, so I had to choose between binpacking and preferring line skips before the implicit keyword in parameter list rather than after; I prioritized binpacking.

If you see anything really ugly in the formatted source code, please let me know and I will try to see if I can reconfigure scalafmt to deal with it better.

I am also including a few other links to get the discussion started:

  • GitHub does not yet support .git-blame-ignore-revs on their Web interface, but there is a community request for it.
  • The .git-blame-ignore-revs approach is what is recommended by the (quite popular) Python formatter, black, for migrating coding styles without ruining git blame.
  • An example of a large project using .git-blame-ignore-revs is locust
  • Configuring Git with .git-blame-ignore-revs works both on the command line, and in VS Code (with no extra configuration; I am using GitLens for viewing git blame in the editor)

I still need to remove sections in the coding style that are no longer true, or no longer relevant because they don't need to be enforced manually anymore. However, I thought I would still open the PR as is now, to get the discussion started on this change.

@hepin1989
Copy link

@hepin1989 hepin1989 commented Oct 22, 2020

Will you organize the imports too?

@MaximeKjaer
Copy link
Contributor Author

@MaximeKjaer MaximeKjaer commented Oct 22, 2020

I could add the SortImports scalafmt rewrite rule, but note that this only sorts imports within curly brackets, not import statements (see scalameta/scalafmt#722). For sorting import statements, we would need to add scalafix, with the OrganizeImports rule. So seeing that another tool is needed to properly sort imports, I think it's best to not let scalafmt format imports.

@MaximeKjaer MaximeKjaer force-pushed the MaximeKjaer:scalafmt branch 3 times, most recently from b4c6c24 to 47ceb13 Oct 22, 2020
@MaximeKjaer
Copy link
Contributor Author

@MaximeKjaer MaximeKjaer commented Oct 27, 2020

@gzm0 gates are now passing

project/Build.scala Outdated Show resolved Hide resolved
@MaximeKjaer
Copy link
Contributor Author

@MaximeKjaer MaximeKjaer commented Oct 27, 2020

To-do:

  • Limit scalafmt parallelism to 1 to avoid race conditions
  • Add scalafmt and scalafmtCheck top-level commands (in addition to scalafmtAll and scalafmtCheckAll)
  • Disable scalafmt on scalalib
  • Rebase so that every commit passes CI
  • Edit coding style to reflect that scalafmt is the authoritative style guide
@MaximeKjaer MaximeKjaer force-pushed the MaximeKjaer:scalafmt branch from 47ceb13 to 7fa8592 Oct 29, 2020
@MaximeKjaer MaximeKjaer requested a review from gzm0 Oct 30, 2020
Copy link
Contributor

@gzm0 gzm0 left a comment

On a high level, three comments:

  • A bunch of ways to cause less commits.
  • Comments where it seems that scalafmt / the Scala.js style is buggy.
  • Some of the reformatting breaks other rules we have. How to fix that?
@@ -1146,8 +1152,14 @@ object Build {
delambdafySetting,
noClassFilesSettings,

// Ignore scalastyle for this project
// Ignore scalastyle and scalafmt for this project
scalastyleCheck := {},

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

There is another one of these in the partest project. I'm unsure whether this is just an artifact of the past. IIUC, the code in there was adopted from Scala, but by now is pretty much independent and patching it with upstream commits is not really something we do.

@sjrd any ideas about this?

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

Also, if we need this in multiple places, we should probably put it in a val ignoreCodeStyleSettings or something like this.

<check level="error" enabled="true" class="org.scalastyle.file.NewLineAtEofChecker"/>

<!--Indentation checker, whilst useful gives false positives on the file headers, so disabled for now-->
<check level="error" enabled="false" class="org.scalastyle.file.IndentationChecker"/>

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

Is this not the case anymore, that it gives false positives on the headers?

This comment has been minimized.

@MaximeKjaer

MaximeKjaer Nov 3, 2020
Author Contributor

I don't know if the false positives are gone or not, but in any case there's no need for scalastyle to check indentation anymore as indentation is now scalafmt's responsibility, and is checked by scalafmtCheckAll.

scalastyle-config.xml Show resolved Hide resolved
@@ -22,7 +22,8 @@ trait DoublePredicate { self =>
def and(other: DoublePredicate): DoublePredicate = {
new DoublePredicate {
def test(value: Double): Boolean =
self.test(value) && other.test(value) // the order and short-circuit are by-spec
self.test(value) && other.test(
value) // the order and short-circuit are by-spec

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

Ugh seriously :(

This comment has been minimized.

@MaximeKjaer

MaximeKjaer Nov 3, 2020
Author Contributor

This particular instance was fixed by increasing the line length limit. But in general, it's possible to fix this kind of issue manually in daily development. The thing about scalafmt is that it may not produce the prettiest results on the first try; its output depends somewhat on the whitespace in your input. For instance, suppose you start with this:

self.test(value) && other.test(value) // the order and short-circuit are by-spec

scalafmt may reformat to something ugly:

self.test(value) && other.test(
    value) // the order and short-circuit are by-spec

But if you're dissatisfied with this, you can make scalafmt break differently by manually editing to the following:

self.test(value) && 
other.test(
    value) // the order and short-circuit are by-spec

scalafmt will then reformat to something more correct:

self.test(value) && 
  other.test(value) // the order and short-circuit are by-spec

With format-on-save enabled in your IDE, this kind of thing becomes pretty seamless:

Peek 2020-11-03 18-23

Once we've settled on a stable list of config rules, I can manually go over the ugliest places and fix them.

@@ -1073,7 +1071,8 @@ abstract class GenJSCode[G <: Global with Singleton](val global: G)
!settings.noForwarders && sym.isStatic && !isImplClass(sym) && {
// Reject non-top-level objects unless opted in via the appropriate option
scalaJSOpts.genStaticForwardersForNonTopLevelObjects ||
!sym.name.containsChar('$') // this is the same test that scalac performs
!sym.name.containsChar(
'$') // this is the same test that scalac performs

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

These comment moves are unfortunate, not sure we can do anything about them :(

This comment has been minimized.

@MaximeKjaer

MaximeKjaer Nov 3, 2020
Author Contributor

For really long lines, there's nothing to be done, no. This example, however, is fixed by increasing max line length.

val methodIdent = encodeMethodSym(m)
val originalName = originalNameOfMethod(m)
val jsParams = m.tpe.params.map(genParamDef(_))
val resultType = toIRType(m.tpe.resultType)

js.MethodDef(flags, methodIdent, originalName, jsParams, resultType, Some {
js.MethodDef(flags, methodIdent, originalName, jsParams, resultType,
Some {
genApplyMethod(genLoadModule(moduleClass), m, jsParams.map(_.ref))

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

Another example of less indentation for nested stuff :(

reversedArgs =
genPrimitiveJSRepeatedParam(arg) reverse_::: reversedArgs
reversedArgs = genPrimitiveJSRepeatedParam(
arg) reverse_::: reversedArgs

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

Other example of bad assignment?


A *few* of these rules are checked automatically using Scalastyle, but most of them are too complex to teach to an automated tool.
While most rules are checked automatically by scalafmt and Scalastyle, but there are also a few rules that are too complex to teach to an automated tool.
This document tries to document the manually enforced rules as much as possible to make it easier for everyone to contribute.

This comment has been minimized.

@gzm0

gzm0 Oct 31, 2020
Contributor

I realize now that this is problematic: scalafmt's formatting on the current codebase breaks certain of the rules even though they were followed before (what I have seen is mostly blocks not being present). IMHO this is quite serious :-/

Is there any scalafmt mode that is much more aggressive and formats everything in a consistent manner (not maybe the manner we have right now)? IMO that might be a better alternative.

This comment has been minimized.

CODINGSTYLE.md Show resolved Hide resolved
@sjrd
Copy link
Member

@sjrd sjrd commented Oct 31, 2020

Last time I seriously looked at scalafmt for Scala.js, I came up with the following .scalafmt.conf:
https://github.com/sjrd/scala-js/blob/scalafmt/.scalafmt.conf
Maybe some of the things in there are worth including now.

Also, maybe this could be a good time to reconsider our 80 characters line limit. We've always had 80 soft limit and 120 hard limit. But scalafmt has no concept of soft limit, and that causes all sorts of ugly things, because it will favor an ugly formatting that stays under 80 over something clean that goes a bit over the limit. Maybe we would have way less ugly formattings if we simply dropped that 80 char limit.

MaximeKjaer added 5 commits Oct 29, 2020
Add scalafmt sbt commands.

Remove coding style rules deprecated by scalafmt, and scalastyle rules
overlapping with scalafmt.
@MaximeKjaer MaximeKjaer force-pushed the MaximeKjaer:scalafmt branch from ed43e35 to 4ce36db Nov 3, 2020
@MaximeKjaer
Copy link
Contributor Author

@MaximeKjaer MaximeKjaer commented Nov 3, 2020

Thank you @sjrd, I've rebased with something closer to your config. It's not possible to have both binpacking (which is enabled by default in the Scala.js preset) and newlines.implicitParamListModifierPrefer = before, so I removed the latter from your config.

I also increased the line length limit to 100, which seems to work fine; in the end, most lines are around 80, and a few lines go in the 80-100 range, which nicely follows the spirit of the "soft 80, hard 120" rule. For instance, here's a screenshot of a fairly representative part of GenJSCode.scala, with the editor ruler set to 80 characters:

image

@MaximeKjaer
Copy link
Contributor Author

@MaximeKjaer MaximeKjaer commented Nov 3, 2020

Why did the AppVeyor build break? Checking the source files, not much has changed in the offending files: library/src/main/scala/scala/scalajs/js/package.scala has barely changed (just whitespace in two places), and library/src/main/scala/scala/scalajs/concurrent/QueueExecutionContext.scala hasn't changed at all 😕

@sjrd
Copy link
Member

@sjrd sjrd commented Nov 3, 2020

Why did the AppVeyor build break?

That failure is very weird. I've relaunched the build just because I can't make sense of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Linked issues

Successfully merging this pull request may close these issues.

None yet

5 participants