CLN remove unnecessary trailing commas to get ready for new version of black #35925

MarcoGorelli · 2020-08-27T15:52:10Z

The new version of black is consistent in how it handles the magic trailing commas. So if we upgrade black and apply it, lots of files will be changed. However, the diff needn't be so large if we remove unnecessary trailing commas before upgrading.

E.g. in pandas/core/aggregation.py there is

def reconstruct_func(
    func: Optional[AggFuncType], **kwargs,
) -> Tuple[
    bool, Optional[AggFuncType], Optional[List[str]], Optional[List[int]],
]:

which has an unnecessary trailing comma.

The new version of black would transform this as

def reconstruct_func(
    func: Optional[AggFuncType],
    **kwargs,
) -> Tuple[bool, Optional[AggFuncType], Optional[List[str]], Optional[List[int]],]:

However, if we instead remove the trailing comma and write it as

def reconstruct_func(
    func: Optional[AggFuncType], **kwargs
) -> Tuple[bool, Optional[AggFuncType], Optional[List[str]], Optional[List[int]]]:

then both the current and the new versions of black will be OK with it.

So, PRs to remove some unnecessary trailing commas would be welcome - perhaps keep each PR limited to 5-10 files changed.

Files that (may) need changing are:

MarcoGorelli · 2020-08-27T16:24:15Z

See #35930, #35949, #35950 and for examples of how to open a PR for this.

No need to ask for permission to work on this, just choose a few files to work on and open a PR (optionally, you can comment here indicating that you're working on them)

jorisvandenbossche · 2020-08-27T21:21:02Z

I think the easiest way to find them is actually to let black (using the latest version) reformat the file(s), and then check in the diff for which ones the trailing comma can be removed (and then run black again)

jpribyl · 2020-08-28T16:25:54Z

I am looking at these right now

pandas/core/internals/concat.py
pandas/core/internals/managers.py
pandas/core/internals/ops.py
pandas/core/nanops.py
pandas/core/ops/docstrings.py
pandas/core/reshape/concat.py
pandas/core/reshape/pivot.py
pandas/core/reshape/reshape.py

edit: #35956

@MarcoGorelli if it looks good I can open another with some more later today.

MarcoGorelli · 2020-08-28T16:46:09Z

@jpribyl thanks! Yes, feel free to do so, any help here would be appreciated!

jpribyl · 2020-08-28T17:56:53Z

#35959

with several more files:

pandas/core/series.py
pandas/core/window/ewm.py
pandas/core/window/indexers.py
pandas/core/window/numba_.py
pandas/core/window/rolling.py
pandas/io/formats/css.py
pandas/io/formats/format.py
pandas/io/orc.py


        Comma cleanup for pandas-dev#35925


        Comma cleanup for pandas-dev#35925

metehankutlu · 2020-08-30T15:38:25Z

working on these right now:

pandas/tests/test_multilevel.py
pandas/tests/test_nanops.py
pandas/tests/window/moments/test_moments_consistency_rolling.py
pandas/tests/window/moments/test_moments_ewm.py
pandas/tests/window/moments/test_moments_rolling.py
pandas/tests/window/test_base_indexer.py
pandas/tests/window/test_pairwise.py
pandas/tests/window/test_rolling.py
pandas/tseries/frequencies.py
pandas/util/_test_decorators.py

#35996

metehankutlu · 2020-08-30T16:28:42Z

I'm searching using this regex: ,\n? *\)|,\n? *\] What do you guys think? Is it the correct way?

Edit: ,\n? *(\)|\]|\})

MarcoGorelli · 2020-08-30T16:46:13Z

I'm searching using this regex: ,\n? *\)|,\n? *\] What do you guys think? Is it the correct way?

Edit: ,\n? *(\)|\]|\})

I would just upgrade black, run it on the chosen files, see what's changed, remove trailing commas, and run black again


        Comma cleanup for pandas-dev#35925


        Comma cleanup for pandas-dev#35925


        Comma cleanup for pandas-dev#35925

rajanshoo25 · 2020-09-01T06:26:43Z

I would just upgrade black, run it on the chosen files, see what's changed, remove trailing commas, and run black again

@MarcoGorelli when I am running black again, it changes file back with trailing commas

MarcoGorelli · 2020-09-01T07:47:53Z

I would just upgrade black, run it on the chosen files, see what's changed, remove trailing commas, and run black again

@MarcoGorelli when I am running black again, it changes file back with trailing commas

That's fine. Two things may happen when running the new version of black:

the command is long, and so it splits it into multiple lines, each with a trailing comma;
the command is short, and so it puts it all on one line.

It's the second case in which we want to remove the trailing commas, so that both the current and the new version of black would reformat that line in the same way.

From a quick glance it looks like you've done it correctly anyway, will review later today


        Comma cleanup for #35925 (#36023)


        remove trailing commas for #35925 (#36029)

tiagohonorato · 2020-09-01T21:25:19Z

Working on these files:

pandas/io/sas/sas_xport.py
pandas/io/stata.py
pandas/tests/arithmetic/test_interval.py
pandas/tests/arithmetic/test_numeric.py
pandas/tests/arrays/boolean/test_logical.py

sarthakchaudhary13 · 2020-09-01T21:33:41Z

Now Working on

pandas/tests/scalar/test_na_scalar.py
pandas/tests/scalar/timestamp/test_arithmetic.py
pandas/tests/scalar/timestamp/test_constructors.py
pandas/tests/series/methods/test_argsort.py
pandas/tests/series/methods/test_convert_dtypes.py
pandas/tests/series/methods/test_drop_duplicates.py
pandas/tests/series/methods/test_interpolate.py
pandas/tests/series/methods/test_unstack.py
pandas/tests/series/test_cumulative.py
pandas/tests/test_algos.py

xcz011 · 2020-09-02T02:40:53Z

Working on:

pandas/tests/groupby/aggregate/test_numba.py
pandas/tests/groupby/test_apply.py
pandas/tests/groupby/test_categorical.py
pandas/tests/groupby/test_groupby_dropna.py
pandas/tests/groupby/test_groupby_subclass.py
pandas/tests/groupby/test_size.py
pandas/tests/groupby/test_timegrouper.py
pandas/tests/groupby/transform/test_numba.py


        Comma cleanup for #35925 (#36058)

sarthakchaudhary13 · 2020-09-02T18:15:36Z

Working on these files

pandas/tests/io/test_sql.py
pandas/tests/io/test_stata.py
pandas/tests/plotting/test_frame.py
pandas/tests/resample/test_datetime_index.py
pandas/tests/reshape/merge/test_merge_index_as_string.py
pandas/tests/reshape/test_crosstab.py
pandas/tests/reshape/test_get_dummies.py


        Comma cleanup for pandas-dev#35925 (pandas-dev#36023)


        remove trailing commas for pandas-dev#35925 (pandas-dev#36029)


        Comma cleanup for pandas-dev#35925 (pandas-dev#36058)

satrio-hw · 2020-09-07T12:01:11Z

hello, I'm new to this community.. Now I'm working on these files :

pandas/tests/arrays/categorical/test_replace.py
pandas/tests/arrays/sparse/test_array.py
pandas/tests/arrays/test_array.py
pandas/tests/arrays/test_timedeltas.py
pandas/tests/base/test_conversion.py
pandas/tests/dtypes/test_missing.py
pandas/tests/extension/base/methods.py
pandas/tests/extension/test_sparse.py
pandas/tests/frame/indexing/test_setitem.py


        CLN:pandas-dev#35925

tiagohonorato · 2020-09-08T12:44:37Z

Working on:

pandas/tests/io/pytables/test_timezones.py
pandas/tests/io/test_feather.py
pandas/tests/io/test_s3.py
pandas/tests/io/test_sql.py
pandas/tests/io/test_stata.py
pandas/tests/plotting/test_frame.py
pandas/tests/resample/test_datetime_index.py
pandas/tests/reshape/merge/test_merge_index_as_string.py
pandas/tests/reshape/test_crosstab.py
pandas/tests/reshape/test_get_dummies.py


        CLN: remove unnecessary trailing commas on issues #35925 (#36193)


        CLN: remove unnecessary trailing commas on issues pandas-dev#35925 (p…

…andas-dev#36193)

cjlynch278 · 2020-09-11T17:52:29Z

Working On:
pandas/tests/series/methods/test_interpolate.py
pandas/tests/series/methods/test_unstack.py
pandas/tests/series/test_cumulative.py
pandas/tests/test_algos.py

MarcoGorelli added Clean good first issue labels Aug 27, 2020

simonjayhawkins added this to the Contributions Welcome milestone Aug 27, 2020

This was referenced Aug 27, 2020

remove unnecessary trailing commas #35926

Closed

CLN remove unnecessary trailing commas #35927

Closed

CLN remove unnecessary trailing commas from aggregation #35930

Merged

This was referenced Aug 28, 2020

CLN remove unnecessary trailing commas to get ready for new version of black: _testing -> generic #35949

Merged

CLN remove unnecessary trailing commas to get ready for new version of black: generic -> blocks #35950

Merged

jpribyl mentioned this issue Aug 28, 2020

Issue35925 remove trailing commas #35956

Merged

jpribyl mentioned this issue Aug 28, 2020

Issue35925 remove more trailing commas #35959

Merged

This was referenced Aug 29, 2020

Comma cleanup for Issue #35925 #35970

Closed

Comma cleanup #35971

Closed

JonathanShrek added a commit to JonathanShrek/pandas that referenced this issue Aug 30, 2020

Comma cleanup for pandas-dev#35925

d097609

JonathanShrek added a commit to JonathanShrek/pandas that referenced this issue Aug 30, 2020

Comma cleanup for pandas-dev#35925

c34ed0e

metehankutlu mentioned this issue Aug 30, 2020

Issue35925 Remove trailing commas #35996

Merged

JonathanShrek added a commit to JonathanShrek/pandas that referenced this issue Sep 1, 2020

Comma cleanup for pandas-dev#35925

26c8c23

JonathanShrek added a commit to JonathanShrek/pandas that referenced this issue Sep 1, 2020

Comma cleanup for pandas-dev#35925

da833f1

JonathanShrek added a commit to JonathanShrek/pandas that referenced this issue Sep 1, 2020

Comma cleanup for pandas-dev#35925

21c0cca

JonathanShrek mentioned this issue Sep 1, 2020

Comma cleanup for #35925 #36023

Merged

rajanshoo25 mentioned this issue Sep 1, 2020

remove trailing commas for #35925 #36029

Merged

3 of 3 tasks complete

sarthakchaudhary13 mentioned this issue Sep 1, 2020

CLN remove unnecessary trailing commas in pandas/io #36052

Merged

3 of 3 tasks complete

tiagohonorato mentioned this issue Sep 1, 2020

CLN remove unnecessary trailing commas #36057

Merged

xcz011 mentioned this issue Sep 2, 2020

CLN remove unnecessary trailing commas in groupby tests #36059

Merged

sarthakchaudhary13 mentioned this issue Sep 2, 2020

CLN clearing unnecessary trailing commas #36073

Merged

sarthakchaudhary13 mentioned this issue Sep 3, 2020

CLN removing trailing commas #36101

Merged

rxxg added a commit to rxxg/pandas that referenced this issue Sep 5, 2020

Comma cleanup for pandas-dev#35925 (pandas-dev#36023)

1e1a3ba

rxxg added a commit to rxxg/pandas that referenced this issue Sep 5, 2020

remove trailing commas for pandas-dev#35925 (pandas-dev#36029)

39f6253

rxxg added a commit to rxxg/pandas that referenced this issue Sep 5, 2020

Comma cleanup for pandas-dev#35925 (pandas-dev#36058)

a28bd9c

satrio-hw pushed a commit to satrio-hw/pandas that referenced this issue Sep 7, 2020

CLN:pandas-dev#35925

de14931

satrio-hw mentioned this issue Sep 7, 2020

CLN: remove unnecessary trailing commas on issues #35925 #36193

Merged

tiagohonorato mentioned this issue Sep 8, 2020

CLN remove trailing commas #36222

Merged

3 of 10 tasks complete

rxxg added a commit to rxxg/pandas that referenced this issue Sep 10, 2020

CLN: remove unnecessary trailing commas on issues pandas-dev#35925 (p…

49b9c32

…andas-dev#36193)

cjlynch278 mentioned this issue Sep 11, 2020

CLN: Remove trailing commas #36295

Open

0 of 5 tasks complete

pandas-dev / pandas

CLN remove unnecessary trailing commas to get ready for new version of black #35925

CLN remove unnecessary trailing commas to get ready for new version of black #35925

MarcoGorelli commented Aug 27, 2020 •

edited

MarcoGorelli commented Aug 27, 2020 •

edited

jorisvandenbossche commented Aug 27, 2020

jpribyl commented Aug 28, 2020 •

edited

MarcoGorelli commented Aug 28, 2020

jpribyl commented Aug 28, 2020

metehankutlu commented Aug 30, 2020 •

edited

metehankutlu commented Aug 30, 2020 •

edited

MarcoGorelli commented Aug 30, 2020

rajanshoo25 commented Sep 1, 2020

MarcoGorelli commented Sep 1, 2020 •

edited

tiagohonorato commented Sep 1, 2020

sarthakchaudhary13 commented Sep 1, 2020

xcz011 commented Sep 2, 2020

sarthakchaudhary13 commented Sep 2, 2020 •

edited

satrio-hw commented Sep 7, 2020

tiagohonorato commented Sep 8, 2020

cjlynch278 commented Sep 11, 2020

pandas-dev / pandas

Sponsor pandas-dev/pandas

Join GitHub today

CLN remove unnecessary trailing commas to get ready for new version of black #35925

CLN remove unnecessary trailing commas to get ready for new version of black #35925

Comments

MarcoGorelli commented Aug 27, 2020 • edited

MarcoGorelli commented Aug 27, 2020 • edited

jorisvandenbossche commented Aug 27, 2020

jpribyl commented Aug 28, 2020 • edited

MarcoGorelli commented Aug 28, 2020

jpribyl commented Aug 28, 2020

metehankutlu commented Aug 30, 2020 • edited

metehankutlu commented Aug 30, 2020 • edited

MarcoGorelli commented Aug 30, 2020

rajanshoo25 commented Sep 1, 2020

MarcoGorelli commented Sep 1, 2020 • edited

tiagohonorato commented Sep 1, 2020

sarthakchaudhary13 commented Sep 1, 2020

xcz011 commented Sep 2, 2020

sarthakchaudhary13 commented Sep 2, 2020 • edited

satrio-hw commented Sep 7, 2020

tiagohonorato commented Sep 8, 2020

cjlynch278 commented Sep 11, 2020

MarcoGorelli commented Aug 27, 2020 •

edited

MarcoGorelli commented Aug 27, 2020 •

edited

jpribyl commented Aug 28, 2020 •

edited

metehankutlu commented Aug 30, 2020 •

edited

metehankutlu commented Aug 30, 2020 •

edited

MarcoGorelli commented Sep 1, 2020 •

edited

sarthakchaudhary13 commented Sep 2, 2020 •

edited