Skip to content

/ pandas

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: Can't disable progress bar for read_gbq #33360

Open
de-code opened this issue Apr 7, 2020 · 5 comments · May be fixed by #33477
Open

BUG: Can't disable progress bar for read_gbq #33360

de-code opened this issue Apr 7, 2020 · 5 comments · May be fixed by #33477
Assignees
@sumanau7
Labels
Bug IO Google good first issue
Milestone
Contributions Wel...

Comments

@de-code
Copy link

@de-code de-code commented Apr 7, 2020

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • (optional) I have confirmed this bug exists on the master branch of pandas.

Note: Please read this guide detailing how to provide the necessary information for us to reproduce your bug.

Code Sample, a copy-pastable example

# Your code here
import pandas as pd

pd.read_gbq(..., progress_bar_type=None)

(Due to the BigQuery dependency it's not copy-pastable)

Warning message displayed or tqdm is used:

/path/to/python/site-packages/pandas_gbq/gbq.py:555: UserWarning: A progress bar was requested, but there was an error loading the tqdm library. Please install tqdm to use the progress bar functionality.
  progress_bar_type=progress_bar_type,

That is because if progress_bar_type is set to None it is not passed to pandas_gbq.read_gbq (see #29858 (comment)). But the default for pandas_gbq.read_gbq is tqdm.

Problem description

It should be possible to disable the progress bar.

Expected Output

No progress bar.

Output of pd.show_versions()

INSTALLED VERSIONS ------------------ commit : None python : 3.6.9.final.0 python-bits : 64 OS : Linux OS-release : 4.15.0-91-generic machine : x86_64 processor : x86_64 byteorder : little LC_ALL : None LANG : en_GB.UTF-8 LOCALE : en_GB.UTF-8

pandas : 1.0.3
numpy : 1.18.2
pytz : 2019.3
dateutil : 2.8.1
pip : 20.0.2
setuptools : 39.0.1
Cython : None
pytest : 4.6.9
hypothesis : None
sphinx : None
blosc : None
feather : None
xlsxwriter : None
lxml.etree : None
html5lib : None
pymysql : None
psycopg2 : None
jinja2 : None
IPython : None
pandas_datareader: None
bs4 : None
bottleneck : None
fastparquet : None
gcsfs : None
lxml.etree : None
matplotlib : None
numexpr : None
odfpy : None
openpyxl : None
pandas_gbq : 0.13.1
pyarrow : None
pytables : None
pytest : 4.6.9
pyxlsb : None
s3fs : None
scipy : None
sqlalchemy : None
tables : None
tabulate : None
xarray : None
xlrd : None
xlwt : None
xlsxwriter : None
numba : None
@TomAugspurger TomAugspurger added this to the Contributions Welcome milestone Apr 7, 2020
@TomAugspurger
Copy link
Contributor

@TomAugspurger TomAugspurger commented Apr 7, 2020 •
edited

Thanks for the report. Looks like we need to just always set it at

pandas/pandas/io/gbq.py

Lines 171 to 172 in 6ae710a

if progress_bar_type is not None:
kwargs["progress_bar_type"] = progress_bar_type
.

We'll need to verify that the minimum supported version of gbq will always accept it. Seems like it requires 0.12.0 or newer.
@de-code
Copy link
Author

@de-code de-code commented Apr 7, 2020 •
edited

You could also set the default to something like NOT_SET and compare against that instead of None. (Or use something like kwargs)

EDIT: I guess if you want to keep the (sensible) default of no progressbar then that wouldn't work
@TomAugspurger
Copy link
Contributor

@TomAugspurger TomAugspurger commented Apr 7, 2020

I'm not sure why we have a different default, perhaps a version issue. Ideally we would just follow pandas-gbq (cc @tswast)
@tswast
Copy link
Contributor

@tswast tswast commented Apr 7, 2020

Yes, it was left out due to version issues. progress_bar_type was added in pandas-gbq==0.12.0.

Options:

  1. Increment minimum pandas-gbq version to at least 0.12.0 and pass through the argument value to pandas-gbq.
  2. Use a sentinel value for the default (NOT_SET) and only populate the value in pandas-gbq when set to something else.

By following (2), the default value in pandas-gbq will be used. By following (1), pandas can set a different default value from pandas-gbq.
@sumanau7 sumanau7 linked a pull request that will close this issue Apr 11, 2020
Open
4 of 5 tasks complete
@sumanau7
Copy link
Contributor

@sumanau7 sumanau7 commented Apr 11, 2020

take
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees

@sumanau7 sumanau7

Labels
Bug IO Google good first issue
Projects
None yet
Milestone
  Contributions Wel...
Linked pull requests

Successfully merging a pull request may close this issue.

Send None parameter to pandas-gbq to set no progress bar
4 participants
@tswast @de-code @TomAugspurger @sumanau7
You can’t perform that action at this time.