Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

BUG: read_csv with both names and parse_date raises 'NoneType' TypeError #33699

Closed
quazgar opened this issue Apr 21, 2020 · 1 comment · Fixed by #38431
Closed

BUG: read_csv with both names and parse_date raises 'NoneType' TypeError #33699

quazgar opened this issue Apr 21, 2020 · 1 comment · Fixed by #38431
Labels
Bug IO CSV
Milestone
1.3

Comments

@quazgar
Copy link

@quazgar quazgar commented Apr 21, 2020

This is probably similar to #14792, but still happens with 0.23.3 (a friend even tested it with 1.0.3).

Testcase

import io
s = """A,B,\n1,2"""
pd.read_csv(io.StringIO(s), parse_dates=["B"], names=["B"])

raises:

---------------------------------------------------------------------------
TypeError                                 Traceback (most recent call last)
<ipython-input-146-082b3d7afa0a> in <module>()
      1 import io
      2 s = """A,B,\n1,2"""
----> 3 pd.read_csv(io.StringIO(s), parse_dates=["B"], names=["B"])

/usr/lib/python3/dist-packages/pandas/io/parsers.py in parser_f(filepath_or_buffer, sep, delimiter, header, names, index_col, usecols, squeeze, prefix, mangle_dupe_cols, dtype, engine, converters, true_values, false_values, skipinitialspace, skiprows, nrows, na_values, keep_default_na, na_filter, verbose, skip_blank_lines, parse_dates, infer_datetime_format, keep_date_col, date_parser, dayfirst, iterator, chunksize, compression, thousands, decimal, lineterminator, quotechar, quoting, escapechar, comment, encoding, dialect, tupleize_cols, error_bad_lines, warn_bad_lines, skipfooter, doublequote, delim_whitespace, low_memory, memory_map, float_precision)
    676                     skip_blank_lines=skip_blank_lines)
    677
--> 678         return _read(filepath_or_buffer, kwds)
    679
    680     parser_f.__name__ = name

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _read(filepath_or_buffer, kwds)
    444
    445     try:
--> 446         data = parser.read(nrows)
    447     finally:
    448         parser.close()

/usr/lib/python3/dist-packages/pandas/io/parsers.py in read(self, nrows)
   1034                 raise ValueError('skipfooter not supported for iteration')
   1035
-> 1036         ret = self._engine.read(nrows)
   1037
   1038         # May alter columns / col_dict

/usr/lib/python3/dist-packages/pandas/io/parsers.py in read(self, nrows)
   1887
   1888                 values = self._maybe_parse_dates(values, i,
-> 1889                                                  try_parse_dates=True)
   1890                 arrays.append(values)
   1891

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _maybe_parse_dates(self, values, index, try_parse_dates)
   1946
   1947     def _maybe_parse_dates(self, values, index, try_parse_dates=True):
-> 1948         if try_parse_dates and self._should_parse_dates(index):
   1949             values = self._date_conv(values)
   1950         return values

/usr/lib/python3/dist-packages/pandas/io/parsers.py in _should_parse_dates(self, i)
   1319             else:
   1320                 name = None
-> 1321             j = self.index_col[i]
   1322
   1323             if is_scalar(self.parse_dates):

TypeError: 'NoneType' object is not subscriptable

Notes

This does not happen with engine="python" (you only get a ParseError there).
@siddartha-RE
Copy link

@siddartha-RE siddartha-RE commented Oct 13, 2020

In Pandas 1.0.3 I am seeing the error with the following simple invoication:

df = pd.read_csv('/tmp/csv_no_dates.csv', nrows=500, low_memory=False, parse_dates=[])

Bottom of stack:

  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 676, in parser_f
    return _read(filepath_or_buffer, kwds)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 454, in _read
    data = parser.read(nrows)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1133, in read
    ret = self._engine.read(nrows)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 2080, in read
    values = self._maybe_parse_dates(values, i, try_parse_dates=True)
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 2141, in _maybe_parse_dates
    if try_parse_dates and self._should_parse_dates(index):
  File "/usr/local/lib/python3.6/dist-packages/pandas/io/parsers.py", line 1448, in _should_parse_dates
    j = self.index_col[i]

@phofl phofl mentioned this issue Dec 12, 2020
Merged
5 tasks
@jreback jreback added this to the 1.3 milestone Dec 12, 2020
@phofl phofl mentioned this issue Dec 15, 2020
Closed
3 tasks
@lucebac lucebac mentioned this issue Dec 15, 2021
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
Bug IO CSV
Projects
None yet
Milestone
1.3
Linked pull requests

Successfully merging a pull request may close this issue.

BUG: read_csv raising TypeError for engine=c with names and parse_dates
4 participants
@quazgar @jreback @jbrockmendel @siddartha-RE