Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

csv does not round-trip for complex numbers #98485

Open
sobolevn opened this issue Oct 20, 2022 · 4 comments
Open

csv does not round-trip for complex numbers #98485

sobolevn opened this issue Oct 20, 2022 · 4 comments
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error

Comments

@sobolevn
Copy link
Member

sobolevn commented Oct 20, 2022

Reproduction depends on the quoting.

QUOTE_ALL

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_ALL)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_ALL)
    print(list(reader))  # [['1'], ['', ''], ['1'], ['j'], []]

In this case, we don't have our data back, but at least it does not raise.

QUOTE_NONNUMERIC

Qouting docs:

No automatic data type conversion is performed unless the QUOTE_NONNUMERIC format option is specified (in which case unquoted fields are transformed into floats)

import csv
data = [1, 1j]
with open('example.csv', 'w') as f:
    writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
    writer.writerow(data)

with open('example.csv') as f:
    contents = f.read()
    print(contents)  # 1,1j
    reader = csv.reader(contents, quoting=csv.QUOTE_NONNUMERIC)
    print(list(reader))

# Traceback (most recent call last):
#   File "/Users/sobolev/Desktop/cpython/ex.py", line 11, in <module>
#     print(list(reader))
#           ^^^^^^^^^^^^
# ValueError: could not convert string to float: 'j'

In this case, it raises an error, while trying to convert 1j to float.

Current docs / tests

I cannot find any mentions of complex numbers in tests or docs for csv.

Solutions?

  1. We can say that it works as expected: with this strange QUOTE_ALL formatting and QUOTE_NONNUMERIC exception. Add a test case for it and forget about it
  2. We can try to quote complex as string: in this case it will be treated as "1j". I think it is much better, because it will allow users to convert this value to complex manually
  3. Forbid writting complex numbers. However, I don't think it is a path we should go
@sobolevn sobolevn added the type-bug An unexpected behavior, bug, or error label Oct 20, 2022
@pochmann
Copy link
Contributor

I suggest you first fix your bug. Give csv.reader the file object, not the file contents as a string.

@pochmann
Copy link
Contributor

pochmann commented Oct 20, 2022

Those two programs are identical. You're not using QUOTE_NONNUMERIC.

@sobolevn
Copy link
Member Author

@pochmann thanks!

I suggest you first fix your bug. Give csv.reader the file object, not the file contents as a string.

It does not really matter. See:

» ./python.exe
Python 3.12.0a0 (heads/main:ff173ed2f6, Oct 20 2022, 14:06:56) [Clang 11.0.0 (clang-1100.0.33.16)] on darwin
Type "help", "copyright", "credits" or "license" for more information.
>>> import csv
>>> data = [1, 1j]
>>> with open('example.csv', 'w') as f:
...     writer = csv.writer(f, quoting=csv.QUOTE_NONNUMERIC)
...     writer.writerow(data)
... 
6
>>> with open('example.csv') as f:
...     reader = csv.reader(f, quoting=csv.QUOTE_NONNUMERIC)
...     print(list(reader))
... 
Traceback (most recent call last):
  File "<stdin>", line 3, in <module>
ValueError: could not convert string to float: '1j'

Those two programs are identical. You're not using QUOTE_NONNUMERIC.

This was a copy-paste error, updated.

@akulakov
Copy link
Contributor

akulakov commented Nov 9, 2022

Only addressing the first example, with quoting=csv.QUOTE_ALL or with default quoting, I do get the original data back as expected (on latest dev version):

with open('a.csv') as f:
    reader = csv.reader(f, quoting=csv.QUOTE_ALL)
    for r in reader:
        print(r)
# ['1', '1j']
# ['2', '2j']

So with complex numbers you would need to complex(r[1]) when reading data.

If there was no way to get the original data back, of course it would be more alarming.

@iritkatriel iritkatriel added the stdlib Python modules in the Lib dir label Nov 24, 2023
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
stdlib Python modules in the Lib dir type-bug An unexpected behavior, bug, or error
Projects
Status: No status
Development

No branches or pull requests

4 participants