Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

BUG: Timedelta.total_seconds method is returning wrong values in nanosecond intervals #46819

Open
3 tasks done
Tiarles opened this issue Apr 21, 2022 · 3 comments
Open
3 tasks done
Labels
Bug Regression Timedelta

Comments

@Tiarles
Copy link

@Tiarles Tiarles commented Apr 21, 2022

Pandas version checks

  • I have checked that this issue has not already been reported.

  • I have confirmed this bug exists on the latest version of pandas.

  • I have confirmed this bug exists on the main branch of pandas.

Reproducible Example

import pandas as pd
import numpy as np

duration, Ts = 0.5, 5e-7
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(signal[1])  # 0 days 00:00:00.000000500
print(signal[0])  # 0 days 00:00:00

print(dt.total_seconds())  # 0.0 <- Wrong output

print(dt / pd.Timedelta(seconds=1))  # 5e-07 <- Expected behavior of ``dt.total_seconds()``

duration, Ts = 0.5, 1e-4
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(signal[1])  # 0 days 00:00:00.000100
print(signal[0])  # 0 days 00:00:00

print(dt.total_seconds())  # 0.0001 <- Correct answer

print(dt / pd.Timedelta(seconds=1))  # 0.0001 <- Proof

Issue Description

The total_seconds method from the Timedelta class has an unexpected behavior on Pandas 1.4.2, since Pandas 1.0.5.
For a small difference in nanoseconds scale the number of seconds on the interval returns 0,

Expected Behavior

The expected behavior is like what happens in Pandas 1.0.5, where:

import pandas as pd
import numpy as np

duration, Ts = 0.5, 5e-7
signal = pd.to_timedelta(np.arange(0, duration, Ts), "s")

dt = signal[1] - signal[0]

print(dt.total_seconds())  # = 5e-07

Installed Versions

Python 3.8.4

Env w/ Pandas 1.4.2:

Package Version
numpy 1.22.3
pandas 1.4.2
pip 22.0.4
python-dateutil 2.8.2
pytz 2022.1
setuptools 60.10.0
six 1.16.0
wheel 0.37.1

Env w/ Pandas 1.0.5:

Package Version
numpy 1.22.3
pandas 1.0.5
pip 22.0.4
python-dateutil 2.8.2
pytz 2022.1
setuptools 60.10.0
six 1.16.0
wheel 0.37.1
@Tiarles Tiarles added Bug Needs Triage labels Apr 21, 2022
@Tiarles
Copy link
Author

@Tiarles Tiarles commented Apr 26, 2022

It`s an issue related to datetime package and not pandas itself. Closing

@Tiarles Tiarles closed this as completed Apr 26, 2022
@Tiarles
Copy link
Author

@Tiarles Tiarles commented Apr 27, 2022

Seems that datetime.timedelta ignores nanoseconds operations that pandas should support, so the total_seconds method of pandas.core.indexes.timedeltas.TimedeltaIndex doesn't works as expected.

@Tiarles Tiarles reopened this Apr 27, 2022
@simonjayhawkins simonjayhawkins added Timedelta and removed Needs Triage labels May 28, 2022
simonjayhawkins added a commit to simonjayhawkins/pandas that referenced this issue May 28, 2022
@simonjayhawkins
Copy link
Member

@simonjayhawkins simonjayhawkins commented May 29, 2022

Thanks @Tiarles for the report.

The total_seconds method from the Timedelta class has an unexpected behavior on Pandas 1.4.2, since Pandas 1.0.5.

first bad commit: [3b2c8f6] BUG: nonexistent Timestamp pre-summer/winter DST w/dateutil timezone (#31155)

see #31155 (comment) and #31354

@simonjayhawkins simonjayhawkins added the Regression label May 29, 2022
@simonjayhawkins simonjayhawkins added this to the Contributions Welcome milestone May 29, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Bug Regression Timedelta
Projects
None yet
Development

No branches or pull requests

2 participants