I'm using Python 3.7.12 and trying to understand the behaviour of adding a pandas.TimedeltaIndex object to a datetime.date object, and specifically why I sometimes get a TypeError: unsupported operand type(s) for +: 'TimedeltaArray' and 'datetime.date' error. I am pulling data from a source and loading it into a dataframe with one or more rows, and attempting to add a new column to all rows containing the sum of the datetime.date and pandas.TimedeltaIndex for that row. My code works whenever there are at least two rows in the dataframe, e.g.
import pandas as pd
import logging
logging.basicConfig(level=logging.DEBUG)
data = {'dates': [pd.to_datetime('2017-04-27 15:59:59', format='%Y-%m-%d %H:%M:%S'),
pd.to_datetime('2017-04-28 15:59:59', format='%Y-%m-%d %H:%M:%S')],
'deltas': ['90', '180']}
df = pd.DataFrame(data)
df['adjusted_dates'] = df['dates'].dt.date + pd.TimedeltaIndex(df['deltas'].astype('int64'), unit='D')
# The deltas have been applied and df is in the expected shape
logging.debug("Job done")
If I have exactly one row in my data frame, however, I get the error:
import pandas as pd
import logging
logging.basicConfig(level=logging.DEBUG)
data = {'dates': [pd.to_datetime('2017-04-27 15:59:59', format='%Y-%m-%d %H:%M:%S')],
'deltas': ['90']}
df = pd.DataFrame(data)
# TypeError: unsupported operand type(s) for +: 'TimedeltaArray' and 'datetime.date'
df['adjusted_dates'] = df['dates'].dt.date + pd.TimedeltaIndex(df['deltas'].astype('int64'), unit='D')
# We don't get this far
logging.debug("Job done")
Why am I seeing this error for single-row data frames? Any help would be greatly appreciated.
EDIT: I found another question here on Stack Overflow that answers why my code would sometimes fail: Python Pandas: TypeError: unsupported operand type(s) for +: 'datetime.time' and 'Timedelta'
I was attempting to add a Pandas delta object to a Python datetime, but these two stacks are incompatible. When I have Pandas objects for both operands, the dataframe is updated as expected for single-row as well as multi-row dataframes.
df['adjusted_dates'] = df['dates'] + pd.TimedeltaIndex(df['deltas'].astype('int64'), unit='D')
Now that I see why my code didn't always succeed, I'm confused as to why it didn't always fail?
You can simplify to
import pandas as pd
df = pd.DataFrame({'dates': ['2017-04-27 15:59:59', '2017-04-28 15:59:59'],
'deltas': ['90', '180']})
# no need to provide a format, no need to use an index:
df['adjusted_dates'] = (pd.to_datetime(df['dates']) +
pd.to_timedelta(df['deltas'].astype(int), unit='D'))
# df['adjusted_dates']
# 0 2017-07-26 15:59:59
# 1 2017-10-25 15:59:59
# Name: adjusted_dates, dtype: datetime64[ns]
Or, if you just want to add the timedelta to the date, ignoring the time:
df['adjusted_dates'] = (pd.to_datetime(df['dates']).dt.floor('D') +
pd.to_timedelta(df['deltas'].astype(int), unit='D'))
# df['adjusted_dates']
# 0 2017-07-26
# 1 2017-10-25
# Name: adjusted_dates, dtype: datetime64[ns]
Note that in both cases, you'll have datetime64[ns] datatype. If you floor to the day, the time is just not displayed (defaults to zero / 00:00:00).