1、使用write往odps表写数据时遇到报错
from odps import ODPS
import pandas as pd
# 创建odps实例
o = ODPS(access_id, access_key, project, endpoint)
# odps表对像
t_o = o.get_table(table_name)
# 往odps表写数据
with t_o.open_writer(partition=partition, create_partition=True) as w:
w.write(records)
报错信息如下:
Traceback (most recent call last):
File "D:/yxdwork/home/tools/kaoshi/mc_opt.py", line 107, in <module>
writer_to_odps(table_name, partition, records,o)
File "D:/yxdwork/home/tools/kaoshi/mc_opt.py", line 52, in writer_to_odps
w.write(records)
File "D:\yxdwork\home\pyevn\tools\lib\site-packages\odps\models\table.py", line 684, in write
for record in records:
File "D:\yxdwork\home\pyevn\tools\lib\site-packages\odps\models\table.py", line 656, in <genexpr>
records = (table_object.new_record(vals) for vals in arg)
File "D:\yxdwork\home\pyevn\tools\lib\site-packages\odps\models\table.py", line 826, in new_record
return Record(schema=self.schema, values=values)
File "odps\src\types_c.pyx", line 270, in odps.src.types_c.BaseRecord.__cinit__
File "odps\src\types_c.pyx", line 338, in odps.src.types_c.BaseRecord._sets
File "odps\src\types_c.pyx", line 313, in odps.src.types_c.BaseRecord._set
File "odps\src\types_c.pyx", line 249, in odps.src.types_c.SchemaSnapshot.validate_value
File "odps\src\types_c.pyx", line 131, in odps.src.types_c._validate_timestamp
File "pandas\_libs\tslibs\timestamps.pyx", line 1228, in pandas._libs.tslibs.timestamps.Timestamp.strptime
NotImplementedError: Timestamp.strptime() is not implemented. Use to_datetime() to parse date strings.
报错原因:
pyodps库往odps写数据时,遇到odps表字段类型为timestamp时,需要调用pandas的Timestamp.strptime()方法将数据转换为Timestamp类型后才能写入。而pandas自0.24.0版本后移除了该方法:
version<=0.24.0,如下
version>0.24.0,如下
2、解决方法如下
1、 使用pandas==0.24.0版本的包即可
pip install pandas==0.24.0
2、修改pyodps库源码,使用pandas.to_datetime()替换pandas.Timestamp.strptime()方法步骤如下:
1、找到odps库地址,打到\odps\types.py文件打开,修改第852行,如下图:
2、删除types.py编译后的pyd文件:\odps\src\types_c.cp39-win_amd64.pyd
不删除这个改了types.py也不会生效,一定要删