pandas将某列复制到另一个表,使用Pandas将列从一个DataFrame复制到另一个的最快方法?...

在处理大量数据时,使用Pandas的.loc方法将一列从一个DataFrame复制到另一个可能会较慢。文章提供了一种替代方案,通过先更新小DataFrame,然后在需要时合并所有子DataFrame,以提高处理速度。这种方法适合于独立的更新操作,可以避免在循环中频繁使用.loc。
摘要由CSDN通过智能技术生成

I have a large DataFrame (million +) records I'm using to store core of my data (like a database) and I then have a smaller DataFrame (1 to 2000) records that I'm combining a few of the columns for each time step in my program which can be several thousand time steps . Both DataFrames are indexed the same way by a id column.

the code I'm using is:

df_large.loc[new_ids, core_cols] = df_small.loc[new_ids, core_cols]

Where core_cols is a list of about 10 fields that I'm coping over and new_ids are the ids from the small DataFrame. This code works fine but it is the slowest part of my code my a magnitude of three. I just wanted to know if they was a faster way to merge the data of the two DataFrame together.

I tried merging the data each time with the merge function but process took way to long that is way I have gone to creating a larger DataFrame th

  • 0
    点赞
  • 2
    收藏
    觉得还不错? 一键收藏
  • 0
    评论
评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值