df.drop_duplicates() 详解+用法

drop_duplicates()

1、不定义任何参数，完全删除重复的行数据

2、去除重复的几列行数据

目录

一、代码示例：

二、运行结果：

三、详解：

---

一、代码示例：

import pandas as pd


df = pd.DataFrame({
    'brand': ['Yum Yum', 'Yum Yum', 'Indomie', 'Indomie', 'Indomie'],
    'style': ['cup', 'cup', 'cup', 'pack', 'pack'],
    'rating': [4, 4, 3.5, 15, 5]})
print("---------------------原始数据：")
print(df)
print("------------------------df.drop_duplicates()")
print(df.drop_duplicates())
print("------------------------删除在brand列中重复的数据行")
print(df.drop_duplicates(subset='brand'))
print("------------------------重复行保留第一次出现的行，删除其他行")
print(df.drop_duplicates(keep="first"))
print("----------------------inplace 布尔值，默认为False，是否直接在原数据上删除重复项或删除重复项后返回副本")
print("-----------------inplace=False 删除重复项后返回副本")
print(df.drop_duplicates(inplace=False))
print("-------------df1")
print(df)
print("-----------------inplace=True 直接在原数据上删除重复项")
print(df.drop_duplicates(inplace=True))
print("-------------df2")
print(df)

二、运行结果：

---------------------原始数据：
     brand style  rating
0  Yum Yum   cup     4.0
1  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0
------------------------df.drop_duplicates()
     brand style  rating
0  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0
------------------------删除在brand列中重复的数据行
     brand style  rating
0  Yum Yum   cup     4.0
2  Indomie   cup     3.5
------------------------重复行保留第一次出现的行，删除其他行
     brand style  rating
0  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0
----------------------inplace 布尔值，默认为False，是否直接在原数据上删除重复项或删除重复项后返回副本
-----------------inplace=False 删除重复项后返回副本
     brand style  rating
0  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0
-------------df1
     brand style  rating
0  Yum Yum   cup     4.0
1  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0
-----------------inplace=True 直接在原数据上删除重复项
None
-------------df2
     brand style  rating
0  Yum Yum   cup     4.0
2  Indomie   cup     3.5
3  Indomie  pack    15.0
4  Indomie  pack     5.0

 

三、详解：

drop_duplicates(self, subset: 'Optional[Union[Hashable, Sequence[Hashable]]]' = None, keep: 'Union[str, bool]' = 'first', inplace: 'bool' = False, ignore_index: 'bool' = False)
   

返回：

        DataFrame with duplicate rows removed.
    
    Considering certain columns is optional. Indexes, including time indexes
    are ignored.
    
参数：
    ----------
    subset : 指定重复数据所在的列。column label or sequence of labels, optional
        Only consider certain columns for identifying duplicates, by
        default use all of the columns.
    keep : {'first', 'last', False}, default 'first'
        Determines which duplicates (if any) to keep.
        - ``first`` : 除了第一次出现以外，删除重复项。Drop duplicates except for the first occurrence.
        - ``last`` : 除了第一次出现以外，删除重复项。Drop duplicates except for the last occurrence.
        - False : 删除所有重复项。Drop all duplicates.
    inplace : True:直接在原始数据删除，False：不直接在原始数据删除，并生成一个副本。bool, default False
        Whether to drop duplicates in place or to return a copy.
    ignore_index : bool, default False
        If True, the resulting axis will be labeled 0, 1, …, n - 1.
    
        .. versionadded:: 1.0.0
    
    Returns
    -------
    DataFrame or None
        DataFrame with duplicates removed or None if ``inplace=True``.
    
    See Also
    --------
    DataFrame.value_counts: Count unique combinations of columns.
    
    示例：
    --------
    Consider dataset containing ramen rating.
    
    >>> df = pd.DataFrame({
    ...     'brand': ['Yum Yum', 'Yum Yum', 'Indomie', 'Indomie', 'Indomie'],
    ...     'style': ['cup', 'cup', 'cup', 'pack', 'pack'],
    ...     'rating': [4, 4, 3.5, 15, 5]
    ... })
    >>> df
        brand style  rating
    0  Yum Yum   cup     4.0
    1  Yum Yum   cup     4.0
    2  Indomie   cup     3.5
    3  Indomie  pack    15.0
    4  Indomie  pack     5.0
    
    By default, it removes duplicate rows based on all columns.
    
    >>> df.drop_duplicates()
        brand style  rating
    0  Yum Yum   cup     4.0
    2  Indomie   cup     3.5
    3  Indomie  pack    15.0
    4  Indomie  pack     5.0
    
    To remove duplicates on specific column(s), use ``subset``.
    
    >>> df.drop_duplicates(subset=['brand'])
        brand style  rating
    0  Yum Yum   cup     4.0
    2  Indomie   cup     3.5
    
    To remove duplicates and keep last occurrences, use ``keep``.
    
    >>> df.drop_duplicates(subset=['brand', 'style'], keep='last')
        brand style  rating
    1  Yum Yum   cup     4.0
    2  Indomie   cup     3.5
    4  Indomie  pack     5.0