PySpark---SparkSQL中的DataFrame(四)

1.replace(to_replace, value=_NoValue, subset=None)

"""Returns a new :class:`DataFrame` replacing a value with another value.
:func:`DataFrame.replace` and :func:`DataFrameNaFunctions.replace` are
aliases of each other.
Values to_replace and value must have the same type and can only be numerics, booleans,
or strings. Value can have None. When replacing, the new value will be cast
to the type of the existing column.
For numeric replacements all values to be replaced should have unique
floating point representation. In case of conflicts (for example with `{42: -1, 42.0: 1}`)
and arbitrary replacement will be used.

:param to_replace: bool, int, long, float, string, list or dict.
    Value to be replaced.
    If the value is a dict, then `value` is ignored or can be omitted, and `to_replace`
    must be a mapping between a value and a replacement.
:param value: bool, int, long, float, string, list or None.
    The replacement value must be a bool, int, long, float, string or None. If `value` is a
    list, `value` should be of the same length and type as `to_replace`.
    If `value` is a scalar and `to_replace` is a sequence, then `value` is
    used as a replacement for each item in `to_replace`.
:param subset: optional list of column names to consider.
    Columns specified in subset that do not have matching data type are ignored.
    For example, if `value` is a string, and subset contains a non-string column,
    then the non-string column is simply ignored."""

这个方法通过第一个参数指定要 被替换掉的老的值，第二个参数指定新的值，subset关键字参数指定子集，默认是在整个 DataFrame上进行替换。

注意上面在替换的过程中to_replace和value的类型必须要相同，而且to_replace数据类型只 能是：bool, int, long, float, string, list or dict。value数据类型只能是： bool, int, long, float, string, list or None

df.show()

df.replace([r'\N', ' ', '\t',"''"], None).show()
df.replace([r'\N', ' ', '\t',"''"],None,["name","age"]).show()

还可以针对某列中的值进行对应替换,如下:将Genre列的Female替换成F,将Male替换成M

2.sample(withReplacement=None, fraction=None, seed=None)

"""Returns a sampled subset of this :class:`DataFrame`.
:param withReplacement: Sample with replacement or not (default F