python缺失值插补,使用KNN在Python中缺少值插补

I have a dataset that looks like this

1908 January 5.0 -1.4

1908 February 7.3 1.9

1908 March 6.2 0.3

1908 April NaN 2.1

1908 May NaN 7.7

1908 June 17.7 8.7

1908 July NaN 11.0

1908 August 17.5 9.7

1908 September 16.3 8.4

1908 October 14.6 8.0

1908 November 9.6 3.4

1908 December 5.8 NaN

1909 January 5.0 0.1

1909 February 5.5 -0.3

1909 March 5.6 -0.3

1909 April 12.2 3.3

1909 May 14.7 4.8

1909 June 15.0 7.5

1909 July 17.3 10.8

1909 August 18.8 10.7

I want to replace the NaNs using KNN as the method. I looked up sklearns Imputer class but it supports only mean, median and mode imputation. There is a feature request here but I don't think thats been implemented as of now. Any ideas on how to replace the NaNs from the last two columns using KNN?

Edit:

Since I need to run codes on another environment, I don't have the luxury of installing packages. sklearn, pandas, numpy and other standard packages are the only ones I can use.

解决方案

fancyimpute package supports such kind of imputation, using the following API:

from fancyimpute import KNN

# X is the complete data matrix

# X_incomplete has the same values as X except a subset have been replace with NaN

# Use 3 nearest rows which have a feature to fill in each row's missing features

X_filled_knn = KNN(k=3).complete(X_incomplete)

Here are the imputations supported by this package:

•SimpleFill: Replaces missing entries with the mean or median of each

column.

•KNN: Nearest neighbor imputations which weights samples using the

mean squared difference on features for which two rows both have

observed data.

•SoftImpute: Matrix completion by iterative soft thresholding of SVD

decompositions. Inspired by the softImpute package for R, which is

based on Spectral Regularization Algorithms for Learning Large

Incomplete Matrices by Mazumder et. al.

•IterativeSVD: Matrix completion by iterative low-rank SVD

decomposition. Should be similar to SVDimpute from Missing value

estimation methods for DNA microarrays by Troyanskaya et. al.

•MICE: Reimplementation of Multiple Imputation by Chained Equations.

•MatrixFactorization: Direct factorization of the incomplete matrix

into low-rank U and V, with an L1 sparsity penalty on the elements of

U and an L2 penalty on the elements of V. Solved by gradient descent.

•NuclearNormMinimization: Simple implementation of Exact Matrix

Completion via Convex Optimization by Emmanuel Candes and Benjamin

Recht using cvxpy. Too slow for large matrices.

•BiScaler: Iterative estimation of row/column means and standard

deviations to get doubly normalized matrix. Not guaranteed to converge

but works well in practice. Taken from Matrix Completion and Low-Rank

SVD via Fast Alternating Least Squares.