模型保存(模型持久化pickle模块和joblib模块)


https://scikit-learn.org/stable/modules/model_persistence.html

Model persistence(模型持久化)

训练了模型之后,我们希望可以保存下来,遇到新样本时直接使用已经训练好的保存了的模型,而不用重新再训练模型。本节介绍pickle在保存模型方面的应用。(After training a scikit-learn model, it is desirable to have a way to persist the model for future use without having to retrain. The following section gives you an example of how to persist a model with pickle. We’ll also review a few security and maintainability issues when working with pickle serialization.)

1、persistence example

It is possible to save a model in the scikit by using Python’s built-in persistence model, namely pickle:

>>> from sklearn import svm
>>> from sklearn import datasets
>>> clf = svm.SVC()
>>> iris = datasets.load_iris()
>>> X, y = iris.data, iris.target
>>> clf.fit(X, y)  
SVC(C=1.0, cache_size=200, class_weight=None, coef0=0.0, degree=3, gamma=0.0,
  kernel='rbf', max_iter=-1, probability=False, random_state=None,
  shrinking=True, tol=0.001, verbose=False)

>>> import pickle
>>> s = pickle.dumps(clf)
>>> clf2 = pickle.loads(s)
>>> clf2.predict(X[0])
array([0])
>>> y[0]
0

有些情况下(more efficient on objects that carry large numpy arrays internally)使用joblib’s 代替pickle (joblib.dump & joblib.load)。之后我们甚至可以在另一个pathon程序中load保存好的模型(pickle也可以。。。):

>>> from sklearn.externals import joblib
>>> <strong>joblib.dump(clf, 'filename.pkl') 
>>> clf = joblib.load('filename.pkl') 

Note :dump and load functions also accept file-like object instead of filenames. More information on data persistence with Joblib is available here.

2、security & maintainability limitations

pickle (and joblib by extension)在maintainability and security方面有些问题,因为:

Never unpickle untrusted data
Models saved in one version of scikit-learn might not load in another version.
为了能够在scikit-learn未来的版本中重构已保存好的模型,需要pickled时添加一些metadata:

The training data, e.g. a reference to a immutable snapshot
The python source code used to generate the model
The versions of scikit-learn and its dependencies
The cross validation score obtained on the training data
further discussion,refer this talk by Alex Gaynor.

个人微信公众号,专注于学习资源、笔记分享,欢迎关注。我们一起成长,一起学习。一直纯真着,善良着,温情地热爱生活,,如果觉得有点用的话,请不要吝啬你手中点赞的权力,谢谢我亲爱的读者朋友
五角钱的程序员,专注于学习资源、笔记分享。

评论
添加红包

请填写红包祝福语或标题

红包个数最小为10个

红包金额最低5元

当前余额3.43前往充值 >
需支付:10.00
成就一亿技术人!
领取后你会自动成为博主和红包主的粉丝 规则
hope_wisdom
发出的红包
实付
使用余额支付
点击重新获取
扫码支付
钱包余额 0

抵扣说明:

1.余额是钱包充值的虚拟货币,按照1:1的比例进行支付金额的抵扣。
2.余额无法直接购买下载,可以购买VIP、付费专栏及课程。

余额充值